Method and system for determining growth stage of a plant

Abstract

A method of determining a growth stage of a plant is disclosed. The method includes illuminating the plant with an illumination light. The illumination light elicits a response light from the plant. The method also includes detecting the response light from the plant, and based on the detected response light, determining the growth stage of the plant. In the method, (i) illuminating the plant includes illuminating with at least partially polarized illumination light, and / or (ii) detecting the response light includes polarizing filtering the response light.

Description

Technical Field

[0001] This disclosure relates to methods and systems for determining the growth stages of plants, and in particular to methods and systems using polarized light. Background Technology

[0002] Harvesting plants (such as flowering plants) at the right time can be very important. Harvesting too late can cause plants to spoil quickly after harvest because they may have become overripe. Harvesting too early may prevent plants from developing the desired characteristics in their flowers, such as the presence of certain compounds and / or the desired shape or form.

[0003] For example, cannabis plants mature within a specific timeframe, and harvesting the flowers at the right time is highly preferred for optimal cultivation results. The evolution of the cannabis flower maturation process involves many stages, including the formation of the stigma and the formation of trichomes on the flower (approximately when the stigma turns brown and dries).

[0004] The trichomes gradually grow, become prominent, and eventually change from transparent / clear to white / milky, and finally to amber. The optimal time to harvest the flowers is when less than 10% of the trichomes have turned amber and the majority are white / milky. When more trichomes begin to turn amber, the buds become overripe. However, this can vary depending on the species.

[0005] The production of cannabis compounds for pharmaceutical purposes requires products to meet pharmaceutical industry standards. Not only should the plants be produced under controlled conditions and without pesticides, but the active ingredients should also be present in appropriate amounts and uniformly throughout the canopy.

[0006] To date, judging the maturity of hemp flowers still relies on human experience and remains a labor-intensive task. Farmers need to walk through the plantation, assess the hairs (e.g., with a magnifying glass), pick mature flowers based on their expert judgment, and leave immature flowers on the plant until the next harvest exploration.

[0007] According to Livingston et al., authors of "Cannabis glandulartrichomes alter morphology and metabolite content during flower maturation" (2020), 101, 37-56, the characteristics of trichomes and the genes highly expressed within them have been revealed, potentially offering new opportunities for informed harvest timing. The authors investigated the characteristics of cannabis trichomes using advanced microscopy techniques, such as two-photon laser scanning fluorescence microscopy, in which redshifted pulsed lasers were used to penetrate living tissue and excite the intrinsic fluorescence of metabolites. They further utilized transmission electron microscopy and microcapillary probing. However, it is clear that this technique is not suitable for field use, especially by inexperienced users.

[0008] Therefore, in view of the above, there is a need in the art for a method that can accurately and easily determine the growth stage of a plant.

[0009] US 2019 / 259108 A1 discloses in the chapter on “Sensors” that the germination, growth and maturation of plants can be measured by sensors and cameras or other imaging methods such as radar, lidar or ultrasonic sensor systems.

[0010] The scientific publication Jun-Li Xu et al., “A polarized hyperspectral imaging system for in vivo detection: Multiple applications in sunflower leaf analysis,” *Computers and Electronics in Agriculture*, Vol. 158, pp. 258-270, discloses a study that aimed to investigate the potential of raw polarized hyperspectral imaging (HSI) settings for real-world sunflower leaves in the 400-1000 nm spectral domain. The paper concludes that both surface and subsurface features of sunflower leaves are classified in the growth stages of young and mature leaves, and that R… BS Equally important in the spectrum, R BS The spectrum is a backscattering reflection spectrum that combines multiple scattering reflections from incident ray-polarized light and weak scattering reflections from incident ray-polarized light. Summary of the Invention

[0011] To this end, a method for determining the growth stage of a plant is disclosed. The method includes illuminating the plant with illumination light. The illumination light induces a response light from the plant. The method further includes detecting the response light from the plant and determining the growth stage of the plant based on the detected response light. In this method, (i) illuminating the plant includes illuminating it with illumination light that is at least partially polarized, and / or (ii) detecting the response light includes polarization filtering the response light.

[0012] The inventors have recognized that the optical properties of at least a portion of a plant can change during its growth. For example, new structures and / or materials can develop in some parts of the plant, possessing optical properties different from those of existing structures and / or materials constituting the plant. In particular, these different optical properties may be related to polarization characteristics. The polarization direction of incident light rotated by the newly developed structure may differ from that of other parts of the plant. Additionally or alternatively, the newly developed structure may selectively reflect only polarizations of a specific direction, while the rest of the plant does not exhibit this selective reflection, or at least exhibits a different selectivity in reflecting specific polarizations.

[0013] The method disclosed herein conveniently utilizes this distinct polarization property by simply illuminating the plant with at least partially polarized light and / or detecting the response light using one or more polarization filters. This allows for the revelation of polarization-active structures / materials that form or disappear during plant growth. Plant growth can also be referred to as the development or maturation of a plant or its parts (such as a flower). Therefore, growth stages can also be referred to as developmental stages and / or maturation stages.

[0014] Advantageously, because this method requires relatively simple components (such as polarizing filters), it can be easily implemented in the field in real time without damaging or even touching the plants. Farmers can enter the field and very easily determine the growth stage of individual plants, and can decide to harvest the plants or one or more parts thereof.

[0015] Illuminating a plant does not necessarily involve illuminating the entire plant; only specific parts of the plant can be illuminated, such as the parts on which structures with different optical properties develop (e.g., flowers).

[0016] Determining growth stages may involve classifying plants into one of a set of two or more categories, each associated with a different growth stage. In this case, one category might be associated with an early growth stage, another with a mid-growth stage, and yet another with a late growth stage. An early growth stage could be a stage where the plant's flowers or other parts have not yet begun to mature; a mid-growth stage could be a stage where the flowers or other parts have begun to mature but are not yet fully mature; and a late growth stage could be a stage where the flowers or other parts are fully mature. The step of determining growth stages can be performed by a data processing system, i.e., it can be carried out by a computer.

[0017] Determining growth stages can involve quantifying the structures / materials of interest on a specific part of the plant. This quantification then indicates the plant's growth stage.

[0018] In one embodiment, the growth stage of the plant is determined based on a training dataset and / or the quantification of the structures / materials of interest described above is performed, wherein corresponding detected responses from one or more plants are associated with corresponding human expert assessments of the plant growth stage and / or corresponding human expert assessments of the number of structures / materials of interest on the plant. In one example, it can be determined that the detected response light is similar to the response light from a plant verified by human experts to be fully mature.

[0019] Response light may include illumination light reflected and / or transmitted by the plant. Additionally or alternatively, response light may include photoluminescence, such as fluorescence. This may be the case, for example, if illumination light excites the plant or a part thereof. Photoluminescence can be understood as the emission of light caused by the absorption of photons.

[0020] The degree of polarization (DOP) can be understood as indicating which part of an electromagnetic wave is polarized. For example, a fully polarized wave may have a 100% degree of polarization, while an unpolarized wave may have a 0% degree of polarization. Partially polarized waves, and therefore can be represented by the superposition of polarized and unpolarized components, have a degree of polarization between 0 and 100%. The degree of polarization can be the ratio of the power carried by the polarized component of the wave to the power carried by the entire wave.

[0021] Illuminating light that is at least partially polarized is, for example, having a degree of polarization (DOP) of at least 5%, at least 10%, at least 20%, at least 30%, at least 40%, at least 50%, at least 60%, at least 70%, at least 80%, or at least 90%. If the light is not at least partially polarized, it can be called unpolarized light.

[0022] If light is polarized in more than one (e.g., two or three) directions, it can be understood as at least partially polarized. In this case, the electromagnetic wave can be represented by a superposition of several polarized components and optional unpolarized components. Such at least partially polarized illumination light, for example, has a ratio between the power of at least 5% or at least 10% or at least 20% or at least 30% or at least 40% or at least 50% or at least 60% or at least 70% or at least 80% or at least 90% of the components in one direction and the power carried by the wave as a whole.

[0023] The polarization direction of light can be defined as parallel to the direction of the electric field describing the electromagnetic wave of light. Furthermore, it should be understood that a specific polarization direction can be understood to include a range of directions, such as all directions between -2 degrees and +2 degrees from the specified direction. For example, if the polarization direction is 20 degrees relative to a reference direction, then a 20-degree polarization can include all components of light with polarization directions between 18 degrees and 22 degrees.

[0024] The method may include harvesting at least a portion of the plant (e.g., the plant's flowers) based on a determined growth stage.

[0025] In one embodiment, the plant is a cannabis plant, such as a Cannabis species (Cannabis) Cannabis Sativa plant), hemp plant (Cannabis Sativa L. plant) or Indian cannabis plant ( Incica (plant). In this embodiment, the illuminated plant may include specifically illuminating the female flowers of the cannabis plant, since female flowers are generally consumed, while male flowers are not. Female flowers possess so-called special metabolites, such as cannabinoid metabolites, which are known to cause psychoactive and / or pharmacological effects. As described above, as the cannabis plant matures, trichomes grow on the flowers, particularly on the canopy. The trichomes grow, become prominent, and also change color multiple times. The inventors have recognized that the trichomes possess polarization characteristics that differ from the rest of the plant, at least at certain stages of their development. Therefore, these distinct polarization characteristics can be used to assess the presence and quantity of trichomes, and thus can be used to assess plant maturity, particularly flower maturity. It is readily understood that when the plant's growth stage is known, the optimal harvest time can be easily determined. As described above, cannabis flowers mature within a specific timeframe, and harvesting the flowers at the appropriate time is highly preferred for obtaining optimal cultivation results.

[0026] The flowering period of cannabis plants typically involves multiple sub-stages. Determining the growth stage of a cannabis plant is preferably done by identifying which of these multiple sub-stages the female flower is in.

[0027] In one embodiment, detecting the response light includes, optionally by a computer, the step of obtaining data representing the detected response light. In such an embodiment, the method includes determining the plant's growth stage based on the obtained data. An advantage of this embodiment is that it enables the automatic determination of the plant's growth stage, for example, using a data processing system (such as an image processing system).

[0028] In one embodiment, the obtained data indicates at least one of the following -Total intensity of the response light -The average intensity of the response light, - The maximum intensity of the response light - The minimum intensity of the response light.

[0029] These parameters are easy to process and compare, for example, easier than images. This embodiment eliminates the need to capture and process plant images. Each of the above parameters can be compared with an appropriate threshold. If, for example, a parameter is above or below such a threshold, then the plant or plant part can be determined to be ready for harvest. Of course, if the plant is divided into three or more growth stages, several thresholds can be used. Therefore, this embodiment allows for simplified analysis for determining the growth stages of a plant, and thus enables rapid determination of growth stages. This rapid determination is particularly advantageous when the method is performed in real time in the field.

[0030] In one embodiment, the method includes illuminating the plant with a first illumination light, wherein the first illumination light evokes a first response light from the plant, and detecting the first response light from the plant. In this embodiment, the method includes subsequently illuminating the plant with a second illumination light, wherein the second illumination light evokes a second response light from the plant, and detecting the second response light from the plant. This embodiment also includes determining a growth stage of the plant based on the detected first and second response lights. This embodiment allows for precise determination of the growth stage, as explained further below.

[0031] The steps of detecting the first response light and detecting the second response light can be separated in time by at least 5 hours, preferably at least one day. This advantageously allows for monitoring the development of the response light. Preferably, in this embodiment, the first and second illumination lights are identical, for example, in the sense that they have the same polarization direction or that both the first and second illumination lights are unpolarized.

[0032] In this embodiment, the first illumination light is at least partially polarized, and the second illumination light is unpolarized. This embodiment allows for the convenient identification of polarization-active portions of a plant, for example, by simply switching between polarized and unpolarized illumination light. As a result of this switching, the response light from the polarization-active portions changes in a manner different from that of other parts of the plant. For example, assuming the first and second illumination lights have the same total intensity, the first and second response lights of the unpolarization-active portions of the plant would be identical because these portions similarly reflect any polarization direction. However, the first and second response lights of the polarization-active portions are typically different. If, for example, these portions selectively reflect a particular polarization direction, and the degree of presence of this particular polarization direction in the first illumination light differs from its degree of presence in the second illumination light (which would be typical), then the second response light of these portions has a higher or lower intensity than the first response light of these portions.

[0033] Unpolarized light can be configured as growth light to stimulate plant growth.

[0034] In one embodiment, the first illumination light is at least partially polarized in a first direction, and the second illumination light is at least partially polarized in a second direction different from the first direction. An advantage of this embodiment is that no polarizing filter is required in the detection path. In fact, detecting the response light could involve a human observer simply looking at the plant as it is subsequently illuminated by the first and second illumination lights.

[0035] In the case of a cannabis plant, the trichomes reflect light only in a specific polarization direction. However, the rest of the plant reflects unpolarized light, even when the plant is illuminated by polarized light. One possible explanation for this is the internal scattering of the illuminating light, which would refer to... Figure 4B Detailed description. Therefore, the intensity of the response light from the portion without trichomes is independent of the polarization direction of the illumination light. On the other hand, since trichomes selectively reflect certain polarization directions, the response light from the trichomes depends on the polarization direction of the illumination light. To illustrate, if we assume that a set of trichomes only reflects light with a first specific polarization direction, then if the polarization direction of the illumination light is perpendicular to the first specific polarization direction, the intensity of the response light from this set of trichomes will be close to zero, but if the polarization direction of the illumination light is parallel to the first specific polarization direction, the intensity of the response light will be much higher. Therefore, by changing the polarization direction of the illumination light, the intensity of the response light from the polarization-active portion can vary. This results in a "striking" effect. For example, if the change in the polarization direction of the illumination is "rapid" and "repetitive," then an easily observable "flickering" illumination effect is produced.

[0036] Preferably, the steps of this embodiment are performed relatively quickly to each other, such that any difference between the first detected response light and the second detected response light is primarily caused by the different polarization directions of the illumination light, and to a lesser extent by other factors. For example, it would be undesirable if the plant moved between detecting the first response light and detecting the second response light.

[0037] In one embodiment, detecting response light from a plant includes polarizing the response light to obtain a first response light having a first polarization direction, detecting the first response light, and then polarizing the response light to obtain a second response light having a second polarization direction different from the first polarization direction. This embodiment also includes detecting the second response light. In this embodiment, the method includes determining the plant's growth stage based on the detected first and second response lights. An advantage of this embodiment is that the illumination light does not need to be polarized.

[0038] Preferably, the steps of this embodiment are performed relatively quickly to each other, such that any difference between the first detected response light and the second detected response light is substantially caused by the different directions of polarization filtering, and to a lesser extent by other factors.

[0039] The subsequent two steps of polarization filtering response light can be performed by adjusting the polarization direction of the polarization filter. This can be achieved by mechanically rotating the polarization filter and / or by changing the polarization filter's position in the detection path.

[0040] In one embodiment, detecting response light from a plant includes polarizing the response light to obtain a first response light having a first polarization direction, detecting the first response light, and then detecting the response light without polarization filtering. The detected response light without polarization filtering may be referred to as a second response light. In this embodiment, the method includes determining the plant's growth stage based on the detected first and second response lights. This embodiment makes it possible to conveniently identify polarization-active portions of a plant, for example, by moving a polarization filter into and out of the detection path. This can allow the identification of polarization-active portions based on principles similar to those described in the embodiments, where subsequent polarized and unpolarized illumination light is used.

[0041] In one embodiment, the method includes illuminating the plant with at least partially polarized illumination light and polarizing the response light in the detection step. This embodiment is particularly advantageous if the polarization-active portion of the plant alters the polarization direction of the incident illumination light, while the remainder of the plant reflects unpolarized light. As a result, the response light will have a relatively strong polarization component in a certain direction, which can be easily detected in the response light using a polarization filter.

[0042] In one embodiment, the method includes obtaining multiple datasets. Each dataset represents detected response light detected at a respective time. Optionally, the multiple datasets include a first dataset representing first response light detected at a first time and a second dataset representing second response light detected at a second time. This embodiment also includes determining the plant's growth stage based on the evolution of the response light indicated by the multiple datasets over time. This embodiment allows for determining the plant's growth stage based on the plant's development over time. This allows for more accurate determination of the growth stage.

[0043] In this embodiment, determining the plant's growth stage can be done by identifying the plant's future growth stage. This embodiment allows for the prediction of plant growth based on historical measurements. Advantageously, this allows for the prediction of the optimal harvest time. Historical trends can further guide farmers in making decisions regarding growth control, harvesting, or the effectiveness of light formulations. For typical plant developmental stages, the detected response light or determined plant growth stage can be plotted on an empirical value map along a time axis.

[0044] This embodiment may in particular include determining a corresponding growth stage based on each dataset, and determining the (current) growth stage based on a previously determined corresponding growth stage.

[0045] Preferably, each dataset includes environmental data such as growing conditions, light intensity, watering schedule, humidity, and temperature. If these parameters differ, the growth stage may be delayed or advanced. In one embodiment, detecting the response light includes an optional computer-implemented step of obtaining data representing an image of the plant. An image of a portion of the plant (e.g., a flower) can be understood as an image of the plant. This image indicates the detected response light of various parts of the plant, for example, indicating the intensity of the response light, and the image includes one or more specific image regions having relatively high or relatively low intensities. The one or more specific image regions correspond to one or more polarization-active portions of the plant, which contain optically active materials and / or materials exhibiting polarization-selective reflection. The method includes determining the growth stage of the plant based on the number and / or size and / or light intensity of the one or more specific image regions. This embodiment is particularly advantageous if the method both includes illuminating the plant with at least partially polarized illumination light and includes polarization-filtering the response light in the detection step.

[0046] In the case of a cannabis plant, one or more high-intensity areas can be understood as representing the portion of the plant containing trichomes.

[0047] The determination of a growth stage can be based, for example, on the number and / or size of one or more polarimetric active portions relative to the size of a plant part, where such polarimetric active portions are expected to form or disappear during plant growth. For instance, if the plant is a cannabis plant, trichomes are expected to form on the female flower. The growth stage can then be determined based on the percentage of the female flower surface covered by trichomes.

[0048] In this embodiment, determining the growth stage of a plant may include determining how many pixels in an image have light intensities above a certain threshold.

[0049] In an embodiment, such as that for detecting first and second response lights as described above, detecting the response lights includes an optional computer-implemented step of obtaining data representing a first image and a second image of the plant. The first image indicates first response light from various parts of the plant, and the second image indicates second response light from various parts of the plant. Furthermore, the first image includes one or more image regions corresponding to one or more parts of the plant, and the second image includes one or more image regions corresponding to said one or more parts of the plant. The one or more image regions of the second image have a higher or lower intensity than the one or more image regions of the first image. In this embodiment, the growth stage of the plant is determined based on the intensity difference between the one or more regions of the first and second images. This embodiment allows for the automatic determination of plant growth based on two or more images acquired from the plant.

[0050] In one embodiment, the method includes storing a first image associated with the physical location of various parts of the plant. For example, the method may include storing the physical location of the plant and / or storing the physical location of the plant and the position of the various parts relative to the plant. For illustration, the method may include storing the first image already obtained from a plant located in the x-th row, y-th aisle of a greenhouse, more precisely from the top flower of that plant. This allows the system to automatically retrieve historical images of a particular plant for comparison based on the plant's location.

[0051] In one embodiment, the one or more parts of the plant are plant parts where polarization-active materials are expected to form or disappear during plant growth. In this case, the first and second response lights can be detected after a relatively long period of time (e.g., one day or more). In the case of cannabis plants, this embodiment, for example, allows for the automated monitoring of the number of trichomes on female flowers over time.

[0052] In one embodiment, the one or more portions of the plant are one or more polarization-active portions of the plant, comprising optically active materials and / or materials exhibiting polarization-selective reflectivity. Preferably, in this embodiment, the first and second response lights are detected relatively rapidly relative to each other. In this embodiment, the first and second response lights can be understood as identifiers for the polarization-active portions. In the case of cannabis plants, this embodiment enables the automatic detection of trichomes on female flowers.

[0053] In one embodiment, the method includes identifying regions of interest (ROIs) in one or more images represented by acquired data, the ROIs corresponding to one or more plant portions in which polarization-active portions are expected to form or disappear during plant growth. In this embodiment, the method includes analyzing the identified ROIs to determine the plant's growth stage. An advantage of this embodiment is that it allows computational resources to be used only on relevant regions of the image; for example, in the case of a cannabis plant, computational resources are used only on the image region corresponding to the female flower of that plant.

[0054] Identifying regions of interest can be done using machine learning algorithms known in the art. For example, a neural network can be trained to identify female flowers of a cannabis plant in an image.

[0055] The image may have already been obtained using 3D ranging technology, thus representing a 3D model of the plant. Plant parts that are expected to form or lose polarization-active regions, such as the female flower of a cannabis plant, can then be identified based on their morphology. This identification can also be performed using machine learning algorithms.

[0056] In one embodiment, the illumination light is at least partially linearly polarized.

[0057] In one embodiment, the illumination light is narrowband light.

[0058] In the case of cannabis plants, the trichomes turn milky white or amber as they change their diffuse reflectance during maturation. Different narrow-band detection ranges can be used depending on the detection requirements. Cyan or blue narrow-band detection can be used to distinguish between milky white, clear, and amber trichomes.

[0059] Narrow-band illumination and / or imaging can further increase the signal-to-noise ratio. Narrow-band light, as used here, can be understood as light with a linewidth between 10 and 30 nm.

[0060] In one embodiment, the method includes presenting an indication of a determined plant growth stage on a display of an augmented reality system, allowing a user viewing the augmented reality display to see the plant and its determined growth stage. This embodiment greatly simplifies harvesting because, for example, farmers only need to point a camera at the plant and immediately see if it is ready for harvest.

[0061] The display can be a transparent display. In this case, the image of the plant does not need to be displayed on the screen, because the user can see the plant through the transparent display.

[0062] For example, presenting a user with an instruction to pick a cannabis plant flower on an augmented reality display can be understood as presenting an instruction on the plant's growth stage. This instruction indicates that the plant has reached a growth stage suitable for harvesting. Similarly, presenting a user with an instruction to leave the flower (i.e., not to pick the flower) on an augmented reality display can be understood as presenting an instruction on the plant's growth stage, because this instruction indicates that the plant has not yet reached a growth stage suitable for harvesting.

[0063] One aspect of this disclosure relates to a system for determining a plant growth stage. The system includes a light source illuminating the plant with illumination light, which induces a response light from the plant. The system also includes a detector for detecting the response light from the plant. Furthermore, the system includes a data processing system configured to determine the plant growth stage based on the detected response light, wherein (i) the illumination light is at least partially polarized, and / or (ii) the detector includes a polarizing filter.

[0064] Therefore, a system is provided that can automatically assess the status of hemp flowers in real time and on a plant-by-plant basis, provide farmers / harvesters with input and guidance on harvesting / not harvesting decisions, track harvesting, and further allow prediction of the next flowering and harvesting time.

[0065] In a preferred embodiment, the system is embodied as a handheld device. This can be understood as a person being able to carry the entire system with one or two hands. In another preferred embodiment, the system is embodied as a head-mounted system. This can be understood as a person being able to wear the entire system on their head.

[0066] The light source can be configured to produce monochromatic or multispectral light. The detector can include a narrowband sensor, such as a narrowband spectral imaging sensor.

[0067] In one embodiment, the light source is configured to generate a first illumination light and subsequently generate the second illumination light described above. In this embodiment, the data processing system is configured to control the light source.

[0068] As a light source, a light source with an "electrically tunable polarization multimode polarization point" as described in WO18114368A1 can be conveniently used.

[0069] The data processing system can be configured in particular to enable this embodiment of the system to perform any of the methods described above using the first and second illumination lights.

[0070] In one embodiment, the detector is configured to polarize-filter the response light to obtain a first response light having a first polarization direction, and subsequently polarize-filter the response light to obtain a second response light having a second polarization direction different from the first polarization direction. In this embodiment, the data processing system is configured to control the detector.

[0071] The data processing system can be configured in particular to cause this embodiment of the system to perform any of the methods described above, wherein the response light is polarized filtered.

[0072] In one embodiment, the system includes an augmented reality system comprising a display. In this embodiment, the augmented reality system is configured to present an indication of a defined growth stage of a plant on the display, allowing a user viewing the display to see the plant and its defined growth stage. In this embodiment, the system is preferably embodied as a head-mounted device.

[0073] In one embodiment, the detector includes a camera for recording images of the plant, such as the first and second images described above. This is advantageous because it allows for real-time assessment of the plant's growth stage and immediate presentation of the results to the user.

[0074] In one embodiment, the augmented reality system is configured to at least partially present the recorded image on the display. Therefore, in this embodiment, both the plant and indications of the defined growth stage are presented on the display.

[0075] In one embodiment, the system includes a handheld or head-mounted device in which the camera and augmented reality display are integrated.

[0076] One aspect of this disclosure relates to an augmented reality (AR) system for presenting plant information about plants, such as cannabis plants. The AR system includes a display for showing said information. The AR system also includes a camera for recording images of the plants. The AR system further includes a data processing system. The data processing system is configured to determine plant information. In one example, determining the information includes scanning an RFID tag or general tag associated with the plant to obtain a plant identifier, and retrieving plant information from a database using the obtained plant identifier. The identity of a particular plant can also be determined based on the physical location of that particular plant. Based on the location of a particular plant, the system may be able to determine which plant is being surveyed. An illumination network grid can be used to determine the location of the plant being surveyed. The illumination network grid can, for example, be equipped with location sensing. Any method of determining the location of the plant being surveyed by combining a plan view of the plant can be used to determine which plant is being surveyed. In another example, the data processing system is configured to determine plant information based on recorded images, for example using methods for determining the growth stage of the plant described herein. The data processing system is also configured to control the display of the AR system to display the determined plant information, so that a user viewing the display sees the plant and the determined plant information. AR systems can include any features of augmented reality systems incorporated into systems used to determine the growth stages of plants.

[0077] Plant information includes, for example, information about the plant's age, growth rate, species, historical measurements associated with the plant, and actions to be taken on the plant.

[0078] In an embodiment of the AR system, the data processing system is configured to determine the action to be performed on the plant, such as watering or harvesting it. In this embodiment, the data processing is configured to control a display so that a user viewing the display perceives both the plant and the determined action as plant information.

[0079] Preferably, the data processing system is configured to store any actions performed on the plants, for example, by transmitting data indicating such actions to a central database. This allows tracking of actions performed by a user on a given plant (e.g., harvesting) and the plant's state after these actions (e.g., which flowers were picked, the number of mature and immature / remaining flowers, and the state of immature flowers).

[0080] Preferably, at least the display is implemented as a head-mounted device. The system may, for example, include a carrier backbone and augmented reality goggles as the display. The display may be a transparent display.

[0081] In one embodiment, to better analyze the maturity of flowers, the AR system may include a spectral analysis device (e.g., a hyperspectral or multispectral camera) to analyze images of plants recording color, reflectance, polarization rotation, etc.

[0082] Optionally, the AR system includes photoactuation devices to allow users to see plant maturation, such as polarized light sources, dual (or multi) wavelength light sources, etc., thereby allowing information to be provided directly to the user without post-processing.

[0083] Optionally, the AR system includes photoactuation devices to enable embedded sensors to observe / detect the maturity of the flower (which may be the same as or different from the photoactuation described above).

[0084] One aspect of this disclosure relates to a computer program containing instructions that, when executed by a data processing system used to determine the growth stage of a plant, cause the system to perform one or more (e.g., all) steps of the method described herein for determining the growth stage of a plant.

[0085] One aspect of this disclosure relates to a non-transitory computer-readable storage medium on which the aforementioned computer program is stored.

[0086] As those skilled in the art will appreciate, aspects of the present invention can be embodied as systems, methods, or computer program products. Therefore, aspects of the present invention can take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware aspects, which are generally referred to herein as “circuit,” “module,” or “system.” The functionality described in this disclosure can be implemented as algorithms executed by a computer’s processor / microprocessor. Furthermore, aspects of the present invention can take the form of computer program products embodied in one or more computer-readable media having computer-readable program code embodied thereon (e.g., stored thereon).

[0087] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the context of this invention, a computer-readable storage medium can be any tangible medium that can contain or store a program used by or in conjunction with an instruction execution system, apparatus, or device.

[0088] Computer-readable signal media may include propagated data signals having computer-readable program code embodied therein (e.g., in baseband or as part of a carrier wave). Such propagated signals may take any of a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium that is not a computer-readable storage medium and may convey, propagate, or transmit a program used by or in conjunction with an instruction execution system, apparatus, or device.

[0089] The program code embodied on a computer-readable medium can be transmitted using any suitable medium—including, but not limited to, wireless, wired, fiber optic, cable, RF, or any suitable combination thereof. The computer program code for carrying out the operations of various aspects of the invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Java™, Smalltalk, or C++) and conventional procedural programming languages ​​(such as the "C" programming language or similar programming languages). The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or can be connected to an external computer (e.g., via the Internet provided by an Internet service provider).

[0090] Various aspects of the invention are described below with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, particularly a microprocessor or central processing unit (CPU), to produce a machine such that the instructions, executable via the processor of the computer, other programmable data processing apparatus, or other device, create means for implementing the functions / actions specified in the flowchart illustrations and / or one or more block diagram blocks.

[0091] These computer program instructions may also be stored in a computer-readable medium that can instruct a computer, other programmable data processing apparatus, or other device to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of writing, which includes instructions that implement functions / actions specified in flowcharts and / or one or more block diagrams.

[0092] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide for implementing the functions / actions specified in the flowchart and / or one or more block diagram boxes.

[0093] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, comprising one or more executable instructions for implementing a specified logical function(s). It should also be noted that in some alternative implementations, the functions described in the blocks may not appear in the order shown in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or sometimes these blocks may be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a system based on dedicated hardware, or a combination of dedicated hardware and computer instructions, that performs the specified function or action.

[0094] In addition, a computer program for carrying out the methods described herein is provided, as well as a non-transitory computer-readable storage medium for storing the computer program. For example, the computer program may be downloaded (updated) to existing systems or stored during the manufacture of such systems.

[0095] Unless otherwise explicitly stated, elements and aspects discussed with respect to a particular embodiment or with respect to a particular embodiment may be suitably combined with elements and aspects of other embodiments. Embodiments of the invention will be further described with reference to the accompanying drawings, which schematically illustrate embodiments according to the invention. It should be understood that the invention is not limited in any way to these specific embodiments. Attached Figure Description

[0096] Various aspects of the invention will be explained in more detail with reference to the exemplary embodiments shown in the accompanying drawings, in which: Figures 1A-1C Several embodiments of the method and system are shown; Figures 2A-2D Several embodiments of the method and system are shown, wherein the illumination light is polarized and the response light is polarized filtered; Figures 3A-3D Several embodiments of the system are illustrated schematically; Figure 4A The anatomical structure of the cannabis plant was described; Figure 4B This illustrates a possible explanation for why certain parts of a plant have different polarization properties; Figure 5 An example of an image that can be captured by an embodiment of the system is shown, such an image being captured when an embodiment of the method is performed; Figures 6A-6B Two images are shown, based on which the growth stage of the depicted plant can be determined; Figure 7 This illustrates, according to one embodiment, how historical data can be used to determine the growth stage of a plant; Figure 8A The system, including an augmented reality system, is illustrated schematically. Figure 8B This illustrates the user's perception when viewing the display of the augmented reality system; Figure 9 A data processing system according to an embodiment is illustrated schematically; Figure 10 A detector according to an embodiment is shown. Detailed Implementation

[0097] In the accompanying drawings, the same reference numerals indicate similar or identical elements.

[0098] Figure 1A(Top) An embodiment of the method and system is shown, wherein a plant 6 is illuminated with at least partially polarized (e.g., linearly polarized) illumination light 5. In this example, polarized illumination light 5 is generated by applying a polarizing filter 4a to light generated by a light source 2, which is configured to generate unpolarized light. The light source 2 and the polarizing filter 4a together form a light source 14, which is configured to illuminate the plant 6 with at least partially polarized illumination light 5.

[0099] Illumination light 5 induces response light 7 from the plant. Illumination light can be understood as causing the plant to respond by emitting response light. Response light may include treated illumination light. Such treatment may include reflection and / or absorption and / or alteration of the polarization of the illumination light.

[0100] The response light 7 is detected by detector 16, which in one example includes telescope 10 and imaging system 12, such as a camera. The detector may additionally or alternatively include a person with polarized glasses and / or a light sensor and / or a camera and / or a time-of-flight camera.

[0101] Figure 1A (Bottom) This shows the same system at a later point in time. Now, another polarizing filter 4b is used, which has a different polarization direction than polarizing filter 4a. Therefore, the illumination light 5' has a different polarization direction than the illumination light 5. As a result, assuming that the plant 6 includes a polarization-active portion, the response light 7' is different from the response light 7. The response light 7 can be referred to as the first response light, and the response light 7' can be referred to as the second response light. As described above, based on the first response light 7 and the second response light 7', the growth stage of the plant 6 can be determined. Although Figures 1 and 2 schematically show the response light 7 transmitted through the plant 6, the response light can also be light reflected back from the plant.

[0102] Therefore, the polarization direction of the illumination light can be changed in this system. Of course, although... Figure 1A Only two different polarization directions of the illumination light are shown, but in one embodiment, the polarization direction of the illumination light is continuously rotated, for example, more than 180 degrees.

[0103] This change in the polarization direction of illumination light can be referred to as dynamically polarized light. This light can be generated by a standard light source with a mechanically rotatable polarizer mounted in front of it. The polarization direction of the light is then sequentially moved (“rotated”) at the speed of a rotating external polarization disk. Dynamically polarized light can also be generated using an MR 16 light source with an embedded mechanically rotating polarizer. Furthermore, dynamically polarized light can be generated by dual (or more) polarized spots with electrically tunable polarization. The use of multiple chip-scale microlight sources integrated into a second optical module allows for electrical switching of the polarization direction at high frequencies (using a driver). Dynamically polarized illumination light can also be generated, for example, by controlling a liquid crystal-based polarizer in front of the light source, using illumination based on switchable laser sources with different orientations (lasers generate polarized light), or using pixelated light sources with pixels having different polarization directions.

[0104] Figure 1A The system is well-suited for producing a "flash effect," which makes the polarization-active parts of plants (such as the hairs on hemp flowers) highly visible to users (harvesters) or automated harvesters. As mentioned above, this flash effect is caused by the different responses of the polarization-active parts to varying polarized illumination light.

[0105] As mentioned above, compared to light sources that post-process polarized light, dynamically polarized light can be generated using laser sources (primarily Nd:YAG lasers, which are linearly polarized, and diode lasers, which are less polarized or even elliptically polarized). The laser source can be attached to an arm or incorporated into a horticultural light to scan the plant from top to bottom for in-depth canopy measurements of maturity.

[0106] Figure 1B An embodiment is shown in which detector 16 is configured to subsequently polarize and filter the response light in different directions. Figure 1B (Top) A system including a detector is shown that polarizes the response light, in this example using a polarizing filter 8a with a first polarization direction to obtain a first response light 7'. Figure 1B (Bottom) shows the same system at a later point in time. Now, the detector polarizes and filters the response light 7 to obtain a response light 7'' with a different polarization, which can be referred to as the second response light. In this example, a polarizing filter 8b is used, which has a different polarization direction than filter 8a. Based on the first and second response lights, the growth stage of plant 6 can be determined. This embodiment also causes the aforementioned flashing effect, which can be observed by humans and / or automated systems.

[0107] In practice, users can use an inspection filter, where they can (manually, electrically, or mechanically) change the polarization to enhance the polarization characteristics of the hair as described above.

[0108] Detector 16 can be a special camera with a dedicated on-chip polarizer, such as Figure 10 As shown (Source: Sony, IEDM2016, Lecture No. 8.7). This imaging system uses individual pixels 44 of the imaging sensor 46. x The upper has several polarization filters with different orientations 42 x Filter 42 x This can be understood as being configured to polarize and filter the response light, as described herein. Filter 42 x Formed during the semiconductor process that forms the imaging sensor. Individual pixels 44 are also shown. x On-chip lens 40 x This on-chip lens 40x can also be called a condenser lens, and it is located at the corresponding image sensor pixel 44. x The imaging system described above. This camera with a dedicated on-chip polarizer is primarily designed for industrial inspection applications, where polarization detection can be used to highlight defects in materials, and can also be advantageously used to determine the growth stage of plants.

[0109] Figure 1C One embodiment is shown, wherein the illumination light 5 is at least partially polarized, and the response light 7 is polarized-filtered to obtain polarized response light 7'. The polarization-active portion can rotate the direction of the incident illumination light, while the remainder of the plant can reflect unpolarized light even if the illumination light is polarized. In this case, the response light will have a relatively strong component in one polarization direction, which can be achieved using, for example... Figure 1C The filter 8 shown becomes visible.

[0110] Figure 2A It shows the results at two corresponding moments. Figure 1C The system uses a polarization-filtered response light detector to obtain polarized light with different orientations at two moments. In this example, two different polarization filters 8a and 8b with different polarization directions are used. If the polarization-active part rotates its polarization direction while the rest of the plant reflects unpolarized light, the intensity of the response light from the polarization-active part will change as the polarization direction in the detection system 16 changes, while the intensity of the response light from the other parts of the plant will remain substantially the same. Therefore, this embodiment allows for easy detection of the polarization-active part. Note that in this embodiment, the response light 7' can be referred to as the first response light, and the response light 7'' can be referred to as the second response light.

[0111] Figure 2B It shows the two corresponding moments. Figure 1CThe system. Here, light source 14 illuminates the plant with first illumination light 5 at a first moment and with second illumination light 5' at a second moment. The first and second illumination lights have different polarization directions. In this example, the detector polarization filters the response light 7 in the same way at both moments. Based on reference... Figure 2A Using the same principles described, this embodiment also allows visualization of the polarization-active components of the plant.

[0112] In one embodiment, the polarization direction of the illumination light at a first moment is different from the polarization direction of the illumination light at a second moment, and the direction of polarization filtering at the first moment is different from that at the second moment. Preferably, the relative direction of the illumination light with respect to the polarization filtering direction performed by the detector 16 is different at the first moment than that relative direction at the second moment. For example, if the angle between the polarization direction of the illumination light and the polarization filter of the detector at the first moment is 10 degrees, then at the second moment that angle is not 10 degrees, for example, it is 50 degrees.

[0113] Figure 2C One embodiment is shown in which a light source 14 is configured to generate a first illumination light 5, and then an illumination light 5'. Specifically, this embodiment shows the light source illuminating a plant 6 with at least partially polarized light 5 (referred to as the first illumination light) at a first moment, and illuminating the plant 6 with unpolarized light 5' (referred to as the second illumination light) at a second moment. This embodiment also allows visualization of polarization-active portions. In a particular embodiment, the light source can continuously switch between unpolarized and polarized light, for example, once per second.

[0114] Figure 2D One embodiment is shown in which the detector is configured to polarize-filter the response light to obtain a first response light 7' having a first polarization direction, and subsequently not polarize-filter the response light to obtain a second, unpolarized response light 7. In a particular embodiment, the detector can continuously switch between a state in which it polarizes and filters the response light and a state in which it does not polarize and filters the response light, for example, once per second.

[0115] Preferably, such as Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 2C , Figure 2DAs shown, when the method involves detecting a first response light at a first moment and a second response light at a second moment, and involves influencing the polarization and / or polarization direction of the illumination light and / or response light somewhere between the first and second moments, the first and second response lights are detected rapidly after each other, for example, within 10 minutes, preferably within 1 minute, and most preferably within 10 seconds. To create a dynamic flickering effect, and when using a solid-state-based electronic solution, the system can transition between states even faster (e.g., at frequencies of 10–50 Hz), making the effect highly effective for human observers. These embodiments can be understood as performing this influence to cause a difference between the first and second response lights. Based on these differences, polarization-active portions can be identified, making them visible. Therefore, other factors are not expected to cause this difference, such as the formation of additional polarization-active portions on plants. This could happen, for example, if the second response light is detected a week later than the first response light.

[0116] Figure 3A A system 20 according to an embodiment is shown. The system includes a light source 14 and a detector 16. The light source 14 can be any light source described herein, and the detector can be any detector described herein. In this example, the light source 14 includes a light source 2 configured to produce unpolarized light and a polarizing filter 4. The light source 14 can therefore produce at least partially polarized illumination light 5. The detector can be configured to detect response light 7. In one embodiment, the detector is configured to polarize-filter the response light as described herein.

[0117] System 20 also includes a data processing system configured to determine the plant's growth stage based on the detected response light 7. For this purpose, detector 16 can be communicatively connected to the data processing system. The data processing system can be understood as a control module of system 20.

[0118] Preferably, the data processing system 100 is configured to control the light source 14. In one example, the data processing system 100 is configured to control the light source 2 and the polarizing filter 4. When appropriate software is executed, the data processing system 100 can be configured to turn the light source 2 on and off and / or control the intensity of the light generated by the light source 2, and can be configured to control the polarizing filter 4. The latter may include controlling the polarization direction of the filter 4, which can be achieved by mechanically moving (e.g., rotating) the polarizing filter 4 and / or by moving the polarizing filter 4 into and out of the path of the illumination light 5.

[0119] The data processing system 100 can also be configured to control the detector 16 when executing appropriate software, for example by controlling a polarization filter contained in the detector described herein. Such control can move (e.g., rotate) the polarization filter to change its polarization direction and / or completely shift the polarization into and out of the path of the response light 7.

[0120] Optionally, system 20 includes a display 22. Data processing system 100 may also control this display. In one example, the data processing system causes the display to show indications of a specific growth stage of the plant, such as instructions to harvest the plant.

[0121] Figure 3B An embodiment of the method is shown, wherein a light source 14 is suspended on the plant to be inspected, and a person uses a magnifying glass to detect the response light 7. The light source 14 can be any light source described herein. The method described herein can be readily implemented within existing lighting infrastructure, making this solution inexpensive and easy to maintain. Existing lighting structures, for example, only require the (controllable) polarizing filter to form the light source 14 described herein.

[0122] Figure 3C An embodiment of the system is shown, in which the light source 14 and the detector 16 are separate. A data processing system is not shown, but it may be located within the housing of the detector 16 (e.g., inside the camera 16); or at least communicatively connected to the detector 16, for example via a wireless network (e.g., Wi-Fi).

[0123] In one embodiment, a light signal can be applied if a determined growth stage indicates that the plant is ready for harvest. In this case, light source 14 or any other light can indicate this. In one example, a light source above the plant may begin to flash or change color to indicate to the farmer or automated harvesting system that the plant is ready for harvest.

[0124] Additionally or alternatively, the data processing system 100 can transmit a message to another device (such as a farmer's handheld device) that a specific plant has reached full maturity. When these are analyzed, the data processing system can generate overall figures on the maturity of various plants. Machine learning-assisted tools can use these figures to accumulate data from previous harvests and correlate them with historical climate and light parameters to aid / automate decision-making.

[0125] Figure 3D An embodiment of system 20 is shown, wherein system 20 is integrated into a handheld device. Specifically, system 20 can be understood as an inspection tool polarizer and human detection for on-site observation. The inspection tool dynamically controls the polarization of the illuminator and the observation sensor. The user aims the illumination light 5 at the area containing the hair-like structures (e.g., using a "crosshair"), and the dynamic polarization effect can be seen on the built-in display 22, which in this example is located on the back of the handheld device.

[0126] Figure 4AThe anatomical properties of cannabis plants, particularly those in the Cannabis genus, are shown. The female flowers, which are typically consumed, are located at the apex of the plant. Furthermore, during plant growth, especially during the maturation of the female flowers, polarization-active trichomes develop on these flowers. Therefore, as the flower develops, its optical properties, particularly its polarization characteristics, change. Thus, these changes in polarization characteristics can be linked to different growth stages.

[0127] Hemp flowers typically go through eight stages to mature. These eight stages can be understood as sub-growth stages within the flower's maturation growth stage. It should be understood that determining the plant's growth stages can be reflected in determining these sub-growth stages. The eight stages of hemp flower maturation are as follows: 1. Female hemp flowers do not contain any stigma; 2- The first stigma appears on the twisted dough stick; 3- Three layers of stigmas appear on the twisted dough stick; 4- The stigma forms continuously on the flower; 5- The flower forms a corolla with a white / milky white stigma; 6. New flowers cease to bloom, and the stigma begins to dry (turn brown) - trichome scoring also begins; 7- The stigma is dry, and the trichomes are erect and milky white; 8. The flower buds are ripe and ready for harvest; the trichomes are plump, with clear color, and less than 10% of them turn amber.

[0128] The transition from stage 7 to stage 8 is very rapid and difficult to detect. The method disclosed in this paper makes it possible to accurately determine when stage 8 has been reached.

[0129] Figure 4B This explains why trichomes reflect polarized light 7a, while the rest of the plant (e.g., leaves) reflects unpolarized light 7b. Light 7a and light 7b together can be understood as constituting the response light.

[0130] Leaves can partially reflect polarized light 7a and unpolarized light 7b. However, as shown, due to light scattering within the leaf, the degree of polarization (DOP) is low (more unpolarized light is reflected than polarized light). In the case of trichomes, the DOP is high, resulting in more polarized reflected light. When observing leaves with varying polarization directions, unpolarized reflected light will dominate, and therefore the polarization direction will not affect the intensity of the reflected light. However, when observing trichomes with different polarization directions, the amount of light observed can vary considerably.

[0131] If a polarized incident beam is used instead of the unpolarized incident beam 5, the leaf will still reflect unpolarized light due to internal scattering. Therefore, regardless of the polarization of the light source, the leaf will still reflect light in any polarization direction. However, the hair-like structure only reflects light in a specific polarization direction. By changing the polarization direction of the light source over time, the observer will perceive a shimmering effect from the hair-like structure even without a visible polarizing filter.

[0132] Figure 5 An image 30 is shown of a plant, particularly a cannabis plant, and more specifically, a female flower of a cannabis plant. This image 30 can be represented by data obtained by measuring the response light. The image can be understood as indicating the response light from various parts of the plant. Figure 5 It is also shown that a region of interest 32 can be identified, corresponding to a plant part where polarization-active components are expected to form or disappear during plant growth. In this example, region of interest 32 corresponds to the female flower of a cannabis plant. Subsequent analysis can be restricted to this region of interest 32, which allows for efficient and rapid data processing, which in turn allows for rapid (e.g., real-time) determination of the plant's growth stage.

[0133] As described above, this region of interest 32 can be identified based on machine learning techniques. Furthermore, this region of interest 32 can be implemented using 3D sensing techniques. In this case, 3D ranging signals are used to obtain a representation of the plant's three-dimensional shape. Then, plant parts on which polarization-active regions are expected to form or disappear can be identified based on the plant's characteristic morphology.

[0134] Figure 6A and Figure 6B First and second images of a plant, specifically a cannabis plant, and more particularly, a female flower of a cannabis plant are shown respectively. Clearly, Figure 6B Including than Figure 6A The corresponding region in the image has a higher intensity. Two image regions in different images can be understood as corresponding image regions if they both represent the same part of the plant. This difference between the first and second images is caused by the subsequent detection of the first and second response lights, where the first and second response lights are different. This difference between the first and second response lights can be achieved by one or more of the methods described above, such as changing the polarization of the illumination, changing the polarization filter in the detection path, switching between polarized and unpolarized illumination light, switching between detection with polarization filtering and detection without polarization filtering, or a combination of these actions.

[0135] In particular, the trichomes shown as white dots have a higher intensity in the second image than in the first image. These images demonstrate that the polarization of light can be used to detect the location and maturity (development) of cannabis plants (e.g., Cannabis species).

[0136] Figure 7 This demonstrates how polarization characteristics can be recorded over time, providing growers with historical trends to make decisions regarding growth control, harvesting, or the effectiveness of light formulations.

[0137] Figure 7 The graph shows the total response light intensity from a specific flower on a specific plant over time. Each data point in the graph corresponds to a dataset representing the response light detected at the corresponding moment. The evolution of the response light over time can help determine the current or even future growth stage.

[0138] Figure 7 The diagram shows that for a specific flower, the initial total intensity of the response light is relatively low, and at some point the total intensity increases until a steady period of constant total light intensity is reached. For example, if the total intensity has reached this steady period, the flower may be ready for harvest. Therefore, in this case, the growth stage can be determined based on the evolution of the response light over time, indicated by multiple datasets.

[0139] For different plants, the duration of the increase in total light intensity may be similar. Then, once the increase has begun, it becomes possible to predict when the total light intensity will reach a plateau. Therefore, this could allow for the prediction of when flowers are ready for harvest.

[0140] Figure 8A An embodiment of a system 20 including an augmented reality system 102 is shown. In such an embodiment, a display 22 can be understood as part of the augmented reality system. The augmented reality system is configured to present an indication of a defined growth stage of a plant on the display, allowing a user viewing the display to see the plant and its defined growth stage. The display 22 may be a transparent display.

[0141] Detector 16 may include a camera for recording images of the plant. Augmented reality system 102 may be configured to at least partially display these recorded images on display 22.

[0142] Flower pickers / farmers can enter the greenhouse to check the maturity of the hemp flowers and pick the flowers ready for harvest.

[0143] Preferably, the display and camera are integrated into a head-mounted device such as AR glasses, allowing the picker / farmer to use his / her hands to perform "normal" picking actions, such as picking flowers.

[0144] Through AR glasses, pickers can see “real” plants and overlay the harvestable locations (flowers) onto the image. Information about ripeness is created in real-time based on sensors embedded in / in the AR glasses. This sensing is combined with image processing based on object (ripe flower) recognition (and potentially further, specific light stimulation). The information provided to the picker allows him / her to harvest the ripe flowers.

[0145] Flowers in early stages of development (e.g., grouped according to "completely immature" / "almost ripe" / "almost ready for harvest") can also be tagged during data analysis, and this information can be stored for later use, thus creating predictive capabilities. Therefore, the day after pickers re-enter the greenhouse (or at the next harvest time), flowers judged to be "almost ready for harvest" may be prioritized for pickers; for example, pickers may be directed to these locations in the greenhouse, allowing for a more efficient workflow (preventing the viewing of plants that are not ready for harvest at all, or plants whose flowers have already been harvested).

[0146] To enable this last function, the camera on the AR glasses can also track the picker's movements (generating stored information about the harvested plants / flowers, who harvested them, how many were harvested, when, and at what maturity level, also allowing for the capture of potential errors and control over ownership of the harvested flowers). Additional expert input data provided by the picker can also be captured using this camera, for example, based on a gesture-based UI. Alternatively, audio functionality can be used as the UI. Picker / expert data can be, for example, parameters such as consistency / inconsistency with machine learning's judgment of maturity, or any type of anomaly flagged via the UI in the crop (e.g., disease outbreaks) or horticultural system.

[0147] This information may not be needed in real time and can be stored (later) throughout the harvesting tool and used for the next harvest cycle or for more in-depth data analysis (e.g., optimizing forecasts for current and future crops).

[0148] Figure 8B An example of a user's perception while viewing display 22 of an augmented reality system is shown. The augmented reality display 22 presents a defined growth stage of a plant next to it, in this example, in the form of instructing the user to harvest the plant.

[0149] When the display is opaque, the display 22 can also present, for example, plant images recorded by the camera 16. In this case, indicators of the defined growth stage are overlaid on the recorded image, for example.

[0150] Figure 9A block diagram illustrating a data processing system according to an embodiment is shown. The data processing system may be embedded in a single device (e.g., AR goggles) or distributed across a combination of wearable devices and real-time accessible cloud-enabled functions (e.g., storage and data processing). Or other similar options.

[0151] like Figure 9 As shown, the data processing system 100 may include at least one processor 102 coupled to a memory element 104 via a system bus 106. Thus, the data processing system can store program code within the memory element 104. Furthermore, the processor 102 can execute program code accessed from the memory element 104 via the system bus 106. In one aspect, the data processing system may be implemented as a computer suitable for storing and / or executing program code. However, it should be understood that the data processing system 100 may be implemented in the form of any system including a processor and memory capable of performing the functions described herein.

[0152] Memory element 104 may include one or more physical memory devices, such as, for example, local memory 108 and one or more mass storage devices 110. Local memory may refer to random access memory or other non-persistent storage devices generally used during the actual execution of the program code. Mass storage devices may be implemented as hard disk drives or other persistent data storage devices. Processing system 100 may also include one or more cache memories (not shown) that provide temporary storage for at least some of the program code to reduce the number of times the program code must be retrieved from mass storage device 110 during execution.

[0153] Optionally, the input / output (I / O) devices depicted as input device 112 and output device 114 can be coupled to the data processing system. Examples of input devices may include, but are not limited to, a keyboard, or a pointing device such as a mouse. Examples of output devices may include, but are not limited to, a monitor or display, or a speaker. The input and / or output devices can be coupled to the data processing system directly or through an intermediate I / O controller.

[0154] In embodiments, the input and output devices may be implemented as a combined input / output device (in... Figure 9(Dashed lines are used to illustrate input device 112 and output device 114). An example of such a combined device is a touch-sensitive display, sometimes also called a “touchscreen display” or simply a “touchscreen.” In such embodiments, input to the device can be provided by the movement of a physical object, such as, for example, a user’s finger or stylus, on or near the touchscreen display. Network adapter 116 may also be coupled to the data processing system to enable it to couple to other systems, computer systems, remote network devices, and / or remote storage devices via an intermediate private or public network. The network adapter may include a data receiver for receiving data transmitted to the data processing system 100 from said systems, devices, and / or networks, and a data transmitter for transmitting data from the data processing system 100 to said systems, devices, and / or networks. Modems, cable modems, and Ethernet cards are examples of different types of network adapters that can be used with the data processing system 100.

[0155] like Figure 9 As illustrated, memory element 104 can store application program 118. In various embodiments, application program 118 may be stored in local memory 108, one or more mass storage devices 110, or separately from local memory and mass storage devices. It should be understood that data processing system 100 may further execute an operating system that facilitates the execution of application program 118. Figure 9 (Not shown in the image). The application program 118, implemented in the form of executable program code, can be executed by the data processing system 100 (e.g., by the processor 102). In response to executing the application program, the data processing system 100 can be configured to perform one or more operational or method steps described herein.

[0156] In one aspect of the invention, the data processing system 100 may represent the augmented reality system described herein.

[0157] In another aspect, data processing system 100 may represent a client data processing system. In this case, application 118 may represent a client application that, when executed, configures data processing system 100 to perform the various functions described herein with reference to "client". Examples of clients may include, but are not limited to, personal computers, portable computers, mobile phones, etc.

[0158] In another aspect, the data processing system 100 may represent a server. For example, the data processing system may represent an (HTTP) server, in which case the application 118 may be configured to perform (HTTP) server operations when executed.

[0159] Various embodiments of the present invention can be implemented as a program product for use with a computer system, wherein the program(s) of the program product define the functionality of the embodiments (including the methods described herein). In one embodiment, the program(s) may be contained on a variety of non-transitory computer-readable storage media, wherein, as used herein, the expression “non-transitory computer-readable storage media” includes all computer-readable media, with the sole exception of transient propagation signals. In another embodiment, the program(s) may be contained on a variety of transient computer-readable storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media on which information is permanently stored (e.g., read-only memory devices within a computer, such as CD-ROM discs readable by a CD-ROM drive, ROM chips, or any type of solid-state non-volatile semiconductor memory); and (ii) writable storage media on which changeable information is stored (e.g., flash memory, floppy disks within a floppy disk drive or hard disk drive, or any type of solid-state random access semiconductor memory). The computer program may run on the processor 102 described herein.

[0160] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” (“a” or “an”) and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms “comprise” and / or “comprising” specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0161] All the means or steps plus functional elements in the following claims are intended to include any structure, material, action, and equivalent for performing a function in combination with other claimed elements as specifically claimed. Descriptions of embodiments of the invention have been shown for illustrative purposes, but are not intended to be exhaustive or limited to the embodiments in the disclosed forms. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. Embodiments have been chosen and described in order to best explain the principles of the invention and some practical applications, and to enable others skilled in the art to understand the invention with respect to various embodiments with various modifications suitable for particular intended uses.

Claims

1. A method for determining the growth stage of a plant, the method comprising: - Illuminate the plant with illumination light, which elicits a response light from the plant. - Detect the response light from the plant, and - Determine the growth stage of the plant based on the detected response light; The detection of the response light from the plant includes: - Control the polarization filter to polarize light in the first polarization direction; - The response light is polarized and filtered using the polarization filter to obtain a first response light having the first polarization direction, and the first response light is detected. Subsequently, the polarization filter is controlled to polarize the light in the second polarization direction, and the response light is polarized and filtered using the polarization filter to obtain a second response light having the second polarization direction, and the second response light is detected, wherein the second polarization direction is different from the first polarization direction. The determination of the plant's growth stage based on the detected response light includes: - The growth stage of the plant is determined based on the difference between the detected first and second response lights caused by the polarization-active portion of the plant. Detecting the response light includes obtaining data representing the detected response light, wherein the plant growth stage is determined based on the obtained data, wherein the obtained data indicates at least one of the following: -The total intensity of the response light -The average intensity of the response light -The maximum intensity of the response light - The minimum intensity of the response light.

2. The method of claim 1, wherein the step of obtaining data representing the detected response light is performed by a computer.

3. The method of claim 1, wherein detecting the response light includes the step of obtaining data representing an image of the plant, the image indicating the detected response light of various parts of the plant, and the image including one or more image regions corresponding to one or more polarization-active portions of the plant, the one or more polarization-active portions comprising optically active materials and / or comprising materials exhibiting polarization-selective reflection.

4. The method of claim 3, wherein the step of obtaining data representing an image of the plant is performed by a computer.

5. The method of claim 1, wherein illuminating the plant with illumination light comprises: The plant is illuminated with at least partially polarized light. The illumination of the plant with at least partially polarized light comprises: - Illuminate the plant with a first illumination light, wherein the first response light is detected based on the first illumination light. - Illuminate the plant with a second illumination light, wherein the second response light is detected based on the second illumination light.

6. The method according to claim 5, Wherein the first illumination light is at least partially polarized light, and the second illumination light is unpolarized light, or The first illumination light is at least partially polarized in a first direction, and the second illumination light is at least partially polarized in a second direction different from the first direction.

7. The method of claim 5, further comprising: - Obtain multiple datasets, each representing the response light detected at a given time. - The growth stage of the plant is determined based on the evolution of the response light over time as indicated by the multiple datasets.

8. The method of claim 5, wherein detecting the response light comprises The step of obtaining data representing a first image and a second image of the plant, wherein the first image indicates a first response light from various parts of the plant, and the second image indicates a second response light from various parts of the plant, wherein... The first image includes one or more image regions corresponding to one or more parts of the plant, and wherein... The second image includes one or more image regions corresponding to one or more parts of the plant, wherein The image region of the second image has a higher or lower intensity than the image region of the first image, and The growth stage of the plant is determined based on the intensity difference of the image regions between the first image and the second image.

9. The method of claim 8, wherein the step of obtaining data representing the first and second images of the plant is performed by a computer.

10. The method according to claim 3, 4, 8 or 9, further comprising: Identify regions of interest (ROIs) in one or more images represented by the acquired data, wherein the ROIs correspond to one or more plant segments in which polarization-active portions are expected to form or disappear during the plant's growth. Analyze the identified regions of interest to determine the growth stage of the plant.

11. The method according to any one of claims 1-9, further comprising: The indicator of the determined growth stage of the plant is presented on the display of the augmented reality system, so that the user viewing the augmented reality display can see the plant and its determined growth stage.

12. A system for determining the growth stage of a plant (6), the system comprising - A light source (14) for illuminating the plant with illumination light (5), which causes a response light (7) from the plant (6). - A polarizing filter for polarizing the response light; - Detector (16), for detecting the response light (7) from the plant (6), and - A data processing system (100) is configured to determine the growth stage of the plant (6) based on the detected response light (7); The detector (16) mentioned above includes a polarizing filter; and The data processing system (100) therein is adapted to control the light source (14) and the detector (16). - Control the polarization filter to polarize light in the first polarization direction; - The response light is polarized and filtered using the polarization filter to obtain a first response light having the first polarization direction, and the first response light is detected. - Subsequently, the polarization filter is controlled to polarize the light in the second polarization direction, and the response light is polarized and filtered using the polarization filter to obtain a second response light having the second polarization direction, and the second response light is detected, wherein the second polarization direction is different from the first polarization direction, and - The growth stage of the plant is determined based on the difference between the detected first and second response lights caused by the polarization-active portion of the plant. Detecting the response light includes obtaining data representing the detected response light, wherein the plant growth stage is determined based on the obtained data, wherein the obtained data indicates at least one of the following: -The total intensity of the response light -The average intensity of the response light -The maximum intensity of the response light - The minimum intensity of the response light.

13. The system of claim 12, wherein the illumination light (5) is at least partially polarized, and The data processing system (100) therein is adapted to control the light source (14). - Illuminate the plant with a first illumination light, wherein the first illumination light evokes a first response light from the plant, and detect the first response light from the plant. -Then the plant is illuminated with a second illumination light, wherein the second illumination light elicits a second response light from the plant, and the second response light from the plant is detected, and - The growth stage of the plant is determined based on the detected first and second response lights.

14. The system according to claim 12 or 13, further comprising: - An augmented reality system (102) including a display (22), wherein The augmented reality system (102) is configured to present an indication of the determined growth stage of the plant on the display (22), so that a user viewing the display (22) can see the plant and its determined growth stage.

15. The system according to claim 12 or 13, wherein, The detector includes a camera for recording images of the plant.

16. The system according to claim 14, wherein, The detector includes a camera for recording images of the plant, and the augmented reality system is configured to at least partially display the recorded images on the display.

Images