Time-of-Flight Sensing for Horticulture

Abstract

The present invention provides a sensing system (1000) for example agricultural applications, including a radiation generator (100), a sensing device (200), and a control system (300) functionally coupled to the radiation generator (100) and the sensing device (200), wherein the sensing system (1000) has one or more time-of-flight sensing operating modes, wherein the generator (100) is configured to generate pulses of radiation (111) in one or more time-of-flight sensing operating modes, and wherein the sensing device (200) is configured to sense, in one or more time-of-flight sensing modes, the wavelength-dependent spectral intensity of the radiation received by the sensing device (200) as a function of time, to provide a sensing system signal; wherein the sensing system signal indicates the wavelength-dependent spectral intensity distribution of the received radiation as a function of time in one or more time-of-flight sensing modes.

Description

Technical Field

[0001] The present invention relates to a sensing system (and its use). The present invention also relates to an agricultural facility (and its use). Still further, the present invention relates to a plant monitoring method. The present invention also relates to a computer program product (for executing such a plant monitoring method, for example, using the sensing system). Background Art

[0002] Methods, devices and systems for detecting, sensing and identifying objects are known in the art. For example, WO2018 / 194920 describes methods, devices and systems for detecting, sensing and identifying objects using modulated polarized light beams. An example polarization-sensitive device includes an illumination source and a modulator coupled to the illumination source to produce an output light beam, wherein the polarization state or polarization parameter of the output light beam is modulated to produce a plurality of modulated polarized light beams. The device also includes a polarization-sensitive detector positioned to receive a reflected portion of the modulated polarized light beam after reflection from an object and to generate information indicating the modulation and polarization state of the received light beam. According to WO2018 / 194920, this information can be used to enable determination of the distance between the polarization-sensitive device and the object, or to determine polarization-specific properties of the object. WO2016 / 176612A1 discloses an airborne multispectral reflectance LIDAR system. Summary of the Invention

[0003] In large-scale industrial horticultural settings (such as greenhouses and urban farms), as well as in open-air horticulture, there is a trend toward utilizing automated inspection and processing to replace or support farmers and workers. This automation is becoming necessary and, in the future, almost mandatory, because observing, estimating (e.g., regarding ripening, disease onset, etc.), and processing (e.g., harvesting) crops and fruits is extremely time-consuming, manual work is very expensive, and may require specialized personnel. Furthermore, even experts can make mistakes and their analysis or actions can be erratic and subjective. Furthermore, when 24 / 7 monitoring is required, it may be more comfortable to have machines perform monitoring at least part of the time. Therefore, in addition to using automated harvesting and crop processing (e.g., spraying) machines, automated plant and crop / fruit sensing may be desirable. However, prior art systems may have less functionality than desired. Therefore, one aspect of the present invention is to provide an alternative sensing system (and / or agricultural facility and / or (plant) monitoring method, and / or computer program product) that preferably further at least partially alleviates one or more of the aforementioned shortcomings. An object of the present invention is to overcome or ameliorate at least one of the shortcomings of the prior art, or to provide a useful alternative.

[0004] In a first aspect, the present invention provides a sensing system, particularly for agricultural applications, comprising a radiation generator ("generator") and a sensing device (which may be functionally coupled to the generator). Furthermore, the sensing system may include a control system, which may be functionally coupled to the radiation generator and the sensing device (which may also be indicated as a "sensor"). In particular, in embodiments, the sensing system may have one or more time-of-flight sensing operating modes, wherein the generator is configured to generate pulses of radiation in the one or more time-of-flight sensing operating modes. In particular, the sensing device may be configured to sense radiation from a sensed object as a function of time in the one or more time-of-flight sensing modes to provide a sensing system signal. In more specific embodiments, the sensing device may be configured to sense the (wavelength-dependent) spectral intensity—such as a wavelength-dependent spectral intensity distribution—of (incoming) radiation received by the sensing device as a function of time in the one or more time-of-flight sensing modes to provide a sensing system signal. In particular, the spectral intensity, such as the wavelength-dependent spectral intensity distribution, may be a spectral power distribution. The sensing system signal indicates the wavelength-dependent spectral intensity distribution of the received radiation as a function of time in the one or more time-of-flight sensing modes. Thus, in particular in an embodiment, the present invention (therefore) provides a sensing system (in particular for agricultural applications), the sensing system comprising a radiation generator, a sensing device (functionally coupled to the generator), and a control system functionally coupled to the radiation generator and the sensing device, wherein the sensing system has one or more time-of-flight sensing operating modes, wherein the generator is configured to generate pulses of radiation in the one or more time-of-flight sensing operating modes, and wherein the sensing device can be configured to sense radiation from a sensed object as a function of time in the one or more time-of-flight sensing modes to provide a sensing system signal. Furthermore, in particular in an embodiment, the present invention (therefore) provides a sensing system (in particular for agricultural applications), the sensing system comprising a radiation generator, a sensing device (functionally coupled to the generator), and a control system functionally coupled to the radiation generator and the sensing device, wherein the sensing system has one or more time-of-flight sensing operating modes, wherein the generator is configured to generate pulses of radiation in the one or more time-of-flight sensing operating modes, and wherein the sensing device is configured to sense the (wavelength-dependent) spectral intensity—such as the wavelength-dependent spectral intensity distribution—of the radiation received by the sensing device as a function of time in the one or more time-of-flight sensing modes to provide a sensing system signal.

[0005] Furthermore, the sensing system signals can be useful as such, for example, to obtain a better overview of the state of the object(s). However, the sensing system signals can also be used to control (another) device (or apparatus or system). Thus, the present invention can further provide that the sensing system is functionally coupled to an agricultural device, wherein the agricultural device comprises a lighting device for illuminating plants or plant parts with a light recipe, wherein the control system controls the agricultural device to illuminate the plants or plant parts with the light recipe based on the sensing system signals.

[0006] Utilizing (a plurality of) such sensing systems, the present invention provides, in embodiments, a time-of-flight camera with multispectral illumination for horticultural applications. In embodiments, the active illumination of the time-of-flight camera can use multiple wavelengths to acquire range information for obtaining shape properties of a plant. In embodiments, this can be combined with capturing and analyzing intensity signals at different wavelengths, which can contain information about plant health and development and can be used, for example, to classify different plant parts and / or determine the status of these plant parts. Earlier detection of disease and / or obtaining a more accurate status of the plant or individual plants may be possible. In this way, treatment and / or other actions, such as pruning, disease management, or harvesting, can also be performed more efficiently. The term "range information" may specifically refer to distance information.

[0007] As mentioned above, the sensing system can be particularly useful in agricultural applications, such as horticultural applications. The term "agriculture" may specifically refer to the science and technology of cultivating plants and livestock. Here, the term "agriculture" may specifically refer to the science and technology of cultivating plants. This may be indoor or outdoor farming. This may be farming in a greenhouse, in an enclosed plant factory, etc. It may be based solely on daylight, a combination of daylight and artificial light, or solely on artificial light. The term "agriculture" may particularly include at least "horticulture." The present invention particularly relates to horticultural applications, among other things. The term "horticulture" refers to the (intensive) cultivation of plants for human use, a wide variety of activities, including edible plants (fruits, vegetables, mushrooms, culinary herbs) and non-edible crops (flowers, trees and shrubs, lawn grasses, hops, grapes, herbs). Horticulture is a branch of agriculture that involves the art, science, technology, and business of growing plants. It may include the cultivation of medicinal plants, fruits, vegetables, nuts, seeds, herbs, sprouts, mushrooms, algae, flowers, seaweed, and non-food crops such as grasses and ornamental trees and plants. Herein, the term "plant" is used to refer to essentially any species selected from medicinal plants, vegetables, herbs, sprouts, mushrooms, nut-bearing plants, seed-bearing plants, flowering plants, fruit-bearing plants, non-edible crops such as grasses and ornamental trees, etc.

[0008] In this article, the term "plant" is used for essentially all stages. The term "plant part" may refer to roots, stems, leaves, fruits (if any), etc. The term "horticulture" relates to the (intensive) cultivation of plants for human use, and its activities are very diverse, including edible plants (fruits, vegetables, mushrooms, culinary herbs) and non-edible crops (flowers, trees and shrubs, lawn grasses, hops, grapes, herbs). Horticulture is a branch of agriculture that involves the art, science, technology, and commerce of growing plants. It may include the cultivation of medicinal plants, fruits, vegetables, nuts, seeds, herbs, bean sprouts, mushrooms, algae, flowers, seaweed, and non-food crops such as grasses and ornamental trees and plants. Here, the term "plant" is used to refer to essentially any species selected from medicinal plants, vegetables, herbs, sprouts, mushrooms, nut-bearing plants, seed-bearing plants, flowering plants, fruit-bearing plants, non-edible crops such as grasses and ornamental trees, etc. Even more particularly, the term "plant" is used to refer to essentially any species selected from the group consisting of medicinal plants, vegetables, herbs, sprouts, nut-bearing plants, seed-bearing plants, flowering plants, fruit-bearing plants, non-food crops. The terms "horticultural" and "plant" may also refer to one or more of cannabis and poppy (especially Papaver somniferum, also known as "opium poppy" or "breadseed poppy").

[0009] The term "crop" is used herein to refer to horticultural plants that are growing or have been growing. Similar plants grown on a large scale for food, clothing, etc. may be referred to as crops. Crops are non-animal species or varieties that are grown for harvest, for example as food, livestock feed, fuel, or for any other economic purpose. The term "crop" may also refer to a variety of crops. Horticultural crops may particularly refer to food crops (tomatoes, peppers, cucumbers, and lettuce), as well as (potentially) plants with such crops, for example tomato plants, pepper plants, cucumber plants, etc. Horticulture in general may refer here to, for example, both crop and non-crop plants. Examples of crop plants are rice, wheat, barley, oats, chickpeas, peas, cowpeas, lentils, mung beans, black beans, soybeans, common beans, moth beans, linseed, sesame, Khesari, sunflower, eggplant, tomato, cucumber, okra, peanuts, potatoes, corn, loosestrife, rye, alfalfa, radish, cabbage, lettuce, pepper, sunflower, sugar beet, castor, red clover, white clover, safflower, spinach, onion, garlic, turnip, pumpkin, melon, watermelon, cucumber, squash, kenaf, oil palm, with particular interest being tomato, cucumber, pepper, lettuce, watermelon, papaya, apple, pear, peach, cherry, grape and strawberry.

[0010] The term "plant" may also refer to a seed or a seedling. Thus, the term "plant" may generally refer to any stage from seed to (mature) plant. The term "plant" may also refer to a plurality of (different) plants.

[0011] In particular, the sensing system comprises a radiation generator and a sensing device. In addition, the sensing system may also comprise a control system.

[0012] The radiation generator is configured to generate radiation (such as one or more of UV, visible light, and (N)IR).

[0013] In an embodiment, the radiation generator can be configured to generate narrowband radiation, such as, in particular, substantially monochromatic light. With monochromatic light, (undesirable) wavelength-dependent penetration depth and / or wavelength-dependent transmission and / or wavelength-dependent reflection can be prevented. This can allow for relatively easy processing of radiation that returns from the object and can be sensed by the sensing device.

[0014] In an embodiment, the radiation generator may comprise a solid state light source, such as a light emitting diode or a laser diode.

[0015] In an embodiment, the radiation generator may include a plurality of such substantially monochromatic light sources. In such an embodiment, it may be possible to subject the object to different wavelengths. From the detected radiation, information that may be wavelength-dependent may be derived. For example, the penetration of radiation and / or the transmission of radiation and / or the reflection of radiation and / or the scattering of radiation and / or fluorescence (due to irradiation with radiation) may depend on, for example, the plant part, the stage of growth, the presence of disease, the health of the plant, etc., and may therefore be wavelength-dependent, as each may have or result in different optical properties, such as, for example, selected from transmission, reflection, and fluorescence.

[0016] In an embodiment, the radiation generator may include a broadband emitter and one or more optical devices for selecting a narrowband, such as a grating, a refractive element, and / or a filter. Also in such an embodiment, the radiation generator may be configured to generate substantially monochromatic light (of a substantially single wavelength). The term "narrowband" may refer to a band with a full width at half maximum equal to or less than 50 nm; the term "broadband" may refer to a band with a full width at half maximum greater than 50 nm.

[0017] In an embodiment, the radiation generator may comprise an optical amplifier. An optical amplifier is a device that directly amplifies an optical signal without first converting it into an electrical signal. The optical amplifier may comprise a laser amplifier, such as a solid-state amplifier or a doped fiber amplifier, or a semiconductor optical amplifier, among others. With an optical amplifier, a substantially monochromatic wavelength may be selected, although the wavelength may be selected from a wider range, thereby allowing tunability. In such an embodiment, it may be possible to subject the object to different wavelengths. From the detected radiation, information, which may be wavelength-dependent, may be derived (see also above).

[0018] Thus, in one embodiment, the radiation generator may be configured to generate substantially monochromatic light (of a substantially single wavelength). In other embodiments, the radiation generator may be configured to generate substantially monochromatic light at two or more wavelengths. This may occur simultaneously. However, in particular, this may be sequential.

[0019] In other embodiments, the radiation generator can be configured to generate polychromatic light. This can be particularly useful when the sensing device can also be configured to sense multiple different wavelengths. In such embodiments, the effects of radiation of different wavelengths on an object can be sensed (over time).

[0020] Alternatively or additionally, the radiation generator can be configured to generate radiation with a specific polarization. As described above, the penetration of radiation and / or the transmission of radiation and / or the reflection of radiation and / or the scattering of radiation and / or fluorescence (due to irradiation with radiation) can, for example, depend on the plant part, the stage of growth, the presence of disease, the health of the plant, etc., and can therefore be polarization-dependent, as each can have or result in different polarization-dependent optics.

[0021] To this end, the radiation generator may comprise a radiation emitter configured to generate radiation having a specific polarization.Alternatively or additionally, the radiation generator may comprise a radiation emitter generating unpolarized radiation, and one or more optical devices imparting polarization to the radiation.

[0022] In certain embodiments, the radiation generator can be configured to generate radiation with two or more different polarizations, such as linearly polarized light and circularly polarized light, or left-polarized light and right-polarized light, etc. In such embodiments, it may be possible to subject an object to different polarizations. From the detected radiation, information that may depend on polarization may be derived. For example, the detection of chiral molecules (such as sugars, proteins, etc.) may be facilitated. These can be detected, for example, by rotating the polarization plane of the linearly polarized light.

[0023] In an embodiment, the polarization of the radiation provided by the radiation generator may vary over time and / or over spatial position over an emission window of the radiation generator.

[0024] The radiation generator may be configured to provide radiation continuously. However, in particular, the generated radiation may be configured to provide radiation in a pulsed manner. Thus, the radiation may be provided in a pulsed manner, wherein the pulses have the same length and intensity, or the pulses vary in one or more of their duration (pulse width) and intensity.

[0025] In particular, in embodiments, the radiation generator may be configured to provide multiple pulses of radiation, wherein the wavelength(s) of the radiation in the pulses of radiation are different. In this way, the effects of radiation having different wavelengths (or different wavelength distributions) may be sensed over time (in embodiments of ToF mode).

[0026] As will be known to those skilled in the art, multiple pulses of two or more different radiation are typically generated in succession with (at least) a sensing time in between.The two or more pulses may differ, for example, in wavelength and / or polarization.

[0027] In particular, for time-of-flight mode measurements, it may be desirable to generate radiation in a pulsed manner. Even more particularly, the pulse may have a relatively short duration. For example, the pulse may have a pulse width equal to or less than 100 ns (nanoseconds), such as particularly equal to or less than 50 ns, such as equal to or less than 2 ns. Even more particularly, the pulse width may be equal to or less than 1 ns, such as equal to or less than 100 ps (picoseconds), such as particularly equal to or less than 10 ps, ​​such as equal to or less than 1 ps. Again even more particularly, the pulse width may be equal to or less than 100 fs (femtoseconds), such as particularly equal to or less than 10 fs, such as equal to or less than 1 fs. In an embodiment, a 1D ToF system based on a pulsed laser may include a laser pulse transmitter, (necessary) optical devices, (two) receiver channels, and a TDC (time-to-digital converter) or a high-speed ADC (analog-to-digital converter). The laser pulse transmitter may emit short light pulses (such as typically 2 to 50 ns in an embodiment) toward the object, and a transmission event may be defined optically by detecting a fraction of the pulses, or electrically based on a drive signal of the laser diode.

[0028] As described above, the generator is particularly configured to generate pulses of radiation (in one or more time-of-flight sensing operating modes). The term "pulse of radiation" may refer in an embodiment to a pulse defined by two (infinitely small) periods of zero radiation intensity. However, the term "pulse of radiation" may also refer to a modulation of a background signal; in such an embodiment, a pulse may be defined by two (infinitely small) minima of the radiation intensity. Furthermore, the term "pulse of radiation" may also refer to a plurality of such pulses. In an embodiment, a plurality of identical pulses may be generated. In other embodiments, a plurality of pulses may be generated, two or more of which may be different. The pulses may differ (or be the same) in pulse width and pulse height. When multiple pulses are generated, the frequency may be fixed, although this is not necessarily the case.

[0029] In exemplary embodiments, the term “radiation generator” may also refer to a plurality of (different) radiation generators.

[0030] As mentioned above, the sensing system may further comprise a sensing device.

[0031] The sensing device may be configured to sense a specific wavelength.

[0032] The sensing apparatus may also include a sensing device configured to sense multiple wavelengths and one or more optical devices, such as gratings, refractive elements, and / or filters, to select a particular wavelength to be sensed. For example, in certain embodiments, the sensing apparatus may be configured to measure the spectral intensity distribution of the radiation, such as the spectral power distribution.

[0033] Thus, in certain embodiments, the sensing device may be able to sense radiation with different wavelengths (and separate its signals), such as polychromatic light. In particular, in certain embodiments, the sensing device may be able to sense radiation with different wavelengths, such as polychromatic light, over time (see also below). This may be useful in time-of-flight mode. Furthermore, this may be used to measure luminescence and / or this may be used to measure radiation from a radiation generator configured to generate polychromatic light. As described above, this is particularly useful when the sensing device is configured to sense multiple different wavelengths. In such an embodiment, the effects of radiation of different wavelengths on an object (over time) may be sensed.

[0034] The sensing arrangement may be configured to sense a specific polarization, ie radiation having a specific polarization.

[0035] The sensing arrangement may also comprise a sensing device configured to sense a plurality of different polarizations (ie different radiation, at least of different polarization types), and one or more optical components to select a specific polarization.

[0036] In embodiments, the polarization of the radiation sensed by the sensing device may vary over time and / or over spatial position on an entrance window of the sensing device.

[0037] In particular, the sensing device may be configured to measure the radiation time dependency. In this way, for example, a time-of-flight measurement may be performed.

[0038] As described above, the sensing device can be particularly configured to sense radiation from the object. In an embodiment, the radiation can be the same radiation as the radiation provided by the radiation generator, because the radiation can be reflected or transmitted. Alternatively or additionally, in an embodiment, the radiation can be different, because the radiation can be partially absorbed. In this way, the spectral intensity distribution of the radiation emitted from the object can be different from the spectral intensity distribution of the radiation provided by the radiation generator. Alternatively or additionally, in an embodiment, the radiation can be different, because the radiation can be partially absorbed and converted into luminescence. Therefore, the phrase "configured to sense radiation from the object" and similar phrases can refer to, for example, one or more of the radiation after transmission, the radiation after reflection, and the radiation after absorption.

[0039] In embodiments, the term "luminescence" may refer to phosphorescence. In embodiments, the term "luminescence" may also refer to fluorescence. The term "emission" may also be used instead of the term "luminescence."

[0040] The term "sensing device" may also refer to a plurality of the same or different sensing devices.

[0041] In particular, the sensing device may be functionally coupled to the generator. The functional coupling of the sensing device and the generator may be achieved, for example, via a control system. In an embodiment, the control system may give instructions to the generator to generate radiation or a specific type of radiation, and also instruct the sensing device to sense radiation from the object being sensed.

[0042] The sensing device and the generator can be integrated into a single device. In such embodiments, the sensing device and the generator can be configured in a single housing and / or can be mechanically coupled. In embodiments, multiple sensing devices and generators are integrated into a single device.

[0043] In other embodiments, the sensing device and the generator are not necessarily physically coupled, or are physically coupled but movable relative to each other. Thus, in certain embodiments, the radiation generator and the sensing device may be configured to be movable relative to each other. In this way, for example, the angle of incidence of the radiation on the object can be controlled. Furthermore, this can be used to control whether it is sensed in reflection mode and / or transmission mode (see also below). The phrase "movable relative to each other" may refer to an embodiment in which one of them may be movable relative to the other, or an embodiment in which the other is movable relative to this one, and in certain embodiments may also refer to an embodiment in which each is movable relative to the other. Here, "movable" may in particular refer to translationally movable or rotationally movable, in particular rotationally movable. In this way, the sensing device and the generator may be configured in different positions relative to each other.

[0044] In an embodiment, the sensing device and the radiation generator can be configured to measure in reflection mode. Thus, in an embodiment, the optical axis of the sensing device and the optical axis of the radiation generator can be arranged at an angle and can propagate in the same direction. This can be used, for example, for the measurement of reflection and / or luminescence. This angle can be small (or even zero); however, it can also be selected from a range of basic angles equal to 0° and less than 180°. For example, when a semi-transparent mirror or other optical device is used, in an embodiment, this angle can be approximately 0°.

[0045] In an embodiment, the sensing device and the radiation generator may be configured to measure in transmission mode. Thus, in an embodiment, the optical axis of the sensing device and the optical axis of the radiation generator may be configured at a small (or even zero) angle and may propagate in opposite directions.

[0046] In an embodiment, a radiation generator comprises a first plurality of pixels, wherein the radiation generator is configured to generate pulses of radiation in the first plurality of pixels in one or more time-of-flight sensing operating modes, wherein the sensing device is configured to sense the wavelength-dependent spectral intensity of the radiation received by the sensing device in one or more time-of-flight sensing modes via a second plurality of pixels as a function of time to provide a sensing system signal. The first plurality of pixels and the second plurality of pixels can be arranged in a checkered pattern in the same plane. Such an embodiment can be advantageous in ensuring that the generated radiation that is received back and sensed has been coupled into an object (e.g., a plant), propagated through the object, and exited the object at a different location. For example, this can allow properties to be analyzed corresponding to an internal portion of the object. Alternatively, the checkered pattern can be an alternating pattern in the same row.

[0047] In essentially all types of configurations, luminescence, if any, can be sensed.

[0048] The phrase "radiation from the subject" and similar phrases may refer to radiation originating downstream from the subject. In particular, it may refer to radiation emitted from the subject when the subject is irradiated with radiation. Radiation emitted from the subject may be transmitted radiation and / or reflected radiation and / or luminescence. Thus, the phrase "radiation from the subject" and similar phrases may, in embodiments, refer to radiation reflected from the subject or radiation transmitted by the subject.

[0049] Note that in embodiments, the sensing device and the radiation generator can be configured to sense two or more of reflection, transmission, and luminescence. Thus, in certain embodiments, the sensing device and the radiation generator can be configured to measure in both transmission and reflection modes. Furthermore, the radiation generator and the sensing device can be configured so that scattering can be sensed (by the sensing device).

[0050] The terms "upstream" and "downstream" relate to the arrangement of items or features relative to the propagation of radiation from a radiation generating member (here in particular a radiation generator), wherein, relative to a first position in the radiation beam from the radiation generating member, a second position in the radiation beam that is closer to the radiation generating member is "upstream" and a third position in the radiation beam that is further away from the radiation generating member is "downstream".

[0051] As described above, the sensing system may further include a control system functionally coupled to the radiation generator and the sensing device. Specifically, the control system may be configured to control the timing of providing and sensing radiation. In view of a time-of-flight sensing mode, the control system may be configured to control the generation of one or more radiation pulses and measure radiation from the object. Specifically, the radiation measurement phase may be time-dependent. Thus, in embodiments, after a pulse of radiation is generated by the radiation generator, the intensity at one or more wavelengths may be measured over time.

[0052] Thus, as described above, a sensing system may include a radiation generator, a sensing device, and a control system functionally coupled to the radiation generator and the sensing device, wherein the sensing system has one or more time-of-flight sensing operating modes.

[0053] The Time of Flight (ToF) principle can be a method for determining the distance between a sensor and an object based on the time difference between the emission of a signal and its return to the sensor after reflection from the object. In this context, the ToF principle can also alternatively or additionally be used to determine the 3D conformation of a plant, the type of plant, different parts of a plant, the stage of growth, the presence of disease, the health of a plant, etc.

[0054] The term "control" and similar terms particularly refer to at least determining the behavior of an element or supervising the operation of an element. Therefore, "control" and similar terms herein may, for example, refer to imposing an action on an element (determining the action or supervising the operation of an element), such as measuring, displaying, actuating, opening, moving, changing the temperature, etc. In addition, the term "control" and similar terms may additionally include monitoring. Therefore, the term "control" and similar terms may include imposing an action on an element, as well as imposing an action on an element and monitoring the element. Control of the element may be accomplished using a control system, which may also be referred to as a "controller." Therefore, the control system and the element may be functionally coupled, at least temporarily or permanently. The element may include a control system. In embodiments, the control system and the element may not be physically coupled. Control may be accomplished via wired and / or wireless control. The term "control system" may also refer to multiple different control systems, which may be functionally coupled, and wherein, for example, one control system may be a master control system and one or more other control systems may be slave control systems. The control system may include or may be functionally coupled to a user interface.

[0055] The control system can also be configured to receive and execute commands from a remote control. In an embodiment, the control system can be controlled via an app on a device, such as a portable device (e.g., a smartphone or iPhone, tablet, etc.). Thus, the device is not necessarily coupled to the lighting system, but can be (temporarily) functionally coupled to the lighting system.

[0056] Therefore, in an embodiment, the control system can (also) be configured to be controlled by an app on a remote device. In such an embodiment, the control system of the lighting system can be controlled from a control system or in slave mode. For example, the lighting system can be identified by a code, in particular a unique code for the respective lighting system. The control system of the lighting system can be configured to be controlled by an external control system that accesses the lighting system based on knowledge of the (unique) code (via a user interface input of an optical sensor (e.g., a QR code reader)). The lighting system may also include means for communicating with other systems or devices, such as based on Bluetooth, WiFi, LiFi, ZigBee, BLE, WiMAX, or other wireless technologies.

[0057] A system, device, or apparatus may perform an action in a "mode," "operating mode," or "operating mode." Similarly, in a method, an action, phase, or step may be performed in a "mode," "operating mode," "operating mode," or "operable mode." The term "mode" may also refer to a "control mode." This does not exclude that the system, device, or apparatus may also be adapted to provide another control mode, or multiple other control modes. Similarly, this does not exclude that one or more other modes may be performed before and / or after the execution of the mode.

[0058] However, in embodiments, a control system may be available that is adapted to provide at least a control mode. If other modes are available, the selection of such a mode may in particular be performed via a user interface, although other options, such as executing a mode based on sensor signals or a (time) scheme, may also be possible. In embodiments, an operating mode may also refer to a system, or an apparatus, or a device that can only operate in a single operating mode (i.e., "on," without further tunability).

[0059] Therefore, in an embodiment, the control system may be controlled based on one or more of an input signal from a user interface, a sensor signal (of a sensor), and a timer. The term "timer" may refer to a clock and / or a predetermined time scheme.

[0060] As described above, in certain embodiments, the generator is configured to generate pulses of radiation in one or more time-of-flight sensing operating modes, and the sensing device can be configured to sense radiation from the object being sensed as a function of time in one or more time-of-flight sensing modes to provide a sensing system signal.

[0061] The sensing system signal may indicate, inter alia, the intensity of the sensing signal as a function of time (after providing a pulse of radiation). The sensing system signal may also indicate the polarization of the sensed radiation and / or the used radiation. The sensing system signal may also indicate the intensity of the used radiation. Furthermore, the sensed signal may indicate the angular distribution of the sensed radiation.

[0062] In addition, the sensed signal may indicate a spectral intensity distribution of the sensed radiation. In such an embodiment, radiation (from the object) may be sensed at two or more different wavelengths. Therefore, the sensing system signal may also indicate a distribution of spectral intensities or a ratio between spectral intensities. The term "spectral intensity distribution" may refer in an embodiment to an n*m array, where n≥2, and where in an embodiment m=a*n, where a≥1. A row may indicate a wavelength, and one or more rows may indicate an intensity. The intensity may be based on the intensity in photon counts or energy counts (spectral power distribution). A row may include intensity values. Alternatively or additionally, a row may include processed intensity values, such as, for example, a ratio of values, or a derivative, etc.

[0063] As mentioned above, the term "spectral intensity distribution" may refer to spectral power distribution in embodiments.

[0064] For determining the spectral intensity distribution, the sensing means may comprise a wavelength dependent sensor, such as a CCD camera.Thus, in an embodiment, the sensing means may comprise a multi-pixel sensor, optionally comprising a color filter for wavelength dependent detection.

[0065] Alternatively or additionally, the sensing device may include a wavelength-independent sensor in combination with an optical device, such as a wavelength-dependent spectral filter. Such a sensing device may also be an array camera with different spectral filters upstream of the sensing elements of the array camera, where two or more sensing elements are (relatively more) selective for different wavelengths.

[0066] Thus, in particular embodiments, the sensing device can be configured to sense the (wavelength-dependent) spectral intensity of radiation received by the sensing device as a function of time in one or more time-of-flight sensing modes - such as a wavelength-dependent spectral intensity distribution - to provide a sensing system signal.

[0067] The term "sensing system signal" refers to a signal of a sensing system, in particular a signal generated by the sensing system when sensing radiation from an object.

[0068] Some further specific embodiments are discussed herein below.

[0069] As described above, different pulses can be provided in time, wherein the optical properties of the pulses can differ. This can facilitate one or more of 3D mapping of an object, determining the stage of growth, determining the presence of disease, determining the health of a plant, and the like. Alternatively or additionally, the angle of incidence of the radiation can be varied between pulses. This can be accomplished using variable optics and / or a sensing device and radiation generator that are movable relative to one another. In embodiments, the variable optics can include one or more movable mirrors or controllable lenses. Thus, in embodiments, the generator can be configured to generate multiple pulses of radiation in one or more of one or more time-of-flight sensing operating modes, wherein the two or more pulses of radiation differ in one or more of (a) optical properties and (b) angle of incidence of the radiation. Also as described above, in certain embodiments, the two or more pulses of radiation can differ in one or more of (i) polarization and (ii) spectral intensity distribution (of the radiation). For example, two or more pulses, such as three or more (e.g., five) or, for example, ten or more pulses, can be provided, whose peak wavelengths differ by at least 5 nm, and even more particularly, by at least 10 nm.

[0070] In a particular embodiment, the radiation generator may comprise two or more lasers configured to respectively generate radiation having different spectral intensity distributions. Furthermore, in particular in an embodiment, the generator may be configured to generate multiple (continuous) pulses of radiation with two or more lasers in one or more of the one or more time-of-flight sensing operating modes. As described above, this may in particular be combined with a sensing device that may be configured to sense different wavelengths generated by the radiation generator. For example, during corresponding consecutive pulses, pulses having different spectral intensity distributions (such as having different peak wavelengths) may be generated respectively. Alternatively or additionally, pulses having different polarizations may be generated respectively (during corresponding consecutive pulses).

[0071] In certain embodiments, LIDAR can be used. LIDAR can provide a surveying method for measuring the distance to a target by illuminating the target with laser light and measuring the reflected light with a sensor. In certain embodiments, a laser-based time-of-flight camera can be used. In such embodiments, the entire scene or spectral intensity distribution can be captured with each laser pulse.

[0072] For example, according to Osram Opto Semiconductors application note AN106, two main systems can be used to obtain 3D point clouds for LIDAR: 3D flash and scanning LIDAR. With 3D flash LIDAR, a pulsed laser beam is emitted all at once over the entire solid angle of interest. To achieve a specific resolution in the point cloud, an n×m array of photosensitive detectors (such as a photodiode array or a CMOS ToF chip) is required. Scanning LIDAR systems can consist of a focused pulsed laser beam directed into a small solid angle using a mechanically rotating mirror or a MEMS (microelectromechanical system) mirror. Because the high-power pulsed laser beam is controlled to emit only into a small solid angle, the reach of the optical power used can be much greater than with 3D flash systems. The mirrors commonly used (mechanically rotating or MEMS) only allow scanning in one direction. The resolution in this direction is achieved by the pulse repetition frequency of the laser and the scanning frequency of the mirror. To achieve a certain resolution in the other direction, an n×1 array of fast PiN photodiodes or avalanche photodiodes can be used.

[0073] In certain embodiments, the radiation generator may include a multispectral illumination source combined with a processing component. The multispectral illumination source generates light having multiple wavelengths that illuminate the object. In one embodiment, the multispectral illumination source includes a plurality of LEDs, wherein a first LED in the plurality of LEDs is capable of emitting light having a first wavelength, and a second LED in the plurality of LEDs is capable of emitting light having a second wavelength, wherein the first and second wavelengths are different. In another embodiment, the multispectral illumination source is a single-element multicolor LED. Additionally, the generated light may be in the form of light pulses, modulated light, or both. The processing component measures the time it takes for light to travel from the multispectral illumination source to the object and from the object to the processing component. It also has the capability to calculate range information to locate the object and to calculate parameters derived from the light received from the object. In this embodiment, the system may include at least one image sensor to sample the radiation received from the object, and electronic circuitry to control the operation of the multispectral illumination source. In embodiments of the present invention, the apparatus may further include a spectral bandpass filter in the optical path of the processing component to suppress wavelengths not emitted by the multispectral illumination source. This can reduce ambient light levels and result in a better signal-to-noise ratio (SNR) and operability. In an embodiment of the present invention, the apparatus further comprises at least one image sensor for sensing light received from the object, and electronic circuitry for controlling the operation of the multispectral illumination source. In an embodiment of the present invention, the apparatus may further comprise a plurality of spectral bandpass filters positioned proximate to the at least one image sensor to filter the light received from the object on a pixel-by-pixel basis. Additionally, the multispectral illumination source may generate light having illumination spectra tuned to the plurality of spectral bandpass filters. This may increase the system's ability to filter light, thereby improving the system's energy efficiency and robustness. In an embodiment of the present invention, the multispectral illumination source may generate light comprising a subset of wavelengths, wherein each wavelength in the subset has a different modulation frequency, and the processing component may be configured to filter the wavelengths on a pixel-by-pixel basis based on the different modulation frequencies of the plurality of wavelengths.

[0074] Furthermore, it appears that specific wavelengths may be particularly useful because they appear to provide a greater signal-to-noise ratio. In particular, one or more of the following wavelength ranges may be useful: 200-300 nm, 680-720 nm, 920-960 nm, 1080-1120 nm, 1340-1420 nm, and 1850-1890 nm. Therefore, in certain embodiments, the radiation generator is configured to generate radiation in one or more of the one or more time-of-flight sensing operating modes, the radiation having a wavelength selected from the wavelength ranges of 200-300 nm, 680-720 nm, 920-960 nm, 1080-1120 nm, 1340-1420 nm, and 1850-1890 nm. In certain embodiments, the radiation generator can be configured to generate radiation having one or more wavelengths within the wavelength range of 200-300 nm. In such embodiments, for example, a UV LED or UV laser, such as a UV laser diode, can be used. Such a light source can have a relatively fast decay time, which is useful for ToF applications. Alternatively or additionally, the radiation generator can be configured to generate radiation having one or more wavelengths within at least two of the aforementioned wavelength ranges. Radiation having one or more wavelengths within these wavelength ranges can be provided simultaneously or sequentially (see also above). In particular, the radiation generator can be configured to generate one or more wavelengths within at least one or more of the following wavelength ranges of green and NIR. Alternatively, the radiation generator can be configured to generate one or more wavelengths within at least one or more of the following wavelength ranges of blue, green, and red.

[0075] In the context of horticultural lighting, near-UV is defined as one or more wavelengths selected from the 300-400 nm spectral range, blue is defined as one or more wavelengths selected from the 400-500 nm spectral range, white is defined as wavelengths selected from the 400-700 nm spectral range (these selected wavelengths together can constitute white light, such as a combination of blue, green, and red wavelengths), green is defined as one or more wavelengths selected from the 500-600 nm spectral range, red is defined as one or more wavelengths selected from the 600-700 nm spectral range, deep red is defined as one or more wavelengths selected from the 640-700 nm spectral range, and far red is defined as one or more wavelengths selected from the 700-800 nm spectral range. Thus, deep red is a secondary choice to red. NIR is defined as one or more wavelengths selected from the 800-2500 nm spectral range, such as in particular from the 800-2000 nm range, such as, for example, up to about 1200 nm. For example, green may be more easily reflected, and NIR may more easily penetrate plant objects (parts). Furthermore, blue and / or red can be absorbed, for example, by chlorophyll (chlorophyl).

[0076] To facilitate sensing, the sensing system can allow the use of user input information. For example, the user can indicate one or more of the type of plant, different plant parts, stage of growth, presence of disease, health of the plant (and optional type of 3D conformation), etc. On this basis, the sensing system can optimize one or more optical properties and angles of incidence. For example, the transparency of the leaves can be considered, the color of the fruit or flower can be considered, the 3D conformation can be considered, etc. This can be done before the (final) time-of-flight sensing mode. User input information can be provided via a user interface. Examples of user interface devices include, among others, manual buttons, displays, touch screens, keyboards, voice-activated input devices, audio outputs, indicators (e.g., lights), switches, knobs, modems, and network cards. In particular, the user interface device can be configured to allow the user to instruct a device or apparatus or system, the user interface being functionally coupled to the device or apparatus or system, or the user interface being functionally included by the device or apparatus or system. The user interface can particularly include manual buttons, touch screens, keyboards, voice-activated input devices, switches, knobs, etc., and / or optional modems, and network cards, etc. The user interface can include a graphical user interface. The term "user interface" can also refer to a remote user interface, such as a remote control. The remote control can be a standalone dedicated device. However, the remote control can also be a device with an app configured to (at least) control a system or device or apparatus. The user interface is in particular functionally coupled to the control system or can be included in the control system.

[0077] Alternatively or additionally, the sensing system can include another sensor, such as a (simple) camera, which can be used for (preliminary) detection of the 3D conformation of the plant, the type of plant, different plant parts, the stage of growth, the presence of disease, the health of the plant, etc. Based on this, the sensing system can optimize one or more optical properties and angles of incidence. For example, the transparency of the leaves can be taken into account, the color of the fruit or flower can be taken into account, the 3D conformation can be taken into account, etc. This can be done during the initial operating mode. The additional sensor can be, for example, a tag reader.

[0078] The sensing system can sense in an iterative manner. For example, the control system can evaluate one or more of the 3D conformation of the plant, the type of plant, the different parts of the plant, the stage of growth, the presence of disease, the health of the plant, etc. based on the sensed radiation. Based on this, when the evaluation is correct, the optical properties, such as one or more of the wavelength, intensity and polarization (but alternatively or additionally also the angle of incidence (see also above)) can be adjusted based on this evaluation and the expected sensed radiation. In this way, via an iterative process, the optical properties and / or the angle of incidence can be optimized to obtain a sensing system signal that can be reliable and / or can have a low signal-to-noise ratio. For example, the transparency of the leaves can be taken into account, the color of the fruit or flower can be taken into account, the 3D conformation can be taken into account, etc. This can be done during an initial operating mode, which can also be indicated as a "preliminary time-of-flight sensing operating mode".

[0079] Thus, in an embodiment, the control system may use a feedforward loop and / or a feedback loop.Furthermore, in an embodiment, multiple optimization cycles may be applied, such as optimizing, for example, S / R ratio, sensitivity, etc.

[0080] Thus, in particular embodiments, the sensing system may include one or more controllable sensing parameters, wherein the sensing system has an initial operating mode, wherein the values ​​of the one or more controllable sensing parameters are defined depending on one or more of: (i) user input information, (ii) a sensor signal of a sensor (such as a (simple) (digital) camera), and (iii) radiation received in a preliminary time-of-flight sensing operating mode (of the sensing system), and wherein the sensing system is configured to perform one or more of the one or more time-of-flight sensing operating modes using the defined sensing parameters after performing the initial (or preliminary) operating mode.

[0081] As can also be derived from the above, in particular the controllable sensing parameters can be selected from the group consisting of: (i) polarization of the radiation, (ii) spectral intensity distribution of the radiation, (iii) angle of incidence of the radiation, (iv) pulse modulation and / or pulse frequency, and (v) polarization filter upstream of the detector of the sensing device.

[0082] As described above, in embodiments, the polarization of the radiation provided by the radiation generator may vary over time and / or vary over spatial position on the emission window of the radiation generator. Alternatively or additionally, the polarization of the radiation sensed by the sensing device may vary over time and / or vary over spatial position on the entrance window of the sensing device.

[0083] For example, during interactions within plant tissue, the propagation direction of the radiation (and subsequently emitted radiation) may change, becoming multidirectional, etc. Polarization may also change. The present invention also allows for the residence time within the plant tissue to be considered and to provide information. Similarly, the present invention also allows for the (biological) components within the plant (tissue) to be considered. This information can also be derived from the sensed radiation.

[0084] In a particular embodiment, the control system can be configured to determine at least two different types of radiation from an initial operating mode, wherein a first type of radiation has a greater penetration depth in the sensed (plant) object than a second type of radiation, and to perform one or more of the one or more time-of-flight sensing operating modes using the at least two different types of radiation. Alternatively or additionally, the control system can be configured to determine at least two different types of radiation from an initial operating mode, wherein the (plant) object has a greater absorption for the first type of radiation than for the second type of radiation, and to perform one or more of the one or more time-of-flight sensing operating modes using the at least two different types of radiation. Alternatively or additionally, the control system can be configured to determine at least two different types of radiation from an initial operating mode, wherein the (plant) object provides a stronger luminescence under the first type of radiation than under the second type of radiation, and to perform one or more of the one or more time-of-flight sensing operating modes using the at least two different types of radiation. In a particular embodiment, the control system can be configured to determine at least two different types of radiation from an initial operating mode, wherein the first type of radiation has a greater penetration depth in a first portion of the sensed (plant) object than the second type of radiation, and to perform one or more of the one or more time-of-flight sensing operating modes using the at least two different types of radiation. Alternatively or additionally, the control system can be configured to determine at least two different types of radiation from the initial operating mode, wherein the first portion of the (plant) object has a greater absorption of the first type of radiation than the second type of radiation, and to perform one or more of the one or more time-of-flight sensing operating modes using the at least two different types of radiation. Alternatively or additionally, the control system can be configured to determine at least two different types of radiation from the initial operating mode, wherein the first portion of the (plant) object provides a stronger luminescence under the first type of radiation than the second type of radiation, and to perform one or more of the one or more time-of-flight sensing operating modes using the at least two different types of radiation. The terms "first type of radiation" and "second type of radiation" may, in embodiments, specifically refer to radiation having different spectral power distributions, such as different colors. Alternatively, these terms may refer to radiation having (substantially) the same spectral power distribution, but having different polarizations. In yet other embodiments, the terms "first type of radiation" and "second type of radiation" may, in embodiments, specifically refer to radiation having different spectral power distributions, such as different colors, which may optionally also have different polarizations. Note that, by definition, lasers having different wavelengths have different spectral power distributions.

[0085] In certain embodiments, the control system can be configured to determine at least two different types of radiation from an initial operating mode, wherein (a) for a first type of radiation, one or more of the following applies: (i) having a greater penetration depth in a first portion of the sensed (plant) object than a second type of radiation, (ii) being more strongly absorbed in the first portion of the sensed (plant) object than the second type of radiation, and (iii) generating more luminescence from the first portion of the sensed (plant) object than the second type of radiation, and (b) for a second type of radiation, one or more of the following applies: (i) having a greater penetration depth in a second portion of the sensed (plant) object than the first type of radiation, (ii) being more strongly absorbed in the second portion of the sensed (plant) object than the first type of radiation, and (iii) generating more luminescence from the second portion of the sensed (plant) object than the first type of radiation, wherein the first portion of the plant and the second portion of the plant are different; and performing one or more of the one or more time-of-flight sensing operating modes with the at least two different types of radiation. The first plant part and the second plant part can, for example, be selected from the group consisting of a flower, a part of a flower, a leaf, a stem, a branch, an aerial root, and the like. The parts of a flower may for example be selected from the group consisting of petals, stigma, anthers, style, ovules, filaments and sepals (and optionally peduncle).

[0086] In certain embodiments, it may be desirable to sense a deviation from the irradiation position. This may be useful, for example, for measuring propagation length, differences in transmission at different wavelengths, and the like. Thus, in an embodiment, the radiation has a beam cross-section (A1), wherein the sensing device has a field of view cross-section (A2) (in one or more of one or more time-of-flight sensing operating modes), wherein the radiation generator and the sensing device have a predetermined configuration, wherein the beam cross-section (A1) and the field of view cross-section (A2) do not overlap within a predetermined distance (d2) from an entrance window of the sensing device, wherein the predetermined distance (d2) is selected from the range of 0-500 cm. In yet other configurations, wherein, for example, the sensing device and the radiation generator are not contained in a single device, but may be configured to be movable relative to each other, in embodiments the predetermined distance may be greater. Thus, in alternative examples or applications, for example, a predetermined distance selected from the range between 500 cm and 1500 cm may be envisioned. For indoor applications, such as gardening plants, a predetermined distance (d2) selected from the range of 0-500 cm may be sufficient, although other embodiments may also be possible. For example, for outdoor applications, the predetermined distance (d2) may be selected from the range of 0-500 cm, but may be selected at a greater distance in alternative embodiments.

[0087] Of course, when the non-overlap condition is not relevant, the predetermined distance can be longer or shorter. Furthermore, in certain embodiments, the radiation can be sensed via one or more common optical devices. For example, a dichroic mirror can be used in the radiation path of the radiation generator, and the same dichroic mirror can be used in the light path to the sensing device.

[0088] The sensing system signal can be useful as such, for example, for obtaining a better status overview of (multiple) objects. However, the sensing system signal can also be used to control (another) device (or apparatus or system). Therefore, as previously described, the sensing system is functionally coupled to the agricultural equipment, wherein the agricultural equipment is controllable based on the sensing system signal. The functional coupling of the sensing system and the agricultural equipment can be wireless and / or via a wired connection. In particular, the functional coupling of the sensing system and the agricultural equipment can include the control system of the sensing system controlling the agricultural equipment and / or the (another) control system controlling the agricultural equipment based on the sensing system signal. The agricultural equipment can include one or more devices selected from the group consisting of nutrient supply equipment, irrigation equipment, pruning equipment, weeding equipment, harvesting equipment, temperature control equipment, humidity control equipment, horticultural radiation control equipment, ripening equipment, etc. The ripening equipment can be configured to provide one or more of pesticides, herbicides, fungicides, etc. Therefore, the sensing system signal can be used to control an actuator of another device (such as the agricultural equipment) based on this sensing system signal.

[0089] The conditions to which the (growing) plants are subjected are typically defined in a recipe. Thus, the control system can grow the plants according to the recipe. Such a recipe may include a light recipe that defines a predetermined horticultural light intensity. This may mean that the recipe defines a predetermined horticultural light intensity that varies over time. Alternatively or additionally, the recipe may define the predetermined horticultural light intensity as a function of a sensed parameter, such as nutrient intake, leaf size, plant temperature, leaf temperature, root temperature, stem length, fruit size, etc. Other parameters may also be sensed, such as one or more of temperature (in a greenhouse, farm, climate cell, tunnel, etc.), humidity, gas composition, etc. Daylight intensity (sunlight may also be used) may also be a parameter to be sensed. A recipe for a lighting parameter may be indicated as a "light recipe." A light recipe may be comprised of a recipe that also includes one or more of other parameters, such as leaf temperature, root temperature, ambient temperature, etc.

[0090] Herein, the term "horticultural light" is used for artificial light used to illuminate plant subjects during at least a portion of a day. The spectral power distribution of the horticultural light may vary during a day and / or throughout a growing cycle.

[0091] In yet a further aspect, the present invention also provides an agricultural facility comprising a sensing system as defined herein.The agricultural facility may comprise a control system which is the control system of the sensing system or which may be functionally coupled to the control system of the sensing system.

[0092] In an embodiment, the agricultural facility may in particular be selected from the group of a greenhouse and a horticultural arrangement. In an embodiment, the horticultural arrangement may include vertical farm plants. Furthermore, in an embodiment, the agricultural facility may in particular be selected from the group of an open space, a courtyard (such as a vineyard), a plantation, etc. Instead of the term "horticultural arrangement", the term "horticultural factory" is used; in such an embodiment, the term "horticultural factory" refers to a building or other construction (not to a living plant). In particular, the term "horticultural arrangement" refers to a plant factory or a climate unit, in which plants are grown under controlled conditions and in which the plants do not substantially receive natural daylight. Furthermore, such a plant factory can be climatized, such as in the case of a climate unit. Therefore, in an embodiment, the horticultural arrangement comprises such a plant factory or climate unit. In this document, the term plant factory is considered to cover embodiments of a climate unit.

[0093] Typically, in a plant farm, plants are grown in climate units. Each unit is equipped with one or more racks. Each rack has multiple levels for growing plants.

[0094] In a specific embodiment (also as described above), the present invention provides an agricultural facility comprising a sensing system as defined herein, wherein one or more of the radiation generator and the sensing device are configured to be movable, and wherein the control system can in particular be configured to control one or more of (i) the position of the radiation generator and (ii) the position of the sensing device. For example, in an embodiment, one or both of the radiation generator and the sensing device can be translatable along a track. In a specific embodiment, one or more of the radiation generator and the sensing device can be comprised by (different) (plural) drones. Thus, in further specific embodiments, both the radiation generator and the sensing device can be configured to be movable relative to each other (such as via a track and / or a drone).

[0095] As can be derived from the above, in certain embodiments, the control system (of the sensing system or the agricultural facility) can be configured to perform an action based on the sensing system signal. In specific embodiments, the action can be selected from the group consisting of controlling the growing conditions of the plant, controlling the irradiation of the plant or plant part (with one or more of, for example, UV, visible light, and IR radiation), controlling the harvesting of the plant or plant part, controlling the treatment of the plant, and controlling the pruning of the plant. Furthermore, in specific embodiments, the agricultural facility can be selected from the group consisting of a horticultural arrangement, a greenhouse, and an open space. Here, controlling the irradiation refers to irradiating the plant for the purpose of controlling growth, etc.

[0096] In use, the agricultural facility may include a plant support with plants, or a plant support with seeds, or a plant support with seedlings, etc. Thus, in use, the agricultural facility may include a plant support with plants, or a plant support with seeds, or a plant support with seedlings, etc. The term "support" or "plant support" may refer to one or more of a (granular) substrate, an aqueous substrate (in hydroponics), soil, wire (for wire crops), etc., which may be used to grow plants in, on, or along them, etc.

[0097] The control system of such an agricultural facility may control one or more of temperature, humidity, irrigation, nutrient supply, light intensity of horticultural lights, air conditions (including one or more of air temperature, air composition, air flow, etc.) Such a horticultural system may be configured to control one or more of these conditions at different locations in the arrangement.

[0098] In yet another aspect, the present invention provides a monitoring (and optionally controlling) method, in particular a plant monitoring (and optionally controlling) method, comprising performing a time-of-flight sensing operating mode, wherein the method comprises subjecting one or more plants to pulses of radiation, sensing the radiation received by a sensing device as a function of time in the time-of-flight sensing operating mode, and generating a sensing system signal. In particular, the method may comprise performing the time-of-flight sensing operating mode with a sensing system defined herein. In particular embodiments, during the time-of-flight sensing mode, different types of radiation may be provided, which may differ in one or more of (peak) wavelength, spectral intensity distribution, and polarization (see also above).

[0099] In yet another embodiment, the plant monitoring method may further comprise performing an action based on the sensing system signal. In a specific embodiment, the action is selected from the group consisting of controlling growing conditions, controlling harvesting, controlling treatment, and controlling pruning.

[0100] Instead of the phrase "plant monitor (and optional control) method" and similar phrases, the phrase "plant monitoring (and optional control) method" or "plant monitoring (and optional controlling) method", and similar phrases may also be applied.

[0101] In yet another aspect, the present invention provides for the use of time-of-flight spectroscopy in plant monitoring (and optionally control) methods. Furthermore, in one aspect, the present invention provides for the use of time-of-flight spectroscopy in horticultural applications, such as for sensing one or more of the 3D conformation of a plant, the type of plant, different plant parts, stage of growth, presence of disease, health of the plant, depth and tissue information, and the like.

[0102] Thus, in embodiments, time-of-flight spectroscopy may be used in monitoring (and optionally controlling) methods, in particular plant monitoring (and optionally controlling) methods. In embodiments, time-of-flight spectroscopy may be used for sensing and / or monitoring agricultural areas or other types of substantially natural outdoor areas, such as woods, (rain) forests, grasslands, heaths, prairies, savannas, parks, etc. In embodiments, time-of-flight spectroscopy may be used for sensing and / or monitoring agricultural areas or other types of substantially natural outdoor areas, such as rivers, river deltas, lakes, oceans, parks, etc.

[0103] In embodiments, the sensing systems described herein may be used for monitoring (and optionally controlling), in particular plant monitoring (and optionally controlling). In embodiments, the sensing systems described herein may be used for sensing and / or monitoring agricultural areas or other types of substantially natural outdoor areas, such as woods, (rain) forests, grasslands, heaths, prairies, savannas, parks, etc. In embodiments, the sensing systems described herein may be used for sensing and / or monitoring agricultural areas or other types of substantially natural outdoor areas, such as rivers, river deltas, lakes, oceans, parks, etc.

[0104] In yet another aspect, the present invention further provides a software product (or (other) computer program product) that, when executed on a computer, implements the plant monitoring (and optional control) method described herein. Such a computer may include or be a control system as described herein. Thus, in embodiments, such a computer may include or be functionally coupled to a sensing system or agricultural facility as described herein.

[0105] The present invention thus also provides a computer program product that enables the execution of the method as defined herein, for example when loaded on a computer (which is functionally coupled to an agricultural facility). In yet another aspect, the present invention provides a record carrier (or data carrier, such as a USB stick, CD, DVD, etc.) storing a computer program for executing the method described herein. Thus, when run on or loaded into a computer, the computer program product implements or is capable of implementing the method as described herein. Thus, in a further aspect, the present invention provides a computer program product that, when run on a computer functionally coupled to or comprised by an agricultural facility, in particular as defined herein, is capable of implementing the method as described herein.

[0106] The record carrier or computer-readable medium and / or memory may be any recordable medium (e.g., RAM, ROM, removable memory, CD-ROM, hard drive, DVD, floppy disk, or memory card) or may be a transmission medium (e.g., a network including optical fiber, the World Wide Web, cables, and / or wireless channels using, for example, time division multiple access, code division multiple access, or other wireless communication systems). Any known or developed medium that can store information suitable for a computer system may be used as a computer-readable medium and / or memory. Additional memory may also be used. The memory may be long-term memory, short-term memory, or a combination of long-term and short-term memory. The term memory may also refer to multiple memories. The memory may configure the processor / controller to implement the methods, operational actions, and functions disclosed herein. The memory may be distributed or local, and where additional processors may be provided, the processor may be distributed or single. The memory may be implemented as electrical, magnetic, or optical memory, or any combination of these or other types of storage devices. Furthermore, the term "memory" should be interpreted broadly enough to encompass any information that can be read from or written to an address in an addressable space accessible by a processor. According to this definition, information on a network (such as the Internet) remains in memory, for example, because the processor can retrieve the information from the network.

[0107] In one aspect, the present invention also provides a system comprising a sensing system as described herein.

[0108] In various aspects, the present invention provides a light generating system comprising one or more light sources (particularly for generating visible light) and a sensing system as described herein. Furthermore, in such embodiments, a control device of the sensing system can be configured to control the one or more light sources to illuminate plants with a lighting characteristic. Thus, in one embodiment, the present invention provides: a light generating system comprising one or more light sources and a sensing system, the sensing system comprising a radiation generator, a sensing device, and a control system functionally coupled to the radiation generator and the sensing device, wherein the sensing system has one or more time-of-flight sensing operating modes, wherein the generator is configured to generate pulses of radiation in the one or more time-of-flight sensing operating modes, and wherein the sensing device is configured to sense the wavelength-dependent spectral intensity of the radiation received by the sensing device as a function of time in the one or more time-of-flight sensing modes to provide a sensing system signal; wherein the control device of the sensing system can be configured to control the one or more light sources to illuminate plants with a lighting characteristic. The lighting characteristic can be, for example, a light recipe, light color, light intensity, light modulation, color temperature, light pattern, or light scene. The light generating system can be arranged in an agricultural facility. The light generating system can be a luminaire or a lamp. Thus, advantages and / or embodiments applicable to the sensing system according to the present invention may apply mutatis mutandis to the light generating system according to the present invention.

[0109] In a further embodiment, the present invention provides a light generating device comprising (a light generating system comprising) one or more light sources and a sensing system as described herein. Thus, such a system (or device) may include one or more light sources (particularly for generating visible light) and a radiation generator.

[0110] In various aspects, the sensing system according to the present invention may be used mutatis mutandis in aquaculture applications.

[0111] In various aspects, the present invention provides an agricultural robot comprising a sensing system according to the present invention. Furthermore, the agricultural robot may comprise agricultural equipment according to the present invention, such as a picking arm and / or lighting equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] Embodiments of the invention will now be described, by way of example only, with reference to the accompanying schematic drawings in which corresponding reference numerals indicate corresponding parts, and in which:

[0113] Figures 1A-1B Some embodiments of the sensing system are schematically depicted;

[0114] Figures 2A-2B Some aspects are schematically depicted;

[0115] Figures 3A-3B Some embodiments are schematically depicted; and

[0116] Figures 4A-4E Various further aspects are schematically depicted.

[0117] The schematic drawings are not necessarily to scale. DETAILED DESCRIPTION

[0118] Possible robotic solutions for inspecting and / or handling plants and fruit may require the ability to correctly detect plants / fruit. This paper proposes, among other things, an (optimized) time-of-flight solution using active illumination at wavelengths that may be optimal for horticultural applications. By applying wavelengths tailored to the reflectance / absorption / luminescence characteristics of the plant or fruit, the accuracy and robustness of range information can be improved. When two or more wavelengths are applied, the difference in reflectance at each wavelength can be used to classify plant parts or assess plant condition and health (e.g., degree of dehydration or presence of disease). This can be used to assess fruit status (e.g., ripeness, presence or concentration of certain components) to ensure fruit quality and automatically determine the optimal time for harvesting. Furthermore, wavelengths can be selected so that they penetrate, for example, leaves and are reflected by the fruit. This enables detection of fruit even when obscured by leaves. Using additional wavelengths reflected by leaves provides information on how to approach the fruit with a robotic gripper. The provided capabilities—such as leaf vision and multispectral analysis of plants / fruit—are very difficult, subjective, and potentially impossible tasks for (unskilled) workers.

[0119] Among other things, active lighting can be provided for SNR improvement of range measurements. For example, active lighting with a wavelength that is best reflected by the fruit / plant to be inspected can be used. For example, the reflectivity of tomatoes appears to be different from that of leaves in the visible light range, but also in the SWIR range (1000-1900 nm). For example, active lighting at 930 nm appears to be poorly reflected by tomatoes, resulting in a lower SNR of the range signal, whereas using, for example, a 750 nm wavelength would result in a much better SNR. Furthermore, scattering of indirect light reflections can have a negative impact on range measurements. In the case where we are only interested in range information for tomatoes, a wavelength near the chlorophyll absorption spectrum of the leaves and the relatively high reflectivity of tomatoes (such as, for example, 625 nm) can be selected. The wavelength can also be selected so that the influence (interference) of sunlight is reduced (see the spectrum in the right figure).

[0120] The absorption spectra of tomatoes and tomato leaves partially overlap, and therefore a single wavelength can be used to measure the range. However, it can also be useful to use different wavelengths, where at least one wavelength is selected from a wavelength range with little or no overlap. In cases where the fruit and flowers have low reflectivity in the overlapping portion of the leaf spectrum (e.g., the blue flowers in the example below), the effective wavelength used can be selected between 450-500 nm.

[0121] Furthermore, monochromatic / multispectral active illumination for spectral measurements with range and shape compensation is provided, among other things. In further embodiments, active illumination can be used, wherein the intensity of the active illumination can be optimized for the range / geometry and optical properties of the fruit / plant to be inspected. The reflection / absorption / transmission of the active illumination can depend on the distance to the plant, the orientation of the surface at different wavelengths, and the orientation of the surface at different wavelengths. The active illumination can use this information to adjust the intensity of the active (multispectral) illumination source to improve signal quality.

[0122] Furthermore, multispectral active lighting for seeing through leaves can be applied, among other things. When observing fruit and leaves, the difference in their spectral response can be exploited to see through the leaves. For example, wavelengths that (partially) penetrate the vegetation and reflect off the fruit can be used when measuring distance. By having the leaves reflect additional wavelengths, automated solutions can use the measured 3D properties of the fruit and leaves to automatically harvest the fruit.

[0123] In embodiments, multispectral active lighting can be applied for plant growth and health monitoring. For example, when using two or more specific wavelengths, the growth or health of a plant can be determined by taking the ratio of the spectral responses. For example, the ripeness of a flower or fruit can be determined by monitoring the reflectance across a first spectral range as a reference, while also using the reflectance at a subset of wavelengths within this first range (or even outside of this first range). Similarly, disease can be detected from comparing multiple responses.

[0124] Furthermore, in embodiments, multispectral active illumination for penetration measurement can be applied. Light penetration in plants may depend on the wavelength used. Long-wavelength red or near-infrared (NIR) light can penetrate deeper into leaves than blue light, likely due to higher scattering at cell membranes and cell-to-cell interfaces. The measured distances of different spectral components can be used to determine the penetration depth of different wavelengths.

[0125] In embodiments, multi-spectral active illumination for fluorescence measurement can be applied. For example, fluorescence reabsorption within a leaf can depend on the emission wavelength. Due to the characteristics of the chlorophyll absorption spectrum, red fluorescence can be more likely to be (at least partially) reabsorbed by the chlorophyll within the leaf than far-red fluorescence. Similarly, when broadband radiation of red light is provided, this can have a greater probability of being (at least partially) reabsorbed by the chlorophyll within the leaf than broadband radiation of far-red light. Active illumination can be used for both range sensing and fluorescence measurement.

[0126] Furthermore, in embodiments, multispectral active illumination for scattering and BRDF measurements can be provided. For example, 3D information can be used to reconstruct the surface and, thereby, the incident angle of the effective light, to measure the properties of the reflected light. Light scattering or the bidirectional reflectance distribution function (BRDF) can be used to monitor plant development / health. The scattering effect may depend on the wavelength used.

[0127] In an embodiment, multi-spectral active illumination for polarization measurement can be applied. For example, different polarizations (see Figure 3B ) and time-of-flight sensors that use elements of polarized and unpolarized radiation. Plants' effects on polarization can be used to monitor plant development (e.g., the presence of cannabis flowers or their maturity level, based on, for example, the detection and monitoring of the amount and size of trichomes or terpenes within them). Another option is to use (un)polarized light sources with sensing elements having different polarizations. However, combinations of polarized radiation and sensing devices that can measure different polarizations are also possible.

[0128] In yet another embodiment, multispectral active illumination for internal propagation / scattering measurements can be applied. For example, a time-of-flight signal can be used to measure the propagation of light within a plant. By partially illuminating the plant with active illumination, unilluminated areas can be used as measurement locations. One can measure the time it takes for light to enter the plant, propagate through the plant (part), and then be received by the sensor (see Figure 4C (Variant II)). The direct return reflection is measured by the sensing element observing the area illuminated by the active lighting, while the indirect reflection (which includes the propagation through the vegetation) is measured via the sensing element observing the non-illuminated part (see Figure 4C (Variant II)). The propagation of light through the plant also depends on the wavelength, thus providing additional information. The propagation time also depends on scattering effects within the plant, thus providing information, for example, on the type and amount of scattering locations (e.g., cell type, size, and number, osmotic pressure in the cells, hydration level of the tissue, etc.).

[0129] With the help of monochromatic and / or polychromatic active illumination range information can be extracted, which can be used to adjust / calibrate the spectral information.With the help of monochromatic and / or polychromatic active illumination shape (3D conformation) information can be extracted, which can be used to adjust / calibrate the spectral information.

[0130] Figures 1A-1B Some embodiments of the sensing system 1000 are schematically depicted, such as here for agricultural applications. Reference 1 refers to very schematically depicted vegetative objects such as plants, trees, bushes, etc.

[0131] The sensing system includes a radiation generator 100, a sensing device 200 (functionally coupled to the generator 100), and a control system 300 functionally coupled to the radiation generator 100 and the sensing device 200. Two or more of the radiation generator 100, the sensing device 200, and the control system 300 may be integrated into a single device. However, they may also be included in two or more different devices: see Examples I and III, which show three separate devices; and Example II, in which the radiation generator 100 and the sensing device 200 are included in a single device.

[0132] In embodiments, the radiation generator 100 can be configured to generate multiple pulses of radiation 111 in one or more of one or more time-of-flight sensing operating modes, wherein in particular embodiments, two or more pulses of radiation 111 differ in one or more of (a) optical properties and (b) angle of incidence of the radiation 111 (see further below).

[0133] The sensing system 1000 has one or more time-of-flight sensing operating modes, wherein the generator 100 is configured to generate pulses of radiation 111 in the one or more time-of-flight sensing operating modes, and wherein the sensing device 200 is configured to sense the (wavelength-dependent) spectral intensity of the radiation received by the sensing device 200 as a function of time in the one or more time-of-flight sensing modes - such as a wavelength-dependent spectral intensity distribution - to provide a sensing system signal.

[0134] Figure 1B Schematically, embodiments are depicted in which radiation can be measured at different angles (variant I) or can be generated at different angles (variant II); the latter may result in different angles of incidence. Of course, variants I and II can also be combined.

[0135] The return radiation that can be sensed by the sensing device 200 may be reflected by the plant object 1, and / or may be reflected or scattered after transmitting through at least a portion of the plant object 1, and / or may be luminescence from the plant object 1 due to illumination by the radiation 111.

[0136] Figure 2A (Variant I) Very schematically depicts the different transmissions (transmission curves) of two different plant parts, such as, for example, leaves and stems or leaves and fruits. Radiations 1111 and 1112 and λ1 and λ2 indicate wavelengths of radiation that will have different effects on different plant parts. A first wavelength λ1 will be absorbed by the first plant part (solid curve) and will be substantially not absorbed by the second plant part (dashed curve); a second wavelength λ2 will be absorbed by the second plant part (dashed curve) and may be substantially not absorbed by the first plant part (solid curve). Variant II very schematically depicts the luminescence at different wavelengths of radiation. With λ a Illumination provides a clear luminescence band, while using λ b There is basically no luminescence under irradiation (the weak luminescence observed is also relative to λ a Note that there may be parts of the plant that emit virtually no light at all.

[0137] Figure 2B (Top) Schematically depicted are multiple pulses of radiation 111. These pulses may all be identical.

[0138] Here, for example, the pulses are substantially identical in pulse duration and distance between pulses, but the pulses may differ in spectral intensity distribution (and / or polarization). This is done using λ1, λ2, and λ n To indicate, n One of the aforementioned wavelengths, or even yet another wavelength different from λ1 and λ2 may be indicated.Alternatively or additionally, the pulses may also differ in polarization.

[0139] Furthermore, alternatively or additionally, the pulses may differ in one or more of pulse duration, pulse intensity (pulse height), pulse repetition rate, etc.

[0140] Generally speaking, with ToF detection in mind, the pulses are relatively short and have short rise and fall times. Pulses as short as about 1.25ns full width at half maximum (FWHM) may also be possible.

[0141] Following the pulse, or each pulse, radiation from the object can be detected. As described above, this can include reflected radiation, transmitted radiation, scattered radiation, or luminescence. This radiation can be detected as a function of time after the pulse. This sensing can be performed for one or more, and in particular, multiple, wavelengths of radiation, and in particular at at least two different times after the pulse.

[0142] Figure 2B(Bottom) Two embodiments or variants are schematically depicted, in which the radiation is measured after each pulse at three different times. In variant I, the spectral intensity distribution over a range of wavelengths is sensed; in variant II, the intensity is measured at two or more different wavelengths (here as an example, three different wavelengths). The spectral intensity can vary over time (after a pulse) and can also vary over time with respect to each other (after a pulse). Based on the spectral intensity and its time dependence, one or more of the 3D conformation of the plant, the type of plant, the different plant parts, the stage of growth, the presence of disease, the health of the plant, etc. can be determined. Thus, based on the sensing system signal, one or more of the 3D conformation of the plant, the type of plant, the different plant parts, the stage of growth, the presence of disease, the health of the plant, etc. can be determined.

[0143] As will be known to those skilled in the art, multiple pulses of two or more different radiation are typically generated in succession with (at least) a sensing time in between.The two or more pulses may differ, for example, in wavelength and / or polarization.

[0144] Figure 3A Some embodiments of a radiation generator 100 are schematically depicted, wherein the radiation generator comprises at least two radiation sources, indicated with reference numeral 110 .

[0145] Variant I schematically depicts a cross-sectional view of an embodiment of a radiation generator 100 in which two different radiation sources 110 (here, radiative laser sources 150) each generate radiation 151, but each having a different spectral intensity distribution, such as blue and red laser light. The different radiations are indicated by reference numerals 151' and 151". Using optics, the beams can be combined to provide a single beam. The radiation 111 escaping from the radiation generator 100 can be indicated by reference numerals 111' and 111", respectively. Note that these radiations 111' and 111" can be substantially identical to radiations 151' and 151", as the optics used can be selected to have substantially no effect on the spectral intensity distribution. Thus, in an embodiment, the radiation generator 100 can include two or more lasers 150, each configured to generate radiation 111 having a different spectral intensity distribution, and wherein the generator 100 is configured to generate multiple pulses of radiation 111 (continuously) in one or more of one or more time-of-flight sensing operating modes using the two or more lasers 150.

[0146] Variant II very schematically depicts a front view of an embodiment of a radiation generator 100, here comprising four different radiation sources 110. These can be configured to generate radiation 111, but with different spectral intensity distributions.

[0147] Figure 3AVariant III in FIG schematically depicts a radiation generator 100 similar to variant II, but now with a filter 120 downstream of the radiation source 110. The filter can be used to change the spectral intensity distribution of the respective radiation of the respective radiation source 110. Alternatively or additionally, the filter can be used to impart polarization to the respective radiation of the respective radiation source 110.

[0148] The radiation source 110 may provide the (ToF) control mode radiation 111 simultaneously, in particular when the sensing system can also detect different spectral intensities (such as wavelength-dependent spectral intensity distributions) and / or polarizations. In an alternative (ToF) control mode, the radiation source 110 may provide the radiation 111 sequentially.

[0149] In variant IV, the radiation source 110 can be configured to provide radiation with different wavelengths, in particular broadband radiation, although, for example, a white multi-LED radiation source is also possible. Alternatively or additionally, the radiation source 110 can be configured to provide radiation with different polarizations (or the radiation is unpolarized). Downstream of the radiation source, an optical wheel with a plurality of filters 120 can be provided. The filters can be used to select a subset of wavelengths and / or impose a specific polarization on the radiation 111.

[0150] Note that it may be possible to generate light beams with different optical paths (unlike variants I and IV) with radiation sources 110 arranged in different positions relative to each other (see variants II and III).

[0151] Thus, as described above, in embodiments, the generator 100 can be configured to generate multiple pulses of radiation 111 in one or more of one or more time-of-flight sensing operating modes, wherein the two or more pulses of radiation 111 differ in one or more of (a) optical properties and (b) angle of incidence of the radiation 111, and wherein the optical properties are selected from the group consisting of i polarization (of the radiation 111) and ii spectral intensity distribution.

[0152] Similar variations to those schematically depicted above with respect to the radiation generator 100 are also possible with Figure 3B The sensing device 200 shown in FIG.

[0153] Variant 1 schematically depicts an embodiment of a sensing arrangement 200 comprising a plurality (here four) of sensing devices 210. These sensing devices 210 may be configured to sense the same type of radiation (but inherently at (slightly) different angles), or may be configured to sense different types of radiation in terms of spectral intensity distribution and / or polarization.

[0154] Variant II schematically depicts an embodiment of a sensing apparatus 200 having multiple (here, four) sensing devices 210, which may, for example, be capable of sensing radiation of multiple wavelengths and / or be substantially polarization-independent. However, an optical filter 215 may be configured for a selected specific wavelength and / or polarization. In particular, the optical filter 215 is thus arranged upstream of the sensing device 210.

[0155] In variant III, an embodiment of a sensing apparatus 200 is schematically depicted, for example, with a single sensing device 210, which may be capable of sensing radiation of multiple wavelengths and / or substantially independent of polarization. However, a filter 215 included in the filter wheel can be used to select a specific wavelength and / or a specific polarization. In particular, the filter 215 is thus arranged upstream of the sensing device 210.

[0156] Note that in Figure 3B In variants III and IV, the small square 110 is behind the plane of the drawing and Figure 3B In variants II and III of , the (shaded) small square 210 is behind the plane of the drawing.

[0157] Figure 4A Some possible stages and variations of the sensing method are schematically depicted. Reference numeral UI indicates user input, reference numeral SI indicates sensor input, and II indicates input via an iterative process. Reference numeral C indicates a calculation stage, wherein based on one or more of the user input UI, the sensor input SI, and the iterative input II, optimized sensing parameters can be selected. Then, the sensing stage (S) can be initiated or started, which generates a sensing system signal SSS. Such a sensing system signal SSS can be used to derive or obtain information about the 3D conformation of the plant, the type of plant, different plant parts, the stage of growth, the presence of disease, the health of the plant, etc. The sensing system signal SSS can also be used as an input (trigger) for another device. Here, AS can refer to, for example, an agricultural system, which can include a device specifically for performing agricultural actions. For example, the action can be selected from the group consisting of controlling the growth conditions of the plant, controlling the irradiation of the plant or plant part, controlling the harvesting of the plant or plant part, controlling the treatment of the plant, and controlling the pruning of the plant.

[0158] Thus, in an embodiment, the sensing system 1000 may include one or more controllable sensing parameters, wherein the sensing system 1000 has an initial operating mode, wherein the value of the one or more controllable sensing parameters is defined depending on one or more of: (i) user input information, (ii) sensor signals from the sensor 310 (see also FIG. Figure 4B), and (iii) radiation received in a preliminary time-of-flight sensing operating mode (of the sensing system 1000), and wherein the sensing system 1000 is configured to perform one or more of the one or more time-of-flight sensing operating modes using defined sensing parameters after performing the initial operating mode.

[0159] As described above, the controllable sensing parameters are selected from the group consisting of: (i) polarization of the radiation 111, (ii) spectral intensity distribution of the radiation 111, (iii) angle of incidence of the radiation 111, (iv) pulse modulation and / or pulse frequency, and (v) a polarization filter 215 upstream of the detector 210 of the sensing device 200.

[0160] In an embodiment, the control system 300 may be configured to determine at least two different types of radiation 111 from an initial operating mode, wherein a first type of radiation 1111 has a greater penetration depth in the sensed plant object than a second type of radiation 1112, and to perform one or more of the one or more time-of-flight sensing operating modes using the at least two different types of radiation 111 (see also, e.g., Figure 2A (Variant II), in which the different optical properties of different plant parts are schematically depicted).

[0161] Figure 4B An embodiment of a sensing system 1000 is schematically depicted which further comprises a sensor 310. The sensor may in particular comprise an optical sensor, such as a camera. A plurality of (different types of) sensors may also be applied. In addition, Figure 4B An embodiment of an agricultural facility 2000 is also depicted very schematically. The agricultural facility 2000, such as a greenhouse, may include a sensing system 1000. In certain embodiments, one or more of the radiation generator 100 and the sensing device 200 is configured to be movable. Here, both can move along a track or can be moved along a track. In particular, the control system 300 can be configured to control one or more of (i) the position of the radiation generator 100 and (ii) the position of the sensing device 200. Optionally, the sensor 310 may also be configured to be movable. Alternatively or additionally, the sensor 310 may be included by one or more of the radiation generator 100 and the sensing device 200. Note that the sensor 310 is different from the sensing device.

[0162] like Figure 4B As schematically depicted in FIG, in an embodiment, both the radiation generator 100 and the sensing device 200 may be configured to be movable relative to each other.

[0163] As described above, the control system 300 is configured to perform an action based on the sensing system signal, wherein the action is selected from the group consisting of controlling growing conditions for plants, controlling illumination of plants or plant parts, controlling harvesting of plants or plant parts, controlling treatment of plants, and controlling pruning of plants; and wherein the agricultural facility 2000 is selected from the group consisting of a horticultural arrangement, a greenhouse, and an open space.

[0164] An embodiment of a possible agricultural device 2100, such as, for example, an irrigation and nutrient supply (spraying) system, is depicted very schematically. The sensing system 1000 can be functionally coupled to such agricultural device 2100. The agricultural device 2100 can be controlled based on the sensing system signals. Of course, other types of agricultural devices can also be used alternatively or additionally.

[0165] Thus, the present invention also provides a plant monitoring method comprising performing a time-of-flight sensing operating mode, wherein the method comprises subjecting one or more plants to pulses of radiation 111, sensing the radiation received by a sensing device as a function of time in the time-of-flight sensing operating mode, and generating a sensing system signal, and wherein the method comprises performing the time-of-flight sensing operating mode with a sensing system 1000 as described herein. Such a plant monitoring method may further comprise performing an action based on the sensing system signal, wherein the action is selected from the group consisting of controlling growing conditions, controlling harvesting, controlling treatment, and controlling pruning.

[0166] Figure 4B An embodiment in variant 1 is schematically depicted, in which the field of view (FoV) of sensor system 200 overlaps with (the beam of) radiation 111 from radiation generator 100 at a specific distance. Here, as an example, the focused beam of radiation 111 is depicted as a diverging sensing range. However, the sensing device may also utilize optics such that, for example, the focal point of radiation generator 100 and the focal point of sensing device 200 can overlap. Radiation 111 has a beam cross section A1. Sensing device 200 has a field of view cross section A2 (in one or more of one or more time-of-flight sensing operating modes). Therefore, at one or more distances, these can at least partially overlap, as is the case in variant 1. In variant 1, distance d1 indicates the distance from (the entrance window 205 of) sensing device 200 at which (the beam of) radiation 111 from radiation generator 100 and the field of view (FoV) of sensor system 200 overlap. Distance d1 may be a range below which there is no overlap and above which there is no overlap. Figure 4C (Variant I) indicates the lowest value of d1.

[0167] In variant II, sensing system 1000 is used such that radiation generator 100 and sensing device 200 have a predetermined configuration, wherein beam cross section A1 and field of view cross section A2 do not overlap within a predetermined distance d2 from entrance window 205 of sensing device 200, wherein predetermined distance d2 is selected from the range of 0-500 cm. In such an embodiment, the transmission and / or absorption of radiation 111 in plant object 1 can also be sensed.

[0168] Of course, other embodiments than those schematically depicted may be possible.

[0169] Figure 4D Schematically depicts some phenomena that may occur when irradiating a plant object 1 with radiation 111. Furthermore, two or more of these phenomena may occur together. Variant I indicates transmission, variant II indicates reflection; variant III indicates scattering, and variant IV indicates luminescence. Note that perpendicular irradiation may also result in reflection and / or scattering. Furthermore, in embodiments, irradiation at certain angles may also result in (some) transmission and / or luminescence.

[0170] Figure 4E Two diagrams explaining some phenomena are schematically depicted.

[0171] Assume that a plant object is illuminated with a narrow beam of radiation 111, also indicated by reference numeral 111a. The radiation measured by the sensing device may be indicated by radiation 111b, which may have a lower intensity due to one or more of absorption (see absorption curve A) and scattering and / or reflection outside the field of view of the sensing system. Absorption may result in loss (and / or conversion; see also below) of light.

[0172] Assume that a plant object is illuminated with broadband radiation 111, also indicated by reference numeral 111c. The radiation measured by the sensing device may be indicated by radiation 111d, which may have a lower intensity due to one or more of absorption (see absorption curve A) and scattering and / or reflection outside the field of view of the sensing system. Absorption may result in loss (or conversion; see also below) of light. Furthermore, since absorption is wavelength-dependent, the sensed radiation 111d may have a different wavelength dependency than the illuminating radiation 111c.

[0173] Assume that the plant object is illuminated with a narrow beam of radiation 111, also indicated in variant II by reference numeral 111e. The radiation measured by the sensing device can be indicated by radiation 111f, which may have a lower intensity due to one or more of absorption (see absorption curve A) and scattering and / or reflection outside the field of view of the sensing system. Absorption may result in loss and / or conversion of light; here, the emitted radiation 111g is also schematically depicted. Therefore, in an embodiment, illumination with radiation 111 (here indicated as 111e) can result in some reflection and / or scattering and / or transmission, as well as (some) emission (indicated by reference numeral 111f).

[0174] Thus, in embodiments, the present invention can provide an optical range sensor with active illumination optimized for observing plants and fruits to simultaneously extract 3D and spectral information, wherein one or more of (i) a single-pixel or multi-pixel (array / matrix) time-of-flight imager, (b) a laser scanner, and (c) a solid-state LIDAR can be applied. In embodiments, multispectral active illumination for time-of-flight ranging can be applied, wherein one or more of the following can be applied: (i) an optical sensor that responds to a broad spectrum combined with an active light source having one or more monochromatic wavelengths, the light sources being activated sequentially; and (ii) two or more sensing elements having different filters or different spectral sensitivities to have different spectral responses combined with an active light source having a broad spectral range. Thus, combinations can also be applied. In embodiments, monochromatic / multispectral active illumination using wavelengths that match the reflectance and / or transmission optical properties of fruits / plants can be applied, which can achieve one or more of the following functions: (i) improving signal quality for range measurements, (ii) providing spectral information with range information, (iii) seeing through leaves, (iv) sorting between fruits / leaves based on spectral reflectance at a single or multiple wavelengths, (v) assessing plant development, plant health, and disease detection based on spectral reflectance at a single or multiple wavelengths, etc. In embodiments, monochromatic / multispectral active illumination using wavelengths that match the reflectance optical properties of fruits / plants with 3D surface reconstruction can be applied, thereby achieving one or more of the following functions: (i) scattering / bidirectional reflectance distribution function measurements, (ii) reflectance property measurements, (iii) polarization property measurements, etc. In embodiments, monochromatic / multispectral active illumination using wavelengths that match the transmission optical properties of fruits / plants with 3D surface reconstruction achieves the following functions: (i) scattering properties, (ii) penetration depth in materials at specific wavelengths, etc. In an embodiment, an algorithm may be provided and / or used that can use the measured reflectance and transmission properties and the reconstructed shape properties for a specific wavelength to extract one or more of the following information: (i) health status, (ii) growth / development status, (iii) absence / presence of disease, (iv) maturity, etc. Additionally or alternatively, the spectral and shape properties of the inspected plants can be provided to a plant management system that can be used to create historical profiles and trends, and optimize growth cycles and detect anomalies. Additionally or alternatively, the spectral and shape properties of the inspected plants can be used to autonomously determine the quality of, for example, fruit, and guide robot / drone actuation to pick such qualified fruit through a shielded leaf canopy. The present invention can be applied to, among other things, one or more of, for example, growth monitoring, quality monitoring, crop and fruit assessment, property assessment, fruit harvesting, and utilization of lighting infrastructure placement and functions.

[0175] Range information (or distance information) can be used, for example, to compensate for signal attenuation depending on the measured distance. Similarly, shape information derived from a pixelated or scanning ToF solution can be used, for example, to account for the angle of incidence relative to an actively illuminated surface.

[0176] The term "plurality" means two or more.

[0177] Those skilled in the art will understand the terms "substantially" or "essentially" and similar terms herein. The terms "substantially" or "essentially" may also include embodiments with "completely," "entirely," "entirely," etc. Therefore, in embodiments, the adjectives "substantially" or "essentially" may also be removed. Where applicable, the terms "substantially" or "essentially" may also refer to 90% or higher, such as 95% or higher, particularly 99% or higher, even more particularly 99.5% or higher, including 100%.

[0178] The term "comprising" also includes embodiments in which the term "comprising" means "consisting of.

[0179] The term "and / or" specifically refers to one or more of the items mentioned before and after "and / or". For example, the phrase "item 1 and / or item 2" and similar phrases can refer to one or more of item 1 and item 2. The term "comprising" can mean "consisting of" in one embodiment, but can also mean "including at least the defined categories and optionally one or more other categories" in another embodiment.

[0180] Furthermore, the terms first, second, third, etc. in the description and claims are used to distinguish between similar elements and not necessarily to describe a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances, and that the embodiments of the invention described herein are capable of operation in other sequences than described or illustrated herein.

[0181] During operation, an apparatus, device or system may be described herein - among other things. As will be clear to one skilled in the art, the present invention is not limited to methods of operation, or apparatus, devices or systems in operation.

[0182] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims.

[0183] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim.

[0184] The use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. Unless the context clearly requires otherwise, throughout the specification and claims, the words "comprise," "comprising," etc. should be interpreted in an inclusive sense, rather than an exclusive or exhaustive sense; that is, in the sense of "including but not limited to."

[0185] The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.

[0186] The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a device claim, an apparatus claim, or a system claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0187] The present invention also provides a control system that can control a device, apparatus, or system, or can perform the methods or processes described herein. Still further, the present invention provides a computer program product that, when functionally coupled to a device, apparatus, or system or executed on a computer included in the device, apparatus, or system, controls one or more controllable elements of such a device, apparatus, or system.

[0188] The present invention also applies to an apparatus, device or system comprising one or more features described in the specification and / or shown in the accompanying drawings. The present invention also relates to a method or process comprising one or more features described in the specification and / or shown in the accompanying drawings.

[0189] The various aspects discussed in this patent may be combined to provide additional advantages. Furthermore, those skilled in the art will appreciate that embodiments may be combined, and more than two embodiments may be combined. Furthermore, some features may form the basis of one or more divisional applications.

Claims

1. A sensing system (1000) comprising a radiation generator (100), a sensing device (200), and a control system (300) functionally coupled to the radiation generator (100) and the sensing device (200), wherein the sensing system (1000) has one or more time-of-flight sensing operating modes, wherein the radiation generator (100) is configured to generate pulses of radiation (111) in the one or more time-of-flight sensing operating modes, and wherein the sensing device (200) is configured to sense the wavelength-dependent spectral intensity of radiation received by the sensing device (200) from a sensed object as a function of time in the one or more time-of-flight sensing modes to provide a sensing system signal; wherein the sensing system signal is indicative of a wavelength-dependent spectral intensity distribution of received radiation as a function of time in the one or more time-of-flight sensing modes; The radiation generator (100) and the sensing device (200) are configured to be movable relative to each other.

2. A sensing system (1000) according to claim 1, wherein the sensing system (1000) is functionally coupled to an agricultural device (2100), wherein the agricultural device includes a lighting device for illuminating plants or plant parts using a light recipe, and wherein the control system (300) controls the agricultural device (2100) to illuminate the plants or plant parts using the light recipe depending on the sensing system signal.

3. A sensing system (1000) according to claim 1, wherein the radiation generator (100) is configured to generate multiple pulses of radiation (111) in one or more of the one or more time-of-flight sensing operating modes, wherein two or more pulses of radiation (111) differ in the incident angle of the radiation (111).

4. A sensing system (1000) according to claim 2, wherein the radiation generator (100) is configured to generate multiple pulses of radiation (111) in one or more of the one or more time-of-flight sensing operating modes, wherein two or more pulses of radiation (111) differ in the incident angle of the radiation (111).

5. A sensing system (1000) according to any of the preceding claims 1-4, wherein the radiation generator (100) is configured to generate multiple pulses of radiation (111) in one or more of the one or more time-of-flight sensing operating modes, wherein two or more pulses of radiation (111) differ in (a) optical properties, wherein the optical properties are selected from the group consisting of (i) polarization and (ii) spectral intensity distribution.

6. A sensing system (1000) according to any one of the preceding claims 2-4, wherein the radiation generator (100) comprises two or more lasers (150), the two or more lasers (150) being configured to generate radiation (111) having different spectral intensity distributions, and wherein the radiation generator (100) is configured to generate multiple pulses of radiation (111) in one or more of the one or more time-of-flight sensing operating modes using the two or more lasers (150).

7. A sensing system (1000) according to any of the preceding claims 2-4, wherein the radiation generator (100) is configured to generate radiation (111) in one or more of the one or more time-of-flight sensing operating modes, the radiation (111) having a wavelength selected from the wavelength ranges of 200-300nm, 680-720nm, 920-960nm, 1080-1120nm, 1340-1420nm and 1850-1890nm.

8. A sensing system (1000) according to any one of the preceding claims 2-4, wherein the sensing system (1000) includes one or more controllable sensing parameters, wherein the sensing system (1000) has an initial operating mode, wherein the values ​​of the one or more controllable sensing parameters are defined depending on one or more of the following: (i) user input information, (ii) a sensor signal of a sensor (310), and (iii) radiation received in a preliminary time-of-flight sensing operating mode, and wherein the sensing system (1000) is configured to perform one or more of the one or more time-of-flight sensing operating modes using the defined sensing parameters after performing the initial operating mode.

9. A sensing system (1000) according to claim 8, wherein the controllable sensing parameter is selected from the group consisting of: (i) polarization of the radiation (111), (ii) spectral intensity distribution of the radiation (111), (iii) incident angle of the radiation (111), (iv) pulse modulation and / or pulse frequency, and (v) a polarization filter (215) upstream of the detector (210) of the sensing device (200).

10. A sensing system (1000) according to claim 8, wherein the control system (300) is configured to determine at least two different types of radiation (111) from the initial operating mode, wherein a first type of radiation (1111) has a greater penetration depth in the sensed plant object than a second type of radiation (1112); and to perform one or more of one or more time-of-flight sensing operating modes using the at least two different types of radiation (111).

11. A sensing system (1000) according to claim 9, wherein the control system (300) is configured to determine at least two different types of radiation (111) from the initial operating mode, wherein a first type of radiation (1111) has a greater penetration depth in the sensed plant object than a second type of radiation (1112); and to perform one or more of one or more time-of-flight sensing operating modes using the at least two different types of radiation (111).

12. A sensing system (1000) according to any one of the preceding claims 1-4, wherein the radiation (111) has a beam cross-section (A1), wherein the sensing device (200) has a field of view cross-section (A2), wherein the radiation generator (100) and the sensing device (200) have a predetermined configuration, wherein within a predetermined distance (d2) from an entrance window (205) of the sensing device (200), the beam cross-section (A1) and the field of view cross-section (A2) do not overlap, wherein the predetermined distance (d2) is selected from the range of 0-500 cm.

13. An agricultural facility (2000), comprising a sensing system (1000) according to any one of the preceding claims 1-12, wherein the radiation generator (100) and one or more of the sensing device (200) are configured to be movable, and wherein the control system (300) is configured to control one or more of (i) the position of the radiation generator (100) and (ii) the position of the sensing device (200).

14. The agricultural facility (2000) according to claim 13, wherein both the radiation generator (100) and the sensing device (200) are configured to be movable relative to each other.

15. The agricultural facility (2000) according to claim 13, wherein the control system (300) is configured to perform an action depending on the sensing system signal, wherein the action is selected from the group consisting of controlling the growing conditions of plants, controlling the illumination of plants or plant parts, controlling the harvesting of plants or plant parts, controlling the processing of plants, and controlling the pruning of plants; and wherein the agricultural facility (2000) is selected from the group consisting of a horticultural arrangement, a greenhouse, and an open space.

16. The agricultural facility (2000) according to claim 14, wherein the control system (300) is configured to perform an action depending on the sensing system signal, wherein the action is selected from the group consisting of controlling the growing conditions of plants, controlling the illumination of plants or plant parts, controlling the harvesting of plants or plant parts, controlling the processing of plants, and controlling the pruning of plants; and wherein the agricultural facility (2000) is selected from the group consisting of a horticultural arrangement, a greenhouse, and an open space.

Images