Accurate application of liquids to a target object in an agricultural field

Abstract

The present invention relates to the technical field of precision agriculture. The invention relates to a system and a method for accurate application of a liquid to a target object in an agricultural field.

Description

[0001] Precise application of liquids to a target object in an agricultural field

[0002] The present disclosure relates to the technical field of precision agriculture. The present invention relates to a device and a method for precisely applying a liquid to a target object in an agricultural field.

[0003] The term precision agriculture refers to methods of site-specific and targeted management of agricultural land.

[0004] One task in precision agriculture is to detect objects such as plants, pests, or companion plants in a digitally recorded image, then make a diagnosis, and then perform an action (e.g., treatment). An example of such a task is "ultra-high-precision spraying" (UHPS), in which a tractor drives across a planted field and takes consecutive images of the agricultural field with the attached spray device. Using image processing algorithms, the companion plants, for example, are identified in the images and then sprayed with a herbicide using a spray boom in a single treatment step.

[0005] This type of task requires tracking target objects such as companion plants once they have been detected from the current position on the images taken while driving until the spray boom is positioned close to a target object to be treated due to the tractor movement, in order to then open one or more nozzles at the right moment, e.g. to spray a herbicide, and close them again after the required opening time.

[0006] One challenge with UHPS is scheduling and activating the nozzles at the right time and in the right place, as it must be ensured that the precision of the fluid application to the target objects in the agricultural field is geared to hitting the target object as accurately as possible. For a UHPS with an accuracy of up to 4 cm x 4 cm, very precise nozzle activation scheduling is necessary. For a tractor traveling at a speed of 10 km / h, for example, it already covers a distance (per unit of time) of approximately 278 cm / second = approximately 2.8 mm / millisecond. If you want to hit small target objects with diameters in the cm range, you have to calculate the opening times to the millisecond.

[0007] These and other challenges are addressed by the subject matter of the independent claims. Preferred embodiments can be found in the dependent claims, the present description, and the drawings.

[0008] A first object of the present invention is a device for applying a liquid to at least part of a target object in an agricultural field, comprising a computing and control unit, at least one reservoir for receiving the liquid, at least one nozzle,

[0009] Means for conveying the liquid from the at least one storage container in the direction of the at least one nozzle, at least one sensor unit which can be directed towards the agricultural field towards a target object and is configured to generate at least one image, at least one position-determining unit, wherein the computing and control unit is configured (a) to cause the at least one sensor unit to generate at least one image at time 0 and to receive this at least one image, wherein the computing and control unit is configured (b) to cause the at least one position-determining unit to determine the position of the at least one sensor unit at time t1 and to receive this position data, wherein the computing and control unit is configured,(c) to determine at least a part of a target object - if present - on the at least one image by image processing and to georeference at least a part of this target object using the position data of the at least one sensor unit from time t1 in order to determine the distance Sj from the at least one nozzle to the at least a part of the target object on the image from time t1, wherein the computing and control unit is configured, (d) to cause the at least one position-determining unit to determine the position of the at least one nozzle at time t2 and to receive this position data, wherein the time t2 is a later time than t1, wherein the computing and control unit is configured,(e) determine the distance S2 from the at least one nozzle to the at least one part of the target object based on the position data of the at least one nozzle from time t2 and the position data of the at least one georeferenced part of the target object on the image from time t1, wherein the computing and control unit is configured; (f) determine the speed v2 and direction r2 of the at least one nozzle from time t2 at least partially based on the position data of the at least one nozzle from time t1 and from time t1, wherein the computing and control unit is configured; (g) plan the activation time of the spraying process for the at least one nozzle on at least one part of the target object at least partially based on the distance S2 from the at least one nozzle to the at least one part of the target object from time t1 and the speed v2 and direction r2 of the at least one nozzle from time t2.

[0010] A further subject of the present disclosure is a method for applying a liquid to at least part of a target object in an agricultural field, comprising the steps (1) to (3): in step (1) moving a device for applying a liquid to at least part of a target object in an agricultural field, in step (2) conveying a liquid from at least one storage container towards at least one nozzle during the movement, in step (3a) causing at least one sensor unit to generate at least one image at time t1 and to forward this at least one image to the computing and control unit, in step (3b) causing at least one position determination unit to determine the position of the at least one sensor unit at time t1 and to forward this position data to the computing and control unit,in step (3c) determining at least a part of a target object - if present - on the at least one image by image processing and georeferencing the at least part of the target object using the position data of the at least one sensor unit from time ty to determine the distance Sj from the at least one nozzle to the at least part of the target object on the image from time ti, in step (3d) causing the at least one position determination unit to determine the position of the at least one nozzle at time t2 and to forward this position data to the computing and control unit, wherein time t2 is a later time than ti,in step (3e) determining the distance S2 from the at least one nozzle to the at least one part of the target object based on the position data of the at least one nozzle from time t2 and the position data of the at least one georeferenced part of the target object on the image from time t1, in step (3f) determining the speed v2 and the direction r2 of the at least one nozzle from time t2 at least partially based on the position data of the at least one nozzle from time t1 and time t2, in step (3g) planning the activation time of the spraying process for the at least one nozzle on at least one part of the target object at least partially based on the distance S2 from the at least one nozzle to the at least one part of the target object from time t1 and the speed v2 and direction r2 of the at least one nozzle from time t2.

[0011] The invention is explained in more detail below, without distinguishing between the subject matter of the invention (device, method). Rather, the following explanations are intended to apply analogously to all subject matter of the invention, regardless of the context (device, method) in which they occur.

[0012] The present invention discloses a device for applying a liquid to at least part of a target object in an agricultural field. This means that the liquid can be applied to at least part of a target object in an agricultural field. Alternatively, the liquid can be applied to substantially the entire target object. It is also possible for the liquid to be applied to multiple target objects.

[0013] The liquid may be water or an aqueous solution or suspension. The aqueous solution or suspension may contain one or more nutrients and / or one or more crop protection agents and / or one or more seed treatment agents.

[0014] The term "nutrients" refers to those inorganic and organic compounds from which plants can extract the elements from which their bodies are built. These elements themselves are often also referred to as nutrients. These are usually simple inorganic compounds such as nitrate (NO3) and phosphate (PO4). 3 - 1 and potassium (K +In addition to the core elements of organic matter (C, O, H, N, and P), K, S, Ca, Mg, Mo, Cu, Zn, Fe, B, Mn, CI in higher plants, Co, and Ni are also essential for life. Different compounds can be present for the individual nutrients; for example, nitrogen can be supplied as nitrate, ammonium, or amino acid.

[0015] The term "plant protection product" refers to a product used to protect plants or plant products from pests or to prevent their effects, to destroy undesirable plants or parts of plants, to inhibit or prevent undesirable plant growth, and / or to influence plant life processes in a manner other than by providing nutrients (e.g., growth regulators). Examples of plant protection products include herbicides, fungicides, and other pesticides (e.g., insecticides).

[0016] Growth regulators are used, for example, to increase lodging in cereals by shortening stalk length (intermode shorteners), improve rooting of cuttings, reduce plant height by compression in horticulture, or prevent germination of potatoes. Growth regulators can be, for example, phytohormones or their synthetic analogues.

[0017] The term “target object” describes one or more plants, one or more areas of a field, pests or other objects. Preferably, the term “target object” describes one or more plants. In a further preferred embodiment, the target object is located on the ground or near the ground. The term “at least part of a target object” preferably describes a part (e.g. a leaf, leaf section, stem, stem section, etc.) of one or more plants or one or more pests. In a preferred embodiment, the target objects are individual crop plants or (individual) parts of individual crop plants or individual groups of crop plants. In a further preferred embodiment, the target objects are individual seeds or groups of seeds that are and / or have been sown in a field for crop plants.In a further preferred embodiment, the target objects are one or more companion plants or (individual) parts of individual companion plants or individual groups of companion plants. In a further embodiment, the target objects are plant components infested by pests. Such pests can be animal pests, fungi, viruses, or bacteria.

[0018] The term “cultivated plant” refers to a plant that is purposefully cultivated as a useful or ornamental plant through human intervention.

[0019] The term “companion plants” (often also referred to as weeds) refers to plants of the spontaneous accompanying vegetation (segetal flora) in crop stands, grassland or gardens, which are not deliberately cultivated there and develop, for example, from the seed potential of the soil or via migration.

[0020] The term “agricultural field” refers to a spatially definable area of ​​the earth’s surface that is used for agricultural purposes, in which crops are planted, possibly supplied with nutrients and harvested.

[0021] The term "processing and control unit" refers to components of a computer or processor that perform arithmetic and logical operations, as well as control sequences of instructions and operations. In modern computers, the processing unit and control unit are often closely linked and together form what is known as the "CPU" ("Central Processing Unit"). These two units work together to enable the processing and execution of instructions in a computer. It is also possible that at least certain functions are performed via "cloud"-based computing.

[0022] The liquid is located in a storage container before application. There may be multiple storage containers. Several (different) liquids may be applied. The liquid is applied via one or more nozzles onto at least part of the target object. In one embodiment, the at least one nozzle is a component of an inkjet print head. In this embodiment, the technology used in inkjet printers is used to apply the liquid onto at least part of the target object. The use of inkjet printer technology in agriculture is described, for example, in M.-Idbella et al.: Structure, Functionality, Compatibility with Pesticides and Beneficial Microbes, and Potential Applications of a New Delivery System Based on Ink-Jet Technology, Sensors 2023, 23(6), 3053.In a preferred embodiment, several nozzles are used, which are preferably enclosed by a bar or a strip (hereinafter referred to as “spray bar”).

[0023] In a further preferred variant, the device according to the invention comprises a plurality of nozzles. The term "plurality" preferably means more than ten. The nozzles are preferably arranged such that each nozzle applies liquid in an area with a maximum lateral extent of less than 20 cm, preferably less than 10 cm, most preferably less than 5 cm. The plurality of nozzles can, for example, be arranged next to one another along a spray bar that extends transversely (e.g., at an angle of 90°) to the direction of movement of the device. Each nozzle can be assigned at least one sensor unit (e.g., a camera). In such an embodiment, there is the possibility that a target object leaves the field of view of the at least one sensor unit when cornering, and therefore the adjacent sensor unit is activated.For application, the liquid is conveyed from the at least one storage container toward the at least one nozzle by means of conveying equipment. A pump, for example, can be used to convey the liquid.

[0024] The device according to the invention comprises at least one sensor unit. A sensor unit comprises at least one sensor. A "sensor" is a technical component that can detect certain physical and / or chemical properties and / or the material properties of its environment qualitatively or quantitatively as a measured variable. These variables are detected by means of physical or chemical effects and converted into a further processable, usually electrical or optical signal. A sensor unit can contain means for processing signals supplied by the at least one sensor. A sensor unit can comprise means for transmitting and / or forwarding signals and / or information (e.g., to the control unit). The one or more sensor units can be part of the system and / or connected to it via a communication link (e.g., via radio).The one or more sensor units can be configured to transmit one or more signals to the computing and control unit continuously or at defined time intervals or upon the occurrence of defined events, on the basis of which the computing and control unit carries out the further steps, for example to plan the activation time of the spraying process for the at least one nozzle.

[0025] The at least one sensor unit, which can be directed towards a target object in the direction of the agricultural field, preferably comprises one or more cameras. This means that the term "recording" preferably describes an "image recording". The camera axis of the at least one camera as a component of the device according to the invention can preferably be aligned substantially perpendicular (vertical) to the ground (i.e. the at least one camera points "down" or "straight ahead" with respect to the ground). Preferably, the at least one camera is located (in the direction of travel) in front of the at least one spray unit. This distance is preferably between 30 cm and 100 cm, more preferably between 40 and 60 cm. The camera used according to the invention can comprise an image sensor and optical elements. The image sensor is a device for recording two-dimensional images from light by electrical means.These are typically semiconductor-based image sensors such as charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS) sensors. The optical elements (lenses, apertures, and the like) serve to produce the sharpest possible image of an object on the image sensor.

[0026] The camera can be caused by the computing and control unit to create image recordings of at least part of a target object (list point (a) in the device according to the invention and step (3a) in the method according to the invention) at defined time intervals or continuously. The at least one part of a target object can be, for example, a cultivated plant, a companion plant and / or a pest. The computing and control unit can be the computing and control unit of the device according to the invention or a separate computing and control unit. The computing and control unit of the device according to the invention is preferably used for this purpose. Each image is marked with a timestamp in order to document the exact time the image was taken. It can also happen that no part of a target object is present in an individual image and, for example, only the ground orthe agricultural field is captured without any part of a target object.

[0027] The inventive design of the device comprises, in addition to the at least one sensor unit described above, at least one position-determining unit. The at least one position-determining unit can also comprise one or more sensors. The at least one position-determining unit of the device is configured such that, at the respective time at which the at least one sensor unit generates an image, the position data of the at least one sensor unit (the camera and preferably the position data of the camera lens center) are recorded, and this information is forwarded to the computing and control unit of the device (item (b) for the device and step (3b) for the method).

[0028] The at least one positioning unit can, for example, comprise a receiver of a satellite navigation system (GNSS, Global Navigation Satellite System), colloquially also referred to as a GPS receiver. The Global Positioning System (abbreviation: GPS), officially NAVSTAR GPS, is an example of a global satellite navigation system for positioning; other examples are GLONASS, Galileo, and Beidou. The satellites of such a satellite navigation system communicate their precise position and time via radio codes. To determine the position, a receiver (the "GPS receiver") must receive the signals from at least four satellites simultaneously. The receiver measures the pseudo-signal propagation times, and uses these to determine the current position.

[0029] In a preferred embodiment, at least two GNSS receivers are used. The position of at least one nozzle of the device is preferably determined using a moving baseline differential GNSS (DGPS) installation, which enables a position determination accuracy of less than 10 mm and achieves update rates between 8 and 100 Hz. In the moving baseline DGPS installation, two GNSS receivers with connected GNSS sensors are mounted at a fixed distance on the device according to the invention. One of these assumes the role of a moving baseline, the other that of a rover. For example, these two components can be installed at opposite ends of the spray boom. With this technology, DGPS correction data can be generated during travel, enabling the aforementioned accuracy.Since the distance of the at least one sensor unit (preferably the camera lens) and the at least one nozzle from the GNSS receivers on the device according to the invention is fixed, their positions can be determined very quickly based on the position of the GNSS receivers with only a few computational operations. This means that as soon as the position-determining unit has determined the position of the at least one sensor unit, the position of the at least one nozzle is also known. Analogously, when the position of the at least one nozzle is determined at a specific time, the position of the at least one sensor unit at the same time is also known.

[0030] The computing and control unit of the device (or method) receives the at least one image from time t1 from the sensor unit and the position data from time t1 from the at least one position-determining unit. The computing and control unit is configured to process the images using image processing in order to recognize at least part of a target object—if present—in the images, e.g., a defined plant (e.g., a companion plant), a part of a defined plant (e.g., a leaf), a pest, a plant infested with a pest, and / or another object (list item (c) for the device and step (3c) for the method). Numerous methods and devices for detecting objects in image recordings are described in the prior art (see, for example, WO2020120802A1, W02020120804A1, WO2020229585A1).Different methods can be used for image processing for object recognition, such as feature extraction with deep learning models such as neural networks, support vector machines (SVM), decision trees, etc. Other possible methods that can be used are color-based recognition, texture analysis, shape-based recognition, spectral analysis, color histogram analysis, the histogram of oriented gradients (HOG) method, morphological operations, or a combination of these methods.

[0031] In a preferred embodiment, the computing and control unit is configured to determine at least part of the target object on the image using machine learning. The models used in this image processing are preferably based on deep learning techniques, such as neural networks (for example, a convolutional neural network (CNN)). Such a model is trained with annotated images to learn how to distinguish between different classes, such as a companion plant and other plants. After training, such a model can be applied to new images. A image received by the at least one sensor unit can be preprocessed for use with the model.Preprocessing includes common methods such as normalization or scaling steps, noise removal, changing contrast or other methods to improve data quality, as well as converting the image into the model's input format. The model assigns each pixel in the image to a specific class, for example, whether the pixel belongs to a target object, a non-target object, or the background. This process is also referred to as semantic segmentation. Semantic segmentation with regard to the identification of plants in a field is described in the prior art (see, for example, WO19226869 A1). As part of the postprocessing of the classified data at the pixel level of the model, transformations or analysis operations such as the generation of pixel masks (e.g.binary representation of pixels that belong to the detected target object or not), the generation of "bounding boxes" (rectangular frames around a detected target object), the determination of the centroid (center of the surface of the detected target object that can be used as a representative point for the detected target object), the generation of an outline polygon (a series of points that describe the contour of the detected target object). In addition to semantic segmentation, "object detection" algorithms such as Yolo, SSD, Faster-RCNN, etc. can also be used. With these algorithms, it is possible to directly generate "bounding boxes" for the detected target object instead of the intermediate path via the pixel masks.

[0032] Ideally, image analysis detects the entire target object. However, it can happen that only part of a target object is present in the image, and only this part can be detected. The image processing methods mentioned above, including the pre- and post-processing steps, can also be performed if only part of a target object was detected in the image.

[0033] After image processing, at least a portion of the detected target object is georeferenced with the position data of the at least one sensor unit from time t1 (item (c) in the device and step (3c) in the method). In one embodiment of the device according to the invention, at least one pixel coordinate of at least a portion of the detected target object is converted into position data on the agricultural field. In a further preferred embodiment, this one pixel coordinate is the centroid of the detected target object. In a further preferred embodiment, several pixel coordinates of the detected target object are georeferenced, such as all points of the pixel mask that were assigned to the detected target object or all points within the "bounding box" or the outline polygon of the detected target object. Based on the georeferenced point orThe distance Sj from the at least one nozzle to the at least part of the target object in the image taken at time ti can be determined using the georeferenced points, as well as the knowledge of the camera's optical parameters (e.g., the camera's aperture angle, the camera's vertical and horizontal resolution), and the camera's distance (preferably from the camera lens or from the camera lens center) to the ground of the agricultural field. In a preferred variant of the invention, this calculation is performed using the following formulas:

[0034] (1) (2) where in formula (1) Sy is the number of pixels per unit height on the camera's image sensor (vertical resolution per unit height), y is the vertical resolution of the camera, H is the distance from the camera lens to the ground and a ystands for the vertical aperture angle of the camera, where in formula (2) & is the number of pixels per unit width on the camera's image sensor (horizontal resolution per unit width), x is the horizontal resolution of the camera, H is the distance from the camera lens to the ground and a x stands for the horizontal aperture angle of the camera, where, starting from the pixel coordinate of at least one image, the respective reciprocal of S y and .S calculates the position data of the at least one pixel coordinate of at least a part of the target object on the agricultural field.

[0035] The computing and control unit is further configured to cause the at least one position-determining unit to determine the position of the at least one nozzle at time U and to receive this position data, wherein time t2 is a later time than t1 (item (d) for the device and step (3d) for the method). In a preferred embodiment, the device is in motion so that the position data of the at least one nozzle at time t1 and time t2 are not the same. In a further preferred variant, the device moves toward the target object.

[0036] The computing and control unit is further configured to determine the distance S2 from the at least one nozzle to the at least one part of the target object based on the position data of the at least one nozzle from time U and the position data of the at least one georeferenced part of the target object on the image from time U (item (e) in the device and step (3e) in the method).

[0037] The computing and control unit is further configured to determine the speed V2 and direction of the at least one nozzle from time t2 at least partially based on the position data of the at least one nozzle from time t1 and from time t2 (bullet point (f) in the device and step (3f) in the method). It should be noted that, when multiple nozzles are present, each of the nozzles can have a different speed when the device is cornering. The speed V2 of the at least one nozzle can be determined by knowing the distance traveled by the device between t1 and U. The direction r2 of the at least one nozzle from time U can be determined by knowing the two different position data of the at least one nozzle from time t1 and time t2. The direction between two coordinate points is usually expressed as azimuth or heading.The azimuth is the angle in degrees measured clockwise from the north direction.

[0038] The computing and control unit is further configured to plan the activation time of the spraying process for the at least one nozzle on at least a part of the target object at least partially based on the distance S2 from the at least one nozzle to the at least a part of the target object from time U and the speed V2 and direction / vdcr of at least one nozzle from time U (bullet point (g) in the device and step (3g) in the method).

[0039] In a preferred embodiment, the computing and control unit is further configured to activate the at least one nozzle at least partially based on the planned activation time of the spraying process and to apply the liquid to at least a part of a target object (bullet point (h) in the device and step (3h) in the method).

[0040] In a further preferred embodiment, the computing and control unit is configured to carry out the steps mentioned in points (a) to (f) (steps 3a to 3f of the method) and the calculation of the respective parameters A, ti, v t , rt iteratively (i). The activation time of the spraying process for the at least one nozzle on at least part of a target object in the step mentioned in bullet point (g) can be adjusted as needed through each iterative step. Preferably, during an iterative implementation, the target object is fully identified by the image analysis. This makes it possible to plan the activation time of the spraying process for the entire target object and also to carry out the spraying process accordingly for the entire target object.

[0041] In a preferred embodiment, measurement data (images and - if available - measured position determination data) from several iteration cycles are used for the calculations (bullet points (c), (e), (f) and (g) as well as (c'), (e'), (f ) and (g') in the device and steps (3c), (3e), (3f) and (3g), as well as (3c'), (3e' ), (3f ) and (3g') in the method). Preferably, measurement data from between 2 - 100, more preferably between 3 - 50 and even more preferably between 3 - 10 iteration cycles are used. The larger database allows the measurement and analysis data to be stabilized. At the same time, limiting the iteration cycles prevents old, now irrelevant data from being used for the calculation steps.

[0042] In a further preferred variant of the present device, at least two positioning units are used. Examples of positioning units that can be used are GNSS (Global Navigation Satellite System) receivers, positioning units based on visual odometry technology, radar, trilateration, inertial navigation systems (e.g., gyroscopes, accelerometers), magnetometers, radio navigation systems, etc. The first positioning unit preferably comprises at least one GNSS (Global Navigation Satellite System) receiver, and the second positioning unit is based on visual odometry technology and / or radar. In general, sensor fusion, i.e., the use of multiple positioning units, can improve the accuracy and reliability of positioning. By combining multiple sensors, errors and inaccuracies of individual sensors can be compensated.For example, GPS, visual odometry, and / or radar can be used simultaneously to achieve more precise positioning. Sensor fusion can be achieved using algorithms and techniques such as the Kalman filter or other fused estimation methods to combine data from the various sensors and obtain an optimal position estimate.

[0043] In a further preferred embodiment, the computing and control unit is configured to check, before and / or during a position data determination with the GNSS receiver, whether the signal strength of the GNSS receiver reaches or exceeds a predefined threshold level. If the predefined threshold level is not reached, the position determination is determined by the positioning unit, which comprises visual odometry technology. The definition of a suitable threshold level for a GNSS receiver depends on various factors, such as the sensitivity of the GNSS receiver. A possible threshold could, for example, be in the range of -130 dBm (decibel milliwatts) to -150 dBM.

[0044] In a preferred variant, the signal strength is measured before each execution of the steps described in points (b) and (d) for the device (steps (3b) and (3d) for the method) and it is checked whether the signal strength of the GNSS receiver reaches or exceeds the predefined threshold level. Preferably, this is only carried out before each execution of the step described in points (b) for the device (step (3b) for the method). If, for example, at the time

[0045] Position data determination was carried out using visual odometry and at time t t ,i it is determined that the signal strength of the GNSS receiver now reaches the threshold level, the position data determination for the time t i:1by the GNSS receiver. During the next iteration, a check is again carried out to determine whether the signal strength is sufficient to use the GNSS receiver or whether it is necessary to switch back to visual odometry. Switching between the two positioning units is therefore preferably carried out as seamlessly as possible in order to ensure uninterrupted position determination. In this case, an attempt is preferably made first to determine the position using at least one GNSS receiver. Such a method can also be carried out using a different positioning unit technology, such as radar (see above), instead of visual odometry. The method is explained in more detail below using visual odometry.

[0046] Visual odometry is based on tracking the movement and position of an object (preferably at least part of a target object) in the field using images from at least one camera. By analyzing consecutive images, the computing and control unit identifies visual features that are stable and uniquely identifiable. These are preferably characteristic points of at least part of a target object. The image analysis tracks the movement of the identified features between consecutive images. This can be achieved using methods such as optical flow or feature matching algorithms. Based on the change in the position of the identified features, the movement of the object (preferably at least part of a target object) can be estimated. This estimation can include translation (movement in the x, y, and z directions) and rotation (rotation around the axes).It is also possible to integrate simultaneous localization and mapping (SLAM), which, in addition to the direct positioning, also creates a relative coordinate system (map of the environment). The visual odometry technology can include additional sensor units to improve the accuracy of positioning with this technology using sensor fusion. Such additional sensor units include, for example, inertial sensors such as gyroscopes and accelerometers. Precise calibration of at least one camera and any additional sensor units is required to ensure accurate positioning.

[0047] In one embodiment (when using visual odometry for position determination), the computing and control unit is configured to cause the at least one sensor unit to generate at least one recording at the time t and t, wherein the time interval between t and t is selected to be so short that the recordings overlap (step (a' )), wherein the computing and control unit is configured to generate at the time t,2 - and deviating from the steps described in bullet points (c) and (e) to (g) - in the alternative steps (c'), (e') and (f) - at least partly on the basis of these recordings, at least one of the data selected from the group of distance S ii2 from the at least one nozzle to the at least part of the target object from time t ii2 , speed Vi,i of at least one nozzle from time to direction r itiof the at least one nozzle from time tj,i. The computing and control unit is further configured to determine (in alternative step (g)) the activation time of the spraying process for the at least one nozzle onto at least a part of a target object at least partially based on the distance .SA from the at least one nozzle to the at least a part of the target object from time tt,2 and the speed Vt,i and direction ii of the at least one nozzle from time t i:1 to plan. In the optional step (h), the computing and control unit is configured to activate the at least one nozzle at least partially based on the planned activation time of the spraying process in step (g') and to apply the liquid to at least one part of the target object.

[0048] In a preferred embodiment of the device, the computing and control unit is configured (in alternative step (c ')) to detect matches between the images from the times t i,i and (in alternative step (e )) to calculate the relative image movement and the rotation angle on the basis of this data. In a further preferred variant, the computing and control unit is configured to detect at least one matching pixel movement (preferably at least one pixel of at least part of a target object) of the current (7) image in comparison to the image from the previous iteration (z-7). This procedure is called "image registration" in computer vision. The relative movement and the rotation of this at least one matching pixel can be determined, for example, using a transformation matrix.In the alternative step (f ), the computing and control unit is configured to determine, on the basis of the determined relative movement and the angle of rotation, at least one of the data selected from the group of: the distance .SA from the at least one nozzle to the at least part of the target object at time tt,2, speed Vt,i of the at least one nozzle at time tt,i, direction rt,i of the at least one nozzle at time ti,i.

[0049] The distance St, 2 at time tt,2 can be determined, for example, via the intermediate step using the formula (3):

[0050] Sc = Vi,i ■ (ti,2 - ti,i)

[0051] (3) where Sc is the position of the at least one nozzle at time A2, Vi,i is the speed of the at least one nozzle at time tt,i. The time t t,i describes the time of the recording by the camera (bullet point (a\) in the device and step (3a ,) in the method) in iteration i. The time t i: 2 represents the time after the image processing (after the steps according to bullet point (c\), (e\) for the device and (3c',), (3e',) for the method) in iteration i (ie a time in step f in iteration 7). The distance .SA can be calculated using S c and the calculated rotation angle.

[0052] The speed v, from the time point can be determined, for example, by the formula (4): where Vi is the speed of at least one nozzle at the time and ^ z stands for the relative image movement (determined by image analysis). The denominator of formula (3) is the time difference between the two images from time t i:1 and The time ti, 1 has the same

[0053] Meaning as described in formula (3). The time -7,7 describes the time of the camera recording (bullet point (a ,.;) in the device and step (3a' w) in the method) in iteration i-1. The time interval between A-7,7 and A,i is chosen so short that the recordings overlap.

[0054] The direction r can be determined, for example, using a transformation matrix.

[0055] The times ti and A described above refer to the same iteration. In the method, the alternative steps (a'), (c'), (e'), (f ), (g'), (h') described above are labeled (3a'), (3c'), (3e ), (3f ), (3g'), (3h') in the third step.

[0056] A further embodiment of the invention relates to the device according to the invention, which comprises at least one further sensor unit that can be aligned toward a target object in the direction of the agricultural field. The computing and control unit is configured to cause this at least one further sensor unit to capture a 3D point cloud at time A, and the computing and control unit is further configured to receive this 3D point cloud.

[0057] Suitable sensor units for capturing a 3D point cloud include LIDAR sensors and 3D cameras. If multiple 2D cameras are available, a 3D point cloud at time A can be created using photogrammetric analysis (analysis of images taken from different perspectives). Techniques such as structure from motion (SfM) and triangulation can be used for this. LIDAR sensors emit laser beams and measure how long it takes for the reflected light to return. This enables the creation of 3D point clouds with precise distance data. 3D cameras (e.g., Kinect, RealSense, or others) capture depth information in addition to pixel data, which can be converted into 3D coordinates to create a 3D point cloud. A 3D point cloud is a data structure that represents a collection of three-dimensional points in space.Each point in the point cloud is described by x, y and z coordinates and can also contain additional information such as color or intensity.

[0058] In a preferred embodiment of the invention, the control and computing unit is configured (in step (c) or (c') of the device and step (3c) or (3c') of the method) to project the at least one pixel coordinate of at least a portion of a target object in the agricultural field onto the 3D point cloud data. Preferably, several pixels of the at least a portion of the target object (e.g., pixels of the corner points of the bounding boxes or of a contour polygon, etc.) are projected onto the 3D point cloud data. In a further preferred embodiment of the invention, the control and computing unit is configured to determine the height of the target object in the agricultural field using the 3D point cloud, wherein the distance between the vertically lowest point of the target object and the vertically highest point of the target object in the 3D point cloud is calculated. Image analysis of at least one complete target object is preferred.In a further preferred embodiment, the computing and control unit is configured (item (g) or (g') in the device and step (3g) or (3g') in the method) to determine the activation time of the spraying process for the at least one nozzle on at least part of the target object, at least partially additionally based on the data of the 3D point cloud at time t 1 . Preferably, the calculated height of the target object in the agricultural field is also taken into account (since the flight time of the liquid shortens or lengthens depending on the height of the target object).

[0059] In a further embodiment of the invention, the computing and control unit is configured (item (g) or (g') in the device and step (3g) or (3g') in the method) to at least partially take into account the latency of the at least one nozzle and the flight time of the liquid from the at least one nozzle until it hits at least part of the target object in order to determine the activation time of the spraying process for the at least one nozzle onto at least part of the target object.

[0060] Using the information on the distance S2 from the at least one nozzle to the at least part of the target object at time t2 and the speed v2 and direction r2 of the at least one nozzle at time t2, the nozzle opening time tA (i.e., the activation time for the nozzle opening, which in this disclosure has the same meaning as the activation time of the spraying process) can be determined. In a preferred embodiment of the invention, this nozzle opening time tA is further corrected with respect to the latency of the at least one nozzle LD (i.e., the nozzle opening latency) as well as the flight time of the liquid from the at least one nozzle until it hits the target object tF. The planned nozzle opening time tP can be calculated using the following formula (5): tP = tA - LD - tF

[0061] (5)

[0062] Where tP stands for the planned nozzle opening time, LD for the nozzle opening latency of at least one nozzle and tF for the flight time of the liquid from the at least one nozzle until it hits the target object.

[0063] LD results from various factors such as the nozzle type and design, the fluid pressure, the nature of the computing and control unit, the nozzle size, the fluid viscosity, the maintenance status of at least one nozzle, and the ambient temperature. tF results from the fluid drop velocity, which in turn depends particularly on the fluid pressure and the mechanical properties of the nozzle (and can be determined in the laboratory). Preferably, the calculated height of the target object in the agricultural field is also taken into account when determining tF.

[0064] In a further preferred variant of the invention, the computing and control unit is configured (in bullet points (g) and (g') for the device or (3g) and (3g') for the method) to plan the duration of the nozzle opening. This means the duration of the opening state of at least one nozzle from the time the nozzle opens. The duration of the nozzle opening is preferably based on the size of the target object: depending on the vertical extent of the target object, the nozzle speed is used to calculate how long a nozzle must be open in order to completely treat the entire target object. Therefore, the calculated height of the target object on the agricultural field is preferably also taken into account when determining the duration of the nozzle opening. In a further preferred variant, the nozzle closing latency (which is often greater than the nozzle opening latency due to the pressure) is also taken into account when planning the duration of the nozzle opening.

[0065] The device according to the invention can be part of, or connectable to, a spraying device, for example, an agricultural machine, a robot, or an aircraft (e.g., a drone). Such a spraying device can move autonomously in or over a field or be controlled by a human.

[0066] Embodiments of the present invention are:

[0067] Figures 1 to 3 show, by way of example and schematically, an embodiment of the device according to the invention and the method according to the invention.

[0068] Figure 1 shows the device (10) for applying a liquid (F) to at least part of a target object (PI) in an agricultural field (LF). The device (10) comprises a computing and control unit (11), at least one storage container (12) for receiving the liquid (F), at least one nozzle (13), at least one position-determining unit (14), means (15) for conveying the liquid (F) from the at least one storage container (12) in the direction of the at least one nozzle (13), and at least one sensor unit (16) which is directed towards a target object (PI) in the direction of the agricultural field (F). In Figure 1, the device (10) is moving towards the target object (PI) (i.e., to the left), and the at least one sensor unit (16) is located in front of the at least one nozzle (13) in the direction of movement. The direction of the nozzle (13) is shown obliquely relative to the ground surface.The direction of the nozzle (13) can also be directed straight downwards relative to the ground surface.

[0069] Figure 2 schematically shows steps (a) to (g) or optionally (h) coordinated by the computing and control unit (11). This corresponds to steps 3a to 3g or optionally 3h in the method according to the invention. In (a), the computing and control unit (11) is configured to cause the at least one sensor unit (16) to generate at least one image at time L and to receive this at least one image (in Figure 2, the image is shown as a rectangle with a plant). In (b), the computing and control unit (11) is configured to cause the at least one position-determining unit (14) to determine the position of the at least one nozzle (13) at time tj and to receive this position data (in Figure 2, this is schematically shown with a reduced version of the device (10) from Figure 1 and a small crosshair for acquiring the position data at time L).In (c), the computing and control unit (11) is configured to determine at least a part of a target object (PI) on the at least one image by image processing and to georeference this at least one target object (PI) using the position data of the at least one sensor unit (16) from time t1 (in Figure 2, this is again represented by the small crosshairs on the image) in order to determine the distance Sj from the at least one nozzle (13) to the at least part of the target object (PI) on the image from time 0. In (d), the computing and control unit (11) is configured to cause the at least one position-determining unit (14) to determine the position of the at least one nozzle (13) at time t2 and to receive this position data, wherein time t2 is a later time than t1 (in Figure 2, this is schematically represented analogously to step (b), but at time f1).In (e), the computing and control unit (11) is configured to determine the distance S2 from the at least one nozzle (13) to the at least one part of the target object based on the position data of the at least one nozzle (13) from time t2 and the position data of the at least one georeferenced part of the target object on the image from time t1. In (f), the computing and control unit (11) is configured to determine the speed v2 and direction r2 of the at least one nozzle (13) from time t2 at least partially based on the position data of the at least one nozzle (13) from time t1 and time t2 (Figure 2 schematically shows a smaller version of the device (10), but the position of the at least one nozzle (13) is relevant).In (g), the computing and control unit (11) is configured to (g) plan the activation time of the spraying process for the at least one nozzle (13) onto at least a part of the target object (PI) at least partially based on the distance S2 from the at least one nozzle (13) to the at least a part of the target object (PI) at time t2 and the speed v2 and direction r2 of the at least one nozzle (13) at time t2 (Figure 2 schematically shows a plan for this process). In optional step (h), the computing and control unit (11) is configured to activate the at least one nozzle (13) at least partially based on the planned activation time of the spraying process and to apply the liquid (F) to at least a part of the target object (PI).

[0070] Figure 3 schematically shows the steps (a) to (g) or optionally (h) or (a ), (b), (c'), (d), (e'), (f ) and (g') and optionally (h') coordinated by the computing and control unit (11) in three consecutive iterative cycles i-1, i and i+1. This corresponds to steps 3a to 3g or optionally 3h or (3a ), (3b), (3c'), (3d), (3e'), (3f ) and (3g') and optionally (3h') in the method according to the invention. In iteration cycle i-1, steps (a, / ) to (gz-y) or optionally (h z-y) is performed. The signal strength of the GNSS receiver measured in steps (b / ) and (d,. / ) (at the beginning of the execution of the steps) has reached the predefined threshold level, so that the position determination can be performed with the GNSS receiver. In iteration cycle i, the signal strength of the GNSS receiver measured in step (bi) (and optionally also in step (di)) does not reach the predefined threshold level. Therefore, in addition to these steps, steps (a ,). (c' ; ), (e' ; ), (f ,). (g ) and optionally (h',j). In iteration cycle i+1, the signal strength of the GNSS receiver measured in step (b,+y) and (dt+i) reaches the threshold level again. Therefore, steps (a,+y) to (gt+i) and optionally (h !+ 7) was carried out.

Claims

Patent claims 1. Device (10) for applying a liquid (F) to at least part of a target object (PI) in an agricultural field (LF), comprising a computing and control unit (11), at least one storage container (12) for receiving the liquid (F), at least one nozzle (13), Means (15) for conveying the liquid (F) from the at least one storage container (12) in the direction of the at least one nozzle (13), at least one sensor unit (16) which can be aligned towards the agricultural field (F) towards a target object (PI) and is configured to generate at least one image, at least one position-determining unit (14), wherein the computing and control unit (11) is configured (a) to cause the at least one sensor unit (16) to generate at least one image at time 0 and to receive this at least one image, wherein the computing and control unit (11) is configured (b) to cause the at least one position-determining unit (14) to determine the position of the at least one sensor unit (16) at time t1 and to receive this position data, wherein the computing and control unit (11) is configured,(c) to determine at least a part of a target object (PI) - if present - on the at least one image by image processing and to georeference at least a part of this target object (PI) using the position data of the at least one sensor unit (16) from time t1 in order to determine the distance Sj from the at least one nozzle (13) to the at least a part of the target object (PI) on the image from time t1, wherein the computing and control unit (11) is configured, (d) to cause the at least one position determination unit (14) to determine the position of the at least one nozzle (13) at time t2 and to receive this position data, wherein the time f1 is a later time than t1, wherein the computing and control unit (11) is configured,(e) to determine the distance S2 from the at least one nozzle (13) to the at least one part of the target object based on the position data of the at least one nozzle (13) from time t2 and the position data of the at least one georeferenced part of the target object on the image from time t1, wherein the computing and control unit (11) is configured, (f) to determine the speed v2 and direction r2 of the at least one nozzle (13) from time t2 at least partially based on the position data of the at least one nozzle (13) from time t1 and time t2, wherein the computing and control unit (11) is configured,(g) to plan the activation time of the spraying process for the at least one nozzle (13) on at least a part of the target object (PI) at least partially based on the distance S2 from the at least one nozzle (13) to the at least a part of the target object (PI) at time t2 and the speed v2 and direction r2 of the at least one nozzle (13) at time t2.

2. Device (10) according to one of the preceding claims, wherein the computing and control unit (11) is configured, (h) the at least one nozzle (13) at least partly on the basis of the planned activation time of the spraying process and to apply the liquid (F) to at least part of the target object (PI).

3. Device (10) according to one of the preceding claims, wherein the at least one sensor unit (16) comprises at least one camera and preferably the camera axis of the at least one camera can be aligned substantially perpendicular to the ground.

4. Device (10) according to one of the preceding claims, wherein the computing and control unit (11) is configured in step (c) to carry out the determination of the at least part of a target object (PI) on the image using machine learning.

5. Device (10) according to claim 4, wherein the computing and control unit (11) is configured in step (c) to determine the distance Sj from the at least one nozzle (13) to the at least part of the target object (PI) on the image from time L in such a way that at least one pixel coordinate of the at least part of the target object is converted into position data on the agricultural field (LF).

6. Device (10) according to claim 5, wherein the following formulas are used to calculate the position data of the at least one pixel coordinate of the target object (PI) on the agricultural field (LF): (1) (2) where in formula (1) S y for the number of pixels per unit height on the camera's image sensor (vertical resolution per unit height), y for the vertical resolution of the camera, H for the distance from the camera lens to the ground and a y stands for the vertical aperture angle of the camera, where in formula (2) & stands for the number of pixels per unit width on the image sensor of the camera (horizontal resolution per unit width), x for the horizontal resolution of the camera, H for the distance from the camera lens to the ground and a> for the horizontal aperture angle of the camera, where starting from the pixel coordinate of the at least one image via the respective reciprocal of S yand .S calculates position data of the at least one pixel coordinate of at least one part of the target object (PI) on the agricultural field (LF).

7. Device (10) according to one of the preceding claims, wherein the computing and control unit (11) is configured to carry out the steps mentioned in points (a) to (f) and the calculation of the respective parameters S ti„ Vt, rt iteratively ( ; ) and the activation time of the spraying process for the at least one nozzle (13) on at least a part of the target object (PI) in the step mentioned in item (g) is adjusted as needed by each iterative step.

8. Device (10) according to one of the preceding claims, wherein the at least one position determination unit (14) comprises at least one GNSS (“Global Navigation Satellite System”) receiver, preferably at least two GNSS receivers and even more preferably the at least one GNSS receiver comprises a Moving Baseline Differential GPS installation.

9. Device (10) according to one of claims 7 and 8, wherein at least two position determining units (14) are preferably selected from the group of GNSS (Global Navigation Satellite System) receiver technology, position determining units based on visual odometry technology, radar, trilateration, inertial navigation systems, magnetometers and radio navigation systems and preferably one of these position determining units comprises a GNSS (“Global Navigation Satellite System”) receiver and the second position determining unit comprises a visual odometry technology and / or radar.

10. Device (10) according to claim 9, wherein the computing and control unit (11) is configured to check, before and / or during a position data determination with the GNSS receiver (14), whether the signal strength of the GNSS receiver reaches or exceeds a predefined threshold level and - if the predefined threshold level is not reached - to determine the position determination from the position determination unit which comprises a visual odometry technology.

11. Device (10) according to claim 10, wherein in the position data determination based on the visual odometry technology, the computing and control unit (11) is configured to cause the at least one sensor unit (13) at the time Ai.i and t i:ito generate at least one recording in each case and the time interval between and tt,i is selected so short that the recordings overlap (a ), wherein the computing and control unit (11) is configured at time tt,2- and deviating from the steps described in bullet points (c) and (e) to (g) - in (c', e' and f) at least partly on the basis of these recordings at least one of the data selected from the group of distance S it2 from the at least one nozzle (13) to the at least one part of the target object (PI) at time 4.2, the speed v iti the at least one nozzle (13) from time t i:1to determine the direction rt,i of the at least one nozzle (13) from time tt,i and wherein the computing and control unit is configured (g' ) to plan the activation time of the spraying process for the at least one nozzle onto at least a part of the target object at least partially on the basis of the distance St,2 from the at least one nozzle to the at least a part of the target object from time 4 and the speed Vt,i and direction rt,i of the at least one nozzle from time ti,i.

12. Device (10) according to one of the preceding claims, wherein the device (10) comprises at least one further sensor unit (16b) which can be aligned towards a target object (PI) in the direction of the agricultural field (LF), wherein the computing and control unit (11) is configured to cause the at least one sensor unit (16b) to capture a 3D point cloud at time 4 and to receive this 3D point cloud.

13. Device 10) according to one of the preceding claims, wherein the computing and control unit (11) is configured to at least partially take into account the latency of the at least one nozzle (13) and the flight time of the liquid (F) from the at least one nozzle (13) until it strikes at least part of the target object (PI) for determining the activation time of the spraying process for the at least one nozzle (13) onto at least part of a target object (PI).

14. Device (10) according to one of the preceding claims, the device is part of an agricultural machine, a robot or a drone, or is connectable thereto.

15. A method (100) for applying a liquid (F) to at least part of a target object (PI) in an agricultural field (LF), comprising steps (a) to (c): in step (1) moving a device for applying a liquid (F) to at least part of a target object (PI) in an agricultural field (LF), in step (2) conveying a liquid (F) from at least one storage container (12) towards at least one nozzle (13) during the movement, in step (3a) causing at least one sensor unit (16) to generate at least one image at time 0 and to forward this at least one image to the computing and control unit (11), in step (3b) causing at least one position-determining unit to determine the position of the at least one sensor unit (16) at time t1 and to forward this position data to the computing and control unit (11),in step (3c) determining at least a part of a target object - if present - on the at least one image by image processing and georeferencing the at least one part of the target object using the position data of the at least one sensor unit (16) from time 0 in order to determine the distance Sj from the at least one nozzle (13) to the at least one part of the target object on the image from time 0, in step (3d) causing the at least one position determination unit to determine the position of the at least one nozzle (13) at time t and to forward this position data to the computing and control unit (11), wherein the time t is a later time than t i,in step (3e) determining the distance S2 from the at least one nozzle (13) to the at least one part of the target object (PI) based on the position data of the at least one nozzle from the time t1 and the position data of the at least one georeferenced part of the target object on the recording from the time t2, in step (3f) determining the speed V2 and the direction 1'2 of the at least one nozzle (13) from the time t2 at least partially based on the position data of the at least one nozzle (13) from the time t1 and the time t2, in step (3g) planning the activation time of the spraying process for the at least one nozzle (13) on at least one part of a target object (PI) at least partially based on the distance S2 from the at least one nozzle (13) to the at least one part of the target object (PI) from the time t1 and the speed v2 and direction 1'2 of the at least one nozzle (13) from the time t2.

16. The method (100) of claim 15, further comprising in step (3h) activating the at least one nozzle (13) at least partially based on the planned activation time of the spraying process and applying the liquid to at least a portion of a target object (PI).

Images