Image-based identification of structural features of crop plants

Abstract

The invention relates to a computer-implemented method (10) for the image-based identification of structural features of crop plants in an agricultural field. The invention also relates to a device (100) for applying a liquid to at least one part of an agricultural field on the basis of the identified information relating to the structural features of the crop plants. The invention further relates to a computer-readable storage medium on which the computer-implemented method is encoded.

Description

[0001] Image-based identification of structural features of cultivated plants

[0002] The present invention relates to a computer-implemented method for image-based recognition of structural features of crops in an agricultural field. The invention also relates to a device for applying a liquid to at least a portion of an agricultural field based on the recognized information regarding the structural features of the crops. A further aspect of the invention relates to a computer-readable storage medium on which the computer-implemented method is encoded.

[0003] Growing crops in rows on agricultural fields is a common practice that offers several advantages. By planting crops in rows, farmers can optimize land use, make irrigation and fertilizer application more efficient, and maximize yield per unit area. Furthermore, row arrangement facilitates mechanized cultivation and harvesting. Selecting the optimal row spacing and planting density depends on various factors, including the specific requirements of the crops being grown, soil conditions, climate, and available resources. Farmers must also consider the impact of row cropping on soil health, companion plant management, and pest control.

[0004] Information about the arrangement of crops in rows can help improve the detection and decision-making process for specific treatments. Precise identification of plant rows enables the implementation of new application possibilities. For example, a stronger herbicide (such as one that is phytotoxic to the crop) can be applied between the rows than within them. Similarly, the decision can be made to selectively treat crops accidentally growing outside the rows, as they are typically considered a nuisance.

[0005] Precision agriculture enables the identification of objects such as plants, pests, or companion plants in digitally captured images, allowing for diagnosis and subsequent action (e.g., treatment). One example of such a task is ultra-high-precision spraying (UHPS), where a tractor drives across a planted field, capturing successive images of the agricultural field with an attached sprayer. The challenges of real-time image recognition and information processing in the field, for example, to distinguish between crops and companion plants, lie in managing large amounts of information, processing this data rapidly, and simultaneously executing actions such as partial spraying.Further challenges include adapting to varying light conditions, recognizing plants at different growth stages, and precisely locating and directing the spraying action in real time. Robust algorithms are required to handle variable environmental conditions and unforeseen obstacles.

[0006] The present invention addresses these problems.

[0007] A first object of the present invention is a computer-implemented method for image-based recognition of structural features of crops in an agricultural field, comprising the steps of:

[0008] -(a) Production of at least one photograph of the agricultural field,

[0009] -(b) Generation of at least one segmentation of the at least one image acquisition, -(c) Generation of a first probability value P(A) for the presence of a structural feature of cultivated plants for the at least one segmentation of the at least one image acquisition,

[0010] -(d) Identification and object localization of a first crop plant on the at least one image recording, wherein a second probability value P(B) is generated which corresponds to the confidence value of the object identification and object localization of the first crop plant,

[0011] -(e) Determination of whether the first identified and object-located crop plant, of which at least one segmentation of at least one image capture is at least partially covered,

[0012] -(f) Using the probability values ​​P(A) and P(B) to determine an updated probability value P(A) aktueii_i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image capture, which is at least partially covered by the at least one segmentation.

[0013] Another object of the present invention is a device for applying a liquid to at least a part of an agricultural field, comprising a calculating and control unit, at least one reservoir for receiving the liquid, and at least one nozzle.

[0014] Means for conveying the liquid from the at least one reservoir towards the at least one nozzle, at least one image acquisition unit which can be oriented towards the agricultural field and is configured to produce at least one image, wherein the computing and control unit is configured to (a) cause the at least one image acquisition unit to produce at least one image of the agricultural field and to receive this at least one image, wherein the computing and control unit is configured to (b) produce at least one segmentation of the at least one image, wherein the computing and control unit is configured to (c) for the at least one segmentation of the at least one image, generate a first probability value P(A) for the presence of a structural feature of crop plants, wherein the computing and control unit is configured to(d) to identify and object-locate a first crop plant in the at least one image capture, generating a second probability value P(B) corresponding to the confidence value of the object identification and object-locatement of the first crop plant, wherein the computing and control unit is configured, (e) to determine whether the identified and object-located first crop plant is at least partially encompassed by the at least one segmentation of the at least one image capture, wherein the computing and control unit is configured, (f) the probability values ​​P(A) and P(B) to determine an updated probability value P(A, a(ktueii_i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image acquisition which is at least partially covered by the at least one segmentation, wherein the computing and control unit is configured, (g) the at least one nozzle at least partially on the basis of the information on the updated probability value P(A a to activate ktueii i) and to apply the liquid to at least part of the agricultural field.

[0015] Another object of the present invention is a non-volatile, computer-readable storage medium on which software instructions are stored which, when executed by a processor of a computer system, cause the computer system to perform the following steps:

[0016] -(a) Receiving at least one photograph of an agricultural field,

[0017] -(b) Generation of at least one segmentation of the at least one image acquisition,

[0018] -(c) Generation of a first probability value P(A) for the presence of a structural feature of cultivated plants for the at least one segmentation of the at least one image acquisition,

[0019] -(d) Identification and object localization of a first crop plant on the at least one image recording, wherein a second probability value P(B) is generated which corresponds to the confidence value of the object identification and object localization of the first crop plant,

[0020] -(e) Determination of whether the identified and object-located first crop plant, of which at least one segmentation of at least one image is at least partially covered,

[0021] -(f) Using the probability values ​​P(A) and P(B) to determine an updated probability value P(A) aktueii_i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image capture that is encompassed by the at least one segmentation.

[0022] The invention is explained in more detail below, without distinguishing between the subject matter of the invention (method, device, storage medium). Rather, the following explanations are intended to apply analogously to all subject matter of the invention, regardless of the context in which they are made (method, device, storage medium).

[0023] If the present description or the claims mention steps in a sequence, this does not necessarily mean that the invention is limited to that sequence. Rather, it is conceivable that the steps could also be carried out in a different sequence or even in parallel; unless one step builds upon another, which makes it essential that the building step be carried out subsequently (which will be clear in the specific case). The sequences mentioned thus represent preferred embodiments of the invention.

[0024] The present invention relates to a computer-implemented method for image-based recognition of structural features of crops in an agricultural field.

[0025] The term "structural characteristics" of cultivated plants describes, for example, circular and / or row cultivation. Circular cultivation, also known as circular or circular field irrigation, is a specific cultivation method in which fields are laid out in circular patterns and irrigated from a central point. This method is often used in arid regions where irrigation must be efficient and targeted. Row cultivation refers to the cultivation method in which plants are arranged in parallel rows in the field. This cultivation method allows for efficient field management, the use of agricultural machinery, and targeted irrigation and fertilization.

[0026] The term "cultivated plant" refers to a plant that is intentionally grown by humans as a useful or ornamental plant. The term "companion plants" (often also referred to as weeds) refers to plants of the spontaneous accompanying vegetation (segetal flora) in cultivated plant stands, grasslands, or gardens that are not intentionally cultivated and develop, for example, from the seed bank in the soil or through wind dispersal.

[0027] The term "agricultural field" refers to a spatially definable area of ​​the earth's surface that is used for agricultural purposes, by planting crops, supplying them with nutrients if necessary, and harvesting them.

[0028] In the computer-implemented method, at least one image of the agricultural field is acquired in a first step (a). In one example, the image is acquired using a camera. The camera is attached to an agricultural device, such as a sprayer, and pointed towards the ground. The camera used according to the invention can include an image acquisition sensor and optical elements. The image acquisition sensor is a device for capturing two-dimensional images of light electrically. These are typically semiconductor-based image acquisition sensors, such as CCD (CCD = charge-coupled device) or CMOS (CMOS = complementary metal-oxide-semiconductor) sensors. The optical elements (lenses, apertures, and the like) serve to create the sharpest possible image of an object on the image acquisition sensor.The image format of at least one image can be rectangular or square, for example. Common rectangular image formats are 3:2, 4:3, or 16:9.

[0029] In a further step (b) of the inventive method, at least one segmentation of the at least one image is generated. In one example, the segmentation is a predefined segmentation such as circles or vertical stripes. The generated segmentation on the at least one image has a specific area. If the predefined segmentation is a vertical segmentation, then the segments have a certain segmentation width. The vertical segments can, for example, extend over the entire height of the image, i.e., from the top to the bottom edge. Preferably, the generated segments have substantially the same dimensions. The inventive method aims to divide at least one image into sections such that at least one of these sections shows the structural features of the crop plants (hereinafter referred to as "segments").The segmentation width of the at least one segmentation in step (b) is chosen, for example, to be less than or equal to, and preferably substantially equal to, the width of a spray cone from a nozzle of an agricultural spraying device. The width of a spray cone is also frequently referred to as the "spray width" or "spray cone width." These terms describe the diameter of the cone that is formed when spraying pesticides or fertilizers. This is advantageous because the information regarding the presence of a structural feature of a crop plant at a segment can be used directly to decide whether or not to open the corresponding spray nozzle.

[0030] In a further step (c) of the inventive procedure, a first (a priori) probability value P(A) for the presence of a structural feature of cultivated plants is generated for at least one segmentation of at least one image acquisition. In this step, the probabilities P(A) for each generated segment are initialized. This value is usually constant for all generated segments and is, for example, set to 0.5. Alternatively, for example, if N segments are present (N = number of generated segments), the probability value P(A) for each generated segment can be calculated using the formula P(A) = 1 / N, or other values ​​from the interval (0, 1) (but not equal to zero) can be chosen, since these values ​​adapt quickly (as described below).

[0031] In one example, the probability values ​​range from 0 to 1, where a value of 0 means that an event is impossible, while a value of 1 indicates that an event is certain to occur. Values ​​between 0 and 1 represent the probability with which a particular event can occur, with higher values ​​indicating a greater probability of the event occurring.

[0032] In a further step (d) of the method according to the invention, the identification and object localization of a first crop plant on the at least one image is carried out. Image object identification and image object localization are important tasks in the field of machine vision and computer vision. Image object identification refers to the ability of a system to recognize and classify a specific object in an image. Various methods are used for this purpose, including convolutional neural networks (CNNs) based on deep learning. These networks can extract features from the image and learn to recognize different objects by being trained on extensive training data. Other methods for image object identification include the use of feature descriptors such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features) in combination with machine learning algorithms.

[0033] Image object localization refers to a system's ability to determine the precise position of a detected object within an image. Various techniques can be used for this purpose, including bounding box regression, which allows for the precise localization of the bounding boxes around detected objects, and the application of region-based convolutional neural networks (R-CNNs) and their variants, such as Fast R-CNN and Faster R-CNN. These techniques enable the accurate localization of objects within the image and the drawing of bounding boxes and / or polygons around them. In addition to these methods, techniques such as region-based fully convolutional networks (R-FCN), single-shot multibox detectors (SSDs), and You Only Look Once (YOLO) can also be used for simultaneous object detection and localization.Semantic segmentation methods can also be used, and possible object boundaries can be determined from the resulting masks in a post-processing step.

[0034] In step (d), object identification and localization also generate a second probability value P(B), which corresponds to the confidence score for the object identification and localization of the first crop. This confidence score is typically used in machine learning-based methods, particularly deep learning models for object identification and localization. For example, when using CNNs for object identification and localization, a confidence score is usually generated that indicates how certain the network is that it has correctly identified the detected object and how accurate the object's localization is. R-CNNs and their variants, such as Fast R-CNN and Faster R-CNN, use Region Proposal Networks (RPNs) to identify regions in the image that potentially contain objects.These methods also generate a confidence score for the identification and localization of the detected objects. Methods such as Single Shot Multibox Detector (SSD) and You Only Look Once (YOLO) are designed to efficiently identify and locate objects in a single image. These methods also generate confidence scores for the detected objects to indicate the reliability of the identification and localization.

[0035] In all these methods, the confidence level is used to quantify the reliability and accuracy of object identification and localization. This allows for an assessment of the reliability of the detected objects and the implementation of appropriate measures based on the confidence level, such as displaying warnings or initiating follow-up actions. Step d) is performed for all plants (i.e., companion and crop plants) in the image at least once, because at least the crop plant(s) must be identified. In another embodiment, the companion plants can also be identified and localized. This information can be used to map the companion plants and, if necessary, to treat them specifically with a spray application in step h).

[0036] Alternatively, P(B) could also be a measure of the method's object identification and localization performance, such as the F-measure value of the class "crop" on the test set after complete training of the underlying object identification and localization model. It is also conceivable that P(B) is a combination of the object identification and localization confidence score and the performance measure.

[0037] In a further step (e) of the method according to the invention, it is determined whether the identified and object-located first crop plant is at least partially encompassed by the at least one segmentation of the at least one image acquisition. To determine whether an identified and localized first crop plant is located in a predefined segment of the image acquisition or at least overlaps with this segment, the IoU (Intersection over Union) method can be used, for example. The starting point of the method is the identified and localized first crop plant, which is represented by a boundary box or a polygon that encloses the position and extent of the first crop plant in the image acquisition. To determine whether the boundary box or polygon of the identified and localized first crop plant at least partially overlaps with a predefined segment of the image acquisition, the IoU value is calculated.One could also determine which segmentations a calculated bounding box of a crop plant overlaps with by comparing the horizontal boundaries of the box and the segmentations. Alternative calculation methods are known to experts.

[0038] In a further step (f) of the method according to the invention, the probability values ​​P(A) and P(B) are used to determine an updated probability value P(A). a k tU eii i) is used for the presence of a structural feature of cultivated plants for the area of ​​the at least one image acquisition that is at least partially encompassed by the at least one segmentation. In an example, the said area of ​​the at least one image acquisition refers to the entirety of those segments that at least partially encompass the identified and object-located first cultivated plant.

[0039] The calculation in step (f) can be performed, for example, using various methods employed for updating probabilities and estimates. Some of these methods include: Bayes' rule, maximum likelihood estimation (MLE), expectation maximization (EM), particle filtering, and Kalman filtering.

[0040] The calculation of P(A a k tU eii i) in step (f) of the method according to the invention is preferably carried out using Bayes' rule. The calculation of P(A a ktueii i) is carried out for each of those segments on the at least one image that at least partially include the identified and object-located first (or further) crop plant(s) as follows according to formula (1):

[0041] P(E) = P(B) x P(A) + P(F) x (1 - P(A))

[0042] (1)

[0043] Where P(E) describes the probability whether a segment that at least partially comprises the identified and object-located first (or further) crop plant(s) belongs to a structural feature of crop plants or not. P(A) and P(B) are defined as explained in more detail above in steps c) and d). P(F) can preferably be the false-positive rate of the class "crop plant", as determined by applying the fully trained object identification and object-localization algorithm to a representative test set. In a second step, P(A) a k tU eii i) calculated according to formula (2):

[0044] P(A ak actually _1) = (P(B) XP(A)) / P(E) (2)

[0045] In the next iterative step, P(A) is replaced by the updated P(A) value; specifically, P(A) = P(A) is set. a ktueii_i).

[0046] In a further embodiment of the method according to the invention, in step (g) at least one segmentation is identified on the at least one image image whose probability value P(A) has not been updated after steps (d), (e), and (f) have been carried out for all identified and object-located crop plants on this at least one image image. For this at least one segmentation, the associated probability value P(A) is reduced, preferably by multiplication by a factor less than 1 to generate a P(A). a k tUeii i) for this at least one segment. That is, step (g) is preferably only carried out once all crop plants on the at least one image have been identified and object-located. That is, steps (d), (e), and (f) are each carried out for all crop plants located on the at least first tape recording, as explained in more detail in the next section. Only then is step (g) carried out. Care is taken to ensure that P(A) does not become zero, for example, by applying a lower bound for this value of 0.001 or similar, so that a subsequent update does not change the value of P(A). a k tU eii i) can still effectively change.

[0047] In a further embodiment of the method according to the invention, in step (d') a second crop plant is identified and object-located on at least one image, whereby a third probability value P(C) is generated, which corresponds to the confidence value of the object identification of the second crop plant. In step (e) it is determined whether the identified and object-located second crop plant is at least partially encompassed by the at least one segmentation of the at least one image. In step (f) the probability values ​​P(A) are a k tU eii i) and P(C) to determine a further updated probability value P(A) a k tU eii 2) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image acquisition which is at least partially covered by the at least one segmentation. P(A aktueii 2) then replaces - as described above - P(A a ktueii_i). Preferably, the calculations in step (d), step (e) and step (f) are performed for each identified and object-localized crop plant on the at least one image.

[0048] The information in step (f) can be output to a user, e.g., through a suitable output device such as a monitor, printer, mobile phone, speaker, projector, etc.

[0049] In another embodiment, the identified crop plants are recognized again in subsequent image acquisitions, e.g. by object tracking methods, and it is ensured that P(A aThe parameter `ktueii_i` only receives a new value if a previously unrecognized object has been identified. Object recognition can also be simplified by tracking boundary boxes or determining the IoU value of two objects, provided the pixel shift of the new image has been determined.

[0050] In a further embodiment of the computer-implemented method, in step (h) at least one nozzle of an agricultural device (e.g., an agricultural sprayer) is controlled, at least partially, based on information determined in step (f). The information determined in step (f) contains details about the presence of structural features of crops and thus enables its direct use for controlling spray nozzles of agricultural devices. For example, a herbicide can be used in segments where a high probability of the absence of a structural feature of crops has been determined. Conversely, no herbicide is sprayed in segments where there is a high probability of the presence of a structural feature of crops. Steps (a) to (h) of the computer-implemented method are preferably performed iteratively.In one example, at least one additional image of the agricultural field is generated in (a). It is of secondary importance whether the additional image overlaps with the first image or not. The procedure is carried out with the new image data, and the data calculated using the first image is updated. In another example, at least one additional image of the agricultural field is generated in (a), which at least partially overlaps with the at least one image from step (a). For example, in step (a), continuous (and preferably at least partially overlapping) image captures of the agricultural field can be generated by an agricultural device moving across it (e.g., a sprayer).To transfer the segments generated in the first image acquisition (step b) to new image acquisitions (step (bi)), various techniques and steps can be used. For example, feature extraction can be used to extract features from the new image acquisition that allow the position and orientation of the camera (which is, for example, located on a moving agricultural implement) as well as the environment to be determined. This can be done using feature descriptors such as SIFT, SURF, or CNNs. The extracted features are then used to perform a registration between the new image acquisition and the first image acquisition. Known registration techniques can be used to determine the transformation (e.g., translation, rotation, scaling) required to transfer the new images to the perspective of the first image acquisition.Based on the transformation determined by the registration, the defined segments of the first image acquisition generated in step (b) are transformed accordingly to adapt them to the perspective of the new image acquisition. This may include adjusting the position, size, and orientation of the segments. After transferring the segments to the new image acquisition, a validation can be performed to ensure that the transfer was correct. If necessary, adjustments can be made to ensure that the segments are correctly positioned and oriented in the new image acquisition. In step (c0), P(A) is then replaced by the current P(A) value, as described in more detail above. a ktueii_i) - replaced. Steps (di) to (fi) or (dh) to (fi) and, if applicable, (gi) and / or (h ,) are then performed with the new image acquisition information.

[0051] A further aspect of the invention relates to a device for applying a liquid to at least a part of an agricultural field, comprising a computing and control unit, at least one reservoir for receiving the liquid, at least one nozzle, means for conveying the liquid from the at least one reservoir towards the at least one nozzle, and at least one image acquisition unit that can be oriented towards the agricultural field (e.g., the ground) and is configured to produce at least one image. The computing and control unit is configured (a) to cause the at least one image acquisition unit to produce at least one image of the agricultural field and to receive this at least one image. The computing and control unit is configured (b) to produce at least one segmentation of the at least one image.The computing and control unit is further configured (c) to generate a first probability value P(A) for the presence of a structural feature of crop plants for the at least one segmentation of the at least one image acquisition. The computing and control unit is further configured (d) to identify and object-locate a first crop plant in the at least one image acquisition, generating a second probability value P(B) corresponding to the confidence value of the object identification and object-locatement of the first crop plant. The computing and control unit is further configured (e) to determine whether the identified and object-located first crop plant is at least partially encompassed by the at least one segmentation of the at least one image acquisition. The computing and control unit is further configured (f) to use the probability values ​​P(A) and P(B) to determine an updated probability value P(A).a ktueii_i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image acquisition that is at least partially covered by the at least one segmentation. The computing and control unit is further configured (g) to use the at least one nozzle at least partially based on the information on the updated probability value P(A a to activate ktueii_i) and apply the liquid to at least part of the agricultural field.

[0052] The liquid can be water, an aqueous solution, or a suspension. The aqueous solution or suspension can contain one or more nutrients and / or one or more plant protection products and / or one or more seed treatment agents.

[0053] The term "nutrients" refers to those inorganic and organic compounds from which plants can obtain the elements that make up their bodies. These elements themselves are also often referred to as nutrients. They are mostly simple inorganic compounds such as nitrate (NO₃⁻) and phosphate (PO₄²⁻). 3 - 1 and potassium (K + In addition to the core elements of organic matter (C, O, H, N, and P), the following are essential for life: K, S, Ca, Mg, Mo, Cu, Zn, Fe, B, Mn, Cl (in higher plants), Co, and Ni. Various compounds can be present for the individual nutrients; for example, nitrogen can be supplied as nitrate, ammonium, or amino acids.

[0054] The term "plant protection product" refers to a substance used to protect plants or plant products from harmful organisms or to prevent their effects, to destroy unwanted plants or plant parts, to inhibit or prevent unwanted plant growth, and / or to influence plant life processes in a way other than by providing nutrients (e.g., growth regulators). Examples of plant protection products include herbicides, fungicides, and other pesticides (e.g., insecticides).

[0055] Growth regulators are used, for example, to increase lodging resistance in cereals by shortening stem length (intermodium shorteners), to improve the rooting of cuttings, to reduce plant height through stunting in horticulture, or to prevent potato germination. Growth regulators can be, for example, phytohormones or their synthetic analogs.

[0056] The term "processing unit" refers to components of a computer or processor used to perform arithmetic and logical operations, as well as to control the execution of instructions and operations. In modern computers, the processing unit and the control unit are often closely integrated and together form the so-called "CPU" ("Central Processing Unit"). These two units work together to enable the processing and execution of instructions within a computer. It is also possible that at least certain functions are performed via cloud-based computing.

[0057] The liquid is contained in a reservoir before application. Several reservoirs may be present. Several (different) liquids may be applied. The liquid is applied to at least a portion of the agricultural field via one or more nozzles. In a preferred embodiment, several nozzles are used, preferably encompassed by a bar or strip (hereinafter referred to as a "spray bar"). In another preferred embodiment, the device according to the invention comprises a plurality of nozzles. The term "plural" preferably means more than ten. The nozzles are preferably arranged such that each nozzle sprays liquid in an area with a maximum lateral extent of less than 20 cm. The plurality of nozzles can, for example, be arranged side by side along a spray bar that extends transversely (e.g.,The camera extends at an angle of 90° to the direction of movement of the agricultural device. Each nozzle can be assigned at least one camera. For application, the liquid is conveyed from the at least one reservoir towards the at least one nozzle by means of conveying means. A pump, for example, can be used to convey the liquid. The at least one image acquisition unit, which can be oriented towards the agricultural field (e.g., the ground) and is configured to produce at least one image, is, for example, a camera as described above. The camera axis of the at least one camera, as a component of the device according to the invention, can preferably be oriented substantially perpendicular (vertically) to the ground (i.e., the at least one camera points "downwards" or "straight ahead" with respect to the ground).Preferably, the at least one camera is located (in the direction of travel) in front of the at least one (spray) nozzle. This distance is, for example, between 30 cm and 100 cm.

[0058] The device according to the invention can also be equipped with a positioning unit. Such a system can, for example, include a receiver of a satellite navigation system (GNSS, Global Navigation Satellite System), commonly referred to as a GPS receiver. The Global Positioning System (abbreviated GPS), officially NAVSTAR GPS, is an example of a global satellite navigation system for determining position; other examples are GLONASS, Galileo, and Beidou. The object recognition technology described above can be used in conjunction with the data from a positioning unit to precisely locate and map structural features of crops.

[0059] Another subject of the present disclosure is a non-volatile, computer-readable storage medium on which software instructions are stored which, when executed by a processor of a computer system, cause the computer system to perform the following steps:

[0060] -(a) Receiving at least one photograph of an agricultural field,

[0061] -(b) Production of at least one segmentation of the at least one image acquisition,

[0062] -(c) Generation of a first (a priori) probability value P(A) for the presence of a structural feature of cultivated plants for the at least one segmentation of the at least one image acquisition,

[0063] -(d) Identification and object localization of a first crop plant on the at least one image recording, wherein a second probability value P(B) is generated which corresponds to the confidence value of the object identification and object localization of the first crop plant,

[0064] -(e) Determination of whether the identified and object-located first crop plant, of which at least one segmentation of at least one image is at least partially covered,

[0065] -(f) Using the probability values ​​P(A) and P(B) to determine an updated probability value P(A) a k tU eii i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image capture encompassed by the at least one segmentation.

[0066] Further embodiments of the present invention are:

[0067] 1. Computer-implemented method (10) for image-based recognition of structural features of crops in an agricultural field comprising the steps:

[0068] -(a) Production of at least one image of the agricultural field,

[0069] -(b) Generation of at least one segmentation of the at least one image acquisition,

[0070] -(c) Generation of a first probability value P(A) for the presence of a structural feature of cultivated plants for the at least one segmentation of the at least one image acquisition,

[0071] -(d) Identification and object localization of a first crop plant on the at least one image recording, wherein a second probability value P(B) is generated which corresponds to the confidence value of the object identification and object localization of the first crop plant,

[0072] -(e) Determination of whether the identified and object-located first crop plant, of which at least one segmentation of at least one image is at least partially covered,

[0073] -(f) Using the probability values ​​P(A) and P(B) to determine an updated probability value P(A) a ktueii_i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image capture, which is at least partially covered by the at least one segmentation.

[0074] 2. Computer-implemented method (10) according to embodiment 1, comprising the steps:

[0075] -(d ) Identification and object localization of a second crop plant on the at least one image recording, wherein a third probability value P(C) is generated which corresponds to the confidence value of the object identification of the second crop plant,

[0076] -(e') Determination of whether the identified and object-located second crop plant is at least partially covered by the at least one segmentation of the at least one image acquisition,

[0077] -(f ) Use of the probability values ​​P(A a ktueii_i) and P(C) to determine a further updated probability value P(A) a ktueii 2) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image capture which is at least partially covered by the at least one segmentation.

[0078] 3. Computer-implemented method (10) according to one of the previous embodiments, wherein in step (a) continuous image recordings of the agricultural field can be generated by an agricultural device moving thereon.

[0079] 4. Computer-implemented method (10) according to one of the previous embodiments, wherein in step (b) a vertical segmentation of the at least one image acquisition is generated and the structural feature of the crop plants is a crop plant row.

[0080] 5. Computer-implemented method (10) according to one of the preceding embodiments, wherein the segmentation width of the at least one segmentation in step (b) is selected such that it is less than or equal to the width of a spray cone of a nozzle of an agricultural spraying device.

[0081] 6. Computer-implemented method (10) according to one of the preceding embodiments, wherein the identification and object localization of a crop plant on the at least one image acquisition in step (d) is carried out by generating a bounding box and / or a polygon.

[0082] 7. Computer-implemented method (10) according to one of the preceding embodiments, wherein the calculations in step (d), step (e) and step (f) are performed for each identified and object-located crop plant on the at least one image recording.

[0083] 8. Computer-implemented method (10) according to one of the previous embodiments, wherein the identification and object localization of the crop plant in step (d) is carried out using machine learning, preferably using a neural network.

[0084] 9. Computer-implemented method (10) according to one of the previous embodiments, wherein in step (f) Bayes' rule is used to update the probability value P(A) each time.

[0085] 10. Computer-implemented method (10) according to one of the preceding embodiments, comprising in step: -(g) identifying at least one segmentation on the at least one image whose probability value P(A) was not updated after performing steps (d), (e) and (f) for all identified and object-located crop plants on this at least one image and lowering this probability value P(A), preferably by multiplying by a factor less than 1.

[0086] 11. Computer-implemented method (10) according to one of the preceding embodiments, comprising in step:

[0087] -(h) Control of at least one nozzle of an agricultural device, preferably an agricultural spraying device, at least partially based on information determined in step (f).

[0088] 12. Computer-implemented method (10) according to one of the previous embodiments, wherein steps (a) to (h) are performed iteratively.

[0089] 13. Device (100) for applying a liquid (F) to at least a part of an agricultural field (LF) comprising a calculating and control unit (110), at least one storage container (120) for receiving the liquid (F), at least one nozzle (130),

[0090] Means (150) for conveying the liquid (F) from the at least one reservoir (120) towards the at least one nozzle (130), at least one image acquisition unit (160) which can be oriented towards the agricultural field and is configured to produce at least one image, wherein the computing and control unit (110) is configured (a) to cause the at least one image acquisition unit (160) to produce at least one image of the agricultural field and to receive this at least one image, wherein the computing and control unit (110) is configured, (b) to produce at least one segmentation of the at least one image, wherein the computing and control unit (110) is configured, (c) to generate a first probability value P(A) for the presence of a structural feature of crop plants for the at least one segmentation of the at least one image.wherein the computing and control unit (110) is configured, (d) to identify and object-locate a first crop plant on the at least one image acquisition, generating a second probability value P(B) corresponding to the confidence value of the object identification and object-locatement of the first crop plant, wherein the computing and control unit (110) is configured, (e) to determine whether the identified and object-located first crop plant is at least partially encompassed by the at least one segmentation of the at least one image acquisition, wherein the computing and control unit (110) is configured, (f) the probability values ​​P(A) and P(B) to determine an updated probability value P(A, aktueii_i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image acquisition which is at least partially covered by the at least one segmentation, wherein the computing and control unit (110) is configured, (g) the at least one nozzle (130) at least partially on the basis of the information on the updated probability value P(A a to activate ktueii_i) and apply the liquid (F) to at least part of the agricultural field (LF).

[0091] 14. Device according to embodiment 14, wherein the computing and control unit (110) is configured to (f2) identify at least one segmentation on the at least one image recording whose probability value P(A) has not been updated after performing steps (d), (e) and (f) for all identified and object-located crop plants on this at least one image recording and to lower this probability value P(A), preferably by multiplying by a factor less than 1.

[0092] 15. Non-volatile, computer-readable storage medium (200) on which software instructions are stored which, when executed by a processor of a computer system (300), cause the computer system (300) to perform the following steps:

[0093] -(a) Receiving at least one image of an agricultural field, -(b) Generating at least one segmentation of the at least one image, -(c) Generating a first (a priori) probability value P(A) for the presence of a structural feature of crop plants for the at least one segmentation of the at least one image,

[0094] -(d) Identification and object localization of a first crop plant on the at least one image recording, wherein a second probability value P(B) is generated which corresponds to the confidence value of the object identification and object localization of the first crop plant,

[0095] -(e) Determination of whether the first identified and object-located crop plant, of which at least one segmentation of at least one image capture is at least partially covered,

[0096] -(f) Using the probability values ​​P(A) and P(B) to determine an updated probability value P(A) a ktueii_i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image acquisition that is encompassed by the at least one segmentation.

[0097] The invention is explained in more detail below with reference to figures and examples, without intending to limit the invention to the features and combinations of features mentioned in the figures and examples.

[0098] They show:

[0099] Figure 1 schematically shows the computer-implemented method 10 for image-based recognition of structural features of crops in an agricultural field, comprising the steps: (a) generation of at least one image of the agricultural field, (b) generation of at least one segmentation of the at least one image, (c) generation of a first probability value P(A) for the presence of a structural feature of crops for the at least one segmentation of the at least one image, (d) identification and object localization of a first crop in the at least one image, generating a second probability value P(B) corresponding to the confidence value of the object identification and object localization of the first crop, (e) determination,whether the identified and object-located first crop plant is at least partially encompassed by the at least one segmentation of the at least one image acquisition and (f) use of the probability values ​​P(A) and P(B) to determine an updated probability value P(A, a k tUeii i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image acquisition that is at least partially encompassed by the at least one segmentation. Figure 1 schematically shows the optional (dashed box) steps g) and h). Step g) concerns the identification of at least one segmentation on the at least one image acquisition whose probability value P(A) was not updated after performing steps (d), (e), and (f) for all identified and object-located cultivated plants on this at least one image acquisition. In this step, the probability value P(A) is reduced, preferably by multiplication by a factor less than 1. Step h) concerns an optional control of at least one nozzle of an agricultural device, preferably an agricultural spraying device, at least partially based on information determined in step (f).Alternatively, the information obtained in step f) can be communicated to a user via an output device. Optional steps d), e), and f) involve performing steps d), e), and f) but for a second crop plant in the at least one image. These steps are performed for all crop plants in the at least one image. Step d) is preferably performed for all plants (i.e., companion and crop plants) in the at least one image because the crop plants must first be identified. The figure at the top right indicates that the procedure a) to f), a) to g), and a) to h) can be performed iteratively (a) - f), a0 - g), and a0 - h), respectively.

[0100] Figure 2 schematically illustrates how the at least one image is analyzed by the computer-implemented method 10. In step a), an image of the agricultural field is generated. The image shows six crop plants (three plants in each of two rows). In step b), segments are created (areas between the vertical lines in the image). In step c), each segment is assigned an a priori probability value P(A). Figure 2 shows, for example, the value 0.5 for the first segment. All subsequent segments receive the same a priori probability value. In step d), the first crop plant (top left of the image) is identified and object-located (shown with a target in Figure 2 d). The order of identification and object-localization of the crop plants in the first image can be random.For the first identified and object-located crop plant, a second probability value P(B) is generated (e.g., 0.85, see Figure 2), which corresponds to the confidence value of the object identification and object-localization of the first crop plant. In step e), it is determined whether the first identified and object-located crop plant is at least partially encompassed by at least one segment of the at least one image acquisition. In Figure 2, the third and fourth segments from the left are marked darker because the first identified and object-located crop plant is at least partially encompassed by these segments. In step f), the probability values ​​P(A) and P(B) are used to calculate an updated probability value P(A). a k tUeii i) to determine the presence of a structural feature of cultivated plants for the third and fourth segments from the left. Figure 2 also shows in d) that after carrying out steps a) to f), the procedure can also be carried out for further cultivated plants in the same image (optional steps d'), e') and f). Furthermore, in d) of Figure 2, the last segment (first segment from the right) is shown as an example, which, after carrying out steps (d), (e) and (f), was not updated for all identified and object-located cultivated plants in this image (because it does not include any cultivated plants). In optional step g), the a priori probability value is therefore reduced from the original 0.5. In Figure 2, the factor 0.95 is used as an example, resulting in a P(A) for this segment. a ktueii_i) can be determined to be 0.475. In comparison, the P(A) a k tUeii i) After applying Bayes' rule, the values ​​for the third and fourth segments from the left with the specified values ​​for P(A) and P(B) each increased to 0.85 after calculation for the first identified and object-oriented crop plant. Figure 3 schematically shows the device 100 for applying a liquid F to at least a part (e.g., as shown, a plant) of an agricultural field LF. The device 100 comprises a computing and control unit 110, at least one reservoir 120 for receiving the liquid F, at least one nozzle 130, means 150 for conveying the liquid F from the at least one reservoir 120 towards the at least one nozzle 130, and at least one image acquisition unit 160, which can be aligned towards the agricultural field F.In Figure 1, the device 100 moves from right to left, and the image acquisition unit 160 is located in the direction of movement in front of the at least one nozzle 130. The direction of the nozzle 130 is shown obliquely relative to the ground surface. However, the direction of the nozzle 130 can also be directed straight downwards relative to the ground surface.

[0101] Figure 4 schematically shows the functionality of the device 100 for applying a liquid F to at least a part of an agricultural field LF. The computing and control unit 110 of the device 100 is configured – in step (a) – to cause the at least one image acquisition unit 160 to generate at least one image of the agricultural field and to receive this at least one image. The computing and control unit 110 is configured – in step (b) – to generate at least one segmentation of the at least one image. The computing and control unit 110 is further configured – in step (c) – to generate a first probability value P(A) for the presence of a structural feature of crop plants from the at least one segmentation of the at least one image.The computing and control unit 110 is further configured – in step (d) – to identify and localize a first crop plant in the at least one image, generating a second probability value P(B) corresponding to the confidence value of the object identification and localization of the first crop plant. The computing and control unit 110 is further configured – in step (e) – to determine whether the identified and localized first crop plant is at least partially encompassed by the at least one segmentation of the at least one image. The computing and control unit 110 is further configured – in step (f) – to determine the probability values ​​P(A) and P(B) for an updated probability value P(A). a k tUeii i) to use for the presence of a structural feature of cultivated plants for the area of ​​the at least one image acquisition, which is at least partially covered by the at least one segmentation. The computing and control unit 110 is further configured in step (g) - the at least one nozzle 130 at least partially based on the information on the updated probability value P(A ato activate ktueii_i) and apply the liquid F to at least a portion of the agricultural field LF. Figure 4 schematically shows the optional (dashed box) step f2). The computing and control unit is further configured in step f2) to identify at least one segment on the at least one image whose probability value P(A) has not been updated after performing steps (d), (e), and (f) for all identified and object-located crops on this at least one image. Optional steps d'), e'), and f) involve performing steps d), e), and f) but for a second crop on the at least one image. These steps are performed for all crops on the at least one image. Step d) is preferably performed for all plants (i.e.,The identification of companion and cultivated plants is carried out on the minimum image capture because the cultivated plants must first be identified. It is indicated in the upper right that steps a) to g) or a) to g) including step f2) can be performed iteratively (a) - g,) or a) - g) including step f2,).

[0102] For example, when the device travels 100 times straight ahead across an agricultural field, each segment has different probabilities as to whether it can be assigned to a structural feature of a crop or not. In a row crop, for instance, the different probabilities of the individual (vertical) segments would indicate whether they belong to a crop row or not. If the track shifts or the device 100 turns, the probabilities adjust automatically, because in such a case (from the camera's perspective) the crop row shifts, and other segments are subsequently weighted more or less highly with regard to their probability, depending on how the crop row has shifted.

[0103] Figure 5 shows, on the left, an image of a sugar beet field in which the sugar beets (crop KP) and the companion plants BP have been identified and their locations determined. The sugar beets were planted in rows. On the right side of Figure 5, the original image is shown with border boxes (frames around plants or plant parts) superimposed. At the top of the image, the identified rows are marked: The letter "H" indicates that a row of crop plants is highly likely to be found in these segments. The letter "U" indicates uncertainty: the probability that these segments belong to a crop plant row is neither high nor low. This could be the case, for example, if either a) some crop plants were discovered in this segment not long ago and are no longer present, or b) the first crop plants of a row were discovered.The letter "G" indicates that the probability of these segments overlapping with a crop row is low. Segments with the same or very similar probabilities were grouped together, which explains the different widths of the identified rows. Based on this segment classification, different spray applications can be derived. For example, in the case of companion plant treatment in an agricultural field, herbicide application can be decided by using different herbicides or different herbicide dosages for rows with different probabilities. For instance, a more effective herbicide or a higher dosage could be sprayed for the row with probability "G" than for the row with probability "U".

Claims

PATENT CLAIMS 1. Computer-implemented method (10) for image-based recognition of structural features of crops in an agricultural field comprising the steps: -(a) Production of at least one photograph of the agricultural field, -(b) Generation of at least one segmentation of the at least one image acquisition, -(c) Generation of a first probability value P(A) for the presence of a structural feature of cultivated plants for the at least one segmentation of the at least one image acquisition, -(d) Identification and object localization of a first crop plant on the at least one image recording, wherein a second probability value P(B) is generated which corresponds to the confidence value of the object identification and object localization of the first crop plant, -(e) Determining whether the identified and object-located first crop plant is at least partially covered by the at least one segmentation of the at least one image acquisition, -(f) Using the probability values ​​P(A) and P(B) to determine an updated probability value P(A) a ktueii_i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image capture, which is at least partially covered by the at least one segmentation.

2. Computer-implemented method (10) according to claim 1, comprising the steps: -(d ) Identification and object localization of a second crop plant on the at least one image recording, wherein a third probability value P(C) is generated which corresponds to the confidence value of the object identification of the second crop plant, -(e') Determination whether the identified and object-located second crop plant is at least partially covered by the at least one segmentation of the at least one image acquisition, -(f ) Use of the probability values ​​P(A a ktueii_i) and P(C) to determine a further updated probability value P(A) a k tU eii 2) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image recording which is at least partially covered by the at least one segmentation.

3. Computer-implemented method (10) according to one of the preceding claims, wherein in step (a) continuous image recordings of the agricultural field can be generated by an agricultural device moving thereon.

4. Computer-implemented method (10) according to one of the preceding claims, wherein in step (b) a vertical segmentation of the at least one image acquisition is generated and the structural feature of the crop plants is a crop plant row.

5. Computer-implemented method (10) according to one of the preceding claims, wherein the segmentation width of the at least one segmentation in step (b) is selected such that it is less than or equal to the width of a spray cone of a nozzle of an agricultural spraying device.

6. Computer-implemented method (10) according to one of the preceding claims, wherein the identification and object localization of a crop plant on the at least one Image acquisition in step (d) is carried out by generating a bounding box and / or a polygon.

7. Computer-implemented method (10) according to one of the preceding claims, wherein the calculations in step (d), step (e) and step (f) are performed for each identified and object-located crop plant on the at least one image recording.

8. Computer-implemented method (10) according to any of the preceding claims, wherein the identification and object localization of the crop plant in step (d) is carried out using machine learning, preferably using a neural network.

9. Computer-implemented method (10) according to one of the preceding claims, wherein in step (f) Bayes' rule is used to update the probability value P(A) each time.

10. Computer-implemented method (10) according to any one of the preceding claims, comprising in step: -(g) Identification of at least one segmentation on the at least one image whose probability value P(A) was not updated after performing steps (d) , (e) and (f) for all identified and object-located crop plants on this at least one image and reduction of this probability value P(A), preferably by multiplication by a factor less than 1.

11. Computer-implemented method (10) according to any one of the preceding claims, comprising in step: -(h) Control of at least one nozzle of an agricultural device, preferably an agricultural spraying device, at least partially based on information determined in step (f).

12. Computer-implemented method (10) according to one of the preceding claims, wherein steps (a) to (h) are performed iteratively.

13. Device (100) for applying a liquid (F) to at least a part of an agricultural field (LF) comprising a calculating and control unit (110), at least one storage container (120) for receiving the liquid (F), at least one nozzle (130), Means (150) for conveying the liquid (F) from the at least one reservoir (120) towards the at least one nozzle (130), at least one image acquisition unit (160) which can be oriented towards the agricultural field and is configured to produce at least one image, wherein the computing and control unit (110) is configured (a) to cause the at least one image acquisition unit (160) to produce at least one image of the agricultural field and to receive this at least one image, wherein the computing and control unit (110) is configured, (b) to produce at least one segmentation of the at least one image, wherein the computing and control unit (110) is configured, (c) to generate a first probability value P(A) for the presence of a structural feature of crop plants for the at least one segmentation of the at least one image. wherein the computing and control unit (110) is configured, (d) to identify and object-locate a first crop plant on the at least one image acquisition, generating a second probability value P(B) corresponding to the confidence value of the object identification and object-locatement of the first crop plant, wherein the computing and control unit (110) is configured, (e) to determine whether the identified and object-located first crop plant is at least partially encompassed by the at least one segmentation of the at least one image acquisition, wherein the computing and control unit (110) is configured, (f) the probability values ​​P(A) and P(B) to determine an updated probability value P(A aktueii_i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image acquisition which is at least partially covered by the at least one segmentation, wherein the computing and control unit (110) is configured, (g) the at least one nozzle (130) at least partially on the basis of the information on the updated probability value P(A a k tU eii i) to activate and to apply the liquid (F) to at least part of the agricultural field (LF).

14. Device according to claim 13, wherein the computing and control unit (110) is configured to (f2) identify at least one segmentation on the at least one image acquisition whose probability value P(A) has not been updated after performing steps (d), (e) and (f) for all identified and object-located crop plants on this at least one image acquisition and to lower this probability value P(A), preferably by multiplying by a factor less than 1.

15. Non-volatile, computer-readable storage medium (200) on which software instructions are stored which, when executed by a processor of a computer system (300), cause the computer system (300) to perform the following steps: -(a) Receiving at least one image of an agricultural field, -(b) Generating at least one segmentation of the at least one image, -(c) Generation of a first (a priori) probability value P(A) for the presence of a structural feature of cultivated plants for the at least one segmentation of the at least one image acquisition, -(d) Identification and object localization of a first crop plant on the at least one image recording, wherein a second probability value P(B) is generated which corresponds to the confidence value of the object identification and object localization of the first crop plant, -(e) Determining whether the identified and object-located first crop plant is at least partially covered by the at least one segmentation of the at least one image acquisition, -(f) Using the probability values ​​P(A) and P(B) to determine an updated probability value P(A) aktueii_i) for the presence of a structural feature of cultivated plants for the area of ​​the at least one image acquisition that is encompassed by the at least one segmentation.

Images