Optimized segmentation and / or classification of plant objects

Abstract

The invention relates to a computer-implemented method and intelligent spraying system (100) for the optimized segmentation and / or classification of a plant object on underlying soil for the purpose of treatment with a spraying agent by means of an intelligent spraying system.

Description

[0001] Description

[0002] title

[0003] OPTIMIZED SEGMENTATION AND / OR CLASSIFICATION OF PLANT OBJECTS

[0004] The invention relates to a method and intelligent spraying system for the optimized segmentation and / or classification of a plant object on a field subsoil for spray treatment by means of an intelligent spraying system.

[0005] State of the art

[0006] An intelligent spraying system for applying herbicides is generally known. A camera system detects plants. In a second step, the image data is used to distinguish between crops and weeds. Wherever the camera system detects weeds, appropriate nozzles are activated, thus treating or spraying the weeds with a herbicide. Achieving the cleanest possible separation or segmentation of the plant objects from the respective field subsoil is often challenging.

[0007] Foreign objects such as stones, straw, or even the nature of the soil can influence the quality and accuracy of detection. This is due to the countless variations of different soils and foreign objects on the field subsoil. Therefore, not all cases can be anticipated and intercepted.

[0008] This makes it difficult to ensure the most accurate detection possible. The invention is therefore based on the object of providing a method and / or intelligent spraying system for the optimized segmentation and / or classification of a plant object on a field substrate for spray treatment.

[0009] The problem is solved by a method for the optimized segmentation and / or classification of a plant object on a field substrate, in particular for herbicide treatment using an intelligent spraying system according to the features of patent claim 1. The problem is solved by an intelligent spraying system for the optimized segmentation and / or classification of a plant object on a field substrate for herbicide treatment according to the features of patent claim 9.

[0010] Disclosure of the invention

[0011] According to a first aspect, a computer-implemented method for optimized segmentation and / or classification of a plant object on a field substrate, in particular for treatment with a spraying agent using a spraying system, is provided. The method comprises the following steps:

[0012] - Providing at least one image scene which comprises at least one plant object and a field background and which is captured in particular by an image capture device;

[0013] - Pre-segmenting the image scene using an initially predetermined, dynamically variable NDVI threshold in order to identify the at least one plant object on the basis of an NDVI index and to remove it from the image scene, thereby minimizing in particular interference by the at least one plant object;

[0014] - Calculating a plurality of NDVI indices of the field background remaining in the image scene after pre-segmentation;

[0015] - determining at least one mean of the plurality of NDVI indices of the field background for the image scene;

[0016] - buffering or storing the at least one mean value in a mean value history;

[0017] - Determining the median value based on the history in order to suppress outliers in particular; - Dynamically updating the mean history by displacing each new mean from an image scene provided in a subsequent journal with the oldest mean in the mean history, whereby the median value changes dynamically from a current image scene to an image scene recorded in a subsequent journal; and

[0018] - Segmentation and / or classification of the at least one plant object using the dynamically changing median value as NDVI threshold.

[0019] It is understood that the steps according to the invention, as well as other optional steps, do not necessarily have to be performed in the order shown, but can also be performed in a different order. Furthermore, additional intermediate steps can be provided. The individual steps can also comprise one or more substeps without thereby departing from the scope of the method according to the invention.

[0020] The method according to the invention and the spraying system enable autonomous response to the various subsoils of a field. For this purpose, the field subsoil, or the background or the periphery around the at least one plant object, is dynamically analyzed in order to extract relevant features based on the NDVI indices in a field- and / or subsoil-specific manner. Since the field and / or subsoil properties can change during an application, the NDVI threshold is adapted to the respective current subsoil situation or subsoil condition. The spraying system can thus preferably make decisions based on the field-specific features and, for example, dynamically adapt the NDVI threshold(s) for extracting the at least one plant object.

[0021] The spraying system is configured to apply a spray to plants classified as weeds. A camera system preferably detects plants in the form of at least one image scene. In a further step, the image data is used to distinguish between crops and weeds. Where the camera system detects weeds, spray nozzles or sprayers of the spraying system are preferably activated, which treat or spray the weed in question with at least one spray.

[0022] The spray agent is in particular a spray liquid. The spray agent can comprise or be an agricultural preparation or plant protection product (PPP), in particular a plant protection product concentrate. The spray agent can therefore comprise a pesticide, such as a herbicide, fungicide or an insecticide. The spray agent can also comprise a fertilizer, in particular a fertilizer concentrate. The spray agent can here comprise a growth regulator. The spray agent can comprise a granular spray agent which has been mixed with a carrier liquid. The spray liquid can, for example, be in the form of a liquid, suspension, emulsion, solution or a combination thereof. The spray liquid is preferably in the form of a plant protection product diluted with water or a fertilizer diluted with water. The spray liquid can therefore be, for example, a spray mixture.

[0023] The system preferably independently generates the information required for optimal segmentation and / or classification of at least one plant object in the image scene. Based on this information, the system can preferably make adjustments to the segmentation or the underlying NDVI threshold, thus ensuring stable, background-dependent detection.

[0024] The SmartSpraying System preferentially distinguishes between plant objects and the subsoil using the NDVI index. Plants have a high index, while other objects such as straw, stones, and soil have a low index. As has been shown, soil can also have a high NDVI index depending on its composition and / or texture. For example, soil or a subsoil with a high humus content has a very high NDVI index.

[0025] To determine the baseline level of the NDVI index in the respective image scene, it is preferable to remove all larger plant objects. These would distort the segmentation result or the NDVI index determination and increase the NDVI index of the entire image scene. Since healthy plants have a very high NDVI index, it is preferable to perform an initial segmentation with a fixed and high NDVI threshold.

[0026] According to one embodiment, the NDVI indices each indicate the difference between red light and NIR light in the image scene, and each preferably assumes values ​​between -1 and +1.

[0027] The NDVI (Normalized Difference Vegetation Index) is primarily an index used to assess vegetation activity and condition from satellite or airborne remote sensing data. It is used in agriculture, environmental science, and other disciplines that aim to monitor plant growth or vegetation density. The NDVI is calculated by taking the difference between the near-infrared (NIR) and red light (RED) of an image and dividing them by their sum. NDVI values ​​range from -1 to 1: values ​​close to 1 indicate dense vegetation. Values ​​around 0 indicate bare or sparsely vegetated areas. Negative values ​​can indicate bodies of water.

[0028] An NDVI threshold preferably refers to a specified NDVI value or range of values ​​used to classify or identify specific vegetation states or types. By setting thresholds, specific land cover types, such as dense vegetation, bare patches, or water bodies, can be filtered out and / or highlighted from NDVI data. An example of the use of NDVI thresholds is: An NDVI value of > 0.6 could be interpreted as dense vegetation. A value between 0.2 and 0.6 could be considered sparse or mixed vegetation. A value < 0.2 could be considered bare land or an urban area. It is important to emphasize that such thresholds are not universal and can be adapted depending on the application, region, and specific study objectives.It is preferable to define thresholds based on local knowledge and / or by combining NDVI data with other data sources. According to one embodiment, the initially predetermined, dynamically variable NDVI threshold indicates an NDVI index determined for a healthy plant object to enable an initial separation of the plant object and the field subsoil.

[0029] According to one embodiment, additional sensors and / or image capture devices are used to increase the accuracy of the NDVI index and the resulting plant detection. In principle, various optical sensors suitable for detecting the NDVI index or value can be used.

[0030] According to one embodiment, the mean value history additionally takes environmental conditions, in particular lighting conditions, soil moisture, and / or weather data, into account to optimize detection accuracy. In principle, other factors can also be included.

[0031] According to one embodiment, image processing and NDVI index calculation are performed in real time or near real time to enable immediate adjustment of the spraying system. This ensures optimal herbicide application at all times.

[0032] According to one embodiment, the spray system comprises a machine learning model that optimizes the performance of the spray system over time by continuously learning and dynamically adjusting the median value.

[0033] According to one embodiment, the NDVI indices of the field background remaining in the image scene after pre-segmentation are calculated pixel by pixel or for a pixel grid of a predetermined size. In particular, to save runtime, a pixel grid is preferably generated. Each pixel preferably has a distance of X pixels from its neighboring pixel, where X can be set as desired. In addition, a preferably generated plant mask is excluded. Thus, there is a pixel grid at every Xth pixel in an image scene, with no points or pixel grids preferably only being present on plant objects. Subsequently, the NDVI index is preferably determined on each of these pixel grids. Depending on the image size, this preferably results in several hundred NDVI indices of the field ground included in the image scene. The mean value is then calculated from these NDVI indices.Thus, an average NDVI index of the field background is determined for each pixel grid in an image scene.

[0034] According to a second aspect, a spraying system for the optimized segmentation and / or classification of a plant object on a field substrate is provided, which comprises an image capture device and an evaluation and / or computing device designed to carry out the following steps:

[0035] - Providing at least one image scene which comprises at least one plant object and a field background and which is in particular captured by the image capture device;

[0036] - Pre-segmenting the image scene using an initially predetermined, dynamically variable NDVI threshold in order to identify the at least one plant object on the basis of an NDVI index and to remove it from the image scene, thereby minimizing in particular interference by the at least one plant object;

[0037] - Calculating a plurality of NDVI indices of the field background remaining in the image scene after pre-segmentation;

[0038] - determining at least one mean of the plurality of NDVI indices of the field background for the image scene;

[0039] - buffering or storing the at least one mean value in a mean value history;

[0040] - Determining the median value based on history in order to suppress outliers in particular;

[0041] - Dynamic updating of the mean history, whereby each new mean from an image scene provided in a next journal displaces an oldest mean in the mean history, whereby the median value changes dynamically from a current image scene to an image scene recorded in a next journal; and

[0042] - Segmentation and / or classification of the at least one plant object using the dynamically changing median value as an NDVI threshold or using the dynamically changing median value as an NDVI threshold as a basis for a segmentation and / or classification of the at least one plant object in a subsequent journal.

[0043] The statements made for the procedure apply accordingly to the system. It is understood that linguistic modifications of procedurally formulated features can be reformulated for the system according to common linguistic practice, without such formulations having to be explicitly listed here.

[0044] The invention also claims a computer program with program code for executing at least parts of the method according to the invention in one of its embodiments when the computer program is executed on a computer. In other words, the invention provides a computer program (product) comprising instructions that, when executed by a computer, cause the computer to execute the method / steps of the method according to the invention in one of its embodiments.

[0045] According to the invention, a computer-readable data carrier with program code of a computer program is also proposed for executing at least parts of the method according to the invention in one of its embodiments when the computer program is executed on a computer. In other words, the invention relates to a computer-readable (storage) medium comprising instructions which, when executed by a computer, cause the computer to execute the method / steps of the method according to the invention in one of its embodiments.

[0046] The described designs and further training courses can be combined as desired.

[0047] Further possible embodiments, further developments and implementations of the invention also include combinations of features of the invention described previously or below with regard to the embodiments that are not explicitly mentioned.

[0048] Short description of the drawings

[0049] The accompanying drawings are intended to provide a further understanding of embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain principles and concepts of the invention.

[0050] Other embodiments and many of the aforementioned advantages will become apparent upon review of the drawings. The elements illustrated in the drawings are not necessarily drawn to scale.

[0051] They show:

[0052] Fig. 1 is a schematic flow diagram of an embodiment of the present method;

[0053] Fig. 2 a schematic representation of different substrates of a

[0054] field with different NDVI indices; and

[0055] Fig. 3 is a schematic representation of a dynamic NDVI index

[0056] Adjustment.

[0057] In the figures of the drawings, the same reference symbols designate the same or functionally equivalent elements, parts or components, unless otherwise stated.

[0058] Figure 1 shows a schematic flow diagram of a computer-implemented method for the optimized segmentation and / or classification of a plant object on a field subsoil for intelligent herbicide treatment using an intelligent spraying system. In any embodiment, the method can be carried out at least partially by a system 100, which for this purpose can comprise several components not shown in detail, for example one or more provision devices and / or at least one evaluation and computing device. It is understood that the provision device can be designed jointly with the evaluation and computing device or can be different from it. Furthermore, the system 100 can comprise a storage device and / or an output device and / or a display device and / or an input device.

[0059] According to the invention, the computer-implemented method comprises at least the following steps:

[0060] In a step S1, at least one image scene is provided which has at least one plant object and a field background and which is captured in particular by an image capture device.

[0061] In a step S2, the image scene is pre-segmented using an initially predetermined, dynamically variable NDVI threshold in order to identify the at least one plant object on the basis of an NDVI index and to remove it from the image scene, thereby minimizing in particular interference by the at least one plant object.

[0062] In a step S3, a plurality of NDVI indices of the field background remaining in the image scene after the pre-segmentation are calculated.

[0063] In a step S4, at least one mean value of the plurality of NDVI indices of the field background for the image scene is determined.

[0064] In a step S5, the at least one mean value is temporarily stored or saved in a mean value history.

[0065] In step S6, the median value is determined based on the history, in order to suppress outliers in particular. In step S7, the mean value history is dynamically updated. Each new mean value from an image scene provided in a subsequent journal displaces the oldest mean value in the mean value history, causing the median value to change dynamically from a current image scene to an image scene recorded in a subsequent journal.

[0066] In a step S8, segmentation and / or classification of the at least one plant object takes place using the dynamically changing median value as an NDVI threshold or using the dynamically changing median value as an NDVI threshold as a basis for a segmentation and / or classification of the at least one plant object in a subsequent journal.

[0067] Fig. 2 shows two example images captured by a camera sensor. On the left side, the NDVI threshold is set to 0. The left side shows field segmentations on the ground. Here, the NDVI threshold of 0 is too close to the ground NDVI index of -0.08. On the right side, however, it can be seen that good segmentation is achievable. The distance between the NDVI threshold and the ground NDVI index of -0.28 is large enough. Even very small objects, such as stones or other objects, can be segmented. If the NDVI threshold is set to 0.15, significantly better segmentation can be achieved on the left side. On the right side, however, many of the small objects cannot be segmented.

[0068] For this to happen, the NDVI threshold would have to be set significantly lower. As can be seen from this example, there is no single, good NDVI threshold for every field situation. A uniformly low NDVI threshold risks many missegmentations on the ground, and a uniformly high threshold means that the segmentation algorithm misses many small objects that can be easily separated from the background. The algorithm should therefore be able to decide independently which NDVI threshold is the optimum for segmentation. Fig. 3 shows a schematic representation of dynamic NDVI index adjustment. The mean NDVI indices are stored in a history. In the example shown, this is a history of the last 9 images or image scenes, with the oldest value on the left and the most recent value on the right. To suppress outliers, a median is determined from the history, which in the example shown is 0.31.This median NDVI value is preferably the final result. Each new value from a current image or image scene included in a subsequent journal preferably displaces the oldest value from the history. Thus, the median value changes dynamically from image scene to image scene.

[0069] This median NDVI value can be used as input for further algorithms, such as segmentation. This dynamic adaptation of the plant detection and classification algorithms to the field characteristics ensures significantly more stable and precise detection.

[0070] Journal is preferably determined by the image capture rate (frames per minute, fpm) of the image capture device.

Claims

Claims 1 . Computer-implemented method for the optimized segmentation and / or classification of a plant object on a field substrate, in particular for needs-based spray treatment by means of a spraying system, the method comprising the following steps: Providing (S1) at least one image scene comprising at least one plant object and a field background; Pre-segmenting (S2) the image scene using an initially predetermined, dynamically variable NDVI threshold in order to identify the at least one plant object based on an NDVI index and to remove it from the image scene; Calculating (S3) a plurality of NDVI indices of the field background remaining in the image scene after pre-segmentation; Determining (S4) at least one mean value of the plurality of NDVI indices of the field background for the image scene; Buffering or storing (S5) the at least one mean value in a mean value history; Determine (S6) the median value based on history; Dynamically updating (S7) the mean history by displacing each new mean from an image scene provided in a next journal with the oldest mean in the mean history, whereby the median value changes dynamically from a current image scene to an image scene captured in a next journal; and Segmentation and / or classification of the at least one plant object (S8) using the dynamically changing median value as NDVI threshold.

2. The method according to claim 1, wherein the NDVI indices each indicate the difference between red light and NIR light in the image scene.

3. The method according to claim 1 or 2, wherein the initially predetermined, dynamically variable NDVI threshold indicates an NDVI index determined for a healthy plant object to enable an initial separation of the plant object and the field subsoil.

4. Method according to one of the preceding claims, wherein additional sensors or camera systems are used to increase the accuracy of the NDVI index and the resulting plant detection.

5. Method according to one of the preceding claims, wherein the mean value history additionally takes environmental conditions into account.

6. Method according to one of the preceding claims, wherein the image processing and NDVI index calculation are carried out in real time or near real time to enable immediate adjustment of the spraying system.

7. The method of any preceding claim, wherein the spray system comprises a machine learning model that optimizes the performance of the spray system over time by continuously learning and dynamically adjusting the median value.

8. Method according to one of the preceding claims, wherein the NDVI indices of the field background remaining in the image scene after the pre-segmentation are calculated pixel by pixel or in each case for a pixel grid of a predetermined size.

9. A spraying system (100) for the optimized segmentation and / or classification of a plant object on a field substrate for needs-based spray treatment, comprising an image capture device and an evaluation and / or computing device designed to carry out the following steps: Providing at least one image scene comprising at least one plant object and a field background; Pre-segmenting the image scene using an initially predetermined, dynamically variable NDVI threshold in order to identify the at least one plant object based on an NDVI index and to remove it from the image scene; Calculating a plurality of NDVI indices of the field background remaining in the image scene after pre-segmentation; Determining at least one mean of the plurality of NDVI indices of the field background for the image scene; Buffering or storing the at least one mean value in a mean value history; Determine the median value based on history; Dynamically updating the mean history by displacing each new mean from an image scene provided in a next journal with the oldest mean in the mean history, whereby the median value changes dynamically from a current image scene to an image scene captured in a next journal; and Using the dynamically changing median value as an NDVI threshold as a basis for segmentation and / or classification of the at least one plant object in a subsequent journal.

10. A computer program comprising program code for carrying out at least parts of a method according to any one of claims 1 to 8 when the computer program is executed on a computer.

11. A computer-readable data carrier with program code of a computer program for executing at least parts of a method according to one of claims 1 to 8 when the computer program is executed on a computer.

Images