Rape seedling stage missing seedling detection method based on unmanned aerial vehicle and image processing

Abstract

The invention relates to the technical field of agricultural automation, and discloses a rape seedling stage missing seedling detection method based on an unmanned aerial vehicle and image processing, and the method comprises the following steps: S1, collecting field rape seedling videos for two consecutive days by the unmanned aerial vehicle; s2, identifying and counting seedlings in real time by adopting a YOLOv8 model and a ByteTrack algorithm; s3, comparing the number of seedlings in two consecutive days, and if the count in the next day is increased, repeating the counting work to prevent a non-seedling-emergence area from being misjudged as seedling missing; if the count is not increased, entering the next step; s4, the unmanned aerial vehicle collects seedling photo data again; s5, using a YOLOv8 model to identify seedling positions, and extracting coordinates; s6, using an improved DBSCAN algorithm to classify the coordinate points; s7, fitting a compound row straight line by a least square method, and judging whether a seedling missing area exceeding a distance threshold exists or not; s8, calculating the position where seedlings need to be supplemented with the seedlings; s9, a seedling supplementing instruction is sent to the equipment, and the process is completed. According to the method, the rape planting management efficiency is improved, the labor cost is reduced, and the crop yield and quality can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural automation, and in particular to a method for detecting missing rapeseed seedlings at the seedling stage based on unmanned aerial vehicles and image processing. Background Art

[0002] In agricultural planting, especially the planting of crops such as rapeseed, accurate detection of seedling shortages during the seedling stage is crucial to ensuring crop yields. Traditional seedling shortage detection relies on manual inspections, which is inefficient, time-consuming and labor-intensive, and easily affected by human errors, making it difficult to promote and apply on a large scale. In recent years, with the development of drone technology and image processing technology, automated seedling shortage detection has become possible. However, existing automatic detection methods still have problems such as insufficient accuracy and poor adaptability in complex farmland environments, especially when faced with irregular fields or non-linear working directions.

[0003] In addition, existing methods ignore the effective management and utilization of data, which is not conducive to subsequent optimization and maintenance. Many traditional methods are only applicable to specific types of farmland environments (such as regular fields and straight working directions) and are difficult to adapt to farmlands of different shapes and conditions. Factors such as lighting changes and background interference also affect the recognition accuracy of existing methods, especially for small targets. The lack of comprehensive analysis combined with actual agronomic operations leads to low reliability and practicality of the detection results.

[0004] In summary, although the existing technology has achieved a certain degree of automated seedling shortage detection, there are still many limitations. Therefore, there is an urgent need for a method that can efficiently and accurately detect rapeseed seedling shortages in complex environments, which has good adaptability and robustness and is suitable for different types of farmland environments. The present invention aims to solve the above-mentioned problems and provide a method for detecting rapeseed seedling shortages based on unmanned aerial vehicles and image processing technology, so as to improve the degree of automation, accuracy and scope of application of seedling shortage detection and promote the sustainable development of rapeseed cultivation. To this end, the present application now proposes a method for detecting rapeseed seedling shortages based on unmanned aerial vehicles and image processing. Summary of the invention

[0005] Technical Solution

[0006] To achieve the above object, the present invention provides the following technical solution: a method for detecting missing rapeseed seedlings in the seedling stage based on drone and image processing, comprising the following steps:

[0007] S1: Use a DJI drone Mavic 3T to fly along a predetermined path and record video data of rapeseed seedlings in the field for two consecutive days;

[0008] S2: The YOLOv8 model trained for rapeseed seedlings is used to identify video data in real time, and the ByteTrack tracking algorithm is used to count seedlings between consecutive frames;

[0009] S3: Compare the number of seedlings on the current day with that of the previous day. If the count increases, repeat the counting process; if the count does not increase, proceed to the next step;

[0010] S4: Use a DJI drone Mavic 3T to fly along a specified path and record photo data of rapeseed seedlings in the field;

[0011] S5: Use the trained YOLOv8 model to identify the location of rapeseed seedlings in the photo and extract coordinate information;

[0012] S6: Apply the improved DBSCAN clustering algorithm to classify the coordinate points;

[0013] S7: obtaining a crop row straight line by least squares fitting, and determining whether there is a seedling-deficient area exceeding a set distance threshold;

[0014] S8: After confirming the seedling-deficient area, the specific location where seedlings need to be supplemented is calculated based on the fitted crop row straight line;

[0015] S9: Send the location of the missing seedlings and the seedling replenishment instruction to the seedling replenishment device, and end the whole process.

[0016] Preferably, the specific steps of step S3 are: when the count of the day increases, repeat steps S1 to S2 until the count no longer increases.

[0017] Preferably, in step S4, when the drone flies along the specified path to collect image data, the heading overlap rate is set to 70%, the lateral overlap rate is set to 65%, and the flight altitude is set to 4 meters.

[0018] Preferably, the improved DBSCAN clustering algorithm in step S6 includes changing the traditional circular neighborhood search to a rectangular neighborhood search.

[0019] Preferably, the step S7 obtains the crop row straight line by fitting the least square method, and determines whether there is a seedling-deficient area exceeding the set seedling spacing threshold, and the specific steps are:

[0020] S701: fitting the clustered point coordinates by the least square method to generate crop row straight lines;

[0021] S702: When the distance between the coordinates of two rapeseed plants exceeds a set threshold of 230 pixels, the area is defined as a seedling-deficient area;

[0022] S703: Extracting the coordinates of the missing seedling points based on the fitted crop row straight line.

[0023] Preferably, the improved DBSCAN clustering algorithm comprises the following specific steps: searching a rectangular neighborhood with a width of 200 pixels and a length of 1000 pixels, centered on the coordinates of the rapeseed seedling.

[0024] Preferably, the seedling shortage detection and seedling replenishment system comprises:

[0025] Image acquisition module: obtains rapeseed seedling images of the field to be identified, the rapeseed seedling images contain rapeseed seedling features in several different time periods and different postures; the rapeseed seedling image data comes from images collected by the DJI drone Mavic3T.

[0026] Rapeseed coordinate detection module: input the obtained rapeseed seedling image into the rapeseed seedling detection model obtained by training the YOLOv8 model, perform the first stage of rapeseed seedling target recognition, and obtain coordinate information containing the rapeseed seedling;

[0027] The rapeseed seedling detection module also includes: a first training module, which is used to divide the rapeseed seedling image into training set data, validation set data and test set data; use a labeling tool to label the individual rapeseed seedling bounding boxes in the rapeseed seedling images of the training set data and the test set data; input the rapeseed seedling images in the training set data and the corresponding individual rapeseed seedling bounding boxes into the YOLOv8 model to obtain predicted rapeseed seedling bounding boxes; use the validation set data to perform performance verification on the YOLOv8 model, and when the YOLOv8 model meets the preset performance requirements, use the trained YOLOv8 model as a rapeseed seedling detection model;

[0028] Rapeseed missing seedling location module: The coordinate information of the rapeseed seedlings obtained in the first stage is input into the improved DBSCAN clustering algorithm. The improved DBSCAN clustering algorithm includes changing the traditional circular neighborhood search to a rectangular neighborhood search for rapeseed seedlings, and implementing rapeseed seedling crop row classification. Then, the clustered point coordinates are fitted by the least squares method to generate a crop row straight line. When the distance between the coordinates of two rapeseed plants exceeds the set threshold, the area is defined as a missing seedling area. The missing seedling point coordinates are extracted based on the fitted crop row straight line to complete the second stage of rapeseed seedling missing identification and obtain rapeseed seedling missing location information;

[0029] Rapeseed seedling supplementation module: the rapeseed seedling missing location information obtained in the second stage is transmitted to the seedling supplementation device, thus terminating the entire system.

[0030] Preferably, the system also has learning capability to obtain a better YOLOv8 recognition weight file in real time.

[0031] Preferably, a system is provided for storing image data collected by the drone, intermediate results generated during processing, and managing final results.

[0032] Compared with the prior art, the present invention provides a method for detecting missing rapeseed seedlings in the seedling stage based on drones and image processing, which has the following beneficial effects:

[0033] 1. The method for detecting missing seedlings in the rapeseed seedling stage based on drones and image processing, by using drones and image processing technology to detect missing seedlings and replenish seedlings in the rapeseed seedling stage, can not only efficiently and accurately identify and locate the missing seedling areas in the field, but also automatically calculate the specific positions of the required replenishment seedlings, thereby realizing intelligent replenishment seedlings. The present invention provides a complete set of solutions, from drone image acquisition to missing seedling detection to intelligent replenishment suggestions, aiming to improve agricultural production efficiency, reduce resource waste, ensure the healthy growth of crops, and ultimately realize intelligent and refined management of agricultural production.

[0034] 2. This method for detecting missing rapeseed seedlings in the seedling stage based on drones and image processing uses a pre-trained YOLOv8 model combined with the ByteTrack algorithm to perform real-time, high-precision identification and tracking of rapeseed seedlings, ensuring stable counting between consecutive frames, avoiding the subjectivity and inconsistency of manual detection, and greatly improving detection efficiency. At the same time, by analyzing the data collected by drones, the specific areas and quantities that need to be supplemented are clarified, reducing unnecessary supplementary operations, saving seed resources and labor costs, and achieving the goal of precision agriculture.

[0035] 3. The method for detecting missing seedlings in the rapeseed seedling stage based on drones and image processing, the improved DBSCAN clustering algorithm and the least squares method for fitting crop row straight lines enable the system to automatically determine the missing seedling area and calculate the reasonable seedling filling position, which is conducive to mechanized seedling filling and overcomes the problems of traditional manual transplanting and seedling filling, which is labor-intensive, time-consuming, costly and ineffective. In this way, it promotes the balanced growth and high and stable yield of rapeseed in different regions, adapts to the needs of modern agriculture, is suitable for large-scale farmland management, and is especially suitable for large-scale planting scenarios in modern farms, providing a scientific basis and technical support for agricultural production. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The present invention is a flowchart of the overall working process of a method for detecting missing rapeseed seedlings based on drone and image processing in an embodiment of the present invention.

[0037] Figure 2 This is an example diagram of seedling counting performed by YOLOv8 combined with the ByteTrack algorithm in an embodiment of the present invention.

[0038] Figure 3This is a diagram showing the effect of YOLOv8 extracting the coordinates of the center point of rapeseed seedlings in an embodiment of the present invention.

[0039] Figure 4 This is a diagram showing the clustering effect of the improved DBSCAN algorithm in an embodiment of the present invention.

[0040] Figure 5 This is a graph showing the clustering results of the improved DBSCAN algorithm in an embodiment of the present invention.

[0041] Figure 6 This is a diagram showing the effect of least squares method fitting crop rows in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Example 1

[0044] like Figure 1 As shown, a method for detecting missing rapeseed seedlings in the seedling stage based on drone and image processing includes the following steps:

[0045] S1: Use a DJI Mavic 3T drone to fly along a predetermined path and record video data of rapeseed seedlings in the field for two consecutive days.

[0046] S2: Input the acquired video data into the pre-trained YOLOv8 model to identify rapeseed seedlings in the video in real time, and combine the ByteTrack tracking algorithm to count the seedlings between consecutive frames (such as Figure 2 The figure shows an example of seedling counting using YOLOv8 combined with ByteTrack algorithm).

[0047] It should be noted that the YOLOv8 model has been specially trained to accurately identify rapeseed seedlings and effectively distinguish other background information, significantly improving recognition accuracy. At the same time, combined with the ByteTrack algorithm, stable tracking of the same seedling in the video stream is achieved to ensure the reliability of counting. The specific implementation method is: Figure 2 The direction of the yellow arrow in the middle is the direction of video movement. A virtual counting line is set in the video. Whenever the identification frame of the seedling crosses this line, a counting operation is automatically triggered, thereby accurately counting the number of seedlings.

[0048] S3: Compare the number of seedlings on the current day with that of the previous day. If the count increases, repeat steps S1 to S2 until the count stops increasing to prevent the area without seedlings from being misjudged as an area lacking seedlings, resulting in ineffective seedling replenishment. If the count does not increase, proceed to the next step.

[0049] S4: Use a DJI Mavic 3T drone to fly along a specified path and record photo data of rapeseed seedlings in the field, setting the heading overlap rate to 70%, the lateral overlap rate to 65%, and the flight altitude to about 4 meters.

[0050] It should be noted that the heading overlap rate is set to 70%, the lateral overlap rate is set to 65%, and the flight altitude is maintained at about 4 meters. Such parameter settings ensure that there is enough overlap between adjacent photos, which not only ensures the continuity and integrity of the image, but also effectively covers the entire monitoring area, ensuring that there will be no loss of seedling information due to blind spots or insufficient overlap.

[0051] S5: Input the photo data into the trained YOLOv8 model to identify the location of the rapeseed seedlings in the photo and extract the coordinate information. The yellow dots in the figure are the output coordinate information markers. Figure 3 The figure shows the effect of YOLOv8 in extracting the coordinates of the center point of rapeseed seedlings).

[0052] Example 2

[0053] This embodiment describes in detail the improved DBSCAN clustering algorithm in step S6 based on the above steps.

[0054] like Figure 4 As shown, the specific process is:

[0055] S601: A rectangular neighborhood search of the rapeseed seedling is performed with the coordinates of the rapeseed seedling as the center and a rectangle with a width of 200 pixels and a length of 1000 pixels;

[0056] S602: performing cluster analysis on the coordinate points in the expanded area to form a clustering result based on the distribution characteristics of rapeseed seedlings;

[0057] S603: Classification is performed based on the clustering results to prepare for the least square method of fitting crop row lines in Example 3.

[0058] It should be noted that the parameters of the improved DBSCAN algorithm are modified according to the agronomy of rapeseed planting. The row spacing of rapeseed planting is about 50 cm and the plant spacing is about 15 cm. In order to accurately classify the rows, in the experiment, we determined the modified DBSCAN algorithm, which takes the coordinates of the rapeseed seedling as the center point and then performs neighborhood search according to a rectangular area of ​​fixed size. The width of this rectangle is set to 200 pixels and the length is set to 1000 pixels to ensure that the relevant area around the seedling can be accurately and comprehensively covered. (e.g. Figure 5 The figure shows the result after clustering using the DBSCAN algorithm).

[0059] Example 3

[0060] This embodiment describes step S7 in detail based on the above steps.

[0061] S7: The crop row straight line is obtained by least squares fitting, and it is determined whether there is a seedling-deficient area exceeding the set spacing threshold (such as Figure 6 The figure shows the effect of least squares method fitting crop rows. The specific steps are:

[0062] S701: fitting the clustered point coordinates by the least square method to generate crop row straight lines;

[0063] S702: When the distance between the coordinates of two rapeseed plants exceeds a set threshold of 230 pixels, the area is defined as a seedling-deficient area;

[0064] S703: Extracting the coordinates of the missing seedling points based on the fitted crop row straight line.

[0065] It should be noted that the distance threshold of 230 pixels is set based on the rapeseed seedling planting specifications. Specifically, the rapeseed planting spacing is usually 15 cm. When the actual distance between rapeseed seedlings exceeds 30 cm, it is considered that there is a lack of seedlings. In order to accurately measure, we use Zhang Zhengyou's calibration method to determine the internal parameter matrix of the DJI drone Mavic 3T wide-angle camera to calculate the conversion relationship between pixels and actual distance. The internal parameter matrix is ​​as follows:

[0066]

[0067] It is stipulated that the drone takes images at an altitude of 4 meters. Through the calculation of the pixel internal reference matrix, it is known that the actual distance corresponding to 1 pixel unit is about 1.331 mm. The rapeseed seedling spacing exceeding 30 cm, that is, 230 pixels, is defined as missing seedlings. The distance threshold of the missing seedling area is defined as 230 pixels. Figure 6 The area framed by the middle dotted line is the area lacking seedlings.

[0068] S8: After confirming the seedling-deficient area, the specific position of the seedling replacement is calculated based on the fitted crop row straight line.

[0069] It should be noted that the specific location information of the seedling replacement will be transmitted to an unmanned aerial vehicle field rapeseed seedling database system for storing the image data collected by the unmanned aerial vehicle, the intermediate results generated during the processing, and the management of the final results.

[0070] S9: Send the location of the missing seedlings and the seedling replenishment instruction to the seedling replenishment device, and end the whole process.

[0071] The above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention.

[0072] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting missing rapeseed seedlings based on drones and image processing, characterized in that The following steps are involved: S1: Use a DJI drone Mavic 3T to fly along a predetermined path and record video data of rapeseed seedlings in the field for two consecutive days; S2: The YOLOv8 model trained for rapeseed seedlings is used to identify video data in real time, and the ByteTrack tracking algorithm is used to count seedlings between consecutive frames; S3: Compare the number of seedlings on the current day with that of the previous day. If the count increases, repeat the counting process; if the count does not increase, proceed to the next step; S4: Use a DJI drone Mavic 3T to fly along a specified path and record photo data of rapeseed seedlings in the field; S5: Use the trained YOLOv8 model to identify the location of rapeseed seedlings in the photo and extract coordinate information; S6: Apply the improved DBSCAN clustering algorithm to classify the coordinate points; S7: obtaining a crop row straight line by least squares fitting, and determining whether there is a seedling-deficient area exceeding a set distance threshold; S8: After confirming the seedling-deficient area, the specific location where seedlings need to be supplemented is calculated based on the fitted crop row straight line; S9: Send the location of the missing seedlings and the seedling replenishment instruction to the seedling replenishment device, and end the whole process.

2. The method for detecting missing rapeseed seedlings in the seedling stage based on drone and image processing according to claim 1, characterized in that: The specific steps of step S3 are as follows: when the count of the day increases, steps S1 to S2 are repeated until the count stops increasing.

3. The method for detecting missing rapeseed seedlings in the seedling stage based on drone and image processing according to claim 1, characterized in that: In the step S4, when the drone flies along the designated path to collect image data, the heading overlap rate is set to 70%, the lateral overlap rate is set to 65%, and the flight altitude is set to 4 meters.

4. The method for detecting missing rapeseed seedlings in the seedling stage based on drone and image processing according to claim 1, characterized in that: The improved DBSCAN clustering algorithm in step S6 includes changing the traditional circular neighborhood search to a rectangular neighborhood search.

5. The method for detecting missing rapeseed seedlings in the seedling stage based on drone and image processing according to claim 1, characterized in that: The step S7 obtains the crop row straight line by least squares fitting, and determines whether there is a seedling-deficient area exceeding the set seedling spacing threshold. The specific steps are: S701: fitting the clustered point coordinates by the least square method to generate crop row straight lines; S702: When the distance between the coordinates of two rapeseed plants exceeds a set threshold of 230 pixels, the area is defined as a seedling-deficient area; S703: Extracting the coordinates of the missing seedling points based on the fitted crop row straight line.

6. The method for detecting missing rapeseed seedlings in the seedling stage based on drone and image processing according to claim 4, characterized in that: The specific steps of the improved DBSCAN clustering algorithm are as follows: a rectangular neighborhood search with a width of 200 pixels and a length of 1000 pixels, centered on the coordinates of the rapeseed seedling.

7. A rapeseed seedling deficiency detection and seedling supplementation system based on drone and image processing applied to claims 1-6, characterized in that: The seedling shortage detection and seedling replenishment system comprises: Image acquisition module: obtains rapeseed seedling images of the field to be identified, the rapeseed seedling images contain rapeseed seedling features in several different time periods and different postures; the rapeseed seedling image data comes from images collected by the DJI drone Mavic 3T. Rapeseed coordinate detection module: input the obtained rapeseed seedling image into the rapeseed seedling detection model obtained by training the YOLOv8 model, perform the first stage of rapeseed seedling target recognition, and obtain coordinate information containing the rapeseed seedling; The rapeseed seedling detection module also includes: a first training module, which is used to divide the rapeseed seedling image into training set data, validation set data and test set data; use a labeling tool to label the individual rapeseed seedling bounding boxes in the rapeseed seedling images of the training set data and the test set data; input the rapeseed seedling images in the training set data and the corresponding individual rapeseed seedling bounding boxes into the YOLOv8 model to obtain predicted rapeseed seedling bounding boxes; use the validation set data to perform performance verification on the YOLOv8 model, and when the YOLOv8 model meets the preset performance requirements, use the trained YOLOv8 model as a rapeseed seedling detection model; Rapeseed missing seedling location module: The coordinate information of the rapeseed seedlings obtained in the first stage is input into the improved DBSCAN clustering algorithm. The improved DBSCAN clustering algorithm includes changing the traditional circular neighborhood search to a rectangular neighborhood search for rapeseed seedlings, and implementing rapeseed seedling crop row classification. Then, the clustered point coordinates are fitted by the least squares method to generate a crop row straight line. When the distance between the coordinates of two rapeseed plants exceeds the set threshold, the area is defined as a missing seedling area. The missing seedling point coordinates are extracted based on the fitted crop row straight line to complete the second stage of rapeseed seedling missing identification and obtain rapeseed seedling missing location information; Rapeseed seedling supplementation module: the missing rapeseed seedling location information obtained in the second stage is transmitted to the seedling supplementation device, thus terminating the entire system.

8. The method for detecting missing rapeseed seedlings in the seedling stage based on drone and image processing according to claim 7, characterized in that: The system also has learning capabilities to obtain a better YOLOv8 recognition weight file in real time.

9. A UAV field rapeseed seedling database system, characterized by: It includes a system for storing image data collected by drones, intermediate results generated during processing, and managing final results.

Images