A high-precision automatic extraction method for the vector of tea row boundaries based on tea garden UAV images

Abstract

The present invention discloses a method for automatically and highly accurately extracting the vector of tea row boundaries based on UAV images of tea gardens, comprising the following steps: obtaining UAV multispectral images, annotating the region of interest to obtain the tea garden plot ROI mask; performing binary classification processing based on threshold segmentation to obtain the initial classification image of tea trees - soil; extracting the boundary contour and detecting straight lines from the initial classification image of tea trees - soil, and calculating the main direction slope of the tea rows of tea trees; generating parallel sampling paths based on the main direction slope of the tea rows and classifying them; grouping the parallel sampling paths according to the same category to obtain a scan line group of continuous tea tree or soil categories; filling the soil regions therein to complete the highly accurate extraction of the tea row boundaries. The method of the present invention realizes the highly accurate vector extraction of tea row boundaries by optimizing the segmentation boundary and eliminating discrete misclassified patches, and overcomes the problem that the tea row boundaries are unclear in the existing extraction of tea rows in tea gardens when there are missing tea trees, withered branches, and discolored leaves.

Description

Technical Field

[0001] The present invention relates to the field of smart agriculture, and particularly to a method for automatically and highly accurately extracting tea row boundary vectors based on tea garden UAV images. Background Art

[0002] In modern tea garden planting, tea trees are usually cultivated in a strip-like arrangement to form neat tea rows. This planting pattern is designed based on the growth characteristics of tea trees and the convenience of tea garden management. Tea trees are planted at fixed intervals according to the requirements of row spacing and plant spacing on flat or terraced land, forming an orderly tea row structure. The planting method with tea row arrangement not only improves the space utilization rate of the tea garden but also provides a basis for the standardized management and mechanized operation of the tea garden, which is one of the important characteristics of efficient production in modern tea gardens.

[0003] Tea row vector data is the core basis for smart tea garden operations and precise monitoring, but its extraction process is often interfered with. Currently, general methods rely on pixel analysis. Usually, multi-spectral images are obtained by UAVs, and then extraction and recognition are carried out through threshold segmentation or deep learning. However, both of these methods face the problems of misclassification and missed classification. This is because due to pests and diseases causing chlorosis of tea tree leaves, dead branches and missing plants, the spectral characteristics of tea trees are similar to those of the soil background, resulting in a decrease in their separability, thus leading to the defects of misclassification and missed classification, and phenomena such as blurred boundaries and discrete patches; although deep learning has improved compared to the threshold segmentation method that relies on vegetation indices and bands, it is limited by the labeled data in the tea garden and still cannot completely overcome the problems of misclassification and missed classification. In addition, the objects extracted by the above methods are tea trees with a complete canopy structure, which is not completely consistent with the actual tea row structure in the tea garden, and the complete tea row cannot be extracted, resulting in a deviation between the extraction result and the actual tea row structure, which restricts the reliability of precise monitoring in the tea garden. Summary of the Invention

[0004] The first object of the present invention is to overcome the problems in the extraction of tea rows in the existing tea gardens, such as unclear tea row boundaries and low standardization degree caused by stress factors such as pests and diseases, resulting in missing plants, dead branches, and leaf color changes of tea trees. Analyze the continuous structural characteristics of tea tree arrangement, optimize the segmentation boundary, eliminate discrete misclassified patches, generate a regular binary mask that conforms to the ridge row distribution and actual planting situation in the tea garden, where the tea tree area is the foreground and the soil is the background, and highly accurately extract the tea row boundary vector in the tea tree area.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for automatically and highly accurately extracting tea row boundary vectors based on tea garden UAV images, comprising the following steps:

[0007] Step S1: Obtain the multi-spectral images of the tea garden captured by a drone, label the region of interest to obtain the ROI mask of the tea garden plot; perform binary classification using the threshold segmentation method to obtain the initial tea plant - soil classification image;

[0008] Step S2: Extract the boundary contours and detect straight lines from the initial tea plant - soil classification image, and calculate the main direction slope of the tea rows;

[0009] Step S3: Generate parallel sampling paths based on the main direction slope of the tea rows, and classify the parallel sampling paths;

[0010] Step S4: Detect the class changes of adjacent parallel sampling paths, merge continuous regions of the same class, obtain a scan line group of continuous tea plant or soil classes, fill the soil regions among them, and complete the extraction of the tea row boundary vector.

[0011] Among them, in Step S3, the image data of the tea garden is converted into a structured array to realize the standardized acquisition of data and the extraction of tea row features.

[0012] The second object of the present invention is to provide a tea row recognition device for implementing the above method, including the following modules:

[0013] An image acquisition module, which uses a drone to obtain the multi-spectral images of the tea garden;

[0014] An image preprocessing module, which is used to label the region of interest of the multi-spectral images of the tea garden and perform processing to obtain the initial tea plant - soil classification image of the tea garden plot and the ROI mask of the tea garden plot;

[0015] An edge extraction and straight line detection module, which is used to extract the tea plant - soil boundary in the initial tea plant - soil classification image and calculate the main direction of the tea rows;

[0016] A parallel resampling module, which uses the ROI mask of the tea garden plot as the target classification area, and generates and classifies parallel sampling paths according to the main direction of the tea rows;

[0017] A region merging and filling module, which detects the class changes of adjacent parallel sampling paths, merges continuous regions of the same class, and fills the soil regions;

[0018] An output module, which generates the final tea row and soil segmentation map according to the filling result and outputs the visualization result.

[0019] The third object of the present invention is to provide an electronic device, including a processor and a memory, where the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the above method.

[0020] The fourth object of the present invention is to provide a machine-readable storage medium storing machine-executable instructions, which, when called and executed by a processor, cause the processor to implement the above method.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0022] Based on the initial classification result of threshold segmentation, the present invention, based on the unique ridge-row planting pattern of tea gardens, generates scan lines through the main direction of tea rows in UAV images, and combines continuous scan line classification and region filling technology to accurately extract the tea row region, overcomes the problem that some tea trees are misclassified as soil in traditional classification methods, repairs the gaps and discontinuous regions existing in the tea rows in tea garden images, realizes efficient and accurate extraction of tea rows, generates a regular binary classification map that conforms to the actual planting situation of tea gardens, significantly improves the accuracy and standardization degree of tea garden classification data, and provides a new idea for tea tree image processing and tea row boundary extraction.

[0023] The present invention integrates multi-spectral analysis and the ridge-row structure characteristics of tea gardens, analyzes the continuous spatial structure characteristics of the arrangement of tea trees, ensures the integrity and boundary accuracy of tea row vectors, provides reliable support for the intelligent management of tea gardens, and can quickly and automatically perform large-scale monitoring on remote sensing platforms such as UAVs, saving labor costs and improving farmland management efficiency. Description of the Drawings

[0024] Figure 1 It is a flowchart of the method of the present invention.

[0025] Figure 2 It is the image after the processing of step S1, where A is the multi-spectral image of the tea garden, B is the initial classification image of tea trees and soil, and C is the ROI mask of the tea garden plot.

[0026] Figure 3 It is the image after the processing of step S2, where A is the schematic diagram of edge detection and contour extraction, and B is the schematic diagram of line detection.

[0027] Figure 4 It is the image after the processing of step S3.

[0028] Figure 5 It is the reclassification effect diagram, where A is the original multi-spectral image of the tea garden, B is the initial classification image after threshold segmentation, and C is the optimized binary classification image obtained by the method of the present invention. Detailed Embodiments

[0029] The following further describes the present invention with reference to the drawings and embodiments.

[0030] As Figure 1As shown in the figure, the present invention provides a high-precision automatic extraction method for the vector of tea row boundaries based on UAV images of tea gardens, including the following steps:

[0031] Step S1: Input the multi-spectral images of the tea garden collected by the UAV. The multi-spectral images include four bands, namely Red, Green, Blue, and Near Infrared (NIR). Taking each independent tea garden plot as a processing unit, the following operations are performed:

[0032] Manually label the tea garden plot as the region of interest (ROI), which serves as the input data for subsequent classification processing (as shown in A in Figure 2 ), and generate the ROI mask of the tea garden plot (as shown in C in Figure 2 ).

[0033] In addition, the threshold segmentation method is used to perform binary classification processing on the tea garden plot images: Obtain the Green band image I Green (x, y) from the multi-spectral image, where (x, y) is the pixel coordinate.

[0034] According to the spectral separability of tea trees and soil in the Green band, set the classification threshold T class = 0.08 (determined by statistically analyzing the reflectance distributions of tea tree and soil samples), and establish a binary classification decision rule:

[0035]

[0036] where 1 represents the vegetation area and 0 represents the soil area; according to the classification result C(x, y), output the initial classification image of tea tree - soil (as shown in B in Figure 2 ), where the vegetation area is marked as 1 (green) and the soil area is marked as 0 (black).

[0037] Step S2: Obtain the slope of the tea row; specifically:

[0038] For the initial classification image of tea tree - soil in Step S1, use the Canny edge detection algorithm to extract the boundary contour between the tea tree area and the soil background (as shown in A in Figure 3 ). Specifically, after suppressing noise through Gaussian filtering, calculate the image gradient magnitude and direction, and perform non-maximum suppression and hysteresis threshold processing using double thresholds to output the pixel-level continuous edge contour.

[0039] Map the edge contour pixel points from the Cartesian coordinate system to the Hough parameter space (ρ - θ space), and establish a linear parameterization model:

[0040]

[0041] Among them, (x, y) are the coordinates of the edge points, ρ is the distance from the straight line to the origin, and θ is the angle between the straight line and the x-axis. All edge points are traversed, and each (ρ, θ) in the parameter space is accumulated and voted to generate a voting distribution matrix in the parameter plane.

[0042] Set the minimum voting number threshold T vote = 200 (the threshold is positively correlated with the input image resolution and is dynamically adjusted according to the actual size of the ROI), and filter the candidate point set {(ρ vote , θ i )} in the parameter space whose voting number exceeds T. Each candidate point corresponds to a straight line in the Cartesian coordinate system, and its equation is: i )} Each candidate point corresponds to a straight line in the Cartesian coordinate system, and its equation is:

[0043]

[0044] Output the detected straight line set (as shown by B in Figure 3 ). Optimize the direction consistency of the original detected straight lines, eliminate the abnormal slopes outside the interquartile range, retain the candidate slope subset that conforms to the spatial distribution law of the tea rows, and then take the average value as the main direction slope k of the tea rows in the region.

[0045] Step S3, Scan line classification, specifically:

[0046] Use the tea garden plot ROI mask generated in step S1 as the target classification area;

[0047] Based on the main direction slope k of the tea rows obtained in step S2, generate an index line perpendicular to the main direction of the tea rows (slope k' = -1 / k), and its starting endpoint is dynamically selected according to the sign of k:

[0048] When k > 0, use the starting point (0, 0) of the left boundary of the tea garden plot ROI as the initial position.

[0049] When k ≤ 0, use the starting point (right, 0) of the right boundary of the tea garden plot ROI as the initial position.

[0050] Vertical scan line L i Extend along the k' direction to the opposite boundary to form the initial index line {L i}.

[0051] Traverse all pixel points (x i , y j , y j ) on the index line L, and through each point, generate a parallel scan line along the main direction slope k of the tea rows (the equation is y = k(x - x j ) + y j ), and form the main direction scan line set {L m} that covers the entire target classification area.

[0052] For each scan line L m The intersection point with the ROI mask boundary of the tea garden plot (x start ,y start ) and (x end ,y end ), intercept the effective scanning line segment S according to the two intersection points m (like Figure 4 As shown). On the tea tree-soil initial classification image, along the line segment S m Take the pixel set P m ={P 1 ,P 2 ,P 3 ···,P N}, where p i ∈{0,1} (0 is soil, 1 is tea tree).

[0053] Statistics P m Number of tea tree pixels:

[0054]

[0055] Calculate the proportion:

[0056]

[0057] Set the experience threshold T proportion = 0.7, allowing adjustment according to tea plantation density, T proportion ∈[0.7,0.8]; if tea_proportion ≥ T proportion , determine the scan line segment S m For tea tree area, determine line type tea tree (value 1); if tea_proportion <T proportion , determine the scan line segment S m For the tea ridge area, determine the line type soil (value 0).

[0058] In the present invention, "tea tree pixel" refers to a single pixel whose spectral classification result is vegetation, "tea row" refers to a farming belt formed by continuous tea tree pixels arranged in a linear pattern, and "tea ridge" refers to the soil gap between adjacent tea rows.

[0059] Step S4, graphic image filling; specifically, comprising the following steps:

[0060] In step S3, the attributes (x start ,y start ) and (x end ,y end) and the scan line type (tea tree / soil). Therefore, traverse all classified scan lines and detect the category attributes (tea tree / soil) of adjacent scan lines in real time. When the category attributes of adjacent scan lines are inconsistent or the end of the list is reached, it is determined that the current continuous same-category line segment group ends. For each continuous same-category line segment group, extract the starting point and ending point of the first scan line and the starting point and ending point of the last scan line to form a set of four vertex coordinates.

[0061] Based on the four vertex coordinates, construct a quadrilateral region. The vertex order follows the following rule: starting point of the first scan line → ending point of the first scan line → ending point of the last scan line → starting point of the last scan line.

[0062] If the category of the line segment group is soil, use the fillPoly method in the OpenCV library in python to fill the quadrilateral region with 0 in the ROI mask of the tea garden plot; if it is a tea tree, no processing is done.

[0063] When the traversal ends, perform a closing operation on the last group of continuous same-category line segments to ensure that all regions are filled completely.

[0064] Finally, output the classified and optimized image.

[0065] Process the multi-spectral images of tea gardens in different regions according to the method of the present invention, and the results are as Figure 5 shown; Figure 5 A in Figure 5 is the false color image of the original four-band image, Figure 5 B in Figure 5 is the preliminary classification image obtained after processing by the threshold segmentation method,

[0066] The above embodiments are not limitations on the present invention. The present invention is not limited to the above embodiments. As long as it meets the requirements of the present invention, it belongs to the protection scope of the present invention. In addition, it should be understood that after reading the above description content of the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of this application.

Claims

1. A high-precision automatic extraction method of tea row boundary vectors based on tea garden drone images, characterized in that: The following steps are involved: Step S1, obtaining a multispectral image of the tea garden by using a drone, marking the region of interest to obtain a ROI mask of the tea garden plot; using a threshold segmentation method to perform binary classification processing to obtain an initial classification image containing two types of targets: tea trees and soil; Step S2, performing boundary contour extraction and straight line detection on the initially classified image, and calculating the main direction slope of the tea row; Step S3: Generate parallel sampling paths based on the main direction slope of the tea row, and classify the parallel sampling paths; specifically: 3-1 Based on the main direction slope k of the tea row obtained in step S2, an index line perpendicular to the main direction of the tea row is generated; 3-2 Take the tea garden plot ROI mask as the target classification area, traverse all pixels on the vertical index line, generate parallel scan lines along the main direction slope of the tea row through each pixel, and form a main direction scan line set covering the entire target classification area; 3-3 For each main direction scanning line in the main direction scanning line set, it has two intersection points with the boundary of the target classification area, and the scanning line between the two intersection points is intercepted as a valid scanning line segment; 3-4 Mapping the effective scanning line segment to the initial classification image, counting the number of tea tree pixels in the effective scanning line segment and calculating the proportion, and judging the category of the effective scanning line segment according to the proportion, the category is tea tree or soil; Step S4, detecting the category changes of adjacent parallel sampling paths, merging continuous similar areas, obtaining a scanning line group of continuous tea tree or soil categories, filling the soil area therein, and completing the extraction of the tea row boundary vector.

2. The method for high-precision automatic extraction of tea row boundary vectors based on tea garden drone images according to claim 1 is characterized in that: In step S1, the UAV multispectral image contains four bands, and the Green band image is selected to perform binary classification processing through the threshold segmentation method to obtain the initial classification image.

3. The method for high-precision automatic extraction of tea row boundary vectors based on tea garden drone images according to claim 1 is characterized in that: In step S2, the boundary contour is extracted using the Canny edge detection algorithm.

4. The method for high-precision automatic extraction of tea row boundary vectors based on tea garden drone images according to claim 3 is characterized in that: In step S2, Hough transform is used for line detection; the original detected line is optimized for directional consistency, abnormal slopes outside the interquartile range are eliminated, a subset of candidate slopes that conform to the spatial distribution law of the tea row are retained, and the average value is taken as the main direction slope of the tea row.

5. The method for high-precision automatic extraction of tea row boundary vectors based on tea garden drone images according to claim 1 is characterized in that: In step S4, when the adjacent scan line category attributes are inconsistent or the list is traversed to the end, it is determined that the current continuous similar line segment group ends; for each continuous similar line segment group, the starting point and the ending point of the first scan line, and the starting point and the ending point of the last scan line are extracted to form the coordinates of the four vertices, and then a quadrilateral area is constructed; If the line segment group category is soil, the pixel values ​​in the quadrilateral area are set to zero in the ROI mask of the tea garden plot; If the line segment group category is tea tree, no processing is done; At the end of the traversal, a closing operation is performed on the last set of continuous similar line segments, and the final classified image is output.

6. A tea brand identification device for implementing the method according to any one of claims 1 to 5, characterized in that: Includes the following modules: Image acquisition module, using drones to acquire multispectral images of tea gardens; The image preprocessing module is used to mark the region of interest of the tea garden multispectral image and process it to obtain the tea tree-soil preliminary classification image and the tea garden plot ROI mask; The edge extraction and line detection module is used to extract the boundary between tea trees and soil in the initial classification image and calculate the main direction of the tea row; The parallel resampling module uses the tea garden plot ROI mask as the target classification area, generates parallel sampling paths and classifies them according to the main direction of the tea row; The region merging and filling module detects the category changes of adjacent parallel sampling paths, merges continuous similar regions, and fills the soil region; The output module generates the final tea row and soil segmentation map based on the filling results and outputs the visualization results.

7. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the method according to any one of claims 1 to 5.

8. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the method as described in any one of claims 1 to 5.

Images