Automatic extraction method and device for high-throughput crop plots based on low-altitude remote sensing

Abstract

The application discloses a high-throughput field crop plot automatic extraction method and device based on low-altitude remote sensing. Firstly, high-definition remote sensing images in the seedling stage are obtained by shooting through a UAV, then the images are spliced into DOM images with geographic coordinates through UAV image processing software, a selected extraction area is processed through cutting, division, data conversion and the like, so that a division detection frame of each plot of crops is obtained. In the division link, a processing method combining vegetation index segmentation and filtering is adopted when searching for plot row gaps, and adaptive Fourier transform is introduced to resist the influence of grass damage, and when searching for plot column gaps, a method of calculating each seedling column of the land according to equidistance of two end seedling columns and then correcting according to an energy accumulation diagram is adopted to improve the accuracy when cutting in the column direction. The application provides an effective technical approach for high-throughput extraction of field plots.

Description

Technical Field

[0001] This invention relates to the fields of smart agricultural information technology applications, intelligent unmanned aerial vehicles, agricultural automation, and agricultural bioinformatics, specifically to a method and apparatus for automatic extraction of high-throughput field crop plots based on low-altitude remote sensing. Background Technology

[0002] In modern breeding and cultivation research, crop plots are crucial for seed quality assessment and the selection of superior varieties. Traditional plot division methods primarily rely on manual division using visual observation combined with planting experimental design. These methods are labor-intensive, inefficient, and heavily influenced by human factors, making them unsuitable for large-scale breeding research. With the development of remote sensing and drone technologies, and the ability to extract image features using segmentation methods such as HSV and vegetation index, remote sensing images of crop seedlings acquired by drones equipped with high-definition cameras, combined with threshold segmentation and search algorithms, can generate plot division detection boxes. This significantly improves the accuracy and efficiency of crop plot extraction, enabling rapid and accurate plot extraction.

[0003] However, accurately extracting crop plots from high-resolution remote sensing images taken by drones still faces a series of challenges. First, drone-captured images cover a wide area and are too large to be directly used for extraction. Furthermore, remote sensing images are affected by factors such as shooting angle, lighting conditions, seedling emergence, and weed infestation, making the division and extraction of crop plots more complex. In addition, the irregular design of the row spacing in crop plots also increases the difficulty of detection.

[0004] Therefore, researching a high-throughput extraction method for field plots that combines UAV remote sensing technology with threshold segmentation and search algorithms can not only improve the intelligence level of breeding operations, but also promote the development of precise crop phenotypic detection technology, promote the development of breeding and cultivation research, and serve the selection of high-quality varieties. Summary of the Invention

[0005] (a) Technical problems to be solved This invention aims to solve the technical challenge of rapid and accurate extraction of field crop plots, particularly improving extraction accuracy under complex field conditions such as weed infestation. This invention provides a high-throughput detection method for field crop plots based on unmanned aerial vehicles (UAVs), threshold segmentation, and search algorithms, enabling automatic extraction of field crop plots.

[0006] (II) Technical Solution To address the aforementioned problems, this invention proposes a method and apparatus for automatic extraction of high-throughput field crop plots based on low-altitude remote sensing.

[0007] This invention discloses a high-throughput automatic extraction method for field crop plots based on low-altitude remote sensing, comprising the following steps: Step A: Use a drone to collect visible light remote sensing images of seedlings in the field and stitch the images collected by the drone together. Step B: Select the area to be extracted from the stitched large image, complete the cropping, then rotate the cropping area to adjust the crop seedling rows to the due north-south direction, and record the number of plots R1 and the number of seedling rows R2 in this area; Step C: For the rotated cropped area, the seedlings are segmented by thresholding. In the segmentation mask, the seedlings are set to a value of 1, corresponding to the white part of the mask image, and the other parts are set to a value of 0, corresponding to the black part of the mask image. Step D: Accumulate the mask image along the y-axis of the pixel coordinates to obtain the energy waveform diagram. Each valley in the energy waveform diagram is the passage between the row-oriented plots. Then, use the y-coordinate corresponding to each passage to divide each row-oriented plot. Step E: Process the mask of each row of plots obtained in step D. Accumulate the mask according to the pixel coordinate x-axis to obtain the energy waveform. Calculate the x-coordinate of other seed columns in the same row of plots based on the x-coordinate of the seed columns at both ends. Then correct the coordinates according to the energy waveform to obtain the x-coordinate of the gap between each seed column and cut out each column of seed in each row of plots. Step F: Generate a rectangular bounding box for all seedling rows in the selected area based on the y-coordinate of each aisle and the x-coordinate of the gap between each seedling row, output the result image with the rectangular bounding box, and automatically extract field crop plots using the rectangular bounding box.

[0008] Preferably, in step A, the stitched image is in TIFF format with latitude and longitude coordinates.

[0009] Preferably, in step B, the cropped image is exported as a JPG file, and the image is not compressed during export; it is processed in its original GSD format.

[0010] Preferably, when performing threshold segmentation and row-oriented plot segmentation in steps C and D, if the cropping area has no grass damage or only slight grass damage, threshold segmentation is performed using the HSV color space. The segmented masks are accumulated along the y-direction to obtain an energy waveform diagram. The valley features are highlighted in the energy waveform diagram. The valleys in the waveform diagram are searched in conjunction with grouping to determine the gaps between row-oriented plots. As the search iterates, the search stops when the number of row-oriented gaps obtained is equal to R1+1. The y-coordinates corresponding to the gaps between each row-oriented plot are output.

[0011] Preferably, the threshold values ​​for the H, S, and V channels are set to 33-77, 43-255, and 46-255, respectively.

[0012] Preferably, during threshold segmentation and row-oriented plot division in steps C and D, when the cropping area is severely infested with weeds, the valleys in the energy waveform are lost. Segmentation is performed using the vegetation index ExG combined with the Otsu method, and the segmented mask is processed by median filtering. The energy waveform is subjected to Discrete Fourier Transform (DFT) to obtain the signal spectrum. The main signal is extracted and the main frequency is filtered. After obtaining the main signal, the local minimum value is searched to obtain the row-oriented plot gap. The y-coordinate corresponding to the gap of each row-oriented plot is output.

[0013] Preferably, when performing the filtering process, the size of the filter convolution kernel is set to 5.

[0014] Preferably, in step E, based on the cumulative energy waveform in the x-direction, the x-coordinates of the peaks at both ends of the image are used as the x-coordinates of the first column of seedlings on the left and the first column of seedlings on the right. Using the x-coordinates of the two columns of seedlings at the left and right ends as a reference, and combined with the number of seedling columns R2, reference x-coordinates of each column of seedlings in the middle of the row are generated at equal intervals. Then, a judgment neighborhood is generated for the x-coordinate of each column of seedlings, and the x-coordinate corresponding to the maximum value of the cumulative energy in each neighborhood is replaced with the x-coordinate of that column of seedlings. The coordinates of the midpoint between the columns of seedlings are output as the x-coordinates of the gap between each column of seedlings.

[0015] The present invention also discloses a high-throughput automatic field crop plot extraction device based on low-altitude remote sensing, which uses the aforementioned high-throughput automatic field crop plot extraction method based on low-altitude remote sensing to automatically divide and extract field crop plots.

[0016] (III) Beneficial Effects Compared with the prior art, the present invention has significant positive technical effects, which are specifically manifested in the following aspects.

[0017] (1) Traditional segmentation methods mostly use HSV and HIS color spaces to segment images. In weed-affected areas, weeds and seedlings will be processed as value 1 after segmentation due to color characteristics. The vegetation index segmentation combined with image filtering proposed in this invention can effectively resist the influence of field weeds. Even in the case of extremely severe weed infestation, the probability of weeds in the area being misclassified as value 1 is still less than 30%.

[0018] (2) In existing gap search methods, to accurately locate the row and column gaps in each cell, most methods first divide the row into plots, then locally process each plot to divide the columns. When searching for gaps between row plots, most methods choose to locate each gap using the trough features in the energy map of the selected area segmentation mask in the y-direction. However, when weed infestation is severe, the trough features are significantly lost, resulting in fewer gaps found than the actual number of gaps. This invention introduces an adaptive discrete Fourier transform to process the energy waveform under weed infestation conditions, extract the main signal, and filter the main frequency to ensure the accuracy of row gap search under the influence of weed infestation.

[0019] (3) When dividing each seedling row in each row of the plot, the existing search algorithm also uses the valley in the energy waveform diagram of the x-direction accumulation to locate the gap between seedling rows. It cannot eliminate the influence of missing seedlings, sparse seedlings, and weed damage. The search algorithm proposed in this invention first searches for the x coordinates corresponding to the two seedling rows, combines R2 to generate the reference coordinates of each seedling row, then generates a judgment neighborhood for each seedling row, replaces the coordinates of the seedling row with the x coordinate corresponding to the maximum value of the x-direction energy accumulation in the neighborhood, and finally divides each seedling row by using the midpoint between seedling rows as the gap. This method can effectively overcome the interference of the above-mentioned complex field conditions. Attached Figure Description

[0020] Figure 1 The application flowchart of the method of this invention.

[0021] Figure 2 The segmentation effect diagram of the vegetation index combined with filtering in this invention.

[0022] Figure 3 A schematic diagram of the adaptive Fourier transform of this invention.

[0023] Figure 4 A schematic diagram illustrating the x-coordinate of the seedling column in this invention.

[0024] Figure 5 A schematic diagram illustrating the application of this invention: cell extraction. Detailed Implementation

[0025] To address its technical problems, this invention provides a method and apparatus for automatic extraction of high-throughput field crop plots based on low-altitude remote sensing. The flowchart of the method is shown below. Figure 1 As shown.

[0026] A method for automatic extraction of high-throughput field crop plots based on low-altitude remote sensing includes the following steps: Step 1: Use a drone to collect visible light remote sensing images of seedlings in the field, and stitch the images collected by the drone to obtain an orthophoto DOM with geographic coordinates. Step 2: Select the area to be extracted from the stitched large image, complete the cropping, then rotate the cropping area to adjust the crop seedling rows to the due north-south direction, and record the number of plots R1 and the number of seedling rows R2 in this area; Step 3: For the rotated cropped area, the seedlings are segmented by thresholding. In the segmentation mask, the seedlings are set to a value of 1, corresponding to the white part of the mask image, and the other parts are set to a value of 0, corresponding to the black part of the mask image. Step 4: Accumulate the mask image along the y-axis of the pixel coordinates to obtain the energy waveform diagram. Each trough in the energy waveform diagram is the passage between the row-oriented plots. Then, use the y-coordinate corresponding to each passage to divide each row-oriented plot. Step 5: Process the row-oriented plot masks obtained in Step D respectively. Accumulate the masks according to the pixel coordinate x-axis to obtain the energy waveform diagram. Calculate the x-coordinates of other seed columns in the plot based on the x-coordinates of the seed columns at both ends. Then correct the coordinates according to the energy diagram. Finally, use the midpoint coordinates between seed columns as the x-coordinates of the gap between each seed column to divide each row-oriented plot into each seed column. Step 6: Generate bounding boxes for all seedling rows in the selected area based on the y-coordinate of each aisle and the x-coordinate of the gap between each seedling row. Output the result map with the bounding boxes and use these bounding boxes to automatically extract field crop plots.

[0027] In step 1, the crop seedlings must be clearly visible in the image when the drone collects data.

[0028] Step 2: After selecting the extraction area, crop it and export the cropped image as a JPG file. Do not compress the image during export; save the original GSD file. Rotate the exported image so that the seedling rows are aligned to true north-south, and record the rotation angle D1. Record the number of plots in the row direction R1 and the number of seedling rows R2 for this area.

[0029] Steps 3 and 4 involve thresholding the region image obtained in step 2. The default thresholding space is HSV color space, with H, S, and V channel thresholds set to 33-77, 43-255, and 46-255 respectively. After obtaining the mask, energy waveforms are accumulated along the y-axis. If valley features are prominent in the waveform, the valleys are searched to group the data and determine the gaps between rows. The search stops when the number of row gaps equals R1+1. Otherwise, if valley features are lost in the energy waveform, the iterative search cannot yield R1+1 gaps, indicating severe weed infestation in the area. The vegetation index (ExG) combined with Otsu's method is used for segmentation. The segmented mask is then subjected to median filtering with a kernel size of 5. The segmented mask is accumulated along the y-axis, and a Discrete Fourier Transform (DFT) is performed on the energy waveform to obtain the signal spectrum. The main signal is extracted and the main frequency is filtered. After obtaining the main signal, the local minimum value is searched to determine the row gap. The gap coordinates are then output.

[0030] The combined approach to handling weed in steps 3 and 4 is an improvement on existing row-oriented plot search algorithms. It effectively overcomes the impact of weeds in the region. A comparison of the masking effects of ExG segmentation combined with filtering and HSV segmentation in the case of weed infestation is shown in the figure. Figure 2The original image contains ten rows of seedlings. This area is severely infested with weeds, which grow abundantly around the seedling rows. The mask after HSV segmentation shows that almost all the weeds are misclassified as values ​​of 1. Therefore, the mask corresponding to the seedling rows in the original image has many white mask interferences around it. However, the mask after vegetation index segmentation combined with filtering closely resembles the shape of the seedling rows in the original image, effectively removing the interference from the weeds. The row-by-row search process under weed infestation conditions is described in [link to relevant documentation]. Figure 3 After vegetation index segmentation and filtering, the trough characteristics are basically prominent in the accumulated waveform. However, interference still exists at the second trough (indicated by the arrow), where two troughs appear. A discrete Fourier transform is performed on the waveform to obtain the signal spectrum. The point with the largest amplitude is the main signal, as indicated by the arrow in the figure. Other signal frequencies are removed, and the main signal is selected as follows: Figure 3 (Right) Searching for local minima yields the row spacing.

[0031] Step 5: Based on the gap coordinates between the rows of plots obtained in Steps 3 and 4, the image is divided into multiple parts for processing. The mask of each row of plots is accumulated in the x-direction. Based on the obtained accumulated energy map, the x-coordinates of the peaks at both ends of the image are searched as the x-coordinates of the first column of seedlings on the left and the first column of seedlings on the right. The coordinates of these two columns of seedlings are combined with R2 equidistant to generate the reference x-coordinates of other columns of seedlings in the row of plots. Then, a judgment neighborhood is generated for the x-coordinate of each column of seedlings, and the x-coordinate corresponding to the maximum value of the accumulated energy in each neighborhood is replaced with the x-coordinate of that column of seedlings.

[0032] The search method used in step 5 can solve the problem of inaccurate searching of the gaps between seedling rows under the influence of missing seedlings, sparse seedlings, and weed damage. The x-coordinate of the seedling row located by this search method is shown in [the image / image]. Figure 4 The image contains 13 rows of seedlings. The coordinates of the seedling rows obtained by the algorithm are shown by the white lines in the image, as indicated by the arrows. Even under conditions of severe weed infestation, the x-coordinates obtained by the algorithm can accurately locate the position of each seedling row.

[0033] Step 6: Based on the row and column gap coordinates obtained in Steps 4 and 5, use the OpenCV library to generate the division detection box for each seed column in the selected area. After returning the coordinates to the coordinates in the original image according to the rotation angle D1, save it as a txt file. Combine the latitude and longitude information of the cropped area contained in the jgw file when cropping and exporting to generate a shp format file and deploy it to the stitched DOM image in Step 1.

[0034] The partition detection box generated in step 6 is shown below. Figure 5The image contains two row plots, each with 21 seedling rows. The white boxes in the image represent the detection frames. As can be seen, each seedling row is basically centered within the detection frame, achieving precise row and column segmentation even under severe weed infestation conditions. Segmentation accuracy is evaluated using IoU, with manually drawn and verified frames used as the standard. This method for automatic extraction of field plots achieves an accuracy of 96% under various real-world conditions, with a processing speed of 1000 plots / min, realizing high-throughput, high-precision automatic extraction of field plots.

[0035] The specific examples described in this application are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the specific examples described herein, or substitute them by similar means, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A high-throughput automatic extraction method for field crop plots based on low-altitude remote sensing, characterized in that, Includes the following steps: Step A: Use a drone to collect visible light remote sensing images of seedlings in the field and stitch the images collected by the drone together. Step B: Select the area to be extracted from the stitched large image, complete the cropping, then rotate the cropping area to adjust the crop seedling rows to the due north-south direction, and record the number of plots R1 and the number of seedling rows R2 in this area; Step C: For the rotated cropped area, the seedlings are segmented by thresholding. In the segmentation mask, the seedlings are set to a value of 1, corresponding to the white part of the mask image, and the other parts are set to a value of 0, corresponding to the black part of the mask image. Step D: Accumulate the mask image along the y-axis of the pixel coordinates to obtain the energy waveform diagram. Each valley in the energy waveform diagram is the passage between the row-oriented plots. Then, use the y-coordinate corresponding to each passage to divide each row-oriented plot. Step E: Process the mask of each row of plots obtained in step D. Accumulate the mask according to the pixel coordinate x-axis to obtain the energy waveform. Calculate the x-coordinate of other seed columns in the same row of plots based on the x-coordinate of the seed columns at both ends. Then correct the coordinates according to the energy waveform to obtain the x-coordinate of the gap between each seed column and cut out each column of seed in each row of plots. Step F: Generate a rectangular bounding box for all seedling rows in the selected area based on the y-coordinate of each aisle and the x-coordinate of the gap between each seedling row, output the result map with the rectangular bounding box, and automatically extract field crop plots using the rectangular bounding box. In steps C and D, when performing threshold segmentation and row-oriented plot segmentation, if the cropping area has no grass damage or only slight grass damage, threshold segmentation is performed using the HSV color space. The segmented masks are accumulated along the y-direction to obtain an energy waveform. The valley features are highlighted in the energy waveform. The valleys in the waveform are searched in conjunction with grouping to determine the gaps between row-oriented plots. As the search iterates, the search stops when the number of row-oriented gaps equals R1+1. The y-coordinates corresponding to the gaps between each row-oriented plot are output. In steps C and D, when threshold segmentation and row-oriented plot segmentation are performed, if the cropping area is severely infested with weeds, the troughs in the energy waveform are lost. Segmentation is performed using the vegetation index ExG combined with Otsu's method, and the segmented mask is processed by median filtering. The energy waveform is subjected to Discrete Fourier Transform (DFT) to obtain the signal spectrum. The main signal is extracted and the main frequency is filtered. After obtaining the main signal, the local minimum value is searched to obtain the row-oriented plot gap. The y-coordinate corresponding to the gap of each row-oriented plot is output. In step E, based on the cumulative energy waveform in the x-direction, the x-coordinates of the peaks at both ends of the image are used as the x-coordinates of the first column of seedlings on the left and right. Using the x-coordinates of the two columns of seedlings at the left and right ends as a reference, and combined with the number of seedling columns R2, reference x-coordinates of each column of seedlings in the middle of the row are generated at equal intervals. Then, a judgment neighborhood is generated for the x-coordinate of each column of seedlings, and the x-coordinate corresponding to the maximum value of the cumulative energy in each neighborhood is replaced with the x-coordinate of that column of seedlings. The coordinates of the midpoint between the columns of seedlings are output as the x-coordinates of the gap between each column of seedlings.

2. The method for automatic extraction of high-throughput field crop plots based on low-altitude remote sensing according to claim 1, characterized in that: In step A, the stitched image is in TIFF format with latitude and longitude coordinates.

3. The method for automatic extraction of high-throughput field crop plots based on low-altitude remote sensing according to claim 1, characterized in that: In step B, the cropped image is exported as a JPG file. The image is not compressed during export; it is processed in its original GSD format.

4. The method for automatic extraction of high-throughput field crop plots based on low-altitude remote sensing according to claim 1, characterized in that: The threshold settings for the H, S, and V channels are 33-77, 43-255, and 46-255, respectively.

5. The method for automatic extraction of high-throughput field crop plots based on low-altitude remote sensing according to claim 1, characterized in that: When performing the filtering process, the filter convolution kernel size is set to 5.

6. A high-throughput automatic field crop plot extraction device based on low-altitude remote sensing, characterized in that, It employs the high-throughput automatic extraction method for field crop plots based on low-altitude remote sensing as described in any one of claims 1-5 to automatically divide and extract field crop plots.

Images