A method for checking the manual delineation quality of green crops

Abstract

The application provides a method for checking artificial sketching annotation quality of green crops, which solves the problem of inconvenient quality evaluation of green crop annotation results, and the method comprises the following steps: cutting and processing an optical image, performing color space transformation, screening the color space, performing morphological operation post-processing, performing median filtering, and calculating the intersection-over-union ratio of each image and the whole, so as to realize quality evaluation, and the method has the advantages of convenient quality evaluation of green crop annotation results.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, specifically relating to a method for checking the quality of manual delineation and annotation of green crops. Background Technology

[0002] Agriculture is the foundation of national economic construction and development. In recent years, the development and maturation of drone imagery has promoted the application of information technology in agriculture, facilitating the transformation of traditional agriculture towards a high-efficiency, low-waste model. Drone imagery can be used to extract information about crops (distribution, growth, etc.) for agricultural monitoring and management. A crucial aspect of this is the manual labeling of crops. Currently, various labeling software programs are emerging, allowing for rapid labeling of large-scale drone imagery. However, these programs often prioritize convenience over quality control. Furthermore, most quality control methods rely on manual verification and screening, which is extremely time-consuming and labor-intensive. Therefore, there is currently a lack of rapid and efficient methods for quality control and evaluation of drone imagery labeling results for green crops.

[0003] To address the shortcomings of existing technologies, people have conducted long-term explorations and proposed various solutions. For example, Chinese patent literature discloses a deep learning-based spatiotemporal generalization classification method and system for crops [202110826057.7], which includes: collecting multiple remote sensing images of a set location within a set time period; obtaining real images corresponding to each remote sensing image through manual visual interpretation; the real images are images with labeled crop areas and non-crop areas; constructing a deep convolutional neural network; the deep convolutional neural network includes an input layer, a residual module, a pyramid average pooling module, and an output layer connected in sequence; training the residual module using a natural image dataset, and using the trained residual module as the residual module in the deep convolutional neural network to obtain a pre-trained model; training the pre-trained model with remote sensing images as input and the real images corresponding to each remote sensing image as output to obtain a remote sensing image classification model; and using the remote sensing image classification model to perform image classification.

[0004] The above-mentioned scheme has solved the problems of agricultural monitoring and management to a certain extent, but it still has many shortcomings, such as the lack of quality assessment methods for green crop labeling. Summary of the Invention

[0005] The purpose of this invention is to address the above-mentioned problems by providing a reasonably designed method for checking the quality of manual delineation and labeling of green crops.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for checking the quality of manual marking and labeling of green crops, comprising the following steps:

[0007] S1: Obtain UAV imagery and corresponding manual annotation results;

[0008] S2: Vectorize the manually labeled results and store them in shapefile format;

[0009] S3: Use the vectors from the manually labeled results to segment and crop the corresponding images, and number each segmented image so that it uniquely corresponds to the corresponding vector;

[0010] S4: Perform color space transformation on each segmented red, green, and blue three-channel image;

[0011] S5: Generate a binary image based on the range of green color in the transformed HSV space. In the binary image, 1 represents green and 0 represents non-green.

[0012] S6: Post-process the results of step S5, including closing operations, removing isolated points of a specified size, and holes;

[0013] S7: Vectorize the results and calculate the intersection and union areas of the binary image mask processed by S6 and the corresponding labeled result areas, as well as their ratios;

[0014] S8: Calculate the intersection-union ratio of the overall annotation results, and judge the annotation quality based on the intersection-union ratio of individual annotation samples with the overall samples.

[0015] In the above-mentioned method for checking the quality of manual delineation and annotation of green crops, in step S3, during the cutting process using the annotation results, the original pixel values ​​are maintained inside the annotation box, and the values ​​outside the annotation box are assigned to 0.

[0016] In the above-mentioned method for checking the quality of manual delineation and annotation of green crops, step S4 converts the RGB space to the HSV space.

[0017] In the above-mentioned method for checking the quality of manual delineation and annotation of green crops, step S5, which involves generating a binary image using color filtering in the HSV color space, is as follows:

[0018] S1: Set the filtering range of the green color space according to the sample, including the maximum and minimum values ​​of hue, saturation, and brightness;

[0019] S2: Filter each pixel in the transformed image according to the determined color space. If the hue, saturation and brightness all fall within the set color space, mark it as 1; otherwise, mark it as 0.

[0020] In the above-mentioned method for checking the quality of manual marking of green crops, in step S5, the color space used for screening is set according to the actual color distribution, and the values ​​of hue, saturation and brightness need to be within the defined range of the HSV color space.

[0021] In the above-mentioned method for checking the quality of manual delineation and annotation of green crops, in step S6, the mask obtained by color space filtering is post-processed, and the specific steps are as follows:

[0022] S61: Perform morphological closing operations on the binary mask image;

[0023] S62: Median filtering is used to further remove image patches and noise.

[0024] In the above-mentioned method for checking the quality of manual delineation and annotation of green crops, step S61 first performs morphological dilation operation on the image, and then performs morphological erosion operation.

[0025] In the above-mentioned method for checking the quality of manual delineation and labeling of green crops, the filter size used in step S62 is selected according to the actual situation.

[0026] In the above-mentioned method for checking the quality of manual delineation and labeling of green crops, the formula for calculating the intersection-union ratio in step S7 is as follows:

[0027]

[0028] Where A∩B represents the intersection of the areas of the annotation result A and the color space filtering result B, and A∪B represents the union of the areas of the annotation result A and the color space filtering result B.

[0029] In the above-mentioned method for checking the quality of manual marking of green crops, in step S8, the closer the cross-union ratio of a single sheet is to 1, the better the marking quality is, and the closer it is to 0, the worse the marking quality is; the closer the overall cross-union ratio is to 1, the better the marking quality is, and vice versa.

[0030] Compared with existing technologies, the advantages of this invention are: it meets the practical needs of quickly, efficiently, and accurately evaluating the quality of green crop labeling results under large amounts of UAV data; each operation step is clear and repeatable, enabling rapid judgment of labeling quality in agricultural statistics and artificial intelligence dataset creation, providing a quality basis for the use of labeling results; and the inspection and evaluation process only requires specifying the screening threshold to quickly complete tasks on a large scale, resulting in high quality evaluation efficiency. Attached Figure Description

[0031] Figure 1This is a flowchart of the steps of the present invention;

[0032] Figure 2 This is a schematic diagram illustrating the determination of the correctness of the annotation results according to the present invention;

[0033] Figure 3 This is a schematic diagram illustrating the calculation of the crossover-union ratio according to the present invention. Detailed Implementation

[0034] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0035] like Figure 1-2 As shown, a method for checking the quality of manually drawn annotations of green crops is presented. Based on color space selection, the method evaluates the quality of green crop annotations on high-resolution UAV optical images. This method only requires transforming the image to the HSV color space, generating a mask using a specified color range, and calculating the corresponding cross-union ratio. This method can be used to evaluate the quality of manually drawn green crop annotations under various optical images, solving the current problem of a lack of semi-automatic and automatic methods for evaluating the quality of green crop annotations. Specifically, it includes the following steps:

[0036] S1: Obtain the manually labeled results of green crops and the corresponding UAV optical image base map;

[0037] S2: Vectorize the manually labeled results and store them in shapefile format;

[0038] S3: Use the vectors from the manually labeled results to segment and crop the corresponding images, and number each segmented image according to the vector number of the segmentation, so that it uniquely corresponds to the corresponding vector. In this embodiment, a total of 23,079 labeled objects are generated, which means 23,079 segmented image blocks are generated.

[0039] S4: Perform color space transformation on each segmented red, green, and blue three-channel image, converting the RGB space to the HSV space;

[0040] S5: Generate a binary image based on the range of green color in the transformed HSV space. In the binary image, 1 represents green and 0 represents non-green.

[0041] S6: Post-process the results of step S5, including closing operations, removing isolated points of a specified size, and holes;

[0042] S7: Vectorize the results and calculate the intersection and union areas of the binary image mask processed by S6 and the corresponding labeled result areas, as well as their ratios;

[0043] S8: Calculate the intersection-union ratio of the overall annotation results, and judge the annotation quality based on the intersection-union ratio of individual annotation samples with the overall samples.

[0044] Specifically, in step S3, during the segmentation process using the annotation results, the pixel values ​​inside the annotation box are kept as original, while the values ​​outside the annotation box are assigned to 0.

[0045] Furthermore, an image includes the target object, the background, and noise. To directly extract the target object from a multi-valued digital image, a common method is to set a threshold, dividing the image data into two parts: a group of pixels greater than the threshold and a group of pixels less than the threshold—this is image binarization. In step S5, the steps for generating a binary image using color filtering in the HSV color space are as follows:

[0046] S1: Set the filtering range of the green color space according to the sample, including the maximum and minimum values ​​of hue, saturation, and brightness;

[0047] S2: Filter each pixel in the transformed image according to the determined color space. If the hue, saturation, and brightness all fall within the set color space, mark it as 1; otherwise, mark it as 0. The color space used for filtering is set according to the actual color distribution and must satisfy the requirement that the values ​​of hue, saturation, and brightness are within the defined range of the HSV color space.

[0048] The filtering range is determined according to the HSV color space definition based on the desired color. In this example, green needs to be filtered, corresponding to the H range of [35, 77], the S range of [43, 255], and the V range of [46, 255]. Each pixel is iterated through according to the color space range; if it falls within the specified range, it is marked as 1, otherwise as 0. After filtering, a corresponding mask image is obtained for each image. The mask is a partial occlusion of one image using a binary image. In the mask, 1 represents green crops in the filtering, and 0 represents non-green crops.

[0049] In addition, in step S6, the mask obtained by color space filtering is post-processed, and the specific steps are as follows:

[0050] S61: Perform morphological closing operations on the binary mask image. First, perform morphological dilation on the image, and then perform morphological erosion.

[0051] S62: Median filtering is used to further remove image patches and noise. The filter size is selected according to the actual situation. Post-processing is performed on the calculated mask image, mainly including closing operation and median filtering. First, morphological dilation is performed on the obtained mask image. In this example, the selected calculation template size is 3*3. After the closing operation, the holes in the binary image are filled to a certain extent. Because the outermost pixels are generally not processed in the image convolution operation, the center of the convolution kernel cannot reach the outermost pixels. This requires filling the image boundary first, and then performing the convolution operation according to different filling algorithms. The new image obtained is the filled image. Morphological erosion is performed on the dilated binary image. After the dilation-erosion (i.e., opening operation) processing, the gaps are filled to a certain extent.

[0052] After opening operations, the gaps are connected, but some isolated points still exist. To further reduce these noise points, median filtering is performed. This involves selecting the pixel values ​​of a given pixel and its neighboring pixels, sorting these values, and then using the pixel value in the middle position as the current pixel value. This makes the surrounding pixel values ​​closer to the true values, thus eliminating isolated noise points. The size of the filter template can be determined according to the actual situation; in this example, it is 5*5. After this step, the actual detection and filtering results, after color space filtering and post-processing, are obtained.

[0053] Clearly, the formula for calculating the intersection-union ratio in step S7 is as follows:

[0054]

[0055] like Figure 3 As shown, A∩B represents the intersection of the areas of the annotation result A and the color space filtering result B, and A∪B represents the union of the areas of the annotation result A and the color space filtering result B.

[0056] Generally, an IoU greater than or equal to 0.5 indicates an acceptable result, meaning the detection is correct. If the predictor and the actual bounding box perfectly overlap, the IoU is 1, because intersection equals union. A threshold of 0.5 is generally used to determine the accuracy of the predicted bounding box. A higher IoU indicates a more accurate bounding box. In step S8, the closer the IoU value of a single image is to 1, the better the annotation quality; the closer it is to 0, the worse the annotation quality. Similarly, the closer the overall IoU value is to 1, the better the annotation quality, and vice versa. The IoU of each annotated block's mask is calculated, along with the overall IoU. The image annotation quality is then assessed according to the required IoU; in this example, an IoU less than 80% is considered unacceptable for annotation quality.

[0057] In summary, the principle of this embodiment is as follows: after the optical image is segmented, a color space transformation is performed, followed by color space filtering and morphological collision operation post-processing, median filtering, and then the intersection-union ratio of each image block and the whole image is calculated to achieve quality assessment.

[0058] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

[0059] Although this paper makes frequent use of terms such as vectorization and closing operation, the possibility of using other terms is not excluded. These terms are used merely for the convenience of describing and explaining the essence of this invention; interpreting them as any additional limitation would contradict the spirit of this invention.

Claims

1. A method for inspecting the quality of manual marking and labeling of green crops, characterized in that, Includes the following steps: S1: Obtain UAV imagery and corresponding manual annotation results; S2: Vectorize the manually labeled results and store them in shapefile format; S3: Use the vectors from the manually labeled results to segment the corresponding images, and number each segmented image so that it uniquely corresponds to the corresponding vector; S4: Perform color space transformation on each segmented red, green, and blue three-channel image; convert RGB space to HSV space; S5: Generate a binary image based on the range of green color in the transformed HSV color space. In the binary image, 1 represents green and 0 represents non-green. The steps for generating a binary image using color filtering in the HSV color space are as follows: S51: Set the filtering range of the green color space according to the sample, including the maximum and minimum values ​​of hue, saturation, and brightness. S52: Filter each pixel in the transformed image according to the determined color space. If the hue, saturation and brightness all fall within the set color space, mark it as 1; otherwise, mark it as 0. S6: Post-process the results of step S5, including closing operations, removing isolated points of a specified size, and holes; S7: Vectorize the results and calculate the intersection and union areas of the binary image mask processed by S6 and the corresponding labeled result areas, as well as their ratios; S8: Calculate the intersection-union ratio of the overall annotation results, and judge the annotation quality based on the intersection-union ratio of individual annotation samples with the overall samples.

2. The method for inspecting the quality of manual marking and labeling of green crops according to claim 1, characterized in that, In step S3, during the segmentation process using the annotation results, the pixel values ​​inside the annotation box are maintained, while the values ​​outside the annotation box are assigned to 0.

3. The method for inspecting the quality of manual marking and labeling of green crops according to claim 1, characterized in that, In step S5, the color space used for filtering is set according to the actual color distribution, and the values ​​of hue, saturation, and brightness need to be within the defined range of the HSV color space.

4. The method for inspecting the quality of manual marking and labeling of green crops according to claim 1, characterized in that, In step S6, the mask obtained by color space filtering is post-processed, and the specific steps are as follows: S61: Perform morphological closing operations on the binary mask image; S62: Median filtering is used to further remove image patches and noise.

5. A method for inspecting the quality of manual marking and labeling of green crops according to claim 4, characterized in that, Step S61 first performs morphological dilation on the image, and then performs morphological erosion.

6. A method for inspecting the quality of manual marking and labeling of green crops according to claim 4, characterized in that, The filter size used in step S62 is selected according to the actual situation.

7. A method for inspecting the quality of manual marking and labeling of green crops according to claim 1, characterized in that, The formula for calculating the intersection-union ratio in step S7 is as follows: ； Where A∩B represents the intersection of the areas of the annotation result A and the color space filtering result B, and A∪B represents the union of the areas of the annotation result A and the color space filtering result B.

8. The method for inspecting the quality of manual marking and labeling of green crops according to claim 1, characterized in that, In step S8, the closer the cross-over ratio (CUP) of a single sheet is to 1, the better the annotation quality; the closer it is to 0, the worse the annotation quality. The closer the overall CUP is to 1, the better the annotation quality, and vice versa.

Images