A deep learning-based corn canopy coverage automatic extraction method

Abstract

The present application relates to the field of unmanned aerial vehicle image processing, and particularly relates to a corn canopy coverage automatic extraction method based on deep learning, which comprises the following technical scheme: a test point of corn planting is tested in years as a time unit, N planting densities are set for M hybrid varieties or inbred lines of the test point, so as to improve the diversity of the data set and take into account the characteristics of the canopy coverage in different scenes; the canopy coverage extracted by the FVC-Net model, the SimAM attention mechanism module uses the mean and variance of the feature map to calculate the attention weight without increasing any learning parameters, so as to enhance the ability of the model to extract key features, the up-sampling module with SimAM applies SimAM after convolution to improve the expression of important features, and the model is lightened by using partial convolution and reducing the number of convolution channels; for the automatic extraction of the corn canopy coverage in the low-cost unmanned aerial vehicle RGB image, the method is low in cost, does not need to manually extract features, and can be deployed in the edge device with limited computing resources.

Description

Technical Field

[0001] This invention relates to the field of UAV image processing, and in particular to a method for automatically extracting corn canopy coverage based on deep learning. Background Technology

[0002] Canopy cover is an important phenotypic parameter reflecting maize growth status, defined as the proportion of the vertical projection area of ​​the crop canopy onto the ground to the total land area. Accurate canopy cover extraction is crucial for precise monitoring of maize growth and improved yield prediction accuracy.

[0003] In studies on crop canopy cover extraction, canopy area extraction is mainly based on RGB, multispectral, and hyperspectral imagery. Methods based on multispectral and hyperspectral imagery primarily use vegetation indices such as NDVI and EVI to distinguish between the canopy and soil for canopy cover extraction. While this method is suitable for large-scale crop canopy cover extraction, its high sensor cost limits its widespread adoption. RGB sensors, due to their low cost, are widely used in crop canopy cover extraction research. Specifically, RGB sensor-based methods for maize canopy cover extraction can be categorized into three types: threshold-based area extraction methods, edge and region-based area extraction methods, and image feature and machine learning-based area extraction methods. These maize canopy cover extraction methods often require manual setting of thresholds and extraction features, necessitating different parameters for different environmental conditions, resulting in poor robustness. Therefore, a robust and automatically extractable method is needed for accurate maize canopy cover extraction.

[0004] Deep learning, with its ability to automatically extract features, has gradually become the preferred method for extracting the area of ​​target crops. Deep learning methods have been successfully applied to weed area extraction, plant fruit area extraction, and plant disease extraction. Deep learning models are also increasingly being used in research on crop canopy coverage extraction. The improved U-Net model MCAC-Unet, used for maize canopy coverage extraction, achieved an average pixel accuracy of 93.85% and an average intersection-over-union ratio of 87.51%. Virtual RGB images generated by integrating the SegFormer segmentation model with the CycleGAN model have enabled cross-spatial resolution canopy coverage extraction for rice crops.

[0005] Although the aforementioned studies have focused on using deep learning models to extract crop canopy coverage, current research still faces the following challenges: insufficient dataset diversity, especially when crop growth stages, varieties, light intensity, and weather conditions vary significantly, existing datasets often fail to address the characteristics of canopy coverage under these scenarios; in field environments, edge devices typically possess limited computing resources (such as processing power, storage space, and power consumption), making the excessive number of parameters in deep learning models a major limiting factor in their use.

[0006] In view of this, we propose a deep learning-based method for automatic extraction of maize canopy cover to solve the existing problems. Summary of the Invention

[0007] The purpose of this invention is to provide an automatic method for extracting maize canopy coverage based on deep learning, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: an automatic extraction method for maize canopy coverage based on deep learning, comprising the following steps: selecting experimental sites for maize planting, acquiring RGB images of the maize canopy and obtaining orthophotos of the entire experimental site, then obtaining RGB images of the canopy at the plot scale and labeling them, then dividing the labeled images into training and testing sets, using the training set to train an FVC-Net model, and using the trained model to segment the maize canopy in the testing set, thereby obtaining the canopy coverage extracted by the FVC-Net model; wherein, the FVC-Net model combines the SimAM attention mechanism and uses MobileNetV4 as the encoder and is a lightweight model with reduced channel count. Its core components include a SimAM attention mechanism module, an upsampling module with SimAM, and MobileNetV4 with partial convolution as the encoding layer and reduced convolution channel count.

[0009] Furthermore, a K-year experiment was conducted at the maize planting site, with N planting densities set for M hybrid varieties or inbred lines at the site.

[0010] Furthermore, the RGB images of the drone are stitched together to obtain an orthophoto of the entire test site.

[0011] Furthermore, the orthophoto image of the entire experimental site was cropped according to the location of the planting area to obtain a canopy RGB image at the micro-scale.

[0012] Furthermore, the labeled images are divided into training and testing sets in a 7:3 ratio.

[0013] Furthermore, during the training of the FVC-Net model, the training set is divided into a training set and a validation set in a 9:1 ratio.

[0014] Furthermore, the learning rate was set to 0.001, the batch size to 8, the momentum to 0.9, and the number of rounds to 100.

[0015] Furthermore, the ratio of the number of pixels in the segmented corn canopy to the total number of pixels in the cropped UAV image is calculated to obtain the canopy coverage extracted by the FVC-Net model.

[0016] Furthermore, the ratio of the number of manually marked corn canopy pixels to the total number of pixels in the cropped UAV image is calculated to obtain the manually marked canopy coverage.

[0017] Furthermore, the canopy coverage extracted by the model and the canopy coverage extracted by manual labeling were linearly fitted to verify the statistical accuracy of the model's automatic canopy coverage extraction.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This invention conducts experiments on maize planting sites over a year, setting N planting densities for M hybrid varieties or inbred lines at each site to increase dataset diversity and accommodate canopy cover characteristics under different scenarios. The canopy cover is extracted using the FVC-Net model. The SimAM attention mechanism module calculates attention weights using the mean and variance of the feature maps without adding any learning parameters, enhancing the model's ability to extract key features. The SimAM-enabled upsampling module applies SimAM after convolution to improve the expression of important features. Partial convolution and a reduction in the number of convolution channels achieve model lightweighting. For low-cost automatic extraction of maize canopy cover from UAV RGB imagery, compared to existing canopy cover extraction methods, this method is low-cost, requires no manual feature extraction, and can be deployed on edge devices with limited computing resources. Attached Figure Description

[0019] Figure 1 The flowchart shows a method for extracting maize canopy cover based on the FVC-Net model and UAV RGB imagery. Figure 2 This is a flowchart of the drone image processing workflow; Figure 3 Annotate example images for Eiseg software; Figure 4 This is an architecture diagram of the FVC-Net model; Figure 5 Extract a schematic diagram of canopy coverage from the model; Figure 6 To extract the counting accuracy of canopy coverage for the model. Detailed Implementation

[0020] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments. Example

[0021] like Figure 1 As shown, a method for automatically extracting maize canopy cover based on deep learning includes the following steps: A: Corn planting was carried out at two experimental sites in Xinxiang and Xinjiang. The Xinxiang experimental site conducted a 3-year trial, while the Xinjiang experimental site conducted a 1-year trial. The 20 hybrid varieties at the Xinxiang experimental site were planted at 4 different densities, while the 580 inbred lines at the Xinjiang experimental site were planted at 1 different density.

[0022] B: As Figure 2 As shown in Figure a, RGB images of corn canopies were acquired using a DJI M600 equipped with a Sony Alpha 7 II camera.

[0023] C: such as Figure 2 As shown in b and 2c, the UAV RGB images are stitched together in Agisoft Metashape software based on the coordinates of the ground control points to obtain the orthophoto of the entire test site.

[0024] D: such as Figure 2 As shown in d, the orthophoto in step C is cropped according to the location of the planting area in ArcGIS 10.8 software to obtain a canopy RGB image at the micro-scale.

[0025] E: such as Figure 3 As shown, the corn canopy was annotated in the cell-scale RGB images obtained in step D using Eiseg software, and a total of 32,147 cell-scale corn canopy images were collected.

[0026] F: Divide the labeled images into training and testing sets in a 7:3 ratio.

[0027] G: Train the FVC (Fractional Vegetation Cover)-Net model using the training set. Divide the training set into a training set and a validation set in a 9:1 ratio. Use the validation set to monitor model training and prevent overfitting. Figure 4As shown, the FVC-Net model combines the SimAM attention mechanism with MobileNetV4 as the encoder and is a lightweight model with a reduced number of channels. Its core components include a SimAM attention mechanism module, an upsampling module with SimAM, and MobileNetV4 with partial convolution as the encoding layer and a reduced number of convolution channels. The SimAM attention mechanism module calculates attention weights using the mean and variance of the feature maps without increasing any learning parameters, enhancing the model's ability to extract key features. The upsampling module with SimAM applies SimAM after convolution to improve the expression of important features. The use of partial convolution and a reduction in the number of convolution channels achieves the lightweighting of the model. During training, the learning rate is set to 0.001, the batch size is set to 8, the momentum is set to 0.9, and the epochs are set to 100.

[0028] H: Use the trained model to extract the number of maize canopies in the test set.

[0029] In semantic segmentation, there are three commonly used model performance evaluation metrics: recall, precision, and intersection-over-union (IoU). These metrics are all calculated based on the confusion matrix, and an example of the confusion matrix in this application is shown in Table 1.

[0030] Table 1 Confusion Matrix

[0031] Recall is the proportion of correctly segmented canopy pixels (TP) out of all actual canopy pixels (TP+FN), and the formula is: Recall = TP / (TP + FN).

[0032] Precision is the proportion of correctly segmented canopy pixels (TP) out of all predicted canopy pixels (TP+FP), and the formula is: Precision = TP / (TP + FP).

[0033] Intersection over Union (IoU) is a standard metric for evaluating the accuracy of semantic segmentation. It combines recall and precision, and is expressed as: IoU = TP / (TP + FP + FN).

[0034] The above three metrics are calculated for each image, and the average value of all images is calculated to represent the segmentation performance of the model.

[0035] I: Such as Figure 5As shown, the ratio of the number of corn canopy pixels segmented in step H to the total number of pixels in the cropped UAV image is calculated to obtain the canopy coverage extracted by the FVC-Net model.

[0036] J: Calculate the ratio of the number of manually marked corn canopy pixels in step E to the total number of pixels in the cropped UAV image to obtain the manually marked canopy coverage.

[0037] K: such as Figure 6 As shown, the canopy coverage extracted by the model in step I and the canopy coverage extracted by manual labeling in step J are linearly fitted to verify the statistical accuracy of the automatic extraction of canopy coverage by the model.

[0038] Two metrics are used to quantify the statistical accuracy of automatic canopy coverage extraction: relative root mean square error (rRMSE) and coefficient of determination (R²). 2 ):

[0039]

[0040] Where n is the number of test images, and These are the actual and model-segmented canopy coverage, respectively. yes The average value.

[0041] Test results show that the maize canopy coverage segmentation method based on the FVC-Net model and UAV RGB imagery proposed in this application produces maize canopy coverage that is close to the actual canopy coverage and can reflect the real maize canopy coverage in the field.

[0042] The above specific embodiments are merely several preferred embodiments of the present invention. Based on the technical solutions of the present invention and the relevant teachings of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A method for automatically extracting maize canopy cover based on deep learning, characterized in that, The operation steps include: selecting experimental sites for maize planting, acquiring RGB images of the maize canopy and obtaining orthophotos of the entire experimental site, then obtaining RGB images of the canopy at the plot scale and labeling them, then dividing the labeled images into training and testing sets, using the training set to train the FVC-Net model, and using the trained model to segment the maize canopy in the testing set, thereby obtaining the canopy coverage extracted by the FVC-Net model; wherein, the FVC-Net model combines the SimAM attention mechanism and uses MobileNetV4 as the encoder and is a lightweight model with a reduced number of channels. Its core components include the SimAM attention mechanism module, the upsampling module with SimAM, and MobileNetV4 with partial convolution as the encoding layer and reduced number of convolution channels.

2. The method for automatically extracting maize canopy coverage based on deep learning according to claim 1, characterized in that: A K-year experiment was conducted at a maize planting site, with N planting densities set for M hybrid varieties or inbred lines at the site.

3. The method for automatically extracting maize canopy cover based on deep learning according to claim 1, characterized in that: The RGB images from the drone were stitched together to obtain an orthophoto of the entire test site.

4. The method for automatically extracting maize canopy coverage based on deep learning according to claim 1, characterized in that: The orthophoto of the entire experimental site was cropped according to the location of the planting area to obtain a canopy RGB image at the micro-scale.

5. The method for automatically extracting maize canopy coverage based on deep learning according to claim 1, characterized in that: The labeled images are divided into training and testing sets in a 7:3 ratio.

6. The method for automatically extracting maize canopy cover based on deep learning according to claim 1, characterized in that: During the training of the FVC-Net model, the training set is divided into a training set and a validation set in a 9:1 ratio.

7. The method for automatically extracting maize canopy cover based on deep learning according to claim 6, characterized in that: The learning rate was set to 0.001, the batch size to 8, the momentum to 0.9, and the number of rounds to 100.

8. The method for automatically extracting maize canopy coverage based on deep learning according to claim 1, characterized in that: The canopy coverage extracted by the FVC-Net model is obtained by calculating the ratio of the number of pixels in the segmented corn canopy to the total number of pixels in the cropped UAV image.

9. The method for automatically extracting maize canopy cover based on deep learning according to claim 8, characterized in that: The ratio of the number of manually marked corn canopy pixels to the total number of pixels in the cropped drone image is used to calculate the manually marked canopy coverage.

10. The method for automatically extracting maize canopy cover based on deep learning according to claim 9, characterized in that: Linear fitting was performed between the canopy coverage extracted by the model and the canopy coverage extracted by manual labeling to verify the statistical accuracy of the automatic extraction of canopy coverage by the model.

Images