Unmanned aerial vehicle RGB image and deep learning-based corn whole growth period LAI estimation method

Abstract

The invention relates to the field of unmanned aerial vehicle image processing, in particular to a corn whole growth period LAI estimation method based on unmanned aerial vehicle RGB images and deep learning, and adopts the technical scheme that a corn planting test point is tested by taking a year as a time unit, and the test point comprises a plurality of corn varieties which are composed of a plurality of inbred lines and a plurality of hybrids; therefore, the diversity of a data set is improved, and features in different growth stages are considered. A plurality of deep learning models are selected for standardized classification output, classification heads of all the models are modified into output layers consistent with the number of labels, ImageNet pre-training weights are used for transfer learning initialization, adjustment is carried out on a data set, the learning rate is adjusted within the range of 0.0001-0.001, the step length is 0.0001, and the batch size is changed within the range of 2-64. Selecting an optimal parameter combination generating the lowest convergence loss, and screening out an optimal model parameter with an optimal convergence effect on the verification set; the method has extremely high environmental adaptability, and can realize stable monitoring in cross-year seasons.

Description

Technical Field

[0001] This invention relates to the field of UAV image processing, and in particular to a method for estimating the LAI (Layer Image Size) of maize throughout its entire growth period based on UAV RGB imagery and deep learning. Background Technology

[0002] As the world's highest-yielding food crop, maize is not only a core food source for humans and a source of livestock feed, but also a crucial foundation for industrial raw materials. Against the backdrop of current severe global climate change, shrinking arable land resources, and continuous population growth, accurately monitoring the growth and development of maize is of profound significance for ensuring global food security and promoting the sustainable development of precision agriculture. Leaf area index (LAI), defined as the total area of ​​green leaves on a single side of a unit land surface, is the most fundamental biophysical parameter characterizing plant canopy structure, photosynthetic capacity, transpiration rate, and water-heat balance. Quantitatively obtaining spatiotemporally continuous regional LAI is crucial for monitoring maize growth and estimating yield.

[0003] Because direct measurement methods struggle to acquire long-term, large-area LAI (Local Area Index) observation data, the technological approach for obtaining regional-scale maize LAI is increasingly shifting towards low-altitude remote sensing monitoring using unmanned aerial vehicles (UAVs). UAV platforms, with their significant advantages such as high spatiotemporal resolution, operational flexibility, and minimal cloud interference, have become a core tool for high-throughput phenotypic analysis in the field. Among the many UAV sensors available, RGB cameras are widely used for large-scale farmland monitoring due to their low cost, ease of maintenance, simple data structure, and lack of need for complex radiometric calibration.

[0004] Existing methods for estimating LAI based on remote sensing images mainly fall into three categories: (1) physical model method, which is based on radiative transfer theory and inverts LAI by simulating the scattering and absorption process of light inside the canopy; (2) data assimilation method, which usually combines remote sensing observation information with crop growth models and continuously adjusts the model state variables through optimization algorithms to achieve dynamic prediction of LAI; (3) statistical regression model method, which is currently the most widely used method. Its core logic lies in extracting the spectral vegetation index, texture features or structural parameters of the image and using multiple linear regression or machine learning algorithms to construct a mathematical mapping relationship between features and LAI. However, the inversion logic of existing physical models and data assimilation methods is complex and prone to problems such as ill-conditioned inversion and dependence on initialization parameters. Traditional statistical models are highly dependent on the visible light index and have a serious spectral saturation effect when the canopy closure is high in the middle and late stages of maize growth. Moreover, artificially designed shallow features are difficult to characterize the nonlinear relationship between complex canopy structures and LAI, resulting in insufficient robustness of the full growth period estimation.

[0005] In recent years, with the rapid development of computer vision technology, deep learning methods have been widely applied to agricultural tasks due to their significant advantage of automatically extracting high-dimensional and non-linear features from images. Deep learning models construct an end-to-end mapping architecture through multiple layers of neurons, bypassing the tedious manual feature extraction work and directly mining spatial texture and morphological features related to LAI from the original RGB images, demonstrating potential that surpasses traditional methods.

[0006] Although some progress has been made in estimating LAI using remote sensing imagery, current research still faces the following challenges: existing studies are mostly focused on specific growth stages (such as vegetative growth or flowering), lacking systematic validation of the entire growth cycle, especially the later stages of reproductive growth; and the exploration of optimal architecture for end-to-end models on edge devices in terms of inference efficiency, lightweight design, and cross-year tasks is still insufficient, limiting the practical deployment of this technology in UAV terminals and real-time inversion systems.

[0007] In view of this, we propose a method for estimating the LAI (Label Articulation Area) of maize throughout its entire growth period based on UAV RGB imagery and deep learning to address the existing problems. Summary of the Invention

[0008] The purpose of this invention is to provide a method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning, in order to solve the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning. The operation steps include: selecting experimental sites for maize planting, acquiring RGB images of the maize canopy and LAI data, obtaining orthophotos of the entire experimental site, obtaining canopy RGB images at the plot scale and performing LAI gradient labeling, dividing the dataset into training, validation, and test sets, selecting several deep learning models as standardized classification outputs, modifying the classification heads of all models to output layers consistent with the number of labels, and selecting the optimal model parameters that have the best convergence performance on the validation set.

[0010] Furthermore, a K-year trial was conducted at a maize planting site, which included (M+N) maize varieties, consisting of M inbred lines and N hybrids.

[0011] Furthermore, the RGB images of the drone are stitched together to obtain an orthophoto of the entire test site.

[0012] Furthermore, the orthophoto effect of the entire experimental site was cropped according to the location of the planting area to obtain a canopy RGB image at the micro-scale.

[0013] Furthermore, in LAI gradient labeling, the measured LAI values ​​are discretized according to a preset step size of 0.1, divided into multiple different LAI gradient levels, and the cell images are mapped one by one to their corresponding level labels.

[0014] Furthermore, the dataset is divided into training, validation, and test sets in a 7:1:2 ratio.

[0015] Furthermore, after dividing the dataset into training, validation, and test sets, data augmentation was performed on the training set. The augmentation techniques used included 90°, 180°, and 270° rotations, as well as horizontal and vertical flips.

[0016] Furthermore, the selected deep learning models include six deep learning models: VGG19, ResNet101, EfficientNet, MobileNetV3, GhostNet, and Sequencer.

[0017] Furthermore, in the construction of the deep learning model, the ImageNet pre-trained weights are used for transfer learning initialization, and adjustments are made on the dataset. The learning rate is adjusted in the range of 0.0001 to 0.001, the step size is 0.0001, and the batch size varies between 2 and 64. The optimal combination of parameters that produces the lowest convergence loss is selected.

[0018] Furthermore, after the deep learning model is built, the accuracy and computational efficiency of several deep learning models on the test set are evaluated, and the trained optimal deep learning weights are used to directly predict unknown samples. By calculating the difference between the predicted LAI value and the measured LAI value, the robustness of the model under interannual transfer is verified.

[0019] Compared with the prior art, the beneficial effects of the present invention are: This invention conducts experiments on maize planting sites over a yearly timeframe, including multiple maize varieties comprised of several inbred lines and hybrids to enhance dataset diversity and accommodate characteristics at different growth stages. Several deep learning models are selected as standardized classification outputs, with the classification heads of all models modified to match the number of labels. Transfer learning initialization is performed using ImageNet pre-trained weights, and adjustments are made on the dataset. The learning rate is adjusted from 0.0001 to 0.001, the step size is 0.0001, and the batch size varies from 2 to 64. The optimal parameter combination producing the lowest convergence loss is selected, and the best model parameters with the best convergence performance on the validation set are identified. For low-cost LAI estimation in UAV RGB imagery, compared to existing LAI estimation methods, this method is low-cost, requires no manual feature extraction, has high accuracy and stability, exhibits strong environmental adaptability, and can achieve robust monitoring across yearly seasons. Attached Figure Description

[0020] Figure 1 A schematic diagram of the process for estimating the LAI (Label Intake) of maize throughout its entire growth period based on deep learning and UAV RGB imagery; Figure 2 The process of acquiring and processing drone images; Figure 3 The composition of SunScan and the method for acquiring LAI data; Figure 4 This represents the number of original image samples corresponding to each LAI value in deep learning. Figure 5 For example of data augmentation; Figure 6 The result of estimating LAI for deep learning; Figure 7 To evaluate the interannual transferability of LAI models for deep learning estimation. Detailed Implementation

[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments. Example 1

[0022] like Figure 1 As shown, the method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning includes the following steps: A. Experimental Design: The experimental area is located at the Comprehensive Experimental Base of the Chinese Academy of Agricultural Sciences in Xinxiang City, Henan Province, China. Field trials were conducted in 2022, 2023, and 2024. The maize trial in 2022 included 34 maize varieties, consisting of 17 inbred lines and 17 hybrids. In 2023 and 2024, 20 hybrid maize varieties were planted each year.

[0023] B. Data Acquisition: such as Figure 2 As shown in Figure a, RGB images of the corn canopy were acquired using a DJI M600 equipped with a Sony Alpha 7 II camera, covering the entire growth period of corn (from jointing stage to milk stage); Figure 3 As shown, LAI data were obtained in the field using the SunScan canopy analyzer.

[0024] C. Orthophoto stitching: such as Figure 2 As shown in c, the original aerial images obtained in step B are imported into Agisoft Metashape software for stitching to obtain an orthophoto of the entire test site. Meanwhile, as... Figure 2 As shown in b, high-precision georegistration and spatial correction of the image are performed using the deployed ground control points, providing a spatial reference for subsequent precise masking and cropping at the cell scale.

[0025] D. Cellular cropping: In ArcGIS 10.8 software, crop the orthophoto from step C according to the location of the planting area to obtain a cell-scale canopy RGB image.

[0026] E. LAI gradient labeling: such as Figure 4 As shown, the measured LAI values ​​are discretized according to a preset step size of 0.1, divided into multiple different LAI gradient levels, and the cell images are mapped one by one to their corresponding level labels.

[0027] F. Dataset Construction: The combined dataset from 2022 and 2023 contains 1562 images covering 55 categories. This dataset serves as the primary source for model development and is divided into training, validation, and test sets in a 7:1:2 ratio. For the 2024 growing season, a total of 1232 images covering 75 categories are included. This dataset is specifically reserved for evaluating the time transferability and cross-year generalization performance of deep learning models.

[0028] G. Data Augmentation: Data augmentation was performed on the training set, such as... Figure 5 As shown, the enhancement techniques used include 90°, 180°, and 270° rotations, as well as horizontal and vertical flips, which enlarged the original image to 6402 frames to improve the model's adaptability to different shooting angles and lighting conditions.

[0029] H. Deep Learning Model Construction: Six deep learning models (VGG19, ResNet101, EfficientNet, MobileNetV3, GhostNet, and Sequencer) were selected for standardized classification output. The classification heads of all models were modified to have the same number of labels as the output layer in step F. A transfer learning strategy was adopted, using ImageNet pre-trained weights for transfer learning initialization. Fine-tuning was performed on the 2022-2023 dataset, with the learning rate adjusted from 0.0001 to 0.001 (step size of 0.0001) and the batch size varying from 2 to 64. The optimal parameter combination that produces the lowest convergence loss was selected, and the optimal model parameters with the best convergence performance on the validation set were screened. The optimal parameter combination in this application is shown in Table 1. The model automatically extracts high-order texture and spatial structure features through convolutional layers, and then outputs the probability distribution of each level through the Softmax function.

[0030] Table 1 Optimization Parameters of Deep Learning Models

[0031] I. Model Evaluation: (e.g.) Figure 6 As shown, the accuracy and computational efficiency of six deep learning models on the test set are evaluated.

[0032] Several metrics are used to quantify the accuracy of LAI estimation, namely the coefficient of determination (R²). 2 ), Root Mean Square Error (RMSE) and Relative Root Mean Square Error (rRMSE): , , ,in, and These are the measured value and the predicted value of the i-th sample, respectively. is the average of the measured values, and n is the sample size.

[0033] In addition to core inversion accuracy, this application also comprehensively evaluated six deep learning models from the dimensions of computational efficiency and model complexity. Specific indicators include dataset construction time, modeling time, total time, total FLOPs, and total parameters. All six deep learning models demonstrated excellent performance and application potential in corn LAI estimation, as shown in Table 2.

[0034] Table 2 Performance Evaluation of Deep Learning Models

[0035] J. Cross-year transferability verification: Using the optimal deep learning weights trained in step H for 2022-2023, directly predict the unknown samples for 2024, such as... Figure 7 As shown, the robustness of the model under interannual migration is verified by calculating the difference between the predicted LAI value and the measured LAI value.

[0036] Test results show that the deep learning model performs excellently in both feature representation and environmental adaptability. Leveraging its ability to automatically extract high-level semantic features, the deep learning model effectively overcomes the interference of overlapping closed canopies, achieving high-precision inversion. In cross-year transfer validation, the deep learning model can extract structure-invariant features and maintains excellent generalization performance under challenging environments, fully demonstrating its application potential in long-term agricultural monitoring.

[0037] The above specific embodiments are merely several preferred embodiments of the present invention. Based on the technical solutions of the present invention and the relevant teachings of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. A method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning, characterized in that... The operation steps include: selecting experimental sites for corn planting, acquiring RGB images of the corn canopy and LAI data, obtaining orthophotos of the entire experimental site, obtaining RGB images of the canopy at the micro-scale and performing LAI gradient labeling, dividing the dataset into training, validation and test sets, selecting several deep learning models as standardized classification outputs, modifying the classification heads of all models to the same number of labels as the output layer, and selecting the optimal model parameters that have the best convergence performance on the validation set.

2. The method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning according to claim 1, characterized in that: A K-year trial was conducted at a maize planting site, which included (M+N) maize varieties, consisting of M inbred lines and N hybrids.

3. The method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning according to claim 1, characterized in that: The RGB images from the drone were stitched together to obtain an orthophoto of the entire test site.

4. The method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning according to claim 1, characterized in that: The orthophotos of the entire experimental site were cropped according to the location of the planting area to obtain a canopy RGB image at the micro-scale.

5. The method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning according to claim 1, characterized in that: In LAI gradient labeling, the measured LAI values ​​are discretized according to a preset step size of 0.1, divided into multiple different LAI gradient levels, and the cell images are mapped one by one to their corresponding level labels.

6. The method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning according to claim 1, characterized in that: The dataset was divided into training, validation, and test sets in a ratio of 7:1:

2.

7. The method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning according to claim 1, characterized in that: After dividing the dataset into training, validation, and test sets, data augmentation was performed on the training set. The augmentation techniques used included 90°, 180°, and 270° rotations, as well as horizontal and vertical flips.

8. The method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning according to claim 1, characterized in that: The selected deep learning models include six deep learning models: VGG19, ResNet101, EfficientNet, MobileNetV3, GhostNet, and Sequencer.

9. The method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning according to claim 1, characterized in that: In the construction of the deep learning model, the ImageNet pre-trained weights are used for transfer learning initialization. The model is then adjusted on the dataset with the learning rate ranging from 0.0001 to 0.001, the step size being 0.0001, and the batch size varying between 2 and 64. The optimal combination of parameters that produces the lowest convergence loss is selected.

10. The method for estimating the LAI (Label Area Index) of maize throughout its entire growth period based on UAV RGB imagery and deep learning according to claim 1, characterized in that: After the deep learning model is built, the accuracy and computational efficiency of several deep learning models on the test set are evaluated. The trained optimal deep learning weights are used to directly predict unknown samples. The robustness of the model under interannual transfer is verified by calculating the difference between the predicted LAI value and the measured LAI value.

Images