Unmanned aerial vehicle RGB image cotton yield prediction method based on attention mechanism

Abstract

The invention provides an unmanned aerial vehicle RGB image cotton yield prediction method based on an attention mechanism. The unmanned aerial vehicle RGB image cotton yield prediction method comprises the steps of 1, selecting a proper prediction object; and step 2, spraying a defoliating agent with a reasonable dosage. And step 3, obtaining test data. And 4, constructing a yield prediction model. And 5, predicting the cotton yield. And step 6, evaluating the model. According to the method, the unmanned aerial vehicle carries the visible light camera to collect the cotton canopy RGB image, the problem that the prediction precision of a single feature is not high is solved, image color features and texture features are extracted and combined, data are processed through a principal component analysis method, a CNN-Bi LSTM cotton yield prediction model based on an attention mechanism is established, and the cotton yield prediction precision is improved. And comparing and analyzing a plurality of deep learning yield prediction models of different cotton varieties, and screening out an optimal prediction model, so as to provide reference for rapidly predicting the cotton yield by using the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention belongs to the technical field of crop yield prediction, and in particular relates to a method for predicting cotton yield using unmanned aerial vehicle RGB images based on an attention mechanism. Background Art

[0002] As a key pillar of the agricultural economy, the accurate prediction of cotton yield provides key information and guidance for agricultural producers, decision makers and market participants in agricultural planning, market operations and risk management. The traditional yield method is sampling survey, which requires large-scale destructive sampling of cotton fields and consumes a lot of time and manpower. In recent years, remote sensing technology has been increasingly used in agricultural production monitoring. Satellite remote sensing technology cannot meet the needs of field-scale monitoring due to its susceptibility to weather. UAV remote sensing technology has the advantages of portability, low cost and high resolution, which makes up for this shortcoming and has become a hot topic in precision agriculture research.

[0003] At present, domestic and foreign scholars have conducted a lot of research on crop phenotypic information based on drones equipped with different image collectors (visible light cameras, thermal infrared cameras, and multispectral cameras), and have achieved effective evaluation of crop growth parameters such as biomass and leaf area index. Crop yield prediction has become an important research topic for drone production monitoring. For example, Zhang Jing et al. extracted color and texture features from cotton canopy images and established regression models with cotton yield. The results showed that the stepwise regression model established by color and texture feature variables had the best effect, with an average absolute percentage error of 2.31%. Liu Xinyi et al. obtained the color and texture feature index of wheat fields through RGB image processing, and established a wheat yield prediction model by analyzing the relationship between different color and texture feature indices and wheat yield. It was found that compared with the single color and texture index yield prediction model, the linear regression prediction model established by combining color and texture feature indices had the highest accuracy, with the determination coefficient increased by 23.71% and 16.27% respectively.

[0004] However, the linear regression model has a simple structure and is difficult to expand its application or combine with other features. Xiang Youzhen et al. extracted five vegetation indices with good correlation with yield based on multispectral images from drones, combined with multiple machine learning algorithms, and determined that the random forest (RF) soybean yield prediction model had the highest accuracy, with a determination coefficient of 0.818. In addition, Yang Nan et al. extracted spectral and texture features based on RGB images collected by visible light cameras, and used machine learning models to screen out the optimal yield estimation model for wheat. The results show that the yield prediction model based on random forests is the most effective, with an average absolute percentage error of 1.17%, and multi-feature fusion has higher estimation accuracy than a single feature.

[0005] Because deep learning can make better use of the large number of features in the data set, when there is sufficient data and many relevant features, its prediction effect is usually better than traditional machine learning. It is currently used by many scholars for crop yield prediction.

[0006] Deep learning methods use multi-layer neural networks such as convolutional neural networks (CNN) and long short-term memory neural networks (LSTM). For example, Sun Shaojie et al. used convolutional neural networks to analyze and extract winter wheat planting areas from Sentinel-2 remote sensing data, fused the obtained planting area distribution data with the cultivated land distribution data, extracted and predicted the yield characteristics of the fused data, and found that the determination coefficient of the winter wheat yield prediction method based on convolutional neural networks reached above 0.87; He Xiaohui et al. combined recurrent neural networks (RNN) to select the growth characteristics of winter wheat from 2001 to 2018 and made regression predictions on its yield, which reduced the root mean square error (RMSE) of convolutional neural networks by 20.13%; Wang Xu et al. used remote sensing data, meteorological data and county yields from China's main winter wheat producing areas from 2001 to 2018 to construct a long short-term memory (LSTM)-based prediction method. Liu Haijiao et al. constructed a mango yield prediction model based on a bidirectional long short-term memory network (Bidirectional Long Short-Term Memory, BiLSTM), and compared it with the traditional machine learning model. The results showed that the LSTM model based on all data had the highest accuracy, with an average determination coefficient of 0.853. They also constructed a mango yield prediction model based on a bidirectional long short-term memory network (Bidirectional Long Short-Term Memory, BiLSTM), and selected the factors with higher correlation as the input of the model through correlation analysis between mango yield and meteorological factors. They found that the model had higher prediction accuracy than other machine learning models.

[0007] In addition, hybrid models based on deep learning, represented by the CNN-BiLSTM model, have gradually attracted attention in the field of yield prediction. For example, Dai Jianguo et al. used images of cotton seedling, bud and flowering periods as time series data sets to construct a hybrid model CNN-BiLSTM to estimate cotton yield. They found that compared with the single-structure BiLSTM, the model's determination coefficient increased from 0.851 to 0.885. Then, research on crop yield prediction based on deep learning is mostly focused on food crops such as wheat and corn, and there are relatively few studies on cotton. In addition, existing research on cotton yield prediction is mostly based on the establishment of a model for the province-wide yield based on years of meteorological and cotton phenological information, and rarely on the estimation of cotton yield at the field scale using drone RGB images.

[0008] In summary, the current crop yield forecasting has the following problems:

[0009] (1) The single color or texture feature index yield prediction model has low reliability in complex backgrounds.

[0010] (2) Existing prediction methods mostly use traditional algorithms such as linear regression and empirical models. However, agricultural characteristic data usually contain a large number of nonlinear relationships, time series characteristics, and multi-source heterogeneous data. Traditional machine learning methods usually rely on manually designed features, which requires a lot of domain knowledge and trial and error.

[0011] (3) The prediction of cotton yield is different from that of wheat, apple and other crops, because the cotton canopy image is collected from the cotton plant, but the cotton leaves are relatively large and dense. If the cotton is not processed in the early stage, the collected canopy image will be relatively simple, with high data similarity, resulting in inaccurate prediction results. At the same time, if the cotton plants are not properly handled, the cotton canopy leaves will fall off in large numbers, the canopy information will be reduced, and the soil between the ridges will enter the collected image, which will also cause inaccurate predictions. Summary of the invention

[0012] The purpose of the present invention is to solve the defects of the above-mentioned prior art, and provide a method for predicting cotton yield from drone RGB images based on an attention mechanism. The RGB images of cotton canopies with different leaf densities and varieties are collected by a drone equipped with a visible light camera, which solves the problem of low prediction accuracy of a single feature. The color features and texture features of the image are extracted and combined, and the data is processed by the principal component analysis method. A CNN-BiLSTM cotton yield prediction model based on the attention mechanism is established. Several deep learning yield prediction models of different cotton varieties are compared and analyzed, and the optimal prediction model is screened out, aiming to provide a reference for quickly predicting cotton yield using drones.

[0013] The present invention adopts the following technical solution:

[0014] The cotton yield prediction method of drone RGB images based on attention mechanism includes the following steps:

[0015] Step 1. Choose the appropriate prediction object

[0016] Select cotton fields with reasonable film width, row spacing and plant spacing, and choose the Xinluzao 33 variety that is resistant to lodging and insensitive to defoliants. The film width is 2.05m, the row spacing is (66+10)cm, and the plant spacing is 5.6cm.

[0017] Step 2. Spray defoliants at appropriate doses at different time points

[0018] The defoliant is toloxanil.

[0019] Step 3. Experimental data acquisition

[0020] 301.Use drones to acquire and process images

[0021] Before the cotton harvest, an aerial drone was used to collect high-definition color images of the cotton field in step 1. During image collection, the lens was pointed vertically downward, and photos were taken at equal time intervals. The flight altitude was set to 10 m, the heading and lateral overlap was 80%, the camera shutter speed was 1 / 240 s, the ISO sensitivity was 100, and the image size was 5472 × 3648 pixels in JPG format.

[0022] The acquired images were stitched into orthophotos using software, the stitched orthophotos were stored in TIFF format, and the stitched cotton field images were cropped.

[0023] 302. Image color index feature parameter calculation

[0024] The color feature values ​​of the RGB cotton canopy image were extracted using software. First, the image was normalized and the r, g, and b values ​​were calculated. Then, 12 visible light color feature indices were calculated based on different combinations:

[0025]

[0026] Where r, g, and b are chromaticity coordinates, and R, G, and B represent the digital quantization values ​​of the red, green, and blue channels, respectively.

[0027] The following are the 12 visible light color indices:

[0028] Visible atmospherically resistant index (VARI), Excess green vegetation index (EXG), Excess red vegetation index (EXR), Excess blue vegetation index (EXB), Excess green minus excess red vegetation index (EXGR), Normalized green-red difference index (NGRDI), Green red vegetation index (GRVI), Woebbecke index (WI), Kawashima index (IKAW), Green leaf algorithm (GLI), Red green blue vegetation index (RGBVI) and Vegetativen (VEG).

[0029] Their calculation formulas are as follows:

[0030]

[0031] 303. Image Texture Index Feature Parameter Calculation

[0032] The gray-level co-occurrence matrix is ​​used to extract the energy, contrast, entropy, and inverse difference moment features of the image. The Tamura texture feature is used to extract the roughness, contrast, and orientation features of the image, a total of 7 image texture parameter features.

[0033] 304.Data Production

[0034] 24 high-quality RGB canopy images with different leaf densities were obtained, and the original canopy image dataset was expanded. The RGB cotton canopy images were preprocessed using software, including cropping, segmentation, and splicing. After preprocessing, a brightness enhancer was created to adjust the image brightness, and Gaussian blur was used to change the noise of the image, resulting in 900 cotton canopy images. Finally, 12 color and 7 texture features were extracted from the 900 canopy images, and a dataset was constructed in combination with yield data. The data was divided into a training set and a test set in a ratio of 8:2.

[0035] Step 4. Yield prediction model construction

[0036] 401. The construction of CNN-BiLSTM-ATT model includes: input layer, output layer, convolutional neural network model layer, bidirectional long short-term memory neural network layer, and attention mechanism layer.

[0037] Step 5. Send the data obtained in step 3 to the CNN-BiLSTM-ATT model constructed in step 401 to predict cotton yield.

[0038] Step 6. Evaluate the model

[0039] Perform linear regression analysis on the model prediction results and the real data, and use the coefficient of determination, root mean square error, and mean absolute percentage error as evaluation indicators:

[0040]

[0041] Where n is the total number of samples, i represents the i-th sample in the sample set, and y i is the true value, is the predicted value. 2 It means to normalize the square of the difference between the predicted result and the true value, which is convenient for comparing the difference between models. The value is between 0 and 1. The closer to 1, the better the model. RMSE is often used to indicate the error between the predicted result and the true value of the regression task model. The weight for larger errors is higher. The value is 0 to infinity. The smaller the better. MAPE is a measure of the average size of the error in a set of predictions. It is expressed as a percentage of the actual value. The error value is evenly weighted among all the true values. The value is 0 to infinity. The smaller the value, the higher the prediction accuracy.

[0042] Furthermore, step 305 is included. 12 color features and 7 texture features are processed by principal component analysis, and the dimensions and magnitudes of the feature data are unified. The principal component factors PCA1, PCA2, PCA3 and PCA4 with variance contribution rates of 49.857%, 29.592%, 8.533% and 6.731% are selected respectively. The load value of each characteristic index is recorded, and the four principal component factors and the yield data are used as the final characteristic data.

[0043] Furthermore, the method also includes step 402. Repeatedly training the CNN-BiLSTM-ATT model using Bayesian optimization, adjusting the hyperparameters of the model, and using 20 historical data to predict the output of the next sequence data.

[0044] Beneficial effects of the present invention:

[0045] (1) The cotton canopy images collected by the present invention are treated with defoliants, so that the cotton canopy shows different forms and the interference of redundant leaves is removed, thereby enriching the data source and ultimately ensuring the accuracy of the cotton yield prediction results.

[0046] (2) The cotton variety used in the present invention has less leaf shedding and strong lodging resistance. At the same time, the cotton canopy leaves can still be sealed after falling, and more canopy information is retained, which is more suitable for predicting cotton yield.

[0047] (3). The present invention is based on six deep learning yield prediction models of CNN-BiLSTM-ATT, combined with UAV RGB remote sensing images, calculates the visible light color characteristic index and texture characteristics of cotton canopy, analyzes the correlation between the visible light vegetation index of Xinlu Zao 33, uses principal component analysis to reduce the dimension of features, and forms principal component factors that retain most of the original feature information. The principal component factors are used as input features of the yield prediction model, and the model is optimized using the Bayesian optimization method. Finally, the cotton yield prediction result is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a schematic diagram of the test area (the test area is in Shihezi area of ​​Xinjiang Uygur Autonomous Region).

[0049] Figure 2 It is a convolutional bidirectional long short-term memory neural network model based on the attention mechanism.

[0050] Figure 3 This is the feature correlation heat map of the Xinluzao 33 image.

[0051] Figure 4 This is the heat map of the correlation of image features of Xinlu Zao 50.

[0052] Figure 5 This is a comparison chart of the results of the Xinlu Zao 33 yield prediction model.

[0053] Figure 6 This is a comparison chart of the results of the Xinlu Zao 50 yield prediction model.

[0054] Figure 7 This is the fitting effect diagram of the six cotton yield prediction models of Xinlu Zao 33; A: CNN network model, B: RNN network model; C: LSTM network model, D: BiLSTM network model, E: CNN-BiLSTM network model, F: CNN-BiLSTM-ATT network model.

[0055] Figure 8This is the fitting effect diagram of the six cotton yield prediction models of Xinlu Zao 50; A: CNN network model, B: RNN network model; C: LSTM network model, D: BiLSTM network model, E: CNN-BiLSTM network model, F: CNN-BiLSTM-ATT network model.

[0056] Fig. 9 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] Example

[0059] (1) Materials and methods

[0060] This experiment was conducted from April 2023 to September 2023 in Erlian, Shihezi University Teaching Experimental Field (44°19′N, 85°59′E, altitude: 475m). The climate type is a typical temperate continental climate, the soil is heavy loam, and the soil fertility is relatively high. The cotton varieties tested are Xinlu Zao 33 and Xinlu Zao 50. In April, they were sown in the mode of "one film, three tubes and six rows", with a film width of 2.05m, a row spacing of (66+10)cm, a plant spacing of 5.6cm, and 4 films were planted for each variety. During the entire growth period, field management was carried out according to Xinjiang's high-yield cultivation technology of "short, dense, early, and film", and attention was paid to the prevention and control of diseases, insects, and weeds.

[0061] like Fig. 9 As shown, the test area is Figure 1 As shown in the figure, the whole test area was divided into 6 main plots with a length of 10 m and a width of 8 films, and protection rows were set on both sides of the plots. According to different cotton varieties and leaf densities, the main plot was divided into 6 sub-plots with a length of 10 m and a width of 2.28 m, and 4 plots were planted for each variety. In order to obtain images of leaf densities of different cotton varieties, different doses of defoliants were sprayed in the 6 time periods from T1 to T6 in the main plots. The specific spraying doses were set as C1 (clear water), C2 (150 mL·hm-2 of defoliant), and C3 (150 mL·hm-2 of defoliant). -2 )、C3(Toluolone 300mL·hm -2 )、C4(Toluol 450mL·hm -2 ), spray the 4 films of each variety in the main plot separately. The turbotolone used in this experiment was produced by Bayer, Germany. After the cotton matured, the cotton yield data was obtained manually.

[0062] The spraying times are August 20, August 23, August 30, September 7, September 14 and September 21, 2023 respectively.

[0063] Spraying defoliants can produce multiple samples and enrich sample data. The samples produced can be used to predict yields. If the predicted yield is not high, intervention can be made in advance to apply appropriate fertilizers to improve the harvest.

[0064] (2) Experimental data acquisition

[0065] Before cotton harvest, a Phantom 4 Advanced drone (DJI Innovations Technology Co., Ltd., China) was used to collect high-definition color images of the entire experimental cotton field plot. During image acquisition, the lens was set vertically downward, and photos were taken at equal time intervals. The flight altitude was set to 10 m, the heading and lateral overlap was 80%, the camera shutter speed was 1 / 240 s, the sensitivity ISO value was 100, and the image size was 5472 × 3648 pixels in JPG format. The acquired images were spliced ​​into orthophotos using Pix4Dmapper (Pix4D, Switzerland) software. The spliced ​​orthophotos were stored in TIFF format. According to the plot settings, the spliced ​​cotton field images were cropped into 24 regions of interest for each of the two cotton varieties.

[0066] (3) Image color index feature parameter calculation

[0067] Python was used to extract the color feature values ​​of the RGB cotton canopy image. In order to reduce the influence of shadows, the image was first normalized and the r, g, and b values ​​were calculated. Then, 12 visible light color feature indices were calculated according to different combinations. The 12 visible light color indices used in the present invention are shown in Table 1.

[0068] Table 1 Visible light color index used in the present invention

[0069] Table1 Color index of visible light used in this study

[0070]

[0071]

[0072] Where r, g, and b are chromaticity coordinates, and R, G, and B represent the digital quantization values ​​of the red, green, and blue channels, respectively.

[0073] (4). Calculation of image texture index feature parameters

[0074] The present invention uses gray-level co-occurrence matrix (GLCM) to extract image energy, contrast (contrast-x), entropy, and inverse different moment (IDM) features. Tamura texture features are used to extract image coarseness, contrast (contrast-y), and directionality features, a total of 7 image texture parameter features.

[0075] (5) Dataset creation

[0076] After taking canopy images of two cotton varieties, Xinlu Zao 33 and Xinlu Zao 50, 24 high-quality RGB canopy images of four different leaf densities were obtained. The original canopy image dataset was then expanded, and Photoshop 24.2 and PyCharm 2023.1.2 software were used to preprocess the RGB cotton canopy images, including cropping, segmentation, and splicing. After preprocessing, a brightness enhancer was created to adjust the image brightness, and then Gaussian blur was used to change the noise of the image. Through the above steps, 900 canopy images of each of the two cotton varieties were obtained. Finally, 12 color and 7 texture features were extracted from the 900 canopy images of each variety, and a dataset was constructed in combination with yield data. The data was divided into a training set (720 items) and a test set (180 items) in a ratio of 8:2.

[0077] (6) Construction of production prediction model

[0078] The CNN-BiLSTM-ATT network model structure is as follows Figure 2 As shown in the figure, the CNN-BiLSTM-ATT model consists of three parts, in addition to the input layer and the output layer:

[0079] (6.1) Convolutional Neural Network Model Layer

[0080] ID-CNN is used to process yield feature sequence data to effectively extract local features of the data. Convolutional neural network (CNN) is a very common type of deep learning, and is mostly used in image recognition, natural language processing and other fields. It can be divided into one-dimensional convolution (ID-CNN), two-dimensional convolution (2D-CNN), etc. according to the different data processed, among which 1D-CNN is mainly used in sequence modeling and natural language processing. CNN generally includes input layer, convolution layer, pooling layer, fully connected layer and output layer, and its core is convolution layer, pooling layer and fully connected layer. Since the feature data related to cotton yield is sequence data, the translation feature of the data in a certain direction can be extracted through the convolution layer, which is manifested as the spatial feature of the extracted sequence data. Therefore, the spatial feature of the input cotton yield sequence feature data can be extracted using a one-dimensional convolutional neural network.

[0081] (6.2) Bidirectional long short-term memory neural network model layer

[0082] Bi-LSTM is an improvement of LSTM. It stacks a forward propagation and a backward propagation LSTM layer, and repeatedly trains the input cotton yield sequence feature data and its reverse sequence data to extract time features. Since bi-LSTM can use the information before and after the sequence feature data, it has better context understanding ability, which is helpful for cotton yield prediction. n Indicates the corresponding output.

[0083] (6.3) Attention mechanism layer

[0084] Since the number of extracted cotton sequence features is large, the present invention further optimizes the model through the attention mechanism. The attention mechanism increases the model calculation efficiency by calculating the probability distribution of attention and selecting key features from a large number of features, thereby increasing the model prediction accuracy. n is the amount of attention, a n With h n Multiply and then weighted add to get the final output attention value vector C n .

[0085] The core advantage of the convolutional bidirectional long short-term memory neural network model based on the attention mechanism lies in its excellent feature learning and sequence data analysis capabilities. The model combines the local feature extraction capabilities of CNN, the sequence learning efficiency of BiLSTM, and the focus positioning capabilities of the attention mechanism. The attention mechanism enables the model to identify and give more weight to the most influential parts of the sequence data, thereby improving the ability to accurately predict yields. This structure that integrates advanced cognitive functions allows the model to capture key information and ignore irrelevant interference when processing complex, multivariate sequence data, greatly improving the model's ability to capture complex data patterns and the accuracy of predicting future yield changes, especially when there is noise in the data or significant differences at different scales, it can still maintain the stability and reliability of the prediction. This allows for more accurate and stable yield predictions.

[0086] (6.4) Comparison Model

[0087] Five models, CNN, RNN, LSTM, BiLSTM, and CNN-BiLSTM, were selected for comparison with CNN-BiLSTM-ATT to evaluate the performance of CNN-BiLSTM-ATT in cotton yield prediction. This is because the collected cotton yield feature data is time series data, and these models have excellent performance in processing serialized data. CNN and BiLSTM are introduced in detail in the previous part. RNN can capture the association in sequence data, but there is a long-term dependency problem. As a special RNN structure, LSTM can better solve the gradient disappearance and gradient explosion problems in long sequence training, and performs better when processing longer sequence data. The CNN-BiLST model can achieve end-to-end learning, thereby simplifying the model training process, directly reflecting the mapping relationship from the original input data to the output prediction, and improving the accuracy of the prediction. By comparing these models, we can comprehensively evaluate the advantages of CNN-BiLSTM-ATT in complex data processing and verify its performance in cotton yield prediction.

[0088] (6.5) Evaluation indicators

[0089] In order to verify the accuracy of the yield prediction model, the model prediction results were subjected to linear regression analysis with the actual data. 2 ), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE) are used as evaluation indicators. Their calculation methods (4)-(6) are shown below:

[0090]

[0091] Where n is the total number of samples, i represents the i-th sample in the sample set, and y i is the true value, is the predicted value. 2 It means to normalize the square of the difference between the predicted result and the true value, which is convenient for comparing the difference between models. The value is between 0 and 1. The closer to 1, the better the model. RMSE is often used to indicate the error between the predicted result and the true value of the regression task model. The weight for larger errors is higher. The value is 0 to infinity. The smaller the better. MAPE is a measure of the average size of the error in a set of predictions. It is expressed as a percentage of the actual value. The error value is evenly weighted among all the true values. The value is 0 to infinity. The smaller the value, the higher the prediction accuracy.

[0092] (6.6) Test platform

[0093] The server used in the present invention is Dell PwwerEdge T640, and the main configuration is shown in Table 2, wherein a single graphics card has 16G video memory, each CPU has ten cores and twenty threads. In addition, the hardware platform is shared by multiple people, and each user uses the server by accessing the LXC container in the server. The maximum resources that can be used on each container are also subject to certain restrictions, as follows: the upper limit of the number of threads that can be used by each container is 35, the upper limit of the memory that can be occupied by each container during a single run is 128G, and the remaining resources are not restricted. The deep learning framework used is PyTORCH1.13.1, and Miniconda is used to create and manage the virtual environment.

[0094] 7. Results and Discussion

[0095] (7.1) Principal component analysis

[0096] By analyzing 12 color characteristics and 7 texture characteristics of two cotton varieties, such as Figure 3 and Figure 4 As shown in the figure, it is found that the correlation between the features is higher, and principal component analysis (PCA) is a statistical method that converts multiple variables into several principal components that can describe data information through data dimension reduction. This can eliminate the multicollinearity in the data and remove the redundant information between the features. This helps to improve the interpretability and efficiency of the model. Therefore, by using the principal component analysis method, while ensuring that a large amount of data information is not lost, by eliminating irrelevant or minor features in the data, the risk of overfitting of the model can be reduced and the prediction performance can be improved.

[0097] The multiple characteristics of the two cotton varieties were standardized, and the dimensions and magnitudes of the characteristic data were unified. On the characteristic data set of Xinlu Zao 33 cotton, the principal component factors PCA1, PCA2, PCA3 and PCA4 with variance contribution rates (VCR) of 49.857%, 29.592%, 8.533% and 6.731% were selected, and the cumulative variance contribution rate reached 94.731%. For the characteristic data set of Xinlu Zao 50 cotton, the principal component factors PCA1, PCA2, PCA3 and PCA4 with contribution rates of 50.5495%, 28.416%, 9.908% and 6.607% were selected, and the cumulative variance contribution rate reached 95.480%, and the loading values ​​of each characteristic index of the two cotton varieties were recorded. As shown in Tables 2 and 3. The values ​​of each principal component factor are obtained by taking the characteristic index loading value as the coefficient of the characteristic value index and then summing them up. After screening, each cotton variety uses four main component factors and yield data as the final feature data and is sent to the deep learning network for model training.

[0098] Table 2 Variance contribution rate and loading value of principal component factors of Xinluzao 33

[0099]

[0100] Table 4 Variance contribution rate and loading value of principal component factors of Xinluzao 50

[0101]

[0102] (7.2) Bayesian Optimization

[0103] In order to achieve the best prediction effect of the model, the Bayesian optimization method is used to repeatedly train 6 deep learning prediction models including CNN-BiLSTM-ATT and adjust the hyperparameters of the models. Bayesian optimization is an experimental method for global optimization. Compared with traditional optimization methods, Bayesian optimization can find the optimal value of the objective function with fewer evaluations. The core idea is to describe the objective function by building a probabilistic model and select the next set of optimal parameters for evaluation based on this model. Bayesian optimization can ensure that the adjusted prediction model is close to the optimal parameter solution.

[0104] The basic layer of each model includes one input layer, one intermediate layer and one output layer. At the same time, the hybrid model has one layer for each neural network layer. The adjusted deep learning model parameters are the number of hidden layers of the neural network 0-4, the number of neurons 32-20, the dropout value 0.0-0.6, the batch size batch_size30-150, the learning rate learn_rates0.001-0.01, and the number of iterations epochs30-650. The same training set, test set, time step, features and the same parameter range are used for training and parameter adjustment through the Bayesian optimization method. The final deep learning model is comparable in performance. The model uses the root mean square error as the loss function, uses the Adarm optimizer, the CNN model uses relu as the activation function, and the other models use tanh as the activation function. On the Xinlu Zao 33 cotton yield feature dataset, 20 historical data are used to predict the yield of the next sequence data. The parameters of the deep learning model at the final adjustment are shown in Tables 4 and 5. For the deep learning model at the training, three evaluation indicators are comprehensively considered to select the optimal yield prediction model.

[0105] Table 4 Parameters of Xinluzao 33 deep learning model

[0106]

[0107] Table 5 Parameters of Xinlu Zao 50 deep learning model

[0108]

[0109] (7.3) Yield prediction model results and analysis

[0110] According to the divided data set, the CNN-BiLSTM-ATT prediction model and five comparison models were trained, and the determination coefficient R 2 The results of different yield prediction models on the two datasets on the test set are shown in Tables 6 and 7. Figure 5 and Figure 6 The measured and predicted yields are shown in Figure 7 and Figure 8 shown.

[0111] According to the comparison bar chart between the models, it can be intuitively observed that the CNN-Bilstm-ATT hybrid model proposed in this invention has significant advantages in yield prediction, and can show excellent prediction effects whether on a specific data set or on multiple data sets. 2The value of shows an upward trend, while the values ​​of RMSE and MAPE decrease significantly, indicating that the hybrid model has good generalization ability in predicting yield accuracy. In addition, the depth of the hybrid model structure needs to be balanced to avoid the problem of overfitting of the model.

[0112] Table 6 Results of the yield prediction model for Xinluzao 33

[0113]

[0114] Table 7 Results of the yield prediction model for Xinluzao 50

[0115]

[0116] After comprehensive analysis, for the Xinlu Zao 33 cotton dataset, the six yield prediction models all have good performance, and the determination coefficients of yield prediction on the test set are all over 0.83. The prediction results of the CNN-BiLATM-ATT and CNN-BilSTM models are close, and the determination coefficients R 2 All are greater than 0.89, and the root square error RMSE is less than 62t·hm -2 , the mean absolute percentage error (MAPE) is less than 9%, among which CNN-BiLATM-ATT has the best prediction effect, and there is a good linear relationship between the measured output and the predicted output. The prediction results of CNN, RNN, LSTM and BiLSTM models are relatively low, and the determination coefficient R of CNN is 2 is 0.8327, and the root mean square error RMSE is 0.7731t·hm 2 , the mean absolute percentage error MAPE is 12.07%, and its prediction result is the lowest. The linear relationship between the measured output and the predicted output is also relatively poor. For the three deep learning prediction models of RNN, LSTM, and BiLATM, the determination coefficients R2 are 0.8501, 0.8691, and 0.8792, respectively. The prediction results are higher than the CNN model but lower than the CNN-BiLSTMheCNN-BiLSTM-ATT hybrid model. It can be seen that RNN, LSTM, and BiLSTM are more able to connect the information between the context of sequence data, but the prediction effect is limited by the depth of the network structure.

[0117] Compared with the Xinlu Zao 33 cotton dataset, the prediction effects of the six models of the Xinlu Zao 50 cotton dataset have decreased to varying degrees, but still have good performance. The prediction results of the CNN-BiLSTM-ATT model are the best, and the linear relationship between the measured yield and the predicted yield is the best. The prediction hoops of the BiLSTM and CNN-BiLSTM models are close, and the CNN-BiLSTM determination coefficient R 2is 0.8548, and the root mean square error RMSE is 0.6524t·hm -2 , the mean absolute percentage error MAPE is 9.219%. The prediction results of RNN and LSTM models are also close, and the coefficient of determination R2 exceeds 0.82. The prediction accuracy of RNN model is 2.54%, 2.75% and 4.84% lower than that of BiLSTM, CNN-BiLSTM and CNN-BiLSTM-ATT models respectively. The prediction result of CNN model is the lowest, and the degree of aggregation between measured output and predicted output is also relatively dispersed.

[0118] Liu Xinyi used the color and texture feature data of drone images to predict the yield in the early growth period of wheat, thereby verifying the feasibility of the method. However, when using drones to collect remote sensing images, various extreme conditions, such as differences in temperature and light, may cause the characteristic index prepared based on a single visible light to be relatively random when predicting yield, and the prediction accuracy is often not ideal. The present invention selects to combine the color index features and texture features of cotton canopy images to predict yield and obtains 6 relatively stable deep learning prediction models.

[0119] Different deep learning yield prediction models established on two cotton field datasets have different prediction effects on different datasets. The model has a higher degree of fit and more significant prediction results on the Xinlu Zao 33 dataset. This is because the Xinlu Zao 33 dataset retains more canopy feature information under the same conditions, due to fewer leaf shedding, strong lodging resistance of cotton plants, and more closed cotton canopy ridges. It is more suitable for cotton yield prediction. According to the fitting diagram of the yield prediction model, the per mu yield is 4t·hm -2 To 8t·hm -2 The sample fitting effect is better in the range, because the canopy leaf density in this part of the image is higher and the soil background noise is less. -2 The canopy image has soil noise and the yield is greater than 8t·hm -2 Part of the canopy image is composed of other images with soil noise, which leads to poor model fitting. Among the six deep learning prediction models, the CNN-BiLSTM-ATT model performs best on the two data sets because the hybrid network structure model has a stronger ability to extract sequence data features and connect the contextual relationship of feature data. When there is noise or different scale differences in the data, the model performance is more significant.

[0120] in conclusion

[0121] The present invention is based on CNN-BiLSTM-ATTz as six deep learning yield prediction models, combined with UAV RGB remote sensing images, calculates the visible light color characteristic index and texture feature of cotton canopy, analyzes the correlation between 19 visible light preparation indexes of Xinlu Zao 33 and 50, uses principal component analysis to reduce the dimension of features, and forms principal component factors that retain most of the original feature information. The principal component factors are used as input features of the yield prediction model, and the Bayesian optimization method is used to optimize the model. Finally, cotton yield is predicted, and the following conclusions are obtained:

[0122] Xinlu Zao 33 has a lower sensitivity to defoliants and a tighter cotton canopy ridge closure. Even though its cumulative variance contribution rate is only 94.731%, which is lower than the cumulative variance contribution rate of Xinlu Zao 50 of 95.48%, it still maintains a large number of effective cotton features. The retention of these features significantly improves the performance of the six deep learning yield prediction models on the Xinlu Zao 33 dataset, which is better than the performance on the Xinlu Zao 50 dataset. It is particularly worth noting that the CNN-BiLSTM-ATT model achieves the best effect on the Xinlu Zao 33 dataset, with a determination coefficient R2 of 0.8944, which is higher than 0.8757 on the Xinlu Zao 50 dataset.

[0123] The hybrid network yield prediction model has a more complex structure and can integrate multi-source feature data mainly based on color and texture, which can significantly improve the accuracy of the cotton yield prediction model. The prediction effects of the CNN-BiLSTM and CNN-BiLSTM-ATT models on both data sets are better than those of the single-structure model.

[0124] The CNN-BiLSTM-ATT model cleverly combines the local feature extraction capability of the convolutional network, the sequence data parsing efficiency of the bidirectional long short-term memory network, and the focus function of the attention mechanism. The model has the best yield prediction results on both datasets. On the Xinlu Zao 33 dataset, the determination coefficient R 2 is 0.8944, and the root mean square error RMSE is 0.6142t·hm -2 , the mean absolute percentage error MAPE is 7.31%, and the determination coefficient R 2 is 0.8757, and the root mean square error RMSE is 0.6037t·hm -2 , the mean absolute percentage error MAPE is 6.952%.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting cotton yield from drone RGB images based on attention mechanism, characterized in that: The steps include: Step 1. Choose the appropriate prediction object Choose cotton fields with reasonable film width, row spacing, and plant spacing, and choose varieties that are resistant to lodging and insensitive to defoliants; Step 2. Spray defoliants at appropriate dosages at appropriate times Step 3. Experimental data acquisition 301.Use drones to acquire and process images Before the cotton harvest period, use an aerial drone to collect high-definition color images of the cotton field in step 1; use software to stitch the acquired images into orthophotos, store the stitched orthophotos, and crop the stitched cotton field images; 302. Image color index feature parameter calculation The color feature values ​​of the RGB cotton canopy image were extracted using software. First, the image was normalized and the r, g, and b values ​​were calculated. Then, 12 visible light color feature indices were calculated based on different combinations: In the formula, r, g, and b are chromaticity coordinates, and R, G, and B represent the digital quantization values ​​of the red, green, and blue channels, respectively; 12 visible light color indices: visible light atmospheric impedance vegetation index, super green vegetation index, super red vegetation index, super blue vegetation index, green-red difference index, normalized green-red difference index, green-red vegetation index, Woebbecke index, Kawashima index, green leaf index, red, green and blue vegetation index, plant index, corresponding to the following calculation formulas: EXG=2g―r―b (5) EXR=1.4r―g (6) EXGR=EXG―EXR (8) 303. Image Texture Index Feature Parameter Calculation The gray-level co-occurrence matrix is ​​used to extract the energy, contrast, entropy, and inverse difference moment features of the image. The Tamura texture feature is used to extract the roughness, contrast, and orientation features of the image, for a total of 7 image texture parameter features. 304.Data Production 24 high-quality RGB canopy images with different leaf densities were obtained, and the original canopy image dataset was expanded. The RGB cotton canopy images were preprocessed using software, including cropping, segmentation, and splicing. After preprocessing, a brightness enhancer was created to adjust the image brightness, and Gaussian blur was used to change the noise of the image to obtain the cotton canopy image. Finally, 12 color and 7 texture features were extracted from the canopy image, and a dataset was constructed in combination with yield data. The data was divided into a training set and a test set in a ratio of 8:

2. Step 4. Yield prediction model construction 401. Constructing the CNN-BiLSTM-ATT model includes: input layer, output layer, convolutional neural network model layer, bidirectional long short-term memory neural network layer, and attention mechanism layer; Step 5. Forecast of cotton production The data obtained in step 304 is sent to the CNN-BiLSTM-ATT model constructed in step 401 to predict cotton yield; Step 6. Evaluate the model Perform linear regression analysis on the model prediction results and the real data, and use the coefficient of determination, root mean square error, and mean absolute percentage error as evaluation indicators: Where n is the total number of samples, i represents the i-th sample in the sample set, and y i is the true value, is the predicted value, R 2 It means normalizing the square of the difference between the predicted result and the true value. The value is between 0 and 1. The closer to 1, the better the model. RMSE is often used to indicate the error between the predicted result of a regression task model and the true value. The weight for larger errors is higher. The value is from 0 to infinity. The smaller the better. MAPE is a measure of the average size of the error in a set of predictions. It is expressed as a percentage of the actual value. The error value is evenly weighted among all the true values. The value is from 0 to infinity. The smaller the value, the higher the prediction accuracy.

2. The method according to claim 1, characterized in that Step 1 includes: the cotton variety is Xinlu Zao 33, the film width is 2.05m, the row spacing is 76cm, and the plant spacing is 5.6cm.

3. The method according to claim 1, characterized in that Step 2, select the appropriate time to spray a reasonable dose of defoliant according to the size of the cotton field, the defoliant is Totolon.

4. The method according to claim 1, characterized in that: Step 301 includes: when collecting images, the lens is pointed vertically downward, photos are taken at equal time intervals, the flight altitude is set to 10m, the heading and lateral overlap is set to 80%, the camera shutter speed is set to 1 / 240s, the sensitivity ISO value is set to 100, the acquired image size is 5472×3648 pixels, and the format is JPG.

5. The method according to claim 2, characterized in that: It also includes step 305. 12 color features and 7 texture features are processed using principal component analysis, and the dimensions and magnitudes of the feature data are unified. Principal component factors PCA1, PCA2, PCA3 and PCA4 with variance contributions of 49.857%, 29.592%, 8.533% and 6.731% are selected, the load value of each feature index is recorded, and the four principal component factors and yield data are used as the final feature data.

6. The method according to claim 2, characterized in that The method also includes step 402. Repeatedly training the CNN-BiLSTM-ATT model using Bayesian optimization, adjusting the hyperparameters of the model, and using 20 pieces of historical data to predict the output of the next piece of sequence data.

Images