Sky-ground integrated rice growth monitoring method

Abstract

The invention discloses a sky-ground integrated rice growth vigor monitoring method, and relates to the technical field of remote sensing, and the method comprises the steps: carrying out the modeling of each growth stage of rice, and forming a rice growth model diagram; collecting data to form a rice remote sensing big data image library; training a deep learning feature extractor, and extracting the features of the rice image; estimating rice growth based on remote sensing image big data; estimating rice growth based on the enhanced vegetation index; and carrying out multi-scale cross validation on the accurate identification degree of the rice growth vigor, and finally forming a set of sky-ground integrated rice growth vigor monitoring method. According to the method, a deep learning algorithm and a vegetation enhancement index method are adopted, the growth vigor estimation is performed on remote sensing data of different sources, and the rice growth vigor estimation result is determined through cross validation, so that high-precision rice growth state monitoring is realized; the three remote sensing technologies complement one another to form a comprehensive and multi-angle rice growth monitoring method, and real-time and efficient farmland monitoring management can be achieved.

Description

Technical Field

[0001] The present application relates to three remote sensing technology fields: satellite remote sensing, UAV near-ground remote sensing and in-situ remote sensing, and specifically to a set of sky-ground integrated rice growth monitoring methods. Background Art

[0002] As one of the most important food crops in the world, rice is of great significance to global food security and rural economy. Sichuan Province is one of the important rice producing areas in China, with a rice planting area of ​​29.709 million hectares and a total output of 206.535 million tons. However, there is a lack of experienced researchers to monitor the growth of rice in real time during the rice planting process. As high-standard farmland has become a national key construction project, the traditional artificial rice growth monitoring method covers a small range and has low efficiency, which is difficult to meet the needs of high-standard farmland construction. Summary of the invention

[0003] In order to make up for the shortcomings of the existing technology, this application provides a set of integrated sky-ground rice growth monitoring methods, which uses MODIS data developed by the public satellite remote sensing data processing software NASAMODIS, near-ground remote sensing images of unmanned aerial vehicles, and in-situ remote sensing images of rice collected by ground visual sensors, combined with deep learning classification algorithms and vegetation enhancement index (EVI) curves to complete the monitoring and estimation of rice growth, aiming to solve the problem that agricultural practitioners cannot grasp the growth status of rice in real time.

[0004] This application adopts the following technical solutions:

[0005] A set of sky-ground integrated rice growth monitoring method includes the following steps:

[0006] (1) Modeling each growth stage of rice to form a rice growth model diagram;

[0007] (2) Collect data to form a rice remote sensing big data image library: Use drones equipped with hyperspectral sensors to obtain near-ground remote sensing images, and use rice ground vision sensors to obtain local surface images and field images of rice planting areas to form image big data;

[0008] (3) Train the deep learning feature extractor to extract the features of rice images: input image data, use the deep learning algorithm to forward propagate it, obtain the feature representation of each level, and use the back propagation algorithm to train the deep learning model. Through iterative optimization, the model weights and biases are continuously adjusted to continuously improve the quality of the output results. After the model training is completed, it can be used to extract remote sensing image features;

[0009] (4) Rice growth estimation based on remote sensing image big data: Use the trained deep learning model to extract remote sensing image features, pass the remote sensing image features into the classifier and compare them with the rice growth model map to obtain the growth status of rice;

[0010] (5) Rice growth estimation based on the enhanced vegetation index: On a large scale, by studying the relationship between the vegetation enhancement index and rice growth, the corresponding EVI curve is drawn using MODIS data developed by NASAMODIS, a public satellite remote sensing data processing software, and compared with the standard rice growth EVI curve to determine the rice growth situation;

[0011] (6) Multi-scale cross-validation of the accurate identification of rice growth has formed a set of integrated sky-ground rice growth monitoring methods.

[0012] In the step (3), the deep learning algorithm is a Swin-Transformer deep learning algorithm based on the self-attention mechanism, which indirectly implements the ShiftedWindow operation by shifting the feature map and setting a mask for Attention.

[0013] In the step (4), first, the collected image big data is divided into four categories according to the actual situation, namely, germination stage, growth stage, heading differentiation stage and maturity stage, and these are used as labels for each image; then, the rice growth is estimated by combining the convolutional neural network VGG16 with the Swin-Transformer method, each convolutional block is composed of a convolution-Relu-batch normalization layer, and the output of each convolutional layer is passed to the Swin-TransformerBlock for feature extraction again, thereby enhancing the network model's encoding ability for image spatial information, extracting more abstract and deeper features of the image, and obtaining a feature map with better performance; finally, the obtained feature map is linked to the classifier, and two layers of fully connected layers are used to obtain the weight of each channel in the feature map, and the dimension is reduced, and the probability value of each label is output, and the label with the highest probability value is selected as the estimated result, thereby realizing the estimation of rice growth.

[0014] In the step (5), the MODIS data comes from Google Earth Engine.

[0015] Acquisition of the remote sensing images or data: satellite remote sensing technology is used to obtain large-scale rice field environmental information, including temperature, vegetation index, and vegetation enhancement index; unmanned aerial vehicle near-ground remote sensing technology is used to obtain high-resolution, high-precision rice growth images; in-situ remote sensing technology is used to collect high-resolution local rice growth images and growth environment data.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] The present invention utilizes multi-source image information, adopts deep learning algorithms and vegetation enhancement index methods, predicts the growth of remote sensing data from different sources, and determines the rice growth prediction results through cross-validation, thereby realizing high-precision rice growth status monitoring. Satellite remote sensing technology can obtain large-scale rice field environmental information, such as temperature, vegetation index, vegetation enhancement index, etc., UAV near-ground remote sensing technology can obtain high-resolution and high-precision rice growth images, and in-situ remote sensing technology can collect high-resolution local images of rice growth and growth environment data such as soil moisture and light. The three remote sensing technologies complement each other to form a comprehensive and multi-angle rice growth monitoring method. This application is conducive to the realization of real-time and efficient farmland monitoring and management, and plays an important role in serving agricultural modernization. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is the technical roadmap of the sky-ground integrated rice growth monitoring method of the present invention.

[0019] Figure 2 This is a network structure diagram of rice growth estimation based on remote sensing big data in an embodiment of the present invention.

[0020] Figure 3 It is a graph showing changes in the EVI curve of an embodiment of the present invention.

[0021] Figure 4 1 is the EVI curve of rice with different growth potentials according to the embodiment of the present invention.

[0022] Figure 5 This is an example diagram of the sky-ground integration according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the implementation methods of the present application clearer, the technical solutions in the implementation methods of the present application will be clearly and completely described below.

[0024] 1. Rice growth estimation based on remote sensing big data

[0025] 1. Model the germination stage, growth stage, heading differentiation stage and maturity stage of rice to form a rice growth model diagram.

[0026] 2. Crop growth estimation

[0027] (1) Collect data to form a rice remote sensing big data image library. Use drones equipped with hyperspectral sensors to obtain near-ground remote sensing images, and use rice ground vision sensors to obtain local surface images and field images of rice planting areas to form image big data. Ground vision sensors can be installed in farmland to obtain more comprehensive land information by shooting and recording images from multiple angles. Drones equipped with hyperspectral sensors can fly regularly to obtain high-resolution and high-sensitivity remote sensing image data.

[0028] (2) Train the deep learning feature extractor to extract the features of rice images. Input farmland image data and use the deep learning algorithm to forward propagate it to obtain feature representations at all levels. Use the backpropagation algorithm to train the deep learning model and continuously adjust the model weights and biases through iterative optimization to continuously improve the quality of the output results. Once the model training is completed, it can be used to extract the features of farmland images, providing strong support for subsequent agricultural applications.

[0029] (3) Feature comparison to estimate rice growth. Use the trained deep learning model to extract the features of the rice image and compare it with the rice growth image to estimate the growth stage of the rice. These features reflect the different growth stages of rice and usually include information such as texture, shape, and color.

[0030] 3. Algorithm

[0031] This application uses the Swin-Transformer deep learning algorithm based on the self-attention mechanism to extract remote sensing image features of rice growth stages, and passes the extracted feature map into the fully connected layer to complete classification and recognition. Swin-Transformer is a deep learning model architecture based on Transformer. The difference from the traditional Transformer architecture is that it uses a layered design to divide the large image into multiple small areas of a certain size. Local feature extraction is achieved in each small area through a multi-head self-attention mechanism, which can greatly reduce the number of parameters of the model, while also having good scalability and flexibility.

[0032] In practical applications, this application indirectly implements the ShiftedWindow operation by shifting the feature map and setting a mask for Attention. It can obtain the same calculation results as directly performing the shiftedwindow operation while maintaining the original number of windows, and improve the receptive field of the model, thereby better capturing global features.

[0033] 4. Rice growth estimation based on remote sensing big data

[0034] (1) Network structure

[0035] This application uses the convolutional neural network VGG16 combined with Swin-Transformer to estimate the growth of rice. According to the actual situation, the collected image big data is divided into four categories, namely germination stage, growth stage, heading differentiation stage and maturity stage, and these are used as labels for each image, such as Figure 2 shown.

[0036] Each convolution block of the neural network consists of a convolution-Relu-batch normalization layer, and the output of each convolution layer is passed to the Swin-TransformerBlock for feature extraction again, thereby enhancing the network model's ability to encode image spatial information, extracting more abstract and deeper features of the image, and achieving better performance.

[0037] The network model links the final feature map to the classifier, uses two fully connected layers to obtain the weight of each channel in the feature map, reduces the dimension, and finally outputs the probability value of each label. The label with the highest probability value is selected as the estimated result, thereby realizing the estimation of rice growth.

[0038] (2) Model training

[0039] The model training uses CrossEntropy as the loss function of the model and selects the Adam optimizer to optimize the model (Momentum is set to 0.9, smoothing constants β1 and β2 are set to 0.9 and 0.999 respectively), and trains for 200 epochs. In order to avoid overfitting of the model, the Dropout layer with a selection parameter of 0.7 is used to regularize the model. Finally, the trained model is used to estimate the growth potential of rice.

[0040] (3) Model evaluation indicators

[0041] Evaluation indicators include accuracy, precision, recall and F1-score index.

[0042] 2. Estimation of rice growth based on the vegetation enhancement index (EVI)

[0043] (1) Data source

[0044] The data for this application comes from public satellite images, hydrological and meteorological data, and other data of rice-growing areas in the southwest region of China in Google Earth Engine (GEE). GEE is a service platform based on cloud computing and geographic information data launched by Google, which provides the function of automatically analyzing and processing satellite remote sensing images. Remote sensing images of rice planting in the southwest region are obtained through GEE.

[0045] (2) EVI Curve

[0046] The EVI curve is a remote sensing index curve used to monitor the growth status of vegetation. The curve depicts different stages of vegetation growth over time, with the maximum value usually observed from spring to summer of the year. In rice fields, which are usually flooded before transplanting, low EVI values ​​are observed at this time, and the curve is very gentle, indicating the beginning stage of vegetation growth. However, with the transplanting of rice, the EVI curve gradually rises because the photosynthesis of vegetation accelerates and the chlorophyll content increases. After reaching the saturation point, the curve tends to stabilize, indicating that the vegetation has matured. Therefore, it is reasonable to define the transplanting date of rice as the minimum point of the EVI curve. During the growth and senescence period of rice leaves, the EVI curve begins to decline after the head stage. The maturity of rice is determined by the minimum point of the first derivative of the EVI curve.

[0047] This application uses the Generalized Land Satellite Phenology (GLSP) method to identify annual changes in phenological indicators. The double logistic model used in the GLSP method includes two S-shaped curves, representing the growth and senescence stages of vegetation growth, which can effectively identify different stages of vegetation growth and provide a reliable means to monitor and predict changes in vegetation growth status, such as Figure 3 As shown. The horizontal axis in the figure represents the week of the year, and the vertical axis represents the observed value of the vegetation enhancement index (EVI). The green curve represents the growth stage of the rice crop, and the orange curve represents the senescence stage of the rice crop. SOS and EOS indicate that the first-order derivative of the EVI curve reaches the maximum increase rate and decrease rate in the greening and senescence stages, respectively. V1 is the minimum value of the vegetation enhancement index EVI throughout the year, which is regarded as the background value. V2 is the maximum value of the vegetation enhancement index EVI throughout the year, which is regarded as the mature value. The maximum amplitude of the whole year can be obtained by subtracting V1 from V2.

[0048] Use formula f(x) to fit the EVI value on day t. The specific formula is as follows:

[0049]

[0050] Among them, v1 and v2 are the background and amplitude of EVI throughout the year respectively; the first S-curve Sig1 captures the green stage of vegetation growth; the second S-curve Sig2 captures the aging stage of vegetation growth. The fitting curve calculation formula of Sig1 and Sig2 is as follows:

[0051]

[0052] Rice is transplanted in the early growth stage, when the crop coverage is low and the soil background easily affects the calculation of NDVI. Therefore, the EVI time series curve is selected for construction. However, the conventional composite algorithm has not completely eliminated the influence of factors such as cloud cover, light and terrain on the EVI time series data. These factors cause the EVI time series data to contain noise, thus affecting the accuracy of phenological extraction. Therefore, a filtering method is used to eliminate noise and reconstruct the EVI time series.

[0053] Savitzky-Golay (SG) smoothing filter, also known as least squares method or digital smoothing polynomial, is widely used to smooth noisy signals in phenological research. SG filtering is used to smooth irregular EVI time series. Based on its locally adaptive moving window, it can maintain subtle local changes in time series data. After smoothing, the crop growth cycle can be reflected by the EVI time series curve.

[0054] (3) EVI curve comparison to identify rice growth potential

[0055] By drawing the rice growth EVI curve of the target area and comparing it with the standard EVI curve of rice in the southwest region, the growth conditions of rice in the target area can be identified to formulate analysis and planning that conforms to the rice growth conditions. The characteristics of rice EVI curves under different growth conditions are as follows: Figure 4 shown.

[0056] This application regards the minimum value of the vegetation enhancement index EVI throughout the year as the background value, and the maximum value of the vegetation enhancement index EVI throughout the year as the maturity value. The peak value of the EVI index of standard-growing rice usually occurs in May-June, and the maturity period lasts for a long time. Rice with good growth has high chlorophyll content and high light energy utilization rate, so its EVI index has the characteristics of high stability and long duration, and the corresponding curve peak will rise and move forward compared to the standard EVI curve. However, rice with poor growth has weak photosynthesis ability, relatively low EVI value, and may have problems such as lack of fertilizer, insect pests, and diseases. Therefore, its EVI has the characteristics of short curve period, backward shift of peak value, and large fluctuation.

[0057] This application obtains the EVI index value of the target area based on satellite remote sensing images, fits the EVI curve of the actual growth of rice, and compares it with the standard growth curve to infer and identify the real growth of rice. Based on the identified growth of rice, it can help farmers and agricultural managers identify and solve problems in the rice growth process and improve production efficiency and yield.

[0058] 3. Sky-ground integrated rice growth monitoring method

[0059] This application uses three remote sensing technologies, namely satellite remote sensing, UAV near-ground remote sensing and in-situ remote sensing, to form a sky-ground integrated rice growth monitoring method, such as Figure 5 As shown. This method collects multi-source image information, uses deep learning algorithms and vegetation enhancement index methods to predict the growth of remote sensing data from different sources, and determines the rice growth prediction results through cross-validation, thereby achieving high-precision monitoring of rice growth status. Satellite remote sensing technology can obtain large-scale rice field environmental information, such as temperature, vegetation index, vegetation enhancement index, etc. UAV near-ground remote sensing technology can obtain high-resolution and high-precision rice growth images. In-situ remote sensing technology can collect high-resolution local images of rice growth and growth environment data such as soil moisture and light. The three remote sensing technologies complement each other to form a comprehensive, multi-angle rice growth monitoring method. The specific implementation method is as follows:

[0060] (1) Data collection and data preprocessing

[0061] First, it is necessary to collect data from ground remote sensing, drone near-earth remote sensing, and satellite remote sensing. For ground remote sensing data, sensor equipment (such as high-resolution cameras) is used to collect data from rice planting areas; for drone near-earth remote sensing data, drone equipment equipped with hyperspectral sensors is used to collect data; and satellite remote sensing data comes from MODIS data developed by NASAMODIS, a publicly available satellite remote sensing data processing software, based on statistical algorithms, which is a global vegetation index product synthesized over 16 days with a resolution of 250 meters.

[0062] After data collection is completed, data preprocessing is required. Preprocessing includes operations such as correction, spatial reprojection, data smoothing, and noise processing. These operations can improve data quality and reduce data errors, thereby improving the accuracy and reliability of growth estimation. In addition, in order to better utilize ground remote sensing and UAV near-ground remote sensing data, they need to be fused. Fusion methods include pixel-level fusion, feature-level fusion, and decision-level fusion. Through fusion, the respective advantages of the two types of data can be fully utilized to improve the accuracy and reliability of growth estimation.

[0063] (2) Estimation of rice growth

[0064] This application adopts a rice growth estimation method based on remote sensing big data to process ground remote sensing and drone near-ground remote sensing data. Using deep learning technology, combined with a large amount of remote sensing data and ground observation data, a rice growth estimation model is established. This model can accurately predict the growth of rice crops, including growth status, growth rate, growth cycle, etc.

[0065] Satellite remote sensing data uses a rice growth identification method based on the vegetation enhancement index. This method uses the vegetation enhancement index in satellite remote sensing data and combines the characteristics and laws of rice growth to identify and predict rice growth. It has the advantages of simple operation and wide application, and can realize growth monitoring and prediction in a large range of rice planting areas. At the same time, in order to improve the accuracy of growth identification, a multi-source data fusion method is used to fuse satellite remote sensing data with ground remote sensing and UAV near-ground remote sensing data. Through the application of these methods, comprehensive, real-time, and accurate monitoring and prediction of rice growth can be achieved, providing strong support for the management and decision-making of rice planting.

[0066] (3) Cross-validation

[0067] In this application, in order to avoid inaccurate estimates when the model is overfitted on the training set or does not fully cover the data distribution, a cross-validation method is used. In order to ensure the reliability of the verification results, a variety of cross-validation methods are used, such as K-fold cross-validation, leave-one-out cross-validation, etc. Through cross-validation, the performance of the model on different data sets can be obtained, thereby evaluating the generalization ability and accuracy of the model. After judging the growth of different types of remote sensing data, the cross-validation method is applied to the result verification. By comparing the verification results of different models, a model with a higher accuracy rate can be selected as the actual growth situation of rice. The established sky-ground integrated rice growth monitoring system can realize comprehensive, real-time and accurate monitoring and prediction of rice growth, providing strong support for the management and decision-making of rice planting.

[0068] (4) Feedback and Optimization

[0069] In this application, according to the results of growth potential estimation, the data processing method, data fusion method and model parameters are fed back and optimized to further improve the accuracy and reliability of growth potential estimation. Specifically, by analyzing the error sources and distribution of the growth potential estimation results, the data processing method and data fusion method are optimized. At the same time, the model parameters are also adjusted and optimized to improve the generalization ability and accuracy of the model.

[0070] In addition to optimizing data processing and model parameters, different crop management strategies are set for different land use types and regions based on the results of growth estimation. By analyzing the differences between the growth estimation results and the actual crop growth conditions, corresponding crop management strategies can be formulated for different land use types and regions, including measures such as fertilization, irrigation, and pest control. In this way, agricultural production efficiency can be improved and the goal of sustainable agricultural development can be achieved.

[0071] The above description is only an example of a specific implementation of the present application, and the protection scope of the present application is not limited thereto. Any technical solution that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A set of sky-ground integrated rice growth monitoring method, characterized in that: The following steps are involved: (1) Modeling each growth stage of rice to form a rice growth model diagram; (2) Collect data to form a rice remote sensing big data image library: Use drones equipped with hyperspectral sensors to obtain near-ground remote sensing images, and use rice ground vision sensors to obtain local surface images and field images of rice planting areas to form image big data; (3) Train the deep learning feature extractor to extract the features of rice images: input image data, use the deep learning algorithm to forward propagate it, obtain the feature representation of each level, and use the back propagation algorithm to train the deep learning model. Through iterative optimization, the model weights and biases are continuously adjusted to continuously improve the quality of the output results. After the model training is completed, it can be used to extract remote sensing image features; (4) Rice growth estimation based on remote sensing image big data: Use the trained deep learning model to extract remote sensing image features, pass the remote sensing image features into the classifier and compare them with the rice growth model map to obtain the growth status of rice; (5) Rice growth estimation based on the enhanced vegetation index: On a large scale, by studying the relationship between the vegetation enhancement index and rice growth, the corresponding EVI curve is drawn using MODIS data developed by NASAMODIS, a public satellite remote sensing data processing software, and compared with the standard rice growth EVI curve to determine the rice growth situation; (6) Multi-scale cross-validation of the accurate identification of rice growth has formed a set of integrated sky-ground rice growth monitoring methods.

2. The method for monitoring rice growth in an integrated sky-ground manner according to claim 1, characterized in that: In the step (3), the deep learning algorithm is a Swin-Transformer deep learning algorithm based on the self-attention mechanism, which indirectly implements the ShiftedWindow operation by shifting the feature map and setting a mask for Attention.

3. The method for monitoring rice growth in an integrated sky-ground manner according to claim 1, characterized in that: In the step (4), first, the collected image big data is divided into four categories according to the actual situation, namely, germination stage, growth stage, heading differentiation stage and maturity stage, and these are used as labels for each image; then, the rice growth is estimated by combining the convolutional neural network VGG16 with the Swin-Transformer method, each convolutional block is composed of a convolution-Relu-batch normalization layer, and the output of each convolutional layer is passed to the Swin-TransformerBlock for feature extraction again, thereby enhancing the network model's encoding ability for image spatial information, extracting more abstract and deeper features of the image, and obtaining a feature map with better performance; finally, the obtained feature map is linked to the classifier, and two layers of fully connected layers are used to obtain the weight of each channel in the feature map, and the dimension is reduced, and the probability value of each label is output, and the label with the highest probability value is selected as the estimated result, thereby realizing the estimation of rice growth.

4. The method for monitoring rice growth in an integrated sky-ground manner according to claim 1, characterized in that: In the step (5), the MODIS data comes from Google Earth Engine.

5. The method for monitoring rice growth in an integrated sky-ground manner according to claim 1, characterized in that: The method for acquiring the remote sensing image or data is as follows: using satellite remote sensing technology to acquire large-scale rice field environmental information, including temperature, vegetation index, and vegetation enhancement index; using unmanned aerial vehicle near-ground remote sensing technology to acquire high-resolution, high-precision rice growth images; and using in-situ remote sensing technology to collect high-resolution rice growth local images and growth environment data.

Images