Large-scale winter wheat yield remote sensing estimation method based on unmanned aerial vehicle and satellite image cross-scale information fusion

Abstract

The invention provides a large-scale winter wheat yield remote sensing estimation method based on unmanned aerial vehicle and satellite image cross-scale information fusion, and the method comprises the following steps: obtaining an unmanned aerial vehicle image in a wheat key growth period and a satellite image in the same period through satellite-aircraft-ground synchronous scatter sampling, and constructing an unmanned aerial vehicle platform winter wheat yield estimation model; obtaining a yield sample augmented data set; the method comprises the following steps: constructing a satellite pixel spatial diversity index by using an unmanned aerial vehicle image, realizing cross-scale information fusion, and determining the uncertainty influence of growth difference in a field and mixed pixels on satellite winter wheat yield estimation; and making a sample augmentation optimization strategy based on the optimal spatial resolution index, determining an unmanned aerial vehicle and satellite fused winter wheat yield estimation model, and realizing large-scale field-level winter wheat yield estimation. According to the method, the winter wheat yield can be estimated timely and accurately, and the method has great application potential in the aspects of grain policy making, safety evaluation and the like.

Description

Technical Field

[0001] The present invention belongs to the field of agricultural remote sensing monitoring and is a large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of unmanned aerial vehicle (UAV) and satellite images. Background Art

[0002] Regional-scale yield estimation using remote sensing is a crucial foundation for food security risk assessment and agricultural policy formulation. High-precision and efficient estimation techniques are crucial for ensuring food security and promoting sustainable agricultural development. Satellite remote sensing data, with its wide coverage, short revisit periods, and low cost, has become the primary data source for regional-scale yield estimation. It provides large-scale, continuous surface information, supporting dynamic crop growth monitoring and yield estimation. However, due to limitations in spatial resolution and spectral information, satellite remote sensing data remains significantly inadequate for application in complex cropping areas (such as fragmented fields), making it difficult to accurately capture crop growth variations within fields. In contrast, unmanned aerial vehicle (UAV) remote sensing technology, with its high flexibility and the ability to tailor acquisition conditions to meet demand, can generate ultra-high spatial resolution imagery. It has been widely used for crop yield estimation at the park level. However, the limited observation range of UAV remote sensing technology makes it inadequate for monitoring large regional areas. Existing remote sensing-based yield estimation methods mostly rely on machine learning algorithms, which require large amounts of ground truth data to train high-precision models. However, traditional ground sampling is mostly destructive sampling, which is labor-intensive, time-consuming, and costly, making it difficult to meet the needs of large-scale and timely sampling. In addition, the mismatch between the ground sampling scale and the resolution of satellite data, as well as the impact of internal growth differences in fields, further increase the uncertainty of yield estimates. Therefore, developing a set of efficient yield sample augmentation methods, integrating drone and satellite remote sensing data, and utilizing advanced algorithms are crucial to improving the accuracy and efficiency of regional-scale yield estimates. At the same time, clarifying the impact mechanism of internal growth differences in fields on the uncertainty of regional-scale yield estimates is also a key issue that needs to be addressed. Summary of the Invention

[0003] The technical problem solved by the present invention is to provide a large-scale remote sensing estimation method for winter wheat yield based on cross-scale information fusion of UAV and satellite images. Through UAV multispectral data, a high-precision estimation model for winter wheat yield on the UAV platform is constructed, and combined with Sentinel-2 data, an augmented sample data set of winter wheat yield is successfully obtained. Based on high-spatial-resolution UAV images, the internal spatial heterogeneity index of satellite image plots is constructed to clarify the impact of internal growth differences in plots on satellite wheat yield estimation, and the optimal sample augmentation strategy is determined based on KLD analysis, thereby further obtaining a high-quality winter wheat sample augmentation data set. Finally, based on a deep learning algorithm, a high-precision estimation of winter wheat yield at a regional scale with a spatial resolution of 10 meters is achieved. This method overcomes the dependence of traditional satellite yield estimation methods on ground-measured sample data, has high accuracy, efficiency, and robustness, and can be widely used for large-scale, high-precision, and high-spatial-resolution winter wheat yield estimation.

[0004] The technical solutions for achieving the purpose of the present invention are:

[0005] A large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of UAV and satellite images includes the following steps:

[0006] Step 1: Obtain RedEdge MX multispectral UAV images and Sentinel-2 satellite images of the critical growth period of winter wheat in the study area. Preprocess the RedEdge MX multispectral UAV images using Pix4D software, and preprocess and export the Sentinel-2 satellite images using the remote sensing cloud platform. This includes:

[0007] Step 1-1: Obtain RedEdge MX multispectral UAV images of the key growth period of winter wheat in the study area, and perform UAV image stitching, radiometric correction, geometric correction, and resampling to 1 meter.

[0008] Step 1-2: Obtain Sentinel-2 images of the key growth period of winter wheat in the study area on the remote sensing cloud computing platform, perform cloud mask processing, and export them.

[0009] Step 2: Perform relative radiometric correction on RedEdge MX multispectral UAV imagery based on contemporaneous Sentinel-2 satellite imagery. Use RedEdge MX multispectral UAV imagery during the critical growth period of winter wheat to obtain ground-based sample features. Also, obtain actual wheat yield as sample data. Analyze the sample data to obtain the optimal feature combination, divide the data into training and validation sets, and train a gradient boosting regression (XGBoost) UAV winter wheat yield estimation model to determine the UAV platform winter wheat yield estimation model. Acquire multispectral UAV imagery during the critical growth period of winter wheat using the UAV platform. Input these images into the UAV platform winter wheat yield estimation model to obtain the UAV platform winter wheat yield estimation results. This includes:

[0010] Step 2-1: Using the relative radiometric correction algorithm (MACA), relative radiometric correction is performed based on the Sentinel-2 satellite imagery and RedEdgeMX multispectral UAV imagery from the same period. This reduces the spectral differences between the UAV and satellite sensors and produces the corrected RedEdge MX multispectral UAV imagery.

[0011] Step 2-2: Based on ground-based measured data, spectral indices were extracted using calibrated RedEdge MX multispectral UAV images of winter wheat during its key growth period. Four UAV winter wheat yield estimation models were constructed: gradient boosting regression (XGBoost), random forest regression (RFR), support vector machine regression (SVR), and deep neural network (DNN).

[0012] Step 2-3: Compare four different UAV winter wheat yield estimation models and apply the coefficient of determination R 2 , root mean square error RMSE, relative root mean square error RRMSE evaluation and verification results, and determine the winter wheat yield estimation model XGBoost of the UAV platform;

[0013] Step 2-4: Based on the ground-based measured data, the spectral index is extracted using the calibrated RedEdge MX multispectral UAV imagery of the key growth period of winter wheat. The differences in the spectral index of winter wheat are analyzed to determine the best feature combination, which is recorded as Comb optimal ;

[0014] Steps 2-5: Based on XGBoost and Comb optimal Construct a UAV winter wheat yield estimation model; obtain multispectral UAV images of winter wheat during the key growth period of winter wheat through the UAV platform, input them into the UAV platform winter wheat yield estimation model, and use them to estimate the UAV platform winter wheat yield.

[0015] Step 3: Extract satellite pixel grids using Sentinel-2 satellite imagery and integrate them with the UAV-derived winter wheat yield estimation results obtained in Step 2 to obtain an augmented winter wheat yield sample based on UAV upscaling. This includes:

[0016] Step 3-1: Treat each pixel of the Sentinel-2 image as a grid and perform vector extraction to obtain satellite pixel grid data;

[0017] Step 3-2: Spatial matching of the UAV winter wheat yield estimation image (1 meter) with the satellite pixel grid data (10 meters). Each satellite pixel grid contains 100 UAV yield values and their mean is calculated as the yield value of the satellite pixel grid to obtain an augmented sample of winter wheat yield based on UAV upscaling.

[0018] Step 4: To clarify the impact of internal spatial heterogeneity of fields on satellite winter wheat yield estimation, the spatial heterogeneity of Sentinel-2 satellite pixels was extracted and a sensitivity analysis was conducted by combining UAV multispectral imagery and the satellite pixel grid obtained in Step 3. This was to evaluate the impact of model input on the uncertainty of winter wheat yield estimation and determine the optimal strategy for winter wheat yield estimation. The augmented winter wheat yield sample was then optimized and added to the ground-based measured samples. The optimized augmented winter wheat yield sample was combined with the features obtained from Sentinel-2 satellite imagery during the key growth period of winter wheat as sample data. The data was divided into training and validation sets, and a deep neural network (DNN) was trained to determine the UAV and satellite fusion winter wheat yield estimation model. Sentinel-2 imagery of winter wheat during the key growth period was obtained through a satellite platform and input into the UAV and satellite fusion winter wheat yield estimation model to obtain large-scale field-level winter wheat yield estimation results. The system includes:

[0019] Step 4-1: Obtain the winter wheat yield augmentation sample according to step 3, and combine it with Sentinel-2 of the key growth period of winter wheat

[0020] The features obtained from satellite images were used as sample data and divided into training and validation sets to construct four satellite winter wheat yield estimation models: gradient boosting XGBoost regression, random forest RFR regression, support vector machine SVR regression, and deep neural network DNN.

[0021] Step 4-2: Compare four different satellite winter wheat yield estimation models and apply the coefficient of determination R 2 , root mean square error

[0022] The RMSE and relative root mean square error (RRMSE) were used to evaluate and verify the results, and the DNN model for winter wheat yield estimation based on the fusion of UAV and satellite was determined;

[0023] Step 4-3: To determine the optimal sensitive spatial heterogeneity index, calculate the grayscale image of the drone. Based on the satellite pixel grid data, take each grid as a unit and use the gray level co-occurrence matrix GLCM to calculate the satellite scale spatial heterogeneity index contrast, dissimilarity, correlation, entropy and so on.

[0024] Entropy, inverse difference moment IDM and angular second moment ASM; the specific formula of Grayscale is:

[0025] Grayscale=(0.3×NIR)+(0.59×RED)+(0.11×GREEN)

[0026] Among them, NIR, RED, and GREEN represent the reflectance values of UAV image band 5, band 3, and band 2 respectively;

[0027] Band 5 is 820-860nm, Band 3 is 663-673nm, and Band 2 is 550-570nm;

[0028] The specific formulas of spatial heterogeneity indicators Contrast, Diss, Corr, Entropy, Inverse Difference Moment IDM and Second-Order Angle Moment ASM are as follows:

[0029]

[0030]

[0031]

[0032]

[0033]

[0034]

[0035] Where P(i,j) represents the pixel value at position (i,j) in the Grayscale image, μ represents the mean, and σ represents the standard deviation;

[0036] Step 4-4: Based on the ground survey data, vegetation features were extracted using the calibrated UAV images and input into the random forest RF classifier to build an RF classification model for winter wheat pixel extraction. Based on the satellite pixel grid data, the winter wheat vegetation fraction (WVF), a satellite-scale spatial heterogeneity indicator, was calculated for each grid.

[0037] Step 4-5: Analyze the correlation Pearson_r between satellite-scale winter wheat spatial heterogeneity indicators Contrast, Diss, Corr, Entropy, IDM, ASM and WVF and yield, and use the maximum value of Pearson_r as the criterion to determine the optimal sensitive spatial heterogeneity indicator Entropy_F. The specific formula of Pearson_r is:

[0038]

[0039] Where O and P represent the spatial heterogeneity index and yield value of each sample of the winter wheat yield augmentation sample, respectively. and Represents the spatial heterogeneity index and the mean yield, respectively. Pearson_r has a value between -1 and 1, where -1 indicates a complete negative correlation, 1 indicates a complete positive correlation, and 0 indicates no correlation.

[0040] Steps 4-6: Calculate the similarity KLD values between the winter wheat yield augmentation samples and the ground measured samples in different Entropy_F value ranges based on the Kullback-Leibler divergence KLD, use the minimum KLD value as the criterion to determine the optimal threshold of Entropy_F in each ecological zone, and use the optimal Entropy_F threshold to eliminate the winter wheat yield augmentation samples that are not within the threshold range to obtain the optimized winter wheat yield augmentation samples; specifically, first, normalize the Entropy_F value range to 0 to 1, and divide it into different threshold ranges of 0-0.4, 0-0.5, 0-0.6, 0-0.7, 0 -0.8, 0-0.9 and 0-1; through these Entropy_F threshold ranges, the corresponding ranges of winter wheat yield augmented samples were screened, and the similarity KLD value between the winter wheat yield augmented samples in the Entropy_F threshold range and the ground-measured samples was calculated; secondly, the smaller the KLD value, the higher the similarity between the winter wheat yield augmented samples in the Entropy_F threshold range and the ground-measured samples. By comparing the similarity between the winter wheat yield augmented samples in different Entropy_F threshold ranges and the ground-measured samples, the minimum KLD value was used as the criterion to determine the optimal Entropy_F threshold range. The specific formula of KLD is:

[0041]

[0042] Where P(x) and Q(x) represent the probability distribution of the original data and the augmented data, respectively. A value of 0 indicates that the distribution is exactly the same. A larger value indicates that the difference and information loss between the original data and the augmented data are greater.

[0043] Steps 4-7: The optimized winter wheat yield augmented samples were added to the ground-based measured samples. The features obtained from Sentinel-2 satellite imagery during the key growth period of winter wheat were used as sample data. The data was divided into training and validation sets, and a deep neural network (DNN) was trained to obtain the final UAV and satellite fusion winter wheat yield estimation model, which was used to estimate large-scale field-level winter wheat yield.

[0044] Steps 4-8: Apply the coefficient of determination R 2 , root mean square error RMSE, and relative root mean square error RRMSE evaluation and verification results.

[0045] Step 5: Migrate the final UAV and satellite fusion winter wheat yield estimation model in step 4 to the target area and target year, evaluate the estimation accuracy of winter wheat in different areas and years, and finally obtain large-scale field-level winter wheat yield estimation results.

[0046] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0047] 1. The present invention's large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drones and satellite images uses drones as an intermediate bridge to achieve winter wheat yield sample augmentation using drone data. This method is highly efficient and easy to promote.

[0048] 2. The large-scale remote sensing estimation method for winter wheat yield based on cross-scale information fusion of drone and satellite images of the present invention utilizes cross-scale information fusion of drone data and satellite data to clarify the impact of internal spatial heterogeneity of field plots on satellite yield estimation, and adopts deep learning algorithm to perform large-scale winter wheat yield estimation. This method is highly reliable, universal, and timely.

[0049] 3. The large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images of the present invention can achieve accurate and efficient estimation of winter wheat yield over a large area, and effectively reveal the spatial changes in winter wheat yield in all fields over a large area. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is the technical roadmap of the large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of UAV and satellite images of the present invention.

[0051] Figure 2 This is a principle block diagram of winter wheat yield sample augmentation and spatial heterogeneity index construction in the large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images of the present invention.

[0052] Figure 3This is a graph of the accuracy evaluation results of winter wheat yield estimation in different demonstration counties in 2023 based on the large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images of the present invention, the winter wheat yield estimation model XGBoost based on the drone platform and the optimal feature combination.

[0053] Figure 4 This is a graph showing the accuracy evaluation results of the 2023 winter wheat yield estimation method based on the cross-scale information fusion of drone and satellite images and the winter wheat yield estimation model DNN based on different drone and satellite fusion models.

[0054] Figure 5 This is a Pearson correlation coefficient (r) graph of the winter wheat yield augmented sample and 12 spatial variation indices of the large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite imagery of the present invention (wherein the mark "_B" represents the jointing-heading period, and the mark "_F" represents the flowering-filling period).

[0055] Figure 6 This is a KLD evaluation chart of the similarity between the augmented sample data and the original data of winter wheat yield under seven different threshold ranges determined by Entropy_F for the large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of UAV and satellite images of the present invention.

[0056] Figure 7 This is a comparison chart of the winter wheat yield estimation accuracy of the UAV and satellite fusion winter wheat yield estimation model DNN in different regions under three combinations (combinations #1-3) of the large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of UAV and satellite imagery of the present invention (wherein, (A) Combination #1: Entropy_F range is selected as 0-0.6 (XH and WJ) and 0-0.7 (GY), (B) Combination #2: Entropy_F range is selected as 0-0.7 (XH and WJ) and 0-0.8 (GY), (C) Combination #3: Entropy_F range is uniformly selected as 0-0.4 (GY, XH and WJ)).

[0057] Figure 8 This is the 2023 Jiangsu Province winter wheat yield estimation map based on the large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images of the present invention (wherein, (A) represents the 2023 Jiangsu Province winter wheat yield estimation map produced by the ground-satellite winter wheat yield estimation model, and (B) represents the 2023 Jiangsu Province winter wheat yield estimation map produced by the drone and satellite fusion winter wheat yield estimation model).

[0058] Figure 9This is a comparison chart of winter wheat yield estimates based on a ground-satellite winter wheat yield estimation model and a DNN-based winter wheat yield estimation model for the large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images of the present invention (wherein the first row is a false color composite image of Sentinel-2 images (R: near infrared, G: red, B: blue), the yellow rectangles in the first row represent the actual yield values of the sampling points, and the black rectangles in the last two rows represent the estimated yield values of the sampling points).

[0059] Figure 10 This is a comparison chart of the migration capabilities of the winter wheat estimation model based on the province-wide field measurement data in 2023 for the large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images of the present invention (wherein, the left picture: ground-satellite winter wheat yield estimation model DNN, the right picture: drone and satellite fusion winter wheat yield estimation model DNN). DETAILED DESCRIPTION

[0060] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention.

[0061] The implementation of the present invention is based on UAV multispectral images and Sentinel-2 satellite images. The image parameters and features used are shown in Table 1.

[0062] Table 1 Remote sensing image parameters and related features used in the study

[0063]

[0064]

[0065] This example was implemented in Jiangsu Province, a major wheat-growing province in China. Jiangsu boasts a winter wheat planting area exceeding 30 million mu (approximately 1.5 million hectares) and a yield exceeding 13.6 million tons, ranking fifth nationwide. The province's main production areas are Xuzhou, Huai'an, Nantong, and Taizhou. Furthermore, the method was systematically evaluated and validated using field-level winter wheat yield data from Jiangsu Province in 2023 (a harvest year), effectively demonstrating the advancement, universality, and robustness of the present invention.

[0066] A large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of UAV and satellite images, such as Figure 1As shown in the figure, it mainly includes remote sensing image data acquisition and preprocessing, ground-based winter wheat yield dataset, construction of UAV-scale winter wheat yield estimation model, winter wheat yield sample expansion, determination of the optimal strategy for winter wheat yield estimation based on cross-scale information fusion between UAV and satellite, and large-scale winter wheat yield estimation and verification. Specifically, it includes the following steps:

[0067] Step 1: Obtain RedEdge MX multispectral UAV imagery (1 meter) and Sentinel-2 satellite imagery (10 meters) of the study area during the critical growth period of winter wheat. Preprocess the RedEdge MX multispectral UAV imagery using Pix4D software, and preprocess and export the Sentinel-2 satellite imagery using the remote sensing cloud platform. This includes:

[0068] Step 1-1: Obtain RedEdgeMX multispectral drone images of winter wheat during the key growth periods (booting-heading stage, flowering-grain filling stage) in the study area. The drone flight mission settings are: flight altitude of 120 meters, flight speed of 10 meters per second, heading and lateral overlap rates of 75% and 85% respectively, and obtain geographic control points (GCPs). Using Pix4D software, drone image stitching is performed, radiometric correction is achieved based on the correction plate, and geometric correction is achieved using GCPs. In addition, the drone images are resampled to 1 meter to match the ground sampling. Finally, a total of 60 drone images of three demonstration counties in Jiangsu Province in 2023 (Guanyun County GY, Xinghua City XH and Wujiang District WJ) were obtained.

[0069] Steps 1-2: Acquire 60 Sentinel-2 images of the key growth stages (booting-heading and flowering-grain filling) of winter wheat in 2023 in three demonstration counties using a remote sensing cloud computing platform. Decloud the Sentinel-2 images to obtain cloud-free images. The specific acquisition time of the satellite images used in this invention must be no more than five days different from the acquisition time of the drone images.

[0070] Step 2: Perform relative radiometric correction on RedEdge MX multispectral UAV imagery based on contemporaneous Sentinel-2 satellite imagery. Use RedEdge MX multispectral UAV imagery during the critical growth period of winter wheat to obtain ground-based sample features. Also, obtain actual wheat yield as sample data. Analyze the sample data to obtain the optimal feature combination, divide the data into training and validation sets, and train a gradient boosting regression (XGBoost) UAV winter wheat yield estimation model to determine the UAV platform winter wheat yield estimation model. Acquire multispectral UAV imagery during the critical growth period of winter wheat using the UAV platform. Input these images into the UAV platform winter wheat yield estimation model to obtain the UAV platform winter wheat yield estimation results. This includes:

[0071] Step 2-1: Using the relative radiometric correction algorithm MACA, relative radiometric correction is performed based on the Sentinel-2 satellite imagery and RedEdgeMX multispectral UAV imagery from the same period to reduce the spectral differences between the UAV and satellite sensors and obtain the corrected RedEdge MX multispectral UAV imagery.

[0072] Step 2-2: During the winter wheat maturity period, ground-based yield data were obtained, resulting in 360 county-level yield samples from three demonstration counties in 2023. Four UAV-based winter wheat yield estimation models were constructed based on spectral indices extracted from calibrated RedEdge MX multispectral UAV imagery during the critical growth period of winter wheat: gradient boosting regression (XGBoost), random forest regression (RFR), support vector machine regression (SVR), and deep neural network (DNN).

[0073] Step 2-3: Compare the four different UAV winter wheat yield estimation models in step 2-2 and apply the coefficient of determination R 2 , root mean square error RMSE, and relative root mean square error RRMSE were used to evaluate the accuracy and determine the XGBoost model for winter wheat yield estimation on the UAV platform.

[0074] Steps 2-4: XGBoost was applied to the three demonstration counties respectively. Based on the ground-based measured data, the spectral index was extracted using the calibrated RedEdge MX multispectral drone imagery of the key growth period of winter wheat. The differences in the spectral index of winter wheat in different ecological zones were analyzed to determine the optimal feature combination for each demonstration county, which was recorded as Comb. optimal .

[0075] Step 2-5: Comb the best feature combination obtained based on XGBoost and step 2-4 optimal Construct three demonstration counties’ UAV winter wheat yield estimation models ( Figure 3 ), and used to estimate winter wheat yield from UAV platforms.

[0076] Step 3: Extract satellite pixel grids using Sentinel-2 satellite imagery and integrate them with the winter wheat yield estimation results from the UAV platform obtained in Step 2 to obtain an augmented winter wheat yield sample based on UAV upscaling. This includes:

[0077] Step 3-1: Treat each pixel of the Sentinel-2 image as a grid and perform vector extraction to obtain satellite pixel grid data.

[0078] Step 3-2: Spatial matching of the UAV winter wheat yield estimation image (1 meter) with the satellite pixel grid data (10 meters). Each satellite pixel grid contains 100 UAV yield values and their mean is calculated as the yield value of the satellite pixel grid ( Figure 2 A, D, G), samples with less than 100 drone pixels at the edge of the field were eliminated, and the winter wheat yield augmented samples based on drone upscaling were obtained. A total of 36,590 samples were obtained in the present invention.

[0079] Step 4: To clarify the impact of internal spatial heterogeneity of fields on satellite winter wheat yield estimation, the spatial heterogeneity of Sentinel-2 satellite pixels was extracted and a sensitivity analysis was conducted by combining UAV multispectral imagery and the satellite pixel grid obtained in Step 3. This was to evaluate the impact of model input on the uncertainty of winter wheat yield estimation and determine the optimal strategy for winter wheat yield estimation. The augmented winter wheat yield sample was then optimized and added to the ground-based measured samples. The optimized augmented winter wheat yield sample was combined with the features obtained from Sentinel-2 satellite imagery during the key growth period of winter wheat as sample data. The data was divided into training and validation sets, and a deep neural network (DNN) was trained to determine the UAV and satellite fusion winter wheat yield estimation model. Sentinel-2 imagery of winter wheat during the key growth period was obtained through a satellite platform and input into the UAV and satellite fusion winter wheat yield estimation model to obtain large-scale field-level winter wheat yield estimation results. The specific steps include:

[0080] Step 4-1: Based on the 36,590 winter wheat yield augmentation samples obtained in step 3, the features obtained from Sentinel-2 satellite images during the key growth period of winter wheat are used as sample data, and the training set and validation set are divided.

[0081] Four UAV and satellite fusion winter wheat yield estimation models, including gradient boosting XGBoost regression, random forest RFR regression, support vector machine SVR regression and deep neural network DNN, were used to clarify the contribution and reliability of winter wheat yield sample augmentation to large-scale winter wheat yield estimation.

[0082] Step 4-2: Comparing the winter wheat yield estimation model based on different algorithms of drone and satellite fusion, the advantages of deep learning algorithm DNN are reflected, with the best accuracy ( Figure 4 , R 2 =0.85, RMSE=0.43t / ha, RRMSE=6.65%).

[0083] Step 4-3: To determine the optimal sensitive spatial heterogeneity index, calculate the grayscale image of the drone. Based on the satellite pixel grid data, take each grid as a unit and use the gray level co-occurrence matrix GLCM to calculate the satellite scale spatial heterogeneity index contrast, dissimilarity, correlation, entropy and so on.

[0084] Entropy, inverse difference moment IDM and angular second moment ASM.

[0085] Step 4-4: Based on the ground survey data, wheat and non-wheat sample sets were determined. Vegetation features were extracted using the calibrated UAV imagery and input into the random forest RF classifier. An RF classification model was constructed to extract winter wheat pixels, with an overall accuracy exceeding 90%. Based on the satellite pixel grid data, the winter wheat vegetation fraction (WVF), a satellite-scale spatial heterogeneity indicator, was calculated for each grid.

[0086] Step 4-5: Analyze the spatial heterogeneity of winter wheat at the satellite scale: Contrast, Diss, Corr, Entropy,

[0087] Correlation between IDM, ASM and WVF and yield ( Figure 5 ) to determine the optimal sensitive spatial heterogeneity index Entropy_F. The larger the value, the more obvious the difference in wheat growth within the pixel and the higher the spatial heterogeneity.

[0088] Step 4-6: Normalize the Entropy_F value range to 0 to 1 and divide it into different threshold ranges 0-0.4, 0-0.5,

[0089] 0-0.6, 0-0.7, 0-0.8, 0-0.9 and 0-1; the corresponding ranges of winter wheat yield augmentation samples were screened by these Entropy_F thresholds, and the similarity KLD values between the winter wheat yield augmentation samples in the corresponding ranges of Entropy_F thresholds and the ground measured samples were calculated ( Figure 6 ); The smaller the KLD value, the higher the similarity between the winter wheat yield augmented samples and the ground-measured samples in the range corresponding to the Entropy_F threshold. By comparing the similarities between the winter wheat yield augmented samples and the ground-measured samples in different Entropy_F threshold ranges, the smallest KLD value in each demonstration county corresponds to the Entropy_F

[0090] The threshold ranges are: 0-0.6 (XH and WJ) and 0-0.7 (GY); based on the optimal Entropy_F threshold of each demonstration county, the optimal strategy for winter wheat yield estimation was formulated, and then the winter wheat yield augmentation sample was optimized.

[0091] Steps 4-7: Add the optimized winter wheat yield augmented samples to the ground measured samples, combine them with the features obtained from Sentinel-2 satellite images during the key growth period of winter wheat as sample data, divide them into training sets and validation sets, train the deep neural network (DNN), and determine the winter wheat yield estimation model based on the fusion of drones and satellites ( Figure 7 A); Sentinel-2 images of winter wheat were acquired during the key growth period of winter wheat through satellite platforms and input into the winter wheat yield estimation model based on the fusion of UAV and satellite to obtain large-scale field-level winter wheat yield estimation results ( Figure 8 B), and the results are significantly better than the traditional ground-satellite winter wheat yield estimation model ( Figure 8 A).

[0092] Steps 4-8: Apply the coefficient of determination R 2 , root mean square error RMSE, and relative root mean square error RRMSE evaluation and verification results.

[0093] Step 5: Migrate the final UAV and satellite fusion winter wheat yield estimation model in step 4 to the target area and target year, evaluate the estimation accuracy of winter wheat in different areas and years, and finally obtain large-scale field-level winter wheat yield estimation results.

[0094] Migrate the final UAV and satellite fusion winter wheat yield estimation model in step 4 to the target area and target year, Figure 9 The results showed that compared with the ground-satellite winter wheat yield estimation model, the UAV-satellite fusion winter wheat yield estimation model produced lower intra-field variability and inter-field variability, effectively distinguishing low-yield areas from high-yield areas. In contrast, the ground-satellite winter wheat yield estimation model showed obvious estimation bias, with the yield values of edge pixels higher than those of internal pixels. In addition, 48 provincial-scale yield samples were obtained during the winter wheat maturity period in Jiangsu Province in 2023, and the county-scale UAV-satellite fusion winter wheat yield estimation model was extended to the provincial scale. The UAV-satellite fusion winter wheat yield estimation model can achieve better spatial expansion of the model ( Figure 10 B, R 2 =0.62, RMSE=0.90t / ha, RRMSE=12.63%), indicating that the winter wheat yield estimation model based on the fusion of UAV and satellite has strong robustness, universality and estimation accuracy, and can achieve timely and accurate estimation of winter wheat yield.

Claims

1. A large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of UAV and satellite images, characterized by: The following steps are involved: Step 1: Obtain RedEdge MX multispectral UAV images and Sentinel-2 satellite images of the critical growth period of winter wheat in the study area. Preprocess the RedEdge MX multispectral UAV images using Pix4D software, and preprocess and export the Sentinel-2 satellite images using the remote sensing cloud platform. Step 2: Relative radiometric correction was performed on RedEdge MX multispectral UAV imagery based on contemporaneous Sentinel-2 satellite imagery. Ground-based sample features were obtained using RedEdge MX multispectral UAV imagery during the critical growth period of winter wheat. Actual wheat yield was also obtained as sample data. The sample data was analyzed to determine the optimal feature combination. The data was then divided into training and validation sets, and a gradient boosting regression (XGBoost) UAV winter wheat yield estimation model was trained to determine the UAV platform winter wheat yield estimation model. Multispectral UAV imagery was obtained from the UAV platform during the critical growth period of winter wheat. These images were input into the UAV platform winter wheat yield estimation model to obtain the UAV platform winter wheat yield estimation results. Step 3: Use Sentinel-2 satellite imagery to extract satellite pixel grids and integrate them with the winter wheat yield estimation results from the UAV platform obtained in Step 2 to obtain an augmented winter wheat yield sample based on UAV upscaling. Step 4: To clarify the impact of internal spatial heterogeneity of fields on satellite winter wheat yield estimation, the spatial heterogeneity of Sentinel-2 satellite pixels was extracted and a sensitivity analysis was conducted, combining UAV multispectral imagery and the satellite pixel grid obtained in Step 3. This was to evaluate the impact of model input on the uncertainty of winter wheat yield estimation and determine the optimal strategy for winter wheat yield estimation. The augmented winter wheat yield sample was then optimized and added to the ground-based measured samples. The optimized augmented winter wheat yield sample was combined with the features obtained from Sentinel-2 satellite imagery during the key growth period of winter wheat as sample data, and the data was divided into training and validation sets. A deep neural network (DNN) was trained to determine the UAV and satellite fusion winter wheat yield estimation model. Sentinel-2 imagery of winter wheat was obtained from the satellite platform during the key growth period of winter wheat and input into the UAV and satellite fusion winter wheat yield estimation model to obtain large-scale field-level winter wheat yield estimation results.

2. The large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images according to claim 1 is characterized in that: Step 1 specifically includes: Step 1-1: Obtain RedEdge MX multispectral UAV images of the key growth period of winter wheat in the study area, and perform UAV image stitching, radiometric correction, geometric correction, and resampling to 1 meter. Step 1-2: Obtain Sentinel-2 images of the key growth period of winter wheat in the study area on the remote sensing cloud computing platform, perform cloud mask processing, and export them.

3. The large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images according to claim 1 is characterized in that: Step 2 specifically includes: Step 2-1: Using the relative radiometric correction algorithm (MACA), relative radiometric correction is performed based on the Sentinel-2 satellite imagery and RedEdge MX multispectral UAV imagery from the same period. This reduces the spectral differences between the UAV and satellite sensors and produces the corrected RedEdge MX multispectral UAV imagery. Step 2-2: Based on the ground-based measured data, the spectral index is extracted using the calibrated RedEdge MX multispectral UAV imagery of the key growth period of winter wheat. The differences in the spectral index of winter wheat are analyzed to determine the best feature combination, which is recorded as Comb optimal ; Step 2-3: Based on XGBoost and Comb optimal Construct a winter wheat yield estimation model based on a UAV platform and use it to estimate winter wheat yield based on a UAV platform; Steps 2-4: Apply the coefficient of determination R 2 , root mean square error RMSE, and relative root mean square error RRMSE evaluation and verification results.

4. The large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images according to claim 1 is characterized in that: Step 3 specifically includes: Step 3-1: Treat each pixel of the Sentinel-2 image as a grid and perform vector extraction to obtain satellite pixel grid data; Step 3-2: Spatially match the imagery containing the winter wheat yield estimation results from the drone platform with the satellite pixel grid data. Each satellite pixel grid contains 100 drone yield values, and their mean is calculated as the yield value of the satellite pixel grid to obtain an augmented sample of winter wheat yield based on drone upscaling.

5. The large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images according to claim 1 is characterized in that: Step 4 specifically includes: Step 4-1: To determine the optimal sensitive spatial heterogeneity index, calculate the grayscale image of the drone. Based on the satellite pixel grid data, take each grid as a unit and use the gray level co-occurrence matrix GLCM to calculate the satellite scale spatial heterogeneity index contrast, dissimilarity, correlation, entropy, inverse difference moment (IDM), and angular second moment (ASM). The specific formula of Grayscale is: Grayscale=(0.3×NIR)+(0.59×RED)+(0.11×GREEN) Among them, NIR, RED, and GREEN represent the reflectance values of UAV image band 5, band 3, and band 2 respectively; Band 5 is 820-860nm, Band 3 is 663-673nm, and Band 2 is 550-570nm; Step 4-2: Based on the ground survey data, vegetation features were extracted using the calibrated UAV imagery and input into the random forest RF classifier to construct an RF classification model for winter wheat pixel extraction. Based on the satellite pixel grid data, the winter wheat vegetation fraction (WVF), a satellite-scale spatial heterogeneity indicator, was calculated for each grid. Step 4-3: Analyze the Pearson_r correlation between satellite-scale winter wheat spatial heterogeneity indicators Contrast, Diss, Corr, Entropy, IDM, ASM and WVF and yield, and use the maximum value of Pearson_r as the criterion to determine the most sensitive spatial heterogeneity indicator Entropy_F; Step 4-4: Calculate the similarity KLD values between the winter wheat yield augmentation samples and the ground measured samples within different Entropy_F value ranges based on the Kullback-Leibler divergence KLD. Use the minimum KLD value as the criterion to determine the optimal Entropy_F threshold for each ecological zone. Use the optimal Entropy_F threshold to eliminate the winter wheat yield augmentation samples that are not within the threshold range to obtain the optimized winter wheat yield augmentation samples. Steps 4-5: The optimized winter wheat yield augmented sample is added to the ground-based measured sample. Features obtained from Sentinel-2 satellite imagery during the key growth period of winter wheat are used as sample data. The data is divided into training and validation sets, and a deep neural network (DNN) is trained to obtain a winter wheat yield estimation model that integrates drones and satellites. This model is then used to estimate winter wheat yield at the large-scale field level. Steps 4-6: Apply the coefficient of determination R 2 , root mean square error RMSE, and relative root mean square error RRMSE evaluation and verification results.

6. The large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images according to claim 5 is characterized in that: In step 4-1, the specific formulas of spatial heterogeneity indicators Contrast, Diss, Corr, Entropy, Inverse Difference Moment IDM and Second-Order Angular Moment ASM are as follows: Where P(i,j) represents the pixel value at position (i,j) in the Grayscale image, μ represents the mean, and σ represents the standard deviation.

7. The large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images according to claim 5 is characterized in that: In step 4-3, the specific formula of Pearson_r is: Where O and P represent the spatial heterogeneity index and yield value of each sample in the winter wheat yield augmentation sample, respectively. and They represent the spatial heterogeneity index and the mean yield respectively. The value of Pearson_r is between -1 and 1, where -1 indicates a complete negative correlation, 1 indicates a complete positive correlation, and 0 indicates no correlation.

8. The large-scale winter wheat yield remote sensing estimation method based on cross-scale information fusion of drone and satellite images according to claim 5 is characterized in that: In step 4-4, the minimum KLD value is used as the criterion to determine the optimal threshold of Entropy_F for each ecological zone, including: First, the Entropy_F value range was normalized to 0 to 1, and different threshold ranges were divided into 0-0.4, 0-0.5, 0-0.6, 0-0.7, 0-0.8, 0-0.9 and 0-1. The winter wheat yield augmentation samples in the corresponding ranges were screened by these Entropy_F threshold ranges, and the similarity KLD values between the winter wheat yield augmentation samples in the Entropy_F threshold ranges and the ground measured samples were calculated. The specific formula of KLD is: Where P(x) and Q(x) represent the probability distribution of the original data and augmented data, respectively. A value of 0 indicates that the distribution is exactly the same. A larger value indicates that the difference and information loss between the original data and augmented data are greater. Secondly, the smaller the KLD value, the higher the similarity between the winter wheat yield augmented samples and the ground measured samples in the corresponding range of the Entropy_F threshold. By comparing the similarities between the winter wheat yield augmented samples and the ground measured samples in different Entropy_F threshold ranges, the minimum KLD value was used as the criterion to determine the optimal Entropy_F threshold range.

Images