Multi-variety rice yield prediction method based on time sequence remote sensing image

Abstract

The invention discloses a multi-variety rice yield prediction method based on a time sequence remote sensing image. The method comprises the following steps: S100, performing unmanned aerial vehicle remote sensing data acquisition and processing on an experimental area; s200, extracting time sequence remote sensing image features; s300, based on the extracted time sequence remote sensing image features, creating rice cumulative growth features CGC to represent growth features of rice; s400, dividing rice varieties based on the rice accumulative growth characteristic curve; s500, recording the actual yield of the multi-variety rice in the experimental area; and S600, constructing a rice yield prediction model by adopting a multivariable regression model based on a Transform architecture in rice varieties of a single variety category. The method provided by the invention can effectively solve the problem of low precision of multi-variety rice yield estimation, and can provide help for field fine management and variety breeding under field natural planting conditions and breeding environments.

Description

Technical Field

[0001] The present invention relates to the technical field of rice yield prediction, and in particular to a method for predicting the yield of multiple varieties of rice based on time series remote sensing images. Background Art

[0002] Traditional methods for crop yield estimation primarily include field sampling surveys, weather forecasts, and agronomic forecasts. These methods are time-consuming, labor-intensive, and require expert knowledge, making crop yield estimation complex, time-consuming, and unfeasible for large-scale implementation. Furthermore, traditional manual yield measurement methods on the ground are inefficient and prone to plant damage, hindering subsequent yield prediction. Remote sensing, as a rapid monitoring, analysis, and diagnostic technology, can accurately acquire imagery of crops over large areas, and is therefore widely used in various aspects of smart agriculture. With the development of remote sensing technology, optical satellite data has achieved certain spatial, temporal, and spectral resolutions, and has been widely used for large-scale crop yield prediction since the 1970s. However, the spatial and temporal resolution of satellite data is relatively coarse and susceptible to weather influences, making it difficult to guarantee the timeliness of data acquisition, which significantly impacts regional crop yield forecasts. Unmanned aerial vehicle (UAV) remote sensing technology, with its flexible ability to carry a variety of sensors, efficiently acquires high-resolution spatial and temporal imagery data. In recent years, it has become an effective means of monitoring crop growth. Remote sensing using drones can monitor larger areas than ground-based measurement techniques. Compared to satellite and aircraft observations, it offers superior spatial resolution, more frequent observation opportunities, and lower costs, bringing new possibilities to remote sensing technology. Therefore, drone remote sensing technology has great potential for estimating rice yields in the field.

[0003] Existing crop yield estimation studies using remote sensing rely heavily on differentiated field data arrangements. Most field experiments are designed to control fertilizer gradients, irrigation, and planting patterns. However, actual crop field conditions often struggle to meet these requirements. This makes existing yield estimation models difficult to apply under real-world field conditions. During actual planting, field managers will add sufficient fertilizer to the soil. Rice fields grown by different farmers may vary in variety and planting date. Especially given China's current situation of fragmented farmland, farmers manage their fields with greater precision. This makes uneven fertilizer or irrigation distribution a rare occurrence. Consequently, existing research on yield estimation using remote sensing data from natural field conditions is limited, making it difficult to meet the needs of real-world farmland management.

[0004] The rapid development of drones has increased the frequency and spatiotemporal resolution of crop imagery data. The Vegetation Index (VI) calculated from UAV imagery has been shown to be an effective indicator for crop yield prediction. Furthermore, information such as canopy texture, canopy height, and canopy temperature has been proposed for crop yield estimation. However, for rice, canopy changes are extremely complex over the growth cycle, making it difficult to effectively develop universal yield prediction indicators using these indicators. Some believe that yield is the result of the nonlinear accumulation of crop photosynthetic products, and traditional linear regression models struggle to accurately capture this complex nonlinear relationship, resulting in poor yield prediction using the VI in the late growth stages of crops. Consequently, many studies have favored the use of nonlinear models, such as machine learning, for crop yield prediction. Machine learning can better handle nonlinear relationships between large amounts of data. However, the performance of machine learning models is affected by multiple factors, including training data, input variables, crop type, and growth stage. Consequently, they are less able to handle complex nonlinear relationships and time-series data, and are therefore ineffective in predicting yield for multiple varieties with significant differences in morphological structure and growth period.

[0005] In recent years, phenology has proven to be crucial information for crop yield prediction. Research has shown that dividing time windows into phenological stages allows for more precise extraction of environmental variables, capturing the complex interactions between crop growth and external factors. Current crop yield estimation research primarily focuses on using variables from a single period or integrating multiple periods. When using single-period variables, the correlation between individual period variables and yield, from sowing to maturity, is often analyzed, and highly correlated variables are then selected for direct yield prediction. When using multi-period variables, a matrix of variables from multiple periods is often obtained as input to machine learning or multivariate regression models for direct yield estimation. However, crop variables from different growth stages have different growth dimensions, making direct comparison and integration impossible. Therefore, grouping varieties with similar growth characteristics is crucial for ensuring accurate growth monitoring and yield prediction. Summary of the Invention

[0006] In response to the above technical problems in the related art, the present invention provides a multi-variety rice yield prediction method based on time series remote sensing images, which can solve the above problems.

[0007] To achieve the above technical objectives, the technical solution of the present invention is implemented as follows: A method for predicting the yield of multiple varieties of rice based on time series remote sensing images comprises the following steps: S100, acquire and process UAV remote sensing data in the experimental area; S110, using a drone to acquire and process RGB image data of a rice canopy, where the acquisition time of the RGB image data includes multiple growth periods of the rice; S120, using a drone to acquire and process multispectral image data of rice, wherein the acquisition time of the multispectral image data is consistent with the acquisition time of the RGB image data; S200, extracting time series remote sensing image features; S210: for the pre-processed multispectral image, outlining a multispectral region of interest according to the location of the field in the multispectral image, extracting and calculating the mean reflectance of the field through the multispectral region of interest, and using the mean reflectance as the canopy reflectance of the field; S220, calculate the vegetation index that helps monitor vegetation growth and predict yield by using the canopy reflectance of the field, and use it as a feature for estimating rice yield; S230, obtaining the canopy height and canopy volume of the field; S240, performing tasseled cap transformation on the pre-processed multispectral image to obtain tasseled cap parameters; S300: Create a rice cumulative growth characteristic CGC based on the extracted time series remote sensing image features to characterize the growth characteristics of rice. The specific calculation is as follows: Where RSP represents the remote sensing characteristics of canopy reflectance, vegetation index, canopy height, canopy volume or tasseled cap parameters in the 490 nm, 520 nm, 550 nm, 570 nm, 670 nm, 680 nm, 700 nm, 720 nm, 800 nm, 850 nm, 900 nm and 950 nm bands, and t represents the number of days after rice transplanting. S400, establishing a rice cumulative growth characteristic curve based on a calculation formula for the rice cumulative growth characteristic CGC, wherein the number of days after rice transplanting is used as the abscissa and the cumulative value of the extracted time series remote sensing image features is used as the ordinate, and finely classifying rice varieties using the rice cumulative growth characteristic curve; S500, recording the actual yields of multiple rice varieties in the experimental area; S600. A rice yield prediction model is constructed using a multivariate regression model based on the Transformer architecture for rice varieties within a single variety category, where the actual yield of multiple rice varieties in the experimental area is used as the dependent variable of the prediction model, and the features extracted from the corresponding time series remote sensing images are used as the independent variables of the prediction model.

[0008] Furthermore, the experimental area includes experimental area one and experimental area two, and both experimental area one and experimental area two include several plots, wherein the plots in experimental area one are planted with different hybrid rice varieties, and the plots in experimental area two are planted with rice of different genotypes.

[0009] Furthermore, the processing of RGB image data includes image stitching and three-dimensional reconstruction, generating dense three-dimensional point clouds, digital surface models, and orthophoto stitching images of the experimental area based on the coordinates of the ground control points; the processing of multispectral image data includes geometric processing, radiation processing, and spectral information extraction.

[0010] Furthermore, when acquiring multispectral imaging data of rice, the UAV is equipped with a 12-band multispectral camera, with the central bands being 490nm, 520nm, 550nm, 570nm, 670nm, 680nm, 700nm, 720nm, 800nm, 850nm, 900nm, and 950nm.

[0011] Furthermore, vegetation indices that are helpful for vegetation growth monitoring and yield prediction include: Normalized Difference Vegetation Index: NDVI = (R850nm - R670nm) / (R850nm + R670nm) Green Normalized Difference Vegetation Index: GNDVI = (R850nm - R550nm) / (R850nm + R550nm) Normalized difference red edge index: NDRE = (R850nm - R720nm) / (R850nm + R720nm) Red edge chlorophyll index: CI regedge = R850nm / R720nm-1 Dual-band Enhanced Vegetation Index: EVI2 = 2.5*(R850nm-R670nm) / (R850nm+2.4*R670nm+1) Green edge chlorophyll index: CI green = R850nm / R550nm-1 Among them, R550, R670, R720 and R850 are the canopy reflectance of the fields in the 550nm, 670nm, 720nm and 850nm bands respectively.

[0012] Furthermore, the canopy height and canopy volume of the field plot are obtained in S230 as follows: before transplanting rice to the field, the digital elevation model of the basic ground of the rice field is obtained; after transplanting the seedlings, the digital surface model generated by each aerial photography is subtracted from the digital elevation model of the basic ground to obtain the rice canopy height model; on the rice canopy height model image, the rice canopy range is selected by outlining the region of interest, and the average canopy height of all pixels within each region of interest is calculated as the canopy height of the field plot; after obtaining the rice canopy height model, the canopy volume model is obtained in combination with the ground sampling interval information of each pixel; the canopy volume CV of each field plot can be obtained by outlining the region of interest, and the calculation formula of CV is: Among them H i represents the canopy height of the ith pixel, and GSD is the ground sampling interval.

[0013] Furthermore, in S240 , a tasseled cap transformation is performed on the pre-processed multispectral image to obtain tasseled cap parameters, including brightness, greenness, and the third component.

[0014] Furthermore, the classification of rice varieties in S400 is specifically as follows: in the coordinate of days after transplanting-cumulative growth characteristics, by comparing the angle α formed by the cumulative growth characteristic curves of rice at any two adjacent time points, when the angle α is less than 5°, they are classified into the same category, thereby realizing the fine classification of rice varieties based on the cumulative growth characteristic curve of rice.

[0015] Furthermore, it also includes S700: model verification of the constructed rice yield prediction model, wherein the ten-fold cross-validation method is used in experimental area one to perform model accuracy statistics, and the experimental area two is used as independent verification data to measure the migration ability of the model.

[0016] Beneficial effects of the present invention: The method of the present application can effectively solve the problem of low accuracy in estimating the yield of multiple rice varieties, and can provide assistance for field refined management and variety selection under natural field planting conditions and breeding environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] The present invention will be described in further detail below with reference to the accompanying drawings.

[0019] Figure 1This is a flowchart of a method for predicting multi-variety rice yield based on time series remote sensing images according to an embodiment of the present invention; Figure 2 is a graph showing the change in the green edge chlorophyll index as a function of the number of days after rice transplanting according to an embodiment of the present invention; Figure 3 is a graph showing the cumulative value of the green edge chlorophyll index according to an embodiment of the present invention as a function of the number of days after rice transplanting; Figure 4 yes Figure 3 A magnified view of middle A; Figure 5 is a graph showing the change in canopy height with the number of days after rice transplanting according to an embodiment of the present invention; Figure 6 is a graph showing the cumulative value of canopy height as a function of the number of days after rice transplanting according to an embodiment of the present invention; Figure 7 yes Figure 6 Enlarged view of middle B; Figure 8 This is a schematic diagram of an operation of outlining a multispectral region of interest based on the location of a field in a multispectral image according to an embodiment of the present invention; Figure 9 This is a model framework diagram of the multivariate regression model based on the Transformer architecture described in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of the present invention.

[0021] like Figure 1 As shown, according to the present invention, a method for predicting the yield of multiple varieties of rice based on time series remote sensing images is disclosed, comprising the following steps: S100, acquire and process UAV remote sensing data in the experimental area; S110, using a drone to acquire and process RGB image data of a rice canopy, where the acquisition time of the RGB image data includes multiple growth periods of the rice; S120, using a drone to acquire and process multispectral image data of rice, wherein the acquisition time of the multispectral image data is consistent with the acquisition time of the RGB image data; S200, extracting time series remote sensing image features; S210: for the pre-processed multispectral image, outlining a multispectral region of interest according to the location of the field in the multispectral image, extracting and calculating the mean reflectance of the field through the multispectral region of interest, and using the mean reflectance as the canopy reflectance of the field; S220, calculate the vegetation index that helps monitor vegetation growth and predict yield by using the canopy reflectance of the field, and use it as a feature for estimating rice yield; S230, obtaining the canopy height and canopy volume of the field; S240, performing tasseled cap transformation on the pre-processed multispectral image to obtain tasseled cap parameters; S300: Create a rice cumulative growth characteristic CGC based on the extracted time series remote sensing image features to characterize the growth characteristics of rice. The specific calculation is as follows: Where RSP represents the remote sensing characteristics of canopy reflectance, vegetation index, canopy height, canopy volume or tasseled cap parameters in the 490 nm, 520 nm, 550 nm, 570 nm, 670 nm, 680 nm, 700 nm, 720 nm, 800 nm, 850 nm, 900 nm and 950 nm bands, and t represents the number of days after rice transplanting. S400, establishing a rice cumulative growth characteristic curve based on a calculation formula for the rice cumulative growth characteristic CGC, wherein the number of days after rice transplanting is used as the abscissa and the cumulative value of the extracted time series remote sensing image features is used as the ordinate, and finely classifying rice varieties using the rice cumulative growth characteristic curve; S500, recording the actual yields of multiple rice varieties in the experimental area; S600. A rice yield prediction model is constructed using a multivariate regression model based on the Transformer architecture for rice varieties within a single variety category, where the actual yield of multiple rice varieties in the experimental area is used as the dependent variable of the prediction model, and the features extracted from the corresponding time series remote sensing images are used as the independent variables of the prediction model.

[0022] Example 1: The research data for this application comes from two independent experimental areas, which are used to simulate rice cultivation under natural growth conditions. Experimental area one contains 289 plots, each of which is planted with a different hybrid rice variety. The area of each plot is 1 square meter. Experimental area two contains 48 plots, corresponding to the planting of 48 different genotypes of rice. The area of each plot is about 40 square meters. Except for the different rice genotypes, all field management measures remain consistent. The uniform nitrogen fertilizer application rate is 180 kg / ha. Before transplanting rice, half of the total fertilizer amount is applied in advance as base fertilizer, the remaining quarter is applied during the tillering stage, and the last quarter is applied during the heading stage.

[0023] Example 2: This application used a DJI Phantom 4 Pro quadrotor drone (SZ DJI Technology Co. Ltd., Shenzhen, China) to acquire RGB images of rice canopies. The drone was set at an altitude of 30 meters, with 90% heading overlap and 70% lateral overlap. The drone's built-in gimbal ensured the camera was always pointed vertically downward during acquisition. The camera had an 84° field of view, an image size of 5472 × 3648 pixels, and a ground resolution of 0.8 cm / pixel. Parameters such as aperture and ISO sensitivity were set before the aerial photography according to the actual lighting conditions. These camera parameters remained constant throughout the acquisition process.

[0024] After acquiring RGB images of the experimental area, Agisoft Photoscan Professional software (version 1.4.5, Agisoft LLC, St. Petersburg, Russia) was used for image stitching and 3D reconstruction. Based on the coordinates of the ground control points, a dense 3D point cloud (DPC), digital surface model (DSM), and orthomosaic images of the experimental area were generated. Photoscan software uses structure-from-motion and multi-view stereo algorithms for image processing. RGB data were collected over 10 rice stages: tillering, jointing, booting, heading, and milky maturity, with each collection period lasting 5-10 days.

[0025] Example 3: UAV multispectral imagery was acquired using an S1000 drone (S1000, SZDJI Technology, Co., Ltd., Shenzhen, China) equipped with an MCA series multispectral camera (Tetracam, Inc., Chatsworth, CA, USA). UAV flights were conducted under clear, cloudless, and windless conditions. Data collection was uniformly conducted between 10:00 AM and 2:00 PM local time. The 12-band MCA-12 multispectral camera used (center bands: 490, 520, 550, 570, 670, 680, 700, 720, 800, 850, 900, and 950 nm). Multispectral data acquisition was timed to coincide with RGB image acquisition.

[0026] The UAV multispectral image processing process includes three steps: geometric processing, radiometric processing, and spectral information extraction. The geometric processing of UAV multispectral images acquired with an MCA camera includes vignetting correction, band registration, and distortion correction. This process is performed using PixelWrech2 (Tetracam, Inc., Chatsworth, CA, USA), a specialized MCA image processing software. Radiometric processing primarily involves radiometric calibration, using eight calibration objects with known reflectivity (reflectivity of 3%, 6%, 12%, 24%, 36%, 48%, 56%, and 80%). This radiometric calibration is performed using an empirical linear model, which converts the DN values of the original image into reflectivity.

[0027] Example 4: When extracting features from time series remote sensing images, the extracted features include canopy reflectance, vegetation index, canopy height, canopy volume, and tasseled cap parameters.

[0028] Among them, canopy reflectance: for the pre-processed multispectral image, the multispectral region of interest is outlined according to the location of the field in the multispectral image, and the mean reflectance of the field is extracted and calculated through the multispectral region of interest, and the mean reflectance is used as the canopy reflectance of the field (such as Figure 8 As shown in the figure, the definition of the region of interest is: the relatively homogeneous rice field range after removing the edge position of each field in the remote sensing image, that is, region C).

[0029] Vegetation indices, combining reflectance values from different wavelengths, have been proposed to reflect the dynamic alternations of vegetation throughout its growth cycle. These indices are related to leaf pigmentation, photosynthesis, and plant nutritional status, and are useful for monitoring rice growth. Six vegetation indices proven to be useful for vegetation growth monitoring and yield prediction were selected as features for rice yield estimation. Their calculation formula is as follows: Normalized Difference Vegetation Index: NDVI = (R850nm - R670nm) / (R850nm + R670nm) Green Normalized Difference Vegetation Index: GNDVI = (R850nm - R550nm) / (R850nm + R550nm) Normalized difference red edge index: NDRE = (R850nm - R720nm) / (R850nm + R720nm) Red edge chlorophyll index: CI regedge = R850nm / R720nm-1 Dual-band Enhanced Vegetation Index: EVI2 = 2.5*(R850nm-R670nm) / (R850nm+2.4*R670nm+1) Green edge chlorophyll index: CIgreen = R850nm / R550nm-1 Among them, R550, R670, R720 and R850 are the canopy reflectance of the fields in the 550nm, 670nm, 720nm and 850nm bands respectively.

[0030] After performing band operations on the reflectance image, different vegetation index maps are obtained. Similar to the canopy reflectance acquisition, the mean of the vegetation index of all pixels in the area of interest is used as the canopy vegetation index of the field.

[0031] To accurately determine the canopy height of individual plots and minimize the impact of ground undulation, a Digital Elevation Model (DEM) of the paddy field's underlying ground surface is obtained before transplanting rice seedlings to the field. After transplanting, the Digital Surface Model (DSM) generated from each aerial photo is subtracted from the underlying ground surface DEM to create the rice canopy height model (CHM). On the CHM image, the rice canopy range is selected by delineating the region of interest (ROI). The average canopy height of all pixels within each ROI is calculated as the canopy height for that plot.

[0032] After obtaining the canopy CHM, combined with the ground sampling interval (GSD) information of each pixel, the canopy volume model (CVM) can be obtained. Similar to the CHM, the canopy volume (CV) of each field can be obtained by outlining the area of interest. The calculation formula of CV is: Among them H i represents the canopy height of the ith pixel, and GSD is the ground sampling interval.

[0033] After applying the Tasseled Cap transform to 12-band drone imagery, the result is composed of three factors: brightness, greenness, and the third component. The Tasseled Cap transform is an orthogonal transformation from the spectral space of a surface feature to its feature space. It projects the distribution structure of surface features such as soil and vegetation in multispectral remote sensing into the Tasseled Cap transform space. It also reduces spectral dimensions and concentrates information on a few feature spaces. Its definition formula is: y=Ax+b Where y is the vector after tasseled cap transformation; A is the unit orthogonal matrix, that is, the coefficient matrix of tasseled cap transformation; x is the grayscale value of the image or the apparent reflectivity of the sensor; b is used as an offset vector to avoid negative values in the transformed result.

[0034] After the tasseled cap transformation, the three factors (brightness, greenness, and the third component) are all closely related to the groundscape. Brightness, a weighted sum of 12 band components, reflects the overall brightness variation of the groundscape. Greenness, perpendicular to brightness, is the ratio of the near-infrared to visible bands. It reflects the contrast between the visible bands, especially the red band, and the near-infrared band. It indicates the variation in the greenness of ground vegetation and is closely related to ground vegetation cover, leaf area index, and biomass. The third component is the wetness index, which reflects the moisture characteristics of the soil and vegetation. Similar to calculating the reflectance of a field, the tasseled cap parameters are obtained by defining a rectangular region of interest.

[0035] Embodiment 5: Because the time series characteristics of different canopy reflectance, vegetation index, canopy height, and canopy volume make it difficult to distinguish subtle differences between different rice varieties, a rice cumulative growth characteristic based on different remote sensing characteristics is proposed to characterize rice growth characteristics. The specific calculation is as follows: Among them, RSP represents the remote sensing characteristics of canopy reflectance, vegetation index, canopy height, canopy volume or tassel cap parameters in the 490 nm, 520 nm, 550 nm, 570 nm, 670 nm, 680 nm, 700 nm, 720 nm, 800 nm, 850 nm, 900 nm and 950 nm bands, and t represents the number of days after rice transplanting.

[0036] like Figure 2-7 As shown, CI green Taking the characteristics of canopy height as an example, we select data from 10 fields in the study area to illustrate. Figure 2 and Figure 5 The characteristics of the rice at different times after transplanting are described in green , canopy height) curve, that is, the rice growth characteristic curve of the present invention; using the CGC calculation formula, CI green , the cumulative growth characteristic curve corresponding to the canopy height (such as Figure 3 and Figure 6 shown), cumulative CI green , cumulative canopy height and other parameters are called cumulative growth characteristics of rice; Figure 4 and Figure 7 As shown, the cumulative growth characteristics (cumulative CI green , cumulative canopy height) coordinates, by comparing the angle formed by the cumulative growth characteristic curves of any two adjacent time points (denoted as α ), when the angle is less than 5°, they are divided into the same category, realizing the fine classification of rice varieties based on the cumulative growth characteristic curve.

[0037] Example 6: After all rice genotypes matured, rice yield was measured in each field using manual sampling. After manual harvesting, threshing, drying, moisture content measurement, and weighing, the actual rice yield was calculated based on the number of samples sampled, weight per kilogram, transplanting density, and moisture content.

[0038] After achieving fine classification of rice varieties based on remote sensing features of different time series, a multivariate regression model based on the Transformer architecture was used to construct a rice yield prediction model with multiple variables in a single variety category, in which the features extracted from the drone remote sensing data were used as the independent variable (X) and the corresponding rice yield was used as the dependent variable (Y).

[0039] The proposed multi-variable regression model based on Transformer architecture, named Multihead AttentionRegression (MHAR), is used to predict the yield of multiple rice varieties. Its model framework is as follows Figure 9 Shown, including: Input layer: Input sequence, containing numChannels features (number of independent variables). The input features contain the features of the original data but do not contain position information.

[0040] Positional Embedding Layer: Adds positional embeddings to the input sequence, taking into account temporal information in the sequence. maxPosition is a parameter in the Positional Embedding Layer that defines the maximum number of positions in the sequence. This means that the Positional Embedding Layer generates a positional embedding vector for each position in the sequence, ranging from 0 to maxPosition-1. maxPosition should be at least equal to the length of the longest sequence in the input data.

[0041] Addition layer: Adds the input and position embeddings so that each input feature also contains the position information of that position, which helps the model better capture the relationship between time steps during self-attention calculation.

[0042] Two self-attention layers are the core of the Transformer model. Their primary function is to calculate attention scores between different positions in the sequence to capture dependencies between them. The stacking of two self-attention layers enhances the model's understanding of complex dependencies, capturing multiple dependencies and improving representation capabilities. The first self-attention layer uses a causal mask to ensure sequential dependencies, while the second self-attention layer uses no causal mask and instead leverages global information.

[0043] Index layer: The index layer in the model extracts the output of the last time step of the input sequence. This allows the model to leverage information from the entire sequence to make a final prediction. In this model, the index layer passes the data from the last time step to the fully connected and regression layers for the final prediction output.

[0044] Fully connected layer: This layer linearly transforms a high-dimensional feature vector into a single output value with an output dimension of 1. In regression tasks, the output of a fully connected layer is usually a prediction value. A fully connected layer combines the input features to generate the final prediction value.

[0045] Regression layer: Calculates the error between the predicted value and the true value to guide model training. The model uses the Adam algorithm to optimize the model. Adam (Adaptive Moment Estimation) combines momentum and adaptive learning rate methods and is suitable for training deep neural networks.

[0046] Output layer: outputs the predicted value.

[0047] The MHAR model uses vegetation indices, canopy height, canopy volume, and tassel cap parameters from multiple rice growth stages (including tillering, jointing, booting, heading, and milky stages, with each stage separated by 5-10 days) as input variables. Through a positional encoding layer and a multi-head attention mechanism, it efficiently processes and extracts features from complex spatiotemporal data. Positional encoding adds positional information to the data, enabling the model to understand the temporal order of the input data. The model includes two self-attention layers: the first uses a causal mask to ensure sequential dependencies, while the second, without a causal mask, leverages global information. With temporal features, data from different stages are not subject to interference. Compared to the traditional Transformer model, the MHAR model offers greater flexibility and potential for higher accuracy in the regression task of rice yield prediction. While traditional Transformer models are primarily used in natural language processing and image recognition, the improved MHAR model is more suitable for regression prediction of multivariate spatiotemporal data in fields such as agriculture and environmental science.

[0048] Embodiment seven: To ensure the accuracy of the constructed multi-variety rice yield prediction model, the model needed to be validated. The coefficient of determination (R²), root mean square error (RMSE), and relative root mean square error (RRMSE) were selected as accuracy evaluation metrics for the rice yield estimation model. Ten-fold cross-validation was used to analyze the accuracy. Ten-fold cross-validation is a widely accepted model evaluation method. It divides the dataset into ten equal parts. Each time, one part is set aside as the validation set, and the remaining part is used as the training set. This process is repeated until every part of the data has been used as the test set once. Finally, the test results of the ten iterations are averaged to obtain an evaluation of model performance. This method fully utilizes the data, reduces the variance of the model evaluation, and improves the stability and reliability of the evaluation results. Ten-fold cross-validation was used in experimental area one, while experimental area two served as independent validation data to measure the model's transferability.

[0049] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for predicting the yield of multiple varieties of rice based on time series remote sensing images, characterized in that: The steps include: S100, acquire and process UAV remote sensing data in the experimental area; S110, using a drone to acquire and process RGB image data of a rice canopy, where the acquisition time of the RGB image data includes multiple growth periods of the rice; S120, using a drone to acquire and process multispectral image data of rice, wherein the acquisition time of the multispectral image data is consistent with the acquisition time of the RGB image data; S200, extracting time series remote sensing image features; S210: for the pre-processed multispectral image, outlining a multispectral region of interest according to the location of the field in the multispectral image, extracting and calculating the mean reflectance of the field through the multispectral region of interest, and using the mean reflectance as the canopy reflectance of the field; S220, calculate the vegetation index that helps monitor vegetation growth and predict yield by using the canopy reflectance of the field, and use it as a feature for estimating rice yield; S230, obtaining the canopy height and canopy volume of the field; S240, performing tasseled cap transformation on the pre-processed multispectral image to obtain tasseled cap parameters; S300: Create a rice cumulative growth characteristic CGC based on the extracted time series remote sensing image features to characterize the growth characteristics of rice. The specific calculation is as follows: Where RSP represents the remote sensing characteristics of canopy reflectance, vegetation index, canopy height, canopy volume or tasseled cap parameters in the 490 nm, 520 nm, 550 nm, 570 nm, 670 nm, 680 nm, 700 nm, 720 nm, 800 nm, 850 nm, 900 nm and 950 nm bands, and t represents the number of days after rice transplanting. S400, establishing a rice cumulative growth characteristic curve based on a calculation formula for the rice cumulative growth characteristic CGC, wherein the number of days after rice transplanting is used as the abscissa and the cumulative value of the extracted time series remote sensing image features is used as the ordinate, and finely classifying rice varieties using the rice cumulative growth characteristic curve; S500, recording the actual yields of multiple rice varieties in the experimental area; S600. A rice yield prediction model is constructed using a multivariate regression model based on the Transformer architecture for rice varieties within a single variety category, where the actual yield of multiple rice varieties in the experimental area is used as the dependent variable of the prediction model, and the features extracted from the corresponding time series remote sensing images are used as the independent variables of the prediction model.

2. The method for predicting multi-variety rice yield based on time series remote sensing images according to claim 1, characterized in that: The experimental area includes experimental area one and experimental area two. Both experimental area one and experimental area two contain several plots. The plots in experimental area one are planted with different hybrid rice varieties, and the plots in experimental area two are planted with rice of different genotypes.

3. The method for predicting multi-variety rice yield based on time series remote sensing images according to claim 1, characterized in that: The processing of RGB image data includes image stitching and three-dimensional reconstruction, generating dense three-dimensional point clouds, digital surface models, and orthophoto stitching images of the experimental area based on the coordinates of the ground control points; the processing of multispectral image data includes geometric processing, radiation processing, and spectral information extraction.

4. The method for predicting multi-variety rice yield based on time series remote sensing images according to claim 1, characterized in that: When acquiring multispectral image data of rice, the drone is equipped with a 12-band multispectral camera, with the central bands being 490nm, 520nm, 550nm, 570nm, 670nm, 680nm, 700nm, 720nm, 800nm, 850nm, 900nm, and 950nm.

5. The method for predicting multi-variety rice yield based on time series remote sensing images according to claim 4, characterized in that: Vegetation indices that are helpful for vegetation growth monitoring and yield prediction include: Normalized Difference Vegetation Index: NDVI = (R850nm - R670nm) / (R850nm + R670nm) Green Normalized Difference Vegetation Index: GNDVI = (R850nm - R550nm) / (R850nm + R550nm) Normalized difference red edge index: NDRE = (R850nm - R720nm) / (R850nm + R720nm) Red edge chlorophyll index: CI reg edge = R850nm / R720nm-1 Dual-band Enhanced Vegetation Index: EVI2 = 2.5*(R850nm-R670nm) / (R850nm+2.4*R670nm+1) Green edge chlorophyll index: CI green = R850nm / R550nm-1 Among them, R550, R670, R720 and R850 are the canopy reflectance of the fields in the 550nm, 670nm, 720nm and 850nm bands respectively.

6. The method for predicting multi-variety rice yield based on time series remote sensing images according to claim 1, characterized in that: The specific steps for obtaining the canopy height and canopy volume of a field in S230 are as follows: before transplanting rice to the field, a digital elevation model of the base ground of the rice field is obtained; after transplanting the rice seedlings, the digital surface model generated by each aerial photography is subtracted from the digital elevation model of the base ground to obtain the rice canopy height model; on the rice canopy height model image, the rice canopy range is selected by outlining the region of interest, and the average canopy height of all pixels within each region of interest is calculated as the canopy height of the field; after obtaining the rice canopy height model, the canopy volume model is obtained in combination with the ground sampling interval information of each pixel; the canopy volume CV of each field can be obtained by outlining the region of interest, and the calculation formula of CV is: Among them H i represents the canopy height of the ith pixel, and GSD is the ground sampling interval.

7. The method for predicting multi-variety rice yield based on time series remote sensing images according to claim 1, characterized in that: In S240 , tasseled cap transformation is performed on the pre-processed multispectral image to obtain tasseled cap parameters, including brightness, greenness, and the third component.

8. The method for predicting multi-variety rice yield based on time series remote sensing images according to claim 1, characterized in that: The classification of rice varieties in S400 is specifically as follows: in the coordinate system of days after transplanting - cumulative growth characteristics, by comparing the angle α formed by the cumulative growth characteristic curves of rice at any two adjacent time points, if the angle α is less than 5°, the rice varieties are classified into the same category, thereby realizing the fine classification of rice varieties based on the cumulative growth characteristic curves of rice.

9. The method for predicting multi-variety rice yield based on time series remote sensing images according to claim 2, characterized in that: It also includes S700: performing model validation on the constructed rice yield prediction model, wherein a ten-fold cross-validation method is used in the experimental area one to perform model accuracy statistics, and the experimental area two is used as independent validation data to measure the migration ability of the model.

Images