Crop yield estimation method and device based on linear time sequence characteristics and multi-scale attention

Abstract

The invention provides a crop yield estimation method and device based on linear time sequence features and multi-scale attention, and the method comprises the steps: constructing a yield estimation model which comprises a linear time feature extraction module and a multi-scale attention feature module, obtaining training data formed by crop yields of a plurality of land parcels in historical years and a remote sensing image sequence corresponding to each land parcel, wherein each land parcel is labeled with a yield label; the linear time feature module regards the crop yields of the plots in the training data in historical years and the corresponding years as a time sequence, and environment mode features of the time sequence are extracted through linear regression; the multi-scale attention feature module extracts remote sensing features of a remote sensing image sequence in the training data by using attention mechanisms of different scales; the environment mode features and the remote sensing features are sent into a decoder of the multi-layer perceptron, and the predicted yield is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural crop yield prediction, and in particular to a crop yield estimation method and device based on linear time series characteristics and multi-scale attention. Background Art

[0002] Existing crop yield prediction technologies mainly include but are not limited to the following four categories:

[0003] (1) Methods based on statistical analysis. This method relies on the collection and analysis of a large amount of local data as the basis for prediction. However, its main drawback is that data collection is costly and time-consuming. In addition, human factors in the data collection and analysis process can easily introduce errors, which can affect the accuracy of the prediction results. For example, data entry errors, sampling bias, and other statistical problems can lead to prediction bias.

[0004] (2) Models based on crop growth principles. This method predicts crop yields by deeply understanding each stage of crop planting to harvesting. Its core is to comprehensively consider multiple factors such as soil, climate, water, and light. However, the crop growth process is complex and is affected by the interaction of multiple factors. Even experienced researchers find it difficult to fully simulate it. These models are usually oversimplified, resulting in low prediction accuracy in real scenarios.

[0005] (3) Crop yield prediction based on traditional machine learning methods. With the development of machine learning techniques, more and more researchers are using these techniques to predict crop yield. Machine learning methods are able to manage and analyze complex data patterns, identify and understand the complex relationships between influencing factors without the need for a prior in-depth understanding of the basic mechanisms of crop growth. Machine learning models have the ability to automatically adjust and optimize parameters, which is in contrast to traditional crop models. Common machine learning techniques include random forest regression, support vector regression, multi-layer regression, Lasso regression, and Boosted regression trees. These methods are able to capture complex nonlinear relationships in crop growth and perform well in yield prediction. However, the performance of these models depends on the quality of feature selection, and the manual design of feature selection may affect the performance of the model.

[0006] (4) Crop yield prediction based on deep learning methods. Deep learning simplifies feature engineering by automatically learning and extracting features. With its multi-layer structure, deep learning can effectively process high-dimensional data, extract features at different levels, and capture complex patterns and relationships.

[0007] In crop yield prediction, deep learning-based models mainly rely on tabular records of crop growing environments and remote sensing time-series image data. Most studies focus on one of these two data types due to factors such as accessibility, cost, and error. Models based on crop growing environment records usually include data such as climate variables, precipitation, soil conditions, seed genotype, and geographic location, which together define the growing environment of the crop. A DNN network is used to extract features from heterogeneous data to predict corn yield. A CNN-RNN framework is designed to capture the dependence of crops on different soil and climate variables through one-dimensional convolution, and then an RNN is used to capture the effect of genotype on yield changes over time. This model is further improved by integrating graph neural networks (GNNs) to enhance the adaptability and robustness of the model to different geographical features. Although deep learning outperforms traditional methods such as LASSO and decision trees in yield prediction, the performance of these models is difficult to directly compare due to the diversity of variables affecting crop growth and the inconsistency of input variables selected by researchers. In addition, the data usually comes from sensors or weather stations, and the low sampling density and sensor errors further limit the performance of the model.

[0008] The development of satellite remote sensing technology enables the continuous extraction of spatiotemporal information of crop growth. Existing technologies note the interaction between electromagnetic waves and crop growth, allowing remote sensing technology to provide sufficient spatial coverage, temporal continuity, and availability to monitor crop growth. Therefore, there are more and more deep learning yield estimation methods based on remote sensing. Long short-term memory networks are used to mine temporal relationships from MODIS histogram time series, and Gaussian processes are used to integrate spatiotemporal features to learn the differences in crop growth across regions, but the kernel function for extracting features relies on experience and has very limited representation capabilities. A two-stream convolutional neural network (CNN) is used to extract spatial features of yield distribution from drone remote sensing images. A CNN-RNN structure is used to analyze spatial-spectral changes and temporal dependencies during crop growth. However, these studies do not consider the impact of different crop stages on yield. The model combines attention mechanisms and temporal models to assign weights to features at different crop growth stages to study their contribution to crop yield. Or use prior knowledge to guide attention training, while combining graph convolutional networks to process geospatial topological structures to learn differences in crop growth across regions. However, RNN may cause attention drift problems when accumulating temporal features, and the attention weights guided by prior knowledge may over-rely on expert experience, potentially conflicting with the high-dimensional features learned by the model. The above models only focus on extracting crop growth features from remote sensing sequences. Due to the resolution limitations of remote sensing technology and the inherent invisibility of surface data, these models cannot fully capture the subtle differences in the crop growth environment and its long-term trends.

[0009] Existing models often encounter difficulties when predicting crop yields over large areas. The reason for this is that due to different geographical locations, the climate, soil and other environmental variables in each region have unique effects on crop growth and yield. Existing models are not good enough to deal with differences in crop growth environments in different regions. This limitation comes from both the input data and the model structure itself. Taking the currently popular research scheme that uses remote sensing images as input as an example, although remote sensing images have the advantage of wide coverage, they cannot provide details of the crop growth environment in each region due to resolution limitations. In addition, remote sensing images cannot capture information below the surface, such as the content of various compounds in the soil, which further limits the model's comprehensive understanding of the crop growth environment.

[0010] The two main challenges we face are:

[0011] (1) Existing models mainly rely on extracting crop growth environment features from remote sensing images. However, this method ignores the fact that remote sensing images themselves cannot provide complete growth environment information. Therefore, the model has limitations in understanding the crop growth environment and lacks recognition of differences in characteristics in different regions, which in turn affects the model's prediction performance.

[0012] (2) Limited by the traditional recurrent neural network structure: When trying to fit the impact of crop growth stages in different regions on final yield, some models may be constrained by the traditional recurrent neural network structure, resulting in the problem of attention weight shift. This structural limitation may prevent the model from deeply understanding and representing the dynamic changes of crop growth, further reducing the accuracy and reliability of the prediction. Summary of the invention

[0013] The purpose of the present invention is to overcome the problem that the above-mentioned prior art is limited to extracting crop growth environment characteristics only from remote sensing images, which causes the model to ignore the environmental differences in different regions, and the previous model's attention weight shift to the crop growth stage causes the model to have low accuracy in yield prediction in a large area, and proposes a new yield estimation model.

[0014] In view of the shortcomings of existing technologies, such as Figure 7 As shown, the present invention proposes a crop yield estimation method based on linear time series features and multi-scale attention, which includes:

[0015] A construction step, constructing a yield estimation model including a linear time feature extraction module and a multi-scale attention feature module, wherein the linear time feature extraction module is used to extract environmental pattern features from the crop yield time series of different regions, and the multi-scale attention feature module is used to analyze the real-time growth of crops from the remote sensing image sequence to generate remote sensing features;

[0016] The training step obtains training data consisting of crop yields of multiple plots in historical years and remote sensing image sequences corresponding to each plot, and each plot has been annotated with a yield label; the linear time feature module regards the crop yields of the plots in the training data in historical years and the corresponding years as a time series, and extracts the environmental pattern features of the time series through linear regression; the multi-scale attention feature module uses attention mechanisms of different scales to extract remote sensing features of the remote sensing image sequence in the training data; the environmental pattern features and the remote sensing features are sent to the decoder of the multi-layer perceptron to obtain the predicted yield, and according to the predicted yield and the yield label, a loss function is constructed to back-propagate and train the yield estimation model;

[0017] In the yield estimation step, the historical crop yields and remote sensing image sequences of the plot to be estimated are input into the trained yield estimation model to obtain the yield prediction results of the plot to be estimated.

[0018] The crop yield estimation method based on linear time series features and multi-scale attention, wherein the linear time feature block includes:

[0019] Determine whether there is missing crop yield in historical years. If so, use the balanced binary tree KD to identify the nearest neighbor plots of each missing data, and use the regression parameters of the nearest neighbor plots as the statistical features of the missing data. Otherwise, directly perform linear regression analysis on the time series to obtain statistical features; use a multilayer perceptron to map the statistical features and their corresponding time variables to a high-dimensional space to obtain the environmental pattern features, where the time variable is the difference between the year of data recording and the base year.

[0020] The crop yield estimation method based on linear time series features and multi-scale attention, wherein the multi-scale attention feature module is composed of a convolution block and an attention layer operating at different time scales, and is used to extract effective features and identify trends from remote sensing data within the crop growing season;

[0021] Each band data of each remote sensing image in the remote sensing image sequence is converted into a one-dimensional feature vector, which is passed to the convolution block to obtain the feature coding sequence at different time points. Position and time information are added to the feature coding sequence to obtain the intermediate sequence; the intermediate sequence is input into the attention layer, and the self-attention mechanism is used to capture the dependency between each time step and the correlation between each time span to obtain the remote sensing feature.

[0022] The crop yield estimation method based on linear time series features and multi-scale attention, wherein the multi-scale attention feature module includes two attention layers;

[0023] The first attention layer uses the self-attention mechanism to capture the dependencies between time steps in the intermediate sequence, and the second attention layer is used to capture the correlation of the preset time span;

[0024] The feature sequences obtained by each attention layer are divided into time periods. The features in each time period are sent to the multi-layer perceptron for encoding, and the encoding results are added and input into the feedforward neural network to obtain the remote sensing features.

[0025] like Figure 8 As shown, the present invention also proposes a crop yield estimation device based on linear time series features and multi-scale attention, which includes:

[0026] A construction module is used to construct a yield estimation model including a linear time feature extraction module and a multi-scale attention feature module. The linear time feature extraction module is used to extract environmental pattern features from the crop yield time series in different regions. The multi-scale attention feature module is used to analyze the real-time growth of crops from the remote sensing image sequence to generate remote sensing features.

[0027] The training module obtains training data consisting of crop yields of multiple plots in historical years and remote sensing image sequences corresponding to each plot, and each plot has been annotated with a yield label; the linear time feature module regards the crop yields of the plots in the training data in historical years and the corresponding years as a time series, and extracts the environmental pattern features of the time series through linear regression; the multi-scale attention feature module uses attention mechanisms of different scales to extract remote sensing features of the remote sensing image sequence in the training data; the environmental pattern features and the remote sensing features are sent to the decoder of the multi-layer perceptron to obtain the predicted yield, and according to the predicted yield and the yield label, a loss function is constructed to back-propagate and train the yield estimation model;

[0028] The yield estimation module inputs the historical crop yields and remote sensing image sequences of the land to be estimated into the trained yield estimation model to obtain the yield prediction results of the land to be estimated.

[0029] The crop yield estimation device based on linear time series features and multi-scale attention, wherein the linear time feature block includes:

[0030] Determine whether there is missing crop yield in historical years. If so, use the balanced binary tree KD to identify the nearest neighbor plots of each missing data, and use the regression parameters of the nearest neighbor plots as the statistical features of the missing data. Otherwise, directly perform linear regression analysis on the time series to obtain statistical features; use a multilayer perceptron to map the statistical features and their corresponding time variables to a high-dimensional space to obtain the environmental pattern features, where the time variable is the difference between the year of data recording and the base year.

[0031] The crop yield estimation device based on linear time series features and multi-scale attention, wherein the multi-scale attention feature module is composed of a convolution block and an attention layer operating at different time scales, and is used to extract effective features and identify trends from remote sensing data within the crop growing season;

[0032] Each band data of each remote sensing image in the remote sensing image sequence is converted into a one-dimensional feature vector, which is passed to the convolution block to obtain feature coding sequences at different time points, and position and time information are added to the feature coding sequence to obtain an intermediate sequence; the intermediate sequence is input into the attention layer, and the self-attention mechanism is used to capture the dependency between each time step and the correlation between each time span to obtain the remote sensing feature;

[0033] The multi-scale attention feature module includes two attention layers;

[0034] The first attention layer uses the self-attention mechanism to capture the dependencies between time steps in the intermediate sequence, and the second attention layer is used to capture the correlation of the preset time span;

[0035] The feature sequences obtained by each attention layer are divided into time periods. The features in each time period are sent to the multi-layer perceptron for encoding, and the encoding results are added and input into the feedforward neural network to obtain the remote sensing features.

[0036] The present invention also proposes an electronic device, which includes the crop yield estimation device based on linear time series features and multi-scale attention. The electronic device may be connected to an information display device, and the information display device is used to display the evaluation results according to display parameters and attributes set by the user or through an artificial intelligence model.

[0037] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the crop yield estimation method based on linear time series features and multi-scale attention.

[0038] The present invention also proposes a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the crop yield estimation method based on linear time series features and multi-scale attention are implemented.

[0039] It can be seen from the above scheme that the advantages of the present invention are:

[0040] (1) Statistical evaluation indicators

[0041] Table 3 shows the results of LM-net on the US soybean yield dataset over five years, while Table 4 shows the results on the corn dataset. The best results are marked in bold and the suboptimal results are underlined. It can be seen that the prediction errors of decision trees and SVR are high, which may be due to their poor performance in handling complex nonlinear relationships and temporal dependencies in crop growth. Although CNN-based methods integrate spectral features, they perform poorly in handling the correlation of growth stages. LSTM models address this problem by capturing long-term dependencies, while Gaussian processes enhance performance through spatiotemporal learning but are limited by the Gaussian distribution assumption. Recent methods such as DACM and KSTAGE use attention mechanisms to emphasize key growth stages and perform well in some years, consistent with the original study. However, attention drift and reliance on human experts lead to unstable performance and significant year-to-year fluctuations.

[0042] The above studies only focus on extracting features from remote sensing sequences, ignoring the incomplete representation of environmental factors provided by remote sensing images. In contrast, LM-net achieves satisfactory results. For example, on the soybean dataset, LM-net reduces MAPE by 11.1% and RMSE by 12.8%, which is a significant improvement over the best existing methods. ME represents the overall bias in the forecast, reflecting the national yield level. Most models tend to underestimate, showing bias in capturing long-term environmental factors. Our model significantly improves in ME, indicating that LM-net captures long-term trends more effectively.

[0043] (2) Module ablation analysis

[0044] LM-net consists of two key components: linear temporal feature block and multi-scale attention block. To verify the effectiveness of these components in mining key features from different perspectives, we conducted the following ablation experiments: We used the CNN-RNN model as the baseline and replaced the MSA block with the standard RNN, called wo / MSA; when the LTF block was removed, called wo / LTF. The complete model is called LM-net. Figure 3 Average test results from 2016 to 2020 are shown, where ME* represents the average of the absolute ME values ​​in these years.

[0045] like Figure 5 and 6As shown, the experimental results show that the LTF and MSA blocks significantly improve the performance of the model and reduce the error compared with the baseline model. The LTF block can effectively capture long-term crop growth characteristics, while the MSA block excels in capturing multi-scale features and spatiotemporal correlations. Comparing LM-net with models lacking separate blocks shows that the features extracted by the two blocks are effectively fused through the multi-layer perceptron decoder, reflecting their complementarity and overall improvement in model performance. In addition, the LTF block, as an independent feature extraction module, can be seamlessly integrated into various models. To verify its effectiveness, we integrate the LTF block into the existing model. Figure 4 The experimental results in show that, averaged over a five-year period, all models after integrating the LTF block showed significant performance improvements in soybean and corn yield prediction. The RMSE decreased by 7.2% to 24.5%, and the MAPE decreased by 6.1% to 22.2%. Notably, the decrease in ME* ranged from 36.1% to 84.7%. These results validate the ability of the LTF block to capture long-term environmental and growth changes in crops, complement the features obtained from remote sensing data, and support our hypothesis that yield series provide key information that remote sensing data fail to adequately capture.

[0046] (3) Sensitivity analysis

[0047] To explore the factors that affect model performance, we conducted a sensitivity analysis experiment. The LTF block includes four key parameters: linear regression results, mean and variance of the yield series, time variable T, and confidence variable key. We analyzed the following cases: (1) complete: including all parameters; (2) wo / ke: removing the confidence variable key; (3) wo / li: removing the linear regression results; (4) wo / ye: removing the time variable T; (5) wo / ms: removing the mean and variance.

[0048] Figure 2 Sensitivity analysis results are presented, which show that the model performs best when all parameters are included, showing the contribution of each parameter to the effectiveness of the model. Specifically, the time variable T has the greatest impact on the model. Removing this variable significantly reduces the model performance because the model cannot distinguish the characteristics of LTF blocks at the same location in different time periods, which may lead to information redundancy. On the other hand, removing the confidence variable key slightly reduces the performance, indicating that in most cases, the model has a low reliance on the confidence variable, probably because the historical data in most regions is sufficient to support model predictions without additional confidence adjustments. Removing the linear regression results or the mean and variance leads to an increase in the mean squared error and mean absolute percentage error, indicating the importance of these factors in the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is the overall architecture diagram of LM-net;

[0050] Figure 2 This is a graph of sensitivity analysis experimental results;

[0051] Figure 3 Model prediction performance plot for the US soybean dataset;

[0052] Figure 4 Model prediction performance plot for the US corn dataset;

[0053] Figure 5 This is the result diagram of module ablation experiment;

[0054] Figure 6 This is a graph showing the performance improvement of the LTF module on the existing model;

[0055] Figure 7 is a flow chart of the method of the present invention;

[0056] Figure 8 It is a module diagram of the device of the present invention;

[0057] Fig. 9 This is a schematic diagram of the structure of a first electronic device of the present invention;

[0058] Fig.10 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;

[0059] Fig.11 It is a schematic diagram of the structure of a second electronic device of the present invention.

[0060] Reference numerals:

[0061] A-First electronic device;

[0062] B-Crop yield estimation device based on linear time series features and multi-scale attention;

[0063] C-data acquisition equipment;

[0064] D-information display device;

[0065] 1000 - second electronic device;

[0066] Ⅰ-computational unit;

[0067] II-ROM;

[0068] III-RAM;

[0069] IV-bus;

[0070] V-interface;

[0071] VI - input unit;

[0072] VII-output unit;

[0073] VIII- Storage medium;

[0074] Ⅸ-Communication unit. DETAILED DESCRIPTION

[0075] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0076] Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0077] The processor described in the present invention is the control center of the electronic device, which can be a processor or a general term for multiple processing elements. For example, it can be one or more central processing units (CPUs), or application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0078] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0079] In a specific implementation, as an embodiment, the processor may include one or more CPUs. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). Electronic devices may include: servers, desktop computers, laptops, smart phones, tablet computers, embedded computers, etc., wherein the embedded computers include vehicles and robots, etc.

[0080] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0081] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0082] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0083] It should also be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0084] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0085] It should also be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0086] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0087] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0088] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0089] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0090] To address the above challenges, this paper proposes a new yield estimation model LM-net that integrates linear features and multi-scale spatial self-attention. This model consists of two core modules: a linear time feature extraction module (LTF block) and a multi-scale attention feature module (MSA block). The purpose of the linear time feature extraction module is to extract features from the crop yield time series of different regions to capture the differences between crop growth environments. It regards the historical yield data of each region as a time series and learns the differences in the growth environment of each region by mining the features of the yield series. The linear time feature extraction module is independent of existing prediction models and can be easily integrated into these models to improve the accuracy of prediction. The multi-scale attention feature module is used to improve the crop yield estimation model based on remote sensing images. It is designed to solve the problem of attention shift that may occur during the crop growth feature extraction process and enhance the model's sensitivity to features at different growth stages. By optimizing the feature extraction process, this module can better capture the characteristics of crops at different growth stages, thereby improving the overall prediction performance of the model. Our model was evaluated on the US county-level corn and soybean datasets and showed superior performance to other competing technologies. In addition, integrating the LTF module into existing models can significantly improve the accuracy of yield estimation. Therefore, the present invention includes the following key technical points:

[0091] Key point 1, linear time feature extraction: treat the historical yield data of each region as a time series, and learn the differences in the growing environment of each region by mining the characteristics of the yield series. This module is independent of existing forecasting models and can be easily integrated into these models to improve the accuracy of forecasts.

[0092] Key point 2: Multi-scale attention feature extraction: Use attention mechanisms at different scales to extract the dependencies of different growth stages of crops and enhance the model’s sensitivity to features at different growth stages. By optimizing the feature extraction process, this module can better capture the features of crops at different growth stages, thereby improving the overall prediction performance of the model.

[0093] In order to make the above features and effects of the present invention more clearly and understandably described, embodiments are given below and described in detail with reference to the accompanying drawings. This specification discloses one or more embodiments that include the features of the present invention. The disclosed embodiments are only for illustration. The scope of protection of the present invention is not limited to the disclosed embodiments, and the present invention is defined by the attached claims.

[0094] The overall framework of LM-net is as follows Figure 1 As shown. It can be observed that LM-net contains two important components: the linear time feature block (LTF block) and the multi-scale attention block (MSA block). These two modules extract features from different perspectives. The MSA block captures the real-time dynamic characteristics of crop growth and the changes in each growth stage from the remote sensing image sequence, while the LTF block extracts long-term environmental patterns and trends from the historical yield sequence, which significantly reflects the differences between regions. In this section, we will discuss each component of LM-net in detail.

[0095] The linear time feature module (LTF) treats the historical crop yield of each plot and the corresponding year as a time series during the training process. First, a linear regression model is performed on the two variables to extract information such as slope (β1), intercept (β0), correlation coefficient (r), p value (p) and slope standard error (SE β1 ) and other parameters. These parameters are related to the mean (μ) and variance (σ) of the yield series. 2 ) to establish the statistical characteristics of the region, reflecting the changing characteristics of crop yields in different regions over the years. For most regions, a simple linear regression method can obtain the corresponding statistical characteristics. However, historical yield data in some regions are seriously missing. When the sample size is too small, the accuracy of linear regression is significantly reduced, and some regions even completely lack historical yield labels, making it impossible to obtain corresponding statistical data through linear regression. For areas that cannot be directly modeled, we use feature interpolation algorithms to assign statistical characteristics to these areas. We encode the obtained statistical features into high-dimensional features through a neural network (MLP), and combine them with the remote sensing features extracted by the multi-scale attention module to predict yield. This module performs linear regression, and the input data is the yield of each region in different years and the corresponding year. The training from low-dimensional statistical features to high-dimensional features uses supervised training together with the second module to directly complete the training process of the entire network.

[0096] For the multi-scale attention module (MSA), the input data is the remote sensing time series images of the crop growth cycle of each region. We first use the histogram algorithm to reduce the dimension of the remote sensing data, and then obtain a feature sequence, which represents the remote sensing feature encoding at different time points. Then we use the attention mechanism of different scales to capture the dependencies between each time step. The training of this module is synchronized with the high-dimensional encoder in the LTF block, and the loss function is the MSE loss.

[0097] Linear time feature block:

[0098] The linear-time feature block consists of three basic steps: linear regression, feature interpolation, and high-dimensional mapping. Figure 1 The data processing flow of this module is shown. Its input is the yield sequence {y1,y2,y3,…,y n First, a linear regression analysis is performed on the series to extract data such as the slope (β1), intercept (β0), correlation coefficient (r), p-value (p), and standard error of the slope (SE β1 ) and other parameters. These parameters are related to the mean (μ) and variance (σ) of the yield series. 2 ) to establish the statistical characteristics of the region. For most regions, statistical values ​​can be obtained through the above method. However, historical production data in some regions are seriously missing. When the sample size is too small, the accuracy of linear regression is significantly reduced, and some regions even completely lack historical production labels, making it impossible to obtain corresponding statistical data through linear regression. Therefore, for areas that cannot be directly modeled, feature interpolation becomes necessary.

[0099] In the feature interpolation step, for areas that lack sufficient historical yield data, we assume that geographically close areas have similar crop growing conditions. Based on this, we use the KD tree to identify the nearest neighbor area for each area that cannot be directly modeled, that is, by maintaining a KD tree, find the nearest plots of the plots with missing data, and use the regression parameters of these areas as statistical features of the unmapped areas. However, the reliability of these interpolated statistical features may be reduced. Therefore, we introduced a confidence measurement key to record the reliability of the statistical features. The confidence is initially 1. If the plot is missing data, the confidence is assigned according to the KD tree search path distance. For example, if the search path is 3, the value of the confidence key is 1 / 3. After that, we obtain a feature vector for each area, which can be expressed as:

[0100] F-=[β1,β0,r,p,SE,μ,σ 2 , key]

[0101] When training a model to predict crop yields for the next year, we use historical data up to the current year and need to distinguish data from different years for the same region when inputting the model. We introduce a time variable T to distinguish input features for the same region in different years. The time variable T is defined as follows:

[0102] T=g year -g base

[0103] where g year represents the year of data recording, g base Represents the base year. The base year is a data we need to use when we introduce the variable T when designing the LTF module. When we complete the linear coding, the statistical characteristics of the same plot in different years will become consistent. However, the yield variation of the same plot in different years is very large. If the characteristics of the LTF module of data in different years are all the same, it will cause great interference to the model. Therefore, we introduce the time code T to distinguish the statistical characteristics of different years in the same region. For example, the data starts from 2003 to 2020. When performing time coding, it is inappropriate to directly use the year as T. Therefore, the base year of 2002 is introduced, and the time code becomes 1, 2, 3,..., 18, which can amplify the slope of the linear regression, better improve the feature extraction effect, and thus improve the accuracy of predicted yield.

[0104] At this point, we have obtained all the necessary variables. Finally, we use a multi-layer perceptron to map these statistics to a high-dimensional space, capturing nonlinear relationships and high-order interactions between features. The encoded high-dimensional features are represented as:

[0105]

[0106] Multi-scale attention block:

[0107] It consists of a convolutional block and two attention layers operating at different time scales, and is designed to extract effective features and identify trends from remote sensing data within the crop growing season. In the data processing pipeline, the input data is first preprocessed to convert the data of each band into a one-dimensional feature vector. This feature vector is passed to the convolutional block, which consists of multiple convolution operations, each of which is followed by batch normalization and ReLU activation. Through these operations, we can extract a feature sequence that represents the feature encoding at different time points. In order to better capture the temporal dependency of different stages in the crop growth cycle, we add positional encoding to the generated feature sequence. This step enables the model to distinguish features at different time points by introducing temporal information into the feature sequence.

[0108] Next, the feature sequence is fed into the first layer of attention. In this layer, the self-attention mechanism is used to capture the dependencies between time steps, which means that the model can identify the correlation between a certain point in time and other points in time during the crop growth process. In the second layer of attention, the model focuses on capturing correlations over longer time spans. We divide the feature sequence into time periods, and the features in each time period are fed into a multi-layer perceptron for encoding. These encoded features represent trends or feature changes over a longer period of time and are used for further attention calculations. In this way, the model is able to identify long-term correlations at different stages in the crop growth process.

[0109] In the final stage of the model, we introduced residual connections to ensure the stability and effectiveness of the model. In order to make the outputs from the two layers of attention mechanisms have consistent dimensions, the output of the first layer is processed by a multilayer perceptron. The processed result is added to the output of the second layer and then input into a feedforward neural network for further processing. Finally, all the generated features are combined with the features from other modules and input into the multilayer perceptron decoder to predict the final yield of the crop.

[0110] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0111] like Figure 8 As shown, the present invention also proposes a crop yield estimation device based on linear time series features and multi-scale attention, which includes:

[0112] A construction module is used to construct a yield estimation model including a linear time feature extraction module and a multi-scale attention feature module. The linear time feature extraction module is used to extract environmental pattern features from the crop yield time series in different regions. The multi-scale attention feature module is used to analyze the real-time growth of crops from the remote sensing image sequence to generate remote sensing features.

[0113] The training module obtains training data consisting of crop yields of multiple plots in historical years and remote sensing image sequences corresponding to each plot, and each plot has been annotated with a yield label; the linear time feature module regards the crop yields of the plots in the training data in historical years and the corresponding years as a time series, and extracts the environmental pattern features of the time series through linear regression; the multi-scale attention feature module uses attention mechanisms of different scales to extract remote sensing features of the remote sensing image sequence in the training data; the environmental pattern features and the remote sensing features are sent to the decoder of the multi-layer perceptron to obtain the predicted yield, and according to the predicted yield and the yield label, a loss function is constructed to back-propagate and train the yield estimation model;

[0114] The yield estimation module inputs the historical crop yields and remote sensing image sequences of the land to be estimated into the trained yield estimation model to obtain the yield prediction results of the land to be estimated.

[0115] The crop yield estimation device based on linear time series features and multi-scale attention, wherein the linear time feature block includes:

[0116] Determine whether there is missing crop yield in historical years. If so, use the balanced binary tree KD to identify the nearest neighbor plots of each missing data, and use the regression parameters of the nearest neighbor plots as the statistical features of the missing data. Otherwise, directly perform linear regression analysis on the time series to obtain statistical features; use a multilayer perceptron to map the statistical features and their corresponding time variables to a high-dimensional space to obtain the environmental pattern features, where the time variable is the difference between the year of data recording and the base year.

[0117] The crop yield estimation device based on linear time series features and multi-scale attention, wherein the multi-scale attention feature module is composed of a convolution block and an attention layer operating at different time scales, and is used to extract effective features and identify trends from remote sensing data within the crop growing season;

[0118] Each band data of each remote sensing image in the remote sensing image sequence is converted into a one-dimensional feature vector, which is passed to the convolution block to obtain feature coding sequences at different time points, and position and time information are added to the feature coding sequence to obtain an intermediate sequence; the intermediate sequence is input into the attention layer, and the self-attention mechanism is used to capture the dependency between each time step and the correlation between each time span to obtain the remote sensing feature;

[0119] The multi-scale attention feature module includes two attention layers;

[0120] The first attention layer uses the self-attention mechanism to capture the dependencies between time steps in the intermediate sequence, and the second attention layer is used to capture the correlation of the preset time span;

[0121] The feature sequences obtained by each attention layer are divided into time periods. The features in each time period are sent to the multi-layer perceptron for encoding, and the encoding results are added and input into the feedforward neural network to obtain the remote sensing features.

[0122] like Fig. 9 As shown, the present invention further proposes a first electronic device A in another embodiment, which includes the crop yield estimation device based on linear temporal features and multi-scale attention.

[0123] like Fig.10 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D through a wired or wireless information transmission scheme. The data acquisition device C is used to collect and obtain a remote sensing image sequence to be identified and classified, and the information display device D is used to display the yield prediction results obtained by the analysis of the present invention.

[0124] The information display device D can sort and process the data output by the first electronic device A based on the information display mechanism to improve the readability of the data output by the first electronic device A. The information display mechanism can be manually preset, for example, the data output by the first electronic device A is visually displayed, which can be based on the display parameters and / or attributes set by the user. The display parameters can be, for example, the display data range, and the display attributes can be, for example, the display font, color, whether to scroll and play, etc. The user is presented with the key information specified by the user, such as the output distribution and trend information in the output forecast results, so that the user can understand this information more timely without having to access the secondary page or scroll the page, saving the user's operation. Or the information display mechanism can be an artificial intelligence AI display model, which can learn the user's key information based on the user's previous usage habits, such as viewing time, number of clicks, number of edits, etc., and then automatically present rich and necessary key information to the user.

[0125] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a readable storage medium. When the computer program is executed by a processor, the computer can execute the crop yield estimation method based on linear time series features and multi-scale attention provided by the above methods.

[0126] The present invention also proposes a storage medium VIII in another embodiment for storing a computer program for executing the crop yield estimation method based on linear temporal features and multi-scale attention. It should be understood that the storage medium in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0127] Fig.11 A schematic block diagram of a second electronic device 1000 that can be used to implement an embodiment of the present invention is shown. The second electronic device 1000 electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described and / or required herein. The second electronic device 1000 may be the same or different from the first electronic device A.

[0128] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from a storage medium VIII into a random access memory (RAM) III. In RAM III, various programs and data required for the operation of the device 1000 can also be stored. The computing unit I, ROM II, and RAM III are connected to each other via a bus IV. An input / output (I / O) interface V is also connected to the bus IV.

[0129] A plurality of components in the second electronic device 1000 are connected to the I / O interface V, including: an input unit VI, such as a keyboard, a mouse, etc.; an output unit VII, such as various types of displays, speakers, etc.; a storage medium VIII, such as a disk, an optical disk, etc.; and a communication unit IX, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit IX allows the second electronic device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0130] The computing unit I may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit I performs the various methods and processes described above, such as method steps S1-S3. For example, in some embodiments, the method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage medium VIII. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via ROM II and / or a communication unit IX. When the computer program is loaded into RAM III and executed by the computing unit I, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit I may be configured to execute the method in any other appropriate manner (e.g., by means of firmware).

[0131] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A crop yield estimation method based on linear time series features and multi-scale attention, characterized in that: include: A construction step, constructing a yield estimation model including a linear time feature extraction module and a multi-scale attention feature module, wherein the linear time feature extraction module is used to extract environmental pattern features from the crop yield time series of different regions, and the multi-scale attention feature module is used to analyze the real-time growth of crops from the remote sensing image sequence to generate remote sensing features; The training step is to obtain training data consisting of crop yields of multiple plots in historical years and remote sensing image sequences corresponding to each plot, and each plot has been annotated with a yield label; The linear time feature module regards the crop yields of the plots in the training data in the historical years and the corresponding years as a time series, and extracts the environmental pattern features of the time series through linear regression; the multi-scale attention feature module uses attention mechanisms of different scales to extract the remote sensing features of the remote sensing image sequence in the training data; the environmental pattern features and the remote sensing features are sent to the decoder of the multi-layer perceptron to obtain the predicted yield, and according to the predicted yield and the yield label, the loss function is constructed to back-propagate and train the yield estimation model; In the yield estimation step, the historical crop yields and remote sensing image sequences of the plot to be estimated are input into the trained yield estimation model to obtain the yield prediction results of the plot to be estimated.

2. The crop yield estimation method based on linear time series features and multi-scale attention according to claim 1, characterized in that: The linear time feature block includes: Determine whether there is missing crop yield in historical years. If so, use the balanced binary tree KD to identify the nearest neighbor plots of each missing data, and use the regression parameters of the nearest neighbor plots as the statistical features of the missing data. Otherwise, directly perform linear regression analysis on the time series to obtain statistical features; use a multilayer perceptron to map the statistical features and their corresponding time variables to a high-dimensional space to obtain the environmental pattern features, where the time variable is the difference between the year of data recording and the base year.

3. The crop yield estimation method based on linear time series features and multi-scale attention according to claim 1, characterized in that: The multi-scale attention feature module consists of convolutional blocks and attention layers operating at different time scales to extract effective features and identify trends from remote sensing data within the crop growing season; Each band data of each remote sensing image in the remote sensing image sequence is converted into a one-dimensional feature vector, which is passed to the convolution block to obtain the feature coding sequence at different time points. Position and time information are added to the feature coding sequence to obtain the intermediate sequence; the intermediate sequence is input into the attention layer, and the self-attention mechanism is used to capture the dependency between each time step and the correlation between each time span to obtain the remote sensing feature.

4. The crop yield estimation method based on linear time series features and multi-scale attention according to claim 3, characterized in that: The multi-scale attention feature module includes two attention layers; The first attention layer uses the self-attention mechanism to capture the dependencies between time steps in the intermediate sequence, and the second attention layer is used to capture the correlation of the preset time span; The feature sequences obtained by each attention layer are divided into time periods. The features in each time period are sent to the multi-layer perceptron for encoding, and the encoding results are added and input into the feedforward neural network to obtain the remote sensing features.

5. A crop yield estimation device based on linear time series features and multi-scale attention, characterized in that: include: A construction module is used to construct a yield estimation model including a linear time feature extraction module and a multi-scale attention feature module. The linear time feature extraction module is used to extract environmental pattern features from the crop yield time series in different regions. The multi-scale attention feature module is used to analyze the real-time growth of crops from the remote sensing image sequence to generate remote sensing features. The training module obtains training data consisting of crop yields of multiple plots in historical years and remote sensing image sequences corresponding to each plot, and each plot has been annotated with a yield label; The linear time feature module regards the crop yields of the plots in the training data in the historical years and the corresponding years as a time series, and extracts the environmental pattern features of the time series through linear regression; the multi-scale attention feature module uses attention mechanisms of different scales to extract the remote sensing features of the remote sensing image sequence in the training data; the environmental pattern features and the remote sensing features are sent to the decoder of the multi-layer perceptron to obtain the predicted yield, and according to the predicted yield and the yield label, the loss function is constructed to back-propagate and train the yield estimation model; The yield estimation module inputs the historical crop yields and remote sensing image sequences of the land to be estimated into the trained yield estimation model to obtain the yield prediction results of the land to be estimated.

6. The crop yield estimation device based on linear time series features and multi-scale attention according to claim 5, characterized in that: The linear time feature block includes: Determine whether there is missing crop yield in historical years. If so, use the balanced binary tree KD to identify the nearest neighbor plots of each missing data, and use the regression parameters of the nearest neighbor plots as the statistical features of the missing data. Otherwise, directly perform linear regression analysis on the time series to obtain statistical features; use a multilayer perceptron to map the statistical features and their corresponding time variables to a high-dimensional space to obtain the environmental pattern features, where the time variable is the difference between the year of data recording and the base year.

7. The crop yield estimation device based on linear time series features and multi-scale attention according to claim 5, characterized in that: The multi-scale attention feature module consists of convolutional blocks and attention layers operating at different time scales to extract effective features and identify trends from remote sensing data within the crop growing season; Each band data of each remote sensing image in the remote sensing image sequence is converted into a one-dimensional feature vector, which is passed to the convolution block to obtain feature coding sequences at different time points, and position and time information are added to the feature coding sequence to obtain an intermediate sequence; the intermediate sequence is input into the attention layer, and the self-attention mechanism is used to capture the dependency between each time step and the correlation between each time span to obtain the remote sensing feature; The multi-scale attention feature module includes two attention layers; The first attention layer uses the self-attention mechanism to capture the dependencies between time steps in the intermediate sequence, and the second attention layer is used to capture the correlation of the preset time span; The feature sequences obtained by each attention layer are divided into time periods. The features in each time period are sent to the multi-layer perceptron for encoding, and the encoding results are added and input into the feedforward neural network to obtain the remote sensing features.

8. An electronic device, characterized in that: It includes a crop yield estimation device based on linear time series features and multi-scale attention as described in claims 5-7, and the electronic device is connected to an information display device, which is used to display the evaluation result with display parameters and attributes set by the user or through an artificial intelligence model.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the crop yield estimation method based on linear time series features and multi-scale attention as described in any one of claims 1-4.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the crop yield estimation method based on linear temporal features and multi-scale attention described in any one of claims 1-4 are implemented.

Images