Crop yield estimation method and device based on linear time sequence features and multi-scale attention

Abstract

The present application provides a crop yield estimation method and device based on linear time sequence characteristics and multi-scale attention, comprising: constructing a yield estimation model comprising a linear time feature extraction module and a multi-scale attention feature module, obtaining training data composed of historical year crop yields of multiple plots and corresponding remote sensing image sequences of each plot, and each plot has a labeled yield label; the linear time feature extraction module regards the historical year crop yields of the plots in the training data and the corresponding years as a time sequence, and extracts the environmental pattern features of the time sequence through linear regression; the multi-scale attention feature module extracts remote sensing features of the remote sensing image sequences in the training data using different scale attention mechanisms; the environmental pattern features and the remote sensing features are input into the decoder of the multilayer perceptron to obtain the predicted yield.

Description

Technical Field

[0001] This invention relates to the field of agricultural crop yield prediction technology, and in particular to a crop yield estimation method and apparatus based on linear time series characteristics and multi-scale attention. Background Technology

[0002] Existing crop yield prediction technologies mainly include, but are not limited to, the following four categories:

[0003] (1) Statistical Analysis-Based Methods. This method relies on the collection and analysis of large amounts of local data as the basis for prediction. However, its main drawback is the high cost and time-consuming nature of data collection. Furthermore, human factors during data collection and analysis can easily introduce errors, affecting the accuracy of the prediction results. For example, data entry errors, sampling bias, and other statistical problems can all lead to prediction bias.

[0004] (2) Models based on crop growth principles. This method predicts crop yield by deeply understanding each stage of crop growth from planting to harvest. Its core lies in comprehensively considering multiple factors such as soil, climate, water, and sunlight. However, crop growth is complex and influenced by the interaction of multiple factors, making it difficult even for experienced researchers to fully simulate. These models are often oversimplified, resulting in low prediction accuracy in real-world scenarios.

[0005] (3) Crop yield prediction based on traditional machine learning methods. With the development of machine learning technology, more and more researchers are using these techniques to predict crop yield. Machine learning methods can manage and analyze complex data patterns, identify and understand complex relationships between influencing factors, without requiring prior in-depth understanding of the basic mechanisms of crop growth. Machine learning models have the ability to automatically adjust and optimize parameters, which contrasts with traditional crop models. Common machine learning techniques include random forest regression, support vector regression, multilevel regression, Lasso regression, and Boosted regression trees. These methods can capture complex nonlinear relationships in crop growth and perform well in yield prediction. However, the performance of these models depends on the quality of feature selection, and the manual design of feature selection may affect the model's performance.

[0006] (4) Crop yield prediction based on deep learning methods. Deep learning simplifies feature engineering by automatically learning and extracting features. With its multi-layered structure, deep learning can effectively process high-dimensional data, extract features at different levels, and capture complex patterns and relationships.

[0007] In crop yield prediction, deep learning-based models primarily rely on tabular records of crop growing environments and remote sensing time-series image data. Due to factors such as availability, cost, and error, most research focuses on one of these two data types. Models based on crop growing environment records typically include data on climate variables, precipitation, soil conditions, seed genotypes, and geographic location, which collectively define the crop's growing environment. This study utilizes DNN networks to extract features from heterogeneous data to predict maize yield. A CNN-RNN framework is designed to capture the crop's dependence on different soil and climate variables through one-dimensional convolution, followed by using RNNs to capture the impact of genotype on yield changes over time. This model is further refined by integrating graph neural networks (GNNs) to enhance its adaptability and robustness to different geographic features. Although deep learning outperforms traditional methods such as LASSO and decision trees in yield prediction, the performance of these models is difficult to directly compare due to the diversity of variables affecting crop growth and the inconsistency of input variables chosen by researchers. Furthermore, data often comes from sensors or weather stations, and low sampling density and sensor errors further limit model performance.

[0008] The development of satellite remote sensing technology has enabled the continuous extraction of spatiotemporal information on crop growth. Existing technologies have recognized the interaction between electromagnetic waves and crop growth, allowing remote sensing to provide sufficient spatial coverage, temporal continuity, and availability for monitoring crop growth. Therefore, deep learning-based yield estimation methods based on remote sensing are increasingly prevalent. One approach utilizes Long Short-Term Memory (LSTM) networks to mine temporal relationships from MODIS histogram time series and integrates spatiotemporal features using Gaussian processes to learn differences in crop growth across regions; however, the kernel function for feature extraction is empirically dependent and has very limited representational capabilities. Another approach extracts spatial features of yield distribution from UAV remote sensing images using a dual-stream convolutional neural network (CNN). A CNN-RNN structure is used to analyze spatial-spectral variations and temporal dependencies during crop growth. However, these studies do not consider the impact of different crop stages on yield. A model combining attention mechanisms and temporal models assigns weights to features at different crop growth stages to study their contribution to crop yield. Alternatively, prior knowledge can be used to guide attention training, while graph convolutional networks are combined to process geospatial topology and learn differences in crop growth across regions. However, RNNs may suffer from attention bias when accumulating temporal features, and attention weights guided by prior knowledge may become overly reliant on expert experience, potentially conflicting with the high-dimensional features learned by the model. The aforementioned models focus solely on extracting crop growth features from remote sensing sequences. Due to the resolution limitations of remote sensing technology and the inherent invisibility of surface data, these models cannot fully capture the subtle differences in the crop growth environment and its long-term trends.

[0009] Existing models often encounter difficulties when predicting crop yields over large areas. This is because, due to geographical differences, climate, soil, and other environmental variables in each region have unique impacts on crop growth and yield. Existing models are inadequate in addressing the differences in crop growth environments across regions; this limitation stems from both the input data and the model structure itself. Taking the currently popular research approach using remote sensing images as input as an example, although remote sensing images have the advantage of wide coverage, their resolution limitations prevent them from providing details of the crop growth environment in each region. Furthermore, remote sensing images cannot capture information below the surface, such as the content of various compounds in the soil, which further limits the model's comprehensive understanding of the crop growth environment.

[0010] The two main challenges we currently face are:

[0011] (1) Existing models mainly rely on extracting features of the crop growth environment from remote sensing images. However, this method ignores the fact that remote sensing images themselves cannot provide complete information about the growth environment. Therefore, the model has limitations in understanding the crop growth environment and is insufficient in recognizing the differences in features between different regions, which in turn affects the model's predictive performance.

[0012] (2) Limitations of traditional recurrent neural network structures: When attempting to fit the impact of crop growth stages on final yield in different regions, some models may be constrained by traditional recurrent neural network structures, leading to attention weight shifts. This structural limitation may hinder the model from deeply understanding and representing the dynamic changes in crop growth, further reducing the accuracy and reliability of predictions. Summary of the Invention

[0013] The purpose of this invention is to overcome the problems of existing technologies that are limited to extracting crop growth environment features only from remote sensing images, which leads to the model ignoring the environmental differences in different regions and the low accuracy of yield prediction in large areas due to the shift in attention weights of previous models to crop growth stages. A new yield estimation model is proposed.

[0014] To address the shortcomings of existing technologies, such as Figure 7 As shown, this invention proposes a crop yield estimation method based on linear time-series features and multi-scale attention, including:

[0015] The construction steps involve building a yield estimation model that includes a linear time feature extraction module and a multi-scale attention feature module. The linear time feature extraction module is used to extract environmental pattern features from the time series of crop yields in different regions, and the multi-scale attention feature module is used to analyze the real-time growth of crops from remote sensing image sequences to generate remote sensing features.

[0016] The training process involves acquiring training data consisting of historical crop yields from multiple plots and corresponding remote sensing image sequences for each plot, with each plot labeled with a yield tag. The linear time feature module treats the historical crop yields and corresponding years of the plots in the training data as a time series and extracts environmental pattern features from this time series through linear regression. The multi-scale attention feature module uses attention mechanisms at different scales to extract remote sensing features from the remote sensing image sequences in the training data. The environmental pattern features and the remote sensing features are fed into the decoder of a multilayer perceptron to obtain the predicted yield. Based on the predicted yield and the yield tag, a loss function is constructed and backpropagated to train the yield estimation model.

[0017] The yield estimation step involves inputting the historical crop yields and remote sensing image sequences of the plot to be estimated into the trained yield estimation model to obtain the yield prediction results for the plot.

[0018] The crop yield estimation method based on linear time-series features and multi-scale attention, wherein the linear time-series feature block includes:

[0019] To determine if there are missing data points in the historical crop yield data, if so, a balanced binary tree KD model is used to identify the nearest neighbor plot for each missing data point, and the regression parameters of the nearest neighbor plot are used as the statistical features of the missing data. Otherwise, linear regression analysis is directly performed on the time series to obtain the statistical features. A multilayer perceptron is used to map the statistical features and their corresponding time variables to a high-dimensional space to obtain the environmental pattern features, where the time variable is the difference between the year the data was recorded and the base year.

[0020] The crop yield estimation method based on linear time-series features and multi-scale attention, wherein the multi-scale attention feature module consists of convolutional blocks and attention layers operating at different time scales, used to extract effective features and identify trends from remote sensing data during the crop growing season;

[0021] Each band data of each remote sensing image in the remote sensing image sequence is converted into a one-dimensional feature vector. This feature vector is passed to the convolutional block to obtain the feature encoding sequence at different time points. Position and time information are added to the feature encoding sequence to obtain the intermediate sequence. The intermediate sequence is input into the attention layer, and the self-attention mechanism is used to capture the dependencies between time steps and the correlation between time spans to obtain the remote sensing feature.

[0022] The crop yield estimation method based on linear time-series features and multi-scale attention, wherein the multi-scale attention feature module includes two attention layers;

[0023] The first attention layer uses a self-attention mechanism to capture the dependencies between time steps in the intermediate sequence, and the second attention layer is used to capture the correlations over a preset time span.

[0024] The feature sequences obtained from each attention layer are divided into time periods. The features in each time period are fed into a multilayer perceptron for encoding, and the encoding results are added together and input into a feedforward neural network to obtain the remote sensing feature.

[0025] like Figure 8 As shown, this invention also proposes a crop yield estimation device based on linear time-series features and multi-scale attention, including:

[0026] The module constructs a yield estimation model that includes a linear time feature extraction module and a multi-scale attention feature module. The linear time feature extraction module is used to extract environmental pattern features from the time series of crop yields in different regions, and the multi-scale attention feature module is used to analyze the real-time growth of crops from remote sensing image sequences to generate remote sensing features.

[0027] The training module acquires training data consisting of historical crop yields of multiple plots and corresponding remote sensing image sequences for each plot, with each plot labeled with a yield tag. The linear time feature module treats the historical crop yields of the plots in the training data and the corresponding years as a time series, and extracts environmental pattern features of the time series through linear regression. The multi-scale attention feature module uses attention mechanisms at different scales to extract remote sensing features from the remote sensing image sequences in the training data. The environmental pattern features and the remote sensing features are fed into the decoder of the multilayer perceptron to obtain the predicted yield. Based on the predicted yield and the yield tag, a loss function is constructed and backpropagated to train the yield estimation model.

[0028] The yield estimation module takes the historical crop yield and remote sensing image sequence of the plot to be estimated as input into the trained yield estimation model to obtain the yield prediction results of the plot to be estimated.

[0029] The crop yield estimation device based on linear time-series features and multi-scale attention, wherein the linear time-series feature block includes:

[0030] To determine if there are missing data points in the historical crop yield data, if so, a balanced binary tree KD model is used to identify the nearest neighbor plot for each missing data point, and the regression parameters of the nearest neighbor plot are used as the statistical features of the missing data. Otherwise, linear regression analysis is directly performed on the time series to obtain the statistical features. A multilayer perceptron is used to map the statistical features and their corresponding time variables to a high-dimensional space to obtain the environmental pattern features, where the time variable is the difference between the year the data was recorded and the base year.

[0031] The crop yield estimation device based on linear time-series features and multi-scale attention, wherein the multi-scale attention feature module consists of convolutional blocks and attention layers operating at different time scales, used to extract effective features and identify trends from remote sensing data during the crop growing season.

[0032] Each band data of each remote sensing image in the remote sensing image sequence is converted into a one-dimensional feature vector. This feature vector is passed to the convolutional block to obtain the feature encoding sequence at different time points. Position and time information are added to the feature encoding sequence to obtain the intermediate sequence. The intermediate sequence is input into the attention layer, and the self-attention mechanism is used to capture the dependencies between time steps and the correlation between time spans to obtain the remote sensing feature.

[0033] This multi-scale attention feature module includes two attention layers;

[0034] The first attention layer uses a self-attention mechanism to capture the dependencies between time steps in the intermediate sequence, and the second attention layer is used to capture the correlations over a preset time span.

[0035] The feature sequences obtained from each attention layer are divided into time periods. The features in each time period are fed into a multilayer perceptron for encoding, and the encoding results are added together and input into a feedforward neural network to obtain the remote sensing feature.

[0036] The present invention also proposes an electronic device, including the aforementioned crop yield estimation device based on linear time-series features and multi-scale attention. The electronic device may be connected to an information display device, which is used to display the evaluation results using user-set display parameters, attributes, or through an artificial intelligence model.

[0037] The present invention also proposes a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the crop yield estimation method based on linear time-series features and multi-scale attention.

[0038] The present invention also proposes a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, it implements the steps of the crop yield estimation method based on linear time-series features and multi-scale attention.

[0039] As can be seen from the above solutions, the advantages of the present invention are:

[0040] (1) Statistical evaluation indicators

[0041] Table 3 presents the five-year test results of LM-net on the US soybean yield dataset, while Table 4 shows the results on the maize dataset. The best results are highlighted in bold, and the second-best results are underlined. It can be seen that decision trees and SVRs have high prediction errors, possibly due to their poor performance in handling complex nonlinear relationships and temporal dependencies in crop growth. While CNN-based methods integrate spectral features, they perform poorly in handling the correlations between growth stages. LSTM models address this issue by capturing long-term dependencies, while Gaussian processes enhance performance through spatiotemporal learning but are limited by the Gaussian distribution assumption. Recent methods such as DAMDM and KSTAGE utilize attention mechanisms to emphasize key growth stages, performing well in some years, consistent with the original research. However, attention drift and reliance on human experts lead to unstable performance and significant interannual fluctuations.

[0042] The aforementioned studies focused solely on extracting features from remote sensing sequences, neglecting the incomplete representation of environmental factors provided by remote sensing images. In contrast, LM-net achieved satisfactory results. For example, on the soybean dataset, LM-net reduced MAPE by 11.1% and RMSE by 12.8%, representing a significant improvement over state-of-the-art methods. ME represents the overall bias in predictions, reflecting national yield levels. Most models tend to underestimate this, showing a bias in capturing long-term environmental factors. Our model significantly improves on ME, indicating that LM-net more effectively captures long-term trends.

[0043] (2) Module Ablation Analysis

[0044] LM-net consists of two key components: linear-time feature blocks and multi-scale attention blocks. To verify the effectiveness of these components in mining key features from different perspectives, we conducted the following ablation experiments: using a CNN-RNN model as a baseline, we replaced the MSA block with a standard RNN, termed wo / MSA; when the LTF block was removed, it was termed wo / LTF. The complete model is called LM-net. Figure 3 The average test results from 2016 to 2020 are shown, where ME* represents the average absolute ME value for these years.

[0045] like Figure 5 and 6As shown, experimental results demonstrate that, compared to the baseline model, the LTF and MSA blocks significantly improve model performance and reduce errors. The LTF block effectively captures long-term crop growth characteristics, while the MSA block excels in capturing multi-scale features and spatiotemporal correlations. Comparing LM-net with a model lacking individual blocks shows that the features extracted by these two blocks are effectively fused through the multilayer perceptron decoder, reflecting their complementarity and overall performance improvement. Furthermore, the LTF block, as an independent feature extraction module, can be seamlessly integrated into various models. To validate its effectiveness, we integrated the LTF block into an existing model. Figure 4 The experimental results show that, on average over a five-year period, all models integrating the LTF block exhibited significant performance improvements in soybean and maize yield prediction. RMSE decreased by 7.2% to 24.5%, and MAPE decreased by 6.1% to 22.2%. Notably, ME* decreased by 36.1% to 84.7%. These results validate the ability of the LTF block to capture long-term environmental and growth changes in crops, complement features obtained from remote sensing data, and support our hypothesis that yield sequences provide crucial information that remote sensing data fails to capture adequately.

[0046] (3) Sensitivity analysis

[0047] To explore the factors affecting model performance, we conducted sensitivity analysis experiments. The LTF block includes four key parameters: linear regression results, mean and variance of the yield series, time variable T, and confidence variable key. We analyzed the following cases: (1) complete: including all parameters; (2) wo / ke: removing confidence variable key; (3) wo / li: removing linear regression results; (4) wo / ye: removing time variable T; (5) wo / ms: removing mean and variance.

[0048] Figure 2 The sensitivity analysis results are presented, showing that the model performs best when all parameters are included, demonstrating the contribution of each parameter to model effectiveness. Specifically, the time variable T has the greatest impact on the model. Removing this variable significantly reduces model performance because the model cannot distinguish LTF block features of the same location at different time periods, potentially leading to information redundancy. On the other hand, removing the confidence variable key slightly reduces performance, indicating that in most cases, the model has a low dependence on the confidence variable, possibly because historical data for most regions is sufficient to support model predictions without additional confidence adjustments. Removing linear regression results or the mean and variance leads to an increase in mean squared error and mean absolute percentage error, indicating the importance of these factors in the model. Attached Figure Description

[0049] Figure 1 This is a diagram of the overall architecture of LM-net;

[0050] Figure 2 The results of the sensitivity analysis experiment are shown in the figure.

[0051] Figure 3 A graph showing the prediction performance of the model on the US soybean dataset;

[0052] Figure 4 A graph showing the prediction performance of the model on the US corn dataset;

[0053] Figure 5 Figure showing the results of the module ablation experiment;

[0054] Figure 6 This is a graph showing the performance improvement of the LTF module on existing models;

[0055] Figure 7 This is a flowchart of the method of the present invention;

[0056] Figure 8 This is a block diagram of the device of the present invention;

[0057] Figure 9 This is a schematic diagram of the structure of the first electronic device of the present invention;

[0058] Figure 10 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;

[0059] Figure 11 This is a schematic diagram of the structure of the second electronic device of the present invention.

[0060] Figure label:

[0061] A - First electronic device;

[0062] B-A crop yield estimation device based on linear time-series features and multi-scale attention;

[0063] C-Data acquisition equipment;

[0064] D-Information display device;

[0065] 1000 - Second electronic device;

[0066] Ⅰ-Computational Unit;

[0067] II-ROM;

[0068] III-RAM;

[0069] N-bus;

[0070] V-Interface;

[0071] VI - Input Unit;

[0072] VII - Output Unit;

[0073] VIII - Storage medium;

[0074] IX - Communication Unit. Detailed Implementation

[0075] It should be noted that, in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0076] In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0077] The processor described in this invention is the control center of an electronic device. It can be a single processor or a collective term for multiple processing elements. For example, it can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of this invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0078] Alternatively, the processor can perform various functions of the electronic device by running or executing software programs stored in memory and by calling data stored in memory.

[0079] In a specific implementation, as one example, the processor may include one or more CPUs. Each of these processors may be a single-core processor or a multi-core processor. Here, "processor" can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions). Electronic devices may include servers, desktop computers, laptops, smartphones, tablets, embedded computers, etc., where the embedded computer includes vehicles and robots, etc.

[0080] The memory is used to store the software program that executes the solution of the present invention, and the execution is controlled by the processor. For specific implementation methods, please refer to the above method embodiments, which will not be repeated here.

[0081] It should be noted that the structure of the electronic device shown in the accompanying drawings of this invention does not constitute a limitation thereof. The actual knowledge structure recognition device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0082] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0083] It should also be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0084] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0085] It should also be understood that, in various embodiments of the present invention, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0086] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0087] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0089] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0090] To address the aforementioned challenges, this invention proposes a novel yield estimation model, LM-net, that integrates linear features and multi-scale spatial self-attention. This model consists of two core modules: a Linear Temporal Feature Extraction (LTF) block and a Multi-Scale Attention Feature (MSA) block. The LTF block aims to extract features from crop yield time series from different regions to capture differences in crop growing environments. It treats historical yield data for each region as a time series and learns about regional growing environment differences by mining features from the yield series. The LTF block is independent of existing prediction models and can be easily integrated into them to improve prediction accuracy. The multi-scale attention feature block improves crop yield estimation models based on remote sensing images. It is designed to address the attention shift problem that may occur during crop growth feature extraction, enhancing the model's sensitivity to features at different growth stages. By optimizing the feature extraction process, this module can better capture crop features at different growth stages, thereby improving the overall predictive performance of the model. Our model was evaluated on US county-level maize and soybean datasets, showing superior performance compared to other competing techniques. Furthermore, integrating the LTF block into existing models can significantly improve the accuracy of yield estimation. Therefore, the present invention includes the following key technical points:

[0091] Key Point 1: Linear Time Feature Extraction; This module treats historical yield data for each region as a time series and learns about regional differences in growing environments by mining the features of the yield series. Independent of existing prediction models, this module can be easily integrated into them to improve prediction accuracy.

[0092] Key Point 2: Multi-scale attention feature extraction; This module utilizes attention mechanisms at different scales to extract the dependencies of crop growth at different stages, enhancing the model's sensitivity to features at different growth stages. By optimizing the feature extraction process, this module can better capture the characteristics of crops at different growth stages, thereby improving the overall predictive performance of the model.

[0093] To make the above-mentioned features and effects of the present invention clearer and easier to understand, specific embodiments are described below in conjunction with the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are merely illustrative. The scope of protection of the present invention is not limited to the disclosed embodiments, but is defined by the appended claims.

[0094] The overall framework of LM-net is as follows: Figure 1 As shown, LM-net comprises two important components: the Linear Time Feature Block (LTF block) and the Multi-Scale Attention Block (MSA block). These two modules extract features from different perspectives. The MSA block captures real-time dynamic features of crop growth and changes at various growth stages from remote sensing image sequences, while the LTF block extracts long-term environmental patterns and trends from historical yield sequences, significantly reflecting regional differences. In this section, we will discuss each component of LM-net in detail.

[0095] The Linear Time Feature Module (LTF) treats the historical crop yield of each plot and the corresponding year as a time series during training. It first performs linear regression modeling on the two variables, extracting parameters such as slope (β1), intercept (β0), correlation coefficient (r), p-value (p), and standard error of slope (SE). β1 ) and other parameters. These parameters are related to the mean (μ) and variance (σ) of the yield series. 2 By combining these methods, we establish statistical characteristics for the region, reflecting the variation of crop yields across different areas over the years. For most regions, simple linear regression can obtain the corresponding statistical characteristics. However, historical yield data is severely lacking in some regions. With insufficient sample size, the accuracy of linear regression decreases significantly, and some regions even lack historical yield labels entirely, making it impossible to derive corresponding statistical data through linear regression. For regions that cannot be directly modeled, we use a feature imputation algorithm to assign statistical characteristics to these regions. We then encode the obtained statistical characteristics into high-dimensional features using a neural network (MLP), and combine them with remote sensing features extracted by a multi-scale attention module for yield prediction. This module performs linear regression, with the input data being the yield of each region in different years and the corresponding year. The training from low-dimensional statistical features to high-dimensional features, along with the second module, utilizes supervised training to directly complete the entire network training process.

[0096] For the Multi-Scale Attention (MSA) module, the input data consists of remote sensing time-series images of crop growth cycles for each region. We first use a histogram algorithm to reduce the dimensionality of the remote sensing data, resulting in a feature sequence representing the remote sensing feature encodings at different time points. Next, we utilize attention mechanisms at different scales to capture the dependencies between time steps. The training of this module is synchronized with the high-dimensional encoder in the LTF block, and the loss function is the MSE loss.

[0097] Linear time feature blocks:

[0098] Linear time feature blocks consist of three basic steps: linear regression, feature interpolation, and high-dimensional mapping. Figure 1 This demonstrates the data processing flow of the module, whose input is the output sequence {y1,y2,y3,…,y} corresponding to the region in the image. n First, a linear regression analysis is performed on the sequence to extract parameters such as slope (β1), intercept (β0), correlation coefficient (r), p-value (p), and standard error of slope (SE). β1 ) and other parameters. These parameters are related to the mean (μ) and variance (σ) of the yield series. 2 By combining these methods, statistical characteristics of the region can be established. For most regions, statistical values ​​can be obtained using the methods described above. However, historical yield data is severely lacking in some regions. With insufficient sample size, the accuracy of linear regression decreases significantly, and some regions even lack historical yield labels entirely, making it impossible to derive corresponding statistical data through linear regression. Therefore, feature imputation becomes necessary for regions that cannot be directly modeled.

[0099] In the feature imputation step, for regions lacking sufficient historical yield data, we assume that geographically proximate regions have similar crop growth conditions. Based on this, we use a KD-tree to identify the nearest neighbor regions for each region that cannot be directly modeled. Specifically, by maintaining a KD-tree, we find the nearest neighbor plots to the plots with missing data and use the regression parameters of these regions as statistical features of the unmapped regions. However, the reliability of these imputed statistical features may be reduced. Therefore, we introduce a confidence measurement key to record the reliability of the statistical features. The initial confidence level is 1. If a plot has missing data, the confidence level is assigned according to the distance of the KD-tree search path; for example, if the search path is 3, the confidence level key is 1 / 3. Afterwards, we obtain a feature vector for each region, which can be represented as:

[0100] F-=[β1,β0,r,p,SE,μ,σ 2 [key]

[0101] When training the model to predict next year's crop yield, we use historical data up to the current year and need to distinguish between data from different years within the same region when inputting it into the model. We introduce a time variable T to differentiate the input features of the same region in different years. The time variable T is defined as follows:

[0102] T = g year -g base

[0103] Where g year g represents the year of the data record. base The base year is the data we need when designing the LTF module and introducing the variable T. After completing linear encoding, the statistical characteristics of the same plot of land in different years will become consistent. However, the yield variation of the same plot of land in different years is significant. If the features of the LTF module are all consistent across different years, it will cause huge interference to the model. Therefore, we introduce time encoding T to distinguish the statistical characteristics of different years in the same region. For example, if the data starts from 2003 and continues to 2020, it is not appropriate to directly use the year as T when performing time encoding. Therefore, we introduce the base year as 2002, and the time encoding becomes 1, 2, 3, ..., 18. This can amplify the slope of the linear regression, which can better improve the feature extraction effect and thus improve the accuracy of yield prediction.

[0104] At this point, we have obtained all the necessary variables. Finally, we use a multilayer perceptron to map these statistics into a high-dimensional space, capturing the nonlinear relationships and higher-order interactions between features. The encoded high-dimensional features are represented as:

[0105]

[0106] Multi-scale attention blocks:

[0107] Composed of a convolutional block and two attention layers operating at different time scales, this model aims to extract effective features and identify trends from remote sensing data during the crop growing season. In the data processing flow, the input data is first preprocessed, converting the data for each band into a one-dimensional feature vector. This feature vector is then passed to the convolutional block, which consists of multiple convolutional operations, each followed by batch normalization and ReLU activation. Through these operations, a feature sequence representing feature encodings at different time points is extracted. To better capture the temporal dependencies of different stages in the crop growth cycle, positional encoding is added to the generated feature sequence. This step, by introducing temporal information into the feature sequence, enables the model to distinguish features at different time points.

[0108] Next, the feature sequence is fed into the first attention layer. In this layer, a self-attention mechanism is used to capture dependencies between different time steps, meaning the model can identify the correlation between a given time point and other time points during crop growth. In the second attention layer, the model focuses on capturing correlations over longer time spans. We divide the feature sequence into time periods, and the features within each time period are fed into a multilayer perceptron for encoding. These encoded features represent trends or feature changes over a longer period and are used for further attention calculations. In this way, the model is able to identify long-term correlations between different stages of crop growth.

[0109] In the final stage of the model, we introduce residual connections to ensure its stability and effectiveness. To ensure consistent dimensionality of the outputs from the two attention mechanisms, the output of the first layer is processed through a multilayer perceptron. The processed result is added to the output of the second layer and then fed into a feedforward neural network for further processing. Finally, all generated features are combined with features from other modules and fed into the multilayer perceptron decoder to predict the final crop yield.

[0110] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0111] like Figure 8 As shown, this invention also proposes a crop yield estimation device based on linear time-series features and multi-scale attention, including:

[0112] The module constructs a yield estimation model that includes a linear time feature extraction module and a multi-scale attention feature module. The linear time feature extraction module is used to extract environmental pattern features from the time series of crop yields in different regions, and the multi-scale attention feature module is used to analyze the real-time growth of crops from remote sensing image sequences to generate remote sensing features.

[0113] The training module acquires training data consisting of historical crop yields of multiple plots and corresponding remote sensing image sequences for each plot, with each plot labeled with a yield tag. The linear time feature module treats the historical crop yields of the plots in the training data and the corresponding years as a time series, and extracts environmental pattern features of the time series through linear regression. The multi-scale attention feature module uses attention mechanisms at different scales to extract remote sensing features from the remote sensing image sequences in the training data. The environmental pattern features and the remote sensing features are fed into the decoder of the multilayer perceptron to obtain the predicted yield. Based on the predicted yield and the yield tag, a loss function is constructed and backpropagated to train the yield estimation model.

[0114] The yield estimation module takes the historical crop yield and remote sensing image sequence of the plot to be estimated as input into the trained yield estimation model to obtain the yield prediction results of the plot to be estimated.

[0115] The crop yield estimation device based on linear time-series features and multi-scale attention, wherein the linear time-series feature block includes:

[0116] To determine if there are missing data points in the historical crop yield data, if so, a balanced binary tree KD model is used to identify the nearest neighbor plot for each missing data point, and the regression parameters of the nearest neighbor plot are used as the statistical features of the missing data. Otherwise, linear regression analysis is directly performed on the time series to obtain the statistical features. A multilayer perceptron is used to map the statistical features and their corresponding time variables to a high-dimensional space to obtain the environmental pattern features, where the time variable is the difference between the year the data was recorded and the base year.

[0117] The crop yield estimation device based on linear time-series features and multi-scale attention, wherein the multi-scale attention feature module consists of convolutional blocks and attention layers operating at different time scales, used to extract effective features and identify trends from remote sensing data during the crop growing season.

[0118] Each band data of each remote sensing image in the remote sensing image sequence is converted into a one-dimensional feature vector. This feature vector is passed to the convolutional block to obtain the feature encoding sequence at different time points. Position and time information are added to the feature encoding sequence to obtain the intermediate sequence. The intermediate sequence is input into the attention layer, and the self-attention mechanism is used to capture the dependencies between time steps and the correlation between time spans to obtain the remote sensing feature.

[0119] This multi-scale attention feature module includes two attention layers;

[0120] The first attention layer uses a self-attention mechanism to capture the dependencies between time steps in the intermediate sequence, and the second attention layer is used to capture the correlations over a preset time span.

[0121] The feature sequences obtained from each attention layer are divided into time periods. The features in each time period are fed into a multilayer perceptron for encoding, and the encoding results are added together and input into a feedforward neural network to obtain the remote sensing feature.

[0122] like Figure 9 As shown, in another embodiment of the present invention, a first electronic device A is also proposed, which includes the aforementioned crop yield estimation device based on linear time-series features and multi-scale attention.

[0123] like Figure 10 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D through a wired or wireless information transmission scheme. The data acquisition device C is used to acquire the remote sensing image sequence to be identified and classified, and the information display device D is used to display the yield prediction results obtained by the present invention.

[0124] Information display device D can process and organize the data output by the first electronic device A based on an information display mechanism to improve the readability of the data. This information display mechanism can be manually preset, for example, visualizing the data output by the first electronic device A. It can be based on user-defined display parameters and / or attributes, such as the data range and font, color, and scrolling options. It can present users with their specified key information, such as production distribution and trend information in production forecasts, allowing users to access this information more quickly without needing to access secondary pages or scroll through pages, saving user effort. Alternatively, this information display mechanism can be an artificial intelligence (AI) display model, which can learn the user's key information based on previous usage habits, such as viewing time, number of clicks, and number of edits, and automatically present the user with rich and necessary key information.

[0125] The present invention also provides a computer program product, which includes a computer program that can be stored on a readable storage medium. When the computer program is executed by a processor, the computer is able to execute the crop yield estimation method based on linear time-series features and multi-scale attention provided by the above methods.

[0126] In another embodiment, the present invention also proposes a storage medium VIII for storing a computer program that executes the crop yield estimation method based on linear time-series features and multi-scale attention. It should be understood that the storage medium in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0127] Figure 11 A schematic block diagram of a second electronic device 1000 that can be used to implement embodiments of the present invention is shown. The second electronic device 1000 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein. The second electronic device 1000 may be the same as or different from the first electronic device A.

[0128] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from storage medium VIII into random access memory (RAM) III. The RAM III may also store various programs and data required for the operation of the device 1000. The computing unit I, ROM II, and RAM III are interconnected via bus IV. An input / output (I / O) interface V is also connected to bus IV.

[0129] Multiple components in the second electronic device 1000 are connected to I / O interface V, including: input unit VI, such as a keyboard, mouse, etc.; output unit VII, such as various types of displays, speakers, etc.; storage medium VIII, such as a disk, optical disk, etc.; and communication unit IX, such as a network card, modem, wireless transceiver, etc. Communication unit IX allows the second electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0130] The computing unit I can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit I performs the various methods and processes described above, such as method steps S1-S3. For example, in some embodiments, the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage medium VIII. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1000 via ROM II and / or communication unit IX. When the computer program is loaded into RAM III and executed by computing unit I, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, computing unit I can be configured to perform methods by any other suitable means (e.g., by means of firmware).

[0131] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A crop yield estimation method based on linear time-series features and multi-scale attention, characterized in that, include: The construction steps involve building a yield estimation model that includes a linear time feature extraction module and a multi-scale attention feature module. The linear time feature extraction module is used to extract environmental pattern features from the time series of crop yields in different regions, and the multi-scale attention feature module is used to analyze the real-time growth of crops from remote sensing image sequences to generate remote sensing features. The training process involves acquiring training data consisting of historical crop yields from multiple plots and corresponding remote sensing image sequences for each plot, with each plot labeled with a yield tag. The linear time feature extraction module treats the historical crop yields and corresponding years of the plots in the training data as a time series and extracts environmental pattern features from this time series using linear regression. The multi-scale attention feature module utilizes attention mechanisms at different scales to extract remote sensing features from the remote sensing image sequences in the training data. The environmental pattern features and the remote sensing features are then fed into the decoder of a multilayer perceptron to obtain the predicted yield. Based on the predicted yield and the yield tag, a loss function is constructed and backpropagated to train the yield estimation model. The yield estimation step involves inputting the historical crop yields and remote sensing image sequences of the plot to be estimated into the trained yield estimation model to obtain the yield prediction results for the plot.

2. The crop yield estimation method based on linear time-series features and multi-scale attention as described in claim 1, characterized in that, The linear time feature extraction module includes: To determine if there are missing data points in the historical crop yield data, if so, a balanced binary tree KD model is used to identify the nearest neighbor plot for each missing data point, and the regression parameters of the nearest neighbor plot are used as the statistical features of the missing data. Otherwise, linear regression analysis is directly performed on the time series to obtain the statistical features. A multilayer perceptron is used to map the statistical features and their corresponding time variables to a high-dimensional space to obtain the environmental pattern features, where the time variable is the difference between the year the data was recorded and the base year.

3. The crop yield estimation method based on linear time-series features and multi-scale attention as described in claim 1, characterized in that, This multi-scale attention feature module consists of convolutional blocks and attention layers operating at different time scales, used to extract effective features and identify trends from remote sensing data during the crop growing season. Each band data of each remote sensing image in the remote sensing image sequence is converted into a one-dimensional feature vector. This feature vector is passed to the convolutional block to obtain the feature encoding sequence at different time points. Position and time information are added to the feature encoding sequence to obtain the intermediate sequence. The intermediate sequence is input into the attention layer, and the self-attention mechanism is used to capture the dependencies between time steps and the correlation between time spans to obtain the remote sensing feature.

4. The crop yield estimation method based on linear time-series features and multi-scale attention as described in claim 3, characterized in that, This multi-scale attention feature module includes two attention layers; The first attention layer uses a self-attention mechanism to capture the dependencies between time steps in the intermediate sequence, and the second attention layer is used to capture the correlations over a preset time span. The feature sequences obtained from each attention layer are divided into time periods. The features in each time period are fed into a multilayer perceptron for encoding, and the encoding results are added together and input into a feedforward neural network to obtain the remote sensing feature.

5. A crop yield estimation device based on linear time-series features and multi-scale attention, characterized in that, include: The module constructs a yield estimation model that includes a linear time feature extraction module and a multi-scale attention feature module. The linear time feature extraction module is used to extract environmental pattern features from the time series of crop yields in different regions, and the multi-scale attention feature module is used to analyze the real-time growth of crops from remote sensing image sequences to generate remote sensing features. The training module acquires training data consisting of historical crop yields of multiple plots and remote sensing image sequences corresponding to each plot, with each plot labeled with a yield tag. The linear time feature extraction module treats the historical crop yields of the plots in the training data and the corresponding years as a time series, and extracts the environmental pattern features of the time series through linear regression. The multi-scale attention feature module uses attention mechanisms at different scales to extract remote sensing features from the remote sensing image sequences in the training data. The environmental pattern features and the remote sensing features are fed into the decoder of the multilayer perceptron to obtain the predicted yield. Based on the predicted yield and the yield tag, a loss function is constructed and backpropagated to train the yield estimation model. The yield estimation module takes the historical crop yield and remote sensing image sequence of the plot to be estimated as input into the trained yield estimation model to obtain the yield prediction results of the plot to be estimated.

6. The crop yield estimation device based on linear time-series features and multi-scale attention as described in claim 5, characterized in that, The linear time feature extraction module includes: To determine if there are missing data points in the historical crop yield data, if so, a balanced binary tree KD model is used to identify the nearest neighbor plot for each missing data point, and the regression parameters of the nearest neighbor plot are used as the statistical features of the missing data. Otherwise, linear regression analysis is directly performed on the time series to obtain the statistical features. A multilayer perceptron is used to map the statistical features and their corresponding time variables to a high-dimensional space to obtain the environmental pattern features, where the time variable is the difference between the year the data was recorded and the base year.

7. The crop yield estimation device based on linear time-series features and multi-scale attention as described in claim 5, characterized in that, This multi-scale attention feature module consists of convolutional blocks and attention layers operating at different time scales, used to extract effective features and identify trends from remote sensing data during the crop growing season. Each band data of each remote sensing image in the remote sensing image sequence is converted into a one-dimensional feature vector. This feature vector is passed to the convolutional block to obtain the feature encoding sequence at different time points. Position and time information are added to the feature encoding sequence to obtain the intermediate sequence. The intermediate sequence is input into the attention layer, and the self-attention mechanism is used to capture the dependencies between time steps and the correlation between time spans to obtain the remote sensing feature. This multi-scale attention feature module includes two attention layers; The first attention layer uses a self-attention mechanism to capture the dependencies between time steps in the intermediate sequence, and the second attention layer is used to capture the correlations over a preset time span. The feature sequences obtained from each attention layer are divided into time periods. The features in each time period are fed into a multilayer perceptron for encoding, and the encoding results are added together and input into a feedforward neural network to obtain the remote sensing feature.

8. An electronic device, characterized in that, The crop yield estimation device based on linear time-series features and multi-scale attention as described in any one of claims 5-7 includes an electronic device or an information display device connected to it, which displays the yield prediction result using user-set display parameters, attributes, or through an artificial intelligence model.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the crop yield estimation method based on linear temporal features and multi-scale attention as described in any one of claims 1-4.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the crop yield estimation method based on linear time-series features and multi-scale attention as described in any of claims 1-4.

Images