Vegetation index time sequence prediction method and device based on deep learning

Abstract

The invention discloses a vegetation index time sequence prediction method and device based on deep learning, and the method comprises the following steps: obtaining and processing historical period vegetation indexes and environmental factor data of a preset time period, and obtaining NDVI, EVI and kNDVI data as vegetation indexes, environment factor data including temperature, rainfall, evapotranspiration, wind speed, relative humidity and soil humidity; by determining the intensity and direction of the linear relation between the environmental factors and the vegetation indexes, the environmental factors with high correlation with the vegetation indexes are selected; a CNN-BiLSTM-AM model comprising an input layer, a CNN layer, a BiLSTM layer, an AM layer and an output layer is constructed to predict a vegetation index time sequence, environmental factors with high correlation serve as input data, and NDVI, EVI and kNDVI vegetation indexes serve as output prediction targets. The technical process is simple and clear, and the accuracy of vegetation index time sequence prediction is greatly improved.

Description

Technical Field

[0001] The present invention relates to the application field of remote sensing technology, and more specifically to a vegetation index time series prediction method and device based on deep learning. Background Art

[0002] Vegetation index time series prediction can use remote sensing technology to quantitatively invert vegetation dynamic changes and indicate vegetation growth or degradation. It plays an important role in data and technical support in studying the impact of different climate factors on vegetation growth and the development and change of vegetation under future climate conditions. It has broad application prospects and far-reaching impacts. Vegetation index time series prediction technology has become an important application technology in vegetation remote sensing.

[0003] Vegetation index time series prediction has been carried out based on statistical methods such as minimum absolute shrinkage and selection operators, and machine learning algorithms such as autoregressive moving average, support vector machine and artificial neural network, and some results have been achieved. However, these traditional vegetation index prediction methods have some limitations and cannot fully exploit the intrinsic characteristics of time series data in solving complex and nonlinear problems.

[0004] Deep learning can automatically extract high-dimensional and complex features, especially when dealing with nonlinear and non-stationary data. Deep learning methods have achieved some results in time series prediction. Common methods include convolutional neural network CNN, long short-term memory network LSTM, attention mechanism AM, gated recurrent unit GRU, recurrent neural network RNN, etc. Different deep learning methods have different characteristics in time series prediction. Bidirectional long short-term memory network can effectively capture time series information in both forward and reverse directions, and fully capture the contextual information of the sequence. Convolutional neural network has advantages in two-dimensional data space feature extraction, can automatically learn complex patterns and laws in time series data, and has better computational efficiency and prediction accuracy. The attention mechanism can set different weights according to different parts of the input, helping the model focus on the important parts of the input sequence, thereby improving the performance of time series prediction.

[0005] However, for traditional algorithms and single deep learning methods, the accuracy and generalization ability of vegetation index time series prediction still need to be further enhanced. Summary of the invention

[0006] To solve the above problems, the purpose of the present invention is to provide a vegetation index time series prediction method and device based on deep learning.

[0007] According to one aspect of the present invention, a vegetation index time series prediction method based on deep learning is provided, comprising the following steps:

[0008] Data acquisition and processing steps: obtain and process the historical vegetation index and environmental factor data for a predetermined period;

[0009] Environmental factor screening steps: by determining the strength and direction of the linear relationship between environmental factors and vegetation index, select environmental factors with strong correlation with vegetation index;

[0010] Modeling and prediction steps: A CNN-BiLSTM-AM model structure including an input layer, a CNN layer, a BiLSTM layer, an AM layer, and an output layer is constructed to establish a vegetation index time series model. Environmental factors with a strong correlation with the vegetation index are used as input data, and NDVI, EVI, and kNDVI are used as output prediction targets. The CNN layer includes a convolutional layer and a pooling layer using a ReLU activation function. The BiLSTM layer merges the outputs of the forward and reverse LSTMs, passes the output tensor to the AM layer, maps the weighted data processed by AM to the output layer, processes it using a linear activation function, and outputs a continuous value suitable for the regression task, i.e., the predicted vegetation index.

[0011] Preferably, in the data acquisition and processing step, monthly NDVI and EVI data are generated through quality control, test area and time period extraction, and maximum value synthesis method. The calculation formula kNDVI=tanh(NDVI²) is applied to convert NDVI into monthly kNDVI data, unify all vegetation index data and environmental factor data into a geographic coordinate system, and extract environmental factor data values ​​corresponding to vegetation index samples for modeling and prediction steps.

[0012] Preferably, in the environmental factor screening step, the Pearson correlation analysis method is used to measure the strength and direction of the linear relationship between the environmental factor and the vegetation index, and the correlation coefficient r is calculated. xy :

[0013] ,

[0014] Where: x i and i are vegetation index and environmental factors respectively; and are the average values ​​of vegetation index and environmental factors, n is the number of samples,

[0015] The direction of the correlation is determined by the positive or negative value of the correlation coefficient, and the strength of the correlation is determined by the absolute value of the correlation coefficient. Several environmental factors with strong correlation are selected for the modeling and prediction step.

[0016] Preferably, the Keras framework and the TensorFlow framework are used to train the CNN-BiLSTM-AM model, the feature of each time step is the factor screened in the environmental factor screening step, the vegetation index is the prediction target, the mean absolute error is selected as the loss function, the adaptive motion estimation algorithm is used as the optimizer, and a learning rate decay strategy is used with an initial learning rate of 0.01 and a learning rate decay factor of 0.1.

[0017] Preferably, the CNN layer receives data from the input layer, uses 64 convolution kernels, each convolution kernel has a size of 1*1, and each convolution operation only focuses on the data of one time step; the BiLSTM layer processes the input data simultaneously in both directions of the time series, first processes each time step of the input data sequentially, and calculates the state of the LSTM in the forward and reverse directions respectively; the AM layer calculates the importance weight of each time step or feature, and weights the input data according to these weights, first calculates the attention weight of each time step through a full connection operation, and converts the weight into a probability distribution through normalization to ensure that the sum of the weights of all time steps is 1, secondly, adjusts the dimensional order of the input tensor, exchanges the time dimension with the feature dimension, and aligns the attention weight with the input data, then, multiplies the input data by the calculated attention weight element by element to generate weighted data, and finally, adjusts the dimension of the attention vector in a manner consistent with the dimension of the input data.

[0018] Preferably, the vegetation index time series prediction method based on deep learning also includes a model evaluation step: for the three vegetation indices of EVI, NDVI and kNDVI, a model evaluation index is established, including a determination coefficient R 2 , root mean square error RMSE and mean absolute error MAE:

[0019] ,

[0020] ,

[0021] ,

[0022] in: is the predicted value of vegetation index, y i is the actual value of the vegetation index, is the average of the actual values ​​of the vegetation index, and n is the number of samples.

[0023] According to another aspect of the present invention, a vegetation index time series prediction device based on deep learning is provided, comprising the following modules:

[0024] Data acquisition and processing module: configured to acquire and process the historical vegetation index and environmental factor data of a predetermined period;

[0025] Environmental factor screening module: configured to select environmental factors with strong correlation with vegetation index by determining the strength and direction of the linear relationship between environmental factors and vegetation index;

[0026] Prediction module: It is configured to build a CNN-BiLSTM-AM model including an input layer, a CNN layer, a BiLSTM layer, an AM layer, and an output layer to predict the vegetation index time series. Among them, the environmental factors with strong correlation with the vegetation index are used as input data, and the NDVI, EVI and kNDVI vegetation indices are used as output prediction targets respectively. The CNN layer includes a convolution layer and a pooling layer using the ReLU activation function. The BiLSTM layer merges the outputs of the forward and reverse LSTMs, passes the output tensor to the AM layer, maps the weighted data processed by AM to the output layer, processes it using a linear activation function, and outputs a continuous value suitable for the regression task, that is, the predicted vegetation index.

[0027] According to another aspect of the present invention, a vegetation index time series prediction device based on deep learning is provided, which is constructed by using a CNN-BiLSTM-AM model structure including an input layer, a CNN layer, a BiLSTM layer, an AM layer, and an output layer as a vegetation index time series model, wherein the input data of the input layer is set to multiple environmental factors including temperature, precipitation, and evapotranspiration, the output of the output layer is set to any one of NDVI, EVI, and kNDVI as the vegetation index, the CNN layer includes a convolution layer and a pooling layer using a ReLU activation function, the BiLSTM layer merges the outputs of the forward and backward LSTMs, passes the output tensor to the AM layer, maps the weighted data processed by AM to the output layer, processes it using a linear activation function, and outputs a continuous value suitable for the regression task, that is, any one of NDVI, EVI, and kNDVI as the predicted vegetation index.

[0028] According to another aspect of the present invention, there is provided an electronic device, comprising: a processor for executing a plurality of instructions; and a memory for storing the plurality of instructions; wherein the plurality of instructions are used to be stored in the memory and loaded by the processor to execute the method as described above.

[0029] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the method as described above.

[0030] The technical process of the present invention is simple and clear, which greatly improves the accuracy of vegetation index time series prediction. The present invention integrates the advantages of multiple time series prediction methods, namely the feature extraction ability of convolutional neural networks, the time series data processing advantages of bidirectional long short-term memory networks, and the key information focusing function of attention mechanisms, and proposes a vegetation index prediction method based on convolutional neural networks-bidirectional long short-term memory networks-attention mechanisms (CNN-BiLSTM-AM). This combination can capture short-term local patterns and identify long-term global dependencies, significantly improving the prediction accuracy and robustness of the model, and also performs well in the processing of high-dimensional and multi-feature data, and can efficiently filter redundant information and improve generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A flowchart of vegetation index time series prediction according to an exemplary embodiment of the present invention is schematically shown.

[0032] Figure 2 The structure diagram of the CNN-BiLSTM-AM model is schematically shown.

[0033] Figure 3 Schematically showing the LSTM model results for comparison of different model results.

[0034] Figure 4 The CNN-BiLSTM model results for comparison of different model results are schematically shown.

[0035] Figure 5 The BiLSTM-AM model results for comparison of different model results are schematically shown.

[0036] Figure 6 The CNN-BiLSTM-AM model results for comparison of different model results are schematically shown.

[0037] Figure 7 It is a schematic diagram of the device structure of the present invention.

[0038] Figure 8 It is a schematic diagram of the structure of the electronic device of the present invention. DETAILED DESCRIPTION

[0039] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, the numerical expressions and the numerical values ​​described in these embodiments do not limit the scope of the present invention. It can be understood by those skilled in the art that the terms "first", "second", S1, S2, etc. in the embodiments of the present invention are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor represent the necessary logical order between them. It should also be understood that in the embodiments of the present invention, "multiple" can refer to two or more, and "at least one" can refer to one, two or more. It should also be understood that for any component, data or structure mentioned in the embodiments of the present invention, in the absence of explicit limitation or contrary revelation given in the context, it can generally be understood as one or more. In addition, the term "and / or" in the present invention is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the objects associated with each other are in an "or" relationship. It should also be understood that the description of each embodiment of the present invention emphasizes the differences between the embodiments, and the same or similar aspects can be referred to each other. For the sake of brevity, they will not be described one by one. At the same time, it should be understood that for the convenience of description, the dimensions of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The following description of at least one exemplary embodiment is actually only illustrative and is by no means a limitation on the present invention and its application or use. The techniques, methods and equipment known to ordinary technicians in the relevant fields may not be discussed in detail, but where appropriate, the techniques, methods and equipment should be considered as part of the specification. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0040] Embodiments of the present invention can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc. Terminal devices, computer systems, servers, etc. Electronic devices can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, target programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0041] In an exemplary embodiment of the present invention, a complex neural network model is established through deep learning with a simple and clear technical process, and a method for predicting vegetation index time series based on deep learning is proposed based on automatic extraction of data features and strong nonlinear modeling capabilities.

[0042] Among them, by integrating the advantages of multiple time series prediction methods, namely the feature extraction ability of convolutional neural networks, the time series data processing advantages of bidirectional long short-term memory networks, and the important information attention function of the attention mechanism, a vegetation index prediction method based on convolutional neural network-bidirectional long short-term memory network-attention mechanism (CNN-BiLSTM-AM) is proposed, and environmental factors are comprehensively considered.

[0043] Environmental factors are significantly correlated with changes in vegetation indices. In order to effectively improve the accuracy of vegetation index time series prediction, environmental factors are first screened. Then, for each preferred environmental factor and multiple vegetation indices, the vegetation index prediction method of convolutional neural network-bidirectional long short-term memory network-attention mechanism (CNN-BiLSTM-AM) is used to establish multiple vegetation index time series prediction models, and the vegetation index time series prediction model with good prediction effect is selected.

[0044] Exemplary Methods

[0045] In some embodiments, by screening environmental factors that have a strong correlation with various vegetation indices in historical periods, a vegetation index time series model is established using the CNN-BiLSTM-AM model structure.

[0046] Specifically, the vegetation index time series prediction operation process can be roughly divided into five steps:

[0047] The first step is to obtain and process the vegetation index and environmental factor data in the historical period;

[0048] The second step is to screen environmental factors and select environmental factors with strong correlation with vegetation index;

[0049] The third step is modeling;

[0050] The fourth step is to establish model evaluation indicators;

[0051] The fifth step is to obtain the optimized model.

[0052] The overall process of vegetation index time series prediction operation is shown in Figure 1 As shown. The following will introduce this method in detail according to the overall flow chart of the vegetation index time series prediction method based on deep learning.

[0053] The first step is to obtain and process the vegetation index and environmental factor data in the historical period. The long-term remote sensing data (i.e., the vegetation index data obtained by long-term remote sensing) and long-term environmental data (i.e., the environmental factor data of long-term series) are obtained and unified into the geographic coordinate system, and the environmental factor data values ​​corresponding to the vegetation index samples for modeling and prediction are extracted:

[0054] For example, the Yangtze River Basin was used as the experimental area, the historical period as the predetermined time period, and the period from 2001 to 2020 as the research period. The vegetation indices used were NDVI (Normalized Difference Vegetation Index), EVI (Enhanced Vegetation Index) and kNDVI (KernelNoramlized Difference Vegetation Index), and the environmental factors used were temperature (i.e. air temperature), precipitation, evapotranspiration, wind speed, relative humidity and soil moisture.

[0055] 1. Obtain monthly NDVI, EVI, and kNDVI data for the experimental area over the past 20 years from MOD13A data products.

[0056] Obtain NDVI and EVI data with a temporal resolution of 16 days and a spatial resolution of 500 meters from the MOD13A dataset. After quality control, test area and time period extraction, and maximum value synthesis method, generate monthly NDVI and EVI data. Then apply the calculation formula kNDVI=tanh(NDVI²) to convert NDVI into monthly kNDVI data.

[0057] 2. Obtain relevant environmental factor data such as monthly temperature, precipitation, evapotranspiration, wind speed, relative humidity and soil moisture in the experimental area for 20 years.

[0058] 3. Unify all vegetation index data and environmental factor data into the geographic coordinate system CGS_WGS_1984, and the data type is GeoTIFF format. Extract the environmental factor data values ​​corresponding to the vegetation index samples to participate in subsequent modeling.

[0059] The second step is to screen environmental factors. Pearson correlation analysis method is used to select environmental factors with strong correlation with vegetation index:

[0060] The Pearson correlation analysis method is used to measure the strength and direction of the linear relationship between environmental factors and vegetation index. The calculation method is as follows (1):

[0061] (1)

[0062] Where: r xy is the correlation coefficient, x i and i are vegetation index and environmental factors respectively; and are the average values ​​of vegetation index and environmental factors, respectively, and n is the number of samples.

[0063] The direction of the correlation is determined by the positive or negative value of the correlation coefficient. A positive correlation indicates a positive correlation between the two variables, and a negative correlation indicates a negative correlation between the two variables. The strength of the correlation is determined by the absolute value of the correlation coefficient. A correlation coefficient greater than 0.7 or less than -0.7 is considered a strong correlation, 0.4 to 0.7 or -0.4 to -0.7 is considered a medium correlation, and 0 to 0.4 or -0 to -0.4 is considered a weak correlation.

[0064] As shown in Table 1, temperature, precipitation and evapotranspiration are all positively correlated with the three vegetation indices, with correlation coefficients ranging from 0.550 to 0.752 and an average correlation coefficient of 0.662, indicating strong correlation. Wind speed, relative humidity and soil moisture have both positive and negative correlations with the three vegetation indices, with absolute values ​​of correlation coefficients ranging from 0.121 to 0.492, indicating weak correlation. The top three factors with strong correlation are selected to participate in the vegetation index prediction modeling.

[0065] Table 1 Correlation coefficients between vegetation index and environmental factors

[0066]

[0067] The third step is modeling, vegetation index time series modeling based on deep learning:

[0068] As Figure 2 shown, for example, the Keras and TensorFlow frameworks in Python 3.8 are used to train the CNN-BiLSTM-AM model. The structure of the CNN-BiLSTM-AM model includes an input layer, a CNN layer, a BiLSTM layer, an AM layer, and an output layer.

[0069] <Input layer>

[0070] This input layer provides the structural information of the input data for the subsequent layers of the model, and sets the number of time steps of the input data and the number of data features at each time step. When building the model, the Keras framework automatically understands the dimensions of the data, assigns the correct input and output dimensions to each layer, and passes them into the model.

[0071] Here, the selected data covers the vegetation index (VI) and environmental factor data from 2001 to 2020, a total of 240 months. The number of time steps is 240 (indicating that one month is one step), and the number of features at each time step is 3, namely including 3 features of temperature (TMP), precipitation (PRE), and evapotranspiration (ET). The vegetation index is the prediction target. 70% of this dataset is used for training, and 30% is used for validation.

[0072] <CNN layer>

[0073] This CNN layer is mainly composed of a convolutional layer and a pooling layer, and is mainly used to extract local features from time series data. By sliding the convolutional kernel over the data and combining the pooling layer to reduce the spatial dimensions (such as width and height) of the feature map, the computational amount is effectively reduced and overfitting is prevented. The pooling layer retains the most significant feature information and reduces the dimension of the feature map for more efficient subsequent processing. This process captures the local patterns in the input data and realizes the extraction of important features.

[0074] Here, this CNN layer receives the data from the input layer, uses 64 convolutional kernels, and the size of each convolutional kernel is 1*1. Each convolution operation only focuses on the data of one time step, and introduces a non-linear transformation through the ReLU activation function to enhance the expressive ability of the model. The convolution operation reduces the dimension of the data and maps it into 64 different feature maps, which helps the neural network focus on the most informative features. The CNN layer learns the spatial or temporal relationships of the data through local connections. At each time step, the convolutional kernels share weights and gradually extract the local patterns in each time step, thus capturing meaningful features. The pooling layer downsamples the feature map output by the convolutional layer, reduces the spatial dimension of the data, reduces the computational amount, reduces memory consumption, and enhances the robustness of the model to prevent overfitting.

[0075] <BiLSTM layer>

[0076] The Long Short-Term Memory Network (LSTM) processes the input sequence through cycling and uses its internal gating mechanism (forget gate, input gate, output gate) to determine what information to retain and what to forget at each time step. In this way, the LSTM captures long-term dependencies in the time series and is particularly suitable for processing sequence data with long time-step dependencies. The Bidirectional LSTM, i.e., the Bidirectional Long Short-Term Memory Network, processes the input data simultaneously in two directions of the time series through bidirectional wrapping. The forward LSTM processes the sequence from front to back, and the backward LSTM processes the sequence from back to front. This bidirectional processing method enables the model to capture both forward and backward dependencies in the sequence simultaneously. The BiLSTM combines the outputs of the forward and backward LSTMs, usually through a concatenation operation, to obtain a more enriched representation.

[0077] Here, the BiLSTM first processes each time step of the input data in sequence, calculates the states of the LSTM in both the forward and backward directions respectively. Finally, it combines the forward and backward outputs, outputs a new tensor, and passes it into the directly connected AM layer (h1, h2, ...) to further process this tensor and extract key features.

[0078] <AM layer>

[0079] After being processed by the BiLSTM layer, the output tensor is passed to the AM layer. The AM layer calculates the importance weights for each time step or feature, and weights the input data according to these weights, making the model focus more on the parts that have a greater impact on the final prediction result, filtering out useless information, and improving the accuracy of the model prediction.

[0080] During the operation of this AM layer, first, the attention weights for each time step are calculated through a fully connected operation, and the weights are converted into a probability distribution through normalization to ensure that the sum of the weights for all time steps is 1. Secondly, the dimension order of the input tensor is adjusted, swapping the time dimension and the feature dimension to ensure that the attention weights are aligned with the input data. Then, the input data is multiplied element-wise with the calculated attention weights to generate the weighted data, highlighting the time-step features that are more important for the model prediction. Finally, the dimension of the attention vector is adjusted to be consistent with the dimension of the input data, ensuring that the weighted input data can be correctly operated on with the original data and ensuring that the attention mechanism can effectively act on the input data at all time steps.

[0081] <Output layer>

[0082] The weighted data processed by the AM is mapped to the final output layer and processed using a linear activation function to output continuous values suitable for the regression task, i.e., the predicted vegetation index.

[0083] <Model training and tuning>

[0084] During the model training process, the mean absolute error is used as the loss function, which is the average of the absolute values ​​of the difference between the predicted value and the actual value. The mean absolute error is less sensitive to outliers and can effectively measure the deviation between the predicted value and the actual value in regression tasks, providing a stable optimization direction for the model and helping to evaluate the performance of the model.

[0085] In order to optimize the learning process of the model, the adaptive motion estimation algorithm is used as the optimizer. This algorithm combines the advantages of the momentum method and the root mean square optimization algorithm. By adaptively adjusting the learning rate of each parameter, it effectively handles the sparse gradient problem and performs well when processing large-scale data sets. In the adaptive motion estimation algorithm optimizer, the first and second order moment estimation are referenced at the same time, so that the algorithm can converge quickly and improve the training efficiency when facing different types of training data.

[0086] To further improve the stability of training, the initial learning rate is set to 0.01 to ensure that the model can quickly find a more suitable parameter configuration in the early stage of training. However, in the later stage of training, a larger learning rate will lead to instability in model training or miss the optimal solution. For this reason, a learning rate decay strategy is designed. When the training reaches the 180th training cycle, the learning rate is reduced by 90% with a decay factor of 0.1, that is, the learning rate is adjusted to one tenth of the original. This learning rate decay helps the model gradually converge in the later stage of training, avoids too drastic model parameter updates, thereby improving the generalization ability of the model and reducing the risk of overfitting.

[0087] The total number of rounds of model training was set to 200 training cycles. Through multiple iterations, the model gradually approached the optimal solution and showed a good balance between the training set and the validation set. In general, the goal of these adjustments and optimization strategies is to ensure that the model can learn efficiently while avoiding overfitting and improving generalization capabilities through appropriate loss functions, optimizers, and learning rate decay, ultimately providing an accurate and reliable model for the task of predicting vegetation indices.

[0088] The fourth step is to establish model evaluation indicators to evaluate the vegetation index time series model:

[0089] Establish model evaluation indicators, including the coefficient of determination R 2 , root mean square error RMSE and mean absolute error MAE, each indicator is calculated by the following formulas (2)~(4):

[0090] (2)

[0091] (3)

[0092] (4)

[0093] Where: is the predicted value of vegetation index, y i is the actual value of the vegetation index, is the average of the actual values ​​of the vegetation index, and n is the number of samples.

[0094] For the three vegetation indices of EVI, NDVI and kNDVI, a vegetation index prediction model was established based on the CNN-BiLSTM-AM method. In order to evaluate the prediction effect of the model, a comparison of the modeling using the LSTM method, the CNN-BiLSTM method and the BiLSTM-AM method was performed under the same data conditions. Figure 3 to Figure 6 As shown in the figure, and the prediction accuracy of vegetation indices based on different models is shown in Table 2 below.

[0095] Table 2 Prediction accuracy of vegetation index based on different models

[0096]

[0097] Step 5: Obtain the optimized model and select the vegetation index model:

[0098] All four models above produce good prediction results, and the predicted curves fit the true curves well. However, when the vegetation index changes suddenly, the LSTM model cannot accurately and timely adjust to the actual changes in the vegetation index, resulting in large prediction errors. The CNN-BiLSTM and BiLSTM-AM models can track these changes better than the LSTM model and reflect more subtle differences. The CNN-BiLSTM-AM model can more accurately predict the changes in the vegetation index, especially in capturing small changes at mutation points.

[0099] The CNN-BiLSTM-AM model has high prediction accuracy for EVI, with RMSE=0.077, MAE=0.035, and R 2 =0.943. The CNN-BiLSTM and CNN-BilSTM-AM models have similar effects in predicting NDVI and kNDVI. The LSTM model has poor results in predicting various vegetation indices, especially in predicting kNDVI, with RMSE=0.091, MAE=0.068, and R 2 =0.909. The smaller the RMSE and MAE, the better the R 2 The closer it is to 1, the higher the accuracy of the vegetation index time series model, and the better the prediction effect of the vegetation index time series model.

[0100] The fourth and fifth steps above can also be referred to as model evaluation and optimization steps.

[0101] In summary, the CNN-BiLSTM-AM model can be established for the EVI vegetation index with the best prediction effect by combining CNN, BiLSTM and AM, which can be used to effectively capture the spatiotemporal variation characteristics of the vegetation index. That is, the CNN-BiLSTM-AM model established for the EVI vegetation index has the best prediction effect by combining CNN, BiLSTM and AM, and can be used to effectively capture the spatiotemporal variation characteristics of the vegetation index.

[0102] Exemplary Devices

[0103] like Figure 7 As shown, the vegetation index time series prediction device based on deep learning in this embodiment includes the following modules:

[0104] Data acquisition and processing module: configured to acquire and process the historical vegetation index and environmental factor data of a predetermined period;

[0105] Environmental factor screening module: configured to select environmental factors with strong correlation with vegetation index by determining the strength and direction of the linear relationship between environmental factors and vegetation index;

[0106] Prediction module: It is configured to use the CNN-BiLSTM-AM model structure including input layer, CNN layer, BiLSTM layer, AM layer and output layer as the vegetation index time series model, wherein the environmental factors with strong correlation with the vegetation index are used as input data, and NDVI, EVI and kNDVI are used as output prediction targets respectively. The CNN layer includes a convolution layer and a pooling layer using the ReLU activation function. The BiLSTM layer merges the outputs of the forward and reverse LSTMs, passes the output tensor to the AM layer, maps the weighted data processed by AM to the output layer, processes it using a linear activation function, and outputs a continuous value suitable for the regression task, i.e., the predicted vegetation index.

[0107] In addition, corresponding to the method steps of the above-mentioned embodiment, there may also be a corresponding model evaluation and optimization module, which will not be described in detail.

[0108] Exemplary Electronic Devices

[0109] Figure 8 This is the structure of an electronic device 1 provided by an exemplary embodiment of the present invention. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them. Figure 8 FIG. 1 is a block diagram of an electronic device according to an embodiment of the present disclosure. Figure 8As shown, the electronic device includes one or more processors 2 and a memory 4. The processor 2 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. The memory 4 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may, for example, include a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 2 may run the program instructions to implement the methods of the software programs of the various embodiments of the present disclosure described above and / or other desired functions.

[0110] In one example, the electronic device may further include: an input device 3 and an output device 5, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). In addition, the input device 3 may also include, for example, a keyboard, a mouse, etc. The output device 5 may output various information to the outside. The output device 5 may include, for example, a display, a speaker, a printer, and a communication network and a remote output device connected thereto, etc. Of course, for simplicity, Figure 7 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0111] Exemplary computer program products and computer-readable storage media

[0112] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0113] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0114] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0115] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0116] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0117] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0118] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0119] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0120] It should also be noted that in the apparatus, equipment and method of the present disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present disclosure. The above description of the disclosed aspects is provided to enable any technician in the field to make or use the present disclosure. Various modifications to these aspects are very obvious to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown here, but to the widest scope consistent with the principles and novel features disclosed herein.

[0121] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A vegetation index time series prediction method based on deep learning, characterized in that: The steps include: Data acquisition and processing steps: obtain and process the historical vegetation index and environmental factor data for a predetermined period; Environmental factor screening steps: by determining the strength and direction of the linear relationship between environmental factors and vegetation index, select environmental factors with strong correlation with vegetation index; Modeling and prediction steps: Construct a CNN-BiLSTM-AM model including input layer, CNN layer, BiLSTM layer, AM layer, and output layer to predict the vegetation index time series. The environmental factors with strong correlation with the vegetation index are used as input data, and NDVI, EVI, and kNDVI are used as output prediction targets respectively. The CNN layer includes a convolution layer and a pooling layer using the ReLU activation function. The BiLSTM layer merges the outputs of the forward and reverse LSTMs, passes the output tensor to the AM layer, maps the weighted data processed by AM to the output layer, processes it using a linear activation function, and outputs a continuous value suitable for the regression task, that is, the predicted vegetation index.

2. The vegetation index time series prediction method based on deep learning according to claim 1 is characterized in that: In the data acquisition and processing steps, monthly NDVI and EVI data are generated through quality control, test area and time period extraction, and maximum value synthesis method. The calculation formula kNDVI=tanh(NDVI²) is applied to convert NDVI into monthly kNDVI data. All vegetation index data and environmental factor data are unified into the geographic coordinate system, and the environmental factor data values ​​corresponding to the vegetation index samples are extracted for the modeling and prediction steps.

3. According to the vegetation index time series prediction method based on deep learning in claim 2, in the environmental factor screening step, the Pearson correlation analysis method is used to measure the strength and direction of the linear relationship between the environmental factor and the vegetation index, and the correlation coefficient r is calculated using the following formula (1): xy : （1）， Where: x i and i are vegetation index and environmental factors respectively; and are the average values ​​of vegetation index and environmental factors, n is the number of samples, The direction of the correlation is determined by the positive or negative value of the correlation coefficient, and the strength of the correlation is determined by the absolute value of the correlation coefficient. Several environmental factors with strong correlation are selected for the modeling and prediction step.

4. The vegetation index time series prediction method based on deep learning according to claim 3 is characterized in that: The CNN-BiLSTM-AM model was trained using the Keras framework and the TensorFlow framework. The feature of each time step was the factor screened in the environmental factor screening step. The vegetation index was the prediction target. The mean absolute error was selected as the loss function, the adaptive motion estimation algorithm was used as the optimizer, and the learning rate decay strategy was used with an initial learning rate of 0.01 and a learning rate decay factor of 0.

1.

5. The vegetation index time series prediction method based on deep learning according to any one of claims 1 to 4, characterized in that: The CNN layer receives data from the input layer and uses 64 convolution kernels, each of which is 1*1 in size. Each convolution operation only focuses on the data of one time step. The BiLSTM layer processes the input data in both directions of the time series simultaneously. First, it processes each time step of the input data sequentially and calculates the state of the LSTM in the forward and reverse directions respectively. The AM layer calculates the importance weight of each time step or feature and weights the input data according to these weights. First, it calculates the attention weight of each time step through a fully connected operation and converts the weight into a probability distribution through normalization to ensure that the sum of the weights of all time steps is 1. Secondly, it adjusts the dimensional order of the input tensor, swaps the time dimension with the feature dimension, and aligns the attention weight with the input data. Then, it multiplies the input data by the calculated attention weight element by element to generate weighted data. Finally, it adjusts the dimension of the attention vector in a way consistent with the dimension of the input data.

6. The vegetation index time series prediction method based on deep learning according to claim 1, characterized in that: The model evaluation and optimization step is also included: for the three vegetation indices of EVI, NDVI and kNDVI, a model evaluation index is established, and the model evaluation index includes the determination coefficient R calculated by the following formulas (2) to (4) respectively: 2 , root mean square error RMSE and mean absolute error MAE: （2）， （3）， （4）， in: is the predicted value of vegetation index, y i is the actual value of the vegetation index, is the average of the actual values ​​of the vegetation index, and n is the number of samples.

7. A vegetation index time series prediction device based on deep learning, characterized in that: Includes the following modules: Data acquisition and processing module: configured to acquire and process the historical vegetation index and environmental factor data of a predetermined period; Environmental factor screening module: configured to select environmental factors with strong correlation with vegetation index by determining the strength and direction of the linear relationship between environmental factors and vegetation index; Prediction module: It is configured to build a CNN-BiLSTM-AM model including an input layer, a CNN layer, a BiLSTM layer, an AM layer, and an output layer to predict the vegetation index time series. Among them, the environmental factors with strong correlation with the vegetation index are used as input data, and NDVI, EVI and kNDVI are used as output prediction targets respectively. The CNN layer includes a convolution layer and a pooling layer using the ReLU activation function. The BiLSTM layer merges the outputs of the forward and reverse LSTMs, passes the output tensor to the AM layer, maps the weighted data processed by AM to the output layer, processes it using a linear activation function, and outputs a continuous value suitable for the regression task, that is, the predicted vegetation index.

8. A vegetation index time series prediction device based on deep learning, characterized in that: The vegetation index time series prediction device constructs a CNN-BiLSTM-AM model including an input layer, a CNN layer, a BiLSTM layer, an AM layer, and an output layer to predict the vegetation index time series, wherein the input data of the input layer is set to multiple environmental factors including temperature, precipitation, and evapotranspiration, the output of the output layer is set to any one of NDVI, EVI, and kNDVI as the vegetation index, the CNN layer includes a convolution layer and a pooling layer using a ReLU activation function, the BiLSTM layer merges the outputs of the forward and reverse LSTMs, passes the output tensor to the AM layer, maps the weighted data processed by AM to the output layer, processes it using a linear activation function, and outputs a continuous value suitable for the regression task, that is, any one of NDVI, EVI, and kNDVI as the predicted vegetation index.

9. An electronic device, characterized in that: The electronic device comprises: a processor for executing a plurality of instructions; and a memory for storing a plurality of instructions; wherein the plurality of instructions are used to be stored in the memory and loaded and executed by the processor according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 6.

Images