An Improved Spatiotemporal Prediction Method for ENSO Based on Recursive Gated Convolution and Attention Mechanism

Abstract

This invention discloses an improved ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanisms, comprising the following steps: S1, constructing an ENSO sample dataset; S2, constructing a dense convolutional (Dense Block) module to enhance feature propagation and reduce the number of parameters; S3, constructing a MIMG-Encoder-Decoder architecture improved using an attention mechanism; S4, transposing the convolution to restore the original size of the feature map; S5, constructing and training an ENSO spatiotemporal prediction model; S6, predicting the spatiotemporal sequence. This invention effectively extracts high-dimensional spatial features from spatiotemporal sequence data and suppresses unnecessary redundant features through multi-layered dense convolutional modules. The improved MIMG module can better learn the higher-order non-stationarity of the spatiotemporal evolution of ENSO data. The proposed ENSO spatiotemporal prediction model architecture is theoretically applicable to all spatiotemporal sequences.

Description

Technical Field

[0001] This invention belongs to the field of deep learning spatiotemporal prediction, specifically involving an improved ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanisms. Background Technology

[0002] Spatiotemporal series prediction involves both temporal and spatial factors. Time refers to the sequence of events preceding and following events, while space refers to both the spatial information of the target in the image and its movement and changes, as well as the spatial information of GPS data or latitude and longitude in tabular data. ENSO (El The data for the Southern Oscillation (El Niño Southern Oscillation) prediction uses the spatial information of latitude and longitude from the latter.

[0003] Many existing spatiotemporal sequence models have shortcomings and limitations. Traditional RNN models suffer from severe gradient explosion and vanishing gradient problems and cannot handle long-range dependencies well. To alleviate these problems and further optimize RNN models, Hochreater and Schmidhuber proposed the Long Short-Term Memory (LSTM) model in 1997. By injecting gates, LSTM learns to selectively remember important information and forget secondary information, thus improving RNN model performance. To better apply LSTM models to image sequences, Dr. Shi Xingjian proposed a novel network, Convolutional LSTM, in 2015, which combines convolutional structures with LSTM. This model can learn both spatial and temporal features simultaneously. In 2016, Dr. Shi Xingjian further proposed Trajectory GRU to overcome the local invariance of convolutional structures. In 2017, Yunbo Wang et al. improved the internal structure of ConvLSTM, proposing a zigzag network, PredRNN, to effectively utilize lateral and longitudinal information. In 2018, further improvements were made by proposing PredRNN++, which features an adaptive connection between each time step and each layer, serving both long-term and short-term paths. A Gradient Highway Unit was also introduced to prevent gradient vanishing over long periods. In 2019, Yunbo Wang et al. further proposed the Memory-in-Memory (MIM) network, utilizing the difference signals between adjacent repeated states and modeling non-stationary and near-stationary characteristics in spatiotemporal dynamics through two cascaded, self-updating memory modules. Higher-order non-stationarity is potentially handled by stacking multiple MIM blocks.

[0004] Traditional encoder-decoder models compress all input sequences into a fixed-size vector, inevitably leading to the loss of significant amounts of crucial information and impacting model performance. Furthermore, treating all retained information equally is unreasonable. Attention mechanisms can be viewed as a dynamic selection of important input information, achieved through adaptive weights for features. Generally, attention mechanisms are categorized into four main types: channel attention, spatial attention, temporal attention, and branch attention. Additionally, there are two hybrid attention mechanisms: channel & spatial attention and spatial & temporal attention. Summary of the Invention

[0005] The technical problem to be solved by this invention is to propose an improved ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanism, so as to solve the long-term dependency problem of spatiotemporal sequence prediction and alleviate the impact of spatiotemporal nonstationarity of ENSO data on prediction.

[0006] To achieve the above objectives, this invention provides the following technical solution: an improved ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanisms, comprising the following steps:

[0007] S1. Collect raw ENSO data from the area to be observed, perform preprocessing, and construct an ENSO sample dataset;

[0008] S2. Construct a two-layer Dense Block network to compress the ENSO sample dataset and obtain the high-dimensional spatial features F of the spatiotemporal sequence;

[0009] S3. Input the high-dimensional spatial feature map of the spatiotemporal sequence obtained in step S2 into the attention mechanism connected to the MIMG-Encoder-Decoder architecture for adaptive feature correction, and obtain the final prediction result through the prediction network.

[0010] S4. Using the prediction result obtained in step S3 as input, the feature map is restored to its original size after passing through the transposed convolution module;

[0011] The vector sequence of T frames in the S5 and ENSO sample datasets is used as input, and the corresponding predicted sequence is used as output. The ENSO spatiotemporal prediction model is constructed and trained by combining a two-layer Dense Block network, an attention mechanism to connect the MIMG-Encoder-Decoder architecture, and a transposed convolution module. The training adopts early stopping and automatic decay of the learning rate.

[0012] Further, step S2 described above specifically involves: constructing a convolutional neural network using two layers of Dense Block modules, with X = (B, S, C, H, W) as input, where B represents the batch size, S represents the time step, C represents the number of channels, H represents the height of the input feature map, and W represents the width of the input feature map. The spatial features of each input are extracted through the Dense Block modules, and the input of each layer is the set of the outputs of all previous layers, as shown in the following equation:

[0013] X k-1 =H k ([X0,X1,…,X k-2 ])

[0014] Where X0 represents the first month. The feature values ​​extracted from the SSTA region after the first layer, and the feature X learned by this layer. k-1 It is directly passed as input to all subsequent layers, as shown in the following formula:

[0015] X k =H k ([X0,X1,…,X k-1 ])

[0016] Where, [X0,X1,…,X k-1 [] indicates the first k layers for the first month The feature set extracted by SSTA in the region, H k This represents a composite function of three consecutive operations: batch normalization, ReLU activation function, and 3×3 convolution.

[0017] Furthermore, the aforementioned step S3 includes the following sub-steps:

[0018] S301. Constructing the MIMG-Encoder-Decoder architecture: The Encoder encodes the high-dimensional spatial feature map of the spatiotemporal sequence and converts it into vector form. Then, the Decoder decodes the vector and restores it to the output sequence. In the encoding module, the output after feature extraction by the Dense Block module and attention mechanism is used as the input of ST-LSTM to generate its hidden state. Then, it enters the three-layer MIMG stacked module. The three-layer MIMG stacked module is used in the decoding module. All input sequences share the parameters of the Encoder and all output sequences share the parameters of the Decoder.

[0019] S302. Connect the Dense Block and MIMG-Encoder-Decoder architecture using an Attention module: Use the high-dimensional spatial features F of the spatiotemporal sequence as the input to the Attention module, where F∈R. C*H*WThen, it goes through a one-dimensional convolution M of the channel attention module. c ∈R C*1*1 The convolution result is multiplied by the original image, and the output of the channel attention module is used as the input of the spatial attention module to perform a two-dimensional convolution M in the spatial attention module. s ∈R 1*H*W Then multiply the output result by the original image, as shown in the following formula:

[0020] F′=M c (F)*F

[0021] F″=M S (F′)*F′.

[0022] Furthermore, the aforementioned step S302 includes the following sub-steps:

[0023] The channel attention module described in S302-1 takes the high-dimensional spatial features F of the input spatiotemporal sequence, which first passes through two parallel MaxPool layers and AvgPool layers to reduce the feature map dimension from C*H*W to C*1*1. Then, it passes through an MLP module to compress the number of channels to 1 / r of the original number, and then expands it back to the original number of channels. After passing through the ReLU activation function, two activation results are obtained. These two output results are added element-wise, and then passed through a sigmoid activation function to obtain the CAM output result. Finally, this output result is multiplied by the original image to reduce it back to the size of C*H*W, as shown in the following formula:

[0024]

[0025] Where σ is the activation function, and W0 and W1 are the two weights of the MLP. It is average pooling. It is max pooling; S302-2, spatial attention mechanism first sets the output M of channel attention to maximum pooling; c (F) Average pooling in the channel dimension and max pooling The resulting feature maps are then concatenated, and a 7×7 convolution operation f is applied to the concatenated feature maps. 7x7 Generate the final spatial attention feature map M s (F), as shown in the following formula:

[0026]

[0027] S302-3. The spatial attention mechanism output, namely the features extracted by SSTA in the Nino3.4 region in the spatial and channel dimensions, is used as the input of the ST-LSTM for the next time step.

[0028] Furthermore, the aforementioned step S301 specifically involves: the MIMG module, based on MIM, embeds the residual form rgConv of recursive gated convolution, first using... To obtain a set of projection features and As shown in the following formula:

[0029]

[0030] Wherein, the dimensions of the input features of rgConv are height × width × number of channels H × W × C, and p and q represent the projected features. This is the projection layer that performs channel blending, where k represents the layer number, C0 is the number of channels in the first order, and C... k It is the number of channels of order k+1, and n-1 is the maximum order;

[0031] Then, the gated convolution is performed recursively, as follows:

[0032] p k+1 =f k (q k )⊙g k (p k ) / α,k=0,1,…,n-1

[0033] Among them, f k G represents a convolutional layer in the depth direction. k Indicates matching dimensions in different orders; k represents the number of levels, p k q k Represents the projected features of the k-th layer;

[0034] The output p of the last recursive step n Input to projection layer Then output the result y of rgConv in the form of residuals:

[0035]

[0036] Where p0 is the original feature, p n It is the feature output after n levels of recursion.

[0037] Furthermore, the aforementioned ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanism includes two cascaded temporal memory reuse modules in MIM to replace the temporal forget gate: the non-stationary module MIM-N and the stationary module MIM-S.

[0038] The non-stationary module MIM-N uses As input, it is used to capture the difference between two consecutive hidden representations. The non-stationary changes in MIM-S, based on the differential stationarity assumption, produce different characteristics. Here, t represents the time step, l represents the number of layers, N represents the horizontal transition storage unit in the non-stationary module MIM-N, and D is the differential feature learned by MIM-N and input into MIM-S.

[0039]

[0040] The stationary module MIM-S outputs differential characteristics from the non-stationary module MIM-N. and external time memory cells As input, approximately stationary changes in the spatiotemporal sequence are captured, T is the memory through a virtual forgetting gate, and S represents the horizontal transition storage unit in the stationary module MIM-S:

[0041]

[0042] MIM replaces the forget gate in PredRNN with the final output of two cascaded modules. The non-stationary module MIM-N and the stationary module MIM-S have been updated. First, the feature propagation along the time dimension is addressed, where C represents temporal memory, W represents weights, f, i, and g represent the forget gate, input gate, and input modulation gate, respectively, and * represents convolution.

[0043]

[0044]

[0045]

[0046] Then comes the transmission of spatial feature memory streams, where M represents spatial memory:

[0047]

[0048]

[0049]

[0050]

[0051] Among them, W mg W represents the weight between the spatial memory unit and the input modulation gate. xg Wh represents the weights between the input and the input modulation gate. g W′ is the weight between the hidden layer and the input modulation gate. xf W′ represents the weight between the input and the forget gate. xi b′ represents the weight of the input gate. f It is the bias term of the forgetting gate, b gIt is the bias term of the input modulation gate, b i It is the bias term of the input gate, g t i t It is the input modulation gate and input gate at time t. It is the time memory unit of layer l at time t. All those with the superscript ' represent spatial memory flow features, while those without the superscript ' represent time dimension memory flow features.

[0052] The non-stationary module MIM-N and the stationary module MIM-S combine stationary and non-stationary changes, and then obtain the output through an activation function, as shown in the following equation.

[0053]

[0054]

[0055] Among them, W xo Indicates the output gate weight, b o W represents the bias term of the output gate. co W represents the weight between the time memory unit and the output gate. mo W represents the weight between the spatial memory unit and the output gate. ho This represents the weights between the hidden layer and the output gate. This represents the time memory unit of layer l at time t. o represents the spatial memory unit of layer l at time t. t This represents the output at time t;

[0056] Hidden layer state of the current layer after MIM The projected features p0 are input into rgConv to obtain the output of the hidden state of the current layer.

[0057]

[0058] Further, step S4 described above specifically involves: inputting the output of the MIMG-Encoder-Decoder into a reconstruction network consisting of three deconvolution layers, and reconstructing the extrapolated sequence features into the size of the original sequence; each deconvolution reconstruction network includes a deconvolution layer, a normalization layer, and a LeakyReLU activation function layer; the parameters of the first deconvolution layer are set to a kernel size of (3, 3), 128 channels, and a stride of 2; the parameters of the second deconvolution layer are set to a kernel size of (2, 2), 64 channels, a stride of 2, and padding of 1; the parameters of the third deconvolution layer are set to a kernel size of (3, 3), 1 channel, a stride of 1, and padding of 1.

[0059] Furthermore, the aforementioned step S5 specifically includes:

[0060] The region where the observed features are located The observation of this region at any given time point is represented by a vector. express,

[0061] T-frame data is represented by a fixed-size spatial region of M×N grid points. Each grid point contains C measurements that vary over time. The observations over T time steps are represented by a vector sequence X1, X2, ..., X... T These T frames of data represent the spatiotemporal sequence, which extrapolates future K frames of data given the previous J frames of data. The ENSO spatiotemporal prediction model is constructed as follows:

[0062]

[0063] The predicted sequence can only be obtained after t-level state updates.

[0064] When extrapolating future K-frame data from the spatiotemporal sequence, the loss function is defined as follows:

[0065]

[0066] Where N represents the number of training samples, σ,i∈[1,N],o i Let y represent the predicted output for the i-th sample. i This represents the true label of the i-th sample;

[0067] Simultaneously, cross-validation is used as a training and validation strategy, and the mean squared error (MSE) and mean absolute error (MAE) are selected as the criteria for the detection model score. The formulas for MSE and MAE are as follows:

[0068]

[0069]

[0070] The output y has two subscripts: label and test, which represent the output of the sample label and the test sample, respectively. In the training strategy of this invention, the Adam algorithm is used to optimize the gradient.

[0071] Furthermore, in step S5 mentioned above, the use of early stopping and setting automatic learning rate decay during training specifically involves:

[0072] When the model's error on the validation set is worse than the previous training result, the model parameters from the previous iteration are saved, and the counter is incremented by 1. Training stops when the counter equals the probability. (Preset E)opt If E(t) is the validation set error obtained at iteration number t, then E opt (t)=min t′≤t E va (t′), the growth rate of the generalization error, is given by the following formula:

[0073]

[0074] During network training, an automatic learning rate decay strategy was employed to accelerate the training process, as shown in the following formula:

[0075]

[0076] Where decay_rate is the initial coefficient, epoch i Let α0 represent the i-th training iteration, where α0 is the initial learning rate.

[0077] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention provides... The index's all-season correlation technique improves upon classical convolutional neural networks by 16% and reduces mean squared error by 17%, providing effective forecasts with lead times up to 23 months. It compensates for the scarcity of climate data by using both model simulation and reanalysis data as a dataset. Attached Figure Description

[0078] Figure 1 This is a flowchart of the method of the present invention.

[0079] Figure 2 This is a structural diagram of a Dense Block.

[0080] Figure 3 This is a diagram of the MIMG structure.

[0081] Figure 4 This is the rgConv structure diagram.

[0082] Figure 5 This is the overall architecture diagram of the model.

[0083] Figure 6 This is a structural diagram of the CBAM module.

[0084] Figure 7 This is the overall flowchart.

[0085] Figure 8 This is a screenshot of the Early Stopping effect.

[0086] Figure 9 This is a flowchart of sequence prediction. Detailed Implementation

[0087] To better understand the technical content of the present invention, specific embodiments are described below in conjunction with the accompanying drawings.

[0088] In this invention, various aspects of the invention are described with reference to the accompanying drawings, in which numerous illustrative embodiments are shown. Embodiments of the invention are not limited to those depicted in the drawings. It should be understood that the invention is implemented through any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed herein are not limited to any particular implementation. Furthermore, some aspects of the invention disclosed may be used alone or in any suitable combination with other aspects of the invention disclosed.

[0089] like Figure 1 As shown, the ENSO spatiotemporal prediction method improved by recursive gated convolution and attention mechanism includes the following steps:

[0090] S1. Collect raw ENSO data from the area to be observed, perform preprocessing, and construct an ENSO sample dataset;

[0091] S2. Construct a two-layer Dense Block network to compress the ENSO sample dataset and obtain the high-dimensional spatial features F of the spatiotemporal sequence;

[0092] S3. Input the high-dimensional spatial feature map of the spatiotemporal sequence obtained in step S2 into the attention mechanism connected to the MIMG-Encoder-Decoder architecture for adaptive feature correction, and obtain the final prediction result through the prediction network.

[0093] S4. Using the prediction result obtained in step S3 as input, the feature map is restored to its original size after passing through the transposed convolution module;

[0094] The vector sequence of T frames in the S5 and ENSO sample datasets is used as input, and the corresponding predicted sequence is used as output. The ENSO spatiotemporal prediction model is constructed and trained by combining a two-layer Dense Block network, an attention mechanism to connect the MIMG-Encoder-Decoder architecture, and a transposed convolution module. The training adopts early stopping and automatic decay of the learning rate.

[0095] like Figure 2 As shown in the diagram, the Dense Block structure is as follows: A convolutional neural network is constructed using two Dense Block modules, with X = (B, S, C, H, W) as input, where B represents the batch size, S represents the time step, C represents the number of channels, H represents the height of the input feature map, and W represents the width of the input feature map. The spatial features of each input are extracted through the Dense Block modules. The input of each layer is the set of the outputs of all previous layers, as shown in the following equation:

[0096] X k-1 =H k ([X0,X1,…,X k-2 ])

[0097] Where X0 represents the first month. The SSTA of the region is the feature value extracted after the first layer, while the feature X learned by this layer is... k-1 It is directly passed as input to all subsequent layers, as shown in the following formula:

[0098] X k =H k ([X0,X1,…,X k-1 ])

[0099] Where, [X0,X1,…,X k-1 [] indicates the first k layers for the first month The feature set extracted by SSTA in the region, H k This represents a composite function of three consecutive operations: batch normalization, ReLU activation function, and 3×3 convolution.

[0100] This tight connection exists only within a single Dense Block; there are no tight connections between different Dense Blocks. The output of this module is... Features extracted from the SSTA of a region in the spatial dimension.

[0101] like Figure 5 The diagram shows the overall architecture of the model. First, a Dense Block module and an attention mechanism are used to extract high-dimensional features. Then, a MIMG-Encoder-Decoder architecture is constructed. This Encoder-Decoder structure consists of two parts: the Encoder encodes the high-dimensional spatial feature map of the spatiotemporal sequence, converting it into a vector form; then the Decoder decodes this vector to restore the output sequence. In this invention, both the encoder and decoder use the MIMG algorithm. In the encoding module, since the first layer has no previous layers, we simply use the output after feature extraction by the Dense Block module and attention mechanism in the encoding module as the input to the ST-LSTM to generate its hidden state. Then, it enters a three-layer MIMG stacked module. The decoding module also uses a three-layer MIMG stacked module, where all input sequences share the Encoder's parameters and all output sequences share the Decoder's parameters.

[0102] like Figure 6The diagram shows the CBAM module structure. The CBAM attention mechanism used in this invention is a type of channel & spatial attention. Given a feature map, the CBAM module can sequentially generate attention feature map information in both channel and spatial dimensions. Then, the information from these two feature maps is multiplied with the original input feature map to perform adaptive feature correction, producing the final feature map. CBAM is a lightweight module that can be embedded into any backbone network to improve performance. This invention embeds the CBAM module between the Dense Block and the cell list, thereby focusing on the important features extracted by the Dense Block and suppressing unnecessary noise. The Attention module is used first, connecting to the MIMG-Encoder-Decoder architecture; the high-dimensional spatial features F of the spatiotemporal sequence are used as the input to the Attention module, where F∈R. C*H*W Then, it goes through a one-dimensional convolution M of the channel attention module. c ∈R C*1*1 The convolution result is multiplied by the original image, and the output of the channel attention module is used as the input of the spatial attention module to perform a two-dimensional convolution M in the spatial attention module. s ∈R 1*H*W Then multiply the output result by the original image, as shown in the following formula:

[0103] F′=M c (F)*F

[0104] F″=M S (F′)*F′.

[0105] Specifically, the steps S302-1 to S302-3 are as follows:

[0106] S302-1, the channel attention module treats each channel of the feature map as a feature detector. Therefore, the channel features focus on the useful information "what" in the feature map. To calculate the channel attention features more efficiently, the spatial dimension of the feature map needs to be compressed. Max pooling is used here. Since CBAM is a four-dimensional input, the channel dimension remains unchanged, and the spatial dimension is compressed. This module focuses on what the useful information in the input feature map is. Therefore, this invention merges the sample size B and the sequence length S into a single dimension B*S, which facilitates the extraction of channel attention features by CBAM in the second channel dimension C. The high-dimensional spatial features F of the input spatiotemporal sequence first pass through two parallel MaxPool and AvgPool layers, reducing the feature map dimension from C*H*W to C*1*1. Then, it passes through an MLP module, compressing the number of channels to 1 / r of the original number, and then expanding it back to the original number of channels. After passing through a ReLU activation function, two activation results are obtained. These two outputs are then added element-wise, and then passed through a sigmoid activation function to obtain the CAM output. Finally, this output is multiplied by the original image to restore it to C*H*W size, as shown in the following equation:

[0107]

[0108] Where σ is the activation function, and W0 and W1 are the two weights of the MLP. It is average pooling. It's max pooling; S302-2, the spatial attention mechanism focuses on "where" the effective information on the feature map is. To compute spatial attention, first, the input data F (that is, the output M of the channel attention) is processed. c (F) Average pooling in the channel dimension and max pooling The resulting feature maps are then concatenated, and a convolution operation is performed on the concatenated feature maps to generate the final spatial attention feature map M. s (F), as shown in the following formula:

[0109]

[0110] S302-3. The spatial attention mechanism output, namely the features extracted by SSTA in the Nino3.4 region in the spatial and channel dimensions, is used as the input of the ST-LSTM for the next time step.

[0111] like Figure 3 The diagram shows the MIMG structure. Figure 4 The rgConv structure diagram. The MIMG module, based on MIM, embeds a residual form of recursive gated convolution, rgConv. The input features of rgConv have dimensions of height × width × number of channels H × W × C, where p and q represent projected features. It is the projection layer that performs channel blending, where k represents the layer number and f k G represents a convolutional layer in the depth direction. k This indicates matching dimensions in different orders; first use... To obtain a set of projection features and As shown in the following formula:

[0112]

[0113] Wherein, the dimensions of the input features of rgConv are height × width × number of channels H × W × C, and p and q represent the projected features. This is the projection layer that performs channel blending, where k represents the layer number, C0 is the number of channels in the first order, and C... k It is the number of channels of order k+1, and n-1 is the maximum order.

[0114] Then, the gated convolution is performed recursively, as follows:

[0115] p k+1 =f k (q k )⊙g k (p k ) / α,k=0,1,…,n-1

[0116] Among them, f k G represents a convolutional layer in the depth direction. k Indicates matching dimensions in different orders; k represents the number of levels, p k q k This represents the projection feature of the k-th layer.

[0117] The output p of the last recursive step n Input to projection layer Then output the result y of rgConv in the form of residuals:

[0118]

[0119] Where p0 is the original feature, p n It is the feature output after n levels of recursion.

[0120] MIM includes two cascaded time memory multiplexing modules to replace the time forgetting gate: the non-stationary module MIM-N and the stationary module MIM-S;

[0121] The non-stationary module MIM-N uses As input, it is used to capture the difference between two consecutive hidden representations. The non-stationary changes in MIM-S, based on the differential stationarity assumption, produce different characteristics. Here, t represents the time step, l represents the number of layers, N represents the horizontal transition storage unit in the non-stationary module MIM-N, and D is the differential feature learned by MIM-N and input into MIM-S.

[0122]

[0123] The stationary module MIM-S outputs differential characteristics from the non-stationary module MIM-N. and external time memory cells As input, approximately stationary changes in the spatiotemporal sequence are captured, T is the memory through a virtual forgetting gate, and S represents the horizontal transition storage unit in the stationary module MIM-S:

[0124]

[0125] MIM replaces the forget gate in PredRNN with the final output of two cascaded modules. The non-stationary module MIM-N and the stationary module MIM-S have been updated. First, the feature propagation along the time dimension is addressed, where C represents temporal memory, W represents weights, f, i, and g represent the forget gate, input gate, and input modulation gate, respectively, and * represents convolution.

[0126]

[0127]

[0128]

[0129] Then comes the transmission of spatial feature memory streams, where M represents spatial memory:

[0130]

[0131]

[0132]

[0133]

[0134] Among them, W mg W represents the weight between the spatial memory unit and the input modulation gate. xg Wh represents the weights between the input and the input modulation gate. g W′ is the weight between the hidden layer and the input modulation gate. xf W′ represents the weight between the input and the forget gate. xi b′ represents the weight of the input gate. f It is the bias term of the forgetting gate, b gIt is the bias term of the input modulation gate, b i It is the bias term of the input gate, g t i t It is the input modulation gate and input gate at time t. It is the time memory unit of layer l at time t. All those with the superscript ' represent spatial memory flow features, while those without the superscript ' represent time dimension memory flow features.

[0135] The non-stationary module MIM-N and the stationary module MIM-S combine stationary and non-stationary changes, and then obtain the output through an activation function, as shown in the following equation.

[0136]

[0137]

[0138] Among them, W xo Indicates the output gate weight, b o W represents the bias term of the output gate. co W represents the weight between the time memory unit and the output gate. mo W represents the weight between the spatial memory unit and the output gate. ho This represents the weights between the hidden layer and the output gate. This represents the time memory unit of layer l at time t. o represents the spatial memory unit of layer l at time t. t This represents the output at time t;

[0139] Hidden layer state of the current layer after MIM The projected features p0 are input into rgConv to obtain the output of the hidden state of the current layer.

[0140]

[0141] like Figure 7 The overall flowchart is shown below. The vector sequence of T frames in the ENSO sample dataset serves as input, and its corresponding predicted sequence is output. This is combined with a two-layer Dense Block network and an attention mechanism.

[0142] The MIMG-Encoder-Decoder architecture and the transposed convolution module are used to build and train the ENSO spatiotemporal prediction model, and early stopping and automatic decay of the learning rate are set during training.

[0143] like Figure 8 As shown, early stopping is used to prevent overfitting in the model optimization problem, and the learning rate is set to automatically decay, which speeds up the model optimization to some extent.

[0144] When training deep learning neural networks, we typically aim for the best generalization performance, meaning a good fit to the data. However, all standard deep learning neural network architectures are prone to overfitting. This means that while the network performs better and better on the training set with a lower error rate, its performance on the test set may eventually begin to deteriorate. To improve the model's generalization ability and avoid overfitting, achieving a good balance, this invention introduces the concept of early stopping. Early stopping is a widely used method that outperforms regularization in many cases. See the diagram for the effect of early stopping. Figure 8 The main steps are as follows: Divide the original training dataset into a training set and a validation set. Train only on the training set. Calculate the model's error on the validation set in each epoch. Pre-set a patience (patience ≤ epoch), which represents the maximum number of epochs tolerable for generalization error. When the model's error on the validation set is worse than the previous training result, save the model parameters from the previous iteration and increment the counter. When the counter equals the patience, stop training. Assume E opt If E(t) is the validation set error obtained at iteration number t, then E opt (t)=min t′≤t E va (t′), the growth rate of the generalization error:

[0145]

[0146] In the subsequent network training process, an automatic learning rate decay strategy was adopted to accelerate the training speed. A large learning rate can achieve very fast convergence, so a large value is used initially to avoid getting trapped in local minima. However, as training progresses, an excessively large learning rate may cause the gradient to oscillate around the global minimum, resulting in inconsistent gradients near the convergence point. Therefore, a smaller learning rate can reduce the convergence pace and avoid inconsistent results.

[0147]

[0148] The learning rate will decrease slightly each time the number of training epochs reaches a set value. Here, `decay_rate` is the initial coefficient, and `epoch` is the number of epochs. i Let α0 represent the i-th training iteration, where α0 is the initial learning rate.

[0149] like Figure 9 Sequence prediction flowchart. The region of the feature to be observed... The observation of this region at any given time point is represented by a vector. express,

[0150] T-frame data is represented by a fixed-size spatial region of M×N grid points. Each grid point contains C measurements that vary over time. The observations over T time steps are represented by a vector sequence X1, X2, ..., X... T These T frames of data represent the spatiotemporal sequence, which extrapolates future K frames of data given the previous J frames of data. The ENSO spatiotemporal prediction model is constructed as follows:

[0151]

[0152] The predicted sequence can only be obtained after t-level state updates.

[0153] When extrapolating future K-frame data from the spatiotemporal sequence, the loss function is defined as follows:

[0154]

[0155] Where N represents the number of training samples, σ,i∈[1,N],o i Let y represent the predicted output for the i-th sample. i This represents the true label of the i-th sample.

[0156] Simultaneously, cross-validation is used as a training and validation strategy, and the mean squared error (MSE) and mean absolute error (MAE) are selected as the criteria for the detection model score. The formulas for MSE and MAE are as follows:

[0157]

[0158]

[0159] The subscript of the output y has three representations: train, label, and test, which represent the output of the training process, the sample label, and the test output, respectively. In the training strategy of this invention, the Adam algorithm is used to optimize the gradient.

[0160] While the present invention has been described above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. An improved ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanism, characterized in that, Includes the following steps: S1. Collect raw ENSO data from the area to be observed, perform preprocessing, and construct an ENSO sample dataset; S2. Construct a two-layer Dense Block network to compress the ENSO sample dataset and obtain high-dimensional spatial features of the spatiotemporal sequences. ; S3. Input the high-dimensional spatial feature map of the spatiotemporal sequence obtained in step S2 into the attention mechanism connected to the MIMG-Encoder-Decoder architecture for adaptive feature correction, and obtain the final prediction result through the prediction network; including the following sub-steps: S301. Constructing the MIMG-Encoder-Decoder architecture: The Encoder encodes the high-dimensional spatial feature map of the spatiotemporal sequence and converts it into vector form. Then, the Decoder decodes the vector and restores it to the output sequence. In the encoding module, the output after feature extraction by the Dense Block module and attention mechanism is used as the input of ST-LSTM to generate its hidden state. Then, it enters the three-layer MIMG stacked module. The three-layer MIMG stacked module is used in the decoding module. All input sequences share the parameters of the Encoder and all output sequences share the parameters of the Decoder. The MIMG stacking module embeds the residual form rgConv of recursive gated convolution on top of MIM. First, it uses... To obtain a set of projection features As shown in the following formula: ， Among them, the dimension of the input features of rgConv is high. Width The number of channels is H×W×C, and p and q represent the projection features. It is a projection layer that performs channel blending. Indicates the number of floors. It is the number of channels of the first order. It is the number of channels at order k+1. It is the highest order; Then, the gated convolution is performed recursively, as follows: ， in, Represents a convolutional layer in the depth direction. Indicates matching dimensions in different orders; Indicates the number of floors. Represents the projected features of the k-th layer; The output of the last recursive step Input to projection layer Then output the result of rgConv in the form of residuals. : ， in, It is a primitive feature. It is the feature output after n levels of recursion; S302. Connect the Dense Block and the MIMG-Encoder-Decoder architecture using the Attention module: This integrates the high-dimensional spatial features of the spatiotemporal sequence. As input to the Attention module, Then it goes through a one-dimensional convolution of the channel attention module. The convolution result is multiplied by the original image, and the output of the channel attention module is used as the input of the spatial attention module to perform a two-dimensional convolution of the spatial attention module. Then multiply the output result by the original image, as shown in the following formula: ， ； S4. Using the prediction result obtained in step S3 as input, the feature map is restored to its original size after passing through the transposed convolution module; The vector sequence of T frames in the S5 and ENSO sample datasets is used as input, and the corresponding predicted sequence is used as output. The ENSO spatiotemporal prediction model is constructed and trained by combining a two-layer Dense Block network, an attention mechanism to connect the MIMG-Encoder-Decoder architecture, and a transposed convolution module. The training adopts early stopping and automatic decay of the learning rate.

2. The ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanism as described in claim 1, characterized in that, Step S2 specifically involves constructing a convolutional neural network using two layers of Dense Block modules. For input, where, Indicates batch size. Indicates a time step. Indicates the number of channels. Indicates the height of the input feature map, This represents the width of the input feature map. The spatial features of each input are extracted through the Dense Block module. The input of each layer is the set of the outputs of all previous layers, as shown in the following formula: ， in, This represents the feature values ​​extracted by the SSTA in the Niño3.4 region of month 1 after passing through the first layer; the features learned by this layer. It is directly passed as input to all subsequent layers, as shown in the following formula: , in, This represents the feature set extracted by the first k layers from the SSTA of the Niño3.4 region in month 1. This represents three consecutive operations: batch normalization, ReLU activation function, and 3...

3. The composite function of convolution.

3. The ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanism as described in claim 2, characterized in that, Step S302 includes the following sub-steps: The channel attention module described in S302-1 receives high-dimensional spatial features of the spatiotemporal sequence as input. First, the feature map is processed through two parallel MaxPool and AvgPool layers, reducing its dimension from C*H*W to C*1*1. Then, it passes through an MLP module, compressing the number of channels to 1 / r of the original number, and then expanding it back to the original number of channels. A ReLU activation function is then applied to obtain two activated results. These two outputs are added element-wise, and then passed through a sigmoid activation function to obtain the CAM output. Finally, this output is multiplied by the original image to restore it to C*H*W, as shown in the following formula: ； in, It is an activation function. , These are the two weights of the MLP. It is average pooling. It is max pooling; S302-2, Spatial Attention Mechanism: First, the output of channel attention... Average pooling at the channel dimension and max pooling Then, the resulting feature maps are concatenated, and a filter with a size of [missing value] is applied to the concatenated feature maps. Convolution operation Generate the final spatial attention feature map As shown in the following formula: ， S302-3. The spatial attention mechanism output, namely the features extracted by SSTA in the Nino3.4 region in the spatial and channel dimensions, is used as the input of the ST-LSTM for the next time step.

4. The ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanism as described in claim 3, characterized in that, MIM includes two cascaded time memory multiplexing modules to replace the time forgetting gate: the non-stationary module MIM-N and the stationary module MIM-S; The non-stationary module MIM-N uses As input, it is used to capture the difference between two consecutive hidden representations. The non-stationary changes in these variables, based on the difference stationarity assumption, produce different characteristics. Indicates the time step. Indicates the number of floors. This refers to the horizontal transition memory cell in the non-stationary module MIM-N. It is the difference features learned by MIM-N and input into MIM-S: ， The stationary module MIM-S outputs differential characteristics from the non-stationary module MIM-N. and external time memory cells As input, it captures approximately stationary changes in spatiotemporal sequences. It is through the memory of the virtual forget gate. This refers to the horizontal transition memory cell in the smooth module MIM-S: ， MIM replaces the forget gate in PredRNN with the final output of two cascaded modules. The non-stationary module MIM-N and the stationary module MIM-S were updated, starting with feature propagation of the time-dimensional stream. Represents time memory, Indicates weight, These represent the forget gate, input gate, and input modulation gate, respectively. Representing convolution: ， ， ， Then comes the transmission of spatial feature memory streams. Spatial memory representation: ， ， ， ， in, This represents the weights between the spatial memory unit and the input modulation gate. This represents the weights between the input and the input modulation gate. These are the weights between the hidden layer and the input modulation gate. This represents the weights between the input and the forget gate. Indicates the weights of the input gates. It is the bias term of the forgetting gate. It is the bias term of the input modulation gate. It is the bias term of the input gate. It is the input modulation gate and input gate at time t. It is time t The time memory unit of the layer, all with superscript All represent spatial feature memory stream features, without superscripts. The feature of memory flow is time-dimensional; The non-stationary module MIM-N and the stationary module MIM-S combine stationary and non-stationary changes, and then obtain the output through an activation function, as shown in the following equation. ， ， in, Indicates the output gate weights. This represents the bias term of the output gate. This represents the weight between the time memory unit and the output gate. This represents the weight between the spatial memory unit and the output gate. This represents the weights between the hidden layer and the output gate. Represents time t Layered time memory units, Represents time t The spatial memory unit of the layer, This represents the output at time t; Hidden layer state of the current layer after MIM With projection features The input in the form of rgConv is used to obtain the output of the hidden state of the current layer. : , 。 5. The ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanism as described in claim 1, characterized in that, Step S4 specifically involves: inputting the output of the MIMG-Encoder-Decoder into a reconstruction network consisting of three deconvolution layers, reconstructing the extrapolated sequence features into the size of the original sequence; each deconvolution reconstruction network contains a deconvolution layer, a normalization layer, and a LeakyReLU activation function layer; the parameters of the first deconvolution layer are set to a kernel size of (3,3), 128 channels, and a stride of 2; the parameters of the second deconvolution layer are set to a kernel size of (2,2), 64 channels, a stride of 2, and padding of 1; the parameters of the third deconvolution layer are set to a kernel size of (3,3), 1 channel, a stride of 1, and padding of 1.

6. The ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanism as described in claim 1, characterized in that, Step S5 is as follows: Regions for observation The observation of this region at any point in time is represented by a vector. express, T-frame data is represented by a fixed-size spatial region in the spatial dimension, consisting of M × N grid points. Each grid point contains data that changes over time. A measurement, observed over T time steps, using a vector sequence. These T frames of data represent the spatiotemporal sequence, which extrapolates future K frames of data given the previous J frames of data. The ENSO spatiotemporal prediction model is constructed as follows: ， go through The predicted sequence can only be obtained by updating the layer state. ; When extrapolating future K-frame data from the spatiotemporal sequence, the loss function is defined as follows: ， in, Represents the number of training samples. , This represents the predicted output for the i-th sample. This represents the true label of the i-th sample; Simultaneously, cross-validation is used as a training and validation strategy, and the mean squared error (MSE) and mean absolute error (MAE) are selected as the criteria for the detection model score. The formulas for MSE and MAE are as follows: ， ； The output y has two subscripts: label and test, which represent the output of the sample label and the test sample, respectively. In the training strategy of this invention, the Adam algorithm is used to optimize the gradient.

7. The ENSO spatiotemporal prediction method based on recursive gated convolution and attention mechanism as described in claim 6, characterized in that, In step S5, the early stopping method and automatic learning rate decay are implemented during training as follows: When the model's error on the validation set is worse than the previous training result, the model parameters from the previous iteration are saved, and the counter is incremented by 1. Training stops when the counter equals the probability. If the best validation set error is obtained at iteration number t, then... The growth rate of the generalization error is as follows: ， During network training, an automatic learning rate decay strategy was employed to accelerate the training process, as shown in the following formula: ， in, These are the initial coefficients. Indicates the first This training session It is the initial learning rate.

Images