ENSO long-term prediction method and device based on multi-head space-time attention mechanism, and medium

Abstract

The invention relates to an ENSO long-term prediction method and device based on a multi-head space-time attention mechanism and a medium, and the method comprises the following steps: S1, integrating multi-modal meteorological data, and constructing three-dimensional field data; s2, preprocessing the three-dimensional field data to generate a standardized data set; s3, performing block processing on the standardized data set in a spatial dimension through a Patch function to generate multi-scale spatio-temporal features; s4, mapping the spatio-temporal features to a high-dimensional space through a spatio-temporal embedding mechanism to obtain embedded features; s5, the embedded features are input into a trained ENSO long-term prediction model based on the multi-head space-time attention mechanism, an ENSO prediction result is obtained, and in the model, a Softmax function in the multi-head space-time attention mechanism is replaced with a linear mapping function. Compared with the prior art, the method has the advantages that ENSO long-term prediction can be accurately and efficiently carried out, so that the influence caused by climate disasters is reduced, and more scientific and accurate support is provided for emergency response and disaster prevention.

Description

Technical Field

[0001] The present invention belongs to the technical field of climate prediction, and in particular relates to a method, device and medium for long-term prediction of ENSO based on a multi-head spatiotemporal attention mechanism. Background Art

[0002] ENSO (El Nino-Southern Oscillation) is one of the important factors that cause extreme weather such as floods and droughts. Therefore, improving the accuracy of ENSO prediction is crucial to reducing the impact of climate disasters. Due to the complex composition of ocean data and the difficulty in obtaining it, traditional ENSO prediction methods often only focus on the individual effects of ocean or atmospheric variables and make predictions based on a single variable. This is particularly inadequate when dealing with the interactions between different meteorological variables, resulting in low prediction accuracy, especially in long-term predictions. The prediction error is large and cannot provide a reliable basis for long-term climate warnings.

[0003] The Chinese patent application with publication number CN117688978A discloses a spatiotemporal prediction method for ENSO events that integrates multi-source data. It uses a multi-source data fusion method to combine ocean data and atmospheric data, including sea surface temperature, ocean heat content, radial wind anomalies, and zonal wind anomalies, into multi-dimensional observation data to improve the prediction duration and accuracy of ENSO events. However, this method uses the convolutional layer and attention layer of the convolutional long short-term memory neural network to extract the spatial features of the expanded observation data. The model's ability to extract multi-scale features and computational efficiency need to be further improved. Therefore, it is necessary to design a new ENSO prediction method that can accurately and efficiently perform long-term ENSO predictions to reduce the impact of climate disasters and provide more scientific and accurate support for emergency response and disaster prevention. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a long-term ENSO prediction method, device and medium based on a multi-head spatiotemporal attention mechanism, which can accurately and efficiently perform long-term ENSO prediction to reduce the impact of climate disasters and provide more scientific and accurate support for emergency response and disaster prevention.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] The present invention provides an ENSO long-term prediction method based on a multi-head spatiotemporal attention mechanism, comprising the following steps:

[0007] S1. Obtain sea surface temperature, sea surface wind stress and upper ocean temperature at different depths to construct three-dimensional field data D input ;

[0008] S2, the three-dimensional field data Dinput Preprocessing is performed to remove long-term trends and seasonal changes and generate a standardized data set D norm ;

[0009] S3, patch the standardized data set D norm Perform block processing on the spatial dimension to generate multi-scale spatiotemporal features F patch ;

[0010] S4, the spatiotemporal feature F is embedded in the spatiotemporal feature F through the spatiotemporal embedding mechanism patch Map to high-dimensional space and obtain embedded features F embed , the spatiotemporal embedding mechanism includes time embedding, space embedding and linear embedding;

[0011] S5, embedding the feature F embed Input the trained ENSO long-term prediction model based on multi-head spatiotemporal attention mechanism to obtain ENSO prediction results;

[0012] The ENSO long-term prediction model based on the multi-head spatiotemporal attention mechanism includes an encoder module, a multi-scale feature fusion module and a decoder module connected in sequence. The encoder module is constructed based on the improved multi-head spatiotemporal attention mechanism and is used to extract features from the input data. The improved multi-head spatiotemporal attention mechanism converts the Softmax in the multi-head spatiotemporal attention mechanism into a multi-scale feature fusion module. x The function is replaced by a linear mapping function; the multi-scale feature fusion module is used to fuse the features of different scales output by the encoder module; the decoder module includes multiple decoders, each decoder is used to generate an intermediate prediction result according to the prediction features output by itself and the output of the multi-scale feature fusion module, and the outputs of all decoders are weighted summed to obtain the final prediction result.

[0013] Furthermore, in step S3, the spatiotemporal feature F is generated by the Patch function. patch The specific formula is as follows:

[0014] F patch =Patch(D norm , M scale )

[0015] Among them, M scale Represents a multi-scale partitioning operation.

[0016] Furthermore, in the spatiotemporal embedding mechanism, the specific process of the time embedding is: injecting position information into each time point, and the specific formula is as follows:

[0017]

[0018]

[0019] Among them, PE represents position encoding, d model is the dimension of the embedding vector, pos represents the position of the time point in the sequence, and i represents half of the embedding dimension;

[0020] The specific process of the spatial embedding is: mapping each spatial position to a high-dimensional space through an embedding layer;

[0021] The specific process of linear embedding is: patch Perform compression;

[0022] The embedding feature F obtained by the spatiotemporal embedding mechanism embed The specific expression is as follows:

[0023] F embed =E temporal (F patch )+E spatial (F patch )+E linear (F patch )

[0024] Among them, E temporal (*) is the time embedding function, E spatial (*) is the spatial embedding function, E linear (*) is the linear embedding function.

[0025] Furthermore, the expression of the improved multi-head spatiotemporal attention mechanism is as follows:

[0026] Z=Concat(head 1 , head 2 ,...head h )W O

[0027]

[0028] Among them, Z is the output of the improved multi-head spatiotemporal attention mechanism, Concat(*) represents the feature concatenation operation, and W O is the trainable weight, h represents the number of attention heads, Q i , K i and V i Respectively represent the i-th attention head i The query matrix, key matrix and value matrix of It is a linear mapping function, specifically a Linear Attention function.

[0029] Furthermore, the encoder module includes an improved multi-head spatiotemporal attention mechanism and a feedforward network connected in sequence, and a normalization layer is respectively arranged before and after the feedforward network.

[0030] Furthermore, the output of the feedforward network is F feedforward , the outputs of the normalized layers before and after the feedforward network are Z norm and F scale , the expressions of the above outputs are as follows:

[0031] Z norm =Norm(Z+F embed )

[0032] F feedforward =ReLU(W 1 Z norm +b 1 )

[0033] F scale =Norm(F feedforward +Z norm )

[0034] Among them, Z is the output of the improved multi-head spatiotemporal attention mechanism, Norm(*) represents the normalization operation, ReLU(*) represents the activation function, and W 1 and b 1 are the weight and bias of the network respectively.

[0035] Furthermore, in the decoder module, each decoder includes a multi-head attention mechanism, a normalization layer, a feedforward network layer and a linear layer connected in sequence.

[0036] Furthermore, the ENSO long-term prediction model based on the multi-head spatiotemporal attention mechanism is trained by a back-propagation algorithm and a cyclic training method.

[0037] The present invention also provides an electronic device, comprising a memory, a processor, and a program stored in the memory, wherein the processor implements the above method when executing the program.

[0038] The present invention also provides a computer-readable storage medium on which a computer program is stored, and the program implements the above method when executed by a processor.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. This paper proposes a long-term ENSO prediction method based on a multi-head spatiotemporal attention mechanism. First, a standardized dataset D is constructed based on sea surface temperature, sea surface wind stress, and upper ocean temperature at different depths. norm; Then, the standardized dataset D is patched using the Patch function norm Perform block processing on the spatial dimension to generate multi-scale spatiotemporal features F patch , and map it to a high-dimensional space through the spatiotemporal embedding mechanism to obtain the embedded feature F embed The multi-scale Patch partitioning mechanism can capture the variation characteristics of meteorological variables in different spatial ranges. Especially when dealing with climate phenomena at large and local scales, it can significantly improve the accuracy of spatiotemporal feature extraction and is suitable for capturing the regional impact of ENSO events. Finally, the embedded feature F embed Input the trained ENSO long-term prediction model based on the multi-head spatiotemporal attention mechanism to obtain the ENSO prediction results. The ENSO long-term prediction model based on the multi-head spatiotemporal attention mechanism adopts an encoder-decoder architecture. The encoder module is constructed based on the improved multi-head spatiotemporal attention mechanism. The specific improvement is to convert the Softmap in the multi-head spatiotemporal attention mechanism into a decoder. x The function is replaced by a linear mapping function, which can simultaneously capture the long-range time dependence and complex spatial interaction of meteorological data, process multi-dimensional spatiotemporal characteristics in parallel, greatly reduce the computational complexity, and improve the efficiency and accuracy of the model in long-term data processing, especially showing better robustness in the long-term prediction of ENSO.

[0041] 2. In the process of model training and optimization, the present invention adopts the back propagation method to perform feedback correction on the model to improve the prediction accuracy and reduce the error, thereby improving the long-term prediction accuracy of ENSO events and reducing the prediction error. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flowchart of the ENSO long-term prediction method based on the multi-head spatiotemporal attention mechanism;

[0043] Figure 2 It is a schematic diagram of the structure of the ENSO long-term prediction model based on the multi-head spatiotemporal attention mechanism;

[0044] Figure 3 It is a structural diagram of the encoder;

[0045] Figure 4 A schematic diagram of the decoder structure. DETAILED DESCRIPTION

[0046] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0047] Example:

[0048] This embodiment provides an ENSO long-term prediction method based on a multi-head spatiotemporal attention mechanism, which aims to solve the problem that the existing technology cannot effectively integrate multivariate data in long-term prediction, resulting in insufficient ENSO prediction accuracy. The main steps of this method are: first, obtain three-dimensional field data of the atmosphere and the ocean, and construct three-dimensional field data. Then, remove the long-term trend and seasonal changes in the three-dimensional field data through standardized preprocessing to generate a standardized data set. After data standardization, the three-dimensional data is multi-scale block processed using a multi-scale Patch partitioning mechanism to extract spatiotemporal features at different spatial scales. The multi-scale Patch partitioning can capture the dependency of three-dimensional field data in different spatial dimensions, forming a spatial feature representation with multi-resolution, which is suitable for capturing the regional impact of ENSO events. The processed data is input into the ENSO long-term prediction model based on the multi-head spatiotemporal attention mechanism. The model adopts a convolutional encoder-decoder architecture. The encoder captures the complex interactive relationship between meteorological variables by processing data of different time and space dimensions in parallel. In particular, this embodiment proposes to use Linea r Attend n The mechanism replaces the Softmax in the traditional multi-head spatiotemporal attention mechanism x The function can significantly reduce the computational complexity. Finally, the model is trained and corrected through historical ENSO event data, and the final ENSO prediction results are generated. During the model training and optimization process, the back propagation method is used to feedback correct the model to improve the prediction accuracy and reduce the error, thereby improving the long-term prediction accuracy of ENSO events and reducing the prediction error.

[0049] The above method is Figure 1 As shown, the specific steps include:

[0050] S1. Obtain sea surface temperature (SST), sea surface wind stress (τx and τy) and upper ocean temperature at different depths to construct three-dimensional field data D input .

[0051] S2. Three-dimensional field data D input Perform preprocessing and normalization to remove long-term trends and seasonal changes and generate a standardized data set D containing outliers norm .

[0052] S3, through the Patch function to standardize the data set D norm Perform block processing on the spatial dimension to generate multi-scale spatiotemporal features F patch .

[0053] In order to capture multi-scale spatial features, this embodiment uses the Patch partitioning mechanism to divide the standardized dataset D normMulti-scale block processing is performed. This mechanism can divide the data into several sub-areas according to the spatial resolution of the data. The scale of each area is determined by the block parameter M. scale Dynamically adjust to generate multi-scale spatiotemporal features F patch The specific formula is as follows:

[0054] F patch =Patch(D norm , M scale )

[0055] S4, through the spatiotemporal embedding mechanism, the spatiotemporal features F patch Map to high-dimensional space and obtain embedded features F embed .

[0056] In order to ensure that the model can capture the relative relationship between time points when processing time series data, this embodiment introduces a spatiotemporal embedding mechanism. The spatiotemporal embedding mechanism includes time embedding, space embedding, and linear embedding. The specific process of time embedding is: inject unique location information into each time point. Since the data at different time points have sequential and periodic patterns, the time encoding generated by the combination of sine and cosine functions applies a logarithmic frequency distribution in each dimension to generate a unique embedding that can characterize the position of the time point. The specific formula is as follows:

[0057]

[0058] Among them, PE represents position encoding, d model is the dimension of the embedding vector, pos represents the position of the time point in the sequence, and i represents half of the embedding dimension (because each position generates two values).

[0059] The specific process of spatial embedding is as follows: in order to learn the data relationship between data such as ocean surface temperature and wind stress at different longitude and latitude positions, each spatial position is mapped to a high-dimensional space through an embedding layer, and feature extraction of spatial dimension data is performed through spatial embedding, mainly targeting local changes in space to ensure that spatial information is retained.

[0060] The specific process of linear embedding is: through linear transformation, the spatiotemporal feature F patch Compress and map the original data into a high-dimensional space, retain the original feature information of the data, align the data with the embedding space dimensions, and enable the time and space embedding vectors to be effectively merged into the model input.

[0061] The embedding feature F obtained by the spatiotemporal embedding mechanism embed The specific expression is as follows:

[0062] F embed =E temporal(F patch )+E spatial (F patch )+E linear (F patch )

[0063] Among them, E temporai (*) is the time embedding function, E spatial (*) is the spatial embedding function, E linear (*) is the linear embedding function.

[0064] S5. Embed the feature F embed Input the trained ENSO long-term prediction model based on the multi-head spatiotemporal attention mechanism to obtain the ENSO prediction results.

[0065] like Figure 2 As shown in the figure, the ENSO long-term prediction model based on the multi-head spatiotemporal attention mechanism includes an encoder module, a multi-scale feature fusion module and a decoder module connected in sequence, wherein the encoder module is constructed based on the improved multi-head spatiotemporal attention mechanism and is used to extract features from the input data. The improved multi-head spatiotemporal attention mechanism replaces the Softmax function in the multi-head spatiotemporal attention mechanism with the linear mapping function Linear Attention; the multi-scale feature fusion module is used to fuse features of different scales output by the encoder module; the decoder module includes n decoders, each decoder is used to generate an intermediate prediction result based on the prediction features output by itself and the output of the multi-scale feature fusion module, and the output of all decoders is weighted and summed to obtain the final prediction result. The specific description of each module is as follows:

[0066] like Figure 3 As shown in the figure, the encoder module adopts a stacked multi-layer neural network structure, whose core components are an improved multi-head spatiotemporal attention mechanism, a feedforward network, and a normalization layer. The input of the encoder module is first input into the improved multi-head spatiotemporal attention mechanism for processing, and the weights of multiple attention heads are calculated in parallel to capture the complex spatiotemporal interaction relationship. The specific expression is as follows:

[0067] Z=Concat(head 1 , head 2 ,...head h )W O

[0068]

[0069] Among them, Z is the output of the improved multi-head spatiotemporal attention mechanism, Concat(*) represents the feature concatenation operation, and W O is the trainable weight, h represents the number of attention heads, Q i, K i and V i Respectively represent the i-th head i The query matrix, key matrix and value matrix of is the LinearAttention function. By replacing the Softmax function in the traditional multi-head spatiotemporal attention mechanism with a linear function, the computational complexity can be reduced from O(n 2 ) is optimized to O(nlog(n)), effectively improving the computing efficiency.

[0070] The output of the improved multi-head spatiotemporal attention mechanism is subjected to residual connection and normalization operations and is expressed as:

[0071] Z norm =Norm(Z+F embed )

[0072] Then, the output Z of the improved multi-head spatiotemporal attention mechanism is norm Input the feedforward network to further extract features. The formula is:

[0073] F feedforward =ReLU(W 1 Z norm +b 1 )

[0074] The output F of the feedforward network feedforward After linear transformation and activation function ReLU, residual connection and normalization are applied again, and the final output feature F scale :

[0075] F scale =Norm(F feedforward +Z norm )

[0076] In the above process, Norm(*) represents the normalization operation, ReLU(*) represents the activation function, and W 1 and b 1 are the weight and bias of the network respectively.

[0077] The feature fusion module generates the final feature representation by fusing multi-scale features. The specific operation is as follows: scale1 and F scale2 The fusion formula is:

[0078] F fusion =F scale1 +F scale2

[0079] Among them, F scale1 and F scale2 They represent features of different scales.

[0080] The decoder module receives two inputs: one is the output of the encoder and the fused feature F fusion , and the second is the prediction feature Z output by the decoder itself decoder , thereby achieving sequence-to-sequence learning.

[0081] Each decoder is Figure 4 As shown in Figure 1, it includes a multi-head spatiotemporal attention mechanism, a normalization layer, a feedforward network layer, and a linear layer. The output of the feedforward network is F feedforward , the output of the first and second normalized layers from the decoder input direction is Z norm-1 and Z norm-2 , the output of the linear layer is Y pred , that is, to generate intermediate prediction results for each decoder. The specific formulas of each layer are as follows:

[0082] Z norm-1 =Norm(Z+F embed )

[0083] F decoder =Concat(Z norm-1 , F fusion )

[0084] Z norm-2 =Norm(Z+F decoder )

[0085] F feedforward =ReLU(W 1 Z norm-2 +b 1 )

[0086] Z decoder =Norm(F feedforward +Z norm-2 )

[0087] Y pred =Linear(Dropout(F decoder ))

[0088] Among them, F decoder Represents the first normalized feature Z in the decoder norm-1 The feature F fused with the encoder output fusion The result of concatenation, Concat(*) represents the feature concatenation operation, Linear(*) represents the linear layer.

[0089] The final predicted value Y of the model final It is obtained by weighted combination of intermediate results generated by multiple decoders, which is specifically expressed as:

[0090] Yfinal =W 1 Y pred1 +W 2 Y pred2 +...+W n Y predn

[0091] Among them, W n is the trainable weight, Y predn Intermediate prediction results generated for different decoders.

[0092] The ENSO long-term prediction model based on the multi-head spatiotemporal attention mechanism is calibrated using historical ENSO event data. The model is trained using the back-propagation algorithm and a cyclic training method. The robustness and prediction accuracy of the model on unknown data are verified through cross-validation and performance evaluation.

[0093] The above method breaks through the limitations of traditional prediction methods. It can not only effectively capture the long-range dependencies in time series, but also analyze the long-range correlations between different geographical locations in the spatial dimension. By effectively integrating multimodal meteorological data such as the atmosphere and ocean, and introducing multi-scale patch segmentation processing, the model can extract key features at different spatial scales, further enhancing the accuracy of ENSO prediction. Through the use of linear attention mechanism, the computational efficiency is significantly optimized, enabling the model to efficiently process long time series data. The above innovations not only ensure that the model achieves high prediction accuracy, but also improve the robustness of the model, which can adapt to the diverse application needs in complex climate environments, and provide important support for meteorological disaster prediction and long-term climate research.

[0094] The above method can accurately predict the timing and intensity of ENSO events over a long period of time, and shows strong adaptability and scalability when processing complex meteorological data. It can not only adapt to changes in different geographical locations and atmospheric variables, but also optimize the computational complexity. It can be applied to various scenarios such as climate forecasting, agricultural production planning, and global climate change research. It can provide more reliable climate data support for meteorological departments and improve disaster response efficiency, and has broad application prospects.

[0095] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0096] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A long-term ENSO prediction method based on a multi-head spatiotemporal attention mechanism, characterized in that: The following steps are involved: S1. Obtain sea surface temperature, sea surface wind stress and upper ocean temperature at different depths to construct three-dimensional field data D input ; S2, the three-dimensional field data D input Preprocessing is performed to remove long-term trends and seasonal changes and generate a standardized data set D norm ; S3, patch the standardized data set D norm Perform block processing on the spatial dimension to generate multi-scale spatiotemporal features F patch ; S4, the spatiotemporal feature F is embedded in the spatiotemporal feature F through the spatiotemporal embedding mechanism patch Map to high-dimensional space and obtain embedded features F embed , the spatiotemporal embedding mechanism includes time embedding, space embedding and linear embedding; S5, embedding the feature F embed Input the trained ENSO long-term prediction model based on multi-head spatiotemporal attention mechanism to obtain ENSO prediction results; Among them, the ENSO long-term prediction model based on the multi-head spatiotemporal attention mechanism includes an encoder module, a multi-scale feature fusion module and a decoder module connected in sequence. The encoder module is constructed based on the improved multi-head spatiotemporal attention mechanism and is used to extract features from the input data. The improved multi-head spatiotemporal attention mechanism replaces the Softmax function in the multi-head spatiotemporal attention mechanism with a linear mapping function; the multi-scale feature fusion module is used to fuse features of different scales output by the encoder module; the decoder module includes multiple decoders, each decoder is used to generate an intermediate prediction result based on the prediction features output by itself and the output of the multi-scale feature fusion module, and the outputs of all decoders are weighted summed to obtain the final prediction result.

2. The ENSO long-term prediction method based on a multi-head spatiotemporal attention mechanism according to claim 1, characterized in that: In step S3, the spatiotemporal feature F is generated by the Patch function patch The specific formula is as follows: F patch =Patch(D norm ,M scale ) Among them, M scale Represents a multi-scale partitioning operation.

3. The ENSO long-term prediction method based on a multi-head spatiotemporal attention mechanism according to claim 1, characterized in that: In the spatiotemporal embedding mechanism, the specific process of time embedding is: injecting position information into each time point, and the specific formula is as follows: Among them, PE represents position encoding, d model is the dimension of the embedding vector, pos represents the position of the time point in the sequence, and i represents half of the embedding dimension; The specific process of the spatial embedding is: mapping each spatial position to a high-dimensional space through an embedding layer; The specific process of linear embedding is: patch Perform compression; The embedding feature F obtained by the spatiotemporal embedding mechanism embed The specific expression is as follows: F embed =E temporal (F patch )+E spatial (F patch )+E linear (F patch ) Among them, E temporal (*) is the time embedding function, E spatial (*) is the spatial embedding function, E linear (*) is the linear embedding function.

4. The ENSO long-term prediction method based on a multi-head spatiotemporal attention mechanism according to claim 1, characterized in that: The expression of the improved multi-head spatiotemporal attention mechanism is as follows: Z=Concat(head1,head2,...head h )W O Among them, Z is the output of the improved multi-head spatiotemporal attention mechanism, Concat(*) represents the feature concatenation operation, and W O is the trainable weight, h represents the number of attention heads, Q i , K i and V i Respectively represent the i-th head i The query matrix, key matrix and value matrix of It is a linear mapping function, specifically a Linear Attention function.

5. The ENSO long-term prediction method based on a multi-head spatiotemporal attention mechanism according to claim 1, characterized in that: The encoder module includes an improved multi-head spatiotemporal attention mechanism and a feedforward network connected in sequence, and a normalization layer is respectively arranged before and after the feedforward network.

6. The ENSO long-term prediction method based on a multi-head spatiotemporal attention mechanism according to claim 5, characterized in that: The output of the feedforward network is F feedforward , the outputs of the normalized layers before and after the feedforward network are Z norm and F scale , the expressions of the above outputs are as follows: Z norm =Norm(Z+F embed ) F feedforward =ReLU(W1Z norm +b1) F scale =Norm(F feedforward +Z norm ) Among them, Z is the output of the improved multi-head spatiotemporal attention mechanism, Norm(*) represents the normalization operation, ReLU(*) represents the activation function, W1 and b1 are the weight and bias of the network respectively.

7. The ENSO long-term prediction method based on multi-head spatiotemporal attention mechanism according to claim 1, characterized in that: In the decoder module, each decoder includes a multi-head attention mechanism, a normalization layer, a feedforward network layer and a linear layer connected in sequence.

8. The ENSO long-term prediction method based on multi-head spatiotemporal attention mechanism according to claim 1, characterized in that: The ENSO long-term prediction model based on the multi-head spatiotemporal attention mechanism is trained by a back-propagation algorithm and a cyclic training method.

9. An electronic device comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images