South indian ocean dipole prediction method based on jump attention multi-task learning

Abstract

The application discloses a South Indian Ocean Dipole prediction method based on a leap attention multi-task learning, and belongs to the field of meteorological intelligent information processing. The method comprises the following steps: obtaining original meteorological data and performing pretreatment, and selecting a South Indian Ocean Dipole index as a main task label; a multi-task deep learning model is constructed, including a shared feature extraction module, a leap bidirectional LSTM module and a leap multi-head attention mechanism module, and the deep learning model is trained and optimized by using training set data and labels; and the trained multi-task deep learning model is used to complete a South Indian Ocean Dipole prediction task and an El Nino-Southern Oscillation prediction task. According to the application, the leap attention mechanism is used to directly transmit shallow features to a deep network, so that the gradient vanishing problem is effectively relieved, the multi-scale feature fusion capability is enhanced, knowledge transfer is realized by using a double-branch multi-task learning, and the prediction accuracy and timeliness of the South Indian Ocean Dipole are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of meteorological intelligent information processing, and in particular to a method for predicting the South Indian Ocean dipole based on jump-attention multi-task learning. Background Technology

[0002] The Southern Indian Ocean Dipole (SIOD) is an important climate model for the east-west oscillation of sea surface temperature (SST) in the southern tropical Indian Ocean, and its positive and negative phases have a significant impact on global climate (such as droughts in Australia, floods in East Africa, and the Asian monsoon). Traditional SIOD predictions mainly rely on statistical models and dynamic climate models, but these methods face many challenges. For example, the modeling of nonlinear relationships is insufficient, and the correlation between SIOD and ocean-atmosphere interactions is highly nonlinear. The spatiotemporal dependencies are complex and difficult to capture; the spatial distribution of SIOD events (such as warming in the western Indian Ocean and cooling in the eastern Indian Ocean) and their temporal evolution (seasonal-interannual scales) need to be captured simultaneously, and the rapid growth of observational data (such as satellite remote sensing and reanalysis data) requires efficient feature extraction methods.

[0003] Currently, Tao et al. proposed a convolutional prediction method in their paper (Tao Y, Qiu C, Wang D, et al. Indian Ocean Dipole (IOD) forecasts based on convolutional neural network with sea level pressure predictor[J]. Environmental Research Letters, 2024, 19(10): 104045. DOI: 10.1088 / 1748-9326 / ad7522.). However, this prediction model does not capture its own spatiotemporal dependencies, which may lead to the loss of these features and a decrease in prediction accuracy. In addition, existing models fail to fully utilize the deep coupling features between multi-source meteorological data and lack interpretability analysis of prediction results, making it difficult to meet the needs of practical operational forecasting. In existing deep learning models, the attention mechanism usually adopts a single forward propagation method, making it difficult to effectively transfer shallow features to deep networks, resulting in low feature reuse rate and prominent gradient vanishing problems. Summary of the Invention

[0004] Purpose of the invention: To address the above problems, the purpose of this invention is to provide a prediction method for the South Indian Ocean Dipole based on jump-attention multi-task learning. By constructing a method that integrates a jump-attention mechanism and a multi-task learning framework, the prediction accuracy and timeliness of the South Indian Ocean Dipole Index are improved.

[0005] Technical solution: The present invention provides a method for predicting the South Indian Ocean dipole based on skip-attention multi-task learning, comprising the following steps:

[0006] Step 1: Obtain raw SST data and OHC data. Preprocess the raw data to obtain SSTA anomalies and OHCA anomalies. Divide the anomaly data into training and validation sets. Select the South Indian Ocean Dipole Index as the primary task label and the El Niño-Southern Oscillation Index as the secondary task label.

[0007] Step 2: Construct a multi-task deep learning model, including a shared feature extraction module built from a CNN module, a skip-type bidirectional LSTM module, and a skip-type multi-head attention mechanism module. Use training set data and labels to train and optimize the deep learning model.

[0008] Step 3: Use the trained multi-task deep learning model to complete the South Indian Ocean Dipole prediction task and the El Niño-Southern Oscillation prediction task.

[0009] Preferably, the steps for training and optimizing a deep learning model using training set data and labels include:

[0010] The training set data is input into the shared feature extraction module to extract shared features.

[0011] The shared features are input into two independent task branches. Each branch is processed and connected sequentially through a skip-type bidirectional LSTM module and a skip-type multi-head attention module to perform the South Indian Ocean Dipole prediction task and the El Niño-Southern Oscillation prediction task.

[0012] Preferably, the steps of inputting the training set data into the shared feature extraction module to extract shared features include:

[0013] The outlier feature map is passed through a channel attention layer to generate channel weights, and the channel weights are multiplied with the original feature map channel by channel. Then, the weighted features are downsampled three times in sequence to obtain a four-dimensional tensor feature map, which is used as a shared feature.

[0014] Preferably, the steps of inputting the shared features into two independent task branches, with each branch sequentially passing through a skip-type bidirectional LSTM module and a skip-type multi-head attention module for computation and skip connections, include:

[0015] In any task branch, the shared features are input into a 6-layer LSTM unit for processing to obtain the hidden state sequence; the 4th to 6th layers of the 6-layer LSTM unit adopt a skip connection method;

[0016] The hidden state sequence is input into a 6-layer attention layer for multi-head attention computation to obtain the final output features; among them, layers 4-6 of the 6-layer attention layer adopt a skip connection method.

[0017] Preferably, the step of inputting the shared features into a 6-layer LSTM unit for processing to obtain the hidden state sequence includes:

[0018] The formula for calculating the hidden states of the forward LSTM in layers 1-3 is as follows:

[0019] ,

[0020] The formula for calculating the hidden states of the backward LSTM layers 1-3 is as follows:

[0021] ,

[0022] The formula for calculating the hidden states of the forward LSTM in layers 4-6 is as follows:

[0023] ,

[0024] The formula for calculating the hidden states of the backward LSTM layers 4-6 is as follows:

[0025] ,

[0026] The final hidden state is represented as:

[0027] ,

[0028] in, This represents an LSTM cell, whose calculations include:

[0029] The formula for calculating the forget gate is:

[0030] ,

[0031] The formula for calculating the input gate is:

[0032] ,

[0033] The formula for calculating candidate cell states is:

[0034] ,

[0035] Cell status updated to:

[0036] ,

[0037] The formula for calculating the output gate is:

[0038] ,

[0039] The formula for calculating the hidden state output is:

[0040] ,

[0041] in, Indicates the first The input vector at time t, Indicates the layer index of the LSTM network The outputs of the forget gate, input gate, and output gate are respectively... Let be the cell state at time t. Indicates the candidate cell state, used for updating , Indicates the first The hidden state vector at time step 1. This represents the hidden state dimension of the LSTM. It is the sigmoid activation function. This indicates element-wise multiplication. The hidden states of the first three layers are concatenated with the input of the current layer through skip connections, enabling the direct transfer of shallow features to deeper layers; The weight matrix represents the forget gate. This represents the weight matrix of the input gate. The weight matrix representing the candidate cell state. This represents the weight matrix of the output gate; The bias term representing the forget gate. This represents the bias term of the input gate. Bias terms representing the candidate cell state. This represents the bias term of the output gate.

[0042] Preferably, the step of inputting the hidden state sequence into a 6-layer attention layer for multi-head attention computation to obtain the final output features includes:

[0043] The attention output features of layers 1-3 are calculated separately using the following formulas:

[0044] ,

[0045] ,

[0046] ,

[0047] The attention output features of layers 4-6 are calculated separately using the following formulas:

[0048] ,

[0049] ,

[0050] ,

[0051] in, This represents the multi-head attention mechanism, and the computation process includes:

[0052] The formulas for calculating the Q, K, and V projections are as follows:

[0053] ,

[0054] The formula for calculating single-head attention is:

[0055] ,

[0056] The formula for scaling dot product attention is:

[0057] ,

[0058] The formula for multi-head attention splicing is:

[0059] ,

[0060] in, It is the hidden state sequence of the jump-type bidirectional LSTM output. These are the attention output features of each layer. , , It is a query, key, and value matrix for the attention mechanism. , , Represents the global Q, K, V projection matrices. For the first The projection matrix of each attention head. This represents the scaled dot product attention function. It is a scaling factor. For the number of attention heads, The projection matrix is ​​used to fuse multi-head information and keep the output dimension consistent with the input dimension.

[0061] Preferably, in step 1, the original climate variables are organized in the form of a four-dimensional array, denoted as [time step × spatial latitude × spatial longitude × number of variables]. A spatiotemporal cube is constructed using the sliding window method to generate a four-dimensional tensor, denoted as [batch size, number of channels, height, width]. The channels are derived from the product of the climate variables and the continuous time steps.

[0062] Preferably, the step of generating channel weights from outlier feature maps through a channel attention layer includes:

[0063] Channel features are extracted using average pooling and max pooling, with the following formulas:

[0064] ,

[0065] ,

[0066] The channel weights are generated by a multilayer perceptron with shared weights, and the calculation formula is as follows:

[0067]

[0068] in, Indicates channel weight, This represents the sigmoid function. This represents a multilayer perceptron. Indicates the first Each channel is located in The value, It is an average pooling feature. It is a max pooling feature. Indicates the number of latitude grids. Indicates the number of longitude grids. and Indicates a spatial location index.

[0069] Preferably, in step 2, the loss function for training the multi-task deep learning model is:

[0070] ,

[0071] In the formula, As a balance factor, Losses due to the prediction mission of the South Indian Ocean Dipole. This indicates the loss in the El Niño-Southern Oscillation Index (ELO) forecasting mission.

[0072] Preferably, the loss function for each individual task uses the mean squared error, and the formula is as follows:

[0073] ,

[0074] In the formula, For the sample size, For the true value, These are predicted values.

[0075] Beneficial effects: Compared with the prior art, the significant advantages of this invention are:

[0076] 1. Enhanced multi-task learning collaboration:

[0077] This invention introduces El Niño-Southern Oscillation (ENSO) as an auxiliary task, leveraging its teleconnection with the Southern Indian Ocean Dipole to achieve knowledge transfer during the feature extraction stage. This effectively alleviates the problem of limited training samples in SIOD and improves the model's generalization ability under small sample conditions. In particular, the structural design of setting independent task branches after convolution allows SIOD and ENSO to learn spatiotemporal dependency patterns suitable for their respective tasks, avoiding feature interference between tasks.

[0078] 2. Skipping attention mechanism enhances feature reuse:

[0079] This invention innovatively introduces a skip-attention mechanism into a weather prediction model. Skip-connection structures are used in both the LSTM and attention modules to directly transfer shallow spatiotemporal features to deep networks, effectively alleviating the gradient vanishing problem and enhancing the fusion capability of multi-scale features. Attached Figure Description

[0080] Figure 1 This is a flowchart of the present invention;

[0081] Figure 2 This is a structural diagram of a deep learning model;

[0082] Figure 3 A comparison chart of prediction results from multiple models;

[0083] Figure 4 This is a comparison chart of the prediction errors of multiple models. Detailed Implementation

[0084] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and not intended to limit the scope of the invention. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the embodiments of the present invention, and not all structures.

[0085] In the following description, specific details such as target system architecture and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0086] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0087] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0088] Furthermore, in the description of this application and the appended claims, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0089] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include the target features, structures, or characteristics described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0090] Combination Figure 1 As shown in this embodiment, the South Indian Ocean Dipole prediction method based on skip-attention multi-task learning includes the following steps:

[0091] Step 1: Obtain raw sea surface temperature (SST) and ocean heat capacity (OHC) data. Preprocess the raw data to obtain sea surface temperature anomaly (SSTA) and ocean heat capacity anomaly (OHCA). Select the South Indian Ocean Dipole Index as the primary task label and the El Niño-Southern Oscillation Index as the secondary task label. Divide the processed anomaly data into training and validation sets.

[0092] In one example, the Coupled Model Intercomparison Project Phase 5 (CMIP5) climate model was used to select century-old Sea Surface Temperature (SST) and Ocean Heat Capacity (OHC) data. The data were processed into Sea Surface Temperature Anomalies (SSTA) and Ocean Heat Capacity Anomalies (OHCA). Table 1 shows the dataset selected from CMIP5. It was ensured that the resolution (lat / lon) and units (°C / K) of the climatology and the current data were consistent. The climatology was usually selected as 30 years (such as the World Meteorological Organization standard 1981-2010 or 1991-2020). Temporal smoothing was performed on the climatology and anomalies to reduce noise.

[0093] Table 1. Dataset information selected from CMIP5

[0094]

[0095] Furthermore, in step 1, the original climate variables are organized in the form of a four-dimensional array, denoted as [time step × spatial latitude × spatial longitude × number of variables]. A spatiotemporal cube is constructed using the sliding window method to generate a four-dimensional tensor, denoted as [batch size, number of channels, height, width]. The channels are derived from the product of the climate variables and the continuous time steps.

[0096] In one example, raw Sea Temperature Anomaly (SSTA) or Ocean Heat Capacity Anomaly (OHCA) data are typically organized as a four-dimensional array [time steps × spatial latitude × spatial longitude × number of variables], for example: 36 months × 24 latitude × 72 longitude × 2 variables. To enable efficient processing by a Convolutional Neural Network (CNN), a sliding window method is used to construct a spatiotemporal cube. Using a 3-month time window, data from consecutive time steps are stacked to form a three-dimensional sample, with each sample containing the spatial distribution of variables across multiple time points. Subsequently, the time and variable dimensions are merged into the channel dimension through dimensionality reorganization, ultimately generating a four-dimensional tensor conforming to the PyTorch NCHW format, denoted as: [batch size, number of channels = 6, height = 24, width = 72]. The 6 channels originate from the product of 2 climate variables and 3 consecutive time steps. This transformation method has two important advantages: first, encoding the time dimension information into the channel space allows the two-dimensional convolutional kernel to simultaneously capture spatiotemporal features; second, the concatenation of different climate variables in the channel dimension facilitates the network's learning of spatial correlation patterns between variables.

[0097] To adapt to the unique characteristics of climate data, a series of targeted designs were adopted: in terms of convolution kernel size, a larger meridional kernel (8×4) was selected to capture large-scale changes from the equator to the poles; in the data preprocessing stage, each climate variable was independently standardized to eliminate differences in physical dimensions; in the network structure, a padding strategy was used to maintain the integrity of spatial dimensions and ensure the preservation of edge grid information.

[0098] The Southern Indian Ocean Dipole Index was selected as the primary task label (i.e., the Southern Indian Ocean Dipole prediction task), and the El Niño-Southern Oscillation Index was selected as the secondary task label (i.e., the El Niño-Southern Oscillation prediction task). Specifically, the Southern Indian Ocean Dipole Index was calculated using the difference in sea surface temperature anomalies between the western Pole region (45°S-30°S, 45°E-75°E) and the eastern Pole region (30°S-18°S, 80°E-100°E); the El Niño-Southern Oscillation Index was calculated using the average sea surface temperature anomaly in the Niño3.4 region (5°S-5°N, 170°W-120°W).

[0099] The preprocessed sea surface temperature anomalies (SSTA) and ocean heat capacity anomalies (OHCA) were used to construct a training set, along with the main task label and the auxiliary task label, for subsequent model training.

[0100] Step 2: Construct a multi-task deep learning model, including a shared feature extraction module built from a CNN module, a skip-type bidirectional long short-term memory network (LSTM) module, and a skip-type multi-head attention mechanism module. Use training set data and labels to train and optimize the deep learning model.

[0101] Furthermore, the steps for training and optimizing a deep learning model using training set data and labels include:

[0102] The training set data is input into the shared feature extraction module to extract shared features.

[0103] The shared features are input into two independent task branches. Each branch is processed and connected sequentially through a skip-type bidirectional LSTM module and a skip-type multi-head attention module to perform the South Indian Ocean Dipole prediction task and the El Niño-Southern Oscillation prediction task.

[0104] like Figure 2As shown, the deep learning model includes a shared feature extraction module and two independent task branches arranged sequentially. The shared feature extraction module consists of a channel attention layer and three downsampling layers. The two task branches are used to predict SIOD and ENSO, respectively. Each branch contains an independent skip-type bidirectional LSTM network, a skip-type multi-head attention mechanism, and a fully connected layer. Channel weights are dynamically adjusted based on the input data, rather than being fixedly assigned, thereby improving the flexibility of feature representation. Irrelevant or low signal-to-noise ratio channels are deweighted (e.g., their weights are brought close to 0), improving model robustness, thus enhancing useful features and suppressing redundant information.

[0105] Furthermore, the training set data is input into the shared feature extraction module, and the steps for extracting shared features include:

[0106] The outlier feature maps (preprocessed SSTA sea surface temperature anomalies and OHCA ocean thermal capacity anomalies) are processed through a channel attention layer to generate channel weights. The channel weights are then multiplied with the original feature maps channel by channel. The weighted features are then downsampled three times to obtain a four-dimensional tensor feature map, which is used as a shared feature.

[0107] Furthermore, the step of generating channel weights from the outlier feature map through the channel attention layer includes:

[0108] Channel features are extracted using average pooling and max pooling, with the following formulas:

[0109] ,

[0110] ,

[0111] The channel weights are generated by a multilayer perceptron with shared weights, and the calculation formula is as follows:

[0112] ,

[0113] in, Indicates channel weight, This represents the sigmoid function. This represents a multilayer perceptron. Indicates the first Each channel is located in The value, It is an average pooling feature. It is a max pooling feature. Indicates the number of latitude grids. Indicates the number of longitude grids. and Indicates a spatial location index.

[0114] Furthermore, the channel weights are downsampled three times in sequence. In each downsampling process, a convolutional kernel is first used for feature extraction while keeping the spatial size of the feature weights unchanged. After each convolutional layer, spatial downsampling is performed through tanh activation and max pooling, followed by a dropout layer. After the third downsampling process, the feature weights of the four-dimensional tensor are obtained.

[0115] In one example, after generating channel weight vectors through channel attention, these vectors are fed into a convolutional layer that employs... The convolutional kernels are used for feature extraction, and the spatial size of the feature map is kept constant by setting parameters. After processing by this convolutional layer, the output size remains unchanged. ,in For batch size, This represents the number of convolutional filters, i.e., the number of channels. After each convolutional layer, tanh activation is performed, followed by... Max pooling is used for spatial downsampling with a stride of 2, halving the feature map size. Then, a Dropout layer is applied during the training phase with probability... Randomly zeroing out some neuron outputs effectively prevents overfitting. Both the second and third convolutional layers use this method. Convolution kernel, setting parameters The feature map spatial dimensions remain unchanged. Tanh activation is also performed after each convolutional layer. Max pooling gradually compresses the feature map size to... This hierarchical design achieves feature abstraction from local to global. Considering the computational power requirements, this example... and With values ​​set to 500 and 50 respectively, 500 samples are input at a time, passing through 50 channels. This network structure uses three convolutions and downsampling to convert the initial... The input is effectively compressed to This significantly reduces the number of parameters while preserving key features. After spatial feature extraction, the feature extraction module outputs a dimension of... The four-dimensional tensor, where 500 represents the batch size, 50 corresponds to the number of convolutional filters (channels), and 6 and 18 represent the latitude and longitude grid resolutions after downsampling, respectively.

[0116] Furthermore, the shared features are input into two independent task branches, and each branch sequentially passes through a skip-type bidirectional LSTM module and a skip-type multi-head attention module for computation and skip connections. The steps include:

[0117] In any task branch, the shared features are input into a 6-layer LSTM unit for processing to obtain the hidden state sequence; the 4th to 6th layers of the 6-layer LSTM unit adopt a skip connection method;

[0118] The hidden state sequence is input into a 6-layer attention layer for multi-head attention computation to obtain the final output features; among them, layers 4-6 of the 6-layer attention layer adopt a skip connection method.

[0119] Specifically, the shared features are input into two independent task branches. In the SIOD task branch, a dimension permutation operation is performed on the four-dimensional tensor of the feature weights to convert the latitude in the spatial dimension into the pseudo-time series length; then, the feature channels and longitude points are merged into feature vectors through tensor flattening to form a sequence with a three-dimensional structure.

[0120] The sequence is input into a skip-connected bidirectional LSTM network, which contains 6 layers of LSTM units. Layers 1-3 use normal connections, while layers 4-6 use skip connections. The generation dimension is... The hidden state sequence, where B represents the batch size and T represents the pseudo-time step. This represents the dimension of the hidden state in the LSTM.

[0121] Furthermore, the steps of inputting the shared features into a 6-layer LSTM unit for processing to obtain the hidden state sequence include:

[0122] The formula for calculating the hidden states of the forward LSTM in layers 1-3 is as follows:

[0123] ,

[0124] The formula for calculating the hidden states of the backward LSTM layers 1-3 is as follows:

[0125] ,

[0126] The formula for calculating the hidden states of the forward LSTM in layers 4-6 is as follows:

[0127] ,

[0128] The formula for calculating the hidden states of the backward LSTM layers 4-6 is as follows:

[0129] ,

[0130] The final hidden state is represented as:

[0131] ,

[0132] in, This represents an LSTM cell, whose calculations include:

[0133] The formula for calculating the forget gate is:

[0134] ,

[0135] The formula for calculating the input gate is:

[0136] ,

[0137] The formula for calculating candidate cell states is:

[0138] ,

[0139] Cell status updated to:

[0140] ,

[0141] The formula for calculating the output gate is:

[0142] ,

[0143] The formula for calculating the hidden state output is:

[0144] ,

[0145] in, Indicates the first The input vector at time t, Indicates the layer index of the LSTM network The outputs of the forget gate, input gate, and output gate are respectively... Let be the cell state at time t. Indicates the candidate cell state, used for updating , Indicates the first The hidden state vector at time step 1. This represents the hidden state dimension of the LSTM. It is the sigmoid activation function. This indicates element-wise multiplication. The hidden states of the first three layers are concatenated with the input of the current layer through skip connections, enabling the direct transfer of shallow features to deeper layers; The weight matrix represents the forget gate. This represents the weight matrix of the input gate. The weight matrix representing the candidate cell state. This represents the weight matrix of the output gate; The bias term representing the forget gate. This represents the bias term of the input gate. Bias terms representing the candidate cell state. This represents the bias term of the output gate.

[0146] In one example, a three-stage feature reorganization strategy was designed to establish a deep representation of spatiotemporal coupling, including:

[0147] (1) First, perform a dimension replacement operation to adjust the tensor to This operation converts the spatial dimension of 6 (latitude) into a pseudo-time series length. The climatological basis for this is that, in a standardized latitude and longitude grid, spatial variations in the latitudinal direction often correspond to the temporal evolution characteristics of the climate system. For example, the propagation of sea surface temperature anomalies (SSTA) in the equatorial Pacific region typically manifests as a latitudinal fluctuation signal, and its spatial gradient changes can be compared to the dynamic evolution of a time series.

[0148] (2) Secondly, by tensor flattening, the 50 feature channels and 18 longitude points are merged into a 900-dimensional feature vector, forming The three-dimensional structure.

[0149] (3) Finally, the processed sequence is input into a skip-connected bidirectional LSTM network. This network uses a 6-layer LSTM structure, where layers 1-3 are standard LSTM units, and layers 4-6 receive the hidden states of layers 1-3 through skip connections. That is, layer 4 receives the output of layer 1, layer 5 receives the output of layer 2, and layer 6 receives the output of layer 3. The hidden state dimension is set to 64, and the hidden state is generated... The spatiotemporal feature tensor (128 dimensions after bidirectional concatenation) is used. This jump structure allows shallow temporal features to be directly transmitted to deeper layers, effectively alleviating the gradient vanishing problem and enhancing the ability to capture long temporal dependencies.

[0150] Furthermore, the steps for inputting the hidden state sequence into a 6-layer attention layer for multi-head attention computation to obtain the final output features include:

[0151] The attention output features of layers 1-3 are calculated separately using the following formulas:

[0152] ,

[0153] ,

[0154] ,

[0155] The attention output features of layers 4-6 are calculated separately using the following formulas:

[0156] ,

[0157] ,

[0158] ,

[0159] in, This represents the multi-head attention mechanism, and the computation process includes:

[0160] The formulas for calculating the Q, K, and V projections are as follows:

[0161] ,

[0162] The formula for calculating single-head attention is:

[0163] ,

[0164] The formula for scaling dot product attention is:

[0165] ,

[0166] The formula for multi-head attention splicing is:

[0167] ,

[0168] in, It is the hidden state sequence of the jump-type bidirectional LSTM output. These are the attention output features of each layer. , , It is a query, key, and value matrix for the attention mechanism. , , Represents the global Q, K, V projection matrices. For the first The projection matrix of each attention head. This represents the scaled dot product attention function. It is a scaling factor. For the number of attention heads, The projection matrix is ​​used to fuse multi-head information and keep the output dimension consistent with the input dimension.

[0169] In one example, the skip-connect multi-head attention module uses four attention heads. Layers 1-3 are standard multi-head attention layers, and layers 4-6 receive the attention features from layers 1-3 through skip connections. Specifically, layer 4 receives the weighted sum of layers 1 and 3, layer 5 receives the weighted sum of layers 2 and 4, and layer 6 receives the weighted sum of layers 3 and 5. This skip-connect structure allows the attention weights of shallow layers to directly participate in the attention calculation of deeper layers, enhancing the model's ability to fuse multi-scale spatiotemporal features.

[0170] Furthermore, the steps of training and iterating the deep learning model using the training set data and labels also include:

[0171] The sixth layer feature of the jump-type multi-head attention output The aggregated feature vector is obtained by averaging along the time dimension, as shown in the formula:

[0172] ,

[0173] in, Let represent the feature vector at time step t, and T represent the pseudo-time step number;

[0174] The aggregated features are mapped to the prediction target through a fully connected layer, as follows:

[0175] ,

[0176] in, This is a predicted value, specifically the predicted South Indian Ocean Dipole index. This is the weight matrix. This is a bias term.

[0177] Similarly, in the El Niño-Southern Oscillation (ENSO) task branch, the same network structure as the South Indian Ocean Dipole (SIOD) task branch is adopted, including an independent skip-type bidirectional LSTM network, a skip-type multi-head attention mechanism, and fully connected layers, outputting the predicted value of the El Niño-Southern Oscillation index. During model training, the South Indian Ocean Dipole index is used as the label in the SIOD task branch, and the El Niño-Southern Oscillation index is used as the label in the ENSO task branch.

[0178] Furthermore, in step 2, the loss function for training the multi-task deep learning model is:

[0179] ,

[0180] In the formula, As a balance factor, Losses due to the prediction mission of the South Indian Ocean Dipole. This indicates the loss in the El Niño-Southern Oscillation Index (ELO) forecasting mission.

[0181] In one example, the total loss function of the multi-task deep learning model is a weighted sum of the loss functions of the two tasks, with the balance factor λ set to 0.5 to ensure that the two tasks maintain an appropriate weight balance during training.

[0182] Furthermore, the mean squared error is used as the loss function for each individual task, and the formula is as follows:

[0183] ,

[0184] In the formula, For the sample size, For the true value, These are predicted values.

[0185] In one example, historical sea surface temperature (SST) and heat capacity (OHCC) datasets are preprocessed to derive SST anomalies and OHCC anomalies. The dataset is then divided into training and validation sets to train a deep neural network. The iteration stops when the loss function reaches its minimum. An early stopping mechanism is employed during training: if the validation set loss does not decrease for 10 consecutive cycles, the learning rate is reduced to 0.5 times its original value, and training stops when the learning rate falls below a threshold. Finally, the prediction accuracy is calculated, and the model's accuracy is evaluated based on the distance between observed and true values.

[0186] Step 3: Use the trained multi-task deep learning model to complete the South Indian Ocean Dipole prediction task and the El Niño-Southern Oscillation prediction task.

[0187] The sea surface temperature anomaly (SSTA) and heat capacity anomaly (OHCA) from the validation set are input into the trained deep learning model to predict the South Indian Ocean Dipole Index.

[0188] The target deep learning model constructed in this invention is compared with traditional dynamical model prediction models, including CanCM3, CanCM4, CCSM3, CCSM4, GFDL-aer04, GFDL-FLOR-A06, and GFDL-FLOR-B01, for predicting the South Indian Ocean Dipole Index. CanCM3 and CanCM4 were developed by the Canadian Centre for Climate Modeling and Analysis (CCCma); CCSM3 and CCSM4 were developed by the National Center for Atmospheric Research (NCAR); and the GFDL series models (including GFDL-aer04, GFDL-FLOR-A06, and GFDL-FLOR-B01) were developed by the Geophysical and Fluid Dynamics Laboratory (GFDL). All model data are stored on the NMME system. NMME stands for North American Multi-Model Ensemble, a seasonal forecasting system jointly developed by several leading North American climate research institutions.

[0189] The target deep learning model in this invention is denoted as MultNET. Figure 3This paper demonstrates the predictive skill of various models for the Southern Indian Ocean Dipole (SIOD), using correlation coefficients as a metric and the horizontal axis representing the lead time (1 to 11 months). The results show that the MultNET model maintains high correlation coefficients across all lead times, exhibiting stable and excellent predictive ability; while the predictive skill of traditional climate models (such as CanCM3, CanCM4, CCSM3, CCSM4, GFDL-aer04, GFDL-FLOR-A06, and GFDL-FLOR-B01) decreases significantly with increasing lead time. Overall, MultNET demonstrates a clear advantage over traditional models in SIOD prediction.

[0190] Figure 4 The figure displays the root mean square error (RMSE) of various models for predicting the South Indian Ocean Dipole. The horizontal axis represents the lead time (1 to 11 months), and the vertical axis represents the RMSE value (unit: °C). A smaller value indicates a smaller prediction error and higher accuracy. The figure compares the performance of the MultNET model with several traditional dynamical models, including CanCM3, CanCM4, CCSM3, CCSM4, GFDL-aer04, GFDL-FLOR-A06, and GFDL-FLOR-B01. The results show that MultNET maintains a low RMSE value across all lead times, demonstrating significantly better prediction accuracy than other dynamical models. In contrast, the RMSE of the traditional models increases with lead time, indicating a gradual increase in prediction error. This suggests that MultNET has higher reliability and stability in SIOD prediction. This invention can extend reliable South Indian Ocean Dipole predictions to the next 8 months, outperforming traditional predictions.

[0191] The specific implementation schemes described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific implementation schemes of the present invention and are not intended to limit the scope of the present invention. Any equivalent changes and modifications made by those skilled in the art without departing from the concept and principles of the present invention should fall within the scope of protection of the present invention.

Claims

1. A method for predicting the South Indian Ocean dipole based on skip-attention multi-task learning, characterized in that, Includes the following steps: Step 1: Obtain raw sea surface temperature (SST) data and ocean heat capacity (OHC) data. Preprocess the raw data to obtain sea surface temperature anomalies (SSTA) and ocean heat capacity anomalies (OHCA). Divide the anomaly data into training set and validation set. The South Indian Ocean Dipole Index was selected as the primary task label, and the El Niño-Southern Oscillation Index was selected as the secondary task label. Step 2: Construct a multi-task deep learning model, including a shared feature extraction module built from a CNN module, a skip-type bidirectional LSTM module, and a skip-type multi-head attention mechanism module. Use training set data and labels to train and optimize the deep learning model. Step 3: Use the trained multi-task deep learning model to complete the South Indian Ocean Dipole prediction task and the El Niño-Southern Oscillation prediction task.

2. The South Indian Ocean Dipole Prediction Method Based on Skip-Attention Multi-Task Learning according to claim 1, characterized in that, The steps for training and optimizing a deep learning model using training set data and labels include: The training set data is input into the shared feature extraction module to extract shared features. The shared features are input into two independent task branches. Each branch is processed and connected sequentially through a skip-type bidirectional LSTM module and a skip-type multi-head attention module to perform the South Indian Ocean Dipole prediction task and the El Niño-Southern Oscillation prediction task.

3. The South Indian Ocean Dipole Prediction Method Based on Skip-Attention Multi-Task Learning according to claim 2, characterized in that, The steps for extracting shared features by inputting the training set data into the shared feature extraction module include: The outlier feature map is passed through a channel attention layer to generate channel weights, and the channel weights are multiplied with the original feature map channel by channel. Then, the weighted features are downsampled three times in sequence to obtain a four-dimensional tensor feature map, which is used as a shared feature.

4. The South Indian Ocean Dipole Prediction Method Based on Skip-Attention Multi-Task Learning according to claim 2 or 3, characterized in that, The shared features are input into two independent task branches, and each branch sequentially passes through a skip-type bidirectional LSTM module and a skip-type multi-head attention module for computation and skip connections. The steps include: In any task branch, the shared features are input into a 6-layer LSTM unit for processing to obtain the hidden state sequence; the 4th to 6th layers of the 6-layer LSTM unit adopt a skip connection method; The hidden state sequence is input into a 6-layer attention layer for multi-head attention computation to obtain the final output features; among them, layers 4-6 of the 6-layer attention layer adopt a skip connection method.

5. The South Indian Ocean Dipole Prediction Method Based on Skip-Attention Multi-Task Learning according to claim 4, characterized in that, The steps to process the shared features into a 6-layer LSTM unit to obtain the hidden state sequence include: The formula for calculating the hidden states of the forward LSTM in layers 1-3 is as follows: ， The formula for calculating the hidden states of the backward LSTM layers 1-3 is as follows: ， The formula for calculating the hidden states of the forward LSTM in layers 4-6 is as follows: ， The formula for calculating the hidden states of the backward LSTM layers 4-6 is as follows: ， The final hidden state is represented as: ， in, This represents an LSTM cell, and its calculation process includes: The formula for calculating the forget gate is: ， The formula for calculating the input gate is: ， The formula for calculating candidate cell states is: ， Cell status updated to: ， The formula for calculating the output gate is: ， The formula for calculating the hidden state output is: ， in, Indicates the first The input vector at time t, This represents the layer index of the LSTM network. The outputs of the forget gate, input gate, and output gate are respectively... Let be the cell state at time t. Indicates the candidate cell state, used for updating , Indicates the first The hidden state vector at time step 1. This represents the hidden state dimension of the LSTM. It is the sigmoid activation function. This indicates element-wise multiplication. The first three layers are in a hidden state; The weight matrix represents the forget gate. This represents the weight matrix of the input gate. The weight matrix representing the candidate cell state. This represents the weight matrix of the output gate; The bias term representing the forget gate. This represents the bias term of the input gate. Bias terms representing the candidate cell state. This represents the bias term of the output gate.

6. The South Indian Ocean Dipole Prediction Method Based on Skip-Attention Multi-Task Learning according to claim 5, characterized in that, The steps for inputting the hidden state sequence into a 6-layer attention layer for multi-head attention computation to obtain the final output features include: The attention output features of layers 1-3 are calculated separately using the following formulas: ， ， ， The attention output features of layers 4-6 are calculated separately using the following formulas: ， ， ， in, This represents the multi-head attention mechanism, and the computation process includes: The formulas for calculating the Q, K, and V projections are as follows: ， The formula for calculating single-head attention is: ， The formula for scaling dot product attention is: ， The formula for multi-head attention splicing is: ， in, It is the hidden state sequence of the jump-type bidirectional LSTM output. These are the attention output features of each layer. , , It is a query, key, and value matrix for the attention mechanism. , , Represents the global Q, K, V projection matrices. For the first The projection matrix of each attention head. This represents the scaled dot product attention function. It is a scaling factor. For the number of attention heads, This is the projection matrix.

7. The South Indian Ocean Dipole Prediction Method Based on Skip-Attention Multi-Task Learning according to claim 1, characterized in that, In step 1, the original climate variables are organized in the form of a four-dimensional array, denoted as [time step × spatial latitude × spatial longitude × number of variables]. A spatiotemporal cube is constructed using the sliding window method to generate a four-dimensional tensor, denoted as [batch size, number of channels, height, width]. The channels are derived from the product of the climate variables and the continuous time steps.

8. The South Indian Ocean Dipole Prediction Method Based on Skip-Attention Multi-Task Learning according to claim 3, characterized in that, The steps for generating channel weights from outlier feature maps using a channel attention layer include: Channel features are extracted using average pooling and max pooling, with the following formulas: ， ， The channel weights are generated by a multilayer perceptron with shared weights, and the calculation formula is as follows: ， in, Indicates channel weight, This represents the sigmoid function. This represents a multilayer perceptron. Indicates the first Each channel is located in The value, It is an average pooling feature. It is a max pooling feature. Indicates the number of latitude grids. Indicates the number of longitude grids. and Indicates a spatial location index.

9. The South Indian Ocean Dipole Prediction Method Based on Skip-Attention Multi-Task Learning according to claim 1, characterized in that, In step 2, the loss function for training the multi-task deep learning model is: ， In the formula, As a balance factor, Losses due to the prediction mission of the South Indian Ocean Dipole. This indicates the loss in the El Niño-Southern Oscillation Index (ELO) forecasting mission.

10. The South Indian Ocean Dipole Prediction Method Based on Skip-Attention Multi-Task Learning according to claim 9, characterized in that, The loss function for each individual task uses the mean squared error, and the formula is as follows: ， In the formula, For the sample size, For the true value, These are predicted values.

Images