A long-time dependence modeling and cross-branch fusion sea temperature prediction method and system

Abstract

The application discloses a kind of long time dependence modeling and cross-branch fusion sea temperature prediction method and system in the technical field of marine weather intelligent prediction and deep learning modeling.The method comprises the following steps: using deep convolutional encoder to encode the spatial features of standardized sea temperature time series image sequence, obtaining multi-level spatial features; the multi-level spatial features are two-dimensionally discrete wavelet decomposed, to obtain low-frequency approximation representation, horizontal singular feature, vertical singular feature and diagonal edge feature, then input into wavelet attention module, to obtain short-term time series enhanced features;Multi-level spatial features and low-frequency approximation representation are input into ConvLSTM module, to obtain high-frequency dynamic enhancement features;Short-term time series enhanced features and high-frequency dynamic enhancement features are input into cross-branch attention fusion module, to obtain fusion features, then input into Informer time series encoder and decoder in turn, to obtain prediction results.

Description

Technical Field

[0001] This invention relates to the field of intelligent marine meteorological forecasting and deep learning modeling technology, and in particular to a long-term dependent modeling and cross-branch fusion method and system for sea surface temperature forecasting. Background Technology

[0002] Sea surface temperature is an important physical quantity reflecting ocean dynamics and energy exchange, and it is of great significance for climate prediction, typhoon path analysis, fishery resource distribution, and research on air-sea interaction.

[0003] In recent years, with the development of satellite remote sensing technology and ocean buoy observation networks, it has become possible to acquire high-resolution spatiotemporal sea surface temperature (SST) data. However, SST changes are influenced by various nonlinear dynamic factors, exhibiting significant spatiotemporal coupling and multi-scale characteristics: in the temporal dimension, SST exhibits long-period seasonal variations, short-term eddy disturbances, and random fluctuations; in the spatial dimension, low-frequency trends and high-frequency local details coexist and influence each other. Therefore, how to simultaneously capture long-term dependencies and multi-scale spatial details in deep learning models has become a significant challenge in the field of SST prediction.

[0004] Existing deep learning methods mainly fall into two categories: spatiotemporal prediction models based on convolutional recurrent structures, such as ConvLSTM (Convolutional Long Short-Term Memory), PredRNN (Predictive Recurrent Neural Network), and E3D-LSTM (EIDETIC 3DLong Short-Term Memory). These methods perform short-term predictions by combining convolutional operations with recursive structures, and can learn spatiotemporal dynamics to some extent. However, they suffer from gradient decay and insufficient memory capacity in long-term modeling, limiting the prediction time span. Multi-scale modeling methods based on wavelet or frequency domain decomposition, such as WaveNet (a deep neural network developed by DeepMind that generates raw audio waveforms), are another category. Wavelet decomposition can separate different frequency components and improve the model's sensitivity to details, but most methods only perform simple addition or concatenation between branches, lacking sufficient information exchange mechanisms, resulting in insufficient fusion of low- and high-frequency features. Furthermore, traditional sea surface temperature prediction models often only use recurrent structures, such as RNN (Recurrent Neural Network) / GRU (GateRecurrent Unit), in time series modeling, which have limited ability to capture long-term dependencies.

[0005] In summary, existing sea surface temperature prediction technologies have the following shortcomings:

[0006] 1. In the dual-branch structure, the fusion of low- and high-frequency features is simple, but the information exchange is insufficient;

[0007] 2. It has weak long-term dependency modeling capabilities and struggles to capture complex temporal dynamic changes;

[0008] 3. The prediction results are prone to ambiguity and lack detailed constraints. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the prior art and provide a sea surface temperature prediction method and system that combines long-term dependency modeling and cross-branch fusion. It utilizes a deep convolutional encoder to extract spatial features, achieves multi-scale separation through two-dimensional wavelet decomposition, combines a wavelet attention module and a ConvLSTM module to enhance the interaction of low and high frequency features, and introduces an Informer temporal encoder to efficiently model long-term dependencies. At the same time, multi-scale loss and frequency domain constraints are introduced during the training phase, thereby improving the local texture clarity and temporal consistency of the predicted frames while maintaining the accuracy of overall trend prediction.

[0010] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution:

[0011] In a first aspect, the present invention provides a sea surface temperature prediction method based on long-term dependent modeling and cross-branch fusion, comprising:

[0012] The acquired sea surface temperature time-series image data were normalized to obtain a standardized sea surface temperature time-series image sequence.

[0013] The standardized sea surface temperature time-series image sequence is input into the trained multi-scale sea surface temperature prediction model:

[0014] The standardized sea surface temperature time-series image sequence is spatially encoded using a deep convolutional encoder to obtain multi-level spatial features;

[0015] Two-dimensional discrete wavelet decomposition is performed on the multi-level spatial features to obtain low-frequency approximate representation, horizontal singular features, vertical singular features, and diagonal edge features;

[0016] The multi-level spatial features, low-frequency approximation, horizontal singular features, vertical singular features, and diagonal edge features are input into the wavelet attention module to obtain short-term temporal enhancement features;

[0017] The multi-level spatial features and low-frequency approximation are input into the ConvLSTM module to obtain high-frequency dynamic enhancement features;

[0018] The short-term temporal enhancement features and high-frequency dynamic enhancement features are input into the cross-branch attention fusion module to obtain the fused features;

[0019] The fused features are linearly projected and then input into the Informer temporal encoder to obtain temporally dependent enhanced features.

[0020] The temporal-dependent enhancement features are input into the decoder to obtain the prediction results.

[0021] Optionally, the standardized sea surface temperature time-series image sequence is obtained by the following formula:

[0022] ,in, The mean of the sea surface temperature time series image data. The variance of the sea surface temperature time series image data. This represents time-series sea surface temperature image data. This represents a standardized time-series image sequence of sea surface temperature.

[0023] Optionally, the depth convolutional encoder includes a sequentially connected input layer, multiple convolutional blocks, and an output layer;

[0024] Each of the convolutional blocks includes sequentially connected convolutional layers, normalization layers, and nonlinear activation layers.

[0025] Optionally, the step of inputting the multi-level spatial features, low-frequency approximation, horizontal singular features, vertical singular features, and diagonal edge features into the wavelet attention module to obtain short-term temporal enhancement features includes:

[0026] Will Multi-layered spatial characteristics of time and Short-term temporal enhancement features at time points Input into the reset gate, get Reset the output of the gate at all times ;

[0027] Will Multi-layered spatial characteristics of time and Short-term temporal enhancement features at time points Input into the update gate, get Update the gate output constantly ;

[0028] According to the above Reset the output of the gate at all times and Multi-layered spatial characteristics of time ,generate Candidate activation state at any time ;

[0029] According to the above Candidate activation state at any time and Update the gate output constantly ,generate Timing-gated timing enhancement features ;

[0030] Using 1×1 convolution to approximate low frequencies Horizontal singular features Vertical singularity features and diagonal edge features Channel expansion is performed to obtain the expanded low-frequency approximation. Horizontal singular features Vertical singularity features and diagonal edge features ;

[0031] The Timing-gated timing enhancement features Extended low-frequency approximation Horizontal singular features Vertical singularity features and diagonal edge features By splicing along the channel dimension, we obtain Time-based comprehensive feature representation ;

[0032] The Time-based comprehensive feature representation The input is fed into a wavelet self-attention operator for joint spatial and frequency domain representation learning, resulting in... Short-term temporal enhancement features at time points .

[0033] Optionally, the Reset the output of the gate at all times It can be obtained through the following formula:

[0034] ,

[0035] in, This represents the trainable weight matrix of the reset gate. This represents the Sigmoid activation function;

[0036] The Update the gate output constantly It can be obtained through the following formula:

[0037] ,

[0038] in, This represents the trainable weight matrix of the update gate;

[0039] The Candidate activation state at any time It can be obtained through the following formula:

[0040] ,

[0041] in, This represents the trainable weight matrix used to generate candidate activation states. Represents the hyperbolic tangent function;

[0042] The Timing-gated timing enhancement features It can be obtained through the following formula:

[0043] ;

[0044] The Time-based comprehensive feature representation It can be obtained through the following formula:

[0045] ,

[0046] in, Indicates the concatenation function;

[0047] The Short-term temporal enhancement features at time points It can be obtained through the following formula:

[0048] ,

[0049] in, This represents the wavelet self-attention operator.

[0050] Optionally, the step of inputting the multi-level spatial features and low-frequency approximation into the ConvLSTM module to obtain high-frequency dynamic enhancement features includes:

[0051] Perform a difference operation on the multi-level spatial features and the low-frequency approximation to obtain explicit residual features;

[0052] The explicit residual features are input into the Convolutional Long Short-Term Memory (ConvLSTM) network for temporal modeling to obtain high-frequency dynamic enhancement features;

[0053] The explicit residual characteristics are obtained by the following formula:

[0054] ,

[0055] in, express Multi-layered spatial characteristics of time. express The low-frequency approximation of time. express Explicit residual characteristics at time points;

[0056] The high-frequency dynamic enhancement feature is obtained through the following formula:

[0057] ,

[0058] in, express High-frequency dynamic enhancement features at any given time. express High-frequency dynamic enhancement features at any given time.

[0059] Optionally, the step of inputting the short-term temporal enhancement features and high-frequency dynamic enhancement features into the cross-branch attention fusion module to obtain fused features includes:

[0060] Short-term temporal enhancement features are projected into a query vector using three independent 1×1 convolutional kernels. Key vector Sum value vector ;

[0061] High-frequency dynamically enhanced features are projected into a query vector using three independent 1×1 convolutional kernels. Key vector Sum value vector ;

[0062] For query vector Key vector Sum value vector Perform cross-branch attention calculations to obtain attention aggregation features. ;

[0063] Attention aggregation features After linear transformation, a residual connection is made with short-term temporal enhancement features to obtain the wavelet branch query ConvLSTM branch output. ;

[0064] Key vector Sum value vector and query vector Perform cross-branch attention calculations to obtain attention aggregation features. ;

[0065] Attention aggregation features After linear transformation, a residual connection is made with the high-frequency dynamic enhancement feature to obtain the ConvLSTM branch query wavelet branch output. ;

[0066] The wavelet branch query outputs the ConvLSTM branch. ConvLSTM branch query wavelet branch output Perform adaptive weighted fusion to obtain fused features. ;

[0067] The wavelet query residual output It can be obtained through the following formula:

[0068] ,

[0069] ,

[0070] Where T represents transpose. This represents the softmax activation function. Let the first linear transformation matrix be represented. This indicates short-term temporal enhancement features. Indicates the scaling factor;

[0071] The residual query wavelet output It can be obtained through the following formula:

[0072] ,

[0073] ,

[0074] in, Denotes the second linear transformation matrix. Indicates high-frequency dynamic enhancement features;

[0075] The fusion feature It can be obtained through the following formula:

[0076] ,

[0077] ,

[0078] in, Indicates adaptive weights, This represents the Sigmoid activation function. This represents a multilayer perceptron network. This represents the global average pooling operator.

[0079] Optionally, the time-dependent enhancement sequence is obtained by the following formula:

[0080] ,

[0081] ,

[0082] in, Indicates fusion features, Indicates linear projection. The sequence length representing the fused features. After linear projection The fusion characteristics of moments This represents the input of the Informer timing encoder. Indicates Informer timing encoder. Indicates time-dependent augmentation features, express Temporal dependency enhancement features at specific moments.

[0083] Optionally, the prediction result is obtained using the following formula:

[0084] ,

[0085] in, Indicates decoder, express The prediction results for the time period, express Temporal dependency enhancement features at specific moments.

[0086] Secondly, the present invention provides a sea surface temperature prediction system that relies on long-term modeling and cross-branch fusion, comprising:

[0087] The normalization module is used to normalize the acquired sea surface temperature time series image data to obtain a standardized sea surface temperature time series image sequence.

[0088] The multi-scale sea surface temperature prediction module is used to: input the standardized sea surface temperature time-series image sequence into the trained multi-scale sea surface temperature prediction model.

[0089] The standardized sea surface temperature time-series image sequence is spatially encoded using a deep convolutional encoder to obtain multi-level spatial features;

[0090] Two-dimensional discrete wavelet decomposition is performed on the multi-level spatial features to obtain low-frequency approximate representation, horizontal singular features, vertical singular features, and diagonal edge features;

[0091] The multi-level spatial features, low-frequency approximation, horizontal singular features, vertical singular features, and diagonal edge features are input into the wavelet attention module to obtain short-term temporal enhancement features;

[0092] The multi-level spatial features and low-frequency approximation are input into the ConvLSTM module to obtain high-frequency dynamic enhancement features;

[0093] The short-term temporal enhancement features and high-frequency dynamic enhancement features are input into the cross-branch attention fusion module to obtain the fused features;

[0094] The fused features are linearly projected and then input into the Informer temporal encoder to obtain temporally dependent enhanced features.

[0095] The temporal-dependent enhancement features are input into the decoder to obtain the prediction results.

[0096] Compared with existing technologies, the beneficial effects achieved by this invention are as follows:

[0097] (1) By introducing two-dimensional discrete wavelet decomposition and explicit residual modeling mechanism, the low-frequency large-scale trend and high-frequency local disturbance in the sea surface temperature field are effectively decoupled, realizing the fine modeling of multi-scale and multi-directional features, which significantly improves the model's ability to characterize complex marine dynamic structures such as vortices, fronts and anomalous disturbances; by using cross-branch bidirectional attention mechanism to realize the adaptive interaction and fusion of low-frequency structural information and high-frequency dynamic information, the information redundancy and interference caused by the simple superposition of traditional features are avoided, and the effectiveness and robustness of feature fusion are improved; In the temporal modeling stage, the Informer temporal encoder based on the probabilistic sparse self-attention mechanism is introduced, which greatly reduces the computational complexity while ensuring the ability to model long-term sequence dependencies, and enhances the stability and scalability of the model in the long-term sea surface temperature evolution prediction task; at the same time, multi-scale loss and frequency domain constraints are introduced in the training stage, thereby improving the local texture clarity and temporal consistency of the prediction frame while maintaining the overall trend prediction accuracy;

[0098] (2) The present invention adopts an end-to-end deep learning framework, which can achieve high-precision prediction of complex ocean spatiotemporal sequences without the need for manual feature design. It has good generalization ability and engineering feasibility. It can not only be effectively used for sea surface temperature (SST) prediction, but also be extended to tasks such as climate trend analysis and complex ocean dynamic field evolution modeling. It can also be widely used in marine climate forecasting, fishery resource monitoring, maritime shipping safety early warning and related marine information application fields. Attached Figure Description

[0099] Figure 1 This is a flowchart of a sea surface temperature prediction method based on long-term dependency modeling and cross-branch fusion according to an embodiment of the present invention.

[0100] Figure 2 A structural diagram of a depth convolutional encoder provided according to an embodiment of the present invention;

[0101] Figure 3 This is a structural diagram of a wavelet attention module provided according to an embodiment of the present invention;

[0102] Figure 4 This is a structural diagram of a cross-branch attention fusion module provided according to an embodiment of the present invention. Detailed Implementation

[0103] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.

[0104] It should be noted that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0105] Example 1:

[0106] This invention discloses a sea surface temperature prediction method based on long-term dependency modeling and cross-branch fusion, with reference to... Figure 1 As shown, the specific steps include the following:

[0107] S1, normalize the acquired sea surface temperature time series image data to obtain a standardized sea surface temperature time series image sequence;

[0108] S2, input the standardized sea surface temperature time-series image sequence into the trained multi-scale sea surface temperature prediction model:

[0109] S2.1, use a deep convolutional encoder to encode the spatial features of the standardized sea surface temperature time-series image sequence to obtain multi-level spatial features;

[0110] S2.2, perform two-dimensional discrete wavelet decomposition on the multi-level spatial features to obtain low-frequency approximate representation, horizontal singular features, vertical singular features and diagonal edge features;

[0111] S2.3, The multi-level spatial features, low-frequency approximate representation, horizontal singular features, vertical singular features and diagonal edge features are input into the wavelet attention module to obtain short-term temporal enhancement features;

[0112] S2.4, The multi-level spatial features and low-frequency approximation are input into the ConvLSTM module to obtain high-frequency dynamic enhancement features;

[0113] S2.5, the short-term temporal enhancement features and high-frequency dynamic enhancement features are input into the cross-branch attention fusion module to obtain the fused features;

[0114] S2.6, The fused features are linearly projected and input into the Informer temporal encoder to obtain temporally dependent enhanced features;

[0115] S2.7, Input the temporal dependency enhancement features into the decoder to obtain the prediction result.

[0116] In step S1, this embodiment uses hourly sea surface temperature data of the Atlantic Ocean generated by the numerical model of the Nucleus for European Modeling of the Ocean (NEMO), with a spatial resolution of 0.083°×0.083°. The daily average sea surface temperature data of the Atlantic Ocean obtained in this embodiment is 481×781 pixels in size, spanning from December 28, 2006 to April 5, 2017, a total of 3734 days, and 29 sub-regions of size 64×64 were extracted from it as training and testing sets. To avoid the negative impact caused by data differences, this embodiment uses standardization to preprocess the data. This embodiment uses the Z-score standardization method to preprocess the data. Z-score standardization is a commonly used standardization method applicable to most data types and distributions. In this embodiment, the data is divided into training and test sets. The mean and standard deviation are calculated on the training set, and then these statistics are used to standardize the test set. This avoids introducing training set information into the test set, thus ensuring the model's generalization ability in practical applications. The standardized sea surface temperature time series image sequence is obtained using the following formula:

[0117] ,in, The mean of the sea surface temperature time series image data. The variance of the sea surface temperature time series image data. This represents time-series sea surface temperature image data. This represents a standardized time-series image sequence of sea surface temperature.

[0118] In step S2, this embodiment proposes a multi-scale sea surface temperature prediction model (hereinafter referred to as WaveSANet-Informer). The data processing flow of this model includes steps S2.1 to S2.7; see details below. Figure 2 As shown.

[0119] In step S2.1, the depth convolutional encoder includes a sequentially connected input layer, multiple convolutional blocks, and an output layer; each convolutional block includes a sequentially connected convolutional layer, a normalization layer, and a nonlinear activation layer. Sea surface temperature data at any time by a depth convolutional encoder Mapped to a latent space where wavelet features and complementary features are separable In this process, multi-level spatial features of sea surface temperature images are obtained. .

[0120] In step S2.2, After two-dimensional discrete wavelet decomposition ( It consists of one low-frequency sub-band and three high-frequency detail sub-bands in different directions, namely... Low-frequency approximation Exceptional features in the horizontal and vertical directions and as well as Diagonal edge features The formula is: .

[0121] In step S2.3, refer to Figure 3 As shown, the wavelet attention module is specifically responsible for enhancing global correlation across locations using an attention mechanism, modeling low-frequency trends and multi-directional high-frequency information. The input of the wavelet attention module is... Multi-layered spatial characteristics of time and Short-term temporal enhancement features at time points By fusing features from multiple frequency bands and short-term time series and using wavelet self-attention modeling, short-term time series enhanced features are output.

[0122] To improve the model's ability to describe short-term dependencies in continuous sea surface temperature variations, this embodiment introduces a gated timing enhancement unit based on a gated structure; this structure receives... Multi-layered spatial characteristics of time and Short-term temporal enhancement features at time points As input, the adaptive fusion of historical states and current features is achieved through a combination of reset gates, update gates, and candidate activations.

[0123] First, generate using convolution operations. Reset the output of the gate at all times and Update the gate output constantly This controls the proportion of historical memory deletion and the proportion of current state update, respectively.

[0124] ,

[0125] ,

[0126] in, This represents the Sigmoid activation function, which sets the threshold value to the interval (0,1). This represents the trainable weight matrix of the reset gate. This represents the trainable weight matrix of the update gate;

[0127] Subsequently, according to Reset the output of the gate at all times and Multi-layered spatial characteristics of time ,generate Candidate activation state at any time :

[0128] ,

[0129] in, This represents the trainable weight matrix used to generate candidate activation states. Represents the hyperbolic tangent function;

[0130] Finally, the result was obtained through a gating weighting method. Timing-gated timing enhancement features :

[0131] .

[0132] This process enables the model to adaptively retain or discard historical features at different time steps based on the evolution of the sea surface temperature field, thereby effectively enhancing local dynamic correlation.

[0133] Since wavelet subbands typically have a small number of channels after decomposition and cannot directly participate in attention operations, this embodiment utilizes 1×1 convolution to expand the channels for low-frequency approximation. Horizontal singular features Vertical singularity features and diagonal edge features Channel expansion is performed to obtain the expanded low-frequency approximation. Horizontal singular features Vertical singularity features and diagonal edge features The formula is as follows:

[0134] ,

[0135] ,

[0136] ,

[0137] ,

[0138] To jointly utilize wavelet frequency domain information and short-term time series information, this embodiment concatenates the expanded four types of wavelet subbands with gated time series enhancement features along the channel dimension, forming a multi-band, multi-scale, and time-dependent feature. Time-based comprehensive feature representation :

[0139] ,in, This represents the concatenation function; the fusion process will: The resulting low-frequency trend The high-frequency structural information in the anisotropic structure, The short-term dynamic information in the gating mechanism is integrated into a common feature space, providing rich context for the global dependency modeling of the attention mechanism.

[0140] To further capture the correlation between distant locations based on the fused features, this embodiment introduces a wavelet self-attention operator based on a self-attention mechanism for joint spatial-frequency domain representation learning. This wavelet self-attention operator first applies the fused features... Three independent 1×1 convolutional mappings are applied to generate the query matrix Q, key matrix K, and value matrix V, respectively. Attention weights are then calculated based on a similarity metric to achieve global modeling of the correlations between different locations and frequency bands in the sea surface temperature field, resulting in... Short-term temporal enhancement features at time points :

[0141] ,in, This represents the wavelet self-attention operator.

[0142] In step S2.4, in order to enhance the expression of high-frequency components, this embodiment focuses on multi-level spatial features. and low-frequency approximate subband Perform a difference operation to construct explicit residual characteristics:

[0143] ,in, express Multi-layered spatial characteristics of time. express The low-frequency approximation of time. express The explicit residual features at each time point are used to highlight local perturbations and high-frequency texture variations in the sea surface temperature field. This explicit residual effectively enhances the response in high-dynamic regions, allowing the right branch to focus on modeling rapidly changing ocean dynamic structures.

[0144] Explicit residual characteristics Inputting the Convolutional Long Short-Term Memory (ConvLSTM) network, we perform temporal modeling on it to obtain high-frequency dynamic enhancement features:

[0145] ,in, express The high-frequency dynamic enhancement features at any given time include the temporal enhancement representation of high-frequency perturbations; express High-frequency dynamic enhancement features at any given time are used to characterize historical high-frequency information; This refers to a convolutional long short-term memory unit that updates temporal features through input gates, forget gates, output gates, and gated convolutional structures.

[0146] By utilizing the convolutional gating mechanism of ConvLSTM, this embodiment can maintain the neighborhood structure correlation in the spatial dimension while performing fine-grained modeling of the temporal evolution of local high-frequency perturbations, thereby obtaining a more robust high-frequency dynamic feature representation. This feature will then interact bidirectionally with the multi-band structural features of the left branch in the subsequent cross-branch attention module, achieving joint modeling of trend information and perturbation information.

[0147] In step S2.5, refer to Figure 4 As shown, the step of inputting the short-term temporal enhancement features and high-frequency dynamic enhancement features into the cross-branch attention fusion module to obtain fused features includes:

[0148] Short-term temporal enhancement features are projected into a query vector using three independent 1×1 convolutional kernels. Key vector Sum value vector ;

[0149] High-frequency dynamically enhanced features are projected into a query vector using three independent 1×1 convolutional kernels. Key vector Sum value vector ;

[0150] For query vector Key vector Sum value vector Perform cross-branch attention calculations to obtain attention aggregation features. ;

[0151] Attention aggregation features After linear transformation, a residual connection is made with short-term temporal enhancement features to obtain the wavelet branch query ConvLSTM branch output. :

[0152] ,

[0153] ,

[0154] Where T represents transpose. This represents the softmax activation function. Let the first linear transformation matrix be represented. This indicates short-term temporal enhancement features. Indicates the scaling factor;

[0155] Key vector Sum value vector and query vector Perform cross-branch attention calculations to obtain attention aggregation features. ;

[0156] Attention aggregation features After linear transformation, a residual connection is made with the high-frequency dynamic enhancement feature to obtain the ConvLSTM branch query wavelet branch output. :

[0157] ,

[0158] ,in, Denotes the second linear transformation matrix. Indicates high-frequency dynamic enhancement features, Indicates the scaling factor;

[0159] The wavelet query residual output Wavelet output of residual query Perform adaptive weighted fusion to obtain fused features. :

[0160] ,

[0161] ,

[0162] in, Indicates adaptive weights, This represents the Sigmoid activation function. This represents a multilayer perceptron network. This represents the global average pooling operator.

[0163] In step S2.6, to adapt the spatial features to the sequence input structure of the Informer temporal encoder, this embodiment performs linear mapping or spatial compression to obtain a temporal vector: ,

[0164] in, This represents a linear projection, where D is the hidden dimension of the Informer.

[0165] The T time steps are stacked to form the input sequence of the Informer encoder:

[0166] ,in, The sequence length representing the fused features. After linear projection The fusion characteristics of moments This represents the input of the Informer timing encoder;

[0167] The Informer encoder outputs a temporal dependency augmentation sequence with the same dimension as the input:

[0168] ,in, Indicates Informer timing encoder. Indicates time-dependent augmentation features, express Temporal dependency enhancement features at specific moments.

[0169] The Informer encoder is a stacked structure consisting of multiple layers of Probsparse Self-Attention and a feedforward neural network. Through its unique Probsparse Self-Attention mechanism, the Informer encoder sparsifies self-attention computation, greatly improving computational efficiency. The Informer encoder mainly includes:

[0170] (1) Sparse self-attention mechanism

[0171] Traditional self-attention mechanisms require calculating the similarity between all time steps, resulting in a computational complexity of O(n log n). ,in This refers to the number of time steps. Probsparse Self-Attention reduces computational complexity by selecting important attention distributions; its computational complexity is O(n). The specific steps are as follows:

[0172] 1) Generation of queries and keys:

[0173] For the input sequence Perform matrix calculations to generate a query matrix. s and keys :

[0174] ,

[0175] in, , , It is a trainable matrix.

[0176] 2) Calculate the coefficient attention distribution:

[0177] Using a probabilistic sparsity method, only the important components of the attention distribution are retained. This is achieved by selecting the components that contribute the most to attention. For each query, calculate its similarity to the key:

[0178] First, define the... Sparsity score of each query vector Query :

[0179] ,

[0180] in, Indicates the first A query vector, Indicates the first Transpose of a key vector The feature dimension of the key;

[0181] Choose the highest rated A collection of queries ,right Calculate attention weights:

[0182] ,in, For a sparse attention weight matrix, It is the dimension of the key.

[0183] 3) Weighted summation of values:

[0184] The output of Probsparse Self-Attention is obtained by weighted summation of the attention weights and their corresponding values. :

[0185] ;

[0186] in, It is a matrix obtained by linear transformation of the input sequence. , It is a trainable weight matrix.

[0187] (2) Informer encoder processing flow

[0188] 1) Input embedding:

[0189] The fused feature sequence obtained in the previous step is input into the Informer encoder, and it is converted into a high-dimensional representation through the embedding layer.

[0190] 2) Multi-head self-attention mechanism:

[0191] Multi-head attention computation is performed on the input sequence. This mechanism captures subspace information of the input sequence by computing multiple independent attention heads in parallel.

[0192] ;

[0193] Among them, each attention head It is an independent self-attention computation. It is a trainable weight matrix.

[0194] 3) Feedforward neural networks and residual connections

[0195] The attention output is processed using a feedforward neural network:

[0196] ;

[0197] in, This represents the output of the feedforward neural network. This represents the input features entering the feedforward neural network layer. and It is a weight matrix. and It is a bias vector. This is the activation function. Each self-attention layer and feedforward neural network layer is followed by residual connections and layer normalization to help stabilize model training and accelerate convergence.

[0198] ;

[0199] here, Represents residual connectivity. The sub-layer operation can be represented as a multi-layer self-attention mechanism or a feedforward neural network.

[0200] Through the above processing flow, the Informer encoder can extract the temporal features of sea surface temperature time series, which reflect the complex temporal dependencies in the data. The output of the Informer encoder incorporates latent temporal feature maps of long-term dependencies, including sea surface temperature over the past... The output includes intra-frame trend information, anomalous dynamics, and periodic change patterns. This output will be directly fed into the decoder for sea surface temperature prediction and reconstruction at the next time step.

[0201] In step S2.7, this embodiment designs a decoder structure corresponding to the deep convolutional encoder, which is used to restore the high-dimensional latent features to the pixel space, so as to realize the mapping and reconstruction of spatiotemporal features to physical fields.

[0202] In its design, the decoder employs a convolution-upsampling structure symmetrical to the encoder to ensure gradual recovery of spatial features and reconstruction of details. The input is a temporally encoded output, which is first projected onto the initial channel dimension of the decoder through a linear mapping, followed by layer normalization and activation functions. Subsequently, the decoder recovers the spatial resolution step-by-step through a series of upsampling and convolution operations. Each decoding stage includes upsampling, convolution, normalization, and residual connection modules, the specific calculation of which can be written as follows:

[0203] ;

[0204] in, Indicates the current decoder's... The output feature map of the level, It is the Sigmoid activation function. For batch normalization, It is a 3×3 convolutional layer. This indicates a deconvolution upsampling operation; for The residual mapping function is used for channel and scale matching; through layer-by-layer upsampling, the decoder finally restores the features from low resolution to the spatial scale of the original sea surface temperature field.

[0205] Finally, The input is decoded to reconstruct the sea surface temperature prediction result for the next moment, thereby enabling the prediction of the future sea surface temperature.

[0206] ,in, Indicates decoder, express The prediction results for the time period, express Temporal dependency enhancement features at specific moments.

[0207] This embodiment compares the proposed WaveSANet-Informer with several advanced spatiotemporal sequence prediction networks in recent years on two datasets: TaxiBJ (Beijing taxi traffic dataset) and SST (Sea Surface Temperature dataset). The evaluation metrics include mean squared error (MSE), root mean square error (MAE), and structural similarity (SSIM).

[0208] SSIM is a metric for measuring the similarity between two images. Given two images... and Structural similarity between two images It can be calculated in the following way:

[0209] ;

[0210] in, yes The average value, yes The average value, yes variance yes variance yes and covariance, It is a constant used to maintain stability; , ; It is the dynamic range of pixel values; , The scope of structural similarity (SSIM) is as follows: When two images are exactly the same, the value of SSIM is equal to 1.

[0211] The optimal result in this embodiment is highlighted in bold, while the second-best result is highlighted in underline. Table 1 shows a quantitative comparison of the prediction results of different models for TaxiBJ, including ConvLSTM, PredRNN (Predictive Recurrent Neural Network), LSTM, E3D-LSTM (EIDETIC 3D Long Short-Term Memory), MIM (Masked Image Modeling), PhyDNet (Physical Dynamics Network), SA-ConvLSTM (Self-Attention ConvLSTM), TaylorNet (Symplectic Taylor Neural Networks), and WaveSANet-informer in this embodiment. According to the data in the table, WaveSANet-informer outperforms or reaches the current state-of-the-art prediction models in various metrics.

[0212] Table 1 Quantitative comparison on TaxiBJ

[0213]

[0214] Table 2 shows a quantitative comparison on the SST dataset, where WaveSANet-informer outperforms TaylorNet, the state-of-the-art model on the SST dataset. Overall, the comparison on both datasets demonstrates that this embodiment performs exceptionally well on real-world datasets. This is likely because real-world data often contains more complex data structures and distributions, as well as noise, outliers, and incomplete data. These noises and variations increase data diversity, thereby making the model more robust and capable of generalization.

[0215] Table 2 Quantitative comparisons on SST

[0216]

[0217] In summary, the long-term dependency modeling and cross-branch fusion method for sea surface temperature (SST) prediction provided in this embodiment first normalizes the input SST time-series image data to obtain a standardized multi-frame sequence. Then, each SST frame is input into a deep convolutional encoder to extract multi-layer spatial features and map them to the latent space. Next, a two-dimensional discrete wavelet transform (2D-DWT) is performed on the encoded features, decomposing them into low-frequency trend subbands and multi-directional high-frequency detail subbands. The decomposed multi-scale features are input into a dual-branch network. A cross-branch attention module is introduced before fusion of the branch outputs. The fused feature sequence from multiple time steps is then input into an Informer temporal encoder, which uses a sparse self-attention mechanism to capture long-term dependencies and generate a global temporal feature representation. Finally, the temporal features are input into a decoder, which uses deconvolution and upsampling operations to restore spatial resolution and generate the SST prediction frame for the next time step. During the training phase, structural similarity (SSIM) loss is calculated simultaneously for both the predicted and ground truth frames. Through multi-scale joint optimization, a balanced prediction of global trends and local details can be achieved. It can be effectively used for sea surface temperature (SST) prediction, climate trend analysis and spatiotemporal sequence prediction models of complex ocean dynamic fields, and can be widely applied in fields such as marine climate forecasting, fishery resource monitoring and maritime shipping safety early warning.

[0218] Example 2: Based on the same inventive concept as Example 1, this embodiment of the invention discloses a sea surface temperature prediction system based on long-term dependency modeling and cross-branch fusion, comprising:

[0219] The normalization module is used to normalize the acquired sea surface temperature time series image data to obtain a standardized sea surface temperature time series image sequence.

[0220] The multi-scale sea surface temperature prediction module is used to: input the standardized sea surface temperature time-series image sequence into the trained multi-scale sea surface temperature prediction model.

[0221] The standardized sea surface temperature time-series image sequence is spatially encoded using a deep convolutional encoder to obtain multi-level spatial features;

[0222] Two-dimensional discrete wavelet decomposition is performed on the multi-level spatial features to obtain low-frequency approximate representation, horizontal singular features, vertical singular features, and diagonal edge features;

[0223] The multi-level spatial features, low-frequency approximation, horizontal singular features, vertical singular features, and diagonal edge features are input into the wavelet attention module to obtain short-term temporal enhancement features;

[0224] The multi-level spatial features and low-frequency approximation are input into the ConvLSTM module to obtain high-frequency dynamic enhancement features;

[0225] The short-term temporal enhancement features and high-frequency dynamic enhancement features are input into the cross-branch attention fusion module to obtain the fused features;

[0226] The fused features are linearly projected and then input into the Informer temporal encoder to obtain temporally dependent enhanced features.

[0227] The temporal-dependent enhancement features are input into the decoder to obtain the prediction results.

[0228] The specific functions of each module described above are explained in the relevant content of the method in Embodiment 1, and will not be repeated here.

[0229] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0230] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0231] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0232] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0233] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A sea surface temperature prediction method that relies on long-term modeling and cross-branch fusion, characterized in that, include: The acquired sea surface temperature time-series image data were normalized to obtain a standardized sea surface temperature time-series image sequence. The standardized sea surface temperature time-series image sequence is input into the trained multi-scale sea surface temperature prediction model: The standardized sea surface temperature time-series image sequence is spatially encoded using a deep convolutional encoder to obtain multi-level spatial features; Two-dimensional discrete wavelet decomposition is performed on the multi-level spatial features to obtain low-frequency approximate representation, horizontal singular features, vertical singular features, and diagonal edge features; The multi-level spatial features, low-frequency approximation, horizontal singular features, vertical singular features, and diagonal edge features are input into the wavelet attention module to obtain short-term temporal enhancement features; The multi-level spatial features and low-frequency approximation are input into the ConvLSTM module to obtain high-frequency dynamic enhancement features; The short-term temporal enhancement features and high-frequency dynamic enhancement features are input into the cross-branch attention fusion module to obtain the fused features; The fused features are linearly projected and then input into the Informer temporal encoder to obtain temporally dependent enhanced features. The temporal-dependent enhancement features are input into the decoder to obtain the prediction result; The process of inputting the multi-level spatial features, low-frequency approximation, horizontal singular features, vertical singular features, and diagonal edge features into the wavelet attention module to obtain short-term temporal enhancement features includes: Will Multi-layered spatial characteristics of time and Short-term temporal enhancement features at time points Input into the reset gate, get Reset the output of the gate at all times ; Will Multi-layered spatial characteristics of time and Short-term temporal enhancement features at time points Input into the update gate, get Update the gate output constantly ; According to the above Reset the output of the gate at all times and Multi-layered spatial characteristics of time ,generate Candidate activation state at any time ; According to the above Candidate activation state at any time and Update the gate output constantly ,generate Timing-gated timing enhancement features ; Using 1×1 convolution to approximate low frequencies Horizontal singular features Vertical singularity features and diagonal edge features Channel expansion is performed to obtain the expanded low-frequency approximation. Horizontal singular features Vertical singularity features and diagonal edge features ; The Timing-gated timing enhancement features Extended low-frequency approximation Horizontal singular features Vertical singularity features and diagonal edge features By splicing along the channel dimension, we obtain Time-based comprehensive feature representation ; The Time-based comprehensive feature representation The input is fed into a wavelet self-attention operator for joint spatial and frequency domain representation learning, resulting in... Short-term temporal enhancement features at time points .

2. The sea surface temperature prediction method based on long-term dependency modeling and cross-branch fusion according to claim 1, characterized in that, The standardized sea surface temperature time-series image sequence is obtained by the following formula: ， in, The mean of the sea surface temperature time series image data. The variance of the sea surface temperature time series image data. This represents time-series sea surface temperature image data. This represents a standardized time-series image sequence of sea surface temperature.

3. The sea surface temperature prediction method based on long-term dependency modeling and cross-branch fusion according to claim 1, characterized in that, The depth convolutional encoder includes a sequentially connected input layer, multiple convolutional blocks, and an output layer; Each of the convolutional blocks includes sequentially connected convolutional layers, normalization layers, and nonlinear activation layers.

4. The sea surface temperature prediction method based on long-term dependency modeling and cross-branch fusion according to claim 1, characterized in that, The Reset the output of the gate at all times It can be obtained through the following formula: ， in, This represents the trainable weight matrix of the reset gate. This represents the Sigmoid activation function; The Update the gate output constantly It can be obtained through the following formula: ， in, This represents the trainable weight matrix of the update gate; The Candidate activation state at any time It can be obtained through the following formula: ， in, This represents the trainable weight matrix used to generate candidate activation states. Represents the hyperbolic tangent function; The Timing-gated timing enhancement features It can be obtained through the following formula: ； The Time-based comprehensive feature representation It can be obtained through the following formula: ， in, Indicates the concatenation function; The Short-term temporal enhancement features at time points It can be obtained through the following formula: ， in, This represents the wavelet self-attention operator.

5. The sea surface temperature prediction method based on long-term dependency modeling and cross-branch fusion according to claim 1, characterized in that, The step of inputting the multi-level spatial features and low-frequency approximation into the ConvLSTM module to obtain high-frequency dynamic enhancement features includes: Perform a difference operation on the multi-level spatial features and the low-frequency approximation to obtain explicit residual features; The explicit residual features are input into the Convolutional Long Short-Term Memory (ConvLSTM) network for temporal modeling to obtain high-frequency dynamic enhancement features; The explicit residual characteristics are obtained by the following formula: ， in, express Multi-layered spatial characteristics of time. express The low-frequency approximation of time. express Explicit residual characteristics at time points; The high-frequency dynamic enhancement feature is obtained through the following formula: ， in, express High-frequency dynamic enhancement features at any given time. express High-frequency dynamic enhancement features at any given time.

6. The sea surface temperature prediction method based on long-term dependency modeling and cross-branch fusion according to claim 1, characterized in that, The step of inputting the short-term temporal enhancement features and the high-frequency dynamic enhancement features into the cross-branch attention fusion module to obtain fused features includes: Short-term temporal enhancement features are projected into a query vector using three independent 1×1 convolutional kernels. Key vector Sum value vector ; High-frequency dynamically enhanced features are projected into a query vector using three independent 1×1 convolutional kernels. Key vector Sum value vector ; For query vector Key vector Sum value vector Perform cross-branch attention calculations to obtain attention aggregation features. ; Attention aggregation features After linear transformation, a residual connection is made with short-term temporal enhancement features to obtain the wavelet branch query ConvLSTM branch output. ; Key vector Sum value vector and query vector Perform cross-branch attention calculations to obtain attention aggregation features. ; Attention aggregation features After linear transformation, a residual connection is made with the high-frequency dynamic enhancement feature to obtain the ConvLSTM branch query wavelet branch output. ; The wavelet query residual output Wavelet output of residual query Perform adaptive weighted fusion to obtain fused features. ; The wavelet query residual output It can be obtained through the following formula: ， ， Where T represents transpose. This represents the softmax activation function. Let the first linear transformation matrix be represented. This indicates short-term temporal enhancement features. Indicates the scaling factor; The residual query wavelet output It can be obtained through the following formula: ， ， in, Denotes the second linear transformation matrix. Indicates high-frequency dynamic enhancement features; The fusion feature It can be obtained through the following formula: ， ， in, Indicates adaptive weights, This represents the Sigmoid activation function. This represents a multilayer perceptron network. This represents the global average pooling operator.

7. The sea surface temperature prediction method based on long-term dependency modeling and cross-branch fusion according to claim 1, characterized in that, The time-dependent enhancement sequence is obtained by the following formula: ， ， in, Indicates fusion features, Indicates linear projection. The sequence length representing the fused features. After linear projection The fusion characteristics of moments This represents the input of the Informer timing encoder. Indicates Informer timing encoder. Indicates time-dependent augmentation features, express Temporal dependency enhancement features at specific moments.

8. The sea surface temperature prediction method based on long-term dependency modeling and cross-branch fusion according to claim 1, characterized in that, The prediction result is obtained using the following formula: ， in, Indicates decoder, express The prediction results for the time period, express Temporal dependency enhancement features at specific moments.

9. A sea surface temperature prediction system that relies on long-term modeling and cross-branch fusion, characterized in that, include: The normalization module is used to normalize the acquired sea surface temperature time series image data to obtain a standardized sea surface temperature time series image sequence. The multi-scale sea surface temperature prediction module is used to: input the standardized sea surface temperature time-series image sequence into the trained multi-scale sea surface temperature prediction model. The standardized sea surface temperature time-series image sequence is spatially encoded using a deep convolutional encoder to obtain multi-level spatial features; Two-dimensional discrete wavelet decomposition is performed on the multi-level spatial features to obtain low-frequency approximate representation, horizontal singular features, vertical singular features, and diagonal edge features; The multi-level spatial features, low-frequency approximation, horizontal singular features, vertical singular features, and diagonal edge features are input into the wavelet attention module to obtain short-term temporal enhancement features; The multi-level spatial features and low-frequency approximation are input into the ConvLSTM module to obtain high-frequency dynamic enhancement features; The short-term temporal enhancement features and high-frequency dynamic enhancement features are input into the cross-branch attention fusion module to obtain the fused features; The fused features are linearly projected and then input into the Informer temporal encoder to obtain temporally dependent enhanced features. The temporal-dependent enhancement features are input into the decoder to obtain the prediction result; The process of inputting the multi-level spatial features, low-frequency approximation, horizontal singular features, vertical singular features, and diagonal edge features into the wavelet attention module to obtain short-term temporal enhancement features includes: Will Multi-layered spatial characteristics of time and Short-term temporal enhancement features at time points Input into the reset gate, get Reset the output of the gate at all times ; Will Multi-layered spatial characteristics of time and Short-term temporal enhancement features at time points Input into the update gate, get Update the gate output constantly ; According to the above Reset the output of the gate at all times and Multi-layered spatial characteristics of time ,generate Candidate activation state at any time ; According to the above Candidate activation state at any time and Update the gate output constantly ,generate Timing-gated timing enhancement features ; Using 1×1 convolution to approximate low frequencies Horizontal singular features Vertical singularity features and diagonal edge features Channel expansion is performed to obtain the expanded low-frequency approximation. Horizontal singular features Vertical singularity features and diagonal edge features ; The Timing-gated timing enhancement features Extended low-frequency approximation Horizontal singular features Vertical singularity features and diagonal edge features By splicing along the channel dimension, we obtain Time-based comprehensive feature representation ; The Time-based comprehensive feature representation The input is fed into a wavelet self-attention operator for joint spatial and frequency domain representation learning, resulting in... Short-term temporal enhancement features at time points .

Images