Method for predicting ocean surface temperature sequence based on spatio-temporal double-flow non-stationary perception

Abstract

This invention belongs to the field of image processing technology and discloses a method for predicting ocean surface temperature sequences based on spatiotemporal dual-stream non-stationary sensing. It constructs and trains an ocean surface temperature sequence prediction network based on spatiotemporal dual-stream non-stationary sensing. The network includes a spatial non-stationary feature modeling sub-network, a temporal non-stationary feature modeling sub-network, and a spatiotemporal fusion module based on stacked cross-attention. The satellite remote sensing image of ocean surface temperature to be processed is input into the trained ocean surface temperature sequence prediction network based on spatiotemporal dual-stream non-stationary sensing, and the ocean surface temperature prediction result is output. This invention can simultaneously mine non-stationary features in time and space, effectively explore the spatiotemporal correlation of ocean surface temperature, and improve the accuracy of ocean surface temperature prediction.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a method for predicting ocean surface temperature sequences based on spatiotemporal dual-stream non-stationary sensing. Background Technology

[0002] Ocean surface temperature (SST) prediction methods can be broadly categorized into numerical methods and data-driven methods. Numerical methods use a series of complex physical, chemical, and biological parameters to build mathematical models that predict SST changes; due to the large number of parameters, they typically require very high computational costs. Data-driven methods, on the other hand, focus on solving the SST prediction problem by learning patterns of SST variation from historical SST data to construct prediction models. Compared to numerical methods, data-driven methods do not require extensive knowledge of the ocean and atmosphere and can predict sea surface temperature at a smaller scale and with higher resolution.

[0003] The main data-driven methods include traditional statistical methods and machine learning methods. With the development of deep learning, most current SST prediction methods adopt spatiotemporal prediction models based on recurrent neural networks. These models capture non-stationary features through simple state transitions, establishing simple spatial dependencies while simultaneously establishing temporal dependencies. However, existing methods have the following problems: First, they only focus on building correlations in time series, failing to fully mine the spatial information of ocean surface temperature. Existing ocean surface temperature prediction models often use recurrent structures, performing simple feature extraction on spatial information followed by temporal feature aggregation. However, spatiotemporal data exhibits spatial autocorrelation, and simple feature extraction cannot fully capture potential spatial correlations. Therefore, further in-depth mining of spatial information is needed based on features at different scales. Second, ocean physical spatiotemporal dynamics exhibit complex non-stationarities in both time and space. For example, El Niño and the Southern Oscillation (SOO) can cause abnormal increases in ocean surface temperature. Existing methods cannot simultaneously and adequately model the non-stationarities of ocean surface temperature in both time and space, thus failing to infer uncertainties in spatiotemporal dynamics. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for predicting ocean surface temperature (SST) sequences based on spatiotemporal dual-flow non-stationary sensing. A prediction network based on dual flows is constructed. First, a diagonal recurrent neural network composed of stacked MIM modules captures temporal non-stationary features in the spatiotemporal sequence. Then, parallel sub-networks for modeling temporal and spatial non-stationary features are constructed to simultaneously mine non-stationary features in both time and space. Finally, the temporal and spatial integrated features obtained from the two sub-networks are adaptively learned using a stacked cross-attention spatiotemporal fusion module to learn the weights of temporal and spatial dependencies on the SST field, effectively mining the spatiotemporal correlation of SST and obtaining the final SST prediction result, thus improving the accuracy of SST prediction.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A method for predicting ocean surface temperature sequences based on spatiotemporal dual-stream non-stationary sensing includes the following steps: Step S1, acquiring satellite remote sensing images of ocean surface temperature;

[0007] Step S2: Construct and train an ocean surface temperature sequence prediction network based on spatiotemporal dual-stream nonstationary perception, including a spatial nonstationary feature modeling sub-network, a temporal nonstationary feature modeling sub-network, and a spatiotemporal fusion module based on stacked cross-attention.

[0008] The time-non-stationary feature modeling subnetwork consists of ST-LSTM modules and MIM modules, including a multi-layer network structure with identical architecture. Each layer includes one ST-LSTM module and multiple MIM modules. Modules in the upper and lower layers are stacked and interconnected via zigzag paths. ST-LSTM modules or MIM modules at corresponding positions in each layer are connected via vertical paths. The input is ocean surface temperature satellite remote sensing images X1, X2, ..., X t The ocean surface temperature (SST) time series features are obtained by passing the ST-LSTM through each layer. The hidden states of the outputs of two vertically adjacent modules are passed to the next layer module through a diagonal path. In the MIM module, a difference-based method is used to capture the non-stationary information in the SST time series features. The non-stationary information is captured by a stacked diagonal structure, and the time series non-stationary features of SST are modeled. Finally, the outputs y1, y2, ..., y of the last MIM module in each layer are processed. t Aggregate the data to obtain the SST time-comprehensive feature β;

[0009] The spatial non-stationary feature modeling subnetwork consists of a spatial non-stationary feature extraction module and a spatial feature fusion module, specifically:

[0010] Within the spatial non-stationary feature extraction module, an M*M size satellite remote sensing image of ocean surface temperature is downsampled into feature maps of three scales: M*M, (M*M) / 4, and (M*M) / 8. A convolution operation is performed on the (M*M) / 4 and (M*M) / 8 satellite remote sensing images to capture the small-scale non-stationary feature n. Similarly, a convolution operation is performed on the M*M and (M*M) / 4 satellite remote sensing images to capture the large-scale non-stationary feature N.

[0011] Within the spatial feature fusion module, the original feature map S of M*M, small-scale non-stationary features n, and large-scale non-stationary features N are aggregated. The resulting spatial aggregated features are processed by the sigmoid function and the tanh function, respectively. The sigmoid function strengthens the insignificant non-stationary features in the space to prevent loss and retains the complete spatial comprehensive features. Finally, the corresponding elements of the two function processing results are multiplied to obtain the SST spatial comprehensive feature α.

[0012] The SST temporal comprehensive feature β obtained from the temporal nonstationary feature modeling subnetwork and the SST spatial comprehensive feature α from the spatial nonstationary feature modeling subnetwork are fused through a stacked cross-attention spatiotemporal fusion module. Within the stacked cross-attention spatiotemporal fusion module, the SST spatial comprehensive feature α and the SST temporal comprehensive feature β adaptively learn their own weights for the ocean surface temperature field, thus fusing temporal and spatial information.

[0013] Step S3: Input the satellite remote sensing image of ocean surface temperature to be processed into the trained ocean surface temperature sequence prediction network based on spatiotemporal dual-stream non-stationary sensing, and output the ocean surface temperature prediction result.

[0014] Furthermore, the calculation process for the SST spatial synthesis feature α is as follows:

[0015] o = tanh(W os *S+W oN *N+W on *n)

[0016] g=σ(W gs *S+W gN *N+W gn *n)

[0017]

[0018] Among them W os W oN W on W gs W gN W gn* represents the convolution parameters; * represents the convolution operation; σ is a sigmoid function, applied to the weighted sum of the inputs as an activation function; tanh represents the tanh activation function; n represents small-scale non-stationary features; N represents large-scale non-stationary features; S represents the original feature map; denoted as Hadamard product; o represents the result of aggregating the original feature map S, small-scale non-stationary features n, and large-scale non-stationary features N and then passing it through the tanh function; g represents the result of aggregating the original feature map S, small-scale non-stationary features n, and large-scale non-stationary features N and then passing it through the sigmoid function.

[0019] Furthermore, within the spatiotemporal fusion module based on stacked cross-attention, specifically:

[0020] First, the (M, M)-dimensional SST space comprehensive features α are flattened and mapped using an attention encoding mechanism, respectively, to serve as (M×M, 1)-dimensional query Q. α Sum V α The matrix and the SST temporal comprehensive feature β are also flattened and mapped through an attention encoding mechanism, serving as (M×M,1) dimensional keys K. β Sum V β matrix:

[0021] Q α =W q *α,V α =W v *α

[0022] K β =W k *β,V β =W v *β

[0023] Among them W q W k W v Represents the convolution parameters, * represents the convolution operation, and Q represents the convolution parameters. α V represents the spatial query matrix; α Denotes a spatial value matrix, K β V represents the time key matrix. β Represents a time value matrix;

[0024] Q α and K β Matrix multiplication is performed, and each grid point is modeled using the softmax function to obtain the correlation score e of the SST temporal comprehensive feature β and the SST spatial comprehensive feature α, with dimensions (M×M, M×M), as shown in the following formula:

[0025]

[0026] Where softmax represents the normalization exponential function. Represents matrix multiplication;

[0027] Then, the spatial value matrix V α Matrix multiplication with e on the spatial value matrix V α Modeling each grid point and reshaping the result using the reshape function to obtain the spatial attention feature Z is a method for obtaining spatial attention features Z. α Similarly, the time value matrix V β Matrix multiplication with e on the time-value matrix V β Modeling each grid point and reshaping the result using the reshape function yields the temporal attention feature Z in (M, M) dimensions. β :

[0028]

[0029]

[0030] Where reshape represents the function that transforms the tensor dimensions. V represents matrix multiplication, e represents the correlation score on temporal and spatial features, and V represents matrix multiplication. α V represents the spatial value matrix. β Z represents the time-value matrix. α Z represents spatial attention features. β This indicates the characteristics of time-based attention;

[0031] Finally, by aggregating features and using a multilayer perceptron, the final prediction result of the ocean surface temperature satellite remote sensing image is obtained, as shown below:

[0032] Z = MLP(W z [Z α Z β ])

[0033] Where MLP represents a multilayer perceptron, W Z Z represents the convolution parameters. α Z represents spatial attention features. β Z represents the temporal attention feature, and Z represents the prediction result of ocean surface temperature from satellite remote sensing images.

[0034] Compared with the prior art, the advantages of the present invention are as follows:

[0035] (1) A spatial non-stationary feature modeling sub-network was designed to model the spatial non-stationary features in SST data. First, the input SST spatiotemporal sequence was downsampled to construct spatial features at three scales: small, medium, and large. Then, by performing differential modeling on the multi-scale spatial features, the changes of small-scale and large-scale non-stationary features and large-scale stationary features were fully explored. Finally, a spatial feature fusion module based on a gating mechanism was designed to dynamically model the correlation between changes of non-stationary and stationary features.

[0036] (2) This invention can effectively extract non-stationary features simultaneously in time and space. This invention constructs a prediction network based on dual streams. First, it captures the temporal non-stationary features in the spatiotemporal sequence through a diagonal recurrent neural network composed of stacked MIM modules. Then, it simultaneously mines non-stationary features in time and space by constructing parallel sub-networks for modeling temporal non-stationary features and spatial non-stationary features. Finally, it adaptively learns the weights of temporal and spatial dependencies on the sea surface temperature field by using a stacked cross-attention spatiotemporal fusion module based on the temporal and spatial dependencies obtained from the two sub-networks. The spatiotemporal correlation of sea surface temperature is effectively mined by the stacked cross-attention spatiotemporal fusion module, and the final SST prediction result is obtained, which improves the accuracy of SST prediction. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a network architecture diagram of the present invention;

[0039] Figure 2 This is a schematic diagram of the spatiotemporal fusion module based on stacked cross-attention of the present invention. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0041] This embodiment provides a method for predicting ocean surface temperature sequences based on spatiotemporal dual-stream non-stationary sensing, including the following steps:

[0042] Step S1: Acquire satellite remote sensing images of ocean surface temperature;

[0043] Step S2: Construct and train an ocean surface temperature sequence prediction network based on spatiotemporal dual-stream non-stationary sensing. Step S3: Input the ocean surface temperature satellite remote sensing image to be processed into the trained ocean surface temperature sequence prediction network based on spatiotemporal dual-stream non-stationary sensing, and output the ocean surface temperature prediction result.

[0044] The following is combined Figure 1 This embodiment introduces the ocean surface temperature sequence prediction network structure based on spatiotemporal dual-stream non-stationary sensing, including a spatial non-stationary feature modeling sub-network, a temporal non-stationary feature modeling sub-network, and a spatiotemporal fusion module based on stacked cross-attention. The composition, function, and data processing of each module are described below.

[0045] The subnetwork for modeling temporal non-stationary features employs a recurrent neural network composed of stacked MIM modules to capture temporal non-stationary features. Specifically, it consists of ST-LSTM modules and MIM modules, including a multi-layer network structure with identical architecture. Each layer includes one ST-LSTM module and multiple MIM modules. Modules in the previous layer are stacked and interconnected with modules in the next layer via a zigzag path. ST-LSTM modules or MIM modules at corresponding positions in each layer are connected via a vertical path, and multiple MIM modules are interconnected via a diagonal path. This is used to model the difference information in spatiotemporal prediction. By stacking multiple MIM modules, non-stationary features can be potentially captured from spatiotemporal dynamics, gradually stabilizing the spatiotemporal process and making the predicted future SST time comprehensive features more accurate.

[0046] Input satellite remote sensing images of ocean surface temperature X1, X2, ..., X t The ocean surface temperature (SST) time series features are obtained by passing each layer of the ST-LSTM. However, SST exhibits non-stationary information in natural time and space. Therefore, the hidden states of the outputs of two vertically adjacent modules are passed to the next layer module via a diagonal path. In the MIM module, difference-based operations are used to capture the non-stationary information in the SST time series features. The non-stationary information is captured through a stacked diagonal structure, and the time series non-stationary features of SST are modeled. Finally, the outputs y1, y2, ..., y of the last MIM module in each layer are processed. t Aggregate the data to obtain the SST time-integrated feature β.

[0047] The spatial non-stationary feature modeling subnetwork consists of a spatial non-stationary feature extraction module and a spatial feature fusion module, specifically:

[0048] First, within the spatial non-stationary feature extraction module, due to the slow-changing nature of ocean physical fields, the M*M size satellite remote sensing image of ocean surface temperature is downsampled into feature maps at three scales: M*M, (M*M) / 4, and (M*M) / 8. Larger scale remote sensing images contain richer and more prominent features. A convolution operation is performed on the (M*M) / 4 and (M*M) / 8 satellite remote sensing images to capture the small-scale non-stationary feature n. Similarly, a convolution operation is performed on the M*M and (M*M) / 4 satellite remote sensing images to capture the large-scale non-stationary feature N. Since any non-stationary process can be decomposed into several deterministic time-varying polynomials and a zero-mean random value, appropriately applying the difference operation can reduce the order of the time-varying polynomials, gradually bringing them closer to stationarity. This makes the deterministic and complex trend information predictable, solving the problem of failing to capture spatial non-stationary features.

[0049] Then, within the spatial feature fusion module, the original M*M feature map S, small-scale non-stationary features n, and large-scale non-stationary features N are aggregated. The resulting aggregated spatial features are then processed using the sigmoid function and the tanh function, respectively. The sigmoid function strengthens insignificant non-stationary features in the space to prevent loss, preserving the complete spatial composite features. Finally, the results of the two function processing are multiplied element-wise to obtain the SST spatial composite feature α. The calculation process of the SST spatial composite feature α is shown below:

[0050] o = tanh(W os *S+W oN *N+W on *n)

[0051] g=σ(W gs *S+W gN *N+W gn *n)

[0052]

[0053] Among them W os W oN W on W gs W gN W gn * represents the convolution parameters; * represents the convolution operation; σ is a sigmoid function, applied to the weighted sum of the inputs as an activation function; tanh represents the tanh activation function; n represents small-scale non-stationary features; N represents large-scale non-stationary features; S represents the original feature map; denoted as Hadamard product; o represents the result of aggregating the original feature map S, small-scale non-stationary features n, and large-scale non-stationary features N and then passing it through the tanh function; g represents the result of aggregating the original feature map S, small-scale non-stationary features n, and large-scale non-stationary features N and then passing it through the sigmoid function.

[0054] Finally, the SST temporal comprehensive feature β obtained from the temporal non-stationary feature modeling sub-network and the SST spatial comprehensive feature α from the spatial non-stationary feature modeling sub-network are fused through a stacked cross-attention spatiotemporal fusion module. Within the stacked cross-attention spatiotemporal fusion module, the SST spatial comprehensive feature α and the SST temporal comprehensive feature β adaptively learn their own weights for the ocean surface temperature field, fusing important temporal and spatial information and filtering out unimportant information, making the prediction results more accurate and reliable.

[0055] Combination Figure 2 As shown, within the spatiotemporal fusion module based on stacked cross-attention, specifically:

[0056] First, the (M, M)-dimensional SST space comprehensive features α are flattened and mapped using an attention encoding mechanism, respectively, to serve as (M×M, 1)-dimensional query Q. α Sum V α The matrix and the SST temporal comprehensive feature β are also flattened and mapped through an attention encoding mechanism, serving as (M×M,1) dimensional keys K. β Sum V β matrix:

[0057] Q α =W q *α,V α =W v *α

[0058] K β =W k *β,V β =W v *β

[0059] Among them W q W k W v Represents the convolution parameters, * represents the convolution operation, and Q represents the convolution parameters. α V represents the spatial query matrix; α Denotes a spatial value matrix, K β V represents the time key matrix. β Represents a time value matrix;

[0060] Q α and K βMatrix multiplication is performed, and each grid point is modeled using the softmax function to obtain the correlation score e of the SST temporal comprehensive feature β and the SST spatial comprehensive feature α, with dimensions (M×M, M×M), as shown in the following formula:

[0061]

[0062] Where softmax represents the normalization exponential function. Represents matrix multiplication;

[0063] Then, the spatial value matrix V α Matrix multiplication with e on the spatial value matrix V α Modeling each grid point and reshaping the result using the reshape function to obtain the spatial attention feature Z is a method for obtaining spatial attention features Z. α Similarly, the time value matrix V β Matrix multiplication with e on the time-value matrix V β Modeling each grid point and reshaping the result using the reshape function yields the temporal attention feature Z in (M, M) dimensions. β :

[0064]

[0065]

[0066] Where reshape represents the function that transforms the tensor dimensions. V represents matrix multiplication, e represents the correlation score on temporal and spatial features, and V represents matrix multiplication. α V represents the spatial value matrix. β Z represents the time-value matrix. α Z represents spatial attention features. β This indicates the characteristics of time-based attention;

[0067] Finally, by aggregating features and using a multilayer perceptron, the final prediction result of the ocean surface temperature satellite remote sensing image is obtained, as shown below:

[0068] Z = MLP(W z [Z α Z β ])

[0069] Where MLP represents a multilayer perceptron, W Z Z represents the convolution parameters. α Z represents spatial attention features. β Z represents the temporal attention feature, and Z represents the prediction result of ocean surface temperature from satellite remote sensing images.

[0070] The aforementioned network model was used to model the non-stationary information of the uncertain trends in the dynamic spatiotemporal process of SST, allowing the model to better fit the natural spatiotemporal process of SST and improving the prediction accuracy.

[0071] The training of the network model is not a key design feature of this invention, and will not be elaborated upon here.

[0072] In summary, this invention (1) designs a spatial non-stationary feature modeling sub-network to model the spatial non-stationary features in SST data. First, the input SST spatiotemporal sequence is downsampled to construct spatial features at three scales: small, medium, and large. Then, by performing differential modeling on the multi-scale spatial features, the changes of small-scale and large-scale non-stationary features and large-scale stationary features are fully explored. Finally, a spatial feature fusion module based on a gating mechanism is designed to dynamically model the correlation between changes of non-stationary and stationary features. (2) A prediction network based on dual streams is constructed. First, the temporal non-stationary features in the spatiotemporal sequence are captured by a diagonal recurrent neural network composed of stacked MIM modules. Then, by constructing parallel temporal non-stationary feature modeling sub-networks and spatial non-stationary feature modeling sub-networks, non-stationary features in both time and space are explored simultaneously. Finally, the temporal and spatial comprehensive features obtained from the two sub-networks are adaptively learned by using a stacked cross-attention spatiotemporal fusion module to learn the weights of temporal and spatial dependencies on the sea surface temperature field, thus obtaining the final SST prediction result and improving the accuracy of SST prediction.

[0073] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should be protected by the present invention.

Claims

1. A method for predicting ocean surface temperature sequences based on spatiotemporal dual-stream non-stationary sensing, characterized in that, Includes the following steps: Step S1: Acquire satellite remote sensing images of ocean surface temperature; Step S2: Construct and train an ocean surface temperature sequence prediction network based on spatiotemporal dual-stream nonstationary perception, including a spatial nonstationary feature modeling sub-network, a temporal nonstationary feature modeling sub-network, and a spatiotemporal fusion module based on stacked cross-attention. The time-non-stationary feature modeling subnetwork consists of ST-LSTM modules and MIM modules, including a multi-layer network structure with identical architecture. Each layer includes one ST-LSTM module and multiple MIM modules. Modules in the upper and lower layers are stacked and interconnected via zigzag paths. ST-LSTM modules or MIM modules at corresponding positions in each layer are connected via vertical paths. The input is ocean surface temperature satellite remote sensing images X1, X2, ..., X t The ocean surface temperature (SST) time series features are obtained by passing the ST-LSTM through each layer. The hidden states of the outputs of two vertically adjacent modules are passed to the next layer module through a diagonal path. In the MIM module, a difference-based method is used to capture the non-stationary information in the SST time series features. The non-stationary information is captured by a stacked diagonal structure, and the time series non-stationary features of SST are modeled. Finally, the outputs y1, y2, ..., y of the last MIM module in each layer are processed. t Aggregate the data to obtain the SST time-comprehensive feature β; The spatial non-stationary feature modeling subnetwork consists of a spatial non-stationary feature extraction module and a spatial feature fusion module, specifically: Within the spatial non-stationary feature extraction module, an M*M size satellite remote sensing image of ocean surface temperature is downsampled into feature maps of three scales: M*M, (M*M) / 4, and (M*M) / 8. A convolution operation is performed on the (M*M) / 4 and (M*M) / 8 satellite remote sensing images to capture the small-scale non-stationary feature n. Similarly, a convolution operation is performed on the M*M and (M*M) / 4 satellite remote sensing images to capture the large-scale non-stationary feature N. Within the spatial feature fusion module, the original feature map S of M*M, small-scale non-stationary features n, and large-scale non-stationary features N are aggregated. The resulting spatial aggregated features are processed by the sigmoid function and the tanh function, respectively. The sigmoid function strengthens the insignificant non-stationary features in the space to prevent loss and retains the complete spatial comprehensive features. Finally, the corresponding elements of the two function processing results are multiplied to obtain the SST spatial comprehensive feature α. The SST temporal comprehensive feature β obtained from the temporal nonstationary feature modeling subnetwork and the SST spatial comprehensive feature α from the spatial nonstationary feature modeling subnetwork are fused through a stacked cross-attention spatiotemporal fusion module. Within the stacked cross-attention spatiotemporal fusion module, the SST spatial comprehensive feature α and the SST temporal comprehensive feature β adaptively learn their own weights for the ocean surface temperature field, thus fusing temporal and spatial information. Step S3: Input the satellite remote sensing image of ocean surface temperature to be processed into the trained ocean surface temperature sequence prediction network based on spatiotemporal dual-stream non-stationary sensing, and output the ocean surface temperature prediction result.

2. The ocean surface temperature sequence prediction method based on spatiotemporal dual-stream non-stationary sensing according to claim 1, characterized in that, The calculation process for the SST spatial comprehensive feature α is as follows: o=tanh(W os *S+W oN *N+W on *n) g=σ(W gs *S+W gN *N+W gn *n) Among them W os W oN W on W gs W gN W gn The parameter represents the convolution parameter; * represents the convolution operation; σ is a sigmoid function, applied as an activation function for the weighted sum of inputs; tanh represents the tanh activation function; n represents small-scale non-stationary features; N represents large-scale non-stationary features. S represents the original feature map; denoted as Hadamard product; o represents the result of aggregating the original feature map S, small-scale non-stationary features n, and large-scale non-stationary features N and then passing it through the tanh function; g represents the result of aggregating the original feature map S, small-scale non-stationary features n, and large-scale non-stationary features N and then passing it through the sigmoid function.

3. The ocean surface temperature sequence prediction method based on spatiotemporal dual-stream non-stationary sensing according to claim 2, characterized in that, Within the spatiotemporal fusion module based on stacked cross-attention, specifically: First, the (M, M)-dimensional SST space comprehensive features α are flattened and mapped using an attention encoding mechanism, respectively, to serve as (M×M, 1)-dimensional query Q. α Sum V α The matrix and the SST temporal comprehensive feature β are also flattened and mapped through an attention encoding mechanism, serving as (M×M,1) dimensional keys K. β Sum V β matrix: Q α =W q *a,V α =W v *a K β =W k *b,V β =W v *b Among them W q W k W v Represents the convolution parameters, * represents the convolution operation, and Q represents the convolution parameters. α V represents the spatial query matrix; α Denotes a spatial value matrix, K β V represents the time key matrix. β Represents a time value matrix; Q α and K β Matrix multiplication is performed, and each grid point is modeled using the softmax function to obtain the correlation score e of the SST temporal comprehensive feature β and the SST spatial comprehensive feature α, with dimensions (M×M, M×M), as shown in the following formula: Where softmax represents the normalization exponential function. Represents matrix multiplication; Then, the spatial value matrix V α Matrix multiplication with e on the spatial value matrix V α Modeling each grid point and reshaping the result using the reshape function to obtain the spatial attention feature Z is a method for obtaining spatial attention features Z. α Similarly, the time value matrix V β Matrix multiplication with e on the time-value matrix V β Modeling each grid point and reshaping the result using the reshape function yields the temporal attention feature Z in (M, M) dimensions. β : Where reshape represents the function that transforms the tensor dimensions. V represents matrix multiplication, e represents the correlation score on temporal and spatial features, and V represents matrix multiplication. α V represents the spatial value matrix. β Z represents the time-value matrix. α Z represents spatial attention features. β Indicates the characteristics of time-based attention; Finally, by aggregating features and using a multilayer perceptron, the final prediction result of the ocean surface temperature satellite remote sensing image is obtained, as shown below: Z=MLP(W z [WITH α ,WITH β ]) Where MLP represents a multilayer perceptron, W Z Z represents the convolution parameters. α Z represents spatial attention features. β Z represents the temporal attention feature, and Z represents the prediction result of ocean surface temperature from satellite remote sensing images.

Images