Multi-scale sea surface temperature prediction method and system based on hierarchical cross-modal fusion

Abstract

This invention belongs to the field of sea surface temperature prediction, and specifically relates to a multi-scale sea surface temperature prediction method and system based on hierarchical cross-modal fusion. The method includes the following steps: inputting three modal data—ocean circulation, sea surface temperature, and wind field—and embedding features to obtain small-scale features for each modality; extracting mesoscale and large-scale features from the small-scale features of each modality; and then fusing the large-scale, mesoscale, and small-scale features of each modality in stage 1 for multi-modal feature fusion at the same scale, followed by stage 2 for multi-modal feature fusion at different scales to obtain fused cross-scale features. These features are then output through a convolutional layer after stepwise upsampling and skip connections to predict sea surface temperature. This invention comprehensively characterizes large-scale background information, mesoscale structural changes, and small-scale local disturbance features during the evolution of sea surface temperature, thereby improving the accuracy of sea surface temperature prediction.

Description

Technical Field

[0001] This invention belongs to the field of sea surface temperature prediction, specifically involving a multi-scale sea surface temperature prediction method and system based on hierarchical cross-modal fusion. Background Technology

[0002] Sea surface temperature (SST) is an important physical variable characterizing changes in the marine environment and climate evolution. Accurately predicting the spatiotemporal changes of sea surface temperature is of great significance for marine environmental monitoring, climate change research, and early warning of marine disasters.

[0003] In recent years, with the rapid development of deep learning technology, sea surface temperature (SST) prediction methods based on multi-source ocean observation data have gradually become an important research direction in the field of marine environmental prediction. Existing methods typically use deep learning models such as convolutional neural networks, recurrent neural networks, or Transformers to extract features from multimodal ocean data, thereby learning the spatial and temporal variation patterns of ocean variables. However, these methods suffer from insufficient prediction accuracy and weak long-term prediction capabilities. Therefore, some researchers have proposed a SST prediction method based on multi-scale spatiotemporal attention networks. This method introduces temporal attention modules at different time scales through a multi-scale temporal encoder to capture the dynamic evolution patterns of various ocean regions at different temporal resolutions, and gradually integrates information over a long time range through cross-scale feature fusion. Then, a spatial attention module is introduced at different spatial scales through a multi-scale spatial encoder to model the spatial interaction relationships between different latitude and longitude grids, and gradually converges coarse-grained semantic information and fine-grained local details through a cross-scale fusion mechanism. Finally, a decoder restores the encoded hidden features to the target spatial resolution, and a temporal projection module is used to output the SST prediction results for multiple future time periods, thereby achieving effective modeling of the multi-scale spatiotemporal characteristics of the SST field and improving prediction accuracy and model adaptability.

[0004] The above methods still have shortcomings, including: In the process of multimodal and multi-scale joint modeling, existing methods often mix multimodal fusion at the same scale and multimodal fusion at different scales, making it difficult to establish effective modeling mechanisms for different fusion levels. In the process of multimodal ocean data fusion at the same scale, existing methods usually only use feature splicing or simple attention mechanisms to fuse features from different modalities, lacking effective fusion mechanisms for multimodal information at the same scale. In the process of multimodal ocean data fusion at different scales, existing methods usually use convolution, weighted stacking, and other methods to fuse features at different scales, lacking effective modeling methods for cross-scale multimodal information interaction. Summary of the Invention

[0005] This application addresses the aforementioned technical problems by proposing a multi-scale sea surface temperature prediction method and system based on hierarchical cross-modal fusion. This method solves the problem that existing methods struggle to uniformly model same-scale multi-modal fusion and cross-scale multi-modal interaction, thereby more comprehensively characterizing large-scale background information, mesoscale structural changes, and small-scale local disturbance characteristics in the sea surface temperature evolution process, and improving the accuracy of sea surface temperature prediction.

[0006] To achieve the above objectives, the first aspect of this invention proposes a multi-scale sea surface temperature prediction method based on hierarchical cross-modal fusion, comprising the following steps:

[0007] S1. Multimodal and multiscale feature extraction:

[0008] Input ocean circulation and sea surface temperature modal data are used to embed features to obtain small-scale features for each modality. The small-scale features of each modality are then downsampled stepwise to extract mesoscale and large-scale features for each modality. Input wind field modal data are used to embed features to obtain small-scale features, and the small-scale features are downsampled to obtain mesoscale features.

[0009] S2. Multimodal feature fusion at the same scale:

[0010] Large-scale, mesoscale, and small-scale features of each modality data enter Stage 1. In Stage 1, the large-scale features of ocean circulation and sea surface temperature are input into a geographic-driven slice attention module (GSAM) for fusion processing to obtain fused large-scale features. The mesoscale features of wind field and sea surface temperature are input into another GSAM for the same fusion processing to obtain fused mesoscale features. The small-scale features of sea surface temperature, ocean circulation, and wind field are then stitched together and input into a diffusion-driven slice attention module (DSAM) for local dynamic feature enhancement to obtain small-scale enhanced features.

[0011] S3. Multimodal feature fusion at different scales:

[0012] The process involves fusing large-scale features, meso-scale features, and small-scale enhanced features, which then proceed to stage 2. In stage 2, the fused large-scale features and small-scale enhanced features are input into the time-driven slice attention module (TSAM) for cross-scale feature interaction, resulting in the final feature. Simultaneously, by fusing mesoscale features and small-scale enhanced features into another TSAM, the same cross-scale feature interaction is performed to obtain the features. ;

[0013] S4. Prediction Output:

[0014] and characteristics Concatenation and convolution are performed to obtain cross-scale features. The fused large-scale features are then upsampled and skip-connected with the fused medium-scale features before further upsampling. These upsampled features are then skip-connected with the cross-scale features to obtain small-scale fused features. These small-scale fused features are input into a convolutional layer to output the predicted sea surface temperature. .

[0015] Furthermore, the large-scale features of ocean circulation and sea surface temperature are input into the geographic-driven slice attention module GSAM for fusion processing to obtain fused large-scale features, including:

[0016] S201. Obtain the geographic coordinates of the corresponding spatial grids for large-scale features of ocean circulation and large-scale features of sea surface temperature. Normalization is performed, and geographic features are obtained through location coding. Slicing operations were performed on the large-scale features of ocean circulation and sea surface temperature, respectively, to obtain the corresponding slice weight matrices and ocean circulation slices and sea surface temperature slices. Based on the corresponding slice weight matrices, geographic features were then processed. Weighted aggregation is performed to obtain ocean circulation geographic slices corresponding to ocean circulation slices and sea surface temperature geographic slices corresponding to sea surface temperature slices; the geographic similarity between ocean circulation geographic slices and sea surface temperature geographic slices is calculated to obtain a geographic similarity matrix. ;

[0017] S202. The same slicing operation is performed on the large-scale features of ocean circulation and the large-scale features of sea surface temperature to obtain ocean circulation slices and sea surface temperature slices. Based on the ocean circulation slices, a query vector Q is constructed by querying the linear mapping parameter matrix. Based on the sea surface temperature slices, a key vector K and a value vector V are constructed by the key linear mapping parameter matrix and the value linear mapping parameter matrix, respectively.

[0018] Introducing a geographic similarity matrix As a bias term, the similarity score between the query vector Q and the key vector K is corrected by element-wise multiplication and element-wise addition, and normalized by the Softmax function to obtain the slice attention weight. The slice attention weight is then used to perform a weighted summation on the value vector V to obtain the fused slice.

[0019] S203. Perform a reverse slicing operation on the fused slice features to obtain the reconstructed features of all spatial locations, and then reorganize them to obtain the fused large-scale features.

[0020] Further, the slicing operation includes:

[0021] For input features A linear mapping is performed, and then normalized using the Softmax function to obtain the slice weight matrix for the corresponding mode.

[0022] Based on the slice weight matrix, the input features Weighted aggregation is performed to obtain slice features, and all slice features form a corresponding feature slice sequence.

[0023] The reverse slicing operation includes:

[0024] For input features The slice weight matrix generated during the slicing operation is used to assign the fused slice features back to their corresponding spatial locations, and the features from different slices at the same spatial location are weighted and aggregated to obtain the reconstructed features.

[0025] A reverse slicing operation is performed on all spatial locations to obtain the reconstructed features of all spatial locations, which are then reorganized according to the original spatial grid size to obtain the fused spatial feature map.

[0026] Furthermore, the small-scale features of sea surface temperature, ocean circulation, and wind field are concatenated and input into the diffusion-driven slice attention module DSAM for local dynamic feature enhancement to obtain small-scale enhanced features, including:

[0027] S204, small-scale features of sea surface temperature, small-scale features of ocean circulation, and small-scale features of wind field are concatenated to obtain small-scale features. The small-scale features are input into the diffusion-driven slice attention module DSAM, which performs convolution operations through diffusion convolution kernels. The sum of the convolution results is added to the small-scale features to obtain diffusion-enhanced features.

[0028] S205. Using the same slicing operation as in GSAM, respectively, the diffusion enhancement features are processed. and small-scale features Slicing is performed to obtain diffusion-enhanced feature slice sequences and small-scale feature slice sequences;

[0029] S206. Construct a query vector Q based on the diffusion-enhanced feature slice sequence, and construct the key vector K and value vector V in slice-level attention based on the small-scale feature slice sequence;

[0030] S207. Based on the query vector Q and key vector K, the similarity score is corrected, and normalized using the Softmax function to obtain slice-level attention weights. The slice-level attention weights are then used to perform a weighted summation on the value vector V to obtain the fused slice features. ;

[0031] S208. Using the same reverse slicing operation as in GSAM, the fused slice features are... The spatial feature map is restored, resulting in small-scale enhanced features.

[0032] Furthermore, the fused large-scale features, fused mesoscale features, and small-scale enhanced features enter stage 2. Stage 2 inputs the fused large-scale features and small-scale enhanced features into the time-driven slice attention module (TSAM) for cross-scale feature interaction, resulting in the final feature. ,include:

[0033] S301. The input of large-scale features and small-scale enhanced features into the time-driven slice attention module TSAM is fused. TSAM performs time encoding on the time information of the past ten steps to obtain time features.

[0034] S302 and time features are respectively input into the multilayer perceptron. and multilayer perceptron After being normalized by Softmax, large-scale time weights and small-scale time weights are obtained respectively.

[0035] S303. Temporal modulation is applied to the fused large-scale features and small-scale enhanced features using large-scale temporal weights and small-scale temporal weights, respectively, to obtain the modulated fused large-scale features and modulated small-scale enhanced features.

[0036] S304. Using the same slicing operation as in GSAM, slice the modulated fused large-scale features and the modulated small-scale enhanced features respectively to obtain the fused large-scale feature slice sequence and the small-scale enhanced feature slice sequence.

[0037] S305. Based on the fusion of large-scale feature slice sequences, a query vector Q is constructed by querying the linear mapping parameter matrix. Based on the small-scale enhanced feature slice sequences, a key vector K and a value vector V are constructed by the key linear mapping parameter matrix and the value linear mapping parameter matrix, respectively.

[0038] S306. Based on the query vector Q and key vector K, the similarity score is corrected, and normalized using the Softmax function to obtain slice-level attention weights. The slice-level attention weights are then used to perform a weighted summation on the value vector V to obtain the fused slice features. ;

[0039] S307. Use the same reverse slicing operation as in GSAM to merge the slice features. Restore to spatial feature map, obtain features .

[0040] A second aspect of this invention proposes a hierarchical cross-modal fusion multi-scale sea surface temperature prediction system, comprising:

[0041] The multimodal and multiscale feature extraction module is used for multimodal and multiscale feature extraction: input ocean circulation and sea surface temperature modal data are embedded to obtain small-scale features of each modality, and the small-scale features of each modality are extracted by progressive downsampling to obtain mesoscale and large-scale features of each modality; input wind field modal data are embedded to obtain small-scale features, and the small-scale features are downsampled to obtain mesoscale features;

[0042] A multimodal feature fusion module at the same scale is used for multimodal feature fusion at the same scale: the large-scale, mesoscale, and small-scale features of each modality data enter stage 1. In stage 1, the large-scale features of ocean circulation and the large-scale features of sea surface temperature are input into the geographic-driven slice attention module GSAM for fusion processing to obtain fused large-scale features; the mesoscale features of wind field and the mesoscale features of sea surface temperature are input into another GSAM for the same fusion processing to obtain fused mesoscale features; the small-scale features of sea surface temperature, ocean circulation, and wind field are spliced ​​together and input into the diffusion-driven slice attention module DSAM for local dynamic feature enhancement to obtain small-scale enhanced features;

[0043] A multimodal feature fusion module at different scales is used for multimodal feature fusion at different scales: fusing large-scale features, fusing medium-scale features, and small-scale enhanced features into stage 2. Stage 2 inputs the fused large-scale features and small-scale enhanced features into the time-driven slice attention module TSAM for cross-scale feature interaction to obtain the features. Simultaneously, by fusing mesoscale features and small-scale enhanced features into another TSAM, the same cross-scale feature interaction is performed to obtain the features. ;

[0044] The prediction output module is used to output the predicted sea surface temperature: features and characteristics The system performs concatenation and convolution to obtain cross-scale features. The fused large-scale features are then upsampled and fused with the fused medium-scale features via skip connections, followed by further upsampling. This upsampling is then combined with the cross-scale features to obtain small-scale fused features. These small-scale fused features are input into a convolutional layer to output the predicted sea surface temperature. .

[0045] Compared with the prior art, the advantages of this invention are:

[0046] This invention achieves multimodal feature fusion at the same scale and multimodal feature fusion at different scales through the collaborative design of a Geography-Driven Slice Attention Module (GSAM), a Diffusion-Driven Slice Attention Module (DSAM), and a Time-Driven Slice Attention Module (TSAM). It solves the problem that existing methods are difficult to uniformly model multimodal fusion at the same scale and multimodal interaction across scales, thereby more comprehensively characterizing the large-scale background information, mesoscale structural changes, and small-scale local disturbance features in the evolution of sea surface temperature, and improving the accuracy of sea surface temperature prediction.

[0047] The first aspect of this invention proposes a geographically driven slice attention module (GSAM) and a diffusion-driven slice attention module (DSAM), constructing a fusion mechanism for multimodal features at the same scale. GSAM constructs a geographic similarity matrix by incorporating geographic coordinate information and uses it as a bias term in the slice-level attention weight calculation, thereby enhancing the feature interaction capabilities between geographically proximate regions and improving the model's ability to represent the spatial correlation and dynamic coupling relationships between ocean circulation, wind field, and sea surface temperature. DSAM enhances the local dynamic structure in small-scale features through diffusion convolution kernels and combines the slice attention mechanism to fuse and model local disturbance information, thereby improving the model's ability to characterize small-scale details and local changes. Simultaneously, the slice attention mechanism can also perform weighted fusion of different modal information during regional feature interaction, enhancing the utilization of complementary information between modalities and addressing the problem of insufficient information interaction in traditional same-scale fusion methods.

[0048] The second aspect of this invention proposes a time-driven slice attention module (TSAM) for cross-scale multimodal feature interaction. This module generates dynamic temporal weights for features at different scales through time encoding, enabling the model to adaptively adjust the roles of fused large-scale, fused mesoscale, and small-scale enhanced features in the fusion process according to different time stages. Simultaneously, TSAM utilizes a slice-level attention mechanism to establish cross-scale interactions for fused large-scale, fused mesoscale, and small-scale enhanced features, thereby achieving collaborative modeling of multimodal information at different scales. This module can both preserve large-scale trend information in the sea surface temperature evolution process and enhance the expressive power of small-scale perturbation structures in local areas, thus improving the accuracy and stability of prediction results. This module solves the problem in existing methods where it is difficult to effectively guide the modeling of small-scale enhanced features using fused large-scale and fused mesoscale information in cross-scale multimodal interaction, leading to inaccurate prediction of small-scale structures. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] In the attached diagram:

[0051] Figure 1 This is a flowchart illustrating the multi-scale sea surface temperature prediction method based on hierarchical cross-modal fusion in this specific embodiment.

[0052] Figure 2 This is a schematic diagram of the structure of the geographic-driven slice attention module GSAM in this specific embodiment;

[0053] Figure 3 This is a schematic diagram of the structure of the diffusion-driven slice attention module DSAM in this specific embodiment;

[0054] Figure 4 This is a schematic diagram of the Time-Driven Slice Attention Module (TSAM) in this specific embodiment;

[0055] Figure 5 This is a diagram showing the overall architecture of the multi-scale sea surface temperature prediction system based on hierarchical cross-modal fusion in this specific embodiment. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the accompanying drawings. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0057] Example 1:

[0058] Please see Figure 1 This invention provides a multi-scale sea surface temperature prediction method based on hierarchical cross-modal fusion, comprising the following steps:

[0059] S1. Multimodal and multiscale feature extraction:

[0060] Input ocean circulation and sea surface temperature modal data are used to embed features to obtain small-scale features for each modality. The small-scale features of each modality are then downsampled stepwise to extract mesoscale and large-scale features for each modality. Input wind field modal data are used to embed features to obtain small-scale features, and the small-scale features are downsampled to obtain mesoscale features.

[0061] S2. Multimodal feature fusion at the same scale:

[0062] Large-scale, mesoscale, and small-scale features of each modality data enter Stage 1. In Stage 1, the large-scale features of ocean circulation and sea surface temperature are input into a geographic-driven slice attention module (GSAM) for fusion processing to obtain fused large-scale features. The mesoscale features of wind field and sea surface temperature are input into another GSAM for the same fusion processing to obtain fused mesoscale features. The small-scale features of sea surface temperature, ocean circulation, and wind field are then stitched together and input into a diffusion-driven slice attention module (DSAM) for local dynamic feature enhancement to obtain small-scale enhanced features.

[0063] S3. Multimodal feature fusion at different scales:

[0064] The process involves fusing large-scale features, meso-scale features, and small-scale enhanced features, which then proceed to stage 2. In stage 2, the fused large-scale features and small-scale enhanced features are input into the time-driven slice attention module (TSAM) for cross-scale feature interaction, resulting in the final feature. Simultaneously, by fusing mesoscale features and small-scale enhanced features into another TSAM, the same cross-scale feature interaction is performed to obtain the features. ;

[0065] S4. Prediction Output:

[0066] feature and characteristics Concatenation and convolution are performed to obtain cross-scale features. The fused large-scale features are then upsampled and skip-connected with the fused medium-scale features before further upsampling. These upsampled features are then skip-connected with the cross-scale features to obtain small-scale fused features. These small-scale fused features are input into a convolutional layer to output the predicted sea surface temperature. .

[0067] In some embodiments, large-scale features of ocean circulation and large-scale features of sea surface temperature are input into a geographically driven slice attention module (GSAM) for fusion processing to obtain fused large-scale features, including:

[0068] S201. Obtain the geographic coordinates of the corresponding spatial grids for large-scale features of ocean circulation and large-scale features of sea surface temperature. Normalization is performed, and geographic features are obtained through location coding. Slicing operations were performed on the large-scale features of ocean circulation and sea surface temperature, respectively, to obtain the corresponding slice weight matrices and ocean circulation slices and sea surface temperature slices. Based on the corresponding slice weight matrices, geographic features were then processed. Weighted aggregation is performed to obtain ocean circulation geographic slices corresponding to ocean circulation slices, and sea surface temperature geographic slices corresponding to sea surface temperature slices; the similarity between ocean circulation geographic slices and sea surface temperature geographic slices is calculated to obtain a geographic similarity matrix. ;

[0069] S202. Perform the same Slice operation on the large-scale features of ocean circulation and the large-scale features of sea surface temperature to obtain ocean circulation slices and sea surface temperature slices. Based on the ocean circulation slices, construct the query vector Q by querying the linear mapping parameter matrix. Based on the sea surface temperature slices, construct the key vector K and the value vector V by the key linear mapping parameter matrix and the value linear mapping parameter matrix, respectively.

[0070] Introducing a geographic similarity matrix As a bias term, the similarity score between the query vector Q and the key vector K is corrected by element-wise multiplication and element-wise addition, and normalized by the Softmax function to obtain the slice attention weight. The slice attention weight is then used to perform a weighted summation on the value vector V to obtain the fused slice.

[0071] S203. Perform a reverse slicing operation on the fused slices to obtain the reconstructed features of all spatial locations, and then reorganize them to obtain the fused large-scale features.

[0072] Specifically, large-scale characteristics of ocean circulation Large-scale characteristics of sea surface temperature The input is a geographic-driven tile attention module GSAM, which includes a geographic similarity calculation module and a tile attention module;

[0073] S201. Obtain large-scale features of ocean circulation through the geographic similarity calculation module. Large-scale characteristics of sea surface temperature Geographic coordinates of the corresponding spatial grid Geographic coordinates It is a tensor of size H×W×2, containing two channels that store the longitude and latitude information of the corresponding spatial grid, respectively, for geographic coordinates. Normalization is performed, and geographic features are obtained through location coding. , is represented as:

[0074] ;

[0075] in, This indicates a position encoding operation. Geographic coordinates of the corresponding spatial grid representing the large-scale characteristics of sea surface temperature. Indicates high, Indicates width. This indicates two channels: longitude and latitude. This indicates that geographical features are obtained through location coding. Indicates the number of channels;

[0076] Slicing operations were performed on the large-scale features of ocean circulation and sea surface temperature, respectively, to obtain the corresponding slice weight matrices and ocean circulation slices and sea surface temperature slices. The corresponding slice weight matrix is ​​expressed as follows:

[0077] ;

[0078] ;

[0079] in, This represents the weight matrix of ocean current feature slices. This represents the sea surface temperature slice weight matrix. Represents a geographical slice of ocean circulation. Geographic slices representing sea surface temperature. Indicates the number of slices;

[0080] Specifically, based on the corresponding slice weight matrix, geographical features are respectively... Weighted aggregation is performed to obtain ocean circulation geographic slices corresponding to ocean circulation slices, and sea surface temperature geographic slices corresponding to sea surface temperature slices; the similarity between ocean circulation geographic slices and sea surface temperature geographic slices is calculated to obtain a geographic similarity matrix. , is represented as:

[0081] ;

[0082] in, The elements of the geographic similarity matrix represent the i-th geographic slice of ocean circulation. With the j-th sea surface temperature geographic slice Geographical similarity between them This represents the inner product operation. Let L be the norm of a vector;

[0083] S202, Large-scale characteristics of ocean circulation Large-scale characteristics of sea surface temperature Performing the same slice operation yields ocean circulation slices and sea surface temperature slices, represented as follows:

[0084] ;

[0085] ;

[0086] ;

[0087] ;

[0088] in, , representing the large-scale features of ocean circulation and sea surface temperature corresponding to the nth slice, respectively. Represents a slice of ocean circulation. This represents a slice of sea surface temperature.

[0089] Specifically, the slice attention module constructs a query vector Q based on ocean circulation slices by querying the linear mapping parameter matrix, and constructs a key vector K and a value vector V based on sea surface temperature slices by using the key linear mapping parameter matrix and the value linear mapping parameter matrix, respectively, as follows:

[0090] ;

[0091] ;

[0092] ;

[0093] in, , , These are the query linear mapping parameter matrix, the key linear mapping parameter matrix, and the value linear mapping parameter matrix, respectively.

[0094] Introducing a geographic similarity matrix As a bias term, the similarity score between the query vector Q and the key vector K is corrected by element-wise multiplication and element-wise addition, and then normalized using the Softmax function to obtain the slice attention weight. The value vector V is then weighted and summed using element-wise multiplication using the slice attention weight to obtain the fused slice. , is represented as:

[0095] ;

[0096] in, This represents the normalized exponential function, used to transform slice attention scores into a non-negative attention weight distribution that sums to 1. This represents the feature dimension of the key vector K. Represents the j-th fusion feature, fusion slice .

[0097] S203, Fusion slice Perform a deslicing operation to obtain reconstructed features at all spatial locations, and then recombine them to obtain fused large-scale features. , is represented as:

[0098] .

[0099] Specifically, GSAM can use geographic location information to guide attention interactions between different spatial slices, thereby enhancing the ability of ocean dynamic structure features to characterize sea surface temperature features.

[0100] In practical applications, slicing operations include:

[0101] For input features A linear mapping is performed, and then normalized using the Softmax function to obtain the slice weight matrix for the corresponding mode.

[0102] Based on the slice weight matrix, the input features Weighted aggregation is performed to obtain slice features, and all slice features form a corresponding feature slice sequence.

[0103] The reverse slicing operation includes:

[0104] For input features The slice weight matrix generated during the slicing operation is used to assign the fused slice features back to their corresponding spatial locations, and the features from different slices at the same spatial location are weighted and aggregated to obtain the reconstructed features.

[0105] A reverse slicing operation is performed on all spatial locations to obtain the reconstructed features of all spatial locations, which are then reorganized according to the original spatial grid size to obtain the fused spatial feature map.

[0106] Specifically, the Slice operation is used to map a spatial feature map into several slice features to achieve an aggregated representation of local region features.

[0107] Specifically, let the input features be... Where m represents the type of input feature to be sliced, m can be ocean circulation feature, sea surface temperature feature, wind field feature, diffusion enhancement feature, small-scale feature, meso-scale feature and large-scale feature, H represents feature map height, W represents feature map width, and C represents feature channel number;

[0108] Input features After flattening out the spatial dimensions, it can be represented as ,in, i = 1, 2, ..., N, where N = H × W represents the number of spatial grid points. This represents the eigenvector of the i-th spatial grid point in mode m.

[0109] For input features Perform a linear mapping and normalize using the Softmax function to obtain the slice weight matrix. , is represented as:

[0110] ;

[0111] in, represents the linear mapping function, and Softmax(·) represents the normalization exponential function, which is used to convert the linear mapping result into a non-negative slice weight distribution with a sum of 1; Let represent the slice weight matrix corresponding to mode m, with dimensions N×M, where M represents the number of slices.

[0112] Specifically, the slice weight matrix For input features Perform weighted aggregation to obtain the features of the j-th slice. , is represented as:

[0113] ;

[0114] in, This represents the j-th slice feature corresponding to mode m. This represents the weight of the i-th spatial grid point belonging to the j-th slice.

[0115] Specifically, slice features In fact, it involves multiple input features. The weighted combination allows grid points with similar semantic or spatial features to be aggregated into the same slice, thereby achieving regional-level representation of features and obtaining the corresponding feature slice sequence.

[0116] Specifically, the DeSlice operation is used to remap slice-level features back to the original spatial grid to recover the spatial feature map.

[0117] Specifically, for input features The slice weight matrix generated during the slicing operation is used. The fused slice features are assigned to their corresponding spatial locations, and features from different slices at the same spatial location are weighted and aggregated to obtain the reconstructed features, represented as:

[0118] ;

[0119] in, Let i represent the feature vector at spatial location i after reconstruction. This represents the mapping weight of spatial location i to the j-th slice. Indicates the number of slices. This represents the j-th fused slice feature.

[0120] Specifically, by performing the above-described inverse slicing mapping operation on all spatial locations i=1,2,...,N, the reconstructed features of all spatial locations can be obtained. The reconstructed feature is then reorganized according to the original spatial grid size H×W to obtain the fused spatial feature map.

[0121] In other embodiments, small-scale features of sea surface temperature, ocean circulation, and wind field are concatenated and input into a diffusion-driven slice attention module (DSAM) for local dynamic feature enhancement to obtain small-scale enhanced features, including:

[0122] S204, small-scale features of sea surface temperature, small-scale features of ocean circulation, and small-scale features of wind field are concatenated to obtain small-scale features. The small-scale features are input into the diffusion-driven slice attention module DSAM, which performs convolution operations through diffusion convolution kernels. The sum of the convolution results is added to the small-scale features to obtain diffusion-enhanced features.

[0123] S205. Using the same Slice operation as in GSAM, slice the diffusion enhancement feature and the small-scale feature respectively to obtain the diffusion enhancement feature slice sequence and the small-scale feature slice sequence.

[0124] S206. Construct a query vector Q based on the diffusion-enhanced feature slice sequence, and construct the key vector K and value vector V in slice-level attention based on the small-scale feature slice sequence;

[0125] S207. Based on the query vector Q and key vector K, the similarity score is corrected, and normalized using the Softmax function to obtain slice-level attention weights. The slice-level attention weights are then used to perform a weighted summation on the value vector V to obtain the fused slice features. ;

[0126] S208. Using the same deslicing operation as in GSAM, the fused slice features are... The spatial feature map is restored, resulting in small-scale enhanced features.

[0127] Specifically, S204, small-scale characteristics of sea surface temperature Small-scale characteristics of ocean circulation and small-scale characteristics of wind fields After concatenation, small-scale features are obtained. These small-scale features are input into the diffusion-driven slice attention module DSAM, which performs convolution operations through diffusion convolution kernels. The sum of the convolution results is added to the small-scale features to obtain diffusion-enhanced features, which can enhance the local gradient information and perturbation structure information in the small-scale features, as shown below:

[0128] ;

[0129] ;

[0130] ;

[0131] in, Indicates diffusion enhancement characteristics, Represents small-scale features. Indicates the diffusion step size. These represent the meridional diffusion operator and the zonal diffusion operator, respectively, used to characterize the local diffusion variations of small-scale features in the meridional and zonal directions. and These represent the meridional diffusion convolution kernel and the latitudinal diffusion convolution kernel, respectively. This indicates a convolution operation.

[0132] Specifically, in S205, the same Slice operation as in GSAM is used to slice the diffusion enhancement feature and the small-scale feature respectively, resulting in the diffusion enhancement feature slice sequence and the small-scale feature slice sequence, as follows:

[0133] ;

[0134] ;

[0135] ;

[0136] ;

[0137] in, This represents a slice sequence with diffusion-enhanced features. This represents a sequence of small-scale feature slices. They represent the first Each spatial slice addresses diffusion enhancement features and small-scale features. Indicates diffusion enhancement features slice weights, Representing small-scale features Slice weights.

[0138] Specifically, S206, constructing a query vector Q based on the diffusion-enhanced feature slice sequence, and constructing the key vector K and value vector V in the slice-level attention based on the small-scale feature slice sequence, as follows:

[0139] ;

[0140] ;

[0141] ;

[0142] in, , , These are the query linear mapping parameter matrix, the key linear mapping parameter matrix, and the value linear mapping parameter matrix, respectively.

[0143] Specifically, in S207, the similarity score is corrected based on the query vector Q and the key vector K, and normalized using the Softmax function to obtain slice-level attention weights. The slice-level attention weights are then used to perform a weighted summation on the value vector V to obtain the fused slice features. , is represented as:

[0144] ;

[0145] in, This represents the normalized exponential function, used to map attention scores to a non-negative weight distribution that sums to 1. This represents the feature dimension of the key vector K.

[0146] Specifically, in S208, the same deSlice operation as in GSAM is used to merge the slice features. Restored to a spatial feature map, resulting in small-scale enhanced features. , is represented as:

[0147] ;

[0148] in, Representing small-scale features Slice weights.

[0149] Specifically, the aforementioned DSAM is mainly used to enhance the modeling of small-scale features, so as to highlight the local perturbation structure information and improve the expressive power of small-scale features.

[0150] In some embodiments, the fused large-scale features, fused medium-scale features, and small-scale enhanced features enter stage 2. Stage 2 inputs the fused large-scale features and small-scale enhanced features into the time-driven slice attention module (TSAM) for cross-scale feature interaction to obtain the features. ,include:

[0151] S301. The input of large-scale features and small-scale enhanced features into the time-driven slice attention module TSAM is fused. TSAM performs time encoding on the time information of the past ten steps to obtain time features.

[0152] S302 and time features are respectively input into the multilayer perceptron. and multilayer perceptron After being normalized by Softmax, large-scale time weights and small-scale time weights are obtained respectively.

[0153] S303. Temporal modulation is applied to the fused large-scale features and small-scale enhanced features using large-scale temporal weights and small-scale temporal weights, respectively, to obtain the modulated fused large-scale features and modulated small-scale enhanced features.

[0154] S304. Using the same slicing operation as in GSAM, slice the modulated fused large-scale features and the modulated small-scale enhanced features respectively to obtain the fused large-scale feature slice sequence and the small-scale enhanced feature slice sequence.

[0155] S305. Based on the fusion of large-scale feature slice sequences, a query vector Q is constructed by querying the linear mapping parameter matrix. Based on the small-scale enhanced feature slice sequences, a key vector K and a value vector V are constructed by the key linear mapping parameter matrix and the value linear mapping parameter matrix, respectively.

[0156] S306. Based on the query vector Q and key vector K, the similarity score is corrected, and normalized using the Softmax function to obtain slice-level attention weights. The slice-level attention weights are then used to perform a weighted summation on the value vector V to obtain the fused slice features. ;

[0157] S307. Use the same reverse slicing operation as in GSAM to merge the slice features. Restore to spatial feature map, obtain features .

[0158] Specifically, S301, which integrates large-scale features and small-scale enhanced features, inputs a time-driven slice attention module (TSAM). TSAM processes time information from the past ten steps. Perform time encoding to obtain time features , is represented as:

[0159] ;

[0160] Among them, large-scale fusion features Small-scale enhancement features , , and , Let represent the height and width of the fused large-scale features and small-scale enhanced features, respectively, and t represent the time step. This indicates a time encoding operation.

[0161] Specifically, S302, time characteristics Input to multilayer perceptron respectively and multilayer perceptron After undergoing Softmax normalization, large-scale time weights were obtained. and small-scale time weights , is represented as:

[0162] ;

[0163] ;

[0164] in, This indicates Softmax normalization.

[0165] Specifically, S303 utilizes large-scale time weights. and small-scale time weights The large-scale and small-scale enhanced features were temporally modulated using element-wise multiplication to obtain the modulated fused large-scale features. and modulated small-scale enhancement features , is represented as:

[0166] ;

[0167] ;

[0168] in, This indicates element-wise multiplication.

[0169] Specifically, S306, based on the query vector Q and key vector K, corrects the similarity score, and normalizes it using the Softmax function to obtain slice-level attention weights. These slice-level attention weights are then used to perform a weighted summation of the value vector V to obtain the fused slice features. , is represented as:

[0170] ;

[0171] in, This represents the normalized exponential function, used to map attention scores to a non-negative weight distribution that sums to 1. This represents the feature dimension of the key vector K.

[0172] Specifically, in S307, the same deSlice operation as in GSAM is used to merge the slice features. Restore to spatial feature map, obtain features , is represented as:

[0173] ;

[0174] in, This indicates a reverse slicing operation.

[0175] Specifically, TSAM utilizes time coding to generate dynamic weights for large-scale and small-scale features, as well as dynamic weights for fusing mesoscale and small-scale enhanced features. It also combines slice attention to achieve cross-scale information interaction between large-scale and small-scale enhanced features, and cross-scale information interaction between mesoscale and small-scale enhanced features, thereby enhancing the model's ability to model small-scale sea surface temperature features.

[0176] In some embodiments, features and characteristics The system performs concatenation and convolution to obtain cross-scale features. The fused large-scale features are then upsampled and fused with the fused mid-scale features via skip connections, followed by further upsampling. This fusion is then combined with the cross-scale features to obtain small-scale fused features. These small-scale fused features are input into a convolutional layer, which outputs the predicted sea surface temperature. ,include:

[0177] feature and characteristics The features are concatenated and integrated using a 1×1 convolution to obtain cross-scale features. The fused large-scale features are then upsampled to match the spatial resolution of the fused mesoscale features, and fused with the mesoscale features via skip connections to obtain mesoscale fused features. Subsequently, the mesoscale fused features are further upsampled and fused with the cross-scale features via skip connections to obtain small-scale fused features. Finally, the small-scale fused features are input into a 1×1 convolutional layer to output the predicted sea surface temperature. .

[0178] Example 2:

[0179] Please see Figures 2-5 Based on the same inventive concept, this invention also provides a multi-scale sea surface temperature prediction system based on hierarchical cross-modal fusion, the multi-scale sea surface temperature prediction system comprising:

[0180] The multimodal and multiscale feature extraction module is used for multimodal and multiscale feature extraction: input ocean circulation and sea surface temperature modal data are embedded to obtain small-scale features of each modality, and the small-scale features of each modality are extracted by progressive downsampling to obtain mesoscale and large-scale features of each modality; input wind field modal data are embedded to obtain small-scale features, and the small-scale features are downsampled to obtain mesoscale features;

[0181] A multimodal feature fusion module at the same scale is used for multimodal feature fusion at the same scale: the large-scale, mesoscale, and small-scale features of each modality data enter stage 1. In stage 1, the large-scale features of ocean circulation and the large-scale features of sea surface temperature are input into the geographic-driven slice attention module GSAM for fusion processing to obtain fused large-scale features; the mesoscale features of wind field and the mesoscale features of sea surface temperature are input into another GSAM for the same fusion processing to obtain fused mesoscale features; the small-scale features of sea surface temperature, ocean circulation, and wind field are spliced ​​together and input into the diffusion-driven slice attention module DSAM for local dynamic feature enhancement to obtain small-scale enhanced features;

[0182] A multimodal feature fusion module at different scales is used for multimodal feature fusion at different scales: fusing large-scale features, fusing medium-scale features, and small-scale enhanced features into stage 2. Stage 2 inputs the fused large-scale features and small-scale enhanced features into the time-driven slice attention module TSAM for cross-scale feature interaction to obtain the features. Simultaneously, by fusing mesoscale features and small-scale enhanced features into another TSAM, the same cross-scale feature interaction is performed to obtain the features. ;

[0183] The prediction output module is used to output the predicted sea surface temperature: features and characteristics The system performs concatenation and convolution to obtain cross-scale features. The fused large-scale features are then upsampled and fused with the fused medium-scale features via skip connections, followed by further upsampling. This upsampling is then combined with the cross-scale features to obtain small-scale fused features. These small-scale fused features are input into a convolutional layer to output the predicted sea surface temperature. .

[0184] Example 3:

[0185] This invention utilizes the publicly available OISST V2.1 dataset and the global ocean current product MULTIOBS_GLO_PHY_MYNRT_015_003, selecting the east-west current velocity component uo and the north-south current velocity component vo as input variables for ocean circulation. It also employs the publicly available ERA5 single-layer hourly reanalysis dataset, selecting the east-west wind speed component u10 and the north-south wind speed component v10 at a height of 10 m as input variables for the wind field. The hourly wind field data is processed into daily average data, and the sea surface temperature, ocean current, and wind field data are unified to a spatial resolution of 0.25°×0.25°. Daily data from 2013 to 2022 are selected, resampled to a 32×32 grid using bilinear interpolation, and the first 85% of the data is divided into a training set, while the last 15% is divided into a test set according to chronological order.

[0186] The model was optimized using the Adam optimizer, with an initial learning rate of 0.001 and a batch size of 32. The model was implemented using the PyTorch deep learning framework, and both training and testing were performed on an NVIDIA RTX 4090 GPU. During training, sea surface temperature, ocean circulation, and wind field data for 10 consecutive days were used as model input, and the corresponding future 10 consecutive days of actual sea surface temperature were used as the supervised learning target. A supervised learning strategy was employed to complete the model training.

[0187] This invention uses the mean squared error loss function (MSE) as the training optimization objective, expressed as:

[0188] ;

[0189] in, This represents the sea surface temperature value predicted by the model at the i-th sea surface temperature grid point. This represents the actual sea surface temperature value at the corresponding grid point, and n represents the total number of sea surface temperature grid points involved in the loss calculation.

[0190] By minimizing the aforementioned loss function, the model can continuously adjust the network parameters, reduce the overall error between the predicted sea surface temperature field and the actual sea surface temperature field, thereby improving the accuracy and stability of the sea surface temperature prediction results.

[0191] During training, each batch of training samples is first input into the model for forward propagation to obtain the predicted sea surface temperature for the next 10 days. Then, the mean squared error between the predicted result and the actual sea surface temperature is calculated, the gradient of the network parameters is calculated through backpropagation, and the model parameters are updated using the Adam optimizer. One forward propagation and parameter update for all samples in the training set is counted as one training epoch, and the model is trained for a total of 100 epochs. After training, the model parameters are fixed, and the sea surface temperature prediction performance of the model is evaluated using a test set.

[0192] Experiment content:

[0193] The model setup, training, and testing in this experiment were all performed using the PyTorch deep learning framework. For parameter optimization, the Adam optimizer was used, with its learning rate fixed at 0.001. The model's input was a 10-day historical data sequence, and its output was a prediction of the target value for the next 10 days; the batch size during training was set to 32. To ensure sufficient model convergence, each experiment lasted 100 epochs, and all computational tasks were executed on an NVIDIA RTX 4090 GPU.

[0194] Evaluation indicators and experimental results:

[0195] To comprehensively evaluate the effectiveness of the proposed method in sea surface temperature prediction, this invention employs root mean square error (RMSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and coefficient of determination (R²). 2 The correlation coefficient ρ is used as an evaluation index. Among them, RMSE, MAE, and MAPE are used to measure the magnitude of the error between the model's predicted results and the actual sea surface temperature; the smaller the value, the higher the prediction accuracy. 2 ρ is used to measure the consistency and correlation between the predicted results and the true values. The closer the value is to 1, the better the model's predictive performance.

[0196] To verify the effectiveness of the proposed method, this invention selected the following neural networks as comparative models: Convolutional Long Short-Term Memory (CONV-LSTM), Four CastNet, Dense Dilated Convolutional LSTM (D2CL), Latent Spectral Model (LSM), Shift Window Transformer (Swin Transformer), U-shaped Convolutional Neural Network (U-Net), and U-shaped Fourier Neural Operator (U-FNO). Comparative experiments were conducted using the same dataset and experimental settings. The experimental results are shown in the table below:

[0197]

[0198] The method proposed in this invention achieved the lowest RMSE, MAE, and MAPE in sea surface temperature prediction tasks, while also obtaining the highest R... 2 The correlation coefficient ρ indicates that this invention can effectively reduce the error between the predicted sea surface temperature field and the actual sea surface temperature field, while maintaining a high degree of consistency in spatiotemporal variation. Experimental results further demonstrate the effectiveness of this invention in improving the accuracy and stability of sea surface temperature prediction.

[0199] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-described technical content to create equivalent embodiments without departing from the scope of the present invention. The implementation schemes in the above embodiments can also be further combined or replaced. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A multi-scale sea surface temperature prediction method based on hierarchical cross-modal fusion, characterized in that, Includes the following steps: S1. Multimodal and multiscale feature extraction: Input ocean circulation and sea surface temperature modal data are used to embed features to obtain small-scale features for each modality. The small-scale features of each modality are then downsampled stepwise to extract mesoscale and large-scale features for each modality. Input wind field modal data are used to embed features to obtain small-scale features, and the small-scale features are downsampled to obtain mesoscale features. S2. Multimodal feature fusion at the same scale: The large-scale, meso-scale, and small-scale features of each modal data enter Stage 1. Stage 1 inputs the large-scale features of ocean circulation and sea surface temperature into the geographic-driven slice attention module GSAM for fusion processing to obtain the fused large-scale features. The mesoscale features of wind field and sea surface temperature are input into another GSAM and fused in the same way to obtain fused mesoscale features. The small-scale features of sea surface temperature, ocean circulation and wind field are spliced ​​and input into the diffusion-driven slice attention module DSAM for local dynamic feature enhancement to obtain small-scale enhanced features. S3. Multimodal feature fusion at different scales: The process involves fusing large-scale features, meso-scale features, and small-scale enhanced features, which then proceed to stage 2. In stage 2, the fused large-scale features and small-scale enhanced features are input into the time-driven slice attention module (TSAM) for cross-scale feature interaction, resulting in the final feature. Simultaneously, by fusing mesoscale features and small-scale enhanced features into another TSAM, the same cross-scale feature interaction is performed to obtain the features. ; S4, Prediction Output: feature and characteristics By concatenating and convolving the data, cross-scale features are obtained. The large-scale features are upsampled and then fused with the mesoscale features via skip connections, followed by further upsampling. This process is then combined with cross-scale features to obtain small-scale fused features. These small-scale fused features are input into a convolutional layer to output the predicted sea surface temperature. .

2. The method according to claim 1, characterized in that, Large-scale features of ocean circulation and sea surface temperature are input into the geographic-driven slice attention module GSAM for fusion processing to obtain fused large-scale features, including: S201. Obtain the geographic coordinates of the corresponding spatial grids for large-scale features of ocean circulation and large-scale features of sea surface temperature. Normalization is performed, and geographic features are obtained through location coding. Slicing operations were performed on the large-scale features of ocean circulation and sea surface temperature, respectively, to obtain the corresponding slice weight matrices and ocean circulation slices and sea surface temperature slices. Based on the corresponding slice weight matrices, geographic features were then processed. Weighted aggregation is performed to obtain ocean circulation geographic slices corresponding to ocean circulation slices and sea surface temperature geographic slices corresponding to sea surface temperature slices; the geographic similarity between ocean circulation geographic slices and sea surface temperature geographic slices is calculated to obtain a geographic similarity matrix. ; S202. The same slicing operation is performed on the large-scale features of ocean circulation and the large-scale features of sea surface temperature to obtain ocean circulation slices and sea surface temperature slices. Based on the ocean circulation slices, a query vector Q is constructed by querying the linear mapping parameter matrix. Based on the sea surface temperature slices, a key vector K and a value vector V are constructed by the key linear mapping parameter matrix and the value linear mapping parameter matrix, respectively. Introducing a geographic similarity matrix As a bias term, the similarity score between the query vector Q and the key vector K is corrected by element-wise multiplication and element-wise addition, and normalized by the Softmax function to obtain the slice attention weight. The slice attention weight is then used to perform a weighted summation on the value vector V to obtain the fused slice. S203. Perform a reverse slicing operation on the fused slices to obtain the reconstructed features of all spatial locations, and then reorganize them to obtain the fused large-scale features.

3. The method according to claim 2, characterized in that, Slicing operations include: For input features Perform a linear mapping and normalize it using the Softmax function to obtain the slice weight matrix of the corresponding mode; Based on the slice weight matrix, the input features Weighted aggregation is performed to obtain slice features, and all slice features form a corresponding feature slice sequence. The reverse slicing operation includes: For input features The slice weight matrix generated during the slicing operation is used to assign the fused slice features back to their corresponding spatial locations, and the features from different slices at the same spatial location are weighted and aggregated to obtain the reconstructed features. A reverse slicing operation is performed on all spatial locations to obtain the reconstructed features of all spatial locations, which are then reorganized according to the original spatial grid size to obtain the fused spatial feature map.

4. The method according to claim 3, characterized in that, Small-scale features of sea surface temperature, ocean circulation, and wind field are concatenated and then input into the diffusion-driven slice attention module DSAM for local dynamic feature enhancement to obtain small-scale enhanced features, including: S204, small-scale features of sea surface temperature, small-scale features of ocean circulation, and small-scale features of wind field are concatenated to obtain small-scale features. The small-scale features are input into the diffusion-driven slice attention module DSAM, which performs convolution operations through diffusion convolution kernels. The sum of the convolution results is added to the small-scale features to obtain diffusion-enhanced features. S205. Using the same slicing operation as in GSAM, slice the diffusion enhancement feature and the small-scale feature respectively to obtain the diffusion enhancement feature slice sequence and the small-scale feature slice sequence. S206. Construct a query vector Q based on the diffusion-enhanced feature slice sequence, and construct the key vector K and value vector V in slice-level attention based on the small-scale feature slice sequence; S207. Based on the query vector Q and key vector K, the similarity score is corrected, and normalized using the Softmax function to obtain slice-level attention weights. The slice-level attention weights are then used to perform a weighted summation on the value vector V to obtain the fused slice features. ; S208. Using the same reverse slicing operation as in GSAM, the fused slice features are... The spatial feature map is restored, resulting in small-scale enhanced features.

5. The method according to claim 1, characterized in that, The process involves fusing large-scale features, mesoscale features, and small-scale enhanced features, which then proceed to stage 2. In stage 2, the fused large-scale features and small-scale enhanced features are input into the time-driven slice attention module (TSAM) for cross-scale feature interaction, resulting in the final feature. ,include: S301. The input of large-scale features and small-scale enhanced features into the time-driven slice attention module TSAM is fused. TSAM performs time encoding on the time information of the past ten steps to obtain time features. S302 and time features are respectively input into the multilayer perceptron. and multilayer perceptron After being normalized by Softmax, large-scale time weights and small-scale time weights are obtained respectively. S303. Temporal modulation is applied to the fused large-scale features and small-scale enhanced features using large-scale temporal weights and small-scale temporal weights, respectively, to obtain the modulated fused large-scale features and modulated small-scale enhanced features. S304. Using the same slicing operation as in GSAM, slice the modulated fused large-scale features and the modulated small-scale enhanced features respectively to obtain the fused large-scale feature slice sequence and the small-scale enhanced feature slice sequence. S305. Based on the fusion of large-scale feature slice sequences, a query vector Q is constructed by querying the linear mapping parameter matrix. Based on the small-scale enhanced feature slice sequences, a key vector K and a value vector V are constructed by the key linear mapping parameter matrix and the value linear mapping parameter matrix, respectively. S306. Based on the query vector Q and key vector K, the similarity score is corrected, and normalized using the Softmax function to obtain slice-level attention weights. The slice-level attention weights are then used to perform a weighted summation on the value vector V to obtain the fused slice features. ; S307. Use the same reverse slicing operation as in GSAM to merge the slice features. Restore to spatial feature map, obtain features .

6. A multi-scale sea surface temperature prediction system based on hierarchical cross-modal fusion, characterized in that, include: The multimodal and multiscale feature extraction module is used for multimodal and multiscale feature extraction: input ocean circulation and sea surface temperature modal data are embedded to obtain small-scale features of each modality, and the small-scale features of each modality are extracted by progressive downsampling to obtain mesoscale and large-scale features of each modality; input wind field modal data are embedded to obtain small-scale features, and the small-scale features are downsampled to obtain mesoscale features; The multimodal feature fusion module at the same scale is used for multimodal feature fusion at the same scale: the large-scale features, meso-scale features and small-scale features of each modality data enter stage 1. Stage 1 inputs the large-scale features of ocean circulation and the large-scale features of sea surface temperature into the geographic-driven slice attention module GSAM for fusion processing to obtain the fused large-scale features. The mesoscale features of wind field and sea surface temperature are input into another GSAM and fused in the same way to obtain fused mesoscale features. The small-scale features of sea surface temperature, ocean circulation and wind field are spliced ​​and input into the diffusion-driven slice attention module DSAM for local dynamic feature enhancement to obtain small-scale enhanced features. A multimodal feature fusion module at different scales is used for multimodal feature fusion at different scales: fusing large-scale features, fusing medium-scale features, and small-scale enhanced features into stage 2. Stage 2 inputs the fused large-scale features and small-scale enhanced features into the time-driven slice attention module TSAM for cross-scale feature interaction to obtain the features. Simultaneously, by fusing mesoscale features and small-scale enhanced features into another TSAM, the same cross-scale feature interaction is performed to obtain the features. ; The prediction output module is used to output the predicted sea surface temperature: features and characteristics By concatenating and convolving the data, cross-scale features are obtained. The large-scale features are upsampled and then fused with the mesoscale features via skip connections. This fusion is followed by further upsampling and fusion with cross-scale features to obtain small-scale fused features. These small-scale fused features are then input into a convolutional layer to output the predicted sea surface temperature. .

Images