A Deep Learning-Based Method and System for Reconstructing Ocean Subsurface Salinity

By using deep learning models to extract features and enhance attention in ocean surface satellite remote sensing data, the problem of insufficient spatial resolution in ocean subsurface salinity data was solved, and high-precision ocean subsurface salinity reconstruction was achieved.

CN119068207BActive Publication Date: 2026-11-17NAT UNIV OF DEFENSE TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411176790.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2026-11-17
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

Existing technologies lack sufficient spatial resolution for reconstructing ocean subsurface salinity data, making it difficult to capture small signals, and also suffer from insufficient reconstruction accuracy and generalization ability.

Method used

A deep learning-based approach is adopted to acquire satellite remote sensing observation data of the ocean surface, and feature extraction and attention enhancement are performed using a deep learning model to reconstruct the ocean subsurface salinity, including local and global attention enhancement and multi-scale feature fusion. A deep learning system is constructed for data reconstruction.

Benefits of technology

It achieves high-resolution reconstruction of ocean subsurface salinity data, with better generalization ability and reconstruction accuracy, and is simple and easy to operate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068207B_ABST
    Figure CN119068207B_ABST
Patent Text Reader

Abstract

This application relates to a deep learning-based method and system for reconstructing subsurface salinity in the ocean. The method includes acquiring multiple input data sets, feeding these data sets into a pre-defined deep learning model, extracting corresponding feature vectors from each input data set using the deep learning model, and fusing the feature vectors from multiple input data sets to obtain a first feature map. The first feature map is then enhanced with local and global attention to obtain a second feature map. Finally, the ocean subsurface salinity is reconstructed based on the second feature map. This method addresses the problem of insufficient temporal and spatial resolution in current ocean subsurface salinity data, making it difficult to capture small and medium-sized signals in the subsurface. It can rapidly generate high-resolution ocean subsurface salinity data based on high-resolution sea surface remote sensing observation data. Compared to existing solutions, it offers superior generalization ability, high reconstruction accuracy, low cost, and ease of implementation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of marine data processing technology, and in particular to a method and system for reconstructing marine subsurface salinity based on deep learning. Background Technology

[0002] Salinity, as a variable describing the physical state of the ocean, is considered a key factor in the ocean system, closely related to global ocean circulation, global climate, and ocean-climate system changes. Monitoring long-term and short-term variations in sea surface salinity is crucial. The three-dimensional salinity field of the ocean subsurface is an important parameter for describing the characteristics of ocean physical properties and understanding ocean physical processes. Obtaining accurate three-dimensional salinity information of the ocean subsurface is of great significance for understanding the ocean, developing and utilizing the ocean, and conducting marine scientific research. Ocean salinity field information obtained through in-situ observation methods is sparse in spatiotemporal distribution and lacks temporal continuity. Satellite remote sensing can obtain global ocean surface salinity information, but it cannot obtain three-dimensional salinity information of the ocean subsurface. Therefore, many studies combine ocean surface satellite remote sensing observation data (sea surface temperature, sea surface salinity, sea surface height, and sea surface wind field, etc.) with sparse in-situ profile observation data to establish a mapping relationship from the surface to the underwater layer, achieving the reconstruction of spatiotemporally continuous three-dimensional salinity data of the ocean subsurface based on multi-source sea surface remote sensing data.

[0003] The main methods for reconstructing three-dimensional salinity data of the ocean subsurface can be divided into: statistical methods, dynamic methods, and empirical methods based on sound propagation time.

[0004] Statistical methods are used to establish statistical relationships between sea surface satellite remote sensing observations and subsurface salinity information. These methods primarily include regression statistics, empirical orthogonal decomposition, and variational methods. Empirical orthogonal decomposition performs EOF decomposition on temperature-salinity profile observation data to identify the amplitudes of certain modes and establish regression relationships with sea surface temperature or height. Variational methods use data assimilation to combine observational data with climatological data to generate a three-dimensional salinity field. Dynamic methods utilize dynamic models combined with sea surface information to simulate internal ocean dynamic processes. Based on dynamic constraints or numerical models, satellite remote sensing data of the sea surface is transformed into dynamic control variables, achieving a mapping from surface data to subsurface salinity. Empirical methods based on acoustic propagation time establish empirical models based on sound propagation time to invert underwater temperature-salinity structures.

[0005] The aforementioned traditional methods are limited to a specific region, have poor generalization ability, and the accuracy of reconstructed salinity is not high. Summary of the Invention

[0006] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0007] A first aspect of this application provides a deep learning-based method for reconstructing subsurface salinity in the ocean, the method comprising:

[0008] Acquire various input data, which are satellite remote sensing observation data of the ocean surface;

[0009] By inputting the various input data into a preset deep learning model, the ocean subsurface salinity reconstructed by the deep learning model is obtained;

[0010] The deep learning model reconstructs the ocean subsurface salinity in the following manner:

[0011] The corresponding feature vectors are extracted from each type of input data, and the feature vectors of multiple types of input data are fused to obtain a first feature map;

[0012] The first feature map is enhanced with local and global attention to obtain the second feature map;

[0013] The ocean subsurface salinity is reconstructed based on the second feature map.

[0014] This application provides a deep learning-based method for reconstructing ocean subsurface salinity. This method addresses the problem of insufficient spatial resolution in current ocean subsurface salinity data, making it difficult to capture small and medium-sized signals in the ocean subsurface. It can rapidly generate high-resolution ocean subsurface salinity data based on high-resolution sea surface remote sensing observation data, providing a novel approach for ocean subsurface salinity data reconstruction. Compared to existing solutions, it offers superior generalization ability, high reconstruction accuracy, low cost, and ease of implementation.

[0015] In some embodiments of this application, the step of enhancing the first feature map with local and global attention to obtain the second feature map includes the following steps:

[0016] The first feature map is enhanced with local attention and global attention to obtain the first enhanced feature map. The first enhanced feature map and the second feature map are then stacked by channel dimension to obtain the stacked feature map.

[0017] A second enhancement of local and global attention is performed on the stacked feature map to obtain a second enhanced feature map. The second enhanced feature map, the first feature map, and the stacked feature map are then stacked along the channel dimension to obtain a second feature map.

[0018] The first enhancement step is the same as the second enhancement step, and the first enhancement based on the first feature map, performing local and global attention to obtain the first enhanced feature map, includes:

[0019] SA channel attention enhancement is performed based on the first feature map to obtain the first intermediate feature map;

[0020] The second intermediate feature map is obtained by multiplying the first intermediate feature map by the first feature map in the channel dimension;

[0021] The second intermediate feature map is flattened into a third intermediate feature map;

[0022] Multi-head attention enhancement is performed based on the third intermediate feature map to obtain the fourth intermediate feature map;

[0023] A fifth intermediate feature map is obtained by performing a residual connection based on the fourth intermediate feature map and the third intermediate feature map;

[0024] The sixth intermediate feature map is extracted from the fifth intermediate feature map using a multilayer perceptron.

[0025] The fifth intermediate feature map is flattened to form the seventh intermediate feature map;

[0026] Residual connections are performed based on the seventh intermediate feature map and the sixth intermediate feature map to obtain the feature map after the first enhancement.

[0027] In some embodiments of this application, the step of extracting corresponding feature vectors from each type of input data and fusing the corresponding feature vectors of multiple types of input data to obtain a first feature map includes:

[0028] Extract the corresponding multi-scale feature map from each of the input data;

[0029] Multi-scale fusion is performed on the multi-scale feature maps corresponding to each type of input data to obtain a multi-scale fused feature map corresponding to each type of input data.

[0030] The first feature map is obtained by fusing the multi-scale fusion feature maps corresponding to the various input data.

[0031] In some embodiments of this application, extracting the corresponding multi-scale feature map from each type of input data includes:

[0032]

[0033] Among them, B′ m Let C represent the m-th type of input data. 1×1 C 3×3 C5×5 C 7×7 Let f represent the mapping functions for convolution operations with kernel sizes of 1x1, 3x3, 5x5, and 7x7, respectively. LReLU The mapping function representing the LReLU activation function. Represents a residual join, concat represents a stacking operation, F CNN-Multiscale-1-m This represents the multi-scale feature vector corresponding to the m-th type of input data.

[0034] In some embodiments of this application, the multi-scale fusion of the multi-scale feature maps corresponding to each type of input data includes:

[0035]

[0036] Among them, F CNN-Multiscale-1-m Let C represent the multi-scale feature vector corresponding to the m-th type of input data. 1×1 C 3×3 C 5×5 and C 7×7 f represents the mapping function f for convolution operations with kernels of 1x1, 3x3, 5x5, and 7x7. LReLU The mapping function representing the LReLU activation function. Represents a residual join, concat represents a stacking operation, F CNN-Multiscale-2-m This represents the multi-scale fusion feature vector corresponding to the m-th type of input data.

[0037] In some embodiments of this application, the fusion of multi-scale fusion feature vectors corresponding to multiple input data includes:

[0038]

[0039] Among them, F CNN-Multiscale-2-1 F CNN-Multiscale-2-2 F CNN-Multiscale-2-M This represents the multi-scale fusion feature map corresponding to the first type of input data, the second type of input data, and the Mth type of input data. `concat` represents the stacking operation. 1×1 The mapping function representing a 1x1 convolution operation. f represents the mapping function for average pooling layers of 3x3, 5x5, 7x7, and 9x9. LReLU F represents the mapping function of the LReLU activation function. CNN-Multiscale This represents the first feature map.

[0040] In some embodiments of this application, the loss function of the deep learning model includes:

[0041]

[0042] Where Huber Loss represents the loss function, δ represents the hyperparameter, O′ represents the ocean subsurface salinity output by the deep learning model, and L′ represents the label data.

[0043] In some embodiments of this application, the input data includes: sea surface temperature, sea surface salinity, and sea surface height.

[0044] The second aspect of this application provides a deep learning-based ocean subsurface salinity reconstruction system, the system comprising:

[0045] The data acquisition unit is used to acquire various input data, which are satellite remote sensing observation data of the ocean surface.

[0046] The data reconstruction unit is used to input the various input data into a preset deep learning model to obtain the ocean subsurface salinity reconstructed by the deep learning model.

[0047] The deep learning model reconstructs the ocean subsurface salinity in the following manner:

[0048] The corresponding feature map is extracted from each type of input data, and the feature maps of multiple types of input data are fused to obtain the first feature map;

[0049] The first feature map is enhanced with local and global attention to obtain the second feature map;

[0050] The ocean subsurface salinity is reconstructed based on the second feature map.

[0051] A third aspect of this application provides an electronic device including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform the above-described deep learning-based method for reconstructing ocean subsurface salinity.

[0052] It is understood that the beneficial effects of the second to third aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This application provides a deep learning-based method for reconstructing marine subsurface salinity in one embodiment.

[0055] Figure 2 This is a schematic flowchart of a marine subsurface salinity reconstruction method provided in another embodiment of this application;

[0056] Figure 3 This is a structural diagram of the branch of the first part of the convolutional multi-scale feature extraction and fusion module for extracting multi-scale feature maps of a certain variable, provided in one embodiment of this application;

[0057] Figure 4 This is a structural diagram of the third part of the convolutional multi-scale feature extraction and fusion module provided in one embodiment of this application, which fuses a multi-scale fused feature map of all variables from a wider range;

[0058] Figure 5 This is a structural diagram of a convolutional multi-scale feature extraction and fusion module provided in one embodiment of this application;

[0059] Figure 6 This is a structural diagram of a local attention module provided in one embodiment of this application;

[0060] Figure 7 This is a structural diagram of a multi-head attention module provided in one embodiment of this application;

[0061] Figure 8 This is a structural diagram of a global attention module provided in one embodiment of this application;

[0062] Figure 9 This is a structural diagram of a global and local alternating attention module provided in one embodiment of this application;

[0063] Figure 10 This is a structural diagram of the output module provided in one embodiment of this application;

[0064] Figure 11 This is a structural diagram of a deep learning model provided in one embodiment of this application;

[0065] Figure 12 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0069] like Figure 1 One embodiment of this application provides a deep learning-based method for reconstructing subsurface salinity in the ocean, the method comprising steps S110 to S120:

[0070] Step S110: Acquire various input data, which are satellite remote sensing observation data of the ocean surface.

[0071] Step S120: Input multiple input data into a preset deep learning model to obtain the ocean subsurface salinity reconstructed by the deep learning model;

[0072] The deep learning model reconstructs the salinity of the ocean subsurface layer in the following way:

[0073] The corresponding feature maps are extracted from each type of input data, and the feature maps of multiple input data are fused to obtain the first feature map;

[0074] The first feature map is enhanced with local and global attention to obtain the second feature map.

[0075] The salinity of the ocean subsurface is reconstructed based on the second feature map.

[0076] In this embodiment, the input data is high-precision satellite remote sensing observation data of the ocean surface, including but not limited to: sea surface temperature data, sea surface height data, and sea surface salinity data. Multiple input data are used as input variables for the deep learning model.

[0077] Deep learning models can be built based on various deep learning algorithms, such as models constructed using CNN networks. A deep learning model first extracts corresponding feature maps from each type of input data and fuses these feature maps to obtain a first feature map. This step allows for the extraction and fusion of features from different input variables, resulting in a first feature map with rich feature information from different input variables, providing data support for subsequent processing. Then, the first feature map undergoes local and global attention enhancement to obtain a second feature map. Local attention is used to extract sub-regions that play a crucial role in reconstructing subsurface salinity globally, while global attention calculation maintains focus on these key sub-regions and incorporates more spatial information for interaction, achieving global attention calculation with local emphasis. Finally, the ocean subsurface salinity can be reconstructed based on the second feature map.

[0078] This method addresses the problem of insufficient spatial resolution in current ocean subsurface salinity data, which makes it difficult to capture small and medium-sized signals in the ocean subsurface. It can rapidly generate high-resolution ocean subsurface salinity data based on high-resolution sea surface remote sensing observations, providing a novel approach for ocean subsurface salinity data reconstruction. Compared to existing methods, it offers superior generalization ability, high reconstruction accuracy, low cost, and ease of implementation.

[0079] In some embodiments of this application, local attention and global attention enhancement are applied to the first feature map to obtain a second feature map, including steps S210 to S220. Specifically:

[0080] Step S210: Perform the first enhancement of local attention and global attention based on the first feature map to obtain the first enhanced feature map, and stack the first enhanced feature map and the second feature map by channel dimension to obtain the stacked feature map.

[0081] Step S220: Perform a second enhancement of local attention and global attention based on the stacked feature map to obtain the second enhanced feature map. Stack the second enhanced feature map, the first feature map, and the stacked feature map along the channel dimension to obtain the second feature map.

[0082] The first enhancement step is the same as the second enhancement step, and the first enhancement is performed based on the first feature map, involving both local and global attention, to obtain the enhanced feature map, including:

[0083] SA channel attention enhancement is performed based on the first feature map to obtain the first intermediate feature map;

[0084] The second intermediate feature map is obtained by multiplying the first intermediate feature map by the first feature map in the channel dimension;

[0085] The second intermediate feature map is flattened into a third intermediate feature map;

[0086] Multi-head attention enhancement is performed based on the third intermediate feature map to obtain the fourth intermediate feature map;

[0087] The fifth intermediate feature map is obtained by performing residual connection based on the fourth and third intermediate feature maps;

[0088] The sixth intermediate feature map is extracted from the fifth intermediate feature map using a multilayer perceptron.

[0089] The fifth intermediate feature map is flattened to form the seventh intermediate feature map;

[0090] Residual connections are performed based on the seventh and sixth intermediate feature maps to obtain the feature map after the first enhancement.

[0091] In this embodiment, the first feature map is enhanced with local attention and global attention. The local attention mechanism is based on convolution, and the global attention mechanism is based on Transformer. The SA channel attention of local attention is used to extract the sub-regions that play a key role in the reconstruction of subsurface salinity in the global region. On this basis, multilayer perceptron calculation of global attention is performed to maintain attention to the key sub-regions and introduce more spatial information for interaction. This realizes global attention calculation with local emphasis, which can obtain more global information without ignoring the key sub-regions, and achieve complementary advantages.

[0092] In this embodiment, the first feature map is enhanced twice using dense connections (i.e., channel stacking), which can reduce information loss and ultimately improve the reconstruction effect.

[0093] In some embodiments of this application, a corresponding feature map is extracted from each type of input data, and the feature maps corresponding to multiple input data are fused to obtain a first feature map, including steps S310 to S330. Specifically:

[0094] Step S310: Extract the corresponding multi-scale feature map from each type of input data.

[0095] Step S320: Perform multi-scale fusion on the multi-scale feature maps corresponding to each type of input data to obtain the multi-scale fused feature map corresponding to each type of input data.

[0096] Step S330: Fuse the multi-scale fusion feature maps corresponding to multiple input data to obtain the first feature map.

[0097] In this embodiment, firstly, multi-scale feature maps of each input data are extracted independently; secondly, the multi-scale feature maps of each input data are fused separately to ensure that each grid region of the multi-scale fused feature map of each input data incorporates rich spatial information of a multi-scale range, providing a wider range of surrounding information reference for reconstruction, thus obtaining the multi-scale fused feature map of each input data; finally, from a larger scale range, the multi-scale fused feature maps of all input data are fused together, and by learning the implicit interaction relationships between different input data, information complementarity between different variables is achieved, thereby improving the accuracy of reconstruction.

[0098] This embodiment extracts and fuses multi-scale features of the input variables through multiple convolution operation branches of different sizes. The purpose is to reconstruct the subsurface salinity at each location, which is usually affected by the adjacent regions. By fusing features at different scales, spatial information at different scales will be fully utilized to provide more information support for the subsurface salinity reconstruction, thereby improving the accuracy of the final model in reconstructing salinity.

[0099] like Figures 2 to 11 Another embodiment of this application provides a method for reconstructing marine subsurface salinity, specifically:

[0100] Step S910: Obtain input data and label data, and combine the obtained input data and label data to form the dataset for training and testing the model. Specifically, this includes the following steps:

[0101] Input data types: Daily average sea surface salinity (0.25°×0.25°), sea surface temperature, and sea surface height data are used as input data, and daily average subsurface salinity (0.083°×0.083°) data are used as label data. All the above data are downloaded from 2010 to 2023, and each contains 5113 (N=5113) sample data, that is, sample data at 5113 time points. The sample data of all variables correspond one-to-one at the 5113 time points. The spatial range of the covered sea area is consistent, with longitude coverage from 105°E to 120°E and latitude coverage from 0° to 25°N. Therefore, the latitude span is lat=25° and the longitude span is lon=15°.

[0102] Input data source: Daily average 0.25° × 0.25° sea surface temperature data are from the daily best interpolation V2.1SST data of the National Oceanic and Atmospheric Administration (NOAA);

[0103] The daily average sea surface salinity data of 0.25°×0.25° comes from the SMOS satellite (hybrid orbit) average 10-day and monthly salinity field products of the CATDS center, the downstream data processing center of SMOS;

[0104] The daily average sea surface height data of 0.25° × 0.25° are from the Copernicus Ocean Service’s global ocean grid L4 sea surface height and derived variables reprocessed product from 1993;

[0105] Label data: Daily average 0.083°×0.083° ocean subsurface salinity data are from GLORYS12V1 (Global Ocean Physics Reanalysis) global ocean physical reanalysis product. Therefore, the grid size s°=0.83°; 15 depths [0.5 m, 5 m, 10 m, 15 m, 20 m, 25 m, 35 m, 55 m, 75 m, 100 m, 120 m, 140 m, 160 m, 180 m, 200 m] are selected for reconstruction, i.e. d=15.

[0106] First, the input data is paired with the label data to obtain a time-space matching data pair matrix X. The data pair matrix X is organized as follows:

[0107]

[0108] Here, each input variable in the data pair matrix X is denoted as v. m,n v m,n This represents the nth sample data of the mth variable, where m ∈ [1, M], M = 3. The first three samples correspond to the variables sea surface salinity, sea surface temperature, and sea surface height, respectively, for a total of N = 5113 samples. Each sample data is a column in matrix X. Due to the same time resolution, one time point corresponds to one sample data. Each column contains 3 variables, with the first 3 being the input variables and the last row being the label data l. n This refers to ocean reanalysis salinity data containing multiple depths. Each sample data has a label that corresponds one-to-one with the input variable, maintaining consistency in time and space. Therefore, there are a total of 5113 sample data with labels.

[0109] Then, the paired data matrix X is divided into a training set T and a test set S in an 8:2 ratio. That is, the first 80% of the rows of X are used as the training set T, and the last 20% of the rows are used as the test set S. The training set has 4090 samples and the test set has 1023 samples.

[0110] Take 16 (b=16) sample data points randomly from the training set T or the test set S as a batch B. Standardize the sea surface salinity, sea surface temperature, and sea surface height data in the batch B to a mean of 0 and a standard deviation of 1, to obtain a standardized batch B′. The normalization formula is as follows:

[0111]

[0112] Among them, Bm This represents a certain input variable in batch B. To find the mean of all grid values ​​for a certain variable across all samples in the training set. Let B′ be the standard deviation of all grid values ​​of a certain variable across all samples in the training set after batch standardization. m The corresponding label data also needs to be standardized. The standardization formula for label data is as follows:

[0113]

[0114] Among them, B l This represents the label data in batch B, μ l σ is the mean of all grid values ​​across all depths of all samples in the labeled data of the training set. l The standard deviation of all grid values ​​across all samples and depths in the training set's labeled data is denoted as B′ for this batch of standardized labeled data. l .

[0115] Finally, the standardized batch of data B′ is stored in a 4-dimensional matrix format. The four dimensions are the number of samples in the batch, the number of variables, the latitudinal span, and the longitudinal span, abbreviated as [16, 3, 25°, 15°]. lat represents the latitudinal span of the area for reconstructing subsurface salinity in the horizontal direction, and lon represents the longitudinal span of the reconstructed area in the horizontal direction. The nearest neighbor interpolation algorithm is used to ensure that the horizontal spatial resolution of all input variables is consistent with the label data, which is 1 / 12° × 1 / 12°. The longitudinal and latitudinal spans are converted into grid numbers. The size of a single batch of data P′ is [16, 3, 300, 180].

[0116] Step S920: Build the deep learning model. The expression of the deep learning model is as follows:

[0117]

[0118] in, This illustrates the mapping relationship between the input and output of a deep learning model. The input is a normalized batch of multivariate input B′, and the output is a batch of subsurface ocean salinity data O, θ={W k b k} represents the weights and biases of the k-th layer in the model, specifically including the following steps:

[0119] Set up a deep learning environment, install the Python-based deep learning framework PyTorch on the server, and build deep learning models based on the PyTorch framework.

[0120] Step S920 provides a deep learning model that mainly includes: a convolutional multi-scale feature extraction and fusion module, and a global and local alternating attention module. Based on the built deep learning environment, the convolutional multi-scale feature extraction and fusion module is created to extract multi-scale features from multiple input variables in the input batch and perform feature fusion to obtain the multi-scale fused feature F. CNN-Multiscale The expression is as follows:

[0121] F CNN-Multiscale =f CNN-Multiscale (B′ n )

[0122] Among them, f CNN-Multiscale This represents the mapping function of the convolutional multi-scale feature extraction and fusion module. This module extracts and fuses multi-scale features of the input variables through multiple convolutional operation branches of different sizes. The purpose is to reconstruct the subsurface salinity at each location. It is usually affected by the adjacent regions. By fusing features at different scales, it can make full use of spatial information at different scales and provide more information support for the subsurface salinity reconstruction.

[0123] The following section focuses on the convolutional multi-scale feature extraction and fusion module:

[0124] The convolutional multi-scale feature extraction and fusion module consists of three parts. First, it independently extracts the multi-scale feature maps of each variable. Second, it fuses the multi-scale feature maps of each variable separately, ensuring that each grid region of the multi-scale fused feature map of each variable incorporates rich spatial information across multiple scales, providing a wider range of surrounding information references for reconstruction, and obtaining the multi-scale fused feature map of each variable. Finally, it fuses the multi-scale fused feature maps of all variables from a larger scale range, and by learning the implicit interaction relationships between different variables, it achieves information complementarity between different variables, thereby improving the accuracy of reconstruction.

[0125] like Figure 3 First, the multi-scale features of each variable are extracted independently by three independent and structurally consistent branches. These branches share parameters; that is, parameter updates in one branch during model training are reflected in the other two branches, reducing the overall number of model parameters and simplifying model complexity. in1 This represents the number of channels in the input feature map. Since each variable is processed separately, the number of channels in the input feature map is C. in1 =1,C out1 The number of channels in the output feature map is set to 16. The specific structure of the branch is as follows: First, there are 4 convolutional layers in parallel. The kernel size of each layer is different, and residual connections are used to avoid gradient explosion or vanishing, reducing the difficulty of model training. Specifically, the first layer is a sequential connection. One 1×1 convolutional kernel (1×1 stride, no edge padding), one LReLU activation layer. The first layer consists of a 3×3 convolutional kernel (1×1 stride, 1 layer of edge padding with padding values ​​set to match neighboring values), a single LReLU activation layer, and residual connections between the results of two LReLU activation layers; the second layer is a sequential connection. One 3×3 convolutional kernel (stride 1×1, one layer of edge padding with padding values ​​set to match neighboring values), and one LReLU activation layer. The first layer consists of a 5×5 convolutional kernel (1×1 stride, 2 layers of edge padding with padding values ​​set to match neighboring values), a single LReLU activation layer, and residual connections between the results of the two LReLU activation layers; the third layer is a sequential connection. One 5×5 convolutional kernel (stride 1×1, two layers of edge padding, padding values ​​set to match neighboring values), and one LReLU activation layer. The first layer consists of a 7×7 convolutional kernel (1×1 stride, 3 layers of edge padding with padding values ​​set to match neighboring values), a LReLU activation layer, and residual connections after two LReLU activation layers; the fourth layer is a sequential connection. One 7×7 convolutional kernel (stride 1×1, 3 layers of edge padding, padding values ​​set to match neighboring values), and one LReLU activation layer. The system consists of four 9×9 convolutional kernels (with a stride of 1×1, 4 layers of edge padding, and padding values ​​set to match those of neighboring values) and one LReLU activation layer. The results of the two LReLU activation layers are then concatenated using residual connections. Finally, the results of the four convolutional layers are merged along the channel dimension to obtain the multi-scale feature map F for a specific variable. CNN-Multiscale-1-m The number of channels is C out1 =16.

[0126] The LReLU activation layer described above uses the LReLU activation function. If the input value is greater than 0, the output value remains unchanged; if it is less than or equal to 0, the output value is the input value multiplied by a small slope value ε. The expression for the LReLU function is as follows:

[0127]

[0128] f LReLU This is a mapping function for the LReLU activation function, where `input` represents the function's input and `active input` represents the activation result. The expressions for the branches that process each variable individually are as follows:

[0129]

[0130] Among them, B′ m For a variable in the input batch, with dimensions [16, 1, 300, 180], F CNN-Multiscale-1-mThis represents a multi-scale feature map of a variable, with dimensions [16, 16, 300, 180], f CNN-Multiscale-1 C represents the mapping function C, which is the branch used to extract multi-scale features of each variable in the first part of the convolutional multi-scale feature extraction and fusion module. 1×1 C 3×3 C 5×5 C 7×7 and C 9×9 Let represent the mapping functions for convolution operations with kernel sizes of 1×1, 3×3, 5×5, 7×7, and 9×9, respectively. This indicates a residual connection, where the grids at corresponding positions of two feature maps are added together, and the resulting size remains unchanged. concat indicates that feature maps are stacked in the channel dimension, increasing the number of channels while keeping the size of the other dimensions unchanged.

[0131] The multi-scale feature maps of each variable are fused separately. Following the branches that independently extract the multi-scale feature maps of each variable, corresponding independent multi-scale feature fusion branches follow. These branch structures are consistent with the branch structures for extracting multi-scale features of each variable, and the parameters of each branch are still shared. By inputting the multi-scale feature map of a certain variable from the preceding sequence into convolutional layers of different sizes, the preceding multi-scale feature maps are fused from different scales to further extract multi-scale features, resulting in a multi-scale fused feature map of a certain variable. The number of channels in the input feature map is C. Since each variable is processed separately, the number of channels in the input multi-scale feature map in a certain variable branch is C. in2 =C out1 =16, the number of channels in a certain multi-scale fused feature map output is C. out2 =64, the modular expression for fusing multi-scale feature maps of various variables is:

[0132]

[0133] Among them, F CNN-Multiscale-2-m This represents a multi-scale fused feature map of a certain variable, with dimensions [16, 64, 300, 180], f CNN-Multiscale-2 This represents the mapping function for the branch that fuses the multi-scale feature maps of various variables in the second part of the convolutional multi-scale feature extraction and fusion module.

[0134] like Figure 4 and Figure 5 To fuse multi-scale feature maps of all variables at a larger scale, the multi-scale feature maps of each variable obtained from the three branches are first stacked in the channel dimension, with a channel count of 3C. out2 =192, then input into 4 independent pooling convolutional layers respectively. The pooling layer expands the receptive field of the convolutional layer, allowing information to be obtained from a larger spatial range. C in3=3C out2 =192, representing the number of channels in the input feature map, C out3 The output feature map has 256 channels. Each branch has the following structure: The first layer is a 3×3 average pooling layer (1×1 stride, 1 layer of edge padding with values ​​matching neighboring values). The pooling layer has the same number of channels as the input feature map, and includes one LReLU activation layer. One 1×1 convolutional kernel, one LReLU activation layer, output feature map contains The first layer has one channel; the second layer is a 5×5 max pooling layer (with a stride of 1×1, two layers of edge padding, and the padding value is set to match the neighboring values), and an LReLU activation layer. One 1×1 convolutional kernel, one LReLU activation layer, output feature map contains The third layer consists of one 7×7 average pooling layer (with a stride of 1×1, 3 layers of edge padding, and the padding values ​​are set to match the neighboring values), and one LReLU activation layer. One 1×1 convolutional kernel, one LReLU activation layer, output feature map contains The fourth layer consists of one 9×9 max pooling layer (with a stride of 1×1, 4 layers of edge padding, and the padding values ​​are set to match the neighboring values), and one LReLU activation layer. One 1×1 convolutional kernel, one LReLU activation layer, output feature map contains Each channel; the edge padding of the above pooling layers is to ensure that the output feature map size remains unchanged; finally, the results of the four layers are stacked in the channel dimension to obtain an output feature map containing C. out3 =256 channels, the modular expression for the multi-scale fusion feature map that integrates all variables is as follows:

[0135]

[0136] Among them, F CNN-Mult i scale-2 This represents the feature map obtained by stacking multi-scale fusion feature maps of three variables, with dimensions [16, 192, 300, 180]. CNN-Multiscale-3 The third part of the convolutional multi-scale feature extraction and fusion module represents the mapping function F, which fuses multi-scale fused feature maps from a larger scale range, integrating all variables. CNN-Multiscale This represents the output feature map of the convolutional multi-scale feature extraction and fusion module, with dimensions [16, 256, 300, 180]. This represents the mapping function for a 3×3 average pooling layer. This represents the mapping function for a 5×5 max pooling layer. This represents the mapping function for a 7×7 average pooling layer. Represents the mapping function for a 9×9 average pooling layer;

[0137] like Figures 6 to 9 The following introduces the global and local alternating attention module:

[0138] The expression for the alternating global and local attention module is:

[0139] F Alternate-Attention =f Alternate-Attention (F CNN-Multiscale )

[0140] Among them, F Alternate-Attention f represents the output feature map of the global and local alternating attention module. Alternate-Attention This represents the mapping function for the alternating global and local attention module. The local attention mechanism of this module is implemented based on convolution, while the global attention mechanism is implemented based on Transformer. Local attention extracts the sub-regions that play a key role in the reconstruction of subsurface salinity in the global context. Based on this, global attention is then calculated to maintain attention to the key sub-regions and introduce more spatial information for interaction. This achieves global attention calculation with local emphasis, obtaining more global information while not neglecting key sub-regions, thus achieving complementary advantages.

[0141] The alternating global and local attention module comprises two basic modules: a local attention module followed by a global attention module. The local attention module is a concatenated SA spatial attention module. After the input multi-channel feature map is processed by the SA spatial attention module, the result is a multi-channel feature map weighted with spatial attention weights. The function expression of the local attention module is as follows:

[0142]

[0143] Among them, F Local-Attention f represents the output feature map of the local attention module. Local-Attention The mapping function represents the local attention module. This indicates that the corresponding dimensions are multiplied, and the calculated attention weights are assigned to the input feature map, f. SA This represents the mapping function of the SA spatial attention module; the local attention module does not change the size of the input feature map.

[0144] The SA spatial attention module first passes the data through a single-channel global max pooling layer and a single-channel global average pooling layer. These two results are then stacked along the channel dimension (2 channels), followed by a 7×7 convolutional layer to reduce the dimensionality to one channel. Finally, a sigmoid activation layer is applied to obtain the spatial attention feature map. At this point, F... SAThe output feature map size of the spatial attention module is [16, 1, H, W]; the SA spatial attention module can be expressed as:

[0145]

[0146] Among them, F SA M represents the output feature map of the SA spatial attention module. SA Let F represent the mapping function of the SA spatial attention module, and let C represent the multi-channel feature map input to the SA spatial attention module. 7×7 This represents a 7×7 convolutional kernel (stride 1×1, 3 layers of edge padding, padding values ​​set to match neighboring values); GloAvgPool represents a channel-wide global average pooling layer, expressed as:

[0147] F mean-C =torch.mean(F, 1, keepdim=True);

[0148] Among them, F mean-C This represents the feature map after channel-wise global average pooling, where "1" indicates averaging over the first dimension, i.e., the channel dimension. F mean-C The dimensions are [16, 1, H, W].

[0149] Similarly, GloMaxPool represents the channel-wide global max pooling layer, and its expression is:

[0150] F max-C =torch.max(F, 1, keepdim=True);

[0151] F max-C This represents the feature map after channel-wise global max pooling, where "1" indicates that the maximum value is taken in the first dimension, i.e., the channel dimension. F max-C The dimensions are [16, 1, H, W]; SA spatial attention module structure diagram.

[0152] The spatial attention feature map F SA Multiplying the corresponding grid of the feature map of the input SA spatial attention module with the feature map of this module yields the output features of the final local attention module structure map; F SA With only one channel, the value of each grid cell represents its importance in planar space, and multiplying corresponding grid cells is equivalent to assigning a spatial weight to the same grid cell across all channels in the feature map batch; the local attention module function expression is:

[0153]

[0154] Among them, F Local-Attention f represents the output feature map of the local attention module. Local-AttentionThe mapping function represents the local attention module. This indicates that the corresponding dimensions are multiplied, and the calculated attention weights are assigned to the input feature map, f. SA This represents the mapping function of the SA spatial attention module; as shown in the above steps, the number of input and output channels of the local attention module is the same, therefore, F Local-Attention Size and Input Feature Map F CNN-Multiscale The dimensions are consistent, all being [16, 256, 300, 180].

[0155] The global attention module is the multi-head attention module of the Transformer encoder. First, the input feature map is flattened to meet the input requirements of the Transformer encoder module. A sliding window of size 3×3 (w=3) is used, with a stride of 1. On the input feature map F, let's say the size is 16×C×H×W, where H×W forms a two-dimensional plane (length and width), resulting in B×C two-dimensional planes. The window slides progressively from top to bottom and from left to right. Each slide flattens the covered two-dimensional region into a one-dimensional vector, ultimately flattening it into a three-dimensional matrix F. flatten The size is 16×N×L, where N=H×W, L=9C, that is, 16 samples, each sample is a two-dimensional matrix of N rows and 9 columns, and then Ff l a tt The input `en` is fed into the multi-head attention module (MHA) of the Transformer encoder to capture global attention on the feature map. This is then fed into a single-layer multilayer perceptron (MLP). Finally, an inverse flattening operation restores the feature map output by the MHA to its original four-dimensional matrix size. To reduce information loss and lower training difficulty, a residual connection is used after the MHA and MLP. The global attention module function expression is as follows:

[0156] F flatten =f flatten (F Local-Attention )

[0157]

[0158] Among them, F Global-Attention f represents the output feature map of the global attention module. Global-Attention f represents the mapping function of the global attention module. MHA The mapping function f represents the multi-head attention module (MHA). MLP Let f be the mapping function of a multilayer perceptron MLP. flatten with f un-flatten The mapping function representing the flattening and unflattening operations. This indicates residual connection, which means that the grids at the same position in two feature maps of the same size are added together.

[0159] First, the input flattened feature map F is processed. flatten After passing through three identical fully connected Linear layers, the third dimension is expanded by a factor of head_num, resulting in three 3D matrices Q, K, and V. With head_num set to 8, the dimensions of Q, K, and V are all 16×(H×W)×72C. Using Python's `reshape` function, Q, K, and V are transformed into 4D matrices with dimensions of 16×8×(H×W)×9C. This involves splitting the original third dimension and performing SDPA scaling pointwise multiplication on Q, K, and V. Specifically, Q and K are first multiplied by a matrix multiplication, and then scaled to... The feature map F is obtained by multiplying the current by V and performing Softmax activation along the last dimension. The activation result is then multiplied by V, reshaped to restore the size to 16×(H×W)×72C, and finally passed through a fully connected Linear layer to restore the size to 16×(H×W)×9C. MHA The implementation process of the MHA multi-head attention module is expressed as follows:

[0160] Q = f Linear (F flatten ) = w Q F+b Q

[0161] K = f Linear (F flatten ) = w K F+b K

[0162] V = f Linear (F flatten ) = w V F+b V

[0163]

[0164] Among them, f Linear w is the activation function for the fully connected layer. Q W K W V Let b represent the weight matrix of the neurons in the three fully connected layers. Q b K b V The three fully connected layer bias values ​​are: reshape1 represents transforming Q, K, and V into a 4-dimensional matrix, resulting in a size of 16×8×(H×W)×9C; reshape2 represents restoring it to a 3-dimensional matrix, resulting in a size of 16×(H×W)×72C. The specific expression of the Softmax activation function is as follows:

[0165]

[0166] Here, input represents the input vector of the activation function. The aforementioned Softmax activation is performed along the last dimension, that is, the Softmax activation is performed sequentially along each vector of this dimension. As can be seen from the above steps, the input and output dimensions of the global attention module are the same.

[0167] By connecting the local attention modules first, followed by the global attention modules, we obtain an alternating global and local attention module, expressed as:

[0168] F Alternate-Attention =f Alternate-Attention (F CNN-Multiscale )

[0169] =f Global-Attention (f Local-Attention (F CNN-Multiscale ))

[0170] Among them, F Alternate-Attention f represents the output feature map of the global and local alternating attention module. Alternate-Attention This represents the mapping function for the global and local alternating attention modules; as can be seen from the above steps, the input and output sizes of the global and local alternating attention modules are consistent.

[0171] like Figure 10 The output module is constructed such that the number of layers in the output feature map matches the number of layers in the label data, resulting in reconstructed subsurface salinity data. The expression for the output module is as follows:

[0172] F Output =f Output (F Alternate-Attention ) = C 3×3 (Relu((f 5×5 (F Alternate-Attention ))))

[0173] Among them, F Output f represents the output feature map of the output module. Output Represents the mapping function of the output module, C 3×3 With C 5×5 Represents the convolution function with kernel sizes of 3×3 and 5×5, and ReLU represents the ReLU activation function.

[0174] After inputting 45 5×5 convolutional kernels into the multi-channel feature map, the dimensionality is reduced to 45 channels.

[0175] The 45-channel feature map is sequentially passed through one ReLU activation layer and 15 3×3 convolutional kernels to reduce the dimensionality of the feature map to a 15-channel output:

[0176] F Output =f Output(F Alternate-Attention ) = C 3×3 (f LReLU ((C 5×5 (F Alternate-Attention ))))

[0177] Among them, F Output f represents the output feature map of the output module. Output Let F represent the mapping function of the output module, and let F represent the multi-channel feature map of the input and output modules.

[0178] Combining one convolutional multi-scale feature extraction and fusion module, two alternating global and local attention modules, and one output module yields the deep learning model, expressed as follows:

[0179]

[0180] in, This represents the mapping relationship between the input and output of a deep learning model. The input is normalized batch multivariate data B′, and the output is batch O of subsurface ocean salinity data, θ={W k b k} represents the weights and biases of the k-th layer of the model.

[0181] The two alternating global and local attention modules are densely connected, i.e., they are stacked using a concat channel-dimensional stacking method. This stacks the feature maps that have passed through each of the alternating global and local attention modules, reducing information loss. The expression is as follows:

[0182]

[0183] That is, the feature map f obtained after passing through the first global and local alternating attention module. Alternate-Attention (F CNN-Multiscale ), and the feature map F of the input global and local alternating attention modules. Multi-CNN After stacking, the second global and local alternating attention module is input, and the final result is a stack of all the aforementioned feature maps, where F is the input feature map with a size of [16, 1024, 300, 180].

[0184] The deep learning model is obtained by combining the effects of one convolutional multi-scale feature extraction and fusion module, two alternating global and local attention modules, and one output module in a specific order, as shown in the following expression:

[0185]

[0186] in, This represents the mapping relationship between the input and output of a deep learning model. The input is normalized batch multivariate data B′, and the output is batch O of subsurface ocean salinity data, θ={W k b k} represents the weights and biases of the k-th layer of the model;

[0187] Step S930: Train the deep learning model using the obtained dataset to obtain the trained deep learning model. This specifically includes the following steps:

[0188] We choose Huber loss as the loss function for the obtained deep learning model. The expression for the Huber loss function is as follows:

[0189]

[0190] Huber Loss combines the advantages of MSE and MAE losses. Firstly, MSE loss is less complex than MAE loss in terms of gradient calculation, as the calculated gradient value is also dynamically changing, allowing MSE loss to converge faster and more accurately. Secondly, regarding outliers, MAE loss is less sensitive to outliers and is suitable for handling data with many outliers, while MSE loss is very sensitive to large deviations and is suitable for scenarios with few outliers. Therefore, Huber loss combines the advantages of both MSE and MAE losses, exhibiting MSE loss characteristics with small errors and MAE loss characteristics with large errors. It performs better in terms of sensitivity to outliers and error balance, and Huber loss is differentiable everywhere, improving the accuracy of gradient calculation when the loss is small. The MSE loss is represented by |L′-O′|, and the MAE loss is represented by |L′-O′|. δ is a hyperparameter set to 0.2, whose magnitude determines the emphasis of Huber loss on MSE and MAE losses. When |L′-O′|≤0.2, it becomes MSE, with the gradient gradually decreasing, ensuring the model obtains the global optimum more accurately. When |L′-O′|>0.2, it becomes similar to MAE, with the gradient always approximating δ, ensuring the parameters are updated quickly during model training. Furthermore, O′ represents the subsurface salinity reconstruction data, and L′ represents the normalized salinity data from multiple depths of the ocean reanalysis, with both having the same size.

[0191] The training set T is input into the deep learning model in batches for training and fitting. The parameters θ = {W} of each layer k b k}, thus obtaining the trained model;

[0192] The test set S data is standardized so that the mean of each variable in the test set is 0 and the standard deviation is 1.

[0193] The standardized test set data is input into the trained model in batches, and the output data of each batch is destandardized to obtain the reconstructed ocean subsurface salinity data of the test set. The destandardization formula is:

[0194] O S =O′ S *μ l +σ l

[0195] Among them, O′ S Indicates the output of the undenormalized test set, O S This represents the destandardized test set output, i.e., the ocean subsurface salinity data reconstructed from the test set.

[0196] RMSE (Root Mean Square Error) is chosen as the evaluation metric for model performance. The expression for RMSE is as follows:

[0197]

[0198] Among them, L′ S For the standardized label data of the test set, O′ S With L′ S The dimensions are all [16, 15, 300, 180], and depth represents the depth of the output subsurface salinity. The root mean square error of RMSE between the output results of each batch on the test set and the true results is calculated. The RMSE of all batches is averaged to obtain the final RMSE to evaluate the model's performance.

[0199] The basic hyperparameters for training a deep learning model in the PyTorch framework are set as follows: The initial number of iterations is epoch = 100, the initial learning rate is lr = 0.001, the initial batch size is batch = 16, the Glorot normal distribution initializer is used to randomly initialize the weights of each layer of the network, and the AdamW iterative optimizer is used, with the two exponential decay rates betas set to (0.9, 0.999) and eps set to 1e. -8 The L2 regularization decay sparsity is set to 0.01. Then, the data from the training set T obtained in step S13 are input into the obtained deep learning model in batches for training and fitting. The parameters θ = {W} of each layer k b k};

[0200] Save the model parameters after training to obtain the trained model.

[0201] like Figure 11Step S940: Acquire real-time sea surface remote sensing data, and use a trained deep learning model to reconstruct subsurface salinity to obtain high-resolution subsurface grid salinity data. This specifically includes the following steps:

[0202] Acquire real-time sea surface remote sensing data and organize the data into a standard input format according to the real-time acquisition data.

[0203] The obtained data is standardized, and each input batch is converted into a four-dimensional matrix format.

[0204] By using data from each batch and sequentially inputting it into a trained deep learning model, ocean subsurface salinity can be reconstructed, thus obtaining high-precision ocean subsurface salinity data.

[0205] One embodiment of this application provides a deep learning-based ocean subsurface salinity reconstruction system, the system comprising:

[0206] The data acquisition unit is used to acquire various input data, which is satellite remote sensing observation data of the ocean surface.

[0207] The data reconstruction unit is used to input various input data into a preset deep learning model to obtain the ocean subsurface salinity reconstructed by the deep learning model.

[0208] The deep learning model reconstructs the salinity of the ocean subsurface layer in the following way:

[0209] The corresponding feature maps are extracted from each type of input data, and the feature maps of multiple input data are fused to obtain the first feature map;

[0210] The first feature map is enhanced with local and global attention to obtain the second feature map.

[0211] The salinity of the ocean subsurface is reconstructed based on the second feature map.

[0212] It should also be noted that this embodiment is based on the same inventive concept as the above embodiments, therefore the content of the above embodiments applies to this embodiment, and will not be repeated here.

[0213] like Figure 12 This application also provides an electronic device, which includes:

[0214] At least one memory;

[0215] At least one processor;

[0216] At least one program;

[0217] The program is stored in memory, and the processor executes at least one program to implement the deep learning-based method for reconstructing ocean subsurface salinity described above in this disclosure.

[0218] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0219] The electronic devices according to embodiments of this application will now be described in detail.

[0220] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0221] The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the program code is stored in the memory 1700 and is called and executed by the processor 1600 to execute the deep learning-based ocean subsurface salinity reconstruction method of the embodiments of this invention.

[0222] The input / output interface 1800 is used to implement information input and output.

[0223] The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0224] Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900);

[0225] The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0226] This invention also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the aforementioned bus interval warning method.

[0227] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0228] The embodiments described in this invention are intended to more clearly illustrate the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0229] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0230] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0231] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0232] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0233] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0234] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0235] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0236] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0237] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0238] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A deep learning-based method for reconstructing marine subsurface salinity, characterized in that, The method includes: The system acquires various input data, which are satellite remote sensing observation data of the ocean surface; the input data includes: sea surface temperature, sea surface salinity, and sea surface height. The various input data are input into a preset deep learning model to obtain the ocean subsurface salinity reconstructed by the deep learning model; The deep learning model reconstructs the ocean subsurface salinity in the following manner: The corresponding feature map is extracted from each type of input data, and the feature maps of multiple types of input data are fused to obtain the first feature map; The first feature map is enhanced with local and global attention to obtain a second feature map; the process of enhancing the first feature map with local and global attention to obtain the second feature map includes the following steps: The first feature map is enhanced with local attention and global attention to obtain the first enhanced feature map. The first enhanced feature map and the second feature map are then stacked by channel dimension to obtain the stacked feature map. A second enhancement of local and global attention is performed on the stacked feature map to obtain a second enhanced feature map. The second enhanced feature map, the first feature map, and the stacked feature map are then stacked along the channel dimension to obtain a second feature map. The first enhancement step is the same as the second enhancement step, and the first enhancement based on the first feature map, performing local and global attention to obtain the first enhanced feature map, includes: SA spatial attention enhancement is performed on the first feature map to obtain the first intermediate feature map; The second intermediate feature map is obtained by multiplying the first intermediate feature map by the first feature map in the channel dimension; The second intermediate feature map is flattened into a third intermediate feature map; Multi-head attention enhancement is performed based on the third intermediate feature map to obtain the fourth intermediate feature map; A fifth intermediate feature map is obtained by performing a residual connection based on the fourth intermediate feature map and the third intermediate feature map; The sixth intermediate feature map is extracted from the fifth intermediate feature map using a multilayer perceptron. The fifth intermediate feature map is flattened to form the seventh intermediate feature map; Residual connections are performed based on the seventh intermediate feature map and the sixth intermediate feature map to obtain the feature map after the first enhancement. The ocean subsurface salinity is reconstructed based on the second feature map.

2. The deep learning-based method for reconstructing marine subsurface salinity according to claim 1, characterized in that, The step of extracting corresponding feature maps from each type of input data and fusing the corresponding feature maps of multiple input data to obtain a first feature map includes: Extract the corresponding multi-scale feature map from each of the input data; Multi-scale fusion is performed on the multi-scale feature maps corresponding to each type of input data to obtain a multi-scale fused feature map corresponding to each type of input data. The first feature map is obtained by fusing the multi-scale fusion feature maps corresponding to the various input data.

3. The deep learning-based method for reconstructing marine subsurface salinity according to claim 2, characterized in that, The step of extracting the corresponding multi-scale feature map from each type of input data includes: in, This represents the m-th type of input data. , , , Let represent the mapping functions for convolution operations with kernel sizes of 1x1, 3x3, 5x5, and 7x7, respectively. The mapping function for a 9x9 convolution operation. The mapping function representing the LReLU activation function. Indicates residual connection, Indicates a stacking operation. The multi-scale feature map represents the input data of the m-th type.

4. The deep learning-based method for reconstructing marine subsurface salinity according to claim 2, characterized in that, The multi-scale fusion of the multi-scale feature maps corresponding to each type of input data includes: in, This represents the multi-scale feature map corresponding to the m-th type of input data. , , and The mapping function represents the convolution operation with kernels of 1x1, 3x3, 5x5, and 7x7. The mapping function for a 9x9 convolution operation. The mapping function representing the LReLU activation function. Indicates residual connection, Indicates a stacking operation. This represents the multi-scale fusion feature map corresponding to the m-th type of input data.

5. The deep learning-based method for reconstructing marine subsurface salinity according to claim 2, characterized in that, The multi-scale fusion feature map, which integrates various input data, includes: in, , , This represents the multi-scale fusion feature map corresponding to the first type of input data, the second type of input data, and the Mth type of input data. Indicates a stacking operation. The mapping function representing a convolution operation with a kernel size of 1x1. , The mapping function between 3x3 and 7x7 average pooling layers. , The mapping function between 5x5 and 9x9 max pooling layers. The mapping function representing the LReLU activation function. This represents the first feature map.

6. The deep learning-based method for reconstructing marine subsurface salinity according to claim 1, characterized in that, The loss function of the deep learning model includes: in, Denotes the loss function, Indicates hyperparameters, This represents the subsurface salinity of the ocean output by the deep learning model. This represents the label data.

7. A deep learning-based system for reconstructing marine subsurface salinity, characterized in that, The system includes: The data acquisition unit is used to acquire various input data, which are satellite remote sensing observation data of the ocean surface; the input data include: sea surface temperature, sea surface salinity, and sea surface height. The data reconstruction unit is used to input the various input data into a preset deep learning model to obtain the ocean subsurface salinity reconstructed by the deep learning model. The deep learning model reconstructs the ocean subsurface salinity in the following manner: The corresponding feature map is extracted from each type of input data, and the feature maps of multiple types of input data are fused to obtain the first feature map; The first feature map is enhanced with local and global attention to obtain a second feature map; the process of enhancing the first feature map with local and global attention to obtain the second feature map includes the following steps: The first feature map is enhanced with local attention and global attention to obtain the first enhanced feature map. The first enhanced feature map and the second feature map are then stacked by channel dimension to obtain the stacked feature map. A second enhancement of local and global attention is performed on the stacked feature map to obtain a second enhanced feature map. The second enhanced feature map, the first feature map, and the stacked feature map are then stacked along the channel dimension to obtain a second feature map. The first enhancement step is the same as the second enhancement step, and the first enhancement based on the first feature map, performing local and global attention to obtain the first enhanced feature map, includes: SA spatial attention enhancement is performed on the first feature map to obtain the first intermediate feature map; The second intermediate feature map is obtained by multiplying the first intermediate feature map by the first feature map in the channel dimension; The second intermediate feature map is flattened into a third intermediate feature map; Multi-head attention enhancement is performed based on the third intermediate feature map to obtain the fourth intermediate feature map; A fifth intermediate feature map is obtained by performing a residual connection based on the fourth intermediate feature map and the third intermediate feature map; The sixth intermediate feature map is extracted from the fifth intermediate feature map using a multilayer perceptron. The fifth intermediate feature map is flattened to form the seventh intermediate feature map; Residual connections are performed based on the seventh intermediate feature map and the sixth intermediate feature map to obtain the feature map after the first enhancement. The ocean subsurface salinity is reconstructed based on the second feature map.

8. An electronic device, characterized in that: It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform a deep learning-based method for reconstructing subsurface salinity of the ocean as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Ocean three-dimensional temperature and salt field space-time forecasting method, device, equipment and medium

    CN117787136A

  • Marine salinity super-resolution reconstruction method based on deep learning SROSRN model

    CN117934274A