Deep learning-based ocean subsurface salinity reconstruction method and system

Through deep learning-based methods, the characteristics of ocean surface satellite remote sensing data are extracted and enhanced, and the problem of insufficient spatial resolution of ocean subsurface salinity data is solved, and high-precision and high-generalized ocean subsurface salinity reconstruction is achieved.

CN119068207APending Publication Date: 2024-12-03NAT UNIV OF DEFENSE TECH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411176790.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the reconstruction of marine subsurface salinity data, the spatial resolution is not high enough, making it difficult to capture small and medium-sized signals in the marine subsurface, and the generalization is not strong, and the salinity accuracy of the reconstruction is not high.

Method used

Using a deep learning-based method, the feature vectors are extracted from satellite remote sensing observation data on the surface of the ocean and enhanced local attention and global attention, the feature maps of multiple input data are fused to reconstruct the subsurface salinity of the ocean.

Benefits of technology

It realizes the rapid generation of high-resolution marine subsurface salinity data, has better generalization capabilities, high reconstruction accuracy, low cost and easy to implement.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068207A_ABST
    Figure CN119068207A_ABST
Patent Text Reader

Abstract

The invention relates to an ocean subsurface salinity reconstruction method and system based on deep learning, and the method comprises the steps: obtaining a variety of input data, inputting the variety of input data into a preset deep learning model, extracting a corresponding feature vector from each kind of input data through the deep learning model, fusing the feature vectors corresponding to various input data to obtain a first feature map; performing local attention and global attention enhancement on the first feature map to obtain a second feature map; and reconstructing the marine subsurface salinity according to the second feature map. The method can solve the problems that the current marine subsurface salinity data is not high enough in time and space resolution and small and medium-sized signals of the marine subsurface are difficult to capture, the high-resolution marine subsurface salinity data can be quickly generated according to the high-resolution sea surface remote sensing observation data, and compared with the existing scheme, the method has a better generalization ability and is suitable for popularization and application. The method has the advantages of high reconstruction precision, low cost and easy implementation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of ocean data processing technology, and in particular to a method and system for reconstructing ocean subsurface salinity based on deep learning. Background Art

[0002] Salinity, as a variable describing the physical state of the ocean, is considered a key factor in the ocean system. It is closely related to global ocean circulation, global climate, and changes in the ocean-climate system. Monitoring the long-term and short-term changes in sea surface salinity is crucial. The three-dimensional salinity field of the ocean subsurface is an important parameter for describing the characteristics of the ocean's physical properties and mastering the ocean's physical movement process. Obtaining accurate three-dimensional salinity information of the ocean subsurface is of great significance for understanding the ocean, developing and utilizing the ocean, and marine scientific research. The ocean salinity field information obtained by field observations is sparse in time and space and has poor temporal continuity. Satellite remote sensing can obtain global ocean surface salinity information, but it cannot obtain three-dimensional salinity information of the ocean subsurface. Therefore, many studies have combined ocean surface satellite remote sensing observation data (sea surface temperature, sea surface salinity, sea surface height, sea surface wind field, etc.) and sparse field profile observation data to establish a mapping relationship from the surface to the underwater, and realized the reconstruction of the spatiotemporal continuous three-dimensional salinity data of the ocean subsurface based on multi-source remote sensing data of the sea surface.

[0003] The main methods for reconstructing three-dimensional salinity data of the ocean subsurface can be divided into: statistical methods, dynamic methods and empirical methods based on sound propagation time.

[0004] Statistical methods establish the statistical relationship between sea surface satellite remote sensing observations and ocean subsurface salinity information, mainly including regression statistics, empirical orthogonal decomposition and variational methods. Among them, empirical orthogonal decomposition decomposes the temperature-salinity profile observation data into EOF, finds the amplitude corresponding to certain modes and establishes a regression relationship with the sea surface temperature or sea surface height. Among them, the variational method uses data assimilation methods to combine observation data with climate data to generate a three-dimensional salinity field; dynamic methods use dynamic models combined with sea surface information to simulate the internal dynamic process of the ocean, and based on dynamic constraints or numerical models, converts satellite remote sensing observation data of the sea surface into dynamic control variables to achieve the mapping of surface data to subsurface salinity; empirical methods based on acoustic propagation time establish empirical models based on acoustic propagation time to invert underwater temperature-salinity structure.

[0005] The above traditional methods are limited to a certain area, have poor generalization, and the reconstructed salinity is not accurate. Summary of the invention

[0006] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0007] A first aspect of an embodiment of the present application provides a method for reconstructing ocean subsurface salinity based on deep learning, the method comprising:

[0008] Acquiring a variety of input data, wherein the input data is satellite remote sensing observation data of the ocean surface;

[0009] By inputting the multiple input data into a preset deep learning model, the ocean subsurface salinity reconstructed by the deep learning model is obtained;

[0010] The deep learning model reconstructs the ocean subsurface salinity in the following way:

[0011] Extracting corresponding feature vectors from each type of input data respectively, and fusing feature vectors corresponding to multiple types of input data to obtain a first feature map;

[0012] Performing local attention and global attention enhancement on the first feature map to obtain a second feature map;

[0013] The ocean subsurface salinity is reconstructed according to the second characteristic map.

[0014] The embodiment of the present application provides a method for reconstructing ocean subsurface salinity based on deep learning. This method can solve the problem that the spatial resolution of the current ocean subsurface salinity data is not high enough and it is difficult to capture small and medium-sized signals in the ocean subsurface. It can quickly generate high-resolution ocean subsurface salinity data based on high-resolution sea surface remote sensing observation data, and provide a new method for reconstructing ocean subsurface salinity data. Compared with existing solutions, this method has the advantages of better generalization ability, high reconstruction accuracy, low cost and easy implementation.

[0015] In some embodiments of the present application, the step of performing local attention and global attention enhancement on the first feature map to obtain a second feature map includes the following steps:

[0016] Performing a first enhancement of local attention and global attention according to the first feature map to obtain a first enhanced feature map, and stacking the first enhanced feature map and the second feature map in a channel dimension to obtain a stacked feature map;

[0017] Performing a second enhancement of local attention and global attention according to the stacked feature map to obtain a second enhanced feature map, and stacking the second enhanced feature map, the first feature map, and the stacked feature map in a channel dimension to obtain a second feature map;

[0018] The first enhancement step is the same as the second enhancement step, and the first enhancement of local attention and global attention is performed according to the first feature map to obtain the feature map after the first enhancement, including:

[0019] Perform SA channel attention enhancement according to the first feature map to obtain a first intermediate feature map;

[0020] Obtain a second intermediate feature map by multiplying the first intermediate feature map by the first feature map in the channel dimension;

[0021] Flattening the second intermediate feature map into a third intermediate feature map;

[0022] Perform multi-head attention enhancement according to the third intermediate feature map to obtain a fourth intermediate feature map;

[0023] Performing a residual connection on the fourth intermediate feature map and the third intermediate feature map to obtain a fifth intermediate feature map;

[0024] Extracting a sixth intermediate feature map from the fifth intermediate feature map according to a multi-layer perceptron;

[0025] De-flattening the fifth intermediate feature map into a seventh intermediate feature map;

[0026] A residual connection is performed based on the seventh intermediate feature map and the sixth intermediate feature map to obtain a feature map after the first enhancement.

[0027] In some embodiments of the present application, extracting corresponding feature vectors from each type of input data respectively, and fusing feature vectors corresponding to multiple types of input data to obtain a first feature map includes:

[0028] Extracting a corresponding multi-scale feature map from each of the input data;

[0029] Performing multi-scale fusion on the multi-scale feature maps corresponding to each of the input data to obtain a multi-scale fused feature map corresponding to each of the input data;

[0030] The multi-scale fusion feature maps corresponding to the multiple input data are fused to obtain the first feature map.

[0031] In some embodiments of the present application, extracting a corresponding multi-scale feature map from each of the input data includes:

[0032]

[0033] Among them, B′ m represents the mth input data, C 1×1 , C 3×3 , C5×5 , C 7×7 They represent the mapping functions of convolution operations with kernel sizes of 1x1, 3x3, 5x5, and 7x7, respectively, f LReLU Represents the mapping function of the LReLU activation function, represents residual connection, concat represents stacking operation, F CNN-Multiscale-1-m Represents the multi-scale feature vector corresponding to the m-th input data.

[0034] In some embodiments of the present application, the multi-scale fusion of the multi-scale feature maps corresponding to each of the input data includes:

[0035]

[0036] Among them, F CNN-Multiscale-1-m represents the multi-scale feature vector corresponding to the m-th input data, C 1×1 , C 3×3 , C 5×5 and C 7×7 The mapping function for convolution operations with kernels of 1x1, 3x3, 5x5, and 7x7, f LReLU Represents the mapping function of the LReLU activation function, represents residual connection, concat represents stacking operation, F CNN-Multiscale-2-m Represents the multi-scale fusion feature vector corresponding to the m-th input data.

[0037] In some embodiments of the present application, the fusing of the multi-scale fusion feature vectors corresponding to the multiple input data includes:

[0038]

[0039] Among them, F CNN-Multiscale-2-1 、F CNN-Multiscale-2-2 、F CNN-Multiscale-2-M represents the multi-scale fusion feature map corresponding to the first input data, the multi-scale fusion feature map corresponding to the second input data, and the multi-scale fusion feature map corresponding to the Mth input data, concat represents a stacking operation, C 1×1 Represents the mapping function of the convolution operation with a convolutional layer size of 1 by 1, represents the mapping function of the average pooling layer of 3x3, 5x5, 7x7 and 9x9, f LReLU Represents the mapping function of the LReLU activation function, F CNN-Multiscale Represents the first feature map.

[0040] In some embodiments of the present application, the loss function of the deep learning model includes:

[0041]

[0042] Wherein, Huber Loss represents the loss function, δ represents a hyperparameter, O ′ represents the ocean subsurface salinity output by the deep learning model, and L ′ represents label data.

[0043] In some embodiments of the present application, the input data includes: sea surface temperature, sea surface salinity and sea surface height.

[0044] The second aspect of the embodiment of the present application provides a deep learning-based ocean subsurface salinity reconstruction system, the system comprising:

[0045] A data acquisition unit, used for acquiring a variety of input data, wherein the input data is satellite remote sensing observation data of the ocean surface;

[0046] A data reconstruction unit, used for inputting the plurality of input data into a preset deep learning model to obtain the ocean subsurface salinity reconstructed by the deep learning model;

[0047] The deep learning model reconstructs the ocean subsurface salinity in the following way:

[0048] Extracting corresponding feature maps from each type of input data respectively, and fusing the feature maps corresponding to the multiple types of input data to obtain a first feature map;

[0049] Performing local attention and global attention enhancement on the first feature map to obtain a second feature map;

[0050] The ocean subsurface salinity is reconstructed according to the second characteristic map.

[0051] A third aspect of an embodiment of the present application provides an electronic device, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the above-mentioned deep learning-based ocean subsurface salinity reconstruction method.

[0052] It can be understood that the beneficial effects of the second to third aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0054] Figure 1 It is a method for reconstructing ocean subsurface salinity based on deep learning provided by an embodiment of the present application;

[0055] Figure 2 It is a flow chart of a method for reconstructing ocean subsurface salinity provided by another embodiment of the present application;

[0056] Figure 3 It is a structural diagram of a branch of extracting a multi-scale feature map of a certain variable in the first part of a convolutional multi-scale feature extraction and fusion module provided by an embodiment of the present application;

[0057] Figure 4 It is a structural diagram of a multi-scale fusion feature map of the third part of the convolutional multi-scale feature extraction and fusion module provided in an embodiment of the present application that fuses all variables from a larger range;

[0058] Figure 5 is a structural diagram of a convolutional multi-scale feature extraction and fusion module provided by an embodiment of the present application;

[0059] Figure 6 is a structural diagram of a local attention module provided by an embodiment of the present application;

[0060] Figure 7 is a structural diagram of a multi-head attention module provided by an embodiment of the present application;

[0061] Figure 8 is a structural diagram of a global attention module provided by an embodiment of the present application;

[0062] Fig. 9 This is a structural diagram of a global and local alternating attention module provided by an embodiment of the present application;

[0063] Fig.10 is a structural diagram of an output module provided by an embodiment of the present application;

[0064] Fig.11 is a structural diagram of a deep learning model provided by an embodiment of the present application;

[0065] Fig.12 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0069] like Figure 1 An embodiment of the present application provides a method for reconstructing ocean subsurface salinity based on deep learning, the method comprising steps S110 to S120:

[0070] Step S110, obtaining a variety of input data, the input data is satellite remote sensing observation data of the ocean surface.

[0071] Step S120, inputting a plurality of input data into a preset deep learning model to obtain the ocean subsurface salinity reconstructed by the deep learning model;

[0072] The deep learning model reconstructs the ocean subsurface salinity in the following way:

[0073] Extracting corresponding feature maps from each type of input data respectively, and fusing the feature maps corresponding to the multiple input data to obtain a first feature map;

[0074] Performing local attention and global attention enhancement on the first feature map to obtain a second feature map;

[0075] Reconstruct ocean subsurface salinity based on the second characteristic map.

[0076] In an embodiment, the input data is satellite remote sensing observation data of the ocean surface (with high precision), including but not limited to: sea surface (sea surface) temperature data, sea surface height data, sea surface salinity data. A variety of input data are used as input variables of the deep learning model.

[0077] The deep learning model can be a model built based on various deep learning algorithms, such as a model built by a CNN network. The deep learning model first extracts the corresponding feature map from each input data, and fuses the corresponding feature maps of multiple input data to obtain the first feature map. Through this step, the features of different input variables can be extracted and fused, so that the first feature map can have rich feature information of different input variables, providing data support for subsequent processing; then the first feature map is enhanced with local attention and global attention to obtain the second feature map. The local attention can be used to extract the sub-regions that play a key role in the reconstruction of subsurface salinity in the global, and then the global attention calculation is used to maintain the focus on the key sub-regions, and more information in the space is introduced for interaction, so as to realize the global attention calculation with local emphasis; finally, the subsurface salinity of the ocean can be reconstructed according to the second feature map.

[0078] This method can solve the problem that the spatial resolution of the current ocean subsurface salinity data is not high enough and it is difficult to capture small and medium-sized signals in the ocean subsurface. It can quickly generate high-resolution ocean subsurface salinity data based on high-resolution sea surface remote sensing observation data, providing a new method for reconstructing ocean subsurface salinity data. Compared with existing solutions, it has better generalization ability, high reconstruction accuracy, low cost and easy implementation.

[0079] In some embodiments of the present application, local attention and global attention enhancement are performed on the first feature map to obtain a second feature map, including: steps S210 to S220. Specifically:

[0080] Step S210, performing a first enhancement of local attention and global attention according to the first feature map to obtain a feature map after the first enhancement, and stacking the feature map after the first enhancement and the second feature map in the channel dimension to obtain a stacked feature map.

[0081] Step S220, performing a second enhancement of local attention and global attention according to the stacked feature map to obtain a second enhanced feature map, and stacking the second enhanced feature map, the first feature map, and the stacked feature map in the channel dimension to obtain a second feature map.

[0082] The first enhancement step is the same as the second enhancement step, and the first enhancement of local attention and global attention is performed according to the first feature map to obtain the feature map after the first enhancement, including:

[0083] Perform SA channel attention enhancement according to the first feature map to obtain a first intermediate feature map;

[0084] According to the multiplication of the first intermediate feature map and the first feature map in the channel dimension, a second intermediate feature map is obtained;

[0085] Flatten the second intermediate feature map into a third intermediate feature map;

[0086] Perform multi-head attention enhancement according to the third intermediate feature map to obtain a fourth intermediate feature map;

[0087] Perform a residual connection on the fourth intermediate feature map and the third intermediate feature map to obtain a fifth intermediate feature map;

[0088] Extracting a sixth intermediate feature map from the fifth intermediate feature map according to a multi-layer perceptron;

[0089] De-flattening the fifth intermediate feature map into a seventh intermediate feature map;

[0090] A residual connection is performed based on the seventh intermediate feature map and the sixth intermediate feature map to obtain a feature map after the first enhancement.

[0091] In this embodiment, the features of the first feature map are enhanced by local attention and global attention, wherein the local attention mechanism is implemented based on convolution, and the global attention is implemented based on Transformer, and the sub-region that plays a key role in the reconstruction of sub-surface salinity in the global layer is extracted through the SA channel attention of the local attention; on this basis, the multi-layer perceptron calculation of the global attention is performed to maintain the attention on the key sub-region, and introduce more information in the space for interaction, so as to realize the global attention calculation with local emphasis, obtain more global information, and at the same time, do not ignore the key sub-region, so as to achieve complementary advantages.

[0092] In this embodiment, the first feature map is enhanced twice and densely connected (i.e., channel stacking) is used to reduce information damage and ultimately improve the reconstruction effect.

[0093] In some embodiments of the present application, a corresponding feature map is extracted from each input data, and the feature maps corresponding to the multiple input data are fused to obtain a first feature map, including: step S310 to step S330. Specifically:

[0094] Step S310: extracting a corresponding multi-scale feature map from each type of input data.

[0095] Step S320, performing multi-scale fusion on the multi-scale feature map corresponding to each input data to obtain a multi-scale fused feature map corresponding to each input data.

[0096] Step S330, fusing the multi-scale fusion feature maps corresponding to the various input data to obtain a first feature map.

[0097] In the embodiment, first, the multi-scale feature map of each input data is extracted independently; secondly, the multi-scale feature maps of each input data are fused respectively to ensure that each grid area of ​​the multi-scale fusion feature of each input data incorporates rich multi-scale spatial information, providing a larger range of surrounding information reference for reconstruction, and obtaining the multi-scale fusion feature map of each input data; finally, from a larger scale range, the multi-scale fusion feature maps of all input data are fused, and by learning the implicit interaction relationship between different input data, the information complementarity between different variables is achieved, thereby improving the accuracy of reconstruction.

[0098] This embodiment extracts and fuses the multi-scale features of the input variables through multiple convolution operation branches of different sizes. The purpose is to reconstruct the sub-surface salinity at each location. It is usually affected by the adjacent areas. By fusing features of different scales, it will make full use of spatial information of different scales to provide more information support for the reconstruction of sub-surface salinity, so as to improve the accuracy of the salinity reconstructed by the final model.

[0099] like Figures 2 to 11 Another embodiment of the present application provides a method for reconstructing ocean subsurface salinity, specifically:

[0100] Step S910, obtaining input data and label data, and forming the obtained input data and label data into a data set for training and testing models. Specifically, the following steps are included:

[0101] Type of input data: daily average 0.25°×0.25° sea surface salinity, sea surface temperature, and sea surface height data are used as input data, and daily average 0.083°×0.083° ocean subsurface salinity data are used as label data; the above data are all downloaded from 2010-2023, and contain 5113 (N=5113) sample data, that is, sample data at 5113 moments, and the sample data of all variables correspond one to one at 5113 moments; the spatial range of the covered sea area is consistent, the longitude coverage range is 105°E to 120°E, and the latitude coverage range is 0° to 25°N, so the latitude span is lat=25°, and the longitude span is lon=15°.

[0102] Input data sources: Daily average 0.25°×0.25° sea surface temperature data comes from the daily optimal interpolation V2.1SST data of the National Oceanic and Atmospheric Administration (NOAA);

[0103] The daily average 0.25°×0.25° sea surface salinity data are derived from the SMOS satellite (hybrid orbit) averaged 10-day and monthly salinity field products from the CATDS Center, the downstream data processing center of SMOS;

[0104] The daily mean 0.25° × 0.25° sea surface height data are from the Copernicus Ocean Service Global Ocean Grid L4 Sea Surface Height and Derived Variables 1993 reprocessed product;

[0105] Label data: The daily average 0.083°×0.083° ocean subsurface salinity data comes from the global ocean physics reanalysis product GLORYS12V1 (GlobalOcean Physics Reanalysis). Therefore, the grid size s°=0.83°; [0.5 m, 5 m, 10 m, 15 m, 20 m, 25 m, 35 m, 55 m, 75 m, 100 m, 120 m, 140 m, 160 m, 180 m, 200 m] are selected, a total of 15 depth layers are reconstructed, that is, d=15.

[0106] First, pair the input data with the label data to obtain a data pair matrix X that matches time and space. The data pair matrix X is organized as follows:

[0107]

[0108] Among them, each input variable in the data pair matrix X is denoted as v m,n , v m,n It is represented as the nth sample data of the mth variable, m∈[1,M], M=3, the first to the third correspond to the variables sea surface salinity, sea surface temperature, and sea surface height, respectively, with a total of N=5113 sample data. One sample data is a column in the matrix X. Since the time resolution is the same, one moment corresponds to one sample data. Each column contains 3 variables, of which the input variables are the first 3, and the last line is the label data l n , that is, ocean reanalysis salinity data containing multiple layer depths. The label of each sample data corresponds to the input variable one by one, and remains consistent in time and space. Therefore, there are a total of 5113 sample data label data.

[0109] Then, the paired data pair matrix X is divided into a training set T and a test set S according to 8:2, that is, the first approximately 80% of the rows of X are used as the training set T, and the last approximately 20% of the rows are used as the test set S, that is, the training set has 4090 sample data and the test set has 1023 sample data.

[0110] Randomly select 16 (b=16) sample data from the training set T or the test set S as a batch B, and standardize the sea surface salinity, sea surface temperature, and sea surface height data to a mean of 0 and a standard deviation of 1 to obtain a batch of data B′ after standardization. The normalization formula is as follows:

[0111]

[0112] Among them, Bm Represents an input variable in batch B. is the mean of all grid values ​​of all samples of a variable in the training set, is the standard deviation of all grid values ​​of a variable in all samples in the training set. The variable after batch standardization is recorded as B′ m ; The corresponding label data must also be standardized. The standardization formula for label data is as follows:

[0113]

[0114] Among them, B l represents the label data in batch B, μ l is the mean of all grid values ​​of all samples in all depth layers of the labeled data in the training set, σ l is the standard deviation of all grid values ​​of all samples in all depth layers of the labeled data in the training set. The standardized labeled data of this batch is recorded as B′ l .

[0115] Finally, a batch of data B ′ after standardization is stored in the format of a 4-dimensional matrix. The four dimensions are the number of samples in the batch, the number of variables, the latitudinal span, and the longitudinal span, which are abbreviated as [16, 3, 25°, 15°]. Lat represents the latitudinal span of the ocean subsurface salinity reconstruction area in the horizontal direction, and lon represents the longitudinal span of the reconstruction area in the horizontal direction. The nearest neighbor interpolation algorithm is used to make the horizontal spatial resolution of all input variables consistent with the label data, which is 1 / 12°×1 / 12°. The longitudinal and latitudinal spans are converted into the number of grids. The size of a single batch of data P ′ is [16, 3, 300, 180].

[0116] Step S920: Build a deep learning model. The expression of the deep learning model is as follows:

[0117]

[0118] in, The mapping relationship between the input and output of the deep learning model is shown in Figure 2. The input is a normalized multivariate input batch B ′, and the output is a batch O of ocean subsurface salinity data. θ = {W k , b k} represents the weight and bias of the kth layer in the model, which specifically includes the following steps:

[0119] Build a deep learning environment, install the Python-based deep learning framework Pytorch on the server, and build a deep learning model based on the Pytorch framework.

[0120] Step S920 provides a deep learning model that mainly includes: a convolutional multi-scale feature extraction and fusion module, a global and local alternating attention module, and a convolutional multi-scale feature extraction and fusion module based on the built deep learning environment. The multi-scale features of multiple input variables in the input batch are extracted and feature fused to obtain a multi-scale fusion feature F. CNN-Multiscale , the expression is as follows:

[0121] F CNN-Multiscale =f CNN-Multiscale (B′ n )

[0122] Among them, f CNN-Multiscale Represents the mapping function of the convolutional multi-scale feature extraction and fusion module; this module extracts and fuses the multi-scale features of the input variables through multiple convolution operation branches of different sizes. The purpose is to reconstruct the sub-surface salinity at each location. It is usually affected by the adjacent areas. Through the fusion of different scale features, it will make full use of different scale spatial information and provide more information support for the reconstruction of sub-surface salinity.

[0123] The following focuses on the convolutional multi-scale feature extraction and fusion module:

[0124] The convolutional multi-scale feature extraction and fusion module is divided into three parts. First, the multi-scale feature map of each variable is extracted independently. Second, the multi-scale feature maps of each variable are fused separately to ensure that each grid area of ​​the multi-scale fusion feature of each variable incorporates rich multi-scale spatial information, providing a larger range of surrounding information reference for reconstruction, and obtaining the multi-scale fusion feature map of each variable. Finally, from a larger scale range, the multi-scale fusion feature maps of all variables are fused. By learning the implicit interaction relationship between different variables, the information complementarity between different variables is achieved, thereby improving the accuracy of reconstruction.

[0125] like Figure 3 First, three independent and structurally consistent branches are used to independently extract the multi-scale features of each variable. The parameters of each branch are shared, that is, the update of the parameters of a branch during the model training process will also be reflected in the other two branches, which reduces the overall number of model parameters and simplifies the model complexity; C in1 Represents the number of channels of the input feature map. Since each variable is processed separately, the number of channels of the input feature map is C in1 =1, C out1 The number of channels representing the output feature map is set to 16. The specific structure of the branch is: first, there are 4 parallel convolution layers. The size of the convolution kernel of each layer is different, and residual connections are used to avoid gradient explosion or disappearance and reduce the difficulty of model training. Specifically, the first layer is a sequential connection 1×1 convolution kernel (with a stride of 1×1 and no edge padding), 1 LReLU activation layer, 3×3 convolution kernels (with a step size of 1×1, 1 layer of edge padding, and the padding value is set to be consistent with the adjacent value), 1 LReLU activation layer, and the residual connection after two LReLU activation layers; the second layer is a sequential connection 3×3 convolution kernels (with a stride of 1×1, 1 layer of edge padding, and the padding value is set to be consistent with the adjacent value), 1 LReLU activation layer, 5×5 convolution kernels (with a step size of 1×1, 2 layers of edge padding, and the padding value is set to be consistent with the adjacent value), 1 LReLU activation layer, and the residual connection after two LReLU activation layers; the third layer is a sequential connection 5×5 convolution kernels (with a step size of 1×1, 2 layers of edge padding, and the padding value is set to be consistent with the adjacent value), 1 LReLU activation layer, 7×7 convolution kernels (with a step size of 1×1, 3 layers of edge padding, and the padding value is set to be consistent with the adjacent value) and 1 LReLU activation layer. The results after two LReLU activation layers are residually connected; the fourth layer is a sequential connection 7×7 convolution kernels (with a stride of 1×1, 3 layers of edge padding, and the padding value is set to be consistent with the adjacent value), 1 LReLU activation layer, A 9×9 convolution kernel (with a step size of 1×1, 4 layers of edge padding, and the padding value is set to be consistent with the adjacent value) and an LReLU activation layer. The results after two LReLU activation layers are residually connected; finally, the results of the four parallel convolution layers are merged in the channel dimension to obtain a multi-scale feature map F of a certain variable. CNN-Multiscale-1-m , the number of channels is C out1 =16.

[0126] The above LReLU activation layer is an LReLU activation function. If the input value is greater than 0, the output value remains unchanged. If it is less than or equal to 0, the output value is the input value multiplied by a smaller slope value ε. The expression of the LReLU function is as follows:

[0127]

[0128] f LReLU is the mapping function of the LReLU activation function, input represents the input of the function, and active input represents the activation result; the expression of the branch that processes each variable separately is as follows:

[0129]

[0130] Among them, B′ m is a variable in the input batch, with a size of [16, 1, 300, 180], F CNN-Multiscale-1-mRepresents a multi-scale feature map of a variable, with a size of [16,16,300,180], f CNN-Multiscale-1 represents the mapping function of the branch that extracts the multi-scale features of each variable in the first part of the convolutional multi-scale feature extraction and fusion module, C 1×1 , C 3×3 , C 5×5 , C 7×7 and C 9×9 They represent the mapping functions of convolution operations with kernel sizes of 1×1, 3×3, 5×5, 7×7, and 9×9, respectively. It represents residual connection, and the grids at corresponding positions of two feature maps are added, and the resulting size remains unchanged. concat means that the feature maps are stacked in the channel dimension, the number of channel layers increases, and the sizes of the remaining dimensions remain unchanged.

[0131] The multi-scale feature maps of each variable are fused separately. After the branches for independently extracting the multi-scale feature maps of each variable, the corresponding independent multi-scale feature fusion branches are followed. The branch structures are consistent with the branch structures for extracting the multi-scale features of each variable, and the parameters of each branch are still shared. The multi-scale feature maps of a certain variable from the previous order are input into convolutional layers of different sizes respectively, and the previous multi-scale feature maps are fused from different scales to further extract multi-scale features. A multi-scale fused feature map of a certain variable is obtained, where the number of channels of the input feature map is C. Since each variable is processed separately, at this time, the number of channels of the input multi-scale feature map in a certain variable branch is C. in2 =C out1 =16, the number of channels of a multi-scale fusion feature map output is C out2 =64, the module expression of the multi-scale feature map that fuses each variable is:

[0132]

[0133] Among them, F CNN-Multiscale-2-m Represents a multi-scale fusion feature map of a variable, with a size of [16, 64, 300, 180], f CNN-Multiscale-2 Represents the mapping function of the branch that fuses the multi-scale feature maps of each variable in the second part of the convolutional multi-scale feature extraction and fusion module.

[0134] like Figure 4 and Figure 5 , from a larger scale range, the multi-scale fusion feature maps of all variables are fused. First, the multi-scale fusion feature maps of each variable obtained by the three branches are stacked in the channel dimension, and the number of channels is 3C out2 = 192, and then input into 4 independent pooling convolution layers respectively. The role of the pooling layer is to expand the receptive field of the convolution layer and obtain information from a larger spatial range. in3=3C out2 =192, indicating the number of channels of the input feature map, C out3 The number of channels of the output feature map is set to 256. The structure of each branch is as follows: the first layer is a 3×3 average pooling layer (the step size is 1×1, the edge is filled with 1 layer, and the filling value is set to be consistent with the adjacent value). The number of channels in the pooling layer is consistent with the number of channels in the input feature map, and there is an LReLU activation layer. 1×1 convolution kernel, 1 LReLU activation layer, the output feature map contains channels; the second layer is a 5×5 maximum pooling layer (with a step size of 1×1, 2 layers of edge padding, and the padding value is set to be consistent with the adjacent value), and an LReLU activation layer. 1×1 convolution kernel, 1 LReLU activation layer, the output feature map contains channels; the third layer is a 7×7 average pooling layer (with a step size of 1×1, 3 layers of edge padding, and the padding value is set to be consistent with the adjacent value), and an LReLU activation layer. 1×1 convolution kernel, 1 LReLU activation layer, the output feature map contains channels; the fourth layer is a 9×9 maximum pooling layer (with a step size of 1×1, 4 layers of edge padding, and the padding value is set to be consistent with the adjacent value), and an LReLU activation layer. 1×1 convolution kernel, 1 LReLU activation layer, the output feature map contains The purpose of the edge padding of the above pooling layer is to ensure that the size of the output feature map remains unchanged; finally, the four layers of results are stacked up in the channel dimension to obtain an output feature map containing C out3 = 256 channels, the module expression of the multi-scale fusion feature map that fuses all variables is as follows:

[0135]

[0136] Among them, F CNN-Mult i scale-2 It represents the feature map obtained by stacking the multi-scale fusion feature maps of three variables, with a size of [16, 192, 300, 180], f CNN-Multiscale-3 The third part of the convolutional multi-scale feature extraction and fusion module represents the mapping function of the multi-scale fusion feature map of all variables from a larger scale range, F CNN-Multiscale Represents the output feature map of the convolutional multi-scale feature extraction and fusion module, with a size of [16, 256, 300, 180], represents the mapping function of the 3×3 average pooling layer, represents the mapping function of the 5×5 maximum pooling layer, represents the mapping function of the 7×7 average pooling layer, Represents the mapping function of the 9×9 average pooling layer;

[0137] like Figures 6 to 9 , the following introduces the global and local alternating attention module:

[0138] The expression of the global and local alternating attention module is:

[0139] F Alternate-Attention =f Alternate-Attention (F CNN-Multiscale )

[0140] Among them, F Alternate-Attention represents the output feature map of the global and local alternating attention module, f Alternate-Attention Represents the mapping function of the global and local alternating attention module; the local attention mechanism of this module is implemented based on convolution, and the global attention is implemented based on Transformer. The sub-regions that play a key role in the reconstruction of sub-surface salinity in the global layer are extracted through local attention. On this basis, global attention calculation is performed to maintain the attention on the key sub-regions, and more information in the space is introduced for interaction to realize global attention calculation with local emphasis, so as to obtain more global information while not neglecting the key sub-regions and realize complementary advantages.

[0141] The global and local alternating attention module contains two basic modules, the local attention module first, and the global attention module later. The local attention module is a series of SA spatial attention modules. After the input multi-channel feature map SA spatial attention module, the result of the weighted spatial attention weight of the multi-channel feature map is obtained. The function expression of the local attention module is:

[0142]

[0143] Among them, F Local-Attention represents the output feature map of the local attention module, f Local-Attention represents the mapping function of the local attention module, Indicates the multiplication of the corresponding dimensions, and assigns the calculated attention weight to the input feature map, f SA Represents the mapping function of the SA spatial attention module. The local attention module does not change the input feature map size;

[0144] The SA spatial attention module first passes through a global maximum pooling layer in the channel dimension and a global average pooling layer in the channel dimension, and then stacks these two results in the channel dimension, with the number of channels being 2. Then, it passes through a 7×7 convolutional layer to reduce the dimension to 1 channel, and then passes through a sigmoid activation layer to obtain the spatial attention feature map. At this time, F SAThe output feature map size of the spatial attention module is [16, 1, H, W]; the SA spatial attention module can be expressed as:

[0145]

[0146] Among them, F SA represents the output feature map of the SA spatial attention module, M SA represents the mapping function of the SA spatial attention module, F represents the multi-channel feature map input to the SA spatial attention module, and C 7×7 Represents a 7×7 convolution kernel (with a step size of 1×1, 3 layers of edge padding, and the padding value is set to be consistent with the adjacent value); GloAvgPool represents the channel global average pooling layer, and the expression is:

[0147] F mean-C =torch.mean(F, 1, keepdim=True);

[0148] Among them, F mean-C represents the feature map after channel global average pooling, "1" means averaging in the first dimension, that is, the channel dimension, F mean-C The size is [16, 1, H, W].

[0149] Similarly, GloMaxPool represents the global maximum pooling layer of the channel, and the expression is:

[0150] F max-C =torch.max(F, 1, keepdim=True);

[0151] F max-C represents the feature map after channel global maximum pooling, "1" means taking the maximum value in the first dimension, that is, the channel dimension, F max-C The size is [16, 1, H, W]; SA spatial attention module structure diagram.

[0152] The spatial attention feature map F SA Multiply the corresponding grid of the feature map of the input SA spatial attention module of this module to obtain the output features of the final local attention module structure diagram; F SA There is only one channel, and the value of each grid represents the importance of the grid in the plane space, and the multiplication of the corresponding grids is equivalent to giving a spatial weight to the same grid of all channels in the feature map batch; the local attention module function expression is:

[0153]

[0154] Among them, F Local-Attention represents the output feature map of the local attention module, f Local-Attentionrepresents the mapping function of the local attention module, Indicates the multiplication of the corresponding dimensions, and assigns the calculated attention weight to the input feature map, f SA represents the mapping function of the SA spatial attention module; from the above steps, we can see that the number of input and output channels of the local attention module is the same, so F Local-Attention Size and input feature map F CNN-Multiscale The sizes are consistent, all [16, 256, 300, 180].

[0155] The global attention module is a multi-head attention module of the Transformer encoder. First, the input feature map is flattened to meet the input requirements of the Transformer encoder module. A sliding window of size 3×3, i.e. w=3, is used. According to the step size 1, the input feature map F is set to a size of 16×C×H×W, where H×W constitutes a two-dimensional plane, i.e., a two-dimensional plane composed of length and width, with a total of B×C two-dimensional planes. It slides step by step from top to bottom and from left to right. Each sliding of the window flattens the covered two-dimensional area into a one-dimensional vector, and finally flattens it into a three-dimensional matrix F. flatten , the size is 16×N×L, where N=H×W, L=9C, that is, 16 samples, each sample is a two-dimensional matrix with N rows and 9 columns, and then Ff l a tt The multi-head attention module MHA of the encoder of the Transformer is input to capture the global attention of the feature map, and then it is input into a 1-layer multi-layer perceptron MLP. Finally, after the unflattening operation, the feature map output by the MHA is restored to the size of the original four-dimensional matrix. At the same time, in order to reduce information loss and reduce the difficulty of training, a residual connection is used after the MHA and MLP. The function expression of the global attention module is:

[0156] F flatten =f flatten (F Local-Attention )

[0157]

[0158] Among them, F Global-Attention represents the output feature map of the global attention module, f Global-Attention represents the mapping function of the global attention module, f MHA represents the mapping function of the multi-head attention module MHA, f MLP Represents the mapping function of the multi-layer perception base MLP, f flatten With f un-flatten The mapping function representing the flattening and unflattening operations, Represents a residual connection, which is the addition of grids at the same position in two feature maps of the same size.

[0159] First, the input flattened feature map F flatten After passing through three identical fully connected Linear layers, the third dimension is expanded by head_num times to obtain three 3D matrices Q, K, and V. Head_num is set to 8, so the sizes of Q, K, and V are all 16×(H×W)×72C. The reshape function of Python is used to convert Q, K, and V into 4D matrices with sizes of 16×8×(H×W)×9C, that is, the original third dimension is disassembled, and Q, K, and V are subjected to SDPA scaling point-by-point multiplication. The specific steps are as follows: first, Q and K are matrix multiplied, and then scaled to times, and perform Softmax activation along the last dimension, perform matrix multiplication on the activation result and V, and then reshape it to a size of 16×(H×W)×72C, and finally restore it to a size of 16×(H×W)×9C through the fully connected layer Linear to obtain the feature map F MHA , the implementation process of the MHA multi-head attention module is expressed as follows:

[0160] Q=f Linear (F flatten )=w Q F+b Q

[0161] K=f Linear (F flatten )=w K F+b K

[0162] V=f Linear (F flatten )=w V F+b V

[0163]

[0164] Among them, f Linear is the fully connected layer activation function, w Q , W K , W V represents the neuron weight matrix of the three fully connected layers, b Q , b K , b V Three fully connected layer bias values, reshape 1 Indicates converting Q, K, and V into a 4-dimensional matrix with a size of 16×8×(H×W)×9C, reshape 2 It means that it is restored to a 3D matrix with a size of 16×(H×W)×72C. The specific expression of the Softmax activation function is:

[0165]

[0166] Among them, input represents the input vector of the activation function, and the Softmax activation is performed along the last dimension, that is, the Softmax activation is performed in sequence along each vector of this dimension; from the above steps, it can be seen that the input and output sizes of the global attention module are consistent.

[0167] The global and local alternating attention module is obtained by connecting the local attention module first and the global attention module later. The expression is:

[0168] F Alternate-Attention =f Alternate-Attention (F CNN-Multiscale )

[0169] =f Global-Attention (f Local-Attention (F CNN-Multiscale ))

[0170] Among them, F Alternate-Attention represents the output feature map of the global and local alternating attention module, f Alternate-Attention Represents the mapping function of the global and local alternating attention module; From the above steps, it can be seen that the input and output sizes of the global and local alternating attention modules are consistent.

[0171] like Fig.10 , construct the output module. The module makes the number of layers of the output feature map consistent with the number of layers of the label data, and obtains the reconstructed subsurface salinity data. The output module expression is as follows:

[0172] F Output =f Output (F Alternate-Attention )=C 3×3 (Relu((f 5×5 (F Alternate-Attention ))))

[0173] Among them, F Output represents the output feature map of the output module, f Output represents the mapping function of the output module, C 3×3 With C 5×5 Relu represents the convolution function with 3×3 and 5×5 convolution kernel sizes, and Relu represents the ReLU activation function.

[0174] After the multi-channel feature map is input into 45 5×5 convolution kernels, the dimension is reduced to 45 channels.

[0175] The feature map of 45 channels is passed through 1 layer of ReLU activation layer and 15 3×3 convolution kernels in sequence, and the feature map is reduced to an output of 15 channels:

[0176] FOutput =f Output (F Alternate-Attention )=C 3×3 (f LReLU ((C 5×5 (F Alternate-Attention ))))

[0177] Among them, F Output represents the output feature map of the output module, f Output represents the mapping function of the output module, and F represents the multi-channel feature map of the input and output module.

[0178] Combining a convolutional multi-scale feature extraction and fusion module, two global and local alternating attention modules, and an output module, we get a deep learning model, which is expressed as follows:

[0179]

[0180] in, Represents the mapping relationship between the input and output of the deep learning model. The input is the normalized batch multivariate data B ′, and the output is the ocean subsurface salinity data batch O, θ = {W k , b k} represents the weights and biases of the kth layer of the model.

[0181] The two global and local alternating attention modules are densely connected, that is, the concat channel dimension is stacked to stack the feature maps that pass through each global and local alternating attention module to reduce information loss. The expression is as follows:

[0182]

[0183] That is, the feature map f obtained by the first global and local alternating attention module Alternate-Attention (F CNN-Multiscale ), and the feature map F of the input global and local alternating attention module Multi-CNN After stacking, the second global and local alternating attention module is input, and the final result is the stacking of all the above feature maps, where F is the input feature map with a size of [16, 1024, 300, 180].

[0184] According to the effect of one convolutional multi-scale feature extraction and fusion module, two global and local alternating attention modules, and one output module, a deep learning model is obtained, which is expressed as follows:

[0185]

[0186] in, Represents the mapping relationship between the input and output of the deep learning model. The input is the normalized batch multivariate data B ′, and the output is the ocean subsurface salinity data batch O, θ = {W k , b k} represents the weight and bias of the kth layer of the model;

[0187] Step S930, using the obtained data set to train the obtained deep learning model to obtain a trained deep learning model, specifically includes the following steps:

[0188] The Huber loss is selected as the loss function of the obtained deep learning model. The expression of the Huber loss function is as follows:

[0189]

[0190] Huber Loss is a combination of MSE loss and MAE loss, combining the advantages of both. On the one hand, MSE loss is better than MAE loss in terms of the complexity of solving the gradient. Since the calculated gradient value also changes dynamically, MSE loss can converge quickly and accurately. On the other hand, in terms of abnormal outliers, MAE loss is not very sensitive to outliers and is suitable for processing data with more outliers, while MSE loss is very sensitive to large deviations and is suitable for scenarios with fewer outliers. Therefore, Huber loss combines the advantages of MSE loss and MAE loss. It manifests as MSE loss when the error is small and MAE loss when the error is large. It performs better in terms of sensitivity to outliers and balanced processing of errors, and Huber loss is differentiable everywhere, which improves the accuracy of gradient calculation when the loss is small. is the MSE loss part, |L ′-O ′| is the MAE loss part, δ is a hyperparameter, set to 0.2, and its size determines the emphasis of Huber loss on MSE loss and MAE loss. When |L ′-O ′|≤0.2, it becomes MSE, and the gradient gradually decreases, which can ensure that the model obtains the global optimal value more accurately; when |L ′-O ′|>0.2, it becomes similar to MAE, and the gradient is always approximately δ, which can ensure that the parameters are updated at a faster speed when training the model; in addition, O ′ is the output of the ocean subsurface salinity reconstruction data, and L ′ is the normalized ocean reanalysis salinity data of multiple layer depths as a label, and the two are of the same size;

[0191] Based on the training set T, the deep learning model is trained in batches and fitted. The parameters of each layer in θ={W k , b k}, get the trained model;

[0192] The test set S data is standardized so that the mean of each variable in the test set is 0 and the standard deviation is 1;

[0193] The standardized test set data is input into the trained model in batches, and each batch of output data is de-standardized to obtain the ocean subsurface salinity data of the reconstructed test set. The de-standardization formula is:

[0194] O S =O′ S *μ l +σ l

[0195] Among them, O′ S represents the undenormalized test set output, O S represents the denormalized test set output, i.e., the ocean subsurface salinity data reconstructed by the test set;

[0196] The RMSE root mean square error is selected as the evaluation indicator of the model quality. The expression of RMSE is as follows:

[0197]

[0198] Among them, L′ S is the label data after the test set is standardized, O′ S and L′ S The sizes are all [16, 15, 300, 180], and depth represents the depth layer of the output subsurface salinity. The RMSE root mean square error between the output results of each batch of the test set and the true results is calculated, and the RMSE of all batches is averaged to obtain the final RMSE to evaluate the effect of the model.

[0199] The basic hyperparameters for training deep learning models in the Pytorch framework are set as follows: the number of initial iterations is epoch = 100, the initial learning rate is lr = 0.001, the initial batch size is batch = 16, the Glorot normal distribution initializer is used to randomly initialize the weights of each layer of the network, the iterative optimizer is AdamW, the two exponential decay betas are set to (0.9, 0.999), and eps is set to 1e -8 , L2 regularization attenuation sparsity is set to 0.01, and then the data in the training set T obtained in step S13 are input into the obtained deep learning model in batches for training, fitting The parameters of each layer in θ={W k , b k};

[0200] Save the model parameters after training to get the trained model.

[0201] like Fig.11Step S940, obtaining real-time sea surface remote sensing data, using the trained deep learning model to reconstruct subsurface salinity, and obtaining high-resolution subsurface grid salinity data, specifically includes the following steps:

[0202] Acquire real-time sea surface remote sensing data and organize it into a standard input format according to the real-time acquired data;

[0203] Standardize the obtained data and convert each input batch into a four-dimensional matrix format;

[0204] By using each batch of data, the trained deep learning model is input in sequence to reconstruct the ocean subsurface salinity, thus obtaining high-precision ocean subsurface salinity data.

[0205] One embodiment of the present application provides an ocean subsurface salinity reconstruction system based on deep learning, the system comprising:

[0206] A data acquisition unit, used for acquiring a variety of input data, the input data being satellite remote sensing observation data of the ocean surface;

[0207] A data reconstruction unit, used for inputting a variety of input data into a preset deep learning model to obtain the ocean subsurface salinity reconstructed by the deep learning model;

[0208] The deep learning model reconstructs the ocean subsurface salinity in the following way:

[0209] Extracting corresponding feature maps from each type of input data respectively, and fusing the feature maps corresponding to the multiple input data to obtain a first feature map;

[0210] Performing local attention and global attention enhancement on the first feature map to obtain a second feature map;

[0211] Reconstruct ocean subsurface salinity based on the second characteristic map.

[0212] It should also be noted that this embodiment and the above-mentioned embodiment are based on the same inventive concept, so the contents of the above-mentioned embodiment are applicable to this embodiment and will not be repeated here.

[0213] like Fig.12 The embodiment of the present application further provides an electronic device, the electronic device comprising:

[0214] at least one memory;

[0215] at least one processor;

[0216] at least one program;

[0217] The program is stored in the memory, and the processor executes at least one program to implement the above-mentioned deep learning-based ocean subsurface salinity reconstruction method implemented in the present disclosure.

[0218] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0219] The electronic device according to the embodiment of the present application is described in detail below.

[0220] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0221] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the program code is stored in the memory 1700, and the processor 1600 calls and executes the ocean subsurface salinity reconstruction method based on deep learning in the embodiment of the present invention.

[0222] Input / output interface 1800, used to implement information input and output;

[0223] The communication interface 1900 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0224] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0225] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0226] An embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned bus interval warning method.

[0227] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0228] The embodiments described in the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0229] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present invention and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0230] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0231] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0232] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0233] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0234] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0235] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0236] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0237] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs.

[0238] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A deep learning-based ocean subsurface salinity reconstruction method. It is characterized in that The method comprises: Acquiring a variety of input data, wherein the input data is satellite remote sensing observation data of the ocean surface; Inputting the multiple input data into a preset deep learning model to obtain the ocean subsurface salinity reconstructed by the deep learning model; The deep learning model reconstructs the ocean subsurface salinity in the following way: Extracting corresponding feature maps from each type of input data respectively, and fusing the feature maps corresponding to the multiple types of input data to obtain a first feature map; Performing local attention and global attention enhancement on the first feature map to obtain a second feature map; The ocean subsurface salinity is reconstructed according to the second characteristic map.

2. The ocean subsurface salinity reconstruction method based on deep learning according to claim 1, It is characterized in that The step of performing local attention and global attention enhancement on the first feature map to obtain a second feature map comprises the following steps: Performing a first enhancement of local attention and global attention according to the first feature map to obtain a first enhanced feature map, and stacking the first enhanced feature map and the second feature map in a channel dimension to obtain a stacked feature map; Performing a second enhancement of local attention and global attention according to the stacked feature map to obtain a second enhanced feature map, and stacking the second enhanced feature map, the first feature map, and the stacked feature map in a channel dimension to obtain a second feature map; The first enhancement step is the same as the second enhancement step, and the first enhancement of local attention and global attention is performed according to the first feature map to obtain the feature map after the first enhancement, including: Perform SA channel attention enhancement according to the first feature map to obtain a first intermediate feature map; Obtain a second intermediate feature map by multiplying the first intermediate feature map by the first feature map in the channel dimension; Flattening the second intermediate feature map into a third intermediate feature map; Perform multi-head attention enhancement according to the third intermediate feature map to obtain a fourth intermediate feature map; Performing a residual connection on the fourth intermediate feature map and the third intermediate feature map to obtain a fifth intermediate feature map; Extracting a sixth intermediate feature map from the fifth intermediate feature map according to a multi-layer perceptron; De-flattening the fifth intermediate feature map into a seventh intermediate feature map; A residual connection is performed based on the seventh intermediate feature map and the sixth intermediate feature map to obtain a feature map after the first enhancement.

3. The ocean subsurface salinity reconstruction method based on deep learning according to claim 2, It is characterized in that The extracting corresponding feature maps from each of the input data respectively and fusing the feature maps corresponding to the multiple input data to obtain a first feature map includes: Extracting a corresponding multi-scale feature map from each of the input data; Performing multi-scale fusion on the multi-scale feature maps corresponding to each of the input data to obtain a multi-scale fused feature map corresponding to each of the input data; The multi-scale fusion feature maps corresponding to the multiple input data are fused to obtain the first feature map.

4. The ocean subsurface salinity reconstruction method based on deep learning according to claim 3, It is characterized in that The step of extracting a corresponding multi-scale feature map from each of the input data comprises: Among them, B′ m represents the mth input data, C 1×1 , C 3×3 , C 5×5 , C 7×7 They represent the mapping functions of convolution operations with kernel sizes of 1x1, 3x3, 5x5, and 7x7, respectively, f LReLU represents the mapping function of the LReLU activation function, ⊕ represents the residual connection, concat represents the stacking operation, and F CNN-Multiscale-1-m Represents the multi-scale feature map corresponding to the m-th input data.

5. The ocean subsurface salinity reconstruction method based on deep learning according to claim 3, It is characterized in that The multi-scale fusion of the multi-scale feature maps corresponding to each of the input data includes: Among them, F CNN-Multiscale-1-m represents the multi-scale feature map corresponding to the m-th input data, C 1×1 , C 3×3 , C 5×5 and C 7×7 The mapping function for convolution operations with kernels of 1x1, 3x3, 5x5, and 7x7, f LReLU Represents the mapping function of the LReLU activation function, represents residual connection, concat represents stacking operation, F CNN-Multiscale-2-m Represents the multi-scale fusion feature map corresponding to the m-th input data.

6. The ocean subsurface salinity reconstruction method based on deep learning according to claim 3, It is characterized in that The multi-scale fusion feature map corresponding to the fusion of the plurality of input data includes: Among them, F CNN-Multiscale-2-1 、F CNN-Multiscale-2-2 、F CNN-Multiscale-2-M represents the multi-scale fusion feature map corresponding to the first input data, the multi-scale fusion feature map corresponding to the second input data, and the multi-scale fusion feature map corresponding to the Mth input data, concat represents a stacking operation, C 1×1 Represents the mapping function of the convolution operation with a convolutional layer size of 1 by 1, represents the mapping function of the average pooling layer of 3x3, 5x5, 7x7 and 9x9, f LReLU Represents the mapping function of the LReLU activation function, F CNN-Multiscale Represents the first feature map.

7. The ocean subsurface salinity reconstruction method based on deep learning according to claim 1, It is characterized in that The loss function of the deep learning model includes: Among them, Huber Loss represents the loss function, δ represents the hyperparameter, O′ represents the ocean subsurface salinity output by the deep learning model, and L′ represents the label data.

8. The ocean subsurface salinity reconstruction method based on deep learning according to claim 1, It is characterized in that The input data include: sea surface temperature, sea surface salinity and sea surface height.

9. A deep learning-based ocean subsurface salinity reconstruction system. It is characterized in that The system comprises: A data acquisition unit, used for acquiring a variety of input data, wherein the input data is satellite remote sensing observation data of the ocean surface; A data reconstruction unit, used for inputting the plurality of input data into a preset deep learning model to obtain the ocean subsurface salinity reconstructed by the deep learning model; The deep learning model reconstructs the ocean subsurface salinity in the following way: Extracting corresponding feature maps from each type of input data respectively, and fusing the feature maps corresponding to the multiple types of input data to obtain a first feature map; Performing local attention and global attention enhancement on the first feature map to obtain a second feature map; The ocean subsurface salinity is reconstructed according to the second characteristic map.

10. An electronic device, Features: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute a deep learning-based ocean subsurface salinity reconstruction method as described in any one of claims 1 to 8.

Citation Information

Patent Citations

  • Three-dimensional ocean temperature and salt field forecasting method, system and equipment based on deep learning

    CN116306318A

  • Ocean three-dimensional temperature and salt field space-time forecasting method, device, equipment and medium

    CN117787136A

  • Marine salinity super-resolution reconstruction method based on deep learning SROSRN model

    CN117934274A