Underwater sound velocity distribution super-resolution construction method based on multi-modal data fusion

Abstract

The application belongs to the technical field of marine observation, and discloses a method for constructing underwater sound velocity distribution super-resolution based on multi-modal data fusion.The application constructs a sound velocity distribution prediction model, the model performs preliminary temperature reconstruction on original space region temperature data through an Unet neural network, and extracts sea surface temperature features through an Mamba neural network; then the preliminary reconstructed temperature data is corrected through the sea surface temperature features to obtain high spatial resolution temperature data; finally, high spatial resolution sound velocity distribution data is obtained through a sound velocity distribution calculation module.The method fuses underwater temperature and sea surface temperature data, realizes fast and accurate prediction of the sound velocity distribution of the entire space region under the condition of original low spatial resolution data, and solves the problem of insufficient spatial resolution in marine sound velocity distribution estimation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of ocean observation, and specifically relates to an underwater sound velocity distribution super-resolution construction method based on multi-modal data fusion. BACKGROUND

[0002] The spatial and temporal variation of underwater sound velocity causes the path of sound signal propagation to bend due to the Snell effect, which brings challenges to accurate ranging for underwater positioning and navigation and efficient energy transmission for communication systems. Therefore, to solve the problem of predicting the sound velocity profile in space, a sound velocity profile prediction method based on a radial basis function (RBF) neural network is proposed by a team of Shandong University. This method uses historical temperature and salinity profiles and position information at the sampling time to construct the temperature and salinity field network, uses historical profile data and position information at the sampling time to construct the sound velocity field network, and constrains the sound velocity field network through the temperature and salinity field network. In 2022, Shandong University of Science and Technology proposed a self-organizing map (SOM) neural network-based full-depth sound velocity profile inversion method. This method makes full use of global satellite remote sensing system sea surface temperature measurement data, historical sound velocity profile data, and optional sea surface sound velocity measurement results. China Ocean University proposed a sound velocity profile prediction method based on a convolutional neural network (CNN) neural network. This method uses historical sound velocity principal component features, sea surface remote sensing temperature, and latitude and longitude coordinates to perform spatial sound velocity data fusion through CNN to construct a sound velocity field.

[0003] Although the method based on the RBF neural network does not require measured sound field observation data, the construction result tends to be the sample mean, and it is difficult to accurately describe the spatial and temporal variation of the sound velocity profile. Compared with the sound velocity profile prediction method based on the RBF neural network, the SOM neural network-based method can construct different spatial sound velocity distributions and better reflect the spatial difference of the sound velocity distribution. However, the SOM neural network-based method cannot fully depict the sound velocity distribution of the seasonal jump layer because the model relies on sea surface temperature and sound velocity data. Although the CNN neural network-based method has improved prediction accuracy compared to the SOM method, the high computational complexity makes it difficult to apply to large-scale sound velocity field construction, and the generalization ability is insufficient.

[0004] Therefore, it is necessary to propose an underwater sound velocity distribution super-resolution construction method based on multi-modal data fusion to solve the above technical problems in the prior art. SUMMARY

[0005] The underwater sound velocity distribution super-resolution construction method based on multi-modal data fusion aims to fuse underwater temperature and sea surface temperature data to quickly and accurately predict the sound velocity distribution of the entire spatial region under the condition of original low spatial resolution data.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solutions.

[0007] The underwater sound velocity distribution super-resolution construction method based on multi-modal data fusion comprises the following steps:

[0008] Step 1, obtaining temperature data for underwater sound velocity distribution super-resolution construction, and sequentially constructing original spatial region temperature data set, prediction task spatial region temperature data set and prediction task spatial region sea surface temperature data set;

[0009] Step 2, each data set in step 1 is divided into a training set and a validation set, and is preprocessed;

[0010] Step 3, building a sound velocity distribution prediction model; the sound velocity distribution prediction model comprises a temperature reconstruction module and a sound velocity distribution calculation module; the temperature reconstruction module comprises an Unet neural network, a Mamba neural network, a cross attention module and a convolution layer;

[0011] The preprocessed original spatial region temperature data is input into the Unet neural network for temperature preliminary reconstruction; the preprocessed sea surface temperature data is input into the Mamba neural network to extract sea surface temperature features;

[0012] The results of the temperature preliminary reconstruction by the Unet neural network and the sea surface temperature features are input into the cross attention module, and the results of the temperature preliminary reconstruction are corrected by the sea surface temperature features;

[0013] Then, the results corrected by the cross attention module are connected in residual with the results of the temperature preliminary reconstruction output by the Unet neural network, and are input into the convolution layer to obtain high spatial resolution temperature data;

[0014] Finally, the high spatial resolution temperature data is input into the sound velocity distribution calculation module to obtain high spatial resolution sound velocity distribution data;

[0015] Step 4, based on the training set and the validation set in step 2, the sound velocity distribution prediction model built in step 3 is trained and verified, and then the trained sound velocity distribution prediction model is used for underwater sound velocity distribution prediction.

[0016] Compared with the prior art, the present application has the following beneficial effects:

[0017] As described above, this invention discloses a method for constructing super-resolution underwater sound velocity distribution based on multimodal data fusion. This method establishes a sound velocity distribution prediction model based on Unet and Mamba neural networks. This model mainly consists of a Unet neural network, a Mamba neural network, and a cross-attention module. The model reconstructs high-spatial-resolution temperature data from low-spatial-resolution raw temperature data using the Unet neural network. Then, the Mamba neural network fully extracts features from the sea surface temperature data and corrects the reconstructed temperature data, improving the accuracy of the reconstructed high-spatial-resolution temperature data. The reconstructed high-spatial-resolution temperature data is then processed by the sound velocity distribution calculation module to obtain high-spatial-resolution sound velocity distribution data. This invention, through multimodal data fusion (fusion of underwater temperature and sea surface temperature data), achieves rapid and accurate prediction of sound velocity distribution across the entire spatial region even with low-spatial-resolution raw data, solving the problem of insufficient spatial resolution in ocean sound velocity distribution estimation. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below.

[0019] Figure 1 This is a flowchart of the underwater sound velocity distribution super-resolution construction method based on multimodal data fusion in the embodiment;

[0020] Figure 2 This is a schematic diagram of the sound speed distribution prediction model in the embodiment;

[0021] Figure 3 for Figure 2 The input sea surface temperature data map;

[0022] Figure 4 for Figure 2 Low spatial resolution temperature data plots input from the database;

[0023] Figure 5 This is a schematic diagram of the Unet neural network structure in the embodiment;

[0024] Figure 6 This is a diagram comparing self-attention and adjacent attention.

[0025] Figure 7 This is a schematic diagram of the SS2D module in the embodiment;

[0026] Figure 8 A comparative diagram showing the RMSE of temperature reconstruction using several methods;

[0027] Figure 9 The RMSE plot of the sound velocity distribution predicted by the linear spatial interpolation method;

[0028] Figure 10 The sound speed distribution RMSE graph predicted by the Unet neural network;

[0029] Figure 11 The sound speed distribution RMSE graph predicted by the method of the embodiment;

[0030] Figure 12 The schematic diagram for the cross-attention module to correct the underwater temperature by the sea surface temperature feature. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments.

[0032] Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0033] In addition, the description such as "first", "second" and the like in the present application is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise specifically limited.

[0034] Embodiment 1

[0035] As shown in Figure 1 The present embodiment describes a method for constructing underwater sound speed distribution super-resolution based on multi-modal data fusion. The method can quickly and accurately predict the sound speed distribution of the entire spatial region under the condition of original low spatial resolution data through multi-modal data fusion (data fusion of underwater temperature and sea surface temperature). The method includes the following steps:

[0036] Step 1, obtaining temperature data for constructing underwater sound speed distribution super-resolution, and sequentially constructing original spatial region temperature data set, prediction task spatial region temperature data set and prediction task spatial region sea surface temperature data set.

[0037] Step 1.1, according to the user demand, determine the low spatial resolution of the original data in the sound speed distribution prediction task, denoted as R°xR°. Select the original spatial region coordinates , where represents the longitude, covering the range of M°~N°, and the step is R°; and Represents the dimension, with a coverage range from P° to Q°, and a step size of R°. In this embodiment, the longitude has a coverage range from 150.5°W to 123.0°W, and the latitude has a coverage range from 28.0°S to 0.5°S. The low spatial resolution is 2.5°×2.5°, and the step size is 2.5° as an example.

[0038] Step 1.2: Determine the high spatial resolution for the prediction according to the requirements of the sound speed distribution prediction task, denoted as S°×S°; the prediction time resolution, represented by . Select the coordinates of the spatial area of the prediction task , represents the longitude, with a coverage range from M° to N°, and a step size of S°; while represents the dimension, with a coverage range from P° to Q°, and a step size of S°; where S < R. In this embodiment, the longitude has a coverage range from 150.5°W to 123.0°W, and the latitude has a coverage range from 28.0°S to 0.5°S. The high spatial resolution is 0.5°×0.5°, and the step size is ×0.5° as an example. In the sound speed distribution prediction task, the prediction time resolution is 1 month, and the time span is 240 months from January 2004 to December 2023.

[0039] Step 1.3: Obtain the temperature data for the underwater sound speed distribution super-resolution construction based on the resolution and coordinate data determined in Step 1.1 and Step 1.2, and construct the original spatial area temperature dataset, the prediction task spatial area temperature dataset, and the prediction task spatial area sea surface temperature dataset according to the obtained temperature data.

[0040] Step 1.3.1: Select the temperature data in the GDCSM_Argo data according to the original spatial area coordinates determined in Step 1.1, stratify it in the vertical direction according to depth, and arrange it in the horizontal direction according to time. Then perform linear spatial interpolation on the selected data. The number of interpolation points between two adjacent spatial positions is R / S - 1. In this embodiment, the number of interpolation points between two adjacent spatial positions is 4. Therefore, the original data spatial resolution is increased from 2.5°×2.�° to 0.5°×0.5°, forming the original spatial area temperature dataset.

[0041] The process of linearly stratifying the temperature data in the GDCSM_Argo data along the vertical depth direction is as follows: from 0 to 10 meters, a layer is divided every 5 meters; from 10 to 180 meters, a layer is divided every 10 meters; from 180 to 460 meters, a layer is divided every 20 meters; from 460 to 500 meters, a separate layer is formed; from 500 to 1300 meters, a layer is divided every 50 meters; from 1300 to 1900 meters, a layer is divided every 100 meters; and depths exceeding 1900 meters are divided into a single layer, for a total of 58 layers.

[0042] Let the original spatial region temperature dataset be , is represented as:

[0043] ;

[0044] in, Represented as the first A set of temperature profile data, temperature dataset of the entire original spatial region. It consists of 240 temperature profile datasets arranged along the time dimension, with time intervals corresponding to the forecast time resolution. The duration is consistently one month. Among them, the first... A set of temperature profile data Represented as:

[0045] ;

[0046] in, For transpose, Representing the entire region in the The moment of the first A set of temperatures at layer depth. Among them, Represented as:

[0047] ;

[0048] in, The selected area has the following coordinates: The first In the temperature profile, the first Layer data.

[0049] Temperature dataset of the original spatial region According to the In the temperature profile data set, the first If the data is expanded, then Specifically, it is expressed as follows:

[0050] .

[0051] Step 1.3.2, Since the spatial resolution of the original GDCSM_Argo data is 1°x1°, the original GDCSM_Argo data is first linearly spatially interpolated, and the number of interpolation points between two adjacent spatial positions is 1. Then, according to the high spatial resolution S°xS determined in step 1.2, the temperature data in the spatially interpolated GDCSM_Argo data is selected, and the temperature data set in the prediction task space region is arranged according to the depth layering in the vertical direction and the time in the horizontal direction to form the prediction task space region temperature data set.

[0052] Let the prediction task space region temperature data set be , which is expressed as:

[0053] ;

[0054] wherein, is the jth temperature profile data set, and the entire prediction task space region temperature data set is composed of 240 temperature profile data sets arranged in the time dimension, and the time interval and the prediction time resolution are consistent, i.e., 1 month. The jth temperature profile data set is expressed as:

[0055] ;

[0056] wherein, is the transpose, represents the temperature set at the jth layer depth of the entire region at the ith time. Wherein, is expressed as:

[0057] ;

[0058] wherein, represents the jth layer data in the jth temperature profile at the coordinate position in the selected region.

[0059] The prediction task space region temperature data set is unfolded according to the jth layer data in the jth temperature profile data set, which is specifically expressed as:

[0060] .

[0061] Step 1.3.3, According to the prediction task space region coordinates​​​​​​​​ The sea surface temperature in the NOAA dataset is selected and arranged in the horizontal direction according to the time dimension to form the forecast task space area sea surface temperature data.

[0062] Let the forecast task space area sea surface temperature dataset be , which is expressed as:

[0063]

[0064] wherein, represents the i-th sea surface temperature set, and the forecast task space area sea surface temperature dataset contains 240 monthly data sets arranged in the time dimension, and the time interval is also consistent with the forecast time resolution of 1 month. The i-th sea surface temperature set is expressed as:

[0065]

[0066] wherein, represents the i-th time sea surface temperature data at the coordinate position in the forecast area. Through the above step 1, the original space area temperature dataset, the forecast task space area temperature dataset, and the forecast task space area sea surface temperature dataset are collected and selected, and then step 2 is entered.

[0067] Step 2, the original space area temperature dataset, the forecast task space area temperature dataset, and the forecast task space area sea surface temperature dataset are divided into training set and validation set, and preprocessed.

[0068] Step 2.1, the original space area temperature dataset, the forecast task space area temperature dataset, and the forecast task space area sea surface temperature dataset are divided into training set and test set.

[0069] According to the natural change law of sound speed data, the data change time period is set to

[0070] , the first temperature data in the original space area temperature dataset , the forecast task space area temperature dataset , and the forecast task space area sea surface temperature dataset are taken as the training set, and the last temperature data are taken as the test set, wherein .

[0071] ​​​​​The data change time period is set to 12 months in this embodiment, and the temperature data of the first 228 months of the data set , are taken as the training set, respectively represented as , and , and the formula is as follows:

[0072] ;

[0073] ;

[0074] .

[0075] The temperature data of the last 12 months in the data set , are taken as the test set, respectively represented as , and , and the formula is as follows:

[0076] ;

[0077] ;

[0078] .

[0079] Step 2.2, data preprocessing is performed on the training set , , and the test set , , in step 2.1.

[0080] First, copy and fill and to the same depth layer as , and the formula is as follows:

[0081] ;

[0082] ;

[0083] Then, the training set is subjected to hierarchical standardization processing, represented as , and the formula is as follows:

[0084] ;

[0085] Specifically, in the training set Each layer depth layer divided in vertical direction is respectively processed according to zero mean standardization, and then processed according to training set The mean and standard deviation obtained in the standardization are respectively 、 、 、 and standardized to obtain 、 、 、 and .

[0086] Through the above step 2, the preprocessing of each data set is completed, and then step 3 is entered.

[0087] Step 3, according to the different depth layers, a sound speed distribution prediction model is built on each depth layer for sound speed distribution prediction.

[0088] As shown in Figure 2 , each sound speed distribution prediction model includes a temperature reconstruction module and a sound speed distribution calculation module. The temperature reconstruction module is composed of Unet neural network, Mamba neural network, cross attention module and convolution layer. In the temperature reconstruction module, first, the low spatial resolution temperature data (original spatial region temperature data) preprocessed in step 2 is input into the Unet neural network for temperature preliminary reconstruction, and the temperature preliminary reconstruction result is output; then the sea surface temperature data preprocessed in step 2 is input into the Mamba neural network for sea surface temperature feature extraction; then the sea surface temperature features extracted by the Mamba neural network and the temperature preliminary reconstruction result output by the Unet neural network are input into the cross attention module, and the temperature preliminary reconstruction result is corrected by the sea surface temperature features; then the result corrected by the cross attention module is connected with the temperature preliminary reconstruction result output by the Unet neural network, and the high spatial resolution temperature data is obtained by inputting the convolution layer; then input into the sound speed distribution calculation module. In the sound speed distribution calculation module, the high spatial resolution temperature data and the salinity data (the salinity data corresponding to the prediction task space region in step 2 in GDCSM_Argo data) are input into the sound speed empirical formula to obtain the high spatial resolution sound speed distribution data.

[0089] The components in the temperature reconstruction module will be described below.

[0090] The Unet neural network is composed of an encoder on the left half and a decoder on the right half, taking layers Unet neural network as hidden layer, and the number of neurons of each corresponding hidden layer is taken as In the encoder part, feature extraction is performed on the input image samples through a series of convolution and down-sampling operations; in the decoder part, the feature maps are gradually restored to the original image size through up-sampling and the skip connection with the decoder part, and finally the image after preliminary reconstruction is output.

[0091] The input of the Unet neural network is the low spatial resolution temperature data determined in step 2, and this embodiment takes 56x56x1 as an example, where the last 1 dimension is the number of channels. This embodiment takes 3 layers of Unet layers as the hidden layer, and the number of neurons in each layer of the hidden layer is taken as 64, 128, and 256 in turn. As shown in Figure 5 shown, the input first passes through two 3x3 convolution layers to extract low-level features (edges, textures) of the underwater temperature data while maintaining the spatial size. Then it is processed by the first down-sampling and convolution module, the second down-sampling and convolution module, and the third down-sampling and convolution module to extract high-level features of the image; in each down-sampling and convolution module, the features are first down-sampled by a max-pooling layer to halve the spatial size, and then pass through two 3x3 convolution layers to extract deeper features. Then the output features of the third down-sampling and convolution module are processed by the first up-sampling and convolution module, the second up-sampling and convolution module, and the third up-sampling and convolution module to gradually restore the multi-scale information of the image; in the first up-sampling and convolution module, the features first pass through an up-convolution layer for up-sampling, then are connected with the output features of the second down-sampling and convolution module through a skip connection, and then are input into the second up-sampling and convolution module after passing through two 3x3 convolution layers; in the second up-sampling and convolution module, the features first pass through an up-convolution layer for up-sampling, then are connected with the output features of the first down-sampling and convolution module through a skip connection, and then are input into the third up-sampling and convolution module after passing through two 3x3 convolution layers; in the third up-sampling and convolution module, the features first pass through an up-convolution layer for up-sampling, then are connected with the input features of the first down-sampling and convolution module through a skip connection, and then are input into a 1x1 convolution layer after passing through two 3x3 convolution layers; finally, the result of preliminary reconstruction of the temperature is obtained by outputting through the 1x1 convolution layer.

[0092] The structure of the Mamba neural network is shown in the upper left corner of the temperature reconstruction module in Figure 2 The Mamba neural network is composed of an adjacency attention module, an SS2D module, and a convolution layer, and takes layers of Mamba neural networks as hidden layers, and each layer of the hidden layer contains L neurons.

[0093] This embodiment takes 3 layers of Mamba neural networks as hidden layers, and each layer of the hidden layer contains 64 neurons.

[0094] The input of the Mamba neural network is the preprocessed sea surface temperature data in the prediction task space region in step 2. The input is first subjected to an adjacent attention module to extract local features of the sea surface temperature data, then subjected to an SS2D module to capture global features, then the output features of the adjacent attention module are connected with the output features of the SS2D module to perform feature fusion, then the fused features are input into a convolution layer to perform local and global feature fusion, and finally the output features of the convolution layer are connected with the input features of the Mamba neural network to perform feature fusion to obtain the sea surface temperature data features.

[0095] The structure of the SS2D module is shown in Figure 7 The module is used to extract global features of the input image. The input 2D image is first divided into small pixel blocks and then expanded into 1D sequences along four different directions from front to back, from back to front, from left to right and from right to left by rows and columns respectively. Then the 1D sequences from the four different directions are input into the selective state space model S6 in parallel to capture sequence features, and finally combined into a 2D image.

[0096] The selective state space model (Selective State Space Models, S6) is a core component of the Mamba neural network. It introduces a selective mechanism based on the state space model (State Space Model, SSM). By adjusting the SSM parameters through input-dependent manner, the model allows selective transmission or forgetting of information according to the current data.

[0097] The overall calculation process of the selective state space model is as follows:

[0098] ;

[0099] Wherein, represents the system input, represents the hidden state vector at the current time, represents the output result at the discrete time, represents the output matrix, represents the direct transfer matrix, and represent the discretized state transition matrix and input matrix, respectively. and are further represented as:

[0100] ;

[0101] ;

[0102] In the formula, is the discretized time step, is a natural constant, denotes a unit matrix.

[0103] As Figure 6 shown, the self-attention allows each pixel in the image to pay attention to all pixels, and unlike the self-attention, the adjacent attention focuses on the pixels in the field around the pixel, which is beneficial to fully extract the local features of the input image.

[0104] As Figure 12 shown, Q in the cross-attention mechanism comes from one sequence, and K and V come from another sequence, which is used to capture the dependency between the two sequences. At the same time, since the solar radiation energy directly acts on the ocean surface, the heat is transferred from the sea surface to the deep layer, and decreases with the increase of depth. Therefore, the sea water temperature in the shallow layer is not much different from the sea surface temperature, and the distribution of the sea water temperature in the shallow depth is highly close to the distribution of the sea surface temperature (SST) in the horizontal and vertical directions, so it can be used to correct the sea water temperature in the shallow depth. Therefore, the cross-attention module is adopted in the present application, and the result of the preliminary reconstruction of the temperature is corrected by the sea surface temperature feature. For the deep depth, the sea surface temperature has limited guidance ability for the underwater temperature. Therefore, considering that the feature of the preliminary reconstruction of the temperature after the Unet neural network processing may be deteriorated in the fusion process in the deep depth, a residual connection is added after the cross-attention module, which is used to maintain the original feature of the preliminary reconstruction of the temperature, and ensures the accuracy of the fusion result.

[0105] The sound speed distribution calculation module will be described below.

[0106] In the sound speed distribution calculation module, the high spatial resolution temperature data output by the temperature reconstruction module and the salinity data are substituted into the Coppens sound speed empirical formula to calculate the high spatial resolution sound speed distribution data. The Coppens sound speed empirical formula is expressed as:

[0107] ;

[0108] In the formula, represents the sound speed, and the unit is m / s; represents the temperature, and the unit is d℃, i.e. d℃=℃*10^(-1); represents the salinity, and the unit is %; represents the depth, and the unit is m.

[0109] Step 4, based on the training set and the validation set in step 2, the sound speed distribution prediction model built in step 3 is trained and validated, and then the trained sound speed distribution prediction model is used to predict the underwater sound speed distribution.

[0110] Step 4.1: Train the sound speed distribution prediction model built in Step 3.

[0111] During model training, the temperature reconstruction module of the sound velocity distribution prediction model at each depth layer is trained separately. Before model training, the learning rate of the network training parameters at each depth layer is determined separately. and the number of training iterations ,in Representing the Layer depth, with values... .

[0112] With the first depth layer For example, in each iteration of the training process, the input to the Unet neural network model is first the standardized original spatial region temperature dataset. The first data in the first row The input to the Mamba neural network model is a standardized dataset of sea surface temperatures for the forecast task's spatial region. The first data in the first row At this point, the model's output is the standardized temperature dataset for the forecast task's spatial region. The first data in the first row .

[0113] Next, the input to the Unet neural network model is the standardized original spatial region temperature dataset. The second data in the first row The input to the Mamba neural network model is a standardized dataset of sea surface temperatures for the forecast task's spatial region. The second data in the first row At this point, the model's output is the standardized temperature dataset for the forecast task's spatial region. The second data in the first row And so on, until all have been traversed. , as well as The first row of data in the three datasets represents all the training data for the first deep layer. The training methods for the other deep layers are the same as those for the first deep layer.

[0114] Since the reconstructed temperature is based on a monthly time resolution, and sea surface temperatures vary from month to month, it is necessary to train the Mamba neural network during the training process to improve its generalization ability in order to fully extract sea surface temperature features.

[0115] Step 4.2: Validate the sound velocity distribution prediction model at each depth layer trained in Step 4.1.

[0116] With the first depth layer For example, firstly, the standardized original spatial region temperature dataset is... and the standardized sea surface temperature dataset for the forecast mission spatial region The first data in the first row, that is as well as Using these as inputs to the Unet and Mamba neural networks respectively, the model's predicted output is denoted as... Next, the standardized original spatial region temperature dataset will be used. and the standardized sea surface temperature dataset for the forecast mission spatial region The second data in the first row, i.e. as well as Using these as inputs to the Unet and Mamba neural networks respectively, the model's predicted output is denoted as... And so on, until all instances are traversed. as well as The first row of data in both datasets represents the data from the first deep layer. At this point, the validation of the first deep layer model is complete, and the predicted output is denoted as... .

[0117] The model validation process for other depth layers is the same as for the first depth layer. Therefore, the data is obtained by organizing the prediction outputs of all depth layer models. :

[0118] ;

[0119] After completing the sound velocity distribution prediction model verification in step 4.2, proceed to step 4.3.

[0120] Step 4.3: Destandardize the data predicted in Step 4.2.

[0121] To accelerate model convergence and improve its generalization ability, zero-mean normalization was performed on the input data in the data preprocessing section of step 2. Therefore, to obtain the actual temperature prediction value, the data from the prediction output of step 4.2 needs to be normalized according to the normalization parameters in step 2. Destandardization, then the actual temperature prediction value It is expressed as follows:

[0122] ;

[0123] After completing the destandardization of the prediction data in step 4.3, proceed to step 4.4.

[0124] Step 4.4: Calculate the root mean square error (RMSE).

[0125] Calculate the actual temperature prediction values ​​for the spatial region of the forecast mission. With verification temperature data The root mean square error (RMSE) between them, where the first... The root mean square error at the layer depth is expressed as:

[0126] ;

[0127] in, Indicates the first Time of the first The predicted actual temperature values ​​for the spatial region of the forecast mission at the layer depth level. Indicates the first Time of the first Verification temperature data on the layer depth layer, Indicates total That moment.

[0128] If the root mean square error (RMSE) does not change significantly compared to the previous round (RMSE no longer decreases and tends to a stable value), then proceed to step 4.5; otherwise, return to step 4.1.

[0129] The model was validated multiple times to obtain the average RMSE between the reconstructed high spatial resolution temperature data and the actual temperature across the entire depth layer. For example... Figure 8 As shown, the RMSE of temperature reconstruction using the linear spatial interpolation method, the Unet neural network alone, and the method of this embodiment are compared respectively. Figure 8 As shown, compared to linear spatial interpolation methods, the method in this embodiment further reduces the mean and variance of the average RMSE across the entire forecast spatial region, resulting in a more concentrated distribution. Furthermore, based on the Unet neural network branch, the Mamba neural network branch can further correct the temperature across the entire depth layer using sea surface temperature, improving the model's temperature reconstruction capability.

[0130] The reconstructed high spatial resolution temperature data is then input into the sound velocity distribution calculation module to obtain the predicted high spatial resolution sound velocity distribution data. Figure 9 , Figure 10 as well as Figure 11 The RMSE (Resolution-to-Screen Error) comparisons between high spatial resolution sound velocity distribution data predicted by linear spatial interpolation, using only the Unet neural network portion, and the sound velocity distribution prediction model of this embodiment, and the actual sound velocity distribution data, are presented respectively. Figure 9 , Figure 10 and Figure 11As shown, the linear space interpolation method has a large prediction error overall, especially in the area near the equator, with the highest error reaching 2.35 m / s; the error of the Unet neural network prediction is obviously improved, but there is still a prediction error of more than about 0.5 m / s in the equatorial area and the area near 30°S of the forecast area; and the prediction error is further reduced by the sound speed distribution prediction model of the embodiment, and the error of the entire forecast area tends to be consistent, all less than about 0.5 m / s.

[0131] Step 4.5, using the trained sound speed distribution prediction model to predict the underwater sound speed distribution.

[0132] According to the low spatial resolution and original spatial region coordinates of the original data in the sound speed distribution prediction task, and the prediction time resolution, high spatial resolution and prediction task spatial region coordinates in the sound speed distribution prediction task, the original spatial region temperature data and the sea surface temperature data of the prediction task spatial region are input into the temperature reconstruction module of the sound speed distribution prediction model to obtain the reconstructed high spatial resolution temperature data, and finally the reconstructed temperature data is input into the sound speed distribution calculation module of the sound speed distribution prediction model, and finally the predicted high spatial resolution sound speed distribution data is obtained.

[0133] The embodiment corrects the predicted large-area underwater temperature data by the sea surface temperature data to obtain higher-precision temperature data, and finally calculates the high-precision sound speed data through the salinity data and the sound speed empirical formula. The embodiment solves the problem that predicting the underwater temperature of the entire large-area region only by a small part of the known underwater temperature data will produce a large error.

[0134] The embodiments of the present application are only used to illustrate the technical solutions of the present application and not to limit, and for those skilled in the art, it can be understood that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and spirits of the present application, the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for constructing super-resolution underwater sound velocity distribution based on multimodal data fusion, characterized in that, Includes the following steps: Step 1: Obtain temperature data for underwater sound velocity distribution super-resolution construction, and sequentially construct the original spatial region temperature dataset, the forecast mission spatial region temperature dataset, and the forecast mission spatial region sea surface temperature dataset. Step 2: Divide each dataset from Step 1 into a training set and a validation set, and perform preprocessing. Step 3: Build a sound velocity distribution prediction model; this sound velocity distribution prediction model includes a temperature reconstruction module and a sound velocity distribution calculation module; the temperature reconstruction module includes a Unet neural network, a Mamba neural network, a cross-attention module, and convolutional layers; The preprocessed raw spatial temperature data is input into the Unet neural network for preliminary temperature reconstruction; the preprocessed sea surface temperature data is input into the Mamba neural network to extract sea surface temperature features. The Mamba neural network consists of an adjacency attention module, an SS2D module, and convolutional layers. The Mamba neural network is used as the hidden layer, with each hidden layer containing L neurons. The preprocessed sea surface temperature data of the forecast task spatial area first passes through the adjacency attention module to extract local features of the sea surface temperature data, then passes through the SS2D module to capture global features, and then the output features of the adjacency attention module and the output features of the SS2D module are connected by a skip connection to perform feature fusion. Then the fused features are input into the convolutional layer for local and global feature fusion. Finally, the output features of the convolutional layer are connected by a skip connection to the input features of the Mamba neural network to perform feature fusion, thus obtaining the sea surface temperature data features. The SS2D module is used to extract global features from the input image. The input 2D image is first segmented into small pixel blocks and then expanded into 1D sequences by rows and columns along four different directions: front to back, back to front, left to right, and right to left. Then, the 1D sequences from the four different directions are input in parallel into the selective state space model S6 to capture sequence features, and finally merged into a 2D image. The selective state space model S6 introduces a selective mechanism on the basis of the state space model, adjusting the SSM parameters in a way that depends on the input, allowing the model to selectively pass or forget information based on the current data. The initial temperature reconstruction results obtained through the Unet neural network and the sea surface temperature features are input into the cross-attention module, and the initial temperature reconstruction results are corrected by the sea surface temperature features. Then, the result corrected by the cross-attention module is residually connected with the initial temperature reconstruction result output by the Unet neural network, and then input into the convolutional layer to obtain high spatial resolution temperature data; Finally, the high spatial resolution temperature data is input into the sound velocity distribution calculation module to obtain high spatial resolution sound velocity distribution data. Step 4: Based on the training set and validation set in Step 2, train and validate the sound speed distribution prediction model built in Step 3, and then use the trained sound speed distribution prediction model to predict the underwater sound speed distribution.

2. The underwater sound velocity distribution super-resolution construction method based on multimodal data fusion according to claim 1, characterized in that, Step 1 specifically involves: Step 1.1: Determine the low spatial resolution R°×R° of the raw data in the sound velocity distribution prediction task, and select the original spatial region coordinates as... ; Represents longitude, covering a range of M° to N°, with a step size of R°; The representative dimension covers a range of P° to Q°, with a step size of R°. Step 1.2: Determine the forecast time resolution in the sound velocity distribution forecasting task. With a high spatial resolution of S°×S°, the coordinates of the spatial region for the prediction mission are selected as follows: ; Represents longitude, covering a range of M° to N°, with a step size of S°; The representative dimension covers a range of P° to Q°, with a step size of S°; where S... <R; Step 1.3: Based on the resolution and coordinate data determined in Steps 1.1 and 1.2, acquire temperature data for underwater sound velocity distribution super-resolution construction, and construct the original spatial region temperature dataset, the forecast mission spatial region temperature dataset, and the forecast mission spatial region sea surface temperature dataset based on the acquired temperature data.

3. The underwater sound velocity distribution super-resolution construction method based on multimodal data fusion according to claim 2, characterized in that, The specific process of step 1.3 is as follows: Step 1.3.1: Based on the original spatial region coordinates determined in Step 1.1 Temperature data from the GDCSM_Argo dataset were selected and arranged vertically by depth and horizontally by time. Then, linear spatial interpolation was performed on the selected data to construct the original spatial temperature dataset. , is represented as: ； in, ,express have A set of temperature profiles, with time intervals consistent with the forecast time resolution; ,express have Each layer has a depth; , , and These represent the coordinate positions in the original spatial region. , , and The first The first temperature profile Layer temperature data; Step 1.3.2: First, linear spatial interpolation is performed on the original GDCSM_Argo data. Then, based on the high spatial resolution S°×S determined in Step 1.2, temperature data are selected from the spatially interpolated GDCSM_Argo data and arranged vertically according to depth and horizontally according to time to form the spatial region temperature dataset for the forecasting task. , is represented as: ； in, , , and These represent the coordinates of the forecast area. , , and The first The first temperature profile Layer temperature data; Step 1.3.3: Determine the coordinates of the spatial region for the forecasting mission based on Step 1.

2. The sea surface temperatures in the NOAA dataset are selected and arranged horizontally according to the time dimension to form the sea surface temperature data for the forecast task spatial region. , is represented as: ； in, , , and These represent the coordinates of the forecast area. , , and The first Sea surface temperature data at specific times.

4. The underwater sound velocity distribution super-resolution construction method based on multimodal data fusion according to claim 3, characterized in that, Step 2 specifically involves: Step 2.1: Based on the natural variation pattern of sound speed data, set the data variation time period to [value missing]. The original spatial region temperature dataset Temperature dataset for forecasting spatial regions And the sea surface temperature dataset for the forecast mission spatial region The front of the middle Using temperature data as the training set, finally 100 temperature data points were used as the validation set, and The training set is represented as... , and The validation set is represented as , and ; Step 2.2, firstly... and Copy and fill to and At the same depth and number of layers, then Stratified standardization process is performed to obtain ; In the training set Each depth layer in the vertical direction is standardized according to zero mean, and then... The mean and standard deviation obtained in the standardization process are respectively for , , , as well as Standardization process is required to obtain , , , as well as .

5. The underwater sound velocity distribution super-resolution construction method based on multimodal data fusion according to claim 1, characterized in that, In step 3, the processing procedure of the Unet neural network is as follows: The Unet neural network consists of an encoder and a decoder. The Unet neural network uses layers as hidden layers, and the number of neurons in each hidden layer is taken sequentially. The encoder extracts features from the input image samples through a series of convolution and downsampling operations; the decoder then gradually restores the original image size from the feature map through upsampling and skip connections with the decoder part, and finally outputs the preliminarily reconstructed image.

6. The underwater sound velocity distribution super-resolution construction method based on multimodal data fusion according to claim 1, characterized in that, In step 3, the processing procedure of the sound velocity distribution calculation module is as follows: In the sound velocity distribution calculation module, the high spatial resolution temperature data and salinity data output by the temperature reconstruction module are substituted into the Coppens sound velocity empirical formula to calculate the high spatial resolution sound velocity distribution data.

7. The underwater sound velocity distribution super-resolution construction method based on multimodal data fusion according to claim 3, characterized in that, In step 3, a sound speed distribution prediction model is built for each depth layer according to the different depth layers.

8. The underwater sound velocity distribution super-resolution construction method based on multimodal data fusion according to claim 7, characterized in that, In step 4, the specific steps for training and validating the sound velocity distribution prediction model are as follows: Step 4.1: Before model training, determine the learning rate of the network training parameters for each depth layer. and the number of training iterations ,in Representing the Layer depth, with values... During model training, the temperature reconstruction module of the sound velocity distribution prediction model at each depth layer is trained separately. First depth layer In each iteration of the sound speed distribution prediction model training process, the Unet neural network is first input... The first data in the first row Mamba neural network input The first data in the first row Then the model output The first data in the first row Then the Unet neural network input The second data in the first row Mamba neural network input The second data in the first row Then the model output The second data in the first row ; And so on, until all have been traversed. , as well as The first row of data in the three datasets, which contains all the training data for the first deep layer; The training methods for sound velocity distribution prediction models at other depth layers are the same as those at the first depth layer. The training method for the sound speed distribution prediction model is the same; Step 4.2: Validate the sound velocity distribution prediction model after each layer of training is completed; For the first depth layer When validating the sound speed distribution prediction model, firstly... The first data in the first row as well as The first data in the first row Using these as inputs to the Unet and Mamba neural networks respectively, the model's predicted output is denoted as... Next, The second data in the first row as well as The second data in the first row Using these as inputs to the Unet and Mamba neural networks respectively, the model's predicted output is denoted as... And so on, until all instances are traversed. as well as The first row of data in both datasets, i.e., the data from the first depth layer, is denoted as the predicted output. ; The verification process of the sound velocity distribution prediction model at other depth layers is the same as that at the first depth layer. The validation process for the sound velocity distribution prediction model is the same across all depth layers, therefore the predicted output data of the sound velocity distribution prediction model at all depth layers are consistent. for: ； in, Indicates the first The moment of the first The model predicts temperature data for the entire forecast region at the layer depth. Step 4.3: Analyze the predicted output data from Step 4.

2. Destandardization, then the actual temperature prediction value It is expressed as follows: ； in, Indicates the first The moment of the first The actual temperature prediction value for the entire forecast area at the layer depth; Step 4.4: Calculate the root mean square error between the actual predicted temperature value and the verification temperature data for the spatial region of the forecast mission, where the first... The root mean square error at the layer depth is expressed as: ； in, Indicates the first Time of the first The predicted actual temperature values ​​for the spatial region of the forecast mission at the layer depth level. Indicates the first Time of the first Verification temperature data on the layer depth layer, Indicates total That moment.

Images