A high-resolution sound velocity profile data compression method based on AE autoencoder

Abstract

The present invention belongs to the technical field of ocean sound speed profile data compression, and in particular, relates to a high-resolution sound speed profile data compression method based on AE autoencoder. High-resolution ocean reanalysis products are used to provide data of global sea areas, and sound speed formula conversion is used to obtain seawater sound speed profile data of specific sea areas; the missing data in the high-resolution sound speed profile data set is supplemented by a sliding interpolation method, and the normalized data set is divided into a training set, a test set, and a validation set; an AE autoencoder sound speed profile data compression and reconstruction network is built, and the network is iteratively trained using a training set and a validation set; the trained AE autoencoder sound speed profile data compression and reconstruction network is used to compress and reconstruct the target data, and its compression performance is evaluated. The present invention provides a high-resolution sound speed profile data compression method based on an AE autoencoder, which significantly compresses the data volume of the ocean high-resolution sound speed profile and can reconstruct the data with high precision.

Description

Technical Field

[0001] The invention belongs to the technical field of compression and reconstruction of ocean sound speed profile data, and in particular relates to a high-resolution sound speed profile data compression method based on an AE autoencoder. Background Art

[0002] The seawater sound velocity is a marine environmental parameter that has an important influence on the propagation of sound in the ocean. High-precision sound velocity profiles are beneficial to the relevant research of marine acoustics, improve the accuracy of various applications, and provide a strong guarantee for the inversion and testing of the marine environment. Due to the large ocean area, irregular depth changes, and large parameter differences, the amount of high-resolution sound velocity profile data is very large. The data file size at a single data sampling moment is in the GB range. It is difficult for ordinary personal computer terminals to store long-term series of high-resolution sound velocity profile data. Therefore, effectively compressing high-resolution sound velocity profile data and reconstructing the compressed data into original data with high precision will reduce the burden of data storage and promote the in-depth development of research in the field of marine sound propagation.

[0003] Previous studies have used dictionary learning to sparsely represent the sound velocity profile. The redundant characteristics of the overcomplete dictionary can capture the intrinsic essential characteristics of the sound velocity profile, better characterize the deep characteristics of the sound velocity profile, and improve the accuracy of the seawater sound velocity profile inversion. In addition, based on high-resolution ocean reanalysis products, efficient parallel calculation of water acoustic field data sets under mesoscale conditions can be achieved, and the running speed of mesoscale vortex water acoustic characteristics can be accelerated by 50%. Traditional methods and research contents have conducted certain research on the representation and feature discovery of sound velocity profiles, but have not yet involved the use of AE autoencoders and related deep learning methods to apply AE autoencoders, a deep learning neural network, to ocean sound velocity profile data compression and reconstruction. Only the compressed data and the weight parameters in the neural network need to be stored to restore the compressed data to the original data with high precision. There is no need to store the information of the sound velocity profile data itself, which significantly reduces the amount of data that needs to be stored. Summary of the invention

[0004] The purpose of the invention is to provide a high-resolution sound velocity profile data compression method based on AE autoencoder. The method is based on the high-resolution, long-time series sound velocity profile data obtained in the high-resolution ocean reanalysis product. Under the premise of ensuring the reconstruction accuracy of the compressed data, the AE autoencoder (Auto-Encoder) in the deep learning technology is used to build an AE autoencoder sound velocity profile compression and reconstruction network, and feature encoding is performed on the high-resolution sound velocity profile data of the wide sea area, which significantly compresses the data volume of the high-resolution sound velocity profile of the ocean, and can reconstruct the data with high precision. The precision of data reconstruction can meet the needs of most hydroacoustic engineering application fields, and provide a technical basis for the application of high-resolution ocean reanalysis products in the field of marine acoustics.

[0005] To achieve the above purpose, the present invention adopts the following technical solution.

[0006] A high-resolution sound velocity profile data compression method based on AE autoencoder comprises the following steps:

[0007] Step 1: Select the sea area location and time span of the target compressed data, and use the sound speed empirical formula to convert the seawater temperature, salinity, and depth data in the high-resolution ocean reanalysis product into a high-resolution sound speed profile data set;

[0008] Step 2: Use the sliding interpolation method to complete the missing data in the high-resolution sound velocity profile dataset, normalize the data, and then divide the normalized dataset into a training set, a test set, and a validation set for training and testing the AE autoencoder sound velocity profile compression and reconstruction network;

[0009] Step 3: Build an AE autoencoder sound velocity profile compression and reconstruction network, use the training set and the validation set to iteratively train the network, calculate the compression performance evaluation index in real time during the training and validation process, and monitor the compression effect of the model;

[0010] Step 4: Use the test set to perform line-by-line and overall compression tests on the trained AE autoencoder sound speed profile compression and reconstruction network to evaluate the compression and reconstruction performance; then compress the entire sound speed profile data set line-by-line and overall to obtain the corresponding two types of compressed data sets.

[0011] The specific steps of step 1 are as follows:

[0012] Step 1.1, select the sea area location and time span of the target compressed data, refer to the data storage structure of the high-resolution ocean reanalysis product, and store the sound velocity profile dataset in the same structure; the horizontal spatial resolution, vertical spatial resolution, time resolution, and time span of the storage structure of the sound velocity profile dataset are consistent with those of the reference high-resolution ocean reanalysis product;

[0013] Step 1.2, using the sound speed empirical formula, convert the seawater temperature, salinity, depth, and data in the high-resolution ocean reanalysis product into high-resolution sound speed. The sound speed empirical formula is as follows:

[0014] c(s,T,p)=c ω (T,p)+a(T,p)s+b(T,p)s 3 / 2 +d(T,p)s 2 ;

[0015] Where: c is the sound velocity of seawater; s, T, and p are the salinity, water temperature, and static pressure of seawater, respectively. The requirements are 0≤s≤40, 0°≤T≤40°, 0Pa≤p≤10 8 Pa;c ω , a, b, and d are empirical functions related to seawater temperature and static pressure; the standard deviation of the calculated sound speed value is 0.19 m / s; the storage structure of the converted sound speed profile data set is consistent with the data storage structure of the reanalysis product, and the sound speed value in the storage cell is a floating point type.

[0016] The specific steps of step 2 are as follows:

[0017] Step 2.1, using the sliding interpolation method, traverse the sound speed profiles in the sound speed profile data set one by one, and fill the missing values ​​with the previous non-missing values ​​in the same sound speed profile data sequence; and extend the sound speed profile to the seabed sediment layer to obtain a sound speed profile data set SSP with a uniform depth layer number D; the data set contains a total of I×J sound speed profile samples, I represents the total number of longitude and latitude grids of the sea area covered by the target compressed data, and J represents the total number of time nodes for outputting sound speed profile data in the target compressed data time period; the data set SSP is represented as a two-dimensional matrix with a matrix structure of I×J rows and D columns;

[0018] Step 2.2, divide the sound velocity profile data set S of uniform depth into a training set T, a validation set V, and a test set S according to a certain ratio as the training data set of the AE autoencoder; according to the size of the iterative batch processing amount (batch_size), divide the training set T and the validation set V equally to obtain a batch training set A and a batch validation set B, each batch of which contains M=batch_size sound velocity profile samples. The batch training set A and the batch validation set B are both expressed in matrix form, and the matrix structure is M rows and D columns;

[0019] The test set is divided into equal parts, and a batch test set C is obtained, each batch of which contains only one sound velocity profile sample, to test the model's performance in compressing the data set one by one. At the same time, the complete test set S is retained to test the model's performance in compressing the data set as a whole.

[0020] The specific steps of step 3 are as follows:

[0021] Step 3.1, build the AE autoencoder sound velocity profile compression and reconstruction network structure, which consists of four parts: normalization layer, encoder network, decoder network, and denormalization layer;

[0022] Taking the training of a batch training set A as an example, the complete structure of the AE autoencoder sound velocity profile compression and reconstruction network is as follows:

[0023] The first part is the normalization layer, which inputs the batch training set A and outputs the normalized batch training set A. norm , using 0-1 normalization, the formula is as follows:

[0024]

[0025] Among them, A max is the maximum sound speed in A, A min To minimize the speed of sound, the above 0-1 normalization process can be implemented by the MinMaxScaler() function in Python's preprocessing library;

[0026] The second part is the encoder network, which converts A norm Input encoder network, the output is the feature encoding dataset Z obtained by compressing the batch training set A , dimensions E, Z A It is also the input of the decoder network; when passing through the encoder network, θ1 is the parameter of the encoder network, and the encoding formula is:

[0027] Z A =f(A norm ;θ1);

[0028] The third part is the decoder network, which converts Z A Input decoder network, the output of decoder network is the reconstructed normalized batch training set θ2 is the parameter of the decoder network, and the decoding formula is:

[0029]

[0030] The fourth part is the denormalization layer, which reconstructs the normalized batch training set Perform denormalization to obtain the reconstructed batch training set The anti-normalization formula is:

[0031]

[0032] Among them, the encoder network is composed of three fully connected layers with dimensions D, 16, and E connected in sequence, and the decoder network is composed of three fully connected layers with dimensions E, 16, and D connected in sequence; the Tanh hyperbolic tangent function is used as the activation function after each fully connected layer, x represents the variable of the input activation function, and the expression of the Tanh activation function is as follows:

[0033]

[0034] In network training, for the M sound velocity profile samples in the batch training set, the normalized batch training set reconstructed by the autoencoder is used. The error of is used as the loss function of, and the loss function is:

[0035]

[0036] The Adam optimizer is used to optimize the model by backpropagating the loss. The optimization goal is to minimize the loss function, that is, to make the decoder reconstruct the data to restore the input encoder data to the maximum extent.

[0037] Step 3.2, train the AE autoencoder sound velocity profile compression and reconstruction network. Each time the training set is iterated, the validation set is iterated immediately. However, it should be noted that the loss of the validation set does not participate in the back propagation and is not used to optimize the model. Instead, it is used to observe the changes in the loss of the validation machine, observe whether the model is overfitting, and adjust the three hyperparameters of the iterative batch size (batch_size), learning rate (learning_rate), and number of iterations (epochs) to achieve better training results.

[0038] After each batch training, the compression efficiency and reconstruction accuracy are introduced to evaluate the compression performance of the model. The compression performance of the model is monitored in real time, and the hyperparameters are adjusted in time to obtain better data compression and reconstruction effects. The compression ratio CR (Compression Ratio) is used to evaluate the compression efficiency of the data. The correlation coefficient CC (CrossCorrelation), root mean square error RMSE (Root Mean Square Error), root mean square percentage error PRD (PercentRoot mean square Diference), mean absolute error MAE (Mean Absolute Error), and mean relative error MRE (Mean Relative Error) are used to evaluate the recovery accuracy of the reconstructed data.

[0039] The calculation expression of compression ratio CR is as follows:

[0040]

[0041] Since both the compressed data set Z and the original data set SSP can be expressed in matrix form, the Z matrix structure is I×J rows and E columns, and the original data set S structure is I×J rows and D columns, the simple compression ratio CR expression used is as follows:

[0042]

[0043] The calculation expression of the correlation coefficient CC is as follows:

[0044]

[0045] Where N represents the total number of sound speed values ​​in the input data set, u(i) is the sound speed value of the i-th point in the input data set, and v(i) is the sound speed value of the i-th point in the reconstructed data set. is the average sound speed value of the input data set, is the average sound velocity value of the reconstructed data set, is the standard deviation of the input data set, is the standard deviation of the output data set; the result of CC is in the range of [-1,1]. The closer it is to 1, the higher the recovery accuracy of the reconstructed data set. CC is dimensionless and can directly represent the similarity between data sets.

[0046] The expression of root mean square error RMSE is as follows:

[0047]

[0048] It is used to represent the difference between the original data set and the reconstructed data set. However, RMSE is more dependent on the size of the sound speed data in the data set. Under the same RMSE, the sound speed value with a larger value should be more accurate. Therefore, it is necessary to introduce the concept of root mean square percentage error (PRD).

[0049] The expression of the root mean square percentage error PRD is as follows:

[0050]

[0051] The expression of mean absolute error MAE is as follows:

[0052]

[0053] The expression of mean relative error MRE is as follows:

[0054]

[0055] The specific steps of step 4 are as follows:

[0056] Step 4.1, after completing the iterative training, use the test set to test the generalization ability of the AE autoencoder sound speed profile compression and reconstruction network; input the batch test set C and the complete test set S into the model respectively, test the compression performance of the model on each sound speed profile and the complete sound speed profile data set, and use relevant indicators to evaluate the performance;

[0057] Step 4.2, input the complete sound speed profile data set SSP into the AE autoencoder sound speed profile compression and reconstruction network, perform line-by-line compression and overall compression respectively, obtain the corresponding two types of compressed data sets, and output the compressed data set and the reconstructed data set;

[0058] Step 4.3, evaluate the compression performance of the entire sound velocity profile data set SSP when compressing it line by line and compressing it as a whole;

[0059] The complete sound speed profile dataset SSP is input into the AE autoencoder sound speed profile compression and reconstruction network, and is compressed line by line and as a whole respectively to obtain the corresponding two types of compressed datasets.

[0060] The invention provides a high-resolution sound velocity profile compression and reconstruction method based on AE autoencoder, which has the following beneficial effects:

[0061] 1. Fully compress the data volume of high-resolution sound velocity profile data sets. Under the premise of ensuring compression accuracy, when storing high-resolution sound velocity profile data sets in the data grid unit format of the high-resolution ocean reanalysis product CORAv2.0, the compression ratio of high-resolution sound velocity profile data sets can usually reach 25 times, which is better than the general lossy compression method. By significantly reducing the data volume, the software and hardware costs of data storage are reduced, which provides convenience for the popularization and application of massive high-resolution sound velocity profile data;

[0062] 2. On the basis of fully compressing the sound velocity profile data, the sound velocity profile data can be reconstructed quickly and with high precision. The average relative error after reconstruction using the compressed sound velocity profile data one by one is less than 0.02%, and the accuracy after reconstruction using the overall compressed data is less than 0.12%. The above accuracy can meet the accuracy requirements of sound velocity profile reconstruction for hydroacoustic engineering applications;

[0063] 3. The compression and reconstruction network structure of the AE autoencoder sound velocity profile is simple and has fewer layers. The training and use of this model have low requirements on the performance of computing equipment. It can realize the compression of high-resolution sound velocity profile data and the storage of compressed data on personal laptops. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is an overall flow chart of an embodiment of the present invention;

[0065] Figure 2 This is the network structure diagram of AE autoencoder sound velocity profile compression and reconstruction;

[0066] Figure 3 is a line graph of loss during the model training process in an embodiment of the present invention;

[0067] Figure 4 It is a line graph of mean absolute error (MAE) during model training in an embodiment of the present invention;

[0068] Figure 5 is a line graph of mean relative error (MRE) during model training in an embodiment of the present invention;

[0069] Figure 6 is a line graph of the root mean square error RMSE during the model training process in an embodiment of the present invention;

[0070] Figure 7 is a line graph of the root mean square percentage error (PRD) during the model training process in an embodiment of the present invention;

[0071] Figure 8 It is a line graph of the correlation coefficient CC during the model training process in an embodiment of the present invention;

[0072] Fig. 9 is a schematic diagram of a high-resolution sound velocity profile data set SSP completed in an embodiment of the present invention;

[0073] Fig.10 is a schematic diagram of a data set obtained by overall compressing the entire SSP in an embodiment of the present invention;

[0074] Fig.11 is a schematic diagram of a data set obtained by overall compression and reconstruction of the entire SSP in an embodiment of the present invention; DETAILED DESCRIPTION

[0075] The present invention is described in detail below in conjunction with specific embodiments.

[0076] In order to make the technical problems, technical solutions and beneficial effects solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0077] The present invention provides a high-resolution sound velocity profile data compression method based on AE autoencoder. Figure 1 Provided is a flow chart of the present invention, Figure 2 The model network structure of the AE autoencoder high-resolution sound velocity profile compression and reconstruction method is given, which specifically includes the following steps:

[0078] Step 1: Select the sea area location and time span of the target compressed data, and use the sound speed empirical formula to convert the seawater temperature, salinity, and depth data in the high-resolution ocean reanalysis product into a high-resolution sound speed profile data set;

[0079] Step 2: Use the sliding interpolation method to complete the missing data in the high-resolution sound velocity profile dataset, normalize the data, and then divide the normalized dataset into a training set, a test set, and a validation set according to a certain ratio for training and testing the AE autoencoder sound velocity profile compression and reconstruction network;

[0080] Step 3: Build an AE autoencoder sound velocity profile compression and reconstruction network, use the training set and the validation set to iteratively train the network, calculate the compression performance evaluation index in real time during the training and validation process, and monitor the compression effect of the model;

[0081] Step 4: Use the test set to perform line-by-line and overall compression tests on the trained AE autoencoder sound speed profile compression and reconstruction network to evaluate the compression and reconstruction performance; then compress the entire sound speed profile data set line-by-line and overall to obtain the corresponding two types of compressed data sets.

[0082] The specific steps of step 1 are as follows:

[0083] Step 1.1, obtain high-resolution ocean reanalysis product dataset;

[0084] The high-resolution ocean reanalysis product used in this embodiment is the CORAv2.0 (China Ocean Reanalysis) reanalysis product. The data of this product comes from the National Marine Information Center. The data website is http: / / mds.nmdis.org.cn. The seawater temperature, salinity, and pressure physical quantity data sets required in the embodiment are downloaded from this website; the ocean reanalysis product uses data assimilation technology to combine historical ocean observation data with ocean dynamic models to reconstruct the historical changes of the complete ocean state field, and obtain a data set with global coverage and full elements, which provides important basic information for marine environmental data. In addition to CORAv2.0, there are many other high-resolution ocean reanalysis products. The scope of application of the present invention is not limited to the CORAv2.0 dataset; it is also applicable to other types of high-resolution ocean reanalysis products, such as the three-dimensional temperature-salinity flow reanalysis product of the ECMWF ocean reanalysis of the European Center for Medium-Range Weather Forecasts, the United States Ocean Circulation and Climate Estimates (ECCO, Estimation of the Circulation and Climate of the Ocean), Simple Ocean Data Assimilation (SODA, Simple Ocean Data Assimilation), the United States Navy Hybrid Coordinate Ocean Model (HYCOM, HYbrid coordinate ocean model) and other products;

[0085] Step 1.2, referring to the grid unit structure of the high-resolution ocean reanalysis product CORAv2.0 dataset, design the grid structure for storing high-resolution sound velocity profile data;

[0086] The grid unit structure of the high-resolution ocean reanalysis product CORAv2.0 dataset used in this embodiment is as follows: the longitude and latitude within the sea area are divided into equally spaced grids (resolution is 0.1°); the time resolution is 3 hours; the depth is divided into non-uniformly spaced grids (a total of 50 layers), and the default depth layers in the CORAv2.0 dataset are [-5, -15, -25, -35, -45, -55, -65, -75.004997, -85.025002, -95 .095001, -105.31000, -115.87000, -127.15000, -139.74001, -154.47000, -172.39999, -194.73500, -222.71001, -257.47000, -299.92999, -350.67999, -409.92999, -477.47000, -552.71002, -634.73499, - 722.40002, -814.46997, -909.73999, -1007.1550, -1105.9050, -1205.5350, -1306.2050, -1409.1500, -1517.0950, -1634.1750, -1765.1350, -1914.1500, -2084.0349, -2276.2251, -2491.2500, -2729.250 0, -2990.2500, -3274.2500, -3581.2500, -3911.2500, -4264.2500, -4640.2500, -5039.2500, -5461.2500, -5906.2500], unit is (meter); Since the temperature, salinity and other element data provided by the high-resolution ocean reanalysis products are physical quantities of ocean water bodies, the values ​​at the land and seabed sediment layers are null (default is NAN);

[0087] Step 1.3, select the scope of the target compressed sea area;

[0088] The longitude range of the target compressed sea area is [175.95°E, 179.95°E], the latitude range is [40.95°N, 44.95°N], the horizontal spatial resolution is 0.1°×0.1°, and the total number of longitude and latitude grids is I=160; the time span is the whole year of 2014, the time resolution is output once every 3 hours, and the total number of data output time points is J=2920; the vertical spatial resolution is D=50, and the specific depth stratification is as described in step 1.2;

[0089] Step 1.4, using the sound speed empirical formula, convert the seawater temperature, salinity, depth, and data in the high-resolution ocean reanalysis product into high-resolution sound speed profile data;

[0090] The empirical formula for the speed of sound is as follows:

[0091] c(s,T,p)=c ω (T,p)+a(T,p)s+b(T,p)s 3 / 2 +d(T,p)s 2 ;

[0092] Where: c is the sound velocity of seawater; s, T, and p are the salinity, water temperature, and static pressure of seawater, respectively. The requirements are 0≤s≤40, 0°≤T≤40°, 0Pa≤p≤10 8 Pa;c ω , a, b, d are empirical functions related to seawater temperature and static pressure; the standard deviation of the sound velocity calculation value is 0.19m / s, which means that the sound velocity value converted by this formula is close to the actual sound velocity value, and the conversion result can meet the needs of the application scenario;

[0093] The storage structure of the converted high-resolution sound velocity profile data set is consistent with the above grid structure, and the sound velocity value in the storage cell is a floating point number type; the specific meanings of the above parameters can be found in "Precise equation of state of seawater for oceanic ranges of salinity, temperature and pressure (CT Chen, FJ Millero. 1977)", "Speed ​​of sound inseawater at high pressures (Chen, Chen-Tung. 1977)", and "Marine Physics (Ye Anle, Li Fengqi. 1992)".

[0094] The specific steps of step 2 are as follows:

[0095] Step 2.1, using the sliding interpolation method, fill in the missing values ​​of the high-resolution sound velocity profile data set and the null values ​​at the seafloor sediments, so that each sound velocity profile sample has corresponding sound velocity values ​​at 50 depth layers, and obtain the completed high-resolution sound velocity profile data set SSP;

[0096] The sound velocity profile samples in the high-resolution sound velocity profile data set are traversed one by one, and the missing values ​​in the sound velocity profile samples are filled with the previous non-missing values ​​in the same sound velocity profile sample; and the sound velocity profile is extended to the seabed sediment layer, so that each sound velocity profile sample has corresponding sound velocity values ​​at 50 depth layers; the completed high-resolution sound velocity profile data set SSP is obtained, which contains a total of I×J=467200 sound velocity profile samples; the data set SSP is represented as a two-dimensional matrix with a matrix structure of 467200 rows and 50 columns;

[0097] Step 2.2, divide the completed sound speed profile data set SSP to obtain the data set required for training and testing the AE autoencoder sound speed profile compression and reconstruction network;

[0098] The completed sound speed profile dataset SSP is divided into a training set T, a validation set V, and a test set S in a ratio of 8:1:1, which are used to train and test the AE autoencoder sound speed profile compression and reconstruction network; the training set T and the validation set V are equally divided according to the size of the iterative batch processing amount, each containing M sound speed profile samples, and a single batch is referred to as a batch training set A and a batch validation set B. The above batch datasets are all represented in matrix form. As mentioned above, the vertical spatial resolution of the dataset is D=50, so the above matrix structure is M rows and 50 columns; the test set is equally divided line by line to obtain a batch test set C containing only one sound speed profile sample in each batch, and the performance of the model on the compression of the dataset line by line is tested; at the same time, the complete test set S is retained to test the performance of the model on the overall compression of the dataset;

[0099] The specific steps of step 3 are as follows:

[0100] Step 3.1, build the AE autoencoder sound velocity profile compression and reconstruction network structure;

[0101] The network consists of four parts: normalization layer, encoding network, decoder network, and denormalization layer; the AE autoencoder sound velocity profile compression and reconstruction network structure built in this example is shown in the attached Figure 2 ;

[0102] Taking the training of a batch training set A as an example, the complete structure of the AE autoencoder sound velocity profile compression and reconstruction network is as follows:

[0103] The first part is the normalization layer, which inputs the batch training set A and outputs the normalized batch training set A. norm , using 0-1 normalization, the formula is as follows:

[0104]

[0105] Among them, A max is the maximum sound speed in A, A minTo minimize the speed of sound, the above 0-1 normalization process can be implemented by the MinMaxScaler() function in Python's preprocessing library;

[0106] The second part is the encoder network, which converts A norm Input encoder network, the output is the feature encoding dataset Z obtained by compressing the batch training set A , dimensions are E, Z A It is also the input of the decoder network; when passing through the encoder network, θ1 is the parameter of the encoder network, and the encoding formula is:

[0107] Z A =f(A norm ;θ1);

[0108] The third part is the decoder network, which converts Z A Input decoder network, the output of decoder network is the reconstructed normalized batch training set θ2 is the parameter of the decoder network, and the decoding formula is:

[0109]

[0110] The fourth part is the denormalization layer, which reconstructs the normalized batch training set Perform denormalization to obtain the reconstructed batch training set The anti-normalization formula is:

[0111]

[0112] Among them, the vertical spatial resolution is D = 50, so the encoder network is composed of three fully connected layers of dimensions 50, 16, and E connected in sequence, and the decoder network is composed of three fully connected layers of dimensions E, 16, and 50 connected in sequence. The two share a fully connected layer of dimension E, which is also called a feature layer, that is, compressed data is obtained after passing through this layer; after each fully connected layer, the Tanh hyperbolic tangent function is used as the activation function, x represents the variable of the input activation function, and the expression of the Tanh activation function is as follows:

[0113]

[0114] In network training, for the M sound velocity profile samples in the batch training set, the normalized batch training set reconstructed by the autoencoder is used. The error of is used as the loss function of, and the loss function is:

[0115]

[0116] The Adam optimizer is used to optimize the model by backpropagating the loss. The optimization goal is to minimize the loss function, that is, to make the decoder reconstruct the data to restore the input encoder data to the maximum extent.

[0117] This step is to iteratively train the AE autoencoder sound velocity profile compression and reconstruction network constructed above using the training set and the validation set; the AE autoencoder sound velocity profile compression and reconstruction network has a simple structure and a small number of layers. The training and use of this model have low requirements on the performance of the computing device and can realize the compression of high-resolution sound velocity profile data and the storage of compressed data on a personal laptop computer;

[0118] Step 3.2, training the AE autoencoder sound velocity profile compression and reconstruction network;

[0119] The learning rate is set to 0.0001, the batch size of the iteration is set to 512, that is, M=512, and the number of iterations is set to 5; the weights of each layer of the network are initialized using Xavier normal distribution, the optimizer is Adam, and the loss function is MSE mean square error; the compressed dimensionality reduction feature encoding dimension is initialized to 2, that is, E=2; during training, each iteration of the training set is followed by an iteration of the validation set, in order to adjust the hyperparameters in a timely manner;

[0120] In addition to the loss output during model training, five indicators, including compression ratio CR, correlation coefficient CC, root mean square error RMSE, root mean square percentage error PRD, mean absolute error MAE, and mean relative error MRE, are calculated and output synchronously to evaluate the changes in compression performance; the model converges after about 2 training cycles; during the entire training process, the training set loss line graph can be found in Figure 3 , see the compression performance index change line chart Figures 4 to 8 ; The loss value and compression performance index value of the first batch of training set and validation set in each iteration are shown in Table 1 and Table 2;

[0121] Table 1 Loss value and compression performance index value of the first batch of training set in each iteration (5 iterations in total)

[0122] Epoch Loss CR CC RMSE PRD MAE MRE 1 0.396 25 0.826 18.322 1.272％ 12.493 0.840％ 2 0.019 25 0.991 2.719 0.182％ 1.819 0.124％ 3 0.019 25 0.992 2.621 0.175％ 1.8136 0.121％ 4 0.018 25 0.992 2.513 0.168％ 1.7157 0.113％ 5 0.016 25 0.994 2.237 0.149％ 1.4952 0.100％

[0123] Table 2 Loss value and compression performance index value of the first batch of validation set in each iteration (5 iterations in total)

[0124] Epoch Loss CR CC RMSE PRD MAE MRE 1 0.014 25 0.992 2.484 0.166％ 1.567 0.116％ 2 0.016 25 0.993 2.417 0.162％ 1.585 0.114％ 3 0.016 25 0.993 2.352 0.157％ 1.529 0.108％ 4 0.016 25 0.993 2.453 0.164％ 1.585 0.112％ 5 0.017 25 0.993 2.459 0.165％ 1.663 0.111％

[0125] Table 1 and Table 2 use compression ratio CR (Compression Ratio) to evaluate the compression efficiency of data respectively; four indicators including correlation coefficient CC (Cross Correlation), root mean square error RMSE (Root Mean Square Error), root mean square percentage error PRD (Percent Root mean square Diference), mean absolute error MAE (Mean Absolute Error), and mean relative error MRE (Mean Relative Error) are used to evaluate the recovery accuracy of reconstructed data;

[0126] The calculation expression of compression ratio CR is as follows:

[0127]

[0128] Since both the compressed data set Z and the original data set S can be expressed in matrix form, the Z matrix structure is I×J rows and E columns, and the original data set SSP structure is I×J rows and D columns, the simple compression ratio CR expression used is as follows:

[0129]

[0130] The calculation expression of the correlation coefficient CC is as follows:

[0131]

[0132] Where N represents the total number of sound speed values ​​in the input data set, u(i) is the sound speed value of the i-th point in the input data set, and v(i) is the sound speed value of the i-th point in the reconstructed data set. is the average sound speed value of the input data set, is the average sound velocity value of the reconstructed data set, is the standard deviation of the input data set, is the standard deviation of the output data set; the result of CC is in the range of [-1,1]. The closer it is to 1, the higher the recovery accuracy of the reconstructed data set. CC is dimensionless and can directly represent the similarity between data sets.

[0133] The expression of root mean square error RMSE is as follows:

[0134]

[0135] It is used to represent the difference between the original data set and the reconstructed data set. However, RMSE is more dependent on the size of the sound speed data in the data set. Under the same RMSE, the sound speed value with a larger value should be more accurate. Therefore, it is necessary to introduce the concept of root mean square percentage error (PRD).

[0136] The expression of the root mean square percentage error PRD is as follows:

[0137]

[0138] The expression of mean absolute error MAE is as follows:

[0139]

[0140] The expression of mean relative error MRE is as follows:

[0141]

[0142] The present invention can fully compress the data volume of a high-resolution sound velocity profile data set while ensuring compression accuracy. When the high-resolution sound velocity profile data set is stored in the data grid unit format of the high-resolution ocean reanalysis product CORAv2.0, the compression ratio of the high-resolution sound velocity profile data set can reach 25 times, which is better than the general lossy compression method. By significantly reducing the amount of data, the software and hardware costs of data storage are reduced, which provides convenience for the popularization and application of massive high-resolution sound velocity profile data.

[0143] The specific steps of step 4 are as follows:

[0144] Step 4.1, after completing the above five iterations of training, use the test set to test the AE autoencoder sound velocity profile compression and reconstruction network;

[0145] The batch test set C (each batch contains only one sound velocity profile sample) and the complete test set S are input into the model respectively, and the compression performance of the model on each sound velocity profile and the complete sound velocity profile data set is tested, and the performance is evaluated using relevant indicators; the comparison of the compression performance of the two is shown in Table 3;

[0146] The experimental results show that the compression ratio of the two types of test sets is 25 times. The average relative error of the reconstructed data compressed item by item is less than 0.02% compared with the original data, and the average relative error of the reconstructed data compressed as a whole is less than 0.12% compared with the original data. Both compression methods can meet most of the accuracy requirements of the reconstruction of the sound velocity profile, and the one-by-one compression method performs better in various compression performance evaluation indicators.

[0147] Table 3 The average values ​​of compression loss and compression performance index for each test set and the average values ​​of compression loss and compression performance index for the entire test set

[0148] Compression method CR CC RMSE PRD MAE MRE Compression line by line 25 0.999 0.325 0.022％ 0.279 0.018％ Overall compression 25 0.993 2.456 0.164％ 1.716 0.115％

[0149] Step 4.2, input the complete sound speed profile data set SSP into the AE autoencoder sound speed profile compression and reconstruction network, perform line-by-line compression and overall compression respectively, and obtain the corresponding two types of compressed data sets; output the compressed data set and the reconstructed data set;

[0150] Step 4.3, when compressing the entire sound velocity profile data set SSP line by line and when compressing the entire data set SSP as a whole, the compression performance is evaluated, see Table 4;

[0151] Table 4 The average values ​​of the loss and compression performance index of the entire sound velocity profile data set for compression line by line, and the loss and compression performance index value of the overall compression of the sound velocity profile data set

[0152] Compression method CR CC RMSE PRD MAE MRE Compression line by line 25 0.999 0.325 0.022％ 0.275 0.018％ Overall compression 25 0.993 2.459 0.165％ 1.718 0.115％

[0153] It can be seen that the average relative error of the reconstructed data compressed line by line is less than 0.02% compared with the original data, and the average relative error of the reconstructed data compressed as a whole is less than 0.12% compared with the original data; in general, both compression methods can meet most of the accuracy requirements of the sound velocity profile reconstruction, and the line by line compression method performs better in various compression performance evaluation indicators;

[0154] The three datasets, namely, the completed high-resolution sound velocity profile dataset SSP, the dataset obtained by overall compression of the entire SSP, and the dataset obtained by overall compression and reconstruction of the entire SSP, are plotted as shown in the attached figure. Figures 9-11 ; It can be seen that the reconstructed data is basically consistent with the original data, and the recovery accuracy is high.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A high-resolution sound velocity profile data compression method based on AE autoencoder, characterized in that: The steps include: Step 1: Select the sea area location and time span of the target compressed data, and use the sound speed empirical formula to convert the seawater temperature, salinity, and depth data in the high-resolution ocean reanalysis product into a high-resolution sound speed profile data set; Step 2: Use the sliding interpolation method to complete the missing data in the high-resolution sound velocity profile dataset, normalize the data, and then divide the normalized dataset into a training set, a test set, and a validation set according to a certain ratio for training and testing the AE autoencoder sound velocity profile compression and reconstruction network; Step 3: Build an AE autoencoder sound velocity profile compression and reconstruction network, use the training set and validation set to iteratively train the network, calculate the compression performance evaluation index in real time during the training and validation process, and monitor the compression effect of the model; the specific steps are as follows: Step 3.1, build the AE autoencoder sound velocity profile compression and reconstruction network structure, which consists of four parts: normalization layer, encoder network, decoder network, and anti-normalization layer; Taking the training of a batch training set A as an example, the complete structure of the AE autoencoder sound velocity profile compression and reconstruction network is as follows: The first part is the normalization layer, which inputs the batch training set A and outputs the normalized batch training set A. norm , using 0-1 normalization, the formula is as follows: Among them, A max is the maximum sound speed in A, A min To minimize the sound speed, the above 0-1 normalization process is implemented by the MinMaxScaler() function in Python's preprocessing library; The second part is the encoder network, which converts A norm Input encoder network, the output is the feature encoding dataset Z obtained by compressing the batch training set A , dimensions are E, Z A It is also the input of the decoder network; when passing through the encoder network, θ1 is the parameter of the encoder network, and the encoding formula is: Z A =f(A norm ;θ1); The third part is the decoder network, which converts Z A Input decoder network, the output of decoder network is the reconstructed normalized batch training set θ2 is the parameter of the decoder network, and the decoding formula is: The fourth part is the denormalization layer, which reconstructs the normalized batch training set Perform denormalization to obtain the reconstructed batch training set The anti-normalization formula is: Among them, the encoder network is composed of three fully connected layers with dimensions D, 16, and E connected in sequence, and the decoder network is composed of three fully connected layers with dimensions E, 16, and D connected in sequence; the Tanh hyperbolic tangent function is used as the activation function after each fully connected layer, x represents the variable of the input activation function, and the expression of the Tanh activation function is as follows: In network training, for the M sound velocity profile samples in the batch training set, the normalized batch training set reconstructed by the autoencoder is used. The error of is used as the loss function of, and the loss function is: The Adam optimizer is used to optimize the model by backpropagating the loss. The optimization goal is to minimize the loss function, that is, to make the decoder reconstruct the data to restore the input encoder data to the maximum extent. Step 3.2, train the AE autoencoder sound velocity profile compression and reconstruction network. Each time the training set is iterated, the validation set is iterated immediately. However, it should be noted that the loss of the validation set does not participate in the back propagation and is not used to optimize the model. Instead, it is used to observe the changes in the loss of the validation machine, observe whether the model is overfitting, and adjust the three hyperparameters of the iterative batch size (batch_size), learning rate (learning_rate), and number of iterations (epochs) to achieve better training results. After each batch training, the compression efficiency and reconstruction accuracy are introduced to evaluate the compression performance of the model. The compression performance of the model is monitored in real time, and the hyperparameters are adjusted in time to obtain better data compression and reconstruction effects. The compression ratio CR (Compression Ratio) is used to evaluate the compression efficiency of the data. The correlation coefficient CC (CrossCorrelation), root mean square error RMSE (Root Mean Square Error), root mean square percentage error PRD (PercentRoot mean square Diference), mean absolute error MAE (Mean Absolute Error), and mean relative error MRE (Mean Relative Error) are used to evaluate the recovery accuracy of the reconstructed data. The calculation expression of compression ratio CR is as follows: Since the compressed data set Z and the original data set SSP are both represented in matrix form, the Z matrix structure is I×J rows and E columns, and the original data set S structure is I×J rows and D columns, the simple compression ratio CR is expressed as follows: The calculation expression of the correlation coefficient CC is as follows: Where N represents the total number of sound speed values ​​in the input data set, u(i) is the sound speed value of the i-th point in the input data set, and v(i) is the sound speed value of the i-th point in the reconstructed data set. is the average sound speed value of the input data set, is the average sound velocity value of the reconstructed data set, σ u = is the standard deviation of the input data set, is the standard deviation of the output data set; the result of CC is in the range of [-1,1]. The closer it is to 1, the higher the recovery accuracy of the reconstructed data set. CC is dimensionless and directly indicates the similarity between data sets. The expression of root mean square error RMSE is as follows: It is used to represent the difference between the original data set and the reconstructed data set. However, RMSE is more dependent on the size of the sound speed data in the data set. Under the same RMSE, the sound speed value with a larger value should be more accurate. Therefore, it is necessary to introduce the concept of root mean square percentage error (PRD). The expression of the root mean square percentage error PRD is as follows: The expression of mean absolute error MAE is as follows: The expression of mean relative error MRE is as follows: Step 4: Use the test set to perform line-by-line and overall compression tests on the trained AE autoencoder sound speed profile compression and reconstruction network to evaluate the compression and reconstruction performance; then compress the entire sound speed profile data set line-by-line and overall to obtain the corresponding two types of compressed data sets.

2. The high-resolution sound velocity profile data compression method based on AE autoencoder according to claim 1 is characterized in that: The specific steps of step 1 are as follows: Step 1.1, select the sea area location and time span of the target compressed data, refer to the data storage structure of the high-resolution ocean reanalysis product, and store the sound velocity profile dataset in the same structure; the horizontal spatial resolution, vertical spatial resolution, time resolution, and time span of the storage structure of the sound velocity profile dataset are consistent with those of the reference high-resolution ocean reanalysis product; Step 1.2, using the sound speed empirical formula, convert the seawater temperature, salinity, depth, and data in the high-resolution ocean reanalysis product into high-resolution sound speed. The sound speed empirical formula is as follows: c(s,T,p)=c ω (T,p)+a(T,p)s+b(T,p)s 3 / 2 +d(T,p)s 2 ; Where: c is the sound velocity of seawater; s, T, and p are the salinity, water temperature, and static pressure of seawater, respectively. The requirements are 0≤s≤40, 0°≤T≤40°, 0Pa≤p≤10 8 Pa;c ω , a, b, and d are empirical functions related to seawater temperature and static pressure; the standard deviation of the calculated sound speed value is 0.19 m / s; the storage structure of the converted sound speed profile data set is consistent with the data storage structure of the reanalysis product, and the sound speed value in the storage cell is a floating point type.

3. The high-resolution sound velocity profile data compression method based on AE autoencoder according to claim 2 is characterized in that: The specific steps of step 2 are as follows: Step 2.1, using the sliding interpolation method, traverse the sound speed profiles in the sound speed profile data set one by one, and fill the missing values ​​with the previous non-missing values ​​in the same sound speed profile data sequence; and extend the sound speed profile to the seabed sediment layer to obtain a sound speed profile data set SSP with a uniform depth layer number D; the data set contains a total of I×J sound speed profile samples, I represents the total number of longitude and latitude grids of the sea area covered by the target compressed data, and J represents the total number of time nodes for outputting sound speed profile data in the target compressed data time period; the data set SSP is represented as a two-dimensional matrix with a matrix structure of I×J rows and D columns; Step 2.2, divide the sound velocity profile data set S of uniform depth into a training set T, a validation set V, and a test set S according to a certain ratio as the training data set of the AE autoencoder; and divide the training set T and the validation set V equally according to the size of the iterative batch processing amount batch_size, so as to obtain a batch training set A and a batch validation set B, each batch of which contains M=batch_size sound velocity profile samples. The batch training set A and the batch validation set B are both expressed in matrix form, and the matrix structure is M rows and D columns; The test set is divided into equal parts one by one to obtain a batch test set C containing only one sound velocity profile sample in each batch, to test the model's performance in compressing the data set one by one; at the same time, the complete test set S is retained to test the model's performance in compressing the data set as a whole.

4. The high-resolution sound velocity profile data compression method based on AE autoencoder according to claim 1 is characterized in that: The specific steps of step 4 are as follows: Step 4.1, after completing the iterative training, use the test set to test the generalization ability of the AE autoencoder sound speed profile compression and reconstruction network; input the batch test set C and the complete test set S into the model respectively, test the compression performance of the model on each sound speed profile and the complete sound speed profile data set, and use relevant indicators to evaluate the performance; Step 4.2, input the complete sound speed profile data set SSP into the AE autoencoder sound speed profile compression and reconstruction network, perform line-by-line compression and overall compression respectively, obtain the corresponding two types of compressed data sets, and output the compressed data set and the reconstructed data set; Step 4.3, evaluate the compression performance of the entire sound velocity profile data set SSP when compressing it line by line and compressing it as a whole; The complete sound speed profile dataset SSP is input into the AE autoencoder sound speed profile compression and reconstruction network, and is compressed line by line and as a whole respectively to obtain the corresponding two types of compressed datasets.

Images