Marine temperature data fusion method and system based on deep learning

Abstract

The invention discloses an ocean temperature data fusion method and system based on deep learning, and the method comprises the steps: obtaining multi-source ocean temperature data which comprises 1D temperature observation data, 2D sea surface data source temperature data and 3D lattice temperature data; preprocessing the acquired multi-source ocean temperature data; performing feature extraction and information fusion on the preprocessed 3D lattice temperature data by using a first data-level fusion sub-network f1 to obtain underwater fusion features; performing feature extraction and information fusion on the preprocessed 2D sea surface data source temperature data and the corresponding spatio-temporal information by using a second data-level fusion sub-network f2 to obtain sea surface and spatio-temporal fusion features; and performing feature extraction and global fusion on the obtained underwater fusion features and sea surface and time-space fusion features by using the feature-level fusion sub-network f3 to obtain a fusion data set. Through the method and the system, the fusion precision of the ocean temperature data is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data fusion, and more specifically, to a method and system for fusing ocean temperature data based on deep learning. Background Art

[0002] Accurate and reliable ocean data are important prerequisites and necessary conditions for ocean scientific research, global change research, and efficient offshore production operations. However, ocean observation data are discrete, with problems such as fragmentation, spatio-temporal discontinuity, and sample imbalance. To further deeply reveal and predict climate change and the evolution of the ocean environment, it is necessary to form spatio-temporally continuous gridded data and conduct scientific research on this basis, which has become a consensus in ocean scientific research.

[0003] Multi-source data fusion methods represented by data assimilation are conventional means to obtain spatio-temporally continuous ocean environment grid data. However, the currently commonly used ocean data assimilation methods are mainly based on statistical theory, with problems such as large computational complexity and low fusion accuracy, and do not consider the physical correlation between multiple elements, spatio-temporal multi-scale characteristics, etc., making it difficult to meet the increasingly refined ocean research.

[0004] Therefore, how to improve the fusion accuracy of ocean data is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a method and system for fusing ocean temperature data based on deep learning, which are used to solve at least some of the technical problems in the background art.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] First, the present invention discloses a method for fusing ocean temperature data based on deep learning, including the following steps:

[0008] Obtain multi-source ocean temperature data, where the multi-source ocean temperature data includes 1D temperature observation data, 2D sea surface data source temperature data, and 3D gridded temperature data;

[0009] Preprocess the obtained multi-source ocean temperature data;

[0010] Use the first data-level fusion sub-network f1 to perform feature extraction and information fusion on the preprocessed 3D gridded temperature data to obtain underwater fusion features;

[0011] Use the second data-level fusion sub-network f2 to perform feature extraction and information fusion on the preprocessed 2D sea surface data source temperature data and the corresponding spatio-temporal information to obtain sea surface and spatio-temporal fusion features;

[0012] The obtained underwater fusion features and sea surface and spatio-temporal fusion features are subjected to feature extraction and global fusion by using the feature-level fusion sub-network f3 to obtain a fusion data set.

[0013] Further, the input layer of the first data-level fusion sub-network f1 includes four Block blocks A, B, C, and D connected in sequence;

[0014] Among them, Block A includes a first improved global attention GAM module, a one-dimensional convolutional layer, a batch normalization layer, an activation function layer, and a second improved global attention GAM module connected in sequence;

[0015] Block B includes a first residual mapping sub-module introducing a skip connection structure, and the first residual mapping sub-module includes a first one-dimensional convolutional layer, a batch normalization layer, an activation function layer, a dropout layer, and a second one-dimensional convolutional layer stacked and connected in sequence;

[0016] Block C includes a second residual mapping sub-module introducing a skip connection structure, and the second residual mapping sub-module includes a batch normalization layer, an activation function layer, a dropout layer, and a one-dimensional convolutional layer stacked and connected in sequence;

[0017] Block D includes a batch normalization layer, an activation function layer, and a third improved global attention GAM module connected in sequence.

[0018] Further, the structures of the first improved global attention GAM module and the second improved global attention GAM module are the same, including a channel attention sub-module and a spatial attention sub-module connected in series;

[0019] The channel attention sub-module first uses a two-layer multi-dimensional perceptron module to amplify the "channel-space" cross-dimensional correlation; then, the feature dimension is restored by inverse permutation; finally, the feature weight values of each channel are obtained through the Sigmoid function;

[0020] The spatial attention sub-module first uses a one-dimensional convolutional operation based on a grouped convolution strategy to fuse the data features output by the channel attention sub-module in the spatial dimension; then, the attention weights of different spatial positions are determined through the Sigmoid function.

[0021] Furthermore, the second data-level fusion sub-network f2 uses a first BN-MLP network to achieve data-level fusion of the temperature data of the sea surface data source and the spatio-temporal information. The number of neurons in the input layer of the first BN-MLP network is the sum of the number of sea surface data source types and the length of the spatio-temporal feature vector. The hidden layer of the first BN-MLP network consists of 3 sequentially connected batch-normalized multi-dimensional perceptron modules. The number of neurons in the linear layer of each multi-dimensional perceptron module is 11, 16, and 8 respectively, and the APReLU function is used to achieve non-linear activation of the features.

[0022] Furthermore, the feature-level fusion sub-network f3 uses a second BN-MLP network to extract features from the underwater fusion features and the sea surface and spatio-temporal fusion features. The input layer of the second BN-MLP network inputs a combined vector of the underwater fusion features and the sea surface and spatio-temporal fusion features. The hidden layer of the second BN-MLP network consists of 3 sequentially connected batch-normalized multi-dimensional perceptron modules. The number of neurons in the linear layer of each multi-dimensional perceptron module is 92, 128, and 64 respectively, and the APReLU function is used to achieve non-linear activation of the features.

[0023] Secondly, the present invention also discloses a deep learning-based ocean temperature data fusion system, including a multi-source ocean temperature data acquisition module: used to acquire multi-source ocean temperature data, and the multi-source ocean temperature data includes 1D temperature observation data, 2D sea surface data source temperature data, and 3D gridded temperature data;

[0024] A data preprocessing module: used to preprocess the acquired multi-source ocean temperature data;

[0025] A first fusion feature acquisition module: uses the first data-level fusion sub-network f1 to extract features and fuse information from the preprocessed 3D gridded temperature data to obtain underwater fusion features, denoted as the first fusion feature;

[0026] A second fusion feature acquisition module: uses the second data-level fusion sub-network f2 to extract features and fuse information from the preprocessed 2D sea surface data source temperature data and the corresponding spatio-temporal information to obtain sea surface and spatio-temporal fusion features, denoted as the second fusion feature;

[0027] A fusion data set acquisition module: uses the feature-level fusion sub-network f3 to extract features and globally fuse the obtained first fusion feature and second fusion feature to obtain a fusion data set.

[0028] Preferably, the input layer of the first data-level fusion sub-network f1 includes four Block blocks A, B, C, and D connected in sequence;

[0029] Among them, Block A includes a first improved global attention GAM module, a one-dimensional convolutional layer, a batch normalization layer, an activation function layer, and a second improved global attention GAM module connected in sequence;

[0030] Block B includes a first residual mapping sub-module introducing a skip connection structure. The first residual mapping sub-module includes a first one-dimensional convolutional layer, a batch normalization layer, an activation function layer, a dropout layer, and a second one-dimensional convolutional layer stacked and connected in sequence;

[0031] Block C includes a second residual mapping sub-module introducing a skip connection structure. The second residual mapping sub-module includes a batch normalization layer, an activation function layer, a dropout layer, and a one-dimensional convolutional layer stacked and connected in sequence;

[0032] Block D includes a batch normalization layer, an activation function layer, and a third improved global attention GAM module connected in sequence.

[0033] Preferably, the first improved global attention GAM module and the second improved global attention GAM module have the same structure, including a channel attention sub-module and a spatial attention sub-module connected in series;

[0034] The channel attention sub-module first uses a two-layer multi-dimensional perceptron module to amplify the "channel-space" cross-dimensional correlation; then, uses inverse permutation to restore the feature dimension; finally, obtains the channel feature weight values through the Sigmoid function;

[0035] The spatial attention sub-module first uses a one-dimensional convolutional operation based on the grouped convolution strategy to fuse the data features output by the channel attention sub-module in the spatial dimension; then, determines the attention weights at different spatial positions through the Sigmoid function.

[0036] Preferably, the second data-level fusion sub-network f2 uses a first BN-MLP network to achieve data-level fusion of the sea surface data source temperature data and the spatio-temporal information. The number of neurons in the input layer of the first BN-MLP network is the sum of the number of sea surface data source types and the spatio-temporal feature vector length; the hidden layer of the first BN-MLP network consists of 3 sequentially connected batch-normalized multi-dimensional perceptron modules. The number of neurons in the linear layer of each multi-dimensional perceptron module is 11, 16, and 8 respectively, and the APReLU function is used to achieve non-linear activation of the features.

[0037] Preferably, the feature-level fusion sub-network f3 uses a second BN-MLP network to extract features from the underwater fusion features and the sea surface and spatio-temporal fusion features; the input layer of the second BN-MLP network inputs a combined vector of the underwater fusion features and the sea surface and spatio-temporal fusion features; the hidden layer of the second BN-MLP network consists of 3 successively connected batch-normalized multi-dimensional perceptron modules, and the number of neurons in the linear layer of each multi-dimensional perceptron module is 92, 128, and 64 respectively, and the APReLU function is used to achieve non-linear activation of the features.

[0038] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a method and system for fusing ocean temperature data based on deep learning, having the following beneficial effects:

[0039] Based on a deep learning neural network, the present invention designs an MDMN network (Marine Data MergeNet) model to fuse multi-source heterogeneous ocean temperature data such as in-situ observation data, satellite remote sensing data, and reanalysis data into a spatio-temporally continuous grid dataset. By using physics-based prior knowledge, vertical profile observations, and gradient information as objective constraints, the MDMN network model can reduce high-frequency stray signals in the vertical direction of ocean temperature data. And a global attention mechanism GAM module is added to improve the fusion performance and interpretability of the model. Compared with other fusion technologies, the fusion technology proposed by the present invention is faster than traditional data assimilation and occupies less computing resources; and the accuracy of the obtained fusion dataset is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings according to the provided drawings without creative efforts.

[0041] Figure 1 It is a schematic diagram of the overall process of the ocean temperature data fusion network based on deep learning (MDMN network model) provided by the embodiment of the present invention.

[0042] Figure 2 It is a schematic diagram of the overall structure of the MDMN network model provided by the embodiment of the present invention.

[0043] Figure 3 It is a schematic diagram of the overall structure of the improved global attention module provided by the embodiment of the present invention.

[0044] Figure 4 It is a schematic diagram of the structure of the channel attention sub-module provided by the embodiment of the present invention.

[0045] Figure 5 Schematic diagram of the spatial attention sub-module structure provided by the embodiment of the present invention. Specific implementation manner

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] Embodiment 1

[0048] The embodiment of the present invention first discloses a method for fusing ocean temperature data based on deep learning, aiming to automatically integrate 1D observation profile data, 2D sea surface data, and 3D gridded data of ocean temperature by constructing an MDMN network (Marine Data Merge Net) model, so as to generate high-resolution and high-precision ocean temperature fusion data. To overcome the difficulties brought by the characteristics of wide source, multi-dimensionality, heterogeneity, etc. of the data sources for fusion processing. The overall data processing flow of the MDMN network (MarineData Merge Net) model is as Figure 1 shown, and the network model can adapt to the inherent multi-dimensional and heterogeneous characteristics of the fusion data sources.

[0049] In this embodiment, multi-source ocean temperature data covering a total of 24 years from 1994 to 2017 is used for analyzing the model fusion results, and the effectiveness of the MDMN network model is verified based on the fusion results.

[0050] The present invention first obtains multi-source ocean temperature data, and the multi-source ocean temperature data includes 1D temperature observation data, 2D sea surface data source temperature data, and 3D gridded temperature data.

[0051] In this embodiment, the 3D gridded temperature data in the ocean temperature fusion task includes 7 sets of ECCO, SODA, ORAS5, HYCOM, CORA, CESM2, and BCC-CSM2-MR, and the 2D sea surface data source temperature data includes 5 sets of OISST, ERSST, HadISST, CCMP, and AVSIO-SLA. The 1D temperature observation data used as the label data source adopts EN4 in-situ ocean temperature observation data.

[0052] Then, preprocess the obtained multi-source ocean temperature data. Specifically, in this embodiment, the sample set is divided by time separation. Samples from 1994, 2006, and 2017 for a total of 3 years are selected as the data fusion test set; samples from the remaining years during 1994 - 2017 are used for training and validation, and are randomly divided into a training set and a validation set at a ratio of 8:2.

[0053] The model optimizer is selected as Adam optimization, the learning rate is adaptively adjusted with an initial value of 0.001, the batch size is 6144, and the number of iterations is set to 200. Model training is carried out accordingly, and finally a fusion data set is formed.

[0054] The present invention fuses the preprocessed multi-source ocean temperature data by constructing an MDMN network. The principle of this MDMN model is as follows:

[0055] As Figure 2 shown, the MDMN model constructed by the present invention is divided into three parts: f1, f2, and f3. f1 and f2 are data-level fusion sub-networks, and f3 is a feature-level fusion sub-network. The specific explanations for each part are as follows:

[0056] The input layer of the f1 sub-network includes four Block blocks: A, B, C, and D. Block A includes an improved GAM, a Conv1d layer, a BN layer, an activation function APReLU layer, and an improved GAM connected in sequence. After multiple verifications, adding an improved global attention mechanism module GAM at both the beginning and the end of the BlockA module can better capture features.

[0057] Block B and Block C introduce skip connections to improve the gradient dispersion / explosion and model degradation problems of deep neural networks. The residual mapping sub-modules in both modules are stacked by a Conv1d layer, a BN layer, an APReLU layer, and a Dropout layer (the discard probability is 0.5 for all). Among them, Block B includes a one-dimensional convolutional layer (Conv1d), a batch normalization layer (BN layer), an adaptive parametric rectified linear unit layer (APReLU layer), a dropout layer (Dropout layer), and a one-dimensional convolutional layer (Conv1d) connected in sequence, and the input and output of Block B are connected through a max pooling layer (MP layer). The number of Block C is greater than 1. As Figure 2As shown in the figure, three Block C modules are adopted. Each Block C consists of a batch normalization layer (BN layer), an adaptively parametric rectifier linear unit layer (APReLU layer), a dropout layer, a one-dimensional convolutional layer (Conv1d), a batch normalization layer (BN layer), an adaptively parametric rectifier linear unit layer (APReLU layer), a dropout layer, and a one-dimensional convolutional layer (Conv1d) connected in sequence. The input and output of Block C are connected through a max-pooling layer (MP layer).

[0058] Block D consists of a BN layer, an APRelu layer, and a GAM connected in sequence; all Conv1d layers use a convolution kernel of size 3×1 to extract features.

[0059] The meanings of the structural layers of each English name involved are as follows:

[0060] GAM: An improved attention structure, the specific description is as follows;

[0061] Conv1d: One-dimensional convolutional layer;

[0062] BN layer: Batch Normalization, representing batch normalization;

[0063] APReLU: An activation function, full name is Adaptively Parametric Rectifier Linear Unit;

[0064] Dropout layer: Dropout layer, a regularization technique used to reduce the overfitting problem of neural networks.

[0065] MP: Max-Pool, max-pooling, is a pooling method in convolutional networks. The specific implementation is to use the maximum value in the feature map area as the output of this area.

[0066] f2 uses BN-MLP to achieve data-level fusion of sea surface data information and the corresponding spatio-temporal information (time and latitude and longitude information) of the sea surface data information. The number of neurons in the input layer is the sum of the number of sea surface data source types and the length of the spatio-temporal feature vector (yr, mon, lat, lon); the hidden layer consists of 3 BN-MLP modules. The number of neurons in the linear layer in each module is 11, 16, and 8 respectively, and the APReLU function is used to achieve non-linear activation of features. Among them, BN-MLP is a batch-normalized multi-dimensional perceptron network.

[0067] The structure of the feature-level fusion sub-network f3 is similar to that of f2. It uses BN-MLP to extract features from Feature 1 and Feature 2, achieving further fusion of multi-source marine data. F3 takes the combined vector of Feature 1 and Feature 2 as input; the hidden layer consists of 3 BN-MLP modules, and the number of neurons in the linear layer of each module is 92, 128, and 64 respectively. The APReLU function is used to achieve non-linear activation of features.

[0068] The improved GAM attention mechanism module is introduced as follows:

[0069] To better extract cross-dimensional features of channels and space and enhance the model's interpretability, the present invention introduces the Global Attention Mechanism (GAM) and makes targeted improvements. To highlight the interaction of temperature profiles of multiple data sources at different depths and enable the model to capture important features in both the feature channel and vertical space dimensions simultaneously, this embodiment makes targeted improvements to GAM and redesigns the sub-module.

[0070] Figure 3 The overall process of the improved GAM model is given. It adopts a serial processing method that first passes through the Channel Attention (CA) sub-module and then through the Spatial Attention (SA) sub-module. Assume the input feature is C is the number of channels, D is the feature depth; the intermediate state and the output feature can be expressed as:

[0071]

[0072] In the formula, M C and M S are the channel attention weight and the spatial attention weight respectively, reflecting the attention degree of the model to the input features in different channels and spaces. represents the element-wise multiplication operation of arrays.

[0073] The structure of the channel attention sub-module is as Figure 4 shown. To retain information in both the channel and space dimensions, a permutation operation is used to achieve the permutation of the input feature dimensions; a two-layer MLP is used to amplify the "channel-space" cross-dimensional correlation; then, an inverse permutation is used to restore the feature dimensions; finally, the Sigmoid function is used to obtain the feature weight values of each channel. The channel attention sub-module can be expressed as:

[0074]

[0075] In the formula, f permu and respectively represent the permutation and reduction functions of the feature dimension, f mlp,2 represents the activation function of the two-layer MLP.

[0076] The spatial attention sub-module focuses on spatial features, and its structure is as Figure 5 shown. First, two Conv1d operations are used to achieve the fusion of data features in the spatial dimension; second, a grouped convolution strategy is adopted to reduce the number of parameters of Conv1d; finally, the attention weights of different spatial positions are determined through the Sigmoid function. The spatial attention mechanism sub-module can be expressed as:

[0077]

[0078] In the formula, represents the one-dimensional grouped convolution operation, and the superscript indicates that the size of the convolution kernel is 3×1.

[0079] The data processing steps of the MDMN deep learning model are as follows:

[0080] First, collect data, including three types of temperature data: 1D, 2D, and 3D. The 1D data is the in-situ temperature observation data of EN4, which is the labeled data; the 2D sea surface data sources are 5 sets in total, namely OISST, ERSST, HadISST, CCMP, and AVSIO-SLA, which are the input data; the 3D gridded data sources include 7 sets in total, namely ECCO, SODA, ORAS5, HYCOM, CORA, CESM2, and BCC-CSM2-MR, which are the input data.

[0081] Secondly, clean the ocean data and interpolate it onto the same spatial resolution grid. In this embodiment, the resolution is 0.25°×0.25° in the horizontal direction and interpolated to 23 standard layers in the vertical direction.

[0082] Input the processed 3D ocean temperature data source into the data-level fusion sub-network f1 for feature extraction and information fusion of 3D gridded data to obtain the underwater fusion feature:

[0083] Feature 1 = f1(T 3D ; θ1) (5)

[0084] T 3D is the 3D sample of sea surface temperature, and θ1 is the parameter of the f1 sub-network.

[0085] Input the 2D data source and the spatio-temporal information into the data-level fusion sub-network f2 for feature extraction and information fusion of sea surface data and spatio-temporal information (year, month, longitude, latitude) to obtain the sea surface and spatio-temporal fusion feature:

[0086] Feature 2 = f2((SSx, st; θ2)(6)

[0087] SSx is the sea surface information sample, and st is the spatio-temporal information sample.

[0088] The feature-level fusion sub-network f3 performs unified feature extraction on the fusion feature 1 and the feature 2, realizes the global fusion of the features of multi-source heterogeneous data, and obtains a fusion data set:

[0089] Fusion data set = f3(feature 1, feature 2); θ3) (7)

[0090] This fusion framework can not only realize the fusion of multi-source heterogeneous data, but also retain the information of the original data sources to the greatest extent, and can automatically extract the corresponding relationships between homogeneous data, which is beneficial to improving the accuracy of the fusion data.

[0091] In this embodiment, we finally use the root mean square error RMSE and the coefficient of determination R

[0092] to evaluate the fusion accuracy of the model. These two indicators represent the error degree between the predicted value and the actual value, and the calculation formulas are as follows respectively. 2

[0093]

[0094] Where T i merge is the fusion value of the ocean temperature, and T i obs is the actual value of the ocean temperature. represents the average value of all the label data of the ocean temperature samples. The smaller the RMSE, the better the fusion effect. The closer R 2 is to 1, the better the effect.

[0095] To intuitively reflect the model effect of this embodiment, the internationally common reanalysis data set is selected for comparison. The comparison shows that the result of the present invention is the best, with RMSE being 0.042°, and R 2 being 0.991.

[0096] Table 1 Comparison of results of different data sets

[0097] Re-analyze the dataset RMSE (°C) <![CDATA[R 2 > CORA 1.113 0.981 ORAS5 1.099 0.982 CESM2 2.174 0.929 BCC-CSM2-MR 2.46 0.909 ECCO 1.988 0.94 HYCOM 0.988 0.985 SODA 0.903 0.988 EAD 1.069 0.983 This embodiment 0.042 0.991

[0098] Embodiment 2

[0099] Based on the same inventive principle as in Embodiment 1, the present invention also discloses a deep learning-based ocean temperature data fusion system for realizing the fusion of multi-source ocean temperature data, including a multi-source ocean temperature data acquisition module: used to acquire multi-source ocean temperature data, and the multi-source ocean temperature data includes 1D temperature observation data, 2D sea surface data source temperature data, and 3D gridded temperature data; ​

[0100] Data preprocessing module: used to preprocess the acquired multi-source ocean temperature data;

[0101] First fusion feature acquisition module: Use the first data-level fusion sub-network f1 to extract features and fuse information from the preprocessed 3D gridded temperature data to obtain underwater fusion features, denoted as the first fusion feature;

[0102] Second fusion feature acquisition module: Use the second data-level fusion sub-network f2 to extract features and fuse information from the preprocessed 2D sea surface data source temperature data and the corresponding spatio-temporal information to obtain sea surface and spatio-temporal fusion features, denoted as the second fusion feature;

[0103] Fusion dataset acquisition module: Use the feature-level fusion sub-network f3 to extract features and globally fuse the obtained first fusion feature and second fusion feature to obtain a fusion dataset.

[0104] Preferably, the input layer of the first data-level fusion sub-network f1 includes four Block blocks A, B, C, and D connected in sequence;

[0105] Among them, Block A includes 1 first improved global attention GAM module, 1 one-dimensional convolutional layer, 1 batch normalization layer, 1 activation function layer, and 1 second improved global attention GAM module connected in sequence;

[0106] Block B includes a first residual mapping sub-module introducing a skip connection structure. The first residual mapping sub-module includes 1 first one-dimensional convolutional layer, 1 batch normalization layer, 1 activation function layer, 1 dropout layer, and 1 second one-dimensional convolutional layer stacked and connected in sequence;

[0107] Block C includes a second residual mapping sub-module introducing a skip connection structure. The second residual mapping sub-module includes 1 batch normalization layer, 1 activation function layer, 1 dropout layer, and 1 one-dimensional convolutional layer stacked and connected in sequence;

[0108] Block D includes 1 batch normalization layer, 1 activation function layer, and 1 third improved global attention GAM module connected in sequence.

[0109] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0110] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for fusing ocean temperature data based on deep learning, characterized in that, It includes the following steps: Obtain multi-source ocean temperature data, where the multi-source ocean temperature data includes 1D temperature observation data, 2D sea surface data source temperature data, and 3D gridded temperature data; Preprocess the obtained multi-source ocean temperature data; Use the first data-level fusion sub-network f1 to perform feature extraction and information fusion on the preprocessed 3D gridded temperature data to obtain underwater fusion features; Use the second data-level fusion sub-network f2 to perform feature extraction and information fusion on the preprocessed 2D sea surface data source temperature data and the corresponding spatio-temporal information to obtain sea surface and spatio-temporal fusion features; Use the feature-level fusion sub-network f3 to perform feature extraction and global fusion on the obtained underwater fusion features and sea surface and spatio-temporal fusion features to obtain a fusion data set.

2. The method for fusing ocean temperature data based on deep learning according to claim 1, wherein The input layer of the first data-level fusion sub-network f1 includes four Block blocks A, B, C, and D connected in sequence; Among them, Block A includes a first improved global attention GAM module, a one-dimensional convolutional layer, a batch normalization layer, an activation function layer, and a second improved global attention GAM module connected in sequence; Block B includes a first residual mapping sub-module introducing a skip connection structure, and the first residual mapping sub-module includes a first one-dimensional convolutional layer, a batch normalization layer, an activation function layer, a dropout layer, and a second one-dimensional convolutional layer stacked and connected in sequence; Block C includes a second residual mapping sub-module introducing a skip connection structure, and the second residual mapping sub-module includes a batch normalization layer, an activation function layer, a dropout layer, and a one-dimensional convolutional layer stacked and connected in sequence; Block D includes a batch normalization layer, an activation function layer, and a third improved global attention GAM module connected in sequence.

3. A method for fusing ocean temperature data based on deep learning according to claim 2, characterized in that The structures of the first improved global attention GAM module and the second improved global attention GAM module are the same, including a channel attention sub-module and a spatial attention sub-module connected in series; The channel attention sub-module first uses a two-layer multi-dimensional perceptron module to amplify the "channel-space" cross-dimensional correlation; Then, the feature dimension is restored by inverse permutation; finally, the feature weight values of each channel are obtained through the Sigmoid function; The spatial attention sub-module first uses a one-dimensional convolutional operation based on the grouped convolution strategy to fuse the data features output by the channel attention sub-module in the spatial dimension; then, the attention weights at different spatial positions are determined through the Sigmoid function.

4. A method for fusing ocean temperature data based on deep learning according to claim 1, wherein, The second data-level fusion sub-network f2 uses the first BN-MLP network to realize the data-level fusion of the sea surface data source temperature data and the spatio-temporal information. The number of neurons in the input layer of the first BN-MLP network is the sum of the number of sea surface data source types and the length of the spatio-temporal feature vector; the hidden layer of the first BN-MLP network consists of 3 sequentially connected batch-normalized multi-dimensional perceptron modules. The number of neurons in the linear layer of each multi-dimensional perceptron module is 11, 16, and 8 respectively, and the APReLU function is used to realize the non-linear activation of the features.

5. A method for fusing ocean temperature data based on deep learning according to claim 1, characterized in that, The feature-level fusion sub-network f3 uses a second BN-MLP network to extract features from the underwater fusion features and the sea surface and spatio-temporal fusion features; the input layer of the second BN-MLP network inputs the combined vector of the underwater fusion features and the sea surface and spatio-temporal fusion features; the hidden layer of the second BN-MLP network consists of 3 successively connected batch-normalized multi-dimensional perceptron modules, and the number of neurons in the linear layer of each multi-dimensional perceptron module is 92, 128, and 64 respectively, and the APReLU function is used to achieve non-linear activation of the features.

6. A marine temperature data fusion system based on deep learning, characterized by including Multi-source marine temperature data acquisition module: used to acquire multi-source marine temperature data, and the multi-source marine temperature data includes 1D temperature observation data, 2D sea surface data source temperature data, and 3D gridded temperature data; Data preprocessing module: used to preprocess the acquired multi-source marine temperature data; First fusion feature acquisition module: uses the first data-level fusion sub-network f1 to extract features and fuse information from the preprocessed 3D gridded temperature data to obtain underwater fusion features, denoted as the first fusion feature; Second fusion feature acquisition module: uses the second data-level fusion sub-network f2 to extract features and fuse information from the preprocessed 2D sea surface data source temperature data and the corresponding spatio-temporal information to obtain sea surface and spatio-temporal fusion features, denoted as the second fusion feature; Fusion data set acquisition module: uses the feature-level fusion sub-network f3 to extract features and globally fuse the obtained first fusion feature and second fusion feature to obtain a fusion data set.

7. The ocean temperature data fusion system based on deep learning according to claim 6, characterized in that, The input layer of the first data-level fusion sub-network f1 includes four Block blocks A, B, C, and D connected in sequence; Among them, Block A includes 1 first improved global attention GAM module, 1 one-dimensional convolutional layer, 1 batch normalization layer, 1 activation function layer, and 1 second improved global attention GAM module connected in sequence; Block B includes a first residual mapping sub-module introducing a skip connection structure, and the first residual mapping sub-module includes 1 first one-dimensional convolutional layer, 1 batch normalization layer, 1 activation function layer, 1 dropout layer, and 1 second one-dimensional convolutional layer stacked and connected in sequence; Block C includes a second residual mapping sub-module introducing a skip connection structure, and the second residual mapping sub-module includes 1 batch normalization layer, 1 activation function layer, 1 dropout layer, and 1 one-dimensional convolutional layer stacked and connected in sequence; Block D includes 1 batch normalization layer, 1 activation function layer, and 1 third improved global attention GAM module connected in sequence.

8. A deep learning-based ocean temperature data fusion system according to claim 7, characterized in that, The structures of the first improved global attention GAM module and the second improved global attention GAM module are the same, including a channel attention sub-module and a spatial attention sub-module connected in series; The channel attention sub-module first uses a two-layer multi-dimensional perceptron module to amplify the "channel-space" cross-dimensional correlation; Then, the feature dimension is restored by inverse permutation; finally, the channel feature weight values are obtained through the Sigmoid function. The spatial attention sub-module first uses one-dimensional convolution operation based on the grouped convolution strategy to achieve the fusion of the data features output by the channel attention sub-module in the spatial dimension; then, the attention weights at different spatial positions are determined through the Sigmoid function.

9. The ocean temperature data fusion system based on deep learning according to claim 6, characterized in that The second data-level fusion sub-network f2 uses the first BN-MLP network to achieve the data-level fusion of the sea surface data source temperature data and the spatio-temporal information. The number of neurons in the input layer of the first BN-MLP network is the sum of the number of sea surface data source types and the length of the spatio-temporal feature vector; the hidden layer of the first BN-MLP network consists of 3 sequentially connected batch-normalized multi-dimensional perceptron modules, and the number of neurons in the linear layer in each multi-dimensional perceptron module is 11, 16, and 8 respectively, and the APReLU function is used to achieve the non-linear activation of the features.

10. A marine temperature data fusion system based on deep learning according to claim 6, characterized in that, The feature-level fusion sub-network f3 uses the second BN-MLP network to extract features from the underwater fusion features and the sea surface and spatio-temporal fusion features; the input layer of the second BN-MLP network inputs the combined vector of the underwater fusion features and the sea surface and spatio-temporal fusion features; the hidden layer of the second BN-MLP network consists of 3 sequentially connected batch-normalized multi-dimensional perceptron modules, and the number of neurons in the linear layer in each multi-dimensional perceptron module is 92, 128, and 64 respectively, and the APReLU function is used to achieve the non-linear activation of the features.

Images