Ocean data reconstruction method based on conditional information to generate missing features

Abstract

The present invention relates to an ocean data reconstruction method for generating missing features based on conditional information, comprising the following steps: time period division and sample set construction; construction of a missing feature generation model; training and verification of the missing feature generation model; construction of an ocean data reconstruction model; and training and verification of the ocean data reconstruction model. The missing feature generation model constructed by the present invention uses controllable conditional information as a constraint to guide a generator in generating an ocean feature field. Compared to the original generator using random perturbations, it is easier to converge, and the generated ocean feature field is more realistic and more compliant with reality. A design combining adversarial loss and L1 loss is used during training to improve the stability and efficiency of the model, as well as the quality and accuracy of the generated feature field.

Description

Technical Field

[0001] The present invention relates to the field of marine science and machine learning technology, and in particular to a method for reconstructing marine data by generating missing features based on conditional information. Background Art

[0002] To address the problem that "traditional linear methods are overly simple and numerical model methods are computationally complex, making it difficult to effectively integrate multi-source heterogeneous marine environmental data," the invention patent, patent number CN202211050583.X, entitled "Intelligent Fusion Method and System for Three-Dimensional Marine Environmental Fields Based on Deep Learning," achieves intelligent fusion of multi-source marine data by constructing a deep learning network model, thereby generating a three-dimensional marine element fusion field that is closer to observation. However, due to the multi-source, heterogeneous, and multi-dimensional characteristics of marine environmental data, the valid years of the data used for fusion vary. The above method works well in the period after the 1990s when marine data is abundant, but has certain shortcomings in the early historical period.

[0003] Due to the relatively late emergence of ocean satellites, early in-situ ocean observation data was relatively scarce, and the types and number of ocean observations available for assimilation were limited. This resulted in poor accuracy and consistency in the three-dimensional gridded data sources before the 1990s. High-resolution, high-precision, high-quality data sources like HYCOM and SODA began relatively late, and intelligent fusion methods lack high-quality data sources covering the early historical period, posing a significant challenge to the historical reconstruction of ocean features. There are two common approaches to addressing the lack of high-quality fusion data sources in the early historical period: retraining the intelligent fusion model from the early historical period, with input features excluding the missing high-quality data sources; and supplementing the missing time period with climatological feature fields from high-quality data sources. However, both methods have limitations in the accuracy of the ocean feature reconstructions, failing to meet the demands of refined scientific research. Summary of the Invention

[0004] In view of the above shortcomings of the prior art, an object of the present invention is to provide an ocean data reconstruction method for generating missing features based on conditional information, so as to solve one or more problems in the prior art.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] The ocean data reconstruction method for generating missing features based on conditional information includes the following steps:

[0007] Time period division and sample set construction;

[0008] Building a missing feature generation model;

[0009] Training and validation of missing feature generation models;

[0010] Building ocean data reconstruction models;

[0011] Training and validation of ocean data reconstruction models.

[0012] Furthermore, the time period division and sample set construction include the following steps:

[0013] Divide the feature missing period P1 and the feature complete period P2;

[0014] The sample set DS1 is constructed by the feature complete period P2 and the sample set DS2 is constructed by the feature missing period P1.

[0015] Furthermore, the missing feature generation model building includes the following steps:

[0016] Construct a generator G to integrate the known fusion data source ODS n Converted to generated fusion data source

[0017] Construct the discriminator D to generate the fusion data source and real fusion data sources Perform feature extraction and calculate the probability of being true;

[0018] Loss function design.

[0019] Furthermore, the loss function in the loss function design is designed based on the adversarial loss, and the loss function is shown as follows:

[0020] Loss(f G ,f D )=Loss L1 (f G )+λLoss GAN (f G ,f D )

[0021] Where: λ is a hyperparameter that controls the adversarial loss Loss GAN (f G ,f D ) and L1 loss Loss L1 (f G ) contributes to the overall optimization goal.

[0022] Furthermore, the adversarial loss calculation formula is as follows:

[0023] Loss GAN (f G ,f D )=E[logf D (ODS real)]+E[log(1-f D (ODS gen ))]

[0024] Where: E is the expectation operator, f D (i) represents the probability obtained after the data is input into the discriminator D.

[0025] Furthermore, the training and verification of the missing feature generation model is based on the fusion data source generated by comparing the sample set DS2 of the P1 period and real fusion data sources The root mean square error RMSE and the coefficient of determination R 2 .

[0026] Furthermore, the ocean data reconstruction model construction includes the following steps:

[0027] Freeze generator G;

[0028] Feature extraction;

[0029] Data reconstruction.

[0030] Furthermore, the feature extraction includes feature extraction of fused data sources. Feature extraction of sea surface environmental elements and temporal and spatial information

[0031] Furthermore, the data reconstruction is based on the global fusion reconstruction of the feature Feat1 of the fusion data source and the feature Feat2 of the sea surface environment elements and spatiotemporal information to obtain the reconstructed ocean element field OD fusion .

[0032] Furthermore, the training and verification of the ocean data reconstruction model is based on the sample set DS1 of the P2 period, and the ocean element field OD is reconstructed by comparison. fusion The observation element field OD corresponding to the sample set DS2 obs The root mean square error RMSE and the coefficient of determination R 2 .

[0033] Compared with the prior art, the beneficial technical effects of the present invention are as follows:

[0034] (1) The missing feature generation model constructed by the present invention guides the generator to generate ocean feature fields by inputting controllable conditional information as constraints. Compared with the original generator using random perturbations, it is easier to converge, and the generated ocean feature fields are more realistic and more in line with reality. During the training process, a design combining adversarial loss and L1 loss is adopted to improve the stability and efficiency of the model as well as the quality and accuracy of the generated feature fields.

[0035] (2) Freeze the parameters of the trained generator and connect it to the ocean data reconstruction model. The feature extraction of 3D fusion data source is performed; the sea surface environmental elements and spatiotemporal information are added to the fusion process in the form of an associated network. This model structure design can learn different degrees of information in different dimensions, effectively avoiding the problem of one-dimensional and two-dimensional information loss caused by the network being too deep, thereby making the reconstruction of the ocean feature field more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flow chart of a method for reconstructing ocean data by generating missing features based on conditional information according to an embodiment of the present invention is shown.

[0037] Figure 2 A schematic diagram showing the composition of the sample set DS1 and the sample set DS2 of the ocean data reconstruction method for generating missing features based on conditional information according to an embodiment of the present invention.

[0038] Figure 3 The structure diagram of the missing feature generation model FFG-cGAN of the ocean data reconstruction method for generating missing features based on conditional information in an embodiment of the present invention is shown.

[0039] Figure 4 The structure diagram of the ocean data reconstruction model FFG-ODF-Net of the ocean data reconstruction method for generating missing features based on conditional information according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clear, the ocean data reconstruction method for generating missing features based on conditional information proposed by the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. According to the following description, the advantages and features of the present invention will become clearer. It should be noted that the drawings are in a very simplified form and are not in precise proportions. They are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, please refer to the drawings. It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the content disclosed in the specification for people familiar with this technology to understand and read. They are not used to limit the conditions for the implementation of the present invention, so they have no technical significance. Any modification of the structure, change in the proportional relationship or adjustment of the size should still fall within the scope of the technical content disclosed by the present invention without affecting the efficacy and purpose that can be achieved by the present invention.

[0041] See also Figure 1 ,The ocean data reconstruction method for generating missing features based on conditional information, includes the following steps:

[0042] Step 1: Time period division and sample set construction.

[0043] Step 1.1: Divide the feature missing period P1 and the feature complete period P2.

[0044] According to the temporal distribution of multi-source ocean data, it is divided into the feature missing period, that is, the time period when the fused data source is missing in the early history, recorded as P1, and the feature complete period, that is, the time period when the fused data source is complete in the recent history, recorded as P2.

[0045] Step 1.2: Construct a sample set DS1 through the feature complete period P2 and construct a sample set DS2 through the feature missing period P1.

[0046] Please continue reading Figure 2 , for sample set DS1, ODS n represents the nth set of fused data sources shared by both P1 and P2, st represents the corresponding spatiotemporal information of the temperature-salinity profile processed by the data feature generation model within the complete feature period of the sample, Indicates the mth set of fused data sources that P2 has but P1 does not have.

[0047] Furthermore, for sample set DS2, represents the mth set of fusion data sources generated by the missing feature generation model FFG-cGAN, SSx represents the corresponding sea surface environmental elements of the temperature and salinity profile processed by the data fusion model during the sample feature missing period, OD obs It represents the observation element field corresponding to the temperature-salinity profile processed by the data fusion model during the sample, i.e., the feature missing period.

[0048] Step 2: Build the missing feature generation model.

[0049] The missing feature generation model FFG-cGAN model is based on the maximum-minimum game theory. It uses the generator G and the discriminator D to compete with each other to generate "fake data" that is indistinguishable from the real fusion data source.

[0050] Please continue reading Figure 1 and Figure 3 , the missing feature generation model building includes the following steps:

[0051] Step 2.1: Build the generator G.

[0052] The goal of the generator G is to transform the known fusion data source ODS n That is, the controllable condition information mapping as input is converted into the generated fusion data source Make it difficult for the discriminator D to distinguish whether the input fusion data source is real or generated by the generator G.

[0053] Step 2.2: Build the discriminator D.

[0054] Generated fusion data source and real fusion data sources Perform feature extraction and calculate the probability of being true, maximize the ability to distinguish between real data and generated data, and thus improve the effect of adversarial training.

[0055] Step 2.3: Loss function design.

[0056] The loss function in the loss function design is designed based on the adversarial loss. The loss function of the missing feature generation model FFG-cGAN is shown in the following formula 1:

[0057] Loss(f G ,f D )=Loss L1 (f G )+λLoss GAN (f G ,f D ) (1)

[0058] Where: λ is a hyperparameter that controls the adversarial loss Loss GAN (f G ,f D ) and L1 loss Loss L1 (f G ) contributes to the overall optimization goal.

[0059] Furthermore, the adversarial loss calculation formula is shown in the following formula 2:

[0060] Loss GAN (f G ,f D )=E[logf D (ODS real )]+E[log(1-f D (ODS gen ))] (2)

[0061] Where: E is the expectation operator, f D (i) represents the probability obtained after the data is input into the discriminator D.

[0062] Step 3: Training and validation of feature-deficient generation model.

[0063] The training and verification of the missing feature generation model is based on the sample set DS1 of the P2 period to train the missing feature generation model FFG-cGAN built in step 2, and compare the generated fusion data source and real fusion data sources The root mean square error RMSE and the coefficient of determination R 2 , and then verify the model effect.

[0064] Step 4: Build ocean data reconstruction model.

[0065] Please continue reading Figure 1 and Figure 4 The ocean data reconstruction model construction includes the following steps:

[0066] Step 4.1: Freeze the generator G.

[0067] Based on the generator G of the missing feature generation model FFG-cGAN trained in step 2, freeze its parameters and build the ocean data reconstruction model FFG-ODF-Net.

[0068] Step 4.2: Feature extraction.

[0069] Combining sample set DS1 and sample set DS2, the feature extraction includes feature extraction of fusion data source Feature extraction of sea surface environmental elements SSx and spatiotemporal information st Two processes. Feature extraction of fusion data source It is to integrate the known fusion data source ODS n and the fused data source generated by generator G in sample set DS2 Splicing is performed to extract the features of the 3D fusion data source. The feature extraction of the sea surface environmental elements SSx and the spatiotemporal information st It is to add the sea surface environmental elements SSx and spatiotemporal information st into the fusion process in the form of an associated network.

[0070] Step 4.3: Data reconstruction.

[0071] The process in step 4.2 above The extracted features are recorded as Feat1, process f D2 The extracted features are recorded as Feat2. The data reconstruction is based on the global fusion reconstruction of the features Feat1 of the fusion data source and the features Feat2 of the sea surface environment elements and temporal and spatial information to obtain the reconstructed ocean element field OD fusion .

[0072] Step 5: Training and validation of ocean data reconstruction model.

[0073] Based on the sample set DS2 of the P1 period, the ocean data reconstruction model FFG-ODF-Net built in step 4 is trained, and the ocean feature field OD is reconstructed by comparison. fusion The observation element field OD corresponding to the sample set DS2 obsThe root mean square error RMSE and the coefficient of determination R 2 , and then verify the model effect.

[0074] End the process.

[0075] Specifically, multi-source sea temperature data from 1994 to 2017 were used to train and validate the missing feature generation model (FFG-cGAN) in the method described in this paper. Data from 1994, 2006, and 2017 were reserved as the test set, and the remaining data served as the training and validation set. The known fusion data sources, i.e., the controllable condition information, were CORA, ORAS5, BCC-CSM2-MR, CESM2, HYCOM climate state, and SODA climate state sea temperature. The output was the ECCO, SODA, and HYCOM sea temperature. The metrics on the test set are shown in Table 1 below.

[0076] Table 1

[0077]

[0078] From Table 1 above, we can conclude that and The overall error is small, and the determination coefficient R 2 Both are higher than 0.98, indicating that the missing feature generation model FFG-cGAN of the method described in the present invention can better fit the real fusion data source.

[0079] Furthermore, multi-source sea temperature data from 1958 to 1981 were used to train and verify the ocean data reconstruction model FFG-ODF-Net in the method described in the present invention. Data from 1958, 1970, and 1981 were selected as the test set, and the remaining data were used as the training and validation set. Regarding the effectiveness of ocean data reconstruction, this embodiment sets up several groups of experiments for comparison. The experimental settings and indicators are shown in Table 2 below.

[0080] Table 2

[0081]

[0082] Table 2 shows that, when comparing Case A with Cases B through E, the average RMSE of the SST reconstruction results using the generated fusion data source is lower than when not using it. This indicates that the addition of high-quality generated fusion data sources significantly improves SST reconstruction. A comparison of Cases B through E reveals that the accuracy of the SST reconstruction results varies with the use of different generated data sources. The order of SST data accuracy is SODA > HYCOM > ECCO. Therefore, the accuracy of the labels corresponding to the generated fusion data source is positively correlated with the accuracy of the final SST reconstruction results. Case E demonstrated the best performance among all experiments, indicating that generating more high-quality fusion data sources can further improve SST reconstruction accuracy.

[0083] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0084] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A method for reconstructing ocean data by generating missing features based on conditional information, characterized by: The following steps are involved: The time period division and sample set construction include the following steps: Divide the feature missing period P1 and feature complete period P2, The sample set DS1 is constructed by the feature complete period P2 and the sample set DS2 is constructed by the feature missing period P1; The missing feature generation model is constructed, and the missing feature generation model is constructed, which includes the following steps: Construct a generator G to integrate the known fusion data source ODS n Converted to generated fusion data source Construct the discriminator D to generate the fusion data source and real fusion data sources Perform feature extraction and calculate the probability of being true, Loss function design, in which the loss function is designed based on adversarial loss, the loss function is shown as follows: Loss(f G ,f D )=Loss L1 (f G )+λLoss GAN (f G ,f D ) Where: λ is a hyperparameter that controls the adversarial loss Loss GAN (f G ,f D ) and L1 loss Loss L1 (f G ) contribution to the overall optimization goal; Training and validation of missing feature generation models; Building ocean data reconstruction models; Training and validation of ocean data reconstruction models.

2. The ocean data reconstruction method for generating missing features based on conditional information according to claim 1, characterized in that: The adversarial loss calculation formula is as follows: Loss GAN (f G ,f D )=E[logf D (ODS real )]+E[log(1-f D (ODS gen ))] Where: E is the expectation operator, f D (i) represents the probability obtained after the data is input into the discriminator D.

3. The ocean data reconstruction method for generating missing features based on conditional information according to claim 1, characterized in that: The training and verification of the missing feature generation model is based on the sample set DS1 of the P2 period, and the fusion data source generated by comparison and real fusion data sources The root mean square error RMSE and the coefficient of determination R 2 .

4. The ocean data reconstruction method for generating missing features based on conditional information according to claim 1, characterized in that: The ocean data reconstruction model construction includes the following steps: Freeze generator G; Feature extraction; Data reconstruction.

5. The ocean data reconstruction method for generating missing features based on conditional information according to claim 4, characterized in that: The feature extraction includes feature extraction of fused data sources Feature extraction of sea surface environmental elements and temporal and spatial information 6. The ocean data reconstruction method for generating missing features based on conditional information according to claim 5, characterized in that: The data reconstruction is based on the global fusion reconstruction of the feature Feat1 of the fusion data source and the feature Feat2 of the sea surface environment elements and time and space information to obtain the reconstructed ocean element field OD fusion .

7. The ocean data reconstruction method for generating missing features based on conditional information according to claim 1, characterized in that: The training and verification of the ocean data reconstruction model is based on the sample set DS2 of the P1 period, and the ocean element field OD is reconstructed by comparison fusion The observation element field OD corresponding to the sample set DS2 obs The root mean square error RMSE and the coefficient of determination R 2 .

Images