A fast reconstruction method of regional marine environment field based on hierarchical joint neural network

Abstract

The present invention discloses a method for rapid reconstruction of a regional marine environment field based on a hierarchical joint neural network, which belongs to the technical field of environmental field reconstruction and is used for marine environment field reconstruction, including obtaining multi-source marine data and preprocessing it, calculating the preprocessed multi-source marine data to obtain a density field, constructing a multi-factor coupling field according to a temperature field, a salinity field, and a density field; making a deep learning sample set, constructing a training hierarchical joint neural network, and using the trained hierarchical joint neural network to reconstruct the regional marine environment field. The present invention takes into account the coupling of marine multi-factor fields, marine stratification characteristics, the differences and connections between observation data of the shallow surface layer and the middle and lower layers of the ocean, and the temporal continuity of marine changes, and realizes high-precision reconstruction of the marine environment field.

Description

Technical Field

[0001] The invention discloses a method for rapid reconstruction of a regional ocean environment field based on a hierarchical joint neural network, and belongs to the technical field of environment field reconstruction. Background Art

[0002] In recent years, artificial intelligence methods such as deep learning have been applied to the reconstruction of marine environmental fields and have made some progress. This type of reconstruction method can achieve ideal inversion accuracy while retaining rich details, and realize super-resolution inversion of marine environmental fields. At the same time, the use of parallel computing on graphics cards can greatly improve the speed of reconstruction operations and realize rapid inversion of marine environmental fields, which is more time-consuming than traditional marine environmental field reconstruction methods with large computational load. However, existing deep learning methods lack the organic combination of neural network structure and real ocean characteristics and observation data characteristics, that is, they lack consideration of the coupling of multi-factor fields in the ocean, the stratification characteristics of the ocean, the differences and connections between observation data in the shallow and middle and lower layers of the ocean, and the temporal continuity of ocean changes. Summary of the invention

[0003] The purpose of the present invention is to provide a method for rapid reconstruction of regional marine environment fields based on a hierarchical joint neural network, so as to solve the problem of insufficient accuracy of the results of marine environment field reconstruction in the prior art.

[0004] A method for rapid reconstruction of regional marine environmental fields based on a hierarchical joint neural network includes obtaining multi-source ocean data and preprocessing it, calculating the density field from the preprocessed multi-source ocean data, and constructing a multi-factor coupling field based on the temperature field, salinity field, and density field; making a deep learning sample set, constructing a trained hierarchical joint neural network, and using the trained hierarchical joint neural network to reconstruct the regional marine environmental field.

[0005] The hierarchical joint neural network inputs shallow layer input data and middle and lower layer input data, the shallow layer input data is input into convolutional neural network module A, and the middle and lower layer input data is input into convolutional neural network module B. The outputs of convolutional neural network module A and convolutional neural network module B are connected and input into the recurrent neural network module, and then the output reconstructs the environmental field.

[0006] The convolutional neural network module A is provided with branches, which sequentially connect the generative adversarial network module and the convolutional neural network module B.

[0007] The convolutional neural network module A and the convolutional neural network module B have the same structure.

[0008] Convolutional neural network module A performs 4 modular processing at the input end. One modular processing includes first performing 2 3×3 convolutions and applying ReLU activation function, and then performing 2×2 maximum pooling;

[0009] The modular process is copied 4 times to the output. After copying, the 2×2 maximum pooling becomes 2×2 upsampling. One modular process includes performing 2×2 upsampling, then performing 2 times of 3×3 convolution and applying the ReLU activation function.

[0010] After performing 4 modular processes on the input side, 2 3×3 convolutions are performed and the ReLU activation function is applied, and then 4 replicated modular processes are performed on the output side;

[0011] Finally, a 1×1 convolution is performed and output.

[0012] The generative adversarial network module includes a generator and a discriminator. The shallow layer environment field is input into the generator to generate the environment field, which is then input into the discriminator together with the middle and lower layer environment fields as training samples, and finally the discrimination is performed.

[0013] The input of the recurrent neural network is the environmental field input. Multiple environmental field inputs are respectively input into the corresponding 3D long short-term memory convolutional neural network, and then input into the hidden layer to finally form the environmental field output.

[0014] Elements of multi-source ocean data include temperature data, salinity data, and density data;

[0015] Types of multi-source ocean data include observational data, reanalysis data, and numerical simulation data.

[0016] Preprocessing includes unified data format, depth correction, spatiotemporal interpolation and quality control.

[0017] The deep learning sample set includes training set, validation set and test set.

[0018] Compared with the prior art, the present invention has the following beneficial effects: the present invention takes into account the coupling of multiple ocean elements, the stratification characteristics of the ocean, the differences and connections between observation data of the shallow surface layer and the middle and lower layers of the ocean, and the temporal continuity of ocean changes, and realizes high-precision reconstruction of the ocean environment field. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a technical flow chart of the present invention;

[0020] Figure 2 It is a schematic diagram of the structure of the hierarchical joint neural network model;

[0021] Figure 3 It is a schematic diagram of a convolutional neural network;

[0022] Figure 4 is a schematic diagram of a generative adversarial network;

[0023] Figure 5 is a schematic diagram of a recurrent neural network;

[0024] Figure 6 Statistical comparison of reconstruction error of layered joint neural network and reconstruction error of Kriging interpolation method

[0025] Figure 7 is an example of the true value of the environmental field in the experiment;

[0026] Figure 8 It is an example of a virtual observation station in the experiment;

[0027] Fig. 9 is an example of the output of the convolutional neural network module A inferring the environment field;

[0028] Fig.10 is an example of the input of the temperature field of the generative adversarial network module at a depth of 0m;

[0029] Fig.11 This is an example of the true value of the temperature field at a depth of 200 m in the experiment;

[0030] Fig.12 This is an example of the output of the temperature field inference value generated by the Generative Adversarial Network module at a depth of 200m;

[0031] Fig.13 is an example of the input of the environment field to the recurrent neural network module at time t;

[0032] Fig.14 is an example of the input of the environment field of the recurrent neural network module at time t+1;

[0033] Fig.15 is an example of the true value of the environmental field at time t+2;

[0034] Fig.16 is an example of the inference value output of the environmental field inferred by the recurrent neural network module at time t+2;

[0035] Fig.17 It is a schematic diagram of the truth value in the layered joint neural network reconstruction effect experiment;

[0036] Fig.18 This is a schematic diagram of the sampling stations in the layered joint neural network reconstruction effect experiment;

[0037] Fig.19 This is a schematic diagram of the neural network reconstruction results in the layered joint neural network reconstruction effect experiment;

[0038] Fig. 20 This is a schematic diagram of the reconstruction results of the Kriging interpolation method in the layered joint neural network reconstruction effect experiment. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] A method for rapid reconstruction of regional marine environmental fields based on a hierarchical joint neural network includes obtaining multi-source ocean data and preprocessing it, calculating the density field from the preprocessed multi-source ocean data, and constructing a multi-factor coupling field based on the temperature field, salinity field, and density field; making a deep learning sample set, constructing a trained hierarchical joint neural network, and using the trained hierarchical joint neural network to reconstruct the regional marine environmental field.

[0041] The hierarchical joint neural network inputs shallow layer input data and middle and lower layer input data, the shallow layer input data is input into convolutional neural network module A, and the middle and lower layer input data is input into convolutional neural network module B. The outputs of convolutional neural network module A and convolutional neural network module B are connected and input into the recurrent neural network module, and then the output reconstructs the environmental field.

[0042] The convolutional neural network module A is provided with branches, which sequentially connect the generative adversarial network module and the convolutional neural network module B.

[0043] The convolutional neural network module A and the convolutional neural network module B have the same structure.

[0044] Convolutional neural network module A performs 4 modular processing at the input end. One modular processing includes first performing 2 3×3 convolutions and applying ReLU activation function, and then performing 2×2 maximum pooling;

[0045] The modular process is copied 4 times to the output. After copying, the 2×2 maximum pooling becomes 2×2 upsampling. One modular process includes performing 2×2 upsampling, then performing 2 times of 3×3 convolution and applying the ReLU activation function.

[0046] After performing 4 modular processes on the input side, 2 3×3 convolutions are performed and the ReLU activation function is applied, and then 4 replicated modular processes are performed on the output side;

[0047] Finally, a 1×1 convolution is performed and output.

[0048] The generative adversarial network module includes a generator and a discriminator. The shallow layer environment field is input into the generator to generate the environment field, which is then input into the discriminator together with the middle and lower layer environment fields as training samples, and finally the discrimination is performed.

[0049] The input of the recurrent neural network is the environmental field input. Multiple environmental field inputs are respectively input into the corresponding 3D long short-term memory convolutional neural network, and then input into the hidden layer to finally form the environmental field output.

[0050] Elements of multi-source ocean data include temperature data, salinity data, and density data;

[0051] Types of multi-source ocean data include observational data, reanalysis data, and numerical simulation data.

[0052] Preprocessing includes unified data format, depth correction, spatiotemporal interpolation and quality control.

[0053] The deep learning sample set includes training set, validation set and test set.

[0054] The technical process of the present invention is as follows Figure 1 As shown, it includes acquiring multi-source ocean temperature and salinity data, preprocessing ocean temperature and salinity data, constructing a multi-factor coupling field, making a deep learning sample set, constructing and training a hierarchical joint neural network, and reconstructing the regional ocean environment field.

[0055] The structure of the hierarchical joint neural network model is as follows Figure 2 As shown, the hierarchical joint neural network inputs shallow layer input data and middle and lower layer input data, the shallow layer input data is input into the convolutional neural network module A, the middle and lower layer input data is input into the convolutional neural network module B, the convolutional neural network module A and the convolutional neural network module B are connected and then input into the recurrent neural network module, and then the reconstructed environment field is output. The convolutional neural network module A is provided with branches, and the branches are sequentially connected to the generative adversarial network module and the convolutional neural network module B. The convolutional neural network module A and the convolutional neural network module B have the same structure.

[0056] Convolutional neural networks such as Figure 3 As shown in the figure, the convolutional neural network module A performs 4 modular processings at the input end. One modular processing includes first performing 2 3×3 convolutions and applying the ReLU activation function, and then performing 2×2 maximum pooling; the 4 modular processings are copied to the output end. After the copy, the 2×2 maximum pooling becomes 2×2 upsampling. One modular processing includes performing 2×2 upsampling, and then performing 2 3×3 convolutions and applying the ReLU activation function; after the input end performs 4 modular processings, it performs 2 3×3 convolutions and applies the ReLU activation function, and then performs 4 copied modular processings at the output end; finally, a 1×1 convolution is performed and output.

[0057] Convolutional neural network module A uses a 5-layer U-Net architecture, and the input is a single-layer environmental field data X1. X1 is a three-dimensional tensor. The first two dimensions are the longitude and latitude of the reconstructed environmental field, and the third dimension is the type of element of the reconstructed environmental field. The number of rows of the data tensor is denoted as h, the number of columns is denoted as w, and the number of channels is denoted as c. The shape of the input data in the embodiment is h=512, w=512, c=3. In the first layer of the neural network, the input data X0 undergoes two convolutions with a kernel of 3×3 and two ReLU activation functions to obtain X′1, that is:

[0058] X′1=ReLU(Conv(ReLU(Conv(X0))));

[0059] In the formula, ReLU represents the activation function, and Conv represents the convolution operation;

[0060] The shape of X′1 is h=512, w=512, c=64. X′1 is downsampled by the maximum pooling with a kernel of 2×2 and enters the second layer of the neural network to form X2:

[0061] X2 = MaxPool(X′1);

[0062] In the formula, MaxPool represents maximum pooling;

[0063] The shape of X2 is h=256, w=256, c=64. In the second layer of the neural network, X2 undergoes two convolutions with a kernel of 3×3 and two ReLU activation functions to obtain X′2, that is:

[0064] X′2=ReLU(Conv(ReLU(Conv(X2))));

[0065] The shape of X'2 is h = 256, w = 256, c = 128. X'2 is downsampled by the maximum pooling with a kernel of 2×2 and enters the third layer of the neural network to form X3:

[0066] X3 = MaxPool(X′2);

[0067] The shape of X3 is h=128, w=128, c=128. In the third layer of the neural network, X3 undergoes two convolutions with a kernel of 3×3 and two ReLU activation functions to obtain X′3, that is:

[0068] X′3=ReLU(Conv(ReLU(Conv(X3))));

[0069] The shape of X'3 is h=128, w=128, c=256. X'3 is downsampled by the maximum pooling with a kernel of 2×2 and enters the fourth layer of the neural network to form X4:

[0070] X4 = MaxPool(X′3);

[0071] The shape of X4 is h=64, w=64, c=256. In the fourth layer of the neural network, X4 undergoes two convolutions with a kernel of 3×3 and two ReLU activation functions to obtain X′4, that is:

[0072] X′4=ReLU(Conv(ReLU(Conv(X4))));

[0073] The shape of X′4 is h=64, w=64, c=512. X′4 is downsampled by the maximum pooling with a kernel of 2×2 and enters the fifth layer of the neural network to form X5:

[0074] X5 = MaxPool(X′4);

[0075] The shape of X5 is h=32, w=32, c=512. In the fourth layer of the neural network, X5 undergoes two 3×3 convolutions and two ReLU activation functions to obtain Y5. The number of channels increases from 512 to 1024 and then decreases to 512, that is:

[0076] Y5=ReLU(Conv(ReLU(Conv(X5))));

[0077] The shape of Y5 is h=32, w=32, c=512. Y5 is upsampled with a kernel of 2×2 and enters the fourth layer of the neural network, and then connected with X′4 to form Y′4:

[0078] Y′4=[X′4,UpSample(Y5)];

[0079] The shape of Y′4 is h=64, w=64, c=1024. Y′4 undergoes two convolutions with a kernel of 3×3 and two ReLU activation functions to obtain Y4, that is:

[0080] Y4=ReLU(Conv(ReLU(Conv(Y′4))));

[0081] The shape of Y4 is h=64, w=64, c=256. Y4 is upsampled with a kernel of 2×2 and enters the third layer of the neural network, and then connected with X′3 to form Y′3:

[0082] Y′3=[X′3,UpSample(Y4)];

[0083] The shape of Y'3 is h = 128, w = 128, c = 512. Y'3 undergoes two convolutions with a kernel of 3×3 and two ReLU activation functions to obtain Y3, that is:

[0084] Y3=ReLU(Conv(ReLU(Conv(Y′3))));

[0085] The shape of Y3 is h=128, w=128, c=128. Y3 is upsampled with a kernel of 2×2 and enters the second layer of the neural network, and then connected with X′2 to form Y′2:

[0086] Y′2=[X′2,UpSample(Y3)];

[0087] The shape of Y'2 is h = 256, w = 256, c = 256. Y'2 undergoes two convolutions with a kernel of 3×3 and two ReLU activation functions to obtain Y2, that is:

[0088] Y2=ReLU(Conv(ReLU(Conv(Y′2))));

[0089] The shape of Y2 is h=256, w=256, c=64. Y2 is upsampled with a kernel of 2×2 and enters the first layer of the neural network, and then connected with X′1 to form Y′1:

[0090] Y′1=[X′1,UpSample(Y2)];

[0091] The shape of Y'1 is h = 512, w = 512, c = 128. Y'1 undergoes two convolutions with a kernel of 3 × 3, two ReLU activation functions, and one convolution with a kernel of 1 × 1 to obtain Y1, that is:

[0092] Y1=Conv1×1(ReLU(Conv(ReLU(Conv(Y′2))))));

[0093] The shape of Y1 is h=512, w=512, c=3.

[0094] The convolutional neural network module A reconstructs one layer of environmental field data each time. In this example, the operation is repeated 10 times to obtain 10 layers of shallow surface environmental fields, denoted as X. s , used in the next step.

[0095] Generate adversarial networks such as Figure 4 As shown, the generative adversarial network module includes a generator and a discriminator. The shallow layer environment field is input into the generator to generate the environment field, which is input into the discriminator together with the middle and lower layer environment fields as training samples, and finally the discrimination is performed.

[0096] In the generator section, enter X s , X s It is a 10-layer environmental field in the shallow layer. Each layer has three ocean elements: temperature, salinity, and density. Therefore, the shape is h = 512, w = 512, and c = 30. s The generated environment field X is obtained through the generator g ,Right now:

[0097] Xg =Generator(X s );

[0098] X g The shape of is h = 512, w = 512, c = 30, that is, a 10-layer environment field is generated. The generator is a bottleneck convolutional neural network. s After 4 downsampling and 4 upsampling, we get X g , the shapes of the 7 intermediate states state 1, state 2, state 3, state 4, state 5, state 6, state 7 are:

[0099]

[0100] The discriminator part, inputs the generated environment field X g and the middle and lower environmental field training samples X s , X s The 10-layer environment field in the middle and lower layers, X s The shape of is h = 512, w = 512, c = 30. The output of the discriminator is a true or false classification judgment, that is: T or F = Discriminator ([X g , X s ]);

[0101] T stands for true, F stands for false, and Discriminator stands for discriminator;

[0102] The generator contains convolutional neural networks and fully connected layers. The input [X g , X s ] After 1 connection, 4 downsampling, and 2 full connection transformations, the output result is obtained. The shapes of the 6 intermediate states are:

[0103]

[0104] The generative adversarial network module is trained using a supervised learning method. During training, the 10 layers of shallow environmental fields are input into the generator to generate the 10 layers of middle and lower environmental fields. These fields are input into the discriminator together with the 10 layers of middle and lower environmental fields in the training samples to obtain the discriminant results about the generator effect. Based on this, binary cross entropy is constructed as the loss function. The discriminator is trained using the gradient ascent method, and the generator is trained using the gradient descent method. The generator and the discriminator are trained alternately. During inference, only the generator is used, the 10 layers of shallow environmental fields are input, and the 10 layers of middle and lower environmental fields are output. The effect is to invert the middle and lower environmental fields with relatively sparse observation data and greater reconstruction difficulty through the shallow marine environmental fields with relatively rich observation data and ideal reconstruction effect.

[0105] Generate adversarial network module input 10 layers of shallow environment field X s , generate 10 layers of generated environment field X g , used in the next step.

[0106] The convolutional neural network module B reconstructs one layer of environmental field data each time. In this example, the operation is repeated 10 times to obtain the lower and middle environmental fields of 10 layers, which are recorded as X. d , X d The shape of is h=512, w=512, c=30 and is used in the next step.

[0107] The reconstructed environmental field of the shallow ocean surface layer and the reconstructed environmental field of the middle and lower ocean layers are combined to obtain the reconstructed environmental field of the entire water depth.

[0108] Connect the reconstructed ocean surface environment field matrix and the reconstructed ocean middle and lower layer environment field matrix in the channel dimension to obtain the full water depth reconstructed environment field matrix X a ,Right now:

[0109] X a =[X s , X d ];

[0110] X a The shape of is h=512, w=512, c=60, which is used in the next step.

[0111] Recurrent neural networks such as Figure 5 As shown, the input of the recurrent neural network is the environment field input. Multiple environment field inputs are respectively input into the corresponding 3D long short-term memory convolutional neural network, and then input into the hidden layer to finally form the environment field output.

[0112] Input I T It is a sequence of full-depth reconstructed environmental fields composed of the full-depth reconstructed environmental fields at time t and several moments before time t, and the output O is the full-depth reconstructed environmental field at the new time t. Among them, the recurrent neural network refers to the recurrent neural network architecture and its derivative models, including RNN, LSTM, GRU, DRNN, BRNN, and ConvLSTM. A ConvLSTM architecture with a depth of 10 is used, that is, the output is obtained by reasoning with a time series of length 10. The internal structure of the ConvLSTM memory cell is similar to that of the traditional LSTM, and the matrix operation in the LSTM is replaced by a convolution operation. The dimension and size of the input data of the network match the result, and the time series I with a step size of 1 day and a length of 10 is used. T , I T The shape is h = 512, w = 512, c = 60, t 1= 10, where h, w, and c have the same meanings as before, representing the number of rows, columns, and channels, respectively. 1 Represents the number of channels in the time dimension. T The output O is obtained through ConvLSTM operation, that is:

[0113] O = ConvLSTM(I T );

[0114] The shape of O is h=512, w=512, c=60, and the number of channels c=60 represents three types of elements at 20 depths. The output O is the three-dimensional ocean multi-element reconstruction environment field at the target time.

[0115] In the present invention, obtaining a hierarchical joint neural network model includes:

[0116] S1 constructs and trains a convolutional neural network module A, whose input is the shallow ocean data and whose output is the reconstructed shallow ocean environment field; S2 constructs and trains a generative adversarial network module, whose input is the reconstructed shallow ocean environment field and whose output is the reconstructed middle and lower ocean environment field; S3 constructs and trains a convolutional neural network module B, whose input is the middle and lower ocean data and the reconstructed middle and lower ocean environment field in S2, and whose output is a new reconstructed middle and lower ocean environment field; S4 combines the reconstructed shallow ocean environment field in S1 with the reconstructed middle and lower ocean environment field in S3 to obtain a full-water-depth reconstructed environment field; S5 constructs and trains a recurrent neural network module, whose input is a full-water-depth reconstructed environment field sequence composed of the full-water-depth reconstructed environment fields at time t and several times before time t, and whose output is a new full-water-depth reconstructed environment field at time t; S6 combines the above neural network modules in process order to obtain a hierarchical joint neural network model.

[0117] Convolutional neural network refers to the convolutional neural network architecture and its derivative models, including LeNet, AlexNet, VGG, NiN, ResNet, GoogleNet, DenseNet, U-Net. Furthermore, the recurrent neural network refers to the recurrent neural network architecture and its derivative models, including RNN, LSTM, GRU, DRNN, BRNN, ConvLSTM.

[0118] In the embodiment, HYCOM three-dimensional ocean temperature and salinity data from 2005 to 2015 were used to produce a sample set with a total number of 100,000; samples were evenly selected over 10 years with a step length of 1 month, thereby achieving sample time diversity; samples were randomly selected from mid- and low-latitude seas and experimental seas around the world at a ratio of 4:1, thereby achieving regional diversity of samples and strengthening the characteristic proportion of key areas; three different sizes of sea areas, 100km×100km, 200km×200km, and 300km×300km, were selected at a ratio of 1:3:1, and then scaled to the same size, thereby achieving diversity in sea area scales; all sample sets were stratified and sampled at a ratio of 7:2:1 as training sets, validation sets, and test sets.

[0119] Convolutional neural network module A uses a 5-layer U-Net architecture with the following structure: Figure 3 As shown in the figure, the input is the shallow surface environment field data, and one layer of data is input each time, with a size of 512×512×3, where the first two dimensions are the longitude and latitude of the environment field, and the third dimension is the element type of the input environment field. The output is the reconstructed shallow surface environment field, with a size of 512×512×3, where the first two dimensions are the longitude and latitude of the reconstructed environment field, and the third dimension is the element type of the reconstructed environment field.

[0120] The module structure of the generative adversarial network is as follows: Figure 4 As shown in the figure, it is divided into two parts: the generator and the discriminator. The convolutional neural network is used to process image features. The supervised learning method is used for training. During training, the 15 layers of shallow environmental fields are input into the generator to generate the 30 layers of middle and lower environmental fields. The 30 layers of middle and lower environmental fields in the training samples are input into the discriminator to obtain the discrimination results about the generator effect. Based on this, the binary cross entropy is constructed as the loss function. The discriminator is trained using the gradient ascent method, and the generator is trained using the gradient descent method. The generator and the discriminator are trained alternately. During inference, only the generator is used, the 15 layers of shallow environmental fields are input, and the 30 layers of middle and lower environmental fields are output. The effect is to invert the middle and lower environmental fields with relatively scarce observation data and greater reconstruction difficulty through the shallow marine environmental field with relatively rich observation data and ideal reconstruction effect.

[0121] Convolutional neural network module B uses a 5-layer U-Net architecture, and its structure is similar to that of convolutional neural network module A, except that the number of input channels of the first convolutional layer is 6. The input is the environmental field data of the middle and lower layers of the ocean and the reconstructed environmental field of the middle and lower layers output by the generative adversarial network module. Each time a layer of data is input, its size is 512×512×6, of which the first two dimensions are the longitude and latitude of the environmental field, and the third dimension is the element type of the middle and lower layer input environmental field and the reconstructed environmental field. The output is a new reconstructed environmental field of the middle and lower layers of the ocean, and its size is 512×512×3, of which the first two dimensions are the longitude and latitude of the reconstructed environmental field, and the third dimension is the element type of the reconstructed environmental field.

[0122] The recurrent neural network uses a ConvLSTM architecture with a depth of 10, that is, the output is obtained by inferring a time series with a length of 10, such as Figure 5 As shown in the figure. The internal structure of the ConvLSTM memory cell is similar to that of the traditional LSTM, with convolution operations replacing the matrix operations in the LSTM. The dimension and size of the network input data match the result, and a time series with a step length of 1 day and a length of 10 is used in time. The output data is a matrix of size 45×512×512×3, where the first dimension is depth, the second dimension is longitude, the third dimension is latitude, and the fourth dimension is feature type. The output data is the three-dimensional ocean multi-factor reconstruction environment field at the target time.

[0123] In the embodiment, the training samples are randomly sampled from the HYCOM sea surface temperature field data from 2005 to 2015, with a total number of 100,000. In terms of time, samples are uniformly selected in 10 years with a step length of 1 month. In terms of space, 4 / 5 samples are randomly selected within the range of 50°S~50°N, and 1 / 5 samples are randomly selected in the experimental sea area to strengthen the feature learning of the experimental area.

[0124] In the sampling sea area, the temperature and salinity samples taken are three-dimensional matrices, whose three dimensions are longitude, latitude and depth, and the values ​​represent temperature and salinity, respectively. The matrix of 512×512×45 is linearly interpolated to obtain the temperature field and salinity field, and then the corresponding density is calculated according to the temperature, salinity and depth to obtain the density field. One of the methods for calculating density in the present invention is: ρ=1.025+0.0133(S-35)+0.00014(T-20)+0.000022P, where ρ represents density, S represents salinity, T represents temperature, and P represents atmospheric pressure.

[0125] Each component module of the hierarchical joint neural network is pre-trained separately. The convolutional neural network module A infers the reconstructed environmental field through the input environmental field. In this embodiment, in order to simulate the actual situation of ocean observation, the input environmental field is set as an array with a mask. The positions with observed values ​​correspond to valid values ​​in the input matrix, and the positions without observed values ​​correspond to NaN (Not a Number) data type in the matrix. Taking the ocean surface temperature as an example, the experimental results are shown in the following example. Figure 7 , Figure 8 , Fig. 9 As shown, Figure 7 is the true value of the environmental field, Figure 8 It is a virtual observation station. Fig. 9 It is the environmental field inferred by the neural network based on the virtual observation results. It can be seen that the inferred environmental field of the neural network is similar to the true value in the direction and distribution of the isotherms.

[0126] The function of convolutional neural network module B is similar to that of convolutional neural network module A and will not be described in detail.

[0127] The generative adversarial network module infers the input environment field of the shallow layer to obtain the output environment field of the middle and lower layers. Taking the temperature field of 200m inferred from the temperature field of 0m as an example, the experimental results are as follows: Fig.10 , Fig.11 , Fig.12 As shown, Fig.10 is the temperature field at 0m depth, Fig.11 is the true value of the temperature field at a depth of 200 m, Fig.12 It is the inference value of the temperature field at a depth of 200m. It can be seen that the inference result of the neural network is close to the true value in terms of the mean value and the rough distribution of high and low temperatures.

[0128] The recurrent neural network module infers the environmental field at the next moment through the input environmental field time series. Taking the sea surface temperature field as an example, the environmental field at time t+2 is inferred from the environmental fields at time t and t. The experimental results are shown in the following example. Fig.13 , such as 14, Fig.15 and Fig.16 , an example of the environmental field input of the recurrent neural network module at time t is as follows Fig.13 As shown, the environmental field input example of the recurrent neural network module at time t+1 is as follows Fig.14 As shown, the true value of the environmental field at time t+2 is as follows Fig.15 As shown, an example of the output of the environmental field inference value inferred by the recurrent neural network module at time t+2 is as follows Fig.16 The figure shows the change of a warm vortex in the lower left corner of the sea area over time, and the inference value at time t+2 is close to the true value.

[0129] After each module of the hierarchical joint neural network is completed and trained, the modules are combined to perform end-to-end fine-tuning training on the hierarchical joint neural network as a whole. The input of the hierarchical joint neural network is the three-dimensional virtual observation value of the marine environment field, and the output is the three-dimensional reconstruction result of the marine environment field. Taking the sea surface temperature field as an example, the neural network result is compared with the result of ordinary Kriging interpolation. The true value of the sea temperature field is as follows Fig.17 As shown in the figure, the virtual sampling stations in the experiment are as follows Fig.18 As shown, the hierarchical joint neural network reconstruction results are as follows Fig.19 As shown, the Kriging interpolation reconstruction result is as follows Fig. 20 As shown in the figure, it can be seen that the results of the hierarchical joint neural network reproduce the high and low temperature areas and isotherm shapes well. Figure 6 As shown, the average values ​​of the root mean square errors of the Kriging interpolation method and the hierarchical joint neural network results are 0.87 and 0.75, respectively. It can be seen that the error of the hierarchical joint neural network method result in the embodiment is smaller than that of the Kriging interpolation method.

[0130] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for rapid reconstruction of regional marine environment fields based on hierarchical joint neural networks, characterized in that: It includes obtaining multi-source ocean data and preprocessing it, calculating the density field from the preprocessed multi-source ocean data, and constructing a multi-factor coupling field based on the temperature field, salinity field, and density field; making a deep learning sample set, constructing a training hierarchical joint neural network, and using the trained hierarchical joint neural network to reconstruct the regional ocean environment field; The hierarchical joint neural network inputs shallow layer input data and middle and lower layer input data, the shallow layer input data is input into the convolutional neural network module A, the middle and lower layer input data and the reconstructed middle and lower layer environment field of the ocean are input into the convolutional neural network module B, the reconstructed shallow layer environment field of the ocean output by the convolutional neural network module A and the new reconstructed middle and lower layer environment field of the ocean output by the convolutional neural network module B are connected and input into the recurrent neural network module, and then the reconstructed environment field is output; The convolutional neural network module A is provided with branches, which are sequentially connected to the generative adversarial network module and the convolutional neural network module B; the input of the adversarial network module is to reconstruct the shallow surface environment field of the ocean, and the output is to reconstruct the middle and lower layer environment field of the ocean.

2. The method for rapid reconstruction of regional marine environment field based on hierarchical joint neural network according to claim 1 is characterized in that: The convolutional neural network module A and the convolutional neural network module B have the same structure.

3. The method for rapid reconstruction of regional marine environment field based on hierarchical joint neural network according to claim 2 is characterized in that: Convolutional neural network module A performs 4 modular processing at the input end. One modular processing includes first performing 2 3×3 convolutions and applying ReLU activation function, and then performing 2×2 maximum pooling; The modular process is copied 4 times to the output. After copying, the 2×2 maximum pooling becomes 2×2 upsampling. One modular process includes performing 2×2 upsampling, then performing 2 times of 3×3 convolution and applying the ReLU activation function. After performing 4 modular processes on the input side, 2 3×3 convolutions are performed and the ReLU activation function is applied, and then 4 replicated modular processes are performed on the output side; Finally, a 1×1 convolution is performed and output.

4. The method for rapid reconstruction of regional marine environment field based on hierarchical joint neural network according to claim 3 is characterized in that: The generative adversarial network module includes a generator and a discriminator. The shallow layer environment field is input into the generator to generate the environment field, which is then input into the discriminator together with the middle and lower layer environment fields as training samples, and finally the discrimination is performed.

5. The method for rapid reconstruction of regional marine environment field based on hierarchical joint neural network according to claim 4 is characterized in that: The input of the recurrent neural network is the environmental field input. Multiple environmental field inputs are respectively input into the corresponding 3D long short-term memory convolutional neural network, and then input into the hidden layer to finally form the environmental field output.

6. The method for rapid reconstruction of regional marine environment field based on hierarchical joint neural network according to claim 5 is characterized in that: Elements of multi-source ocean data include temperature data, salinity data, and density data; Types of multi-source ocean data include observational data, reanalysis data, and numerical simulation data.

7. The method for rapid reconstruction of regional marine environment field based on hierarchical joint neural network according to claim 6 is characterized in that: Preprocessing includes unified data format, depth correction, spatiotemporal interpolation and quality control.

8. The method for rapid reconstruction of regional marine environment field based on hierarchical joint neural network according to claim 7 is characterized in that: The deep learning sample set includes training set, validation set and test set.

Images