Construction method and application of marine environment data prediction model

Abstract

The invention discloses a construction method and application of a marine environment data prediction model, and belongs to the technical field of marine environment data prediction.The model comprises a plurality of single-layer ConvLSTM-SA modules and a dynamic gating network module; the plurality of single-layer ConvLSTM-SA modules are used for processing the data and then inputting the data into the dynamic gating network module; the single-layer ConvLSTM-SA module comprises a one-dimensional convolution layer, a causal convolution layer, a residual sum regular layer, a one-dimensional deconvolution layer and a ConvLSTM-SA layer, and data sequentially enters each layer in the single-layer ConvLSTM-SA module to be processed. The invention further provides application of the model, and the prediction precision can be improved when the model is used for marine environment data prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of marine environment data prediction, and in particular to a method for constructing a marine environment data prediction model and its application. Background Art

[0002] Existing marine environmental data prediction methods fall into two main categories: model-driven numerical simulation and deep learning-based spatiotemporal prediction. Model-driven numerical simulation uses mathematical models and computer technology to simulate and predict the behavior of complex systems. By solving discretized equations numerically, the evolution of marine environmental data over time is determined. While these methods are effective for small-scale data, traditional numerical simulation methods struggle to capture nonlinear relationships in large-scale spatiotemporal ocean data, potentially limiting their adaptability and accuracy.

[0003] Spatiotemporal prediction methods based on deep learning primarily utilize deep neural network models to model and predict spatiotemporal data. These methods have been widely applied in recent years in a variety of fields, including weather forecasting, traffic forecasting, and ocean dynamics simulation. For example, in 2018, Yoo-Geun Ham et al. published a study in Nature on the construction of a convolutional neural network model for spatiotemporal prediction of the multi-year El Niño / Southern Oscillation phenomenon. The study demonstrated that the CNN model significantly outperformed existing forecasting methods for the Nino 3.4 index. Furthermore, the deep learning-based forecasting method was able to better predict the detailed regional distribution of sea surface temperature, with a prediction period of up to one and a half years. Deep learning methods can automatically learn latent features in data through multi-layer neural networks and excel at handling nonlinear relationships, time series analysis, and complex pattern recognition. Therefore, deep learning technology has shown promising results in marine environmental data forecasting.

[0004] However, while current deep learning technologies include deep learning architectures specifically designed for processing time series data (such as LSTM and GRU), accurately capturing complex dynamics over long timescales remains a challenge. This is particularly true when considering seasonal variations, interannual variability, and even the impacts of long-term climate change. Therefore, accurately predicting the temporal and spatial trends of marine environmental data is a pressing issue for those skilled in the art. Summary of the Invention

[0005] In response to the above technical problems, the present invention provides a method for constructing and applying a marine environment data prediction model to improve the accuracy of marine environment data prediction.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: A method for constructing a marine environment data prediction model, the model comprising a plurality of single-layer ConvLSTM-SA modules and a dynamic gating network module; the plurality of single-layer ConvLSTM-SA modules process marine environment data and then input the data into the dynamic gating network module; the single-layer ConvLSTM-SA module comprises a one-dimensional convolution layer, a causal convolution layer, a residual and regularization layer, a one-dimensional deconvolution layer, and a ConvLSTM-SA layer; historical marine environment data is sequentially input into each layer of the single-layer ConvLSTM-SA module for processing and connected to the dynamic gating network module, thereby completing the training of the marine environment data prediction model.

[0007] Furthermore, in the multiple single-layer ConvLSTM-SA modules, each single-layer ConvLSTM-SA module uses a different convolution kernel to extract features from the marine environment data. The convolution kernel size of the single-layer ConvLSTM-SA module is , as follows: (1) Extraction of local spatiotemporal features of marine environment observation data: The historical marine environment observation data is obtained in the input layer based on the parameter information and time information obtained in the input coding layer. To further extract the spatiotemporal characteristics of the marine environment from the spatiotemporal data, a one-dimensional convolution is used to To extract local spatiotemporal features, the operation is as follows: ; in, is the vector embedding, is the local spatiotemporal feature, , moving step length , after the convolution operation, the feature vector dimension is ; (2) Extraction of global spatiotemporal features of marine environmental data, It only contains the spatiotemporal characteristics of the local period of the marine environment. Through the causal convolution module, the spatiotemporal characteristics of marine environmental observation data at all time scales are extracted; ; in, is the spatiotemporal feature after causal convolution; In order to ensure that the output features of the convolutional layer are consistent with the input in time or space, Perform padding operation; after activation layer and Dropout, use tanh as activation function to perform nonlinear transformation; ; Introducing residual connection, one-dimensional deconvolution obtains global spatiotemporal features Global spatiotemporal features : ; Global features are a form of representation that fuses together information from various dimensions in the original data. Global spatiotemporal features It is a two-dimensional data of size H×W, with a C-dimensional vector at each position, which represents the abstract representation of the spatial position at T time steps. Through the reshape operation, it is organized into the input tensor form acceptable to ConvLSTM: For global spatiotemporal features , each time step , spatial location , both have eigenvectors , then construct the spatial features of a single time step: ; Stack all time steps to form the input for time step t: ; Finally, add the batch dimension to get the input tensor: ; Among them, B represents the batch size, T represents the number of time steps, H represents the height, W represents the width, and C represents the number of channels input at each time step.

[0008] The input tensor is fed into the ConvLSTM-SA layer, and the expression is as follows: ; Where, represents the input of the current time step, Represents the hidden state of the previous step; Represents the memory gate input of ConvLSTM-SA at time t, represents the sigmoid activation function, Represents the convolution input at time t The memory gate convolution kernel weights, Indicates the hidden state at time t-1 The convolution kernel weight, * represents the convolution operation, represents the memory gate bias term; Represents the forget gate input of ConvLSTM-SA at time t, Represents the convolution input at time t The weight of the forget gate convolution kernel, Represents the convolution input at time t The weight of the forget gate convolution kernel, represents the memory gate bias term; is a candidate cell state, Represents the convolution input at time t The weight of the forget gate convolution kernel, Indicates the hidden state at time t-1 The convolution kernel weight, * represents the convolution operation, represents the memory gate bias term; Indicates the cell renewal status, Represents the multiplication of the corresponding elements of the matrix, forget gate Determines how much information is discarded, the input gate and candidate cells Determine how many cells are added; is the calculated new hidden state, which will be the input for the next time step and can be used to generate the output; Represents the output gate at time step t, which is a part of Convlstm.

[0009] The expression of the dynamic gating network module is as follows: The dynamic gating network module controls the flow of information through the gating mechanism to solve the long-term dependency problem. The dynamic gating network module is represented by the following expression: ; ; ; The input is processed by tanh and sigmoid to map it to [-1, 1]. Finally, the output is as follows: = ); ; in, It is the reset gate. It is the update gate. is the Sigmoid activation function, is the intermediate hidden state, combined with the reset gate and the candidate state to generate the intermediate hidden state, is the proportion of the previous hidden state retained, The proportion of candidates joining the state, is the output gate in the dynamic gated network module and the output of the multi-layer self-attention mechanism. is the hidden state at the previous moment, is the bias term, is a learnable weight matrix, where represents the weight matrix of input and hidden state, It is a new hidden state determined by the gating mechanism module and the self-attention output.

[0010] The model constructed using the method is applied in the prediction of marine environmental data, wherein the marine environmental data set to be predicted is input into the trained marine environmental data prediction model to obtain the marine environmental data prediction result.

[0011] Three encoding methods are used to perform position encoding, parameter encoding and time encoding on the acquired marine environment dataset and use it as input; the historical spatiotemporal dataset is input into a multi-layer ConvLSTM-SA network model for training, and the mean square error is used as the loss function of the model to obtain the initial training weights, use the initial weights to make predictions, and evaluate the model performance; the mean square error (MSE) is used as the loss function; and a trained model is obtained.

[0012] Furthermore, the three encoding methods are used to perform position encoding, parameter encoding and time encoding on the acquired marine environment dataset and use them as input. The specific formula is: ; As the representation vector after the historical numerical input is filled with 0, is the information encoding of the input data, It is a time series code consisting of years and months. The formula of the encoding network is as follows: ; ; ; A one-dimensional convolutional network is used as the value encoding network, which can effectively extract the local correlation between input features; is the number of input features, The output channel is the feature dimension generated after convolution processing; the temporal coding network It consists of 2 fully connected layers, each with an input dimension of 1 and an output dimension of .

[0013] The beneficial effects of the present invention compared with the existing technology: the multi-layer ConvLSTM-SA network model proposed in the present invention can improve the prediction accuracy for predicting marine environmental data. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is an overall flow chart of the method provided by an embodiment of the present invention; Figure 2This is a schematic diagram of the overall structure of the multi-layer ConvLSTM-SA network model provided by an embodiment of the present invention; Figure 3 Schematic diagram of the single-layer ConvLSTM-SA network model structure provided by an embodiment of the present invention; Figure 4 It is a schematic diagram of the SA network module structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The technical solution of the present invention is further explained below through examples, but the protection scope of the present invention is not limited in any form by the examples.

[0016] Example 1 like Figure 1-4 As shown, a marine environment data prediction model includes multiple single-layer ConvLSTM-SA modules and a dynamic gating network module; the multiple single-layer ConvLSTM-SA modules process the marine environment data and then input it into the dynamic gating network module; the single-layer ConvLSTM-SA module includes a one-dimensional convolution layer, a causal convolution layer, a residual and regularization layer, a one-dimensional deconvolution layer and a ConvLSTM-SA layer, and the historical marine environment data is sequentially input into each layer of the single-layer ConvLSTM-SA module for processing and connected to the dynamic gating network module, thereby completing the training of the marine environment data prediction model.

[0017] As a preferred embodiment, in the multiple single-layer ConvLSTM-SA modules, each single-layer ConvLSTM-SA module uses a different convolution kernel to extract features from the marine environment data. The convolution kernel size of the single-layer ConvLSTM-SA module is , as follows: (1) Extraction of local spatiotemporal features of marine environmental data: Historical observation of marine environmental data is obtained in the input layer based on the parameter information and time information obtained in the input coding layer. To further extract periodic spatiotemporal features from spatiotemporal ocean environment data, a one-dimensional convolution is used to To extract local spatiotemporal features, the operation is as follows: ; in, is the vector embedding, is the local spatiotemporal feature, , moving step length , after the convolution operation, the feature vector dimension is ; (2) Extraction of global spatiotemporal features of marine environmental data, It only contains the spatiotemporal features of the local period. Extract spatiotemporal features on all time scales through the causal convolution module; in, is the spatiotemporal feature after causal convolution; in order to ensure that the output features of the convolution layer are consistent with the input in time or space dimensions, Perform padding operation; after activation layer and Dropout, use tanh as activation function to perform nonlinear transformation; ; Introducing residual connection, one-dimensional deconvolution obtains global spatiotemporal features Global spatiotemporal features : ; Global features are a form of representation that fuses together information from various dimensions in the original data. Global spatiotemporal features It is a two-dimensional data of size H×W, with a C-dimensional vector at each position, which represents the abstract representation of the spatial position at T time steps. Through the reshape operation, it is organized into the input tensor form acceptable to ConvLSTM: For global spatiotemporal features , each time step , spatial location , both have eigenvectors , then construct the spatial features of a single time step: ; Stack all time steps to form the input for time step t: ; Finally, add the batch dimension to get the input tensor: ; Among them, B represents the batch size, T represents the number of time steps, H represents the height, W represents the width, and C represents the number of channels input at each time step.

[0018] The input tensor is fed into the ConvLSTM-SA layer, and the expression is as follows: ; Where, represents the input of the current time step, Represents the hidden state of the previous step; Represents the memory gate input of ConvLSTM-SA at time t, represents the sigmoid activation function, Represents the convolution input at time t The memory gate convolution kernel weights, Indicates the hidden state at time t-1 The convolution kernel weight, × represents the convolution operation, represents the memory gate bias term; Represents the forget gate input of ConvLSTM-SA at time t, Represents the convolution input at time t The weight of the forget gate convolution kernel, Represents the convolution input at time t The weight of the forget gate convolution kernel, represents the memory gate bias term; is a candidate cell state, Represents the convolution input at time t The weight of the forget gate convolution kernel, Indicates the hidden state at time t-1 The convolution kernel weight, × represents the convolution operation, represents the memory gate bias term; Indicates the cell renewal status, Represents the multiplication of the corresponding elements of the matrix, forget gate Determines how much information is discarded, the input gate and candidate cells Determine how many cells are added; is the calculated new hidden state, which will be the input for the next time step and can be used to generate the output; Represents the output gate at time step t, which is a part of Convlstm.

[0019] The SA is a memory-based self-attention module, which is expressed by the following expression: ; is the input hidden state at the current time t, 、 、 are weight matrices used to generate the value vector (ValueVector), key vector (Key Vector) and query vector (Query Vector), respectively. is the query vector, is the key vector, is a vector of values.

[0020] Dot product calculation: query vector With key vector Multiply the transpose of to get a score matrix. This score matrix reflects the similarity between the query vector and each key vector.

[0021] Scaling operation: To prevent the dot product result from being too large and causing the softmax function to saturate, the dot product result is divided by d, where d is the dimension of the vector. Apply the softmax function to the scaled score matrix to convert it into a probability distribution, i.e., the attention score ,These scores indicate the importance of different positions.

[0022] Z is a weighted value vector, obtained by multiplying the attention score by the value vector. This step weights the value vector according to the attention score, allowing the model to focus on the more important parts of the input sequence. Z contains the information processed by the attention mechanism. In a multi-head attention mechanism, multiple such heads are computed in parallel, and their outputs are concatenated or averaged to form the final multi-head attention output.

[0023] The expression of the dynamic gating network module is as follows: The dynamic gating network module controls the flow of information through the gating mechanism to solve the long-term dependency problem. The dynamic gating network module is represented by the following expression: ; ; ; The input is processed by tanh and sigmoid to map it to [-1, 1]. Finally, the output is as follows: = ); ; in, It is the reset gate. It is the update gate. is the Sigmoid activation function, is the intermediate hidden state, combined with the reset gate and the candidate state to generate the intermediate hidden state, is the proportion of the previous hidden state retained, The proportion of candidates joining the state, is the output gate in the dynamic gated network module and the output of the multi-layer self-attention mechanism. is the hidden state at the previous moment, is the bias term, is a learnable weight matrix, where represents the weight matrix of input and hidden state, It is a new hidden state determined by the gating mechanism module and the self-attention output.

[0024] Example 2: A method for constructing the model described in Example 1 using marine environmental data. In this embodiment, it is necessary to construct a model for predicting the chlorophyll content in Bohai Sea, which includes the following steps: Step 1: Obtain the historical dataset of marine chlorophyll content in the Bohai Sea area; Step 2: Input the historical data set into the multi-layer ConvLSTM-SA network model for training; Specifically, the step 2: 1.1 Obtain the historical dataset of marine chlorophyll content in the Bohai Sea area. Use three encoding methods to perform position encoding, parameter encoding, and time encoding on the Bohai Sea marine environmental factor dataset and use it as input: ; As the representation vector after the historical numerical input is filled with 0, is the information encoding of the input data, It is a time series encoding, consisting of years and months. The formula of the encoding network is as follows: ; ; ; A one-dimensional convolutional network is used as the value encoding network, which can effectively extract the local correlation between input features. is the number of input features, The output channel is the feature dimension generated after convolution processing; the temporal coding network It consists of 2 fully connected layers, each with an input dimension of 1 and an output dimension of ,Through this encoding structure, the model can simultaneously consider the temporal relationship ,and time information of the input features, thereby improving the ,understanding and prediction capabilities of spatiotemporal data.

[0025] 1.2 Using different convolution kernels to process multiple parameters and time information containing historical observation data For feature extraction, each The convolution kernel size is .

[0026] (1) Local spatiotemporal feature extraction Contains the parameter information and time information of historical observation data. The input sequence has further extracted periodic spatiotemporal features from the spatiotemporal data based on the input encoding layer in the input layer. The convolution operation is as follows: ; in , moving step length , after the convolution operation, the feature vector dimension is .

[0027] (2) Global spatiotemporal feature extraction It only contains spatiotemporal features on the local period, and extracts spatiotemporal features on all time scales through the causal convolution module: ; In order to ensure that the output features of the convolutional layer are consistent with the input in time or space, Perform padding operation. After activation layer and Dropout, and use tanh as activation function, perform nonlinear transformation: ; Introducing residual connection, one-dimensional deconvolution obtains global spatiotemporal features Global spatiotemporal features .

[0028] The global spatiotemporal features are input into a multi-layer ConvLSTM-SA module. The ConvLSTM-SA module is a memory-based self-attention module connected after the ConvLSTM. The ConvLSTM-SA module specifically comprises: multiple single-layer ConvLSTM-SA modules and a dynamic gating network module. The SA module is a memory-based self-attention module connected after the ConvLSTM. The self-attention module undergoes feature aggregation, memory update, and a dynamic gating network module to produce the final output.

[0029] The historical spatiotemporal dataset is fed into a multi-layer ConvLSTM-SA network model for training. The mean squared error (MSE) is used as the model's loss function to obtain initial training weights. The initial weights are used for prediction and to evaluate model performance. The mean squared error (MSE) is used as the loss function. The expression is as follows: .

[0030] Example 3 uses the model constructed in Example 2 to predict the chlorophyll content in the ocean; The ocean chlorophyll data is obtained through the corresponding equipment. After the multi-layer ConvLSTM-SA is used to extract the spatiotemporal related information from the ocean data to be predicted, the pooling layer is used to reduce the dimension of the spatiotemporal feature tensor. After the deconvolution layer, the fully connected layer is used to output the future ocean chlorophyll content prediction value.

[0031] In this example, to test the prediction performance of the multi-layer ConvLSTM-SA model of the present invention, the common BP network model and CNN network model were selected for comparison. The mean absolute error (MAE) and root mean square error (RMSE) were used as evaluation indicators to evaluate the prediction results of the multi-layer ConvLSTM-SA network model. After comparison, the multi-layer ConvLSTM-SA network model proposed in this invention can improve the prediction accuracy of marine chlorophyll content. The specific calculation results are shown in Figure 1. Table 1 Comparative experimental results of different models .

Claims

1. A method for constructing a marine environment data prediction model, characterized in that: The model includes multiple single-layer ConvLSTM-SA modules and a dynamic gating network module; the multiple single-layer ConvLSTM-SA modules process the marine environment data and then input it into the dynamic gating network module; the single-layer ConvLSTM-SA module includes a one-dimensional convolution layer, a causal convolution layer, a residual and regularization layer, a one-dimensional deconvolution layer and a ConvLSTM-SA layer. The construction method is to sequentially input the marine environment data into each layer of the single-layer ConvLSTM-SA module for processing and connect it to the dynamic gating network module, thereby completing the training of the marine environment data prediction model.

2. The method for constructing a marine environment data prediction model according to claim 1, characterized in that: In the multiple single-layer ConvLSTM-SA modules, each single-layer ConvLSTM-SA module uses a different convolution kernel to extract features from the marine environment data. The convolution kernel size of the single-layer ConvLSTM-SA module is: , the specific construction method is as follows: (1) Extraction of local spatiotemporal features of marine environment observation data: The marine environment observation data is obtained in the input layer based on the parameter information and time information obtained in the input coding layer. To further extract the spatiotemporal characteristics of the marine environment from the spatiotemporal data, a one-dimensional convolution is used to To extract local spatiotemporal features, the operation is as follows: ； in, is the vector embedding, is the local spatiotemporal feature, , moving step length , after the convolution operation, the feature vector dimension is ; (2) Extraction of global spatiotemporal features of marine environmental data It only contains the spatiotemporal characteristics of the local period of the marine environment. Through the causal convolution module, the spatiotemporal characteristics of marine environmental observation data at all time scales are extracted; ； in, is the spatiotemporal feature after causal convolution; In order to ensure that the output features of the convolutional layer are consistent with the input in time or space, Perform padding operation; after activation layer and Dropout, use tanh as activation function to perform nonlinear transformation; ； Introducing residual connection, one-dimensional deconvolution obtains global spatiotemporal features Global spatiotemporal features : ； Global features are a form of representation that combines information from various dimensions in the original data; global spatiotemporal features It is a two-dimensional data of H×W, with a C-dimensional vector at each position, which represents the abstract representation of the spatial position at T time steps; it is reshaped and organized into the input tensor form acceptable to ConvLSTM: For global spatiotemporal features , each time step , spatial location , both have eigenvectors ,So: Construct the spatial features for a single time step: ； Stack all time steps to form the input for time step t: ； Finally, add the batch dimension to get the input tensor: ； Where B represents the batch size, T represents the number of time steps, H represents the height, W represents the width, and C represents the number of channels input at each time step; The input tensor is fed into the ConvLSTM-SA layer, and the expression is as follows: ； Where, represents the input of the current time step, Represents the hidden state of the previous step; Represents the memory gate input of ConvLSTM-SA at time t, represents the sigmoid activation function, Represents the convolution input at time t The memory gate convolution kernel weights, Indicates the hidden state at time t-1 The convolution kernel weight, × represents the convolution operation, represents the memory gate bias term; Represents the forget gate input of ConvLSTM-SA at time t, Represents the convolution input at time t The weight of the forget gate convolution kernel, Represents the convolution input at time t The weight of the forget gate convolution kernel, represents the memory gate bias term; is a candidate cell state, Represents the convolution input at time t The weight of the forget gate convolution kernel, Indicates the hidden state at time t-1 The convolution kernel weight, × represents the convolution operation, represents the memory gate bias term; Indicates the cell renewal status, Represents the multiplication of the corresponding elements of the matrix, forget gate Determines how much information is discarded, the input gate and candidate cells Determine how many cells are added; is the calculated new hidden state, which will be the input for the next time step and can be used to generate the output; Represents the output gate at time step t, which is a part of Convlstm.

3. The method for constructing a marine environment data prediction model according to claim 1, characterized in that: The expression of the dynamic gating network module is as follows: The dynamic gating network module controls the flow of information through the gating mechanism to solve the long-term dependency problem; the dynamic gating network module is represented by the following expression: ； ； ； The input is processed by tanh and sigmoid to map it to [-1, 1]; finally, the output is as follows: = ）； ； in, It is the reset gate. It is the update gate. is the Sigmoid activation function, is the intermediate hidden state, combined with the reset gate and the candidate state to generate the intermediate hidden state, is the proportion of the previous hidden state retained, The proportion of candidates joining the state, is the output gate in the dynamic gated network module and the output of the multi-layer self-attention mechanism. is the hidden state at the previous moment, is the bias term, is a learnable weight matrix, where represents the weight matrix of input and hidden state, It is a new hidden state determined by the gating mechanism module and the self-attention output.

4. The method for constructing a marine environment data prediction model according to claim 1, characterized in that: Three encoding methods are used to perform position encoding, parameter encoding and time encoding on the acquired marine environment dataset and use it as input; the historical spatiotemporal dataset is input into the marine environment data prediction model for training, the mean square error is used as the loss function of the model, the initial training weights are obtained, the initial weights are used for prediction, and the model performance is evaluated; the mean square error (MSE) is used as the loss function; and a trained marine environment data prediction model is obtained.

5. The method for constructing a marine environment data prediction model according to claim 4, characterized in that: The three encoding methods are used to perform position encoding, parameter encoding and time encoding on the acquired marine environment dataset and use them as input. The specific formula is: ; As the representation vector after the historical numerical input is filled with 0, is the information encoding of the input data, It is a time series code consisting of years and months. The formula of the encoding network is as follows: ； ； ； A one-dimensional convolutional network is used as the value encoding network, which can effectively extract the local correlation between input features; is the number of input features, The output channel is the feature dimension generated after convolution processing; the temporal coding network It consists of 2 fully connected layers, each with an input dimension of 1 and an output dimension of .

6. Application of the model constructed by the method according to any one of claims 1 to 5 in the prediction of marine environmental data, characterized in that: The application is to input the ocean environment data set to be predicted into a trained ocean environment prediction model to obtain the ocean environment data prediction result.

Images