Sea surface temperature prediction method based on TCN neural network

Abstract

Along with the development of marine ecological environment protection and aquaculture planning, the invention provides a sea temperature prediction method based on a TCN neural network for the change of an important factor marine surface temperature (SST) and the prediction problem thereof. According to the method, a TCN neural network, an improved attention module and a convolutional bidirectional gated recurrent unit network (CNN-Bi-GRU) are combined. The method comprises the following steps: firstly, extracting local features of SST through TCN; then, according to the time sequence characteristics of the SST, weights are distributed for temperature changes of different areas in the network by utilizing a self-attention mechanism; then, modeling the input sequence from the front direction and the back direction by means of the Bi-GRU; according to the method, the future SST value is predicted by using the multi-dimensional information of the historical data, so that the prediction accuracy is improved.

Description

Technical Field

[0001] The present invention relates to a method for predicting ocean surface temperature based on a TCN neural network, which realizes accurate prediction of ocean temperature through sea surface temperature (SST) time series data. Background Art

[0002] As a key physical parameter in the marine environment, sea surface temperature has a profound impact on the balance of marine ecosystems and the distribution and migration patterns of fishery resources. Accurate prediction of sea surface temperature is of great significance to many fields, including fishery resource management, marine biodiversity protection, climate change research, and marine economic activities.

[0003] Changes in sea surface temperature directly affect the distribution of primary productivity in the ocean, and thus affect the entire marine food chain. In the field of fishery resource management, the prediction of sea surface temperature helps to assess the abundance and distribution of fishery resources and provide a scientific basis for fishing activities. In addition, abnormal increases or decreases in sea surface temperature often lead to

[0004] It is often related to marine disaster events, such as coral bleaching and red tides. Accurate prediction can provide early warning information for marine ecological protection and disaster prevention.

[0005] Traditional sea surface temperature prediction methods mainly include statistical models, dynamic models and hybrid models. Statistical models rely on historical data for prediction, but often lack a deep understanding of physical processes. Dynamic models are built based on physical equations and can simulate complex ocean processes, but they are computationally expensive and sensitive to initial conditions. Hybrid models combine the advantages of statistical and dynamic models, but the model construction and parameter adjustment process are complicated.

[0006] These methods face challenges in practical applications, such as inconsistent data quality, insufficient prediction accuracy, and low computational efficiency. With the development of ocean observation technology and the advancement of data science, a new prediction method is urgently needed to overcome these challenges. Summary of the invention

[0007] This paper aims to construct an intelligent prediction model for marine ecological environment elements based on remote sensing big data. This paper proposes an innovative prediction model for the key parameter of sea surface temperature (SST). The model uses a temporal convolutional network (TCN) to process and predict the changes in sea surface temperature in remote sensing big data, and predicts the sea surface temperature in different sea areas. The model shows excellent prediction ability and broad application prospects.

[0008] The present invention firstly introduces causal convolution and dilated convolution and combines residual connection to capture the long-term dependency of time series data and maintain causality. The long-term dependency in the sequence data is captured through the TCN layer, and the information in the time dimension is retained; then, the convolutional bidirectional gated recurrent unit network (CNN-Bi-GRU) receives the data sequence processed by TCN, and uses its bidirectional structure to deeply analyze the time series relationship of the sea temperature time series data, and obtains the correlation characteristics of time, longitude and latitude and temperature; then, the improved attention module quantifies the feature weights of the CNN-Bi-GRU output data, and further learns the correlation between the logging data; finally, the weighted features are input into the fully connected layer for feature dimensionality reduction, and the predicted sea temperature information is output. The evaluation indicators are root mean square error RMSE, mean absolute error MAE, determination coefficient R2, and mean absolute percentage error MAPE.

[0009] The TCN neural network construction includes the following steps:

[0010] (1) Causal convolution: The length of each hidden layer is the same as the length of the input layer, and zero padding is used to ensure that subsequent layers have the same length. The value of the previous layer at time t only depends on the value of the next layer at time t and before, that is, future information is not considered, which ensures the causal temporal nature of the data when feature information is extracted. The modeling length of causal convolution for time is limited by the size of the convolution kernel. To obtain a deeper receptive field, a multi-layer network or a larger convolution kernel is required, but the increase in the number of network layers and the increase in the size of the convolution kernel are likely to increase the network training time;

[0011] (2) Dilated convolution: There are still some problems with simple causal convolution, such as difficulty in capturing dependencies between longer time points. Dilated convolution allows for intermittent sampling of the convolution time point input, and the sampling rate is controlled by the parameter Dilate. For example, when Dilate is 2, it means that one sample is taken for every 2 points as input. The higher the level, the larger the Dilate used. Therefore, dilated convolution can expand the effective window, so that a smaller number of layers can be used to obtain a larger receptive field.

[0012] For a one-dimensional input sequence x∈R and a filter f3, the dilated convolution operation F at time is

[0013] Where d is the dilation factor Dilate, k is the convolution kernel size, and t-di represents the direction of historical information. When d = 1, it is a normal convolution.

[0014] (3) Residual module: In order to solve the gradient vanishing or exploding problem caused by the introduction of dilated causal convolution, a residual block is introduced in the output layer of TCN. Its principle is to fuse the model input x with the output F(x). This allows the network to learn not only the mapping relationship between input data and output data, but also the residual between input data and output data, and finally obtain the output y of TCN. The formula is as follows:

[0015] y=Activation(x+F(x)) (2)

[0016] The residual module of TCN, which is the activation function, includes two layers of dilated convolution and ReLU functions, and the weights of the convolution kernels are all normalized. The input data skips the intermediate convolution operation, and is directly added to the data after the intermediate convolution operation after 1×1 convolution processing, and the result is used as the output of this layer. The skip layer connection directly connects the feature map of the lower layer to the upper layer. In order to prevent the corresponding number of channels from being inconsistent, 1×1 convolution is used to merge elements to ensure that the two dimensions are the same size. The residual module ensures that each layer can efficiently learn the relationship between the identity mapping, which is very effective in networks with deeper layers. TCN uses a 1×1 convolution kernel as the identity mapping to eliminate the possible difference in width between the residual block input and the residual connection output.

[0017] Furthermore, the attention mechanism includes the following contents:

[0018] The feature attention mechanism is used in the input feature extraction process of the TCN network. The structure of feature attention is as follows Figure 4 As shown in Figure 2, the feature attention is located in the shallow layer of the model, and the input feature data is usually concentrated in a certain numerical range. In this paper, the attention weight is mapped to the interval (0, 1) through the softmax function to improve the comprehensiveness of the weight, and the obtained attention weight coefficients are normalized to obtain the feature attention weight α t =[α 1,t ,α 2,t ,α m,t ,....,α M,t ], where α m,t is the attention weight of the mth feature, denoted as .

[0019]

[0020] x' t =α t ·x t =[α 1,t x 1,t ,α 2,t x 2,t ,....,α M,t x M,t ] (4)

[0021] Then the input feature vector x t Recalculate the weighted vector x' t . Where · represents the Hadamard product. Taking the tth time node as an example, the single time step input feature vector x containing M features t =[x 1,t ,x 2,t ,...,x M,t ]. First, obtain the feature vector of the one-dimensional convolution output, and then input the feature vector into the fully connected layer to obtain the attention weight of each feature. Then, multiply the attention weight and the original feature sequence to obtain the feature sequence after attention screening. Feature attention considers the correlation between the input features and the output labels, strengthens the key factors affecting the prediction, and weakens the less relevant factors.

[0022] Furthermore, the evaluation indicators are as follows:

[0023] The evaluation index formula is as follows: N is the total number of samples, and parameter i represents the i-th sample. Pred is the predicted value, and Ture is the actual value. The root mean square error is the arithmetic square root of the mean square error, which can intuitively observe the degree of dispersion between the predicted value and the actual value. The calculation formula is as follows:

[0024]

[0025] The mean absolute error is the sum of the absolute values ​​of the differences between the predicted values ​​and the actual values. It only measures the average magnitude of the errors in the predicted values, regardless of direction:

[0026]

[0027] The coefficient of determination R reflects how much the fluctuation of the dependent variable is affected by the fluctuation of the independent variable. The closer the value is to 1, the better the model fit is. The calculation formula is as follows:

[0028]

[0029] The mean absolute percentage error not only considers the error between the predicted value and the true value, but also the ratio between the deviation and the true value. The calculation formula is as follows:

[0030] BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is the TCN model framework of the present invention;

[0032] Figure 2 This is a structural diagram of the residual module of the present invention;

[0033] Figure 3 This is a characteristic attention framework diagram of the present invention;

[0034] Figure 4 It is a network diagram of a convolutional bidirectional gated recurrent unit of the present invention;

[0035] Figure 5 It is the loss function discount diagram of the present invention;

[0036] Figure 6 It is a fitting diagram of the prediction and actual results of the present invention;

[0037] Figure 7 This is a sea temperature sea area map predicted by the present invention.

[0038] Figure 8 The experimental results are shown in the figure. DETAILED DESCRIPTION

[0039] The initial time step set in the experiment of the present invention is 12, the prediction type (-feature) is set to 'M', indicating multivariate prediction of multivariate; the dimension of the TCN layer (-model_dim) is set to [64, 128, 256, 512, 1024], which defines the number of layers of TCN and the width of each layer; the learning rate (-lr) is set to 0.00001; the random drop probability (-drop_out) is 0.5 to prevent overfitting; the training round (-epochs) is set to 10; the batch size (-batch_size) is 4; and the model save path (-save_path) is specified as the 'models' directory. The selection of these parameters aims to balance the complexity of the model with the training efficiency, while avoiding overfitting, so as to obtain better results in the experiment.

[0040] This model uses four data sets, namely the sea temperature remote sensing data of the eastern seas of my country, Bohai Sea, Yellow Sea and East China Sea. The range selected for the eastern coastal area is 20°0′~40°0′N, 117°0′~130°0′E; the range selected for Bohai Sea is 37°0′~41°0′N, 117°0′~123°0′E; the range selected for the Yellow Sea is 33°0′~40°0′N, 119°0′~127°0′E; the range selected for the East China Sea is 22°0′~34°0′N, 117°0′~131°0′E.

[0041] As shown in the following table:

[0042] The practical application effect of this model is shown through four charts: training loss function, model fitting and actual prediction:

[0043] This model uses MAE as a parameter to make judgments and predictions. The results of 200 rounds are as follows Figure 5 As shown in the figure, the vertical axis represents the loss value and the horizontal axis represents the training cycle. It can be seen that as the training cycle increases, the loss value shows a downward trend, which indicates that the model is gradually improving. To avoid fitting, the random dropout probability is set to 0.5, and sometimes there will be a short rise and then a decrease, which is an "oscillation" situation. Overall, despite the fluctuations, the loss value still shows a clear downward trend, which indicates that the model is constantly optimizing.

[0044] To verify the difference between the model prediction results and the original image, Figure 6 As shown in the figure, the model fits the sea temperature forecast. The blue line represents historical data and the orange line represents predicted data. It can be seen that the prediction curve of the model roughly follows the trend of historical data, but there are certain deviations in certain time periods. This may mean that the prediction accuracy of the model is lower at certain times and better at other times. Overall, the model is able to capture the basic trend of sea temperature changes, but the match in details is not perfect. It should be noted that since sea temperature is affected by many factors, including seasonality, climate patterns and other external influences, it is very difficult to make a completely accurate prediction. This figure can help evaluate the performance of the model and provide a reference for further optimization.

[0045] The representative forecast results for the next five months in the eastern sea area (117°~130°E, 20°~41°N) are as follows, taking January, March and May as examples. Figure 8 As shown, the original image is the data image of June 2024, and the last 1 month, 3 months and 5 months are the predicted remote sensing images of July 2024, September 2024 and November 2024 respectively. It can be seen that the smoothness of the predicted image and the prediction results have achieved good performance.

Claims

1. A sea temperature prediction method based on TCN neural network, It is characterized in that The method includes: firstly, introducing causal convolution and dilated convolution and combining residual connection to capture the long-term dependency of time series data and maintain causality. The long-term dependency in the sequence data is captured through the TCN layer, and the information in the time dimension is retained; then, the convolutional bidirectional gated recurrent unit network (CNN-Bi-GRU) receives the data sequence processed by TCN, and uses its bidirectional structure to deeply analyze the time series relationship of the sea temperature time series data, and obtain the correlation characteristics of time, longitude and latitude and temperature; then, the improved attention module quantifies the feature weights of the CNN-Bi-GRU output data, and further learns the correlation between the logging data; finally, the weighted features are input into the fully connected layer for feature dimensionality reduction, and the predicted sea temperature information is output. The evaluation indicators are root mean square error RMSE, mean absolute error MAE, determination coefficient R2, and mean absolute percentage error MAPE.

2. According to claim 1, introducing causal convolution and dilated convolution and combining residual connection, Features The residual connection module: When the filter size k and the dilation factor D remain unchanged, the introduction of the residual network can expand the network's receptive field and achieve deeper training. The convolution operation of each layer is replaced by a general residual module, as shown below: X i+1 =Activation(X i +F(X i )) (1) 3. The TCN temporal convolutional network according to claim 1, Features The causal convolution, dilated convolution and no offset padding: The causal convolution is used to ensure that only current and past information is used when predicting future values, without information leakage. The receptive field is increased without losing temporal resolution through dilated convolutions. Dilated convolutions are implemented by applying convolution kernels at intervals. In order to keep the time length of the output the same as the input, the network uses no offset padding before the convolution operation to maintain time alignment when processing sequence data.

4. The convolutional bidirectional gated recurrent unit network (CNN-Bi-GRU) according to claim 1, Features The introduced CNN-Bi-GRU structure can extract local features from both positive and negative directions to improve the global performance of the network. The CNN is used to extract features of preprocessed sensor data. Secondly, the extracted features and labels are fed to the Bi-GRU network.

5. The improved attention module according to claim 1, Features Improve the model's ability to focus on key information through weight distribution. The feature attention is located in the shallow layer of the model, and the input feature data is usually concentrated in a certain numerical range. This paper maps the attention weight to the (0,1) interval through the softmax function to improve the comprehensiveness of the weight, and normalizes the obtained attention weight coefficients to obtain the feature attention weight.

6. According to claim 1, the root mean square error RMSE, mean absolute error MAE, determination coefficient R2, mean absolute percentage error MAPE, specifically include: The evaluation index formula is as follows: Where N is the total number of samples, and parameter i represents the i-th sample. Pred is the predicted value, and Ture is the actual value. The root mean square error is the arithmetic square root of the mean square error, which can intuitively observe the degree of dispersion between the predicted value and the actual value. The calculation formula is as follows: The mean absolute error is the sum of the absolute values ​​of the differences between the predicted values ​​and the actual values. It only measures the average magnitude of the errors in the predicted values, regardless of direction: The coefficient of determination R reflects how much the fluctuation of the dependent variable is affected by the fluctuation of the independent variable. The closer the value is to 1, the better the model fit is. The calculation formula is as follows: The mean absolute percentage error not only considers the error between the predicted value and the true value, but also the ratio between the deviation and the true value. The calculation formula is as follows:

Images