Surface current prediction method based on attention mechanism TCN-LSTM model

Abstract

The present application belongs to the technical field of marine surveying and mapping, and is a surface current prediction method based on an attention mechanism TCN-LSTM model, comprising the following steps: (1) collecting real surface current data through a data acquisition module, (2) processing surface current data using an experimental data processing module, (3) constructing an attention mechanism TCN-LSTM surface current prediction model using a model building module, (4) verifying and testing the surface current data, and (5) predicting the neural network model. The present application method utilizes the characteristics of long short-term memory networks and the advantages of time convolution, and simultaneously adds an attention mechanism, and predicts after processing real historical marine data of a certain sea area measured by a WaMoSII system. The method proposed by the present application has a surface current flow velocity accuracy of 92.9% and a flow direction accuracy of 90.4%.

Description

TECHNICAL FIELD

[0001] The present application relates to a surface current prediction method based on an attention mechanism TCN-LSTM model, and belongs to the technical field of marine surveying and mapping. BACKGROUND

[0002] A sea current is the flow of seawater in the ocean, which generally moves in a certain direction on a large scale. Sea currents can be divided into two types: surface currents and deep currents. Surface currents are mainly driven by wind, atmospheric pressure and ocean topography, and usually have a depth of more than 200 meters. The characteristics of surface currents mainly include flow velocity and flow direction, which represent the strength and movement direction of surface currents. The flow velocity of surface currents is generally from 0 cm / s to 300 cm / s, which is smaller than that of deep currents. The change of surface currents plays an important role in the ocean. Although surface currents have a positive effect, they can also cause certain harm to the environment, economy and society. Therefore, the research on surface current prediction has wide application value.

[0003] Surface currents can be observed in real time by instruments. The floating method is a traditional observation method, and the commonly used instruments for observing surface currents are electromagnetic current meters and acoustic Doppler current profilers, which can realize long-term observation of fixed-point and underway flow velocities, respectively. Surface currents can also be predicted. The prediction of the characteristics of surface currents is roughly divided into three categories, including time series and ocean model prediction, and prediction by neural network model. At present, there are not many methods for predicting surface current parameters by using deep learning, and the prediction accuracy needs to be improved. Due to the complexity of surface currents and the influence of external environmental factors, it is difficult to mine the information between surface currents. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the present application aims to provide a surface current prediction method based on an attention mechanism TCN-LSTM model. The method uses the characteristics of long short-term memory networks and the advantages of time convolution, and adds an attention mechanism. The real historical marine data of a certain sea area measured by the WaMoSII system is processed and predicted. The prediction accuracy of the algorithm proposed in the present application reaches 92.9%, and the flow direction accuracy is lower than that of the flow velocity, but still reaches 90.4%. In the future, the multi-step prediction is better than other models, and ideal results are achieved.

[0005] In order to achieve the above-mentioned application purpose, the technical scheme adopted by the present application is as follows: a surface current prediction method based on an attention mechanism TCN-LSTM model, comprising the following steps:

[0006] 1. A surface current prediction method based on an attention mechanism TCN-LSTM model, characterized by comprising the following steps:

[0007] Step 1, collecting real data of surface current by data acquisition module, using WaMoSII radar system to collect continuous time of marine parameters in a sea area, including collection time, surface current information flow rate Usp, flow direction Udir, maximum wave height Hmax and average wave period Tm2, specifically including the following sub-steps:

[0008] (a) selecting a wide sea area, using WaMoSII wave current monitoring system fixed on the shore, connecting remote PC for data analysis and storage;

[0009] (b) recording the start time of data collection, setting the number of collected images to 32, the storage mode to pol file, and the size of the Cartesian frame to 256*128 for processing and analyzing data;

[0010] (c) exporting the surface current data obtained by WaMoSII through image inversion to an excel table;

[0011] Step 2, processing surface current data using experimental data processing module, specifically including the following sub-steps:

[0012] (a) processing surface current data anomalies, removing outliers and filling, using k-nearest neighbor method to fill missing values to ensure prediction effect, k-nearest neighbor distance formula is described by formula (1),

[0013]

[0014] where a represents the order of eigenvalue, x i represents the first sample, x j represents the second sample, and the selection of k also determines the quality of the model, k cannot be too large or too small, too large may reduce its generalization ability, and too small will affect the accuracy, k represents the number of points in the multi-dimensional space that are closest to the sample to be measured according to the Euclidean distance, here k is 8;

[0015] (b) flow direction data processing, the flow direction changes greatly from 0° to 360°, which increases the difficulty of prediction, since the flow direction across the 360° boundary line presents a cliff-like jump, which will cause instability in the prediction of flow direction, when using deep learning method to predict angle, the angle from 0 to 360 degrees is converted into a continuous, real number range, using sine function or cosine function for coding, then it is used as the input of the model, finally the predicted value is restored to the normal interval by inverse trigonometric function;

[0016] (c) processing surface current data using continuous wavelet transform, continuous wavelet transform formula is described by formula (2),

[0017]

[0018] wherein x(t) represents an input signal, denotes a wavelet basis function, denotes a conjugate of the wavelet basis function, a denotes a scale parameter in continuous wavelet transform, and b denotes a translation parameter in continuous wavelet transform;

[0019] The wavelet transform has a good effect on removing noise in the surface current data. By analyzing the wavelet coefficients of different scales in the surface current data, effective denoising processing is performed, and the real characteristics in the signal are better extracted for subsequent model training and testing.

[0020] (d) performing normalization processing on the surface current data, and the normalization formula is described by formula (3)

[0021]

[0022] wherein x represents the original surface current data, x max denotes the maximum value in the original surface current data set, x min denotes the minimum value in the original surface current data set;

[0023] Step 3, constructing an attention mechanism TCN-LSTM surface current prediction model by using a model building module, processing the input surface current data by LSTM, and converting the surface current data into query, key and value forms by the attention mechanism, which specifically includes the following sub-steps:

[0024] (a) constructing an LSTM unit, inputting the processed surface current data into the LSTM unit, and the input dimension is t*n, wherein: t is the time length of 16000, n is the feature dimension of 5, the number of LSTM layers is 6, the hidden layer unit is 20, after the first layer of LSTM processing, a two-dimensional array of output (16000, 20) is obtained, and this output is taken as the input of the second layer of LSTM, after the second layer of LSTM processing, the obtained output is still a two-dimensional array of (16000, 20), and so on. The LSTM unit structure mainly includes an input gate, a forgetting gate and an output gate, and the calculation formulas are described by formulas (4), (5), (6), (7) and (8),

[0025] i t = σ (W i · [h t-1 , x t + b i ) (4)

[0026] ft = σ(W f [h t-1 , x t ]+ b f ) (5)

[0027] o t = σ(W o [h t-1 , x t ]+ b o ) (6)

[0028]

[0029] h t = o t * tanh(C t ) (8)

[0030] where W i , b i represent the state weight and bias of the input gate respectively, W f , b f represent the state weight and bias of the forget gate respectively, W o , b o represent the state weight and bias of the output gate respectively, σ is the sigmoid activation function, h t-1 is the hidden state at time t-1, represents the tanh layer used to generate new candidate memory cells, C t represents the candidate memory cell, h t represents the hidden state determined by the output gate o t and the candidate memory cell C t ; the selected features are converted into an input format acceptable to the LSTM model, and the processed surface current time series data are converted into a two-dimensional array form, where one dimension represents the time step and the other dimension represents the feature. At each time step, the LSTM unit calculates the output and the new hidden state of the current time step according to the input of the current time step and the hidden state of the previous time step;

[0031] (b) constructing a TCN-LSTM model, inputting the output of the LSTM into the TCN model structure, adding a convolutional layer after the LSTM model as the starting point of the TCN, converting the output of the LSTM model into a feature representation suitable for the TCN; the TCN model mainly comprises a plurality of convolutional layers, each convolutional layer being composed of a convolution kernel and a nonlinear activation function, the convolution kernel sliding in the time dimension to capture the local patterns of the input sequence, thereby extracting feature information between surface currents; the model comprises 3 residual modules, the number of convolutional layers is 3, the number of convolutional kernels of the first two is set to 32, and the number of convolutional kernels of the last one is set to 16, the convolution kernel size is 3, and the expansion factors are 1, 2 and 4 respectively, so that the model can better capture the global features of the surface current during training, and the output of the LSTM (16000, 20) is input into the 3-layer TCN, each layer of the TCN has different numbers of convolutional kernels and expansion factors; the first layer of the TCN has 32 convolutional kernels, the expansion factor is 1, and the convolution kernel size is 3, after processing by the first layer of the TCN, the output is a two-dimensional array of (16000, 32); the second layer of the TCN also has 32 convolutional kernels, the expansion factor is 2, and the convolution kernel size is 3, after processing by the second layer of the TCN, the output is a two-dimensional array of (16000, 32); the third layer of the TCN has 16 convolutional kernels, the expansion factor is 4, and the convolution kernel size is 3, after processing by the third layer of the TCN, the output is a two-dimensional array of (16000, 16); a pooling layer is added to the last of the TCN to reduce the dimension of the feature map and extract the overall features, the convolution represents the current time t only with the current and the information before t time, which is described by formula (9),

[0032] y t =f(x1,x2,…x t ) (9)

[0033] wherein: x t is a one-dimensional vector, y t represents a predicted variable, and f represents a function established between the two, for the input sequence, the convolution formula is described by formula (10),

[0034]

[0035] wherein: s represents input time series surface current information, d is an expansion factor, k is a filter size, and s-d i is position information of the previous sequence time;

[0036] (c) adding an attention mechanism module after the TCN convolution, the attention mechanism is derived from machine vision, which ignores other unimportant information when processing information, and the function of the attention mechanism is described by formula (11),

[0037]

[0038] wherein: (k1, k m ) is the input key sequence, (v1, v m ) is the corresponding value sequence, the corresponding query vector is q, in order to solve the influence of vector dimension on weight, a scaled dot-product attention mechanism is used, wherein the scoring function of attention is a, which is described by formula (12),

[0039]

[0040] wherein: d k is the dimension of the input vector, k is the same length as the query vector q, k T is the transpose matrix of the input vector;

[0041] The surface current data is converted into the form of query, key and value, the similarity between the query and each key is calculated to measure the correlation between the query and different keys. In this step, the similarity is used to calculate the attention weight corresponding to each key, the Softmax function is used to convert the similarity into the attention weight, so that the sum of the attention weights corresponding to all keys is 1, each value is multiplied by the corresponding attention weight, and the sum of all weighted values is calculated. According to the result obtained by weighted summation, the final output is generated. The introduction of attention mechanism helps the model to automatically learn important information in the input sequence, so that the model can pay more attention to the key moments and important features between the surface currents. The feature information between the surface currents is extracted by the TCN model and input into the attention mechanism to allocate weights. The data output after allocation is input into the fully connected layer, and the surface current velocity and flow direction results are obtained after calculation. The two-dimensional array of (16000, 16) is input into the attention mechanism for processing, and the final output is a one-dimensional array of 16.

[0042] (d) The feature vector weighted by attention is taken as the input of the fully connected layer, and the parameters of the fully connected layer are set, including that the input dimension should match the output dimension of the attention mechanism, which is 16, the output dimension is set to 1, a single numerical value is obtained as the output, and the corresponding y t , y t+2 , y t+4 moment surface current parameters are calculated;

[0043] (e) A sliding window is used, the number of sliding windows is set to 10, and the step is 1. The error of the output surface current velocity usp and flow direction uidr is calculated to verify the feasibility of the model.

[0044] Step 4, verification and testing of surface current data, real surface current data collected by radar system WaMoS II, measurement days are 12 days, a total of 16,000 groups, and then these surface current data are divided into training set, validation set and test set according to 8:1:1, and the last 2 days of surface current data are used for verification and testing;

[0045] Step 5, prediction of neural network model, input the processed surface current data into the established neural network model for prediction, input the surface current data according to the method of sliding window, set the size of the window to 10, the input features are 5, the output features are surface current velocity Usp and surface current direction Udir, the parameters of each model are batch_size 512, LSTM hidden layer 20, initial learning rate 0.01, Relu is the activation function, contains 3 residual modules, the number of iterations is 100, predicts the surface current velocity value of the first, second and fourth time, makes continuous surface current velocity prediction on the test set, the optimization algorithm is Adam, calculates the gradient of mini-batch surface current through iteration, calculates the error of the output surface current velocity and direction, the main error evaluation indexes are mean absolute error MAE, mean absolute percentage error MAPE, root mean square error RMSE and determination coefficient R 2 , described by formulas (13) to (16),

[0046]

[0047]

[0048]

[0049]

[0050] wherein: y i represents the i-th real value of the surface current data, represents the i-th predicted value of the surface current data, represents the average value of the surface current data prediction, R 2 The greater the prediction performance is, the smaller the remaining indexes are, and the higher the prediction accuracy is.

[0051] The present application has the beneficial effects that: a surface current prediction method based on an attention mechanism TCN-LSTM model, comprising the following steps: (1) collecting surface current real data through a data acquisition module, (2) processing surface current data using an experimental data processing module, (3) constructing an attention mechanism TCN-LSTM surface current prediction model using a model building module, (4) verifying and testing the surface current data, and (5) predicting the neural network model. Compared with the prior art, the method uses the characteristics of long short-term memory network and the advantages of time convolution, and simultaneously adds an attention mechanism, and predicts after processing the real historical marine data of a certain sea area measured by the WaMoSII system. The method proposed in the present application has a surface current flow velocity prediction accuracy of 92.9%, and a flow direction accuracy of 90.4%. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is the LSTM model structure diagram of the model building module of the present application.

[0053] Figure 2 is the TCN model dilated convolution diagram of the model building module of the present application.

[0054] Figure 3 is the method step flow chart of the present application.

[0055] Figure 4 is the overall schematic diagram of the model building module of the present application.

[0056] Figure 5 is the surface current flow velocity prediction result diagram of the present application.

[0057] Figure 6 is the surface current flow direction prediction result diagram of the present application. DETAILED DESCRIPTION

[0058] As shown in Figure 3 , a surface current prediction method based on an attention mechanism TCN-LSTM model, comprising the following steps:

[0059] Step 1, collecting surface current real data through a data acquisition module, using a WaMoSII radar system to collect continuous time marine parameters in a certain sea area, including collection time, surface current information flow velocity Usp, flow direction Udir, maximum wave height Hmax and average wave period Tm2, specifically including the following sub-steps:

[0060] (a) selecting a wide sea area, using a WaMoSII wave and current monitoring system fixed on the shore, connecting a remote PC for data analysis and storage;

[0061] (b) Record the start time of data acquisition, set the number of collected images to 32, the storage mode to pol file, and the size of the Cartesian frame to 256*128, which is used for processing and analyzing data;

[0062] (c) Export the surface current data obtained by image inversion of WaMoSII to an excel table;

[0063] Step 2, process the surface current data using the experimental data processing module, which includes the following sub-steps:

[0064] (a) Process the surface current data anomaly, eliminate the abnormal value and fill it, use k-neighbor method to fill the null value to ensure the prediction effect, the k-neighbor distance formula is described by formula (1),

[0065]

[0066] Where a represents the order of eigenvalue, x i represents the first sample, x j represents the second sample, the selection of k also determines the quality of the model, k cannot be too large or too small, too large may lead to reduce its generalization ability, if too small will affect the accuracy, k represents the number of points in the multi-dimensional space that are closest to the sample to be measured according to the Euclidean distance, here k is 8;

[0067] (b) Flow direction data processing, the flow direction is transformed by trigonometric function, where the flow direction changes greatly from 0° to 360°, which makes the prediction difficult, since the flow direction across the 360° boundary line presents a cliff type, it will lead to the instability of flow direction prediction in the process of prediction, when using deep learning method to predict the angle, the angle from 0 to 360 degrees is converted into a continuous, real number range, which is encoded by using sine function or cosine function, then it is used as the input of the model, finally the predicted value is restored to the normal interval by using the inverse trigonometric function;

[0068] (c) Process the surface current data using continuous wavelet transform, the continuous wavelet transform formula is described by formula (2),

[0069]

[0070] Where x(t) represents the input signal, represents the wavelet basis function, represents the conjugate of the wavelet basis function, a represents the scale parameter in the continuous wavelet transform, and b represents the translation parameter in the continuous wavelet transform;

[0071] Wavelet transform has good effect on removing noise in surface current data. By analyzing wavelet coefficients of different scales in surface current data, effective denoising processing is performed to better extract real features in the signal for subsequent model training and testing.

[0072] (d) Normalizing the surface current data, the normalization formula is described by formula (3)

[0073]

[0074] Wherein, x represents the original surface current data, x max represents the maximum value in the original surface current data set, x min represents the minimum value in the original surface current data set.

[0075] Step 3, using the model building module to construct the attention mechanism TCN-LSTM surface current prediction model, processing the input surface current data through LSTM, and the attention mechanism converts the surface current data into the form of query, key and value, which includes the following sub-steps:

[0076] (a) Building an LSTM unit, the LSTM model structure is shown in Figure 1 The processed surface current data is input into the LSTM unit, the input dimension is t*n, where: t is the time length of 16000, n is the feature dimension of 5, the LSTM layer is 6, the hidden layer unit is 20, after the first layer LSTM processing, the output (16000, 20) of two-dimensional array is obtained, this output is taken as the input of the second layer LSTM, after the second layer LSTM processing, the output is still a (16000, 20) two-dimensional array, and so on. The LSTM unit structure mainly includes input gate, forget gate and output gate, and the calculation formula is described by formula (4), (5), (6), (7), (8),

[0077] i t =σ(W i ·[h t-1 ,x t ]+b i ) (4)

[0078] f t =σ(W f ·[h t-1 ,x t ]+b f ) (5)

[0079] o t =σ(W o [h t-1 ,xt ]+b o ) (6)

[0080]

[0081] h t =o t *tanh(C t ) (8)

[0082] wherein: W i , b i represent the state weight and bias term of the input gate respectively, W f , b f represent the state weight and bias term of the forget gate respectively, W o , b o represent the state weight and bias term of the output gate respectively, σ is the sigmoid activation function, h t-1 is the hidden state at time t-1, represents the tanh layer used to generate a new candidate memory cell, C t represents the candidate memory cell, h t represents the hidden state is determined by the output gate o t and the candidate memory cell C t ; the selected features are converted into an input format acceptable to the LSTM model, and the processed surface current time series data is converted into a two-dimensional array form, where one dimension represents the time step and the other dimension represents the feature. At each time step, the LSTM unit calculates the output and the new hidden state of the current time step according to the input of the current time step and the hidden state of the previous time step;

[0083] (b) constructing a TCN-LSTM model, the TCN model structure diagram is as shown in Figure 2As shown, the output of the LSTM is input into the TCN model structure, a convolutional layer is added after the LSTM model as the starting point of the TCN, and the output of the LSTM model is converted into a feature representation suitable for the TCN; the TCN model mainly includes multiple convolutional layers, each convolutional layer is composed of a convolution kernel and a nonlinear activation function, and the convolution kernel slides in the time dimension to capture the local patterns of the input sequence, thereby extracting the feature information between the surface currents; the model includes 3 residual modules, the number of convolutional layers is 3, the number of convolutional kernels of the first two is set to 32, and the number of convolutional kernels of the last one is set to 16, the convolution kernel size is 3, and the expansion factors are 1, 2 and 4 respectively, so that the model can better capture the global features of the surface currents during training, and the output of the LSTM(16000, 20) is input into the 3-layer TCN, each layer of the TCN has different number of convolutional kernels and expansion factors; the first layer of the TCN has 32 convolutional kernels, the expansion factor is 1, and the convolution kernel size is 3, after processing by the first layer of the TCN, the output is a two-dimensional array of (16000, 32); the second layer of the TCN also has 32 convolutional kernels, the expansion factor is 2, and the convolution kernel size is 3, after processing by the second layer of the TCN, the output is a two-dimensional array of (16000, 32); the third layer of the TCN has 16 convolutional kernels, the expansion factor is 4, and the convolution kernel size is 3, after processing by the third layer of the TCN, the output is a two-dimensional array of (16000, 16); a pooling layer is added to the last of the TCN to reduce the dimension of the feature map and extract the overall features, the convolution represents that the current time t is only related to the current and the information before t, which is described by formula (9),

[0084] y t =f(x1,x2,…x t ) (9)

[0085] Where: x t is a one-dimensional vector, y t represents the predicted variable, and f represents the function established between the two, for the input sequence, the convolution formula is described by formula (10),

[0086]

[0087] Where: s represents the input time series surface current information, d is the expansion factor, k is the filter size, and s-d·i is the position information of the previous sequence time;

[0088] (c) adding an attention mechanism module after the TCN convolution, the attention mechanism comes from machine vision, which ignores other unimportant information when processing information, and the function of the attention mechanism is described by formula (11),

[0089]

[0090] wherein: (k1, k m ) is the input key sequence, (v1, v m ) is the corresponding value sequence, and the corresponding query vector is q. In order to solve the influence of vector dimension on weight, a scaled dot-product attention mechanism is used, wherein the attention score function is a, which is described by formula (12),

[0091]

[0092] wherein: d k is the dimension of the input vector, k is the same length as the query vector q, and k T is the transpose matrix of the input vector;

[0093] The surface current data is converted into the form of query, key and value. The relevance between the query and different keys is measured by calculating the similarity between the query and each key. In this step, the similarity is used to calculate the attention weight corresponding to each key. The similarity is converted into the attention weight by using the Softmax function, so that the sum of the attention weights corresponding to all keys is 1. Each value is multiplied by the corresponding attention weight, and the weighted values are summed. According to the result obtained by the weighted summation, the final output is generated. The introduction of the attention mechanism helps the model to automatically learn the important information in the input sequence, so that the model can pay more attention to the key moments and important features between the surface currents. The feature information between the surface currents extracted by the TCN model is input into the attention mechanism to assign weights. The data output after the assignment is input into the fully connected layer, and the surface current velocity and direction results are obtained after the calculation. The two-dimensional array of (16000, 16) is input into the attention mechanism for processing, and the final output is a one-dimensional array of 16.

[0094] (d) The feature vector weighted by attention is input into the fully connected layer. The parameters of the fully connected layer are set, including that the input dimension should match the output dimension of the attention mechanism, which is 16, and the output dimension is set to 1, so as to obtain a single numerical value as the output, and the corresponding y t , y t+2 , y t+4 moment surface current parameters are calculated;

[0095] (e) A sliding window is used, the number of sliding windows is set to 10, and the step is 1. The error of the output surface current velocity usp and the flow direction uidr is calculated to verify the feasibility of the model.

[0096] Step 4, verification and testing of surface current data, real surface current data collected by radar system WaMoS II, measurement days are 12 days, a total of 16,000 groups, and then these surface current data are divided into training set, validation set and test set according to 8:1:1, and the last 2 days of surface current data are used for verification and testing;

[0097] Step 5, prediction of neural network model, the processed surface current data are input into the established neural network model for prediction, the surface current data are input according to the method of sliding window, the size of the window is set to 10, the input features are 5, the output features are surface current velocity Usp and surface current direction Udir, the parameters of each model are batch_size 512, LSTM hidden layer 20, initial learning rate 0.01, Relu is the activation function, contains 3 residual modules, the number of iterations is 100, the surface current velocity values of the first, second and fourth time points in the future are predicted, the test set is continuously predicted, the optimization algorithm is Adam, the gradients in the mini-batch surface current are calculated through iteration, the error of the output surface current velocity and direction is calculated, and the main error evaluation indexes are mean absolute error MAE, mean absolute percentage error MAPE, root mean square error RMSE and determination coefficient R 2 , which are described by formulas (13) to (16),

[0098]

[0099]

[0100]

[0101]

[0102] wherein: y i represents the i-th real value of the surface current data, represents the i-th predicted value of the surface current data, represents the average value of the surface current data prediction, R 2 is greater, the prediction performance is better, the rest of the indexes are smaller, and the prediction accuracy is higher. The overall schematic diagram of the model is shown in Figure 4 , the prediction results are shown in Tables 1, 2, Figure 5 and Figure 6 .

[0103] The hardware environment of the experiment is Core i7-10750H, the memory is 32G, the operating system is Window 10, the GUP model is NVIDIA Ge Force RTX2070, the development framework is Tensorflow, the compiling language is Python 3.8.0, and the open environment is Py Charm.

[0104] The surface current flow velocity prediction result is shown in Table 1,

[0105] Table 1

[0106]

[0107] The surface current flow direction prediction result is shown in Table 2,

[0108] Table 2

[0109]

Claims

1. A surface current prediction method based on an attention mechanism TCN-LSTM model, characterized in that, Comprising the following steps: Step 1, collecting real data of surface current by data acquisition module, using WaMoSII radar system to collect continuous time of marine parameters in a sea area, including collection time, surface current information flow velocity Usp, flow direction Udir, maximum wave height Hmax and average wave period Tm2, specifically including the following sub-steps: (a) selecting a wide sea area, using WaMoSII wave current monitoring system fixed on the shore, connecting remote PC for data analysis and storage; (b) recording the start time of collection, setting the number of collected images to 32, the storage mode to pol file, and the size of the Cartesian frame to 256*128 for processing and analyzing data; (c) exporting the surface current data obtained by WaMoSII through image inversion to an excel table; Step 2, processing surface current data using experimental data processing module, specifically including the following sub-steps: (a) processing surface current data anomalies, removing outliers and filling, using k-nearest neighbor method to fill missing values to ensure prediction effect, k-nearest neighbor distance formula is described by formula (1), Wherein, a represents the order of eigenvalue, x i represents the first sample, x j represents the second sample, the selection of k also determines the quality of the model, k cannot be too large or too small, too large may or lead to reduce its generalization ability, if too small will affect the accuracy, k represents the number of points in the multi-dimensional space according to the Euclidean distance to find the nearest neighbor of the sample to be measured, here k takes 8; (b) flow direction data processing, the flow direction changes greatly from 0° to 360°, which increases the difficulty of prediction, since the flow direction across the 360° boundary presents a cliff-like jump, which will cause instability in the flow direction prediction, when using deep learning method to predict the angle, the angle from 0 to 360 degrees is converted into a continuous, real value range, using sine function or cosine function for coding, then the predicted value is obtained by inverse trigonometric function to restore it to the normal interval; (c) processing surface current data using continuous wavelet transform, continuous wavelet transform formula is described by formula (2), wherein x(t) represents an input signal, denotes a wavelet basis function, denotes a conjugate of the wavelet basis function, a denotes a scale parameter in continuous wavelet transform, and b denotes a translation parameter in continuous wavelet transform; the wavelet transform has a good effect on removing noise in the surface current data, and through analysis of wavelet coefficients of different scales in the surface current data, effective denoising processing is performed, and real characteristics in the signal are better extracted for subsequent model training and testing; (d) normalizing surface current data, normalization formula is described by formula (3) wherein x represents the surface current raw data, x max represents the maximum value in the surface current raw data set, x min represents the minimum value in the surface current raw data set; Step 3, constructing attention mechanism TCN-LSTM surface current prediction model using model building module, processing input surface current data through LSTM, attention mechanism converts surface current data into query, key and value forms, specifically including the following sub-steps: (a) building LSTM unit, inputting processed surface current data into LSTM unit, input dimension is t*n, where: t is time length 16000, n is feature dimension 5, LSTM layer number is 6, hidden layer unit is 20, after the first layer LSTM processing, the output (16000, 20) two-dimensional array is obtained, this output is used as the input of the second layer LSTM, after the second layer LSTM processing, the output is still a (16000, 20) two-dimensional array, and so on; LSTM unit structure mainly includes input gate, forget gate and output gate, calculation formula is described by formula (4), (5), (6), (7), (8), ​ i t = σ(W i · [h t-1 , x t ]+ b i ) (4) f t = σ(W f · [h t-1 , x t ]+ b f ) (5) o t = σ(W o [ h t-1 , x t ] + b o ) (6) h t = o t tanh(C t ) (8) where: W i , b i represent the state weight and bias term of the input gate respectively, W f , b f represent the state weight and bias term of the forget gate respectively, W o , b o represent the state weight and bias term of the output gate respectively, σ is the sigmoid activation function, h t-1 is the hidden state at time t-1, represents the tanh layer used to generate new candidate memory cells, C t represents the candidate memory cell, h t represents the hidden state is determined by the output gate o t and the candidate memory cell C t ; the selected features are converted into an input format acceptable to the LSTM model, and the processed surface current time series data are converted into a two-dimensional array form, where one dimension represents the time step and the other dimension represents the feature. At each time step, the LSTM unit calculates the output and the new hidden state of the current time step according to the input of the current time step and the hidden state of the previous time step; (b) constructing a TCN-LSTM model, inputting the output of the LSTM into the TCN model structure, adding a convolutional layer after the LSTM model as the starting point of the TCN, converting the output of the LSTM model into a feature representation suitable for the TCN; the TCN model mainly includes multiple convolutional layers, each convolutional layer is composed of a convolution kernel and a nonlinear activation function, the convolution kernel slides in the time dimension to capture the local patterns of the input sequence, thereby extracting the feature information between the surface currents; the model includes 3 residual modules, the number of convolutional layers is 3, the number of convolution kernels is set to 32 for the first two, and the number of convolution kernels is set to 16 for the last one, the convolution kernel size is 3, and the expansion factors are 1, 2 and 4 respectively, such design makes the model better capture the global features of the surface currents during training, the output of the LSTM(16000, 20) is input to the 3-layer TCN, each layer of the TCN has different number of convolution kernels and expansion factors; the first layer of the TCN has 32 convolution kernels, the expansion factor is 1, and the convolution kernel size is 3, after processing by the first layer of the TCN, the output is a two-dimensional array of (16000, 32); the second layer of the TCN also has 32 convolution kernels, the expansion factor is 2, and the convolution kernel size is 3, after processing by the second layer of the TCN, the output is a two-dimensional array of (16000, 32); the third layer of the TCN has 16 convolution kernels, the expansion factor is 4, and the convolution kernel size is 3, after processing by the third layer of the TCN, the output is a two-dimensional array of (16000, 16); a pooling layer is added, a global average pooling layer is added to the end of the TCN to reduce the dimension of the feature map and extract the overall features, the current time t in each layer of the TCN is only related to the current and the information before t time, which is described by formula (9), y t = f(x1, x2,... x t ) (9) where: x t is a one-dimensional vector, y t represents the predicted variable, f represents the function established between the two, for the input sequence, the convolution formula is described by equation (10), Where: s represents the input time series surface current information, d is the expansion factor, k is the filter size, and s-d·i is the position information of the previous time; (c) adding an attention mechanism module after the TCN convolution, the attention mechanism comes from machine vision, which ignores other unimportant information when processing information, the function of the attention mechanism is described by formula (11), where: (k1, k m ) is the input key sequence, (v1, v m ) is the corresponding value sequence, and the corresponding query vector is q. To address the impact of vector dimension on weight, a scaled dot-product attention mechanism is used, where the attention score function a is described by equation (12). wherein: d k is the dimension of the input vector, k is the same length as the query vector q, k T is the transpose matrix of the input vector; The surface current data is converted into the form of query, key and value, the correlation between the query and different keys is measured by calculating the similarity between the query and each key, in this step, the similarity is used to calculate the attention weight corresponding to each key, the similarity is converted into the attention weight by using the Softmax function, so that the sum of the attention weights corresponding to all keys is 1, each value is multiplied by the corresponding attention weight, and the weighted values are summed, according to the result obtained by the weighted summation, the final output is generated; the introduction of the attention mechanism helps the model to automatically learn the important information in the input sequence, so that the model can pay more attention to the key moments and important features between the surface currents, the feature information between the surface currents is extracted by the TCN model and input into the attention mechanism to allocate the weight, the allocated data is output to the full connection layer, and the surface current velocity and flow direction results are obtained after calculation, a two-dimensional array of (16000, 16) is input into the attention mechanism for processing, and finally a one-dimensional array of 16 is output; (d) taking the attention-weighted feature vector as the input of a fully connected layer, setting the parameters of the fully connected layer, including that the input dimension should be matched with the output dimension in the attention mechanism as 16, setting the output dimension as 1 to obtain a single numerical value as the output, and calculating the surface current parameter at the corresponding y t , y t+2 , y t+4 moment; (e) a sliding window is used, the number of sliding windows is set to 10, the step is 1, the error of the output obtained surface current velocity usp and flow direction uidr is calculated, and the feasibility of the model is verified; Step 4, verify and test the surface current data, the real surface current data collected by the radar system WaMoS II, the measurement days are 12 days, a total of 16,000 groups, and then these surface current data are divided into training set, verification set and test set according to the ratio of 8:1:1, the last two days of surface current data are used for verification and test; Step 5, prediction of the neural network model, the processed surface current data is input into the established neural network model for prediction, the surface current data is input according to the sliding window method, the size of the window is set to 10, the input features are 5, the output features are the surface current velocity Usp and the surface current direction Udir, the parameters of each model are batch_size 512, the LSTM hidden layer is 20, the initial learning rate is 0.01, the Relu is the activation function, contains 3 residual modules, the iteration number is 100, the surface current velocity values of the future 1st, 2nd and 4th time are predicted, the test set is continuously predicted, the optimization algorithm is Adam, the gradient in the mini-batch surface current is calculated quickly through iteration, the output surface current velocity and direction are calculated, the main error evaluation indexes are mean absolute error MAE, mean absolute percentage error MAPE, root mean square error RMSE and determination coefficient R 2 , which are described by formulas (13) to (16). wherein: y i represents the i-th true value of the surface current data, represents the i-th predicted value of the surface current data, represents the average value of the predicted surface current data, R 2 The greater the prediction performance is, the smaller the rest of the indicators are, and the higher the prediction accuracy is.

Images