An effective wave height prediction method based on multi-scale patching

Abstract

A multi-scale patch-based method for predicting significant wave height (SWH) is presented, relating to the field of marine environmental variable prediction. First, historical data of multiple marine environmental variables related to SWH are Gaussian normalized, and training and validation sets are created. Then, the number of layers and branches in each layer of the model are determined. Next, temporal embedding is performed on the encoder layer input of each branch in each layer of the model, followed by multi-head attention layers, feedforward neural network layers, residual connections, and batch normalization operations in the encoder layer. The multi-layer output of the model is then passed through a fully connected linear layer to obtain the network output, which is compared with the normalized SWH observations to obtain the error backpropagation. The network is then fine-tuned, and finally, the network output undergoes inverse scaling to obtain the final predicted value. This multi-scale patch-based prediction method effectively improves the short-term and long-term prediction performance of SWH.

Description

Technical Field

[0001] This invention relates to the field of marine environmental variable prediction, and in particular to an effective wave height prediction method based on multi-scale patching. Background Technology

[0002] Accurate prediction of ocean wave parameters, such as wave height and wave period, is crucial for the planning, design, use, and maintenance of structures in coastal engineering. Wave height observations vary over time and are a typical type of time series data. Prediction of wave height can be mainly categorized into: prediction based on mathematical models, prediction based on data-driven methods, and prediction using a combination of multiple models.

[0003] Numerical modeling methods: For waves, predictions based on mathematical models are generally achieved through wave numerical models. However, the numerical models established by this method are usually approximate or insufficient, and because they require very accurate water depth measurements as input, they are difficult to handle complex and variable wave time series data in practical applications, resulting in limited prediction accuracy and high computational costs.

[0004] Data-driven approaches: Data-driven time series forecasting methods do not rely on mathematical or physical models of the research object. Instead, they start from the characteristics of the data itself to study its internal patterns and make predictions. Key technologies include support vector machines, filters, genetic algorithms, fuzzy systems, and machine learning.

[0005] Hybrid model approach: The hybrid model approach mainly combines numerical modeling methods and data-driven methods, or a combination of multiple data-driven models, to predict effective wave height. Because it uses two or more models, this method is the most complex, computationally resource-intensive, and time-consuming. Summary of the Invention

[0006] The purpose of this invention is to solve the above-mentioned problems in the prior art and provide an effective wave height prediction method based on multi-scale patching, thereby improving the short-term and long-term prediction accuracy of effective wave height.

[0007] This invention segments time series data of marine environmental variables such as significant wave height into several patches and uses these patches as labels to learn complex temporal patterns. The patch size controls the ability to learn temporal patterns of different frequencies; that is, shorter patches are effective for learning local high-frequency patterns, while longer patches are needed to uncover long-term seasonality and trends. Therefore, this invention applies a multi-scale approach to significant wave height prediction.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] An effective wave height prediction method based on multi-scale patching includes the following steps:

[0010] 1) For a certain latitude and longitude grid point, select historical data of multiple marine environmental variables related to significant wave height as training and validation datasets and perform Gaussian normalization according to the variables. Then, divide the training set and validation set according to time order.

[0011] 2) Constructing an effective wave height prediction model: The model consists of... Layer composition ( (And the input is an integer), each layer of input is divided into patches of different sizes by a patch splitter. Number of branches (and is an integer);

[0012] 3) No. Each branch segmented by the layer patch segmenter is input into a relative position embedding layer. After embedding time-series information, it enters a self-attention layer. The self-attention layer calculates the self-attention of the patch-level time-series data in each branch through encoding operations and performs linear projection on the relative position embedding to achieve learnability, thereby capturing the relationship between patches of the same size in each branch. Then, a combination of residual connections, batch normalization, and feedforward networks is used to process the output of the self-attention layer to obtain the model's... Layer Branch output The outputs of each branch are fused to obtain the model's first branch. Layer output And feed it into the next layer of the model;

[0013] 4) Using model number 1 The output of the first layer is used as input, and then input to the second layer. Repeat steps 2) to 3) to obtain the final model. Layer output;

[0014] 5) Model No. The output of the layer is passed through a linear fully connected layer to obtain the output of the effective wave height prediction model based on multi-scale patching. Then, it is compared with the normalized effective wave height observation to obtain the error backpropagation, and the network weights and biases are optimized and adjusted. After training and validation, the output of the effective wave height prediction model based on multi-scale patching is subjected to inverse scale transformation to obtain the final predicted value of the effective wave height.

[0015] In step 1), the marine environmental variables include significant wave height, meridional wind speed, zonal wind speed, mean wave period, and mean wave direction. In order to avoid discontinuities at 0° and 360°, the mean wave direction is converted into the sine and cosine values ​​of the mean wave direction.

[0016] The formula for Gaussian normalization is:

[0017]

[0018] in, and These represent data for a certain marine environmental variable before and after Gaussian normalization, respectively. This represents the average value of the variable's data before Gaussian normalization. This represents the standard deviation of the variable's data before Gaussian normalization.

[0019] In step 2), record the first... layer The input is , length is , No. Patch size at the branch and stride So, the first Layer patch splitter for the first The output after branching is , No. The number of patches in the branch is ,in, ;

[0020] To ensure Can be Integer division requires padding at the end. The last dimension One zero.

[0021] In step 3), the fusion operation includes flattening, connecting, and linearly projecting the output of each branch.

[0022] In step 3), the first The and the first The relative position embedding formula between time series patches is:

[0023]

[0024] Among them, the sign function , , , , Indicates the embedding dimension.

[0025] In step 3), for the network... Layer Branch, number The and the first The attention weights among the time series patches are represented as follows:

[0026]

[0027] All time series patches for the first The attention of each time series patch is:

[0028]

[0029] Represents the learnable linear projective weight coefficients;

[0030] No. Layer patch splitter for the first Branch splitting output Attention is:

[0031] .

[0032] In step 3), the model's... Layer Branch output Represented as:

[0033]

[0034]

[0035] in, Indicates residual connection, Indicates batch normalization, This indicates a feedforward operation.

[0036] In step 3), the model's... Layer output Represented as:

[0037]

[0038] in, , and Indicates the first Each branch output of the layer is flattened and connected. and Indicates the first Weighting coefficients and biases for layer linear projection operations.

[0039] In step 5), during the network training and validation process, the "Adam" optimizer is used, with the mean squared error (MSE) as the loss function and an initial learning rate of lr = 0.0001. The learning rate is then adjusted according to the following formula:

[0040]

[0041] Here, epoch is the number of training rounds for all training samples. Training stops when the loss MSE of the validation dataset does not decrease within a certain number of rounds.

[0042] In step 5), the formula for the inverse scaling transformation is as follows:

[0043]

[0044] in, and These represent the significant wave height data before and after the inverse transform, respectively. This represents the average value of the significant wave height data before the inverse transform. This represents the standard deviation of the effective wave height data before the inverse transform.

[0045] Compared with the prior art, the beneficial effects achieved by the technical solution of this invention are:

[0046] The shorter patches of this invention facilitate higher-resolution modeling of short-term temporal features, while longer patches help learn longer-term seasonality and trends. For time series of ocean environmental variables such as significant wave height, by setting patches of different sizes, different temporal patterns can be modeled under different resolution conditions, which is beneficial for obtaining satisfactory short-term and long-term forecasting results. Attached Figure Description

[0047] Figure 1 This is a diagram of the effective wave height prediction network structure based on multi-scale patches;

[0048] Figure 2 To predict the effective wave height of a certain sea area in 2023 with a time step of 3 hours, a comparison was made between the predicted effective wave height and the corresponding observed values ​​based on a multi-scale patch network.

[0049] Figure 3 To predict the effective wave height of a certain sea area in 2023 with a time step of 144 hours, a comparison was made between the predicted effective wave height and the corresponding observed values ​​based on a multi-scale patch network.

[0050] Figure 4 For a prediction time step of 720 hours, a comparison was made between the predicted and observed significant wave heights for a certain sea area in 2023 using a multi-scale patch-based significant wave height prediction network. Detailed Implementation

[0051] To make the technical problems, technical solutions and beneficial effects of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0052] See Figure 1 The present invention includes the following steps:

[0053] Step 1: For a given latitude and longitude grid point, select historical data of marine environmental multivariates related to significant wave height (i.e., significant wave height, meridional wind speed, zonal wind speed, mean wave period, and mean wave direction) as the training and validation dataset. To avoid discontinuities at 0° and 360°, the mean wave direction is converted into its sine and cosine values. The training and validation datasets are Gaussian normalized according to the variables (i.e., features), and then divided into training and validation sets according to chronological order and a certain proportion of data volume. In this embodiment, specifically:

[0054] (1) In this embodiment, historical data (significant wave height, meridional wind speed, zonal wind speed, mean wave period, and mean wave direction) of a certain location in a certain sea area from 2014 to 2022 are selected as training and verification data.

[0055] (2) According to the variables (i.e. characteristics), the historical data of the location of the sea area from 2014 to 2022 (significant wave height, meridional wind speed, zonal wind speed, mean wave period, and sine and cosine values ​​of mean wave direction) are normalized by Gaussian.

[0056] (3) According to the time sequence, the normalized data of the sea area from 2014 to 2022, a total of 9 years, are divided into training set and validation set with a data volume ratio of 7:2. That is, the data from 2014 to 2020, a total of 7 years, is used as training set, and the data from 2021 to 2022, a total of 2 years, is used as validation set.

[0057] Specifically, in step 1, Gaussian normalization is performed according to the marine environmental variables to ensure that the processed training and validation datasets conform to a standard normal distribution. The formula for Gaussian normalization is:

[0058]

[0059] in, and These represent data for a specific marine environmental variable (e.g., significant wave height) before and after Gaussian normalization. This represents the average value of the variable's data before Gaussian normalization. This represents the standard deviation of the variable's data before Gaussian normalization.

[0060] Step 2: In this embodiment, the effective wave height prediction model based on multi-scale patches is determined to consist of two layers during network training. The input of the first layer is divided into three branches by a patch splitter, with patch sizes of 3h, 144h, and 720h, respectively.

[0061] Specifically, in step 2, the input of the first layer is denoted as... , length is , No. The patch size at the branch is 3h, and the step size is... ;No. The patch size at the branch is 144h, and the step size is... ;No. The patch size at the branch is 720h, and the step size is... Therefore, the training set contains 3286 × 24 = 78864 hours of effective wave height data from 2014 to 2022, which is... The first layer patch splitter for the first layer The output after branching is , No. The number of patches in the branch is The first-level patch splitter for the first layer The output after branching is , No. The number of patches in the branch is The first-level patch splitter for the first layer The output after branching is , No. The number of patches in the branch is In particular, in order to ensure Can be Divisible, for the th The branches need to be filled in at the end. The last dimension has 48 zeros; to ensure Can be Divisible, for the th The branches need to be filled in at the end. The last dimension has 336 zeros.

[0062] Step 3: Perform a temporal relative position embedding operation on each encoder input of the first layer in the multi-scale patch-based effective wave height prediction model. In this embodiment, the specific steps are as follows:

[0063] Embedded vector dimension , No. The and the first The relative position embedding formula between time series patches is:

[0064]

[0065] Among them, the sign function , , , .

[0066] Step 4: After the time series information is embedded, it enters a self-attention layer, which calculates the self-attention of the patch-level time series data in each branch through encoding operations, and performs linear projection on the relative position embedding to make it learnable, thereby capturing the relationship between patches of the same size in each branch.

[0067] Specifically, in step 4, the embedding vector dimension For the first layer of the network Branch, number The and the first The attention weights among time series patches can be represented as:

[0068]

[0069] For the first layer of the network Branch, all time series patches for the first The attention of each time series patch is:

[0070]

[0071] This represents the learnable linear projective weight coefficients.

[0072] The first-level patch splitter for the first... Branch splitting output Attention is:

[0073]

[0074] when hour, .

[0075] Step 5: Use a specific combination of residual connections, batch normalization, and feedforward networks to process the output of the self-attention layer in each encoder layer to obtain the output of each encoder layer in the first layer of the model.

[0076] Specifically, in step 5, the first layer of the model... Branch output It can be represented as:

[0077]

[0078]

[0079] in, Indicates residual connection, This represents Batch Normalization. This indicates a feedforward operation.

[0080] Step 6: Flatten, connect, and linearly project the output of each encoder layer of the first layer of the model to obtain the output of the first layer of the model.

[0081] Specifically, in step 6, the output of the first layer of the model... It can be represented as:

[0082]

[0083] in, , and This indicates that for each branch of level 1 The output is flattened and connected. and This represents the weighting coefficients and biases of the first-level linear projection operation.

[0084] Step 7: Use the output of the first layer of the model as input to the second layer, and repeat steps 2 to 6 to obtain the output of the second layer of the model.

[0085] Step 8: The output of layer 2 of the model is passed through a linear fully connected layer to obtain the output of the effective wave height prediction model based on multi-scale patches. This output is then compared with the normalized effective wave height observations to obtain the error backpropagation, and the network weights and biases are optimized and adjusted. The input dimension d of the linear fully connected layer is... model =512, output dimension c_out=1.

[0086] During network training and validation, the "Adam" optimizer was used, with Mean Square Error (MSE) as the loss function. The initial learning rate was lr=0.0001, and the learning rate was adjusted according to the following formula:

[0087]

[0088] Where epoch is the number of training rounds for all training samples.

[0089] In addition, an early stopping strategy was used: training was stopped if the loss MSE of the validation dataset did not decrease over several rounds.

[0090] During network training, the number of samples passed to the network for training in a single pass is determined to be batch_size = 32; the number of heads for multi-head attention is n_heads = 8; the feedforward neural network layer consists of two convolutions with a kernel size of 1, and the input and output dimensions are d. model = 512, inner dimension is d_ff = 2048; the activation function in the encoder is set to gelu; the random dropout rate used is dropout_rate=0.1; the number of training epochs is train_epochs = 15; the patience used for the early stopping strategy is 3.

[0091] Step 9: After training and validation, the output of the effective wave height prediction model based on multi-scale patching undergoes inverse scaling transformation to obtain the final predicted value of the corresponding prediction time.

[0092] Specifically, in step 9, an inverse scaling transformation is performed on the significant wave height data. The formula for the inverse scaling transformation is as follows:

[0093]

[0094] in, and These represent the significant wave height data before and after the inverse transform, respectively. This represents the average value of the significant wave height data before the inverse transform. This represents the standard deviation of the effective wave height data before the inverse transform.

[0095] According to steps 1-9 in the embodiment, when the prediction time step is selected as 3h, 144h, and 720h respectively, the predicted value of the significant wave height at a certain location in a certain sea area in 2023 based on the multi-scale patching model is compared with the corresponding observed value. The time series results are as follows: Figure 2 , Figure 3 and Figure 4 As shown.

[0096] The root mean square error (RMSE) metric was used to evaluate the predictive ability of the multi-scale patch-based significant wave height prediction model for the 2023 significant wave height at this sea location. The RMSE calculation formula is as follows:

[0097]

[0098] in, This represents the observed significant wave height. This represents the predicted effective wave height. This indicates the number of valid wave height data.

[0099] Table 1

[0100]

[0101] Table 1 shows the prediction accuracy of the effective wave height prediction model based on multi-scale patches. As can be seen from Table 1, the effective wave height prediction performance of the method of this invention is satisfactory in both the short and long term.

Claims

1. An effective wave height prediction method based on multi-scale patches, characterized in that, Includes the following steps: 1) For a certain latitude and longitude grid point, select historical data of multiple marine environmental variables related to significant wave height as training and validation datasets and perform Gaussian normalization according to the variables. Then, divide the training set and validation set according to time order. 2) Construct an effective wave height prediction model, the model consisting of... Layered structure, And the input is an integer; each layer's input is divided into patches of different sizes by a patch splitter. Number of branches And it is an integer; 3) No. Each branch segmented by the layer patch segmenter is input into a relative position embedding layer. After embedding time-series information, it enters a self-attention layer. The self-attention layer calculates the self-attention of the patch-level time-series data in each branch through encoding operations, and performs linear projection on the relative position embedding using learnable linear projection weight coefficients, thereby capturing the relationship between patches of the same size in each branch. Then, a combination of residual connections, batch normalization, and feedforward networks is used to process the output of the self-attention layer to obtain the model's... Layer Branch output ; The outputs of each branch are fused to obtain the model number. Layer output And feed it into the next layer of the model; 4) Using model number 1 The output of the first layer is used as input, and then input to the second layer. Repeat steps 2) to 3) to obtain the model's first layer. Layer output; 5) Model No. The output of the layer is passed through a linear fully connected layer to obtain the output of the effective wave height prediction model based on multi-scale patching. Then, it is compared with the normalized effective wave height observation to obtain the error backpropagation, and the network weights and biases are optimized and adjusted. After training and validation, the output of the effective wave height prediction model based on multi-scale patching is subjected to inverse scale transformation to obtain the final predicted value of the effective wave height. Specifically, in step 3), the embedded vector dimension For the first layer of the network Branch, number The and the first The attention weights among the time series patches are represented as follows: For the first layer of the network Support, all time series patches for the first The attention of each time series patch is: This represents the learnable linear projective weight coefficients. The first-level patch splitter for the first... Branch splitting output Attention is: when hour, .

2. The effective wave height prediction method based on multi-scale patching as described in claim 1, characterized in that: In step 1), the marine environmental variables include significant wave height, meridional wind speed, zonal wind speed, mean wave period, and mean wave direction. In order to avoid discontinuities at 0° and 360°, the mean wave direction is converted into the sine and cosine values ​​of the mean wave direction. The formula for Gaussian normalization is: in, and These represent data for a certain marine environmental variable before and after Gaussian normalization, respectively. This represents the average value of the variable's data before Gaussian normalization. This represents the standard deviation of the variable's data before Gaussian normalization.

3. The effective wave height prediction method based on multi-scale patching as described in claim 1, characterized in that: In step 2), record the first... The input of the layer is , , length is , No. Patch size at the branch and stride So, the first Layer patch splitter for the first The output after branching is , No. The number of patches in the branch is ; in, ; To ensure Can be Integer division requires padding at the end. The last dimension One zero.

4. The effective wave height prediction method based on multi-scale patching as described in claim 1, characterized in that: In step 3), the fusion operation includes flattening, connecting, and linearly projecting the output of each branch.

5. The effective wave height prediction method based on multi-scale patching as described in claim 1, characterized in that: In step 3), the first The and the first The relative position embedding formula between time series patches is: Among them, the sign function , , , , Indicates the embedding dimension.

6. The effective wave height prediction method based on multi-scale patching as described in claim 1, characterized in that: In step 3), the model's... Layer Branch output Represented as: in, Indicates residual connection, Indicates batch normalization, This indicates a feedforward operation.

7. The effective wave height prediction method based on multi-scale patching as described in claim 1, characterized in that: In step 3), the model's... Layer output Represented as: in, , and Indicates the first Each branch output of the layer is flattened and connected. and Indicates the first Weighting coefficients and biases for layer linear projection operations.

8. The effective wave height prediction method based on multi-scale patching as described in claim 1, characterized in that: In step 5), during the network training and validation process, the "Adam" optimizer is used, with the mean squared error (MSE) as the loss function and an initial learning rate of lr = 0.0001. The learning rate is then adjusted according to the following formula: Here, epoch is the number of training rounds for all training samples. Training stops when the loss MSE of the validation dataset does not decrease within a certain number of rounds.

9. The effective wave height prediction method based on multi-scale patching as described in claim 1, characterized in that: In step 5), the formula for the inverse scaling transformation is as follows: in, and These represent the significant wave height data before and after the inverse transform, respectively. This represents the average value of the significant wave height data before the inverse transform. This represents the standard deviation of the effective wave height data before the inverse transform.

Images