Wave prediction method based on transformer space-time mutual coding

Abstract

The application discloses a wave prediction method based on a Transformer space-time mutual coding, comprising the following steps: constructing a feature sample in an entire region, and screening the feature sample; constructing and training a wave prediction model based on an encoder and a decoder structure of the Transformer, using the encoder to perform space-time mutual coding on the screened feature sample, and extracting compact semantic representation; using the decoder to decode according to global space-time semantic information; and obtaining sea state characteristics of a prediction node at each moment through sampling in time and space. The application establishes effective prediction from historical meteorological elements to effective wave height and average cross-zero period. Meanwhile, based on the similarity between a conventional wave cause and an extreme wave cause, the application pre-trains a model in conventional sea state data, provides a good initialization state for the model, reduces an optimization feasible region, and effectively improves the training efficiency of an extreme sea state prediction model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence wave prediction technology, and in particular to a wave prediction method based on Transformer spatiotemporal intercoding. Background Technology

[0002] Extreme waves are high-altitude, highly destructive nonlinear ocean waves that primarily occur during extreme weather events such as typhoons, cold waves, and tsunamis. The interference and nonlinear interactions between wave components are the main causes of their generation. Currently, typhoon wave forecasting mainly relies on numerical models. This method requires first calculating the typhoon field, then using the calculated typhoon field as input to start the wave numerical model, or simultaneously coupling the typhoon field and wave field for calculation. Its disadvantages include low forecasting efficiency and the significant impact of the difficulty in forecasting the typhoon field on the predicted wave field. Therefore, a faster and more accurate method for typhoon wave forecasting is needed.

[0003] With the rapid development of computer performance and hardware, humans have begun to delve into artificial intelligence, hoping that machines can replace humans in learning, understanding, and analyzing more data, enabling machines to possess human-like autonomous thinking patterns. Machine learning is a branch of artificial intelligence, and scientists hope to achieve artificial intelligence through machine autonomous learning. Around 2000, Support Vector Machine (SVM) became the most popular machine learning method at the time. In 2006, Deep Learning was proposed, ushering in a new wave of machine learning. Artificial intelligence can improve automated processes by using artificial neural networks to simulate the structure and function of the human brain for learning. With the development of artificial intelligence, scholars at home and abroad have begun to research and explore the introduction of artificial intelligence into the atmospheric and oceanic fields. Support Vector Machine (SVM) is a supervised learning model for analyzing data in classification and regression analysis, and it has excellent generalization ability. James (2018) et al. used machine learning to predict wave elements, using Support Vector Machine (SVM) to predict the effective wave height of waves, and Multilayer Perceptron (MLP) to predict the characteristic period of waves. In terms of computational efficiency, the computation speed is nearly 5000 times faster than that of traditional numerical model simulation. Long Short-Term Memory (LSTM) is a recurrent neural network that can learn and predict time series. Shuai Liu et al. (2021) developed a machine learning framework based on the SWAN wave model and applied LSTM to complete the independent prediction of wind waves and swells. The results showed that the average absolute error percentage errors of the effective wave height, spectral peak period, wave direction, and diffusion width were less than 5.9%, 3.3%, 3.5%, and 3.3%, respectively. Using existing machine learning technology to solve the task of extreme sea state prediction faces huge challenges: (1) the factors affecting extreme marine environments are unclear; (2) the problem of effective modeling of temporal and spatial correlation features has not yet been solved. Therefore, it is very necessary to use advanced artificial intelligence technology to establish a rapid forecasting technology for extreme sea state waves that can quickly predict the path of extreme weather such as typhoons and predict extreme sea state parameters. This technology can provide early warning for ships at sea and real-time forecasts of operational and combat environments, and provide a basis for rapid decision-making in ship navigation strategy planning and overall performance design.

[0004] Giant waves in extreme sea states have complex formations, influenced by a combination of environmental factors including wind, currents, and waves, with intricate nonlinear relationships among them. Wave prediction requires consideration not only of historical meteorological information but also of the interactions between nodes within the sea area. Considering only the extreme wave field ignores meteorological information from more distant areas, while considering the entire sea area not only increases data volume but also reveals redundant features and wastes network resources. Therefore, extreme waves are the result of the coupling of large-scale spatiotemporal features and small-scale spatiotemporal features, while also incorporating nonlinear characteristics caused by topography and multi-wave interactions (such as...). Figure 1 (As shown) This further increases the difficulty of extracting effective wave features.

[0005] The task of predicting extreme sea states across all time and space involves high model complexity due to the complexity of its physical mechanisms and the complexity of point-by-point intensive prediction. However, the global extreme sea state data sample is limited and diverse, including various cyclonic phenomena such as tropical depressions, tropical storms, and typhoons. The mechanisms by which the related marine environmental factors affect the predicted objects differ, and their influence varies in strength. Therefore, conventional training can easily lead to model convergence difficulties, and complex network models are prone to having too many parameters and long training times. Summary of the Invention

[0006] In view of this, the present invention provides a wave prediction method based on Transformer spatiotemporal intercoding. This method is an efficient feature extraction method for wind-wave spatial and temporal information, which clarifies the factors affecting extreme marine environments, effectively models the spatiotemporal aspects, and can make wave predictions quickly and accurately.

[0007] Therefore, the present invention provides the following technical solution:

[0008] This invention discloses a wave prediction method based on Transformer spatiotemporal intercoding, comprising:

[0009] A feature sample is constructed for the entire region, and the feature sample is then filtered.

[0010] A wave prediction model based on a Transformer encoder and decoder structure is constructed and trained. The encoder is used to perform spatiotemporal inter-encode on the selected feature samples to extract compact semantic representations. The decoder is used to decode based on global spatiotemporal semantic information.

[0011] The sea state characteristics of the predicted nodes at each time point are obtained through temporal and spatial sampling.

[0012] Further, the feature samples are screened, including:

[0013] Determine the current feature set and initialize the importance of the current features;

[0014] Determine the target feature y, train the model with y, and test the importance of each feature;

[0015] Update the importance of each feature;

[0016] Determine if the feature set is too large; if so, delete features with low importance from the current feature set; if not, retain the current feature set.

[0017] Furthermore, the selected features include: mean wave direction, zero-crossing period, spectral peak period, significant wave height, and wind speed u and v components.

[0018] Further, the feature samples are screened, including:

[0019] A subset of features is generated from the full set of features. Based on the exhaustive subset of features, a branch boundary is added by increasing a relevance threshold to separate out the subsets that have been confirmed to yield no good results.

[0020] The feature subset is evaluated using a correlation function. A good feature subset should contain features that are highly relevant to the target.

[0021] The correlation coefficients of each feature subset with the target feature are compared with the stopping criterion. If the evaluation result is better than the stopping criterion, the process stops; otherwise, the next set of feature subsets is generated and feature selection continues.

[0022] Furthermore, spatiotemporal inter-encoding is performed on the selected feature samples based on the Transformer, including:

[0023] The data is divided into multiple sub-inputs based on the time information in the data, and each sub-input contains wind field features and wave features;

[0024] After being split according to temporal information, the sub-inputs are fed into a spatial correlation coding network. Each sub-input is independent in time but has the same structure in space.

[0025] Spatial correlation coding learns the static spatial feature map between various sea states and features through a multi-attention module, and analyzes the global ocean information at each time point, that is, the correlation between nodes;

[0026] Spatial correlation is not directly decoded after encoding; instead, the multiple feature maps output are aggregated to form temporal features.

[0027] Then, the time series after each spatial correlation encoding is used as the input of the temporal correlation encoding. The temporal correlation encoding network is highly similar to the spatial correlation encoding network. The correlation between various temporal features is learned comprehensively through the temporal autoencoder module.

[0028] After spatiotemporal decoding, the time and spatial information in the original data are effectively utilized.

[0029] Furthermore, a wave prediction model based on a Transformer encoder and decoder structure is trained, including:

[0030] The model is pre-trained on normal sea state data by taking advantage of the similarity of the physical mechanisms of normal and extreme sea states, and the training model parameters are saved.

[0031] The model parameters were fine-tuned using extreme sea state data.

[0032] Furthermore, the parameters of the trained model are saved, including:

[0033] Raw meteorological element data are obtained from the meteorological center;

[0034] Meteorological element data are sampled according to time steps;

[0035] The meteorological element samples were divided into training and validation sets;

[0036] The training set is fed into the neural network for training;

[0037] Determine if the loss curve has converged; if so, check the prediction accuracy on the validation set; if not, adjust the training strategy and network structure.

[0038] Determine if the prediction accuracy can be further improved. If so, optimize the network parameters; otherwise, save the network model and parameters.

[0039] Furthermore, sea state characteristics include wave field characteristics and wind field characteristics, with wave height and wave zero-cycle being the prediction targets.

[0040] Advantages and positive effects of the present invention:

[0041] In this invention, an extreme wave field dataset for the Northwest Pacific region (0°N-60°N, 100°E-180°E) is constructed by reanalyzing the meteorological database ERA5 and combining the spatial and temporal ranges of typhoons. With the powerful spatial and temporal information extraction capabilities of Transformer, the implicit and nonlinear information in the historical wave field is analyzed, and an effective prediction system is established from historical meteorological elements (10m wind speed u component, 10m wind speed v component, mean wave direction, mean zero-crossing period, mean spectral peak period, and significant wave height) to significant wave height and mean zero-crossing period.

[0042] This invention provides a feature selection method based on marine environmental elements. First, a feature subset is generated from the full feature set. A branch boundary is added by increasing a relevance threshold to eliminate subsets that negatively impact the results. The feature subset is then evaluated using a relevance function; an excellent feature subset should contain features with a high degree of relevance to the target.

[0043] This invention provides a pre-training strategy for extreme sea state prediction. Based on the similarity between the causes of conventional and extreme waves, model pre-training is performed on conventional sea state data, providing the model with a good initial state, reducing the feasible optimization region, and effectively improving the training efficiency of the extreme sea state prediction model. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram illustrating the complex causes and nonlinear relationships of waves in an embodiment of the present invention;

[0046] Figure 2 This is a flowchart of the feature selection process in an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of a wave prediction network based on Transformer spatiotemporal intercoding in an embodiment of the present invention;

[0048] Figure 4 This is a diagram illustrating the pre-training principle in an embodiment of the present invention.

[0049] Figure 5 This is a schematic diagram illustrating the improvement effect of pre-training on the loss curve in an embodiment of the present invention;

[0050] Figure 6 This is a flowchart illustrating the process of saving optimal network model parameters in an embodiment of the present invention. Detailed Implementation

[0051] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0052] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0053] This invention provides a wave prediction method based on Transformer spatiotemporal intercoding. The meteorological features used in this invention include wave field features and wind field features, with wave height (SWH) and wave zero period (MWP) as the prediction targets. These features are closely correlated both spatially and temporally. This invention analyzes the spatial information between nodes and the temporal information between historical moments. Therefore, a comprehensive prediction approach covering the entire sea area is adopted to strengthen the correlation between nodes and avoid the loss of spatial information. A dense prediction approach involving multiple moments across all nodes is employed, and an encoder-decoder structure is constructed to improve the accuracy of the prediction task. The encoder aggregates temporal and spatial information step by step, extracting a compact semantic representation. The decoder decodes based on global spatiotemporal semantic information. Finally, through temporal and spatial sampling, the wave height, wave period, and other sea state features of each prediction node at each moment are obtained.

[0054] This invention provides an efficient training method based on pre-training and feature selection strategies. It utilizes the similarity of the physical mechanisms of normal and extreme sea states to pre-train the model on normal sea state data, providing a good initialization state for the model, reducing the feasible optimization region, and effectively improving the training efficiency of extreme sea state prediction models, thus solving the problem of efficient training of deep network models. Feature selection techniques are used to select the horizontal and vertical components at 10 meters from the minimum sea level pressure, radius-based wind intensity, pressure at the last closed isobar, maximum sustained wind speed, maximum wind radius, gusts, eye diameter, maximum wind radius, storm speed, wave height, and geopotential height field. In the data analysis and filtering, a feature subset is first generated from the full feature set. Based on the exhaustive feature subset, a branch boundary is added by increasing the relevance threshold to separate subsets that are confirmed to yield no good results, improving selection efficiency. Then, a relevance function is used to evaluate the feature subset; a good feature subset should contain features with a high relevance to the target. The correlation coefficients of each feature subset with the target feature are compared with the stopping criterion. If the evaluation result is better than the stopping criterion, the process stops; otherwise, the next set of feature subsets is generated, and feature selection continues. Ultimately, the two reanalysis fields with the highest correlation—wind field and geopotential height field—were selected to predict typhoon movement.

[0055] This invention provides a wave prediction method based on Transformer spatiotemporal intercoding, which mainly includes three parts:

[0056] 1. Sample construction and feature selection strategies.

[0057] This embodiment uses the typhoon tracks in the Northwest Pacific region (0°N-60°N, 100°E-180°E) provided by the Tropical Cyclone Center of the China Meteorological Administration as a reference. The data source is the fifth-generation atmospheric reanalysis dataset provided by the European Centre for Medium-Range Weather Forecasts (ECMWF). The reanalysis dataset has a resolution of 1 hour, the track dataset has a resolution of 6 hours, and the time span is from 1999 to 2020.

[0058] To better consider global marine meteorological information, especially the nonlinear interactions between nodes, this embodiment does not prune samples based on typhoon locations, but instead constructs samples covering the entire Northwest Pacific region. The feature selection strategy flowchart is as follows: Figure 2 As shown, the specific steps include:

[0059] S11. Determine the current feature set and initialize the importance of the current features, that is, the degree of influence of the current features (such as temperature, wind speed, etc.) on the final predicted object (such as wave height);

[0060] S12. Determine the target feature y, train the model with y, and test the importance of each feature;

[0061] S13, Update the importance of each feature;

[0062] S14. Determine if the feature set is too large; if so, delete features with low importance from the current feature set; if not, retain the current feature set.

[0063] In the data analysis and screening process, a feature subset is first generated from the complete feature set. Based on this exhaustive subset, a branch boundary is added by incorporating a correlation threshold to separate subsets that are confirmed to yield no good results, thus improving selection efficiency. Then, a correlation function is used to evaluate this feature subset; a good feature subset should contain features with a high correlation to the target. The correlation coefficients of each feature subset with the target feature are compared with a stopping criterion. If the evaluation result is better than the stopping criterion, the process stops; otherwise, the next set of feature subsets is generated, and feature selection continues. In this embodiment, the prediction mainly focuses on two meteorological elements: significant wave height and wave spectrum peak period. Therefore, wave elements such as mean wave direction, zero-period crossing, and significant wave height are first considered as input features. Given the correlation between wind and waves, a feature selection strategy is used to screen wind field elements such as wind speed components. Finally, it is determined that the longitudinal and lateral components of wind speed at 10 meters are very important for wave prediction. In addition, this feature screening process also involves features such as minimum sea level pressure, radius-based wind intensity, pressure of the last closed isobar, and maximum sustained wind speed. Finally, the features of this embodiment are determined to be: mean wave direction, zero-period crossing, spectral peak period, significant wave height, and wind speed u and v components.

[0064] 2. Wave prediction.

[0065] The encoder aggregates temporal and spatial information step by step to extract a compact semantic representation. The decoder decodes based on global spatiotemporal semantic information and finally obtains sea state features such as wave height and wave period at each time point through temporal and spatial sampling.

[0066] The meteorological features used in this embodiment include wave field features and wind field features, with wave height and wave zero-cycle as the prediction targets. Due to the special nature of ocean information, there are close correlations between various features and between different times. This embodiment analyzes the spatial information between various nodes and the temporal information between historical times. Therefore, a comprehensive prediction method covering the entire ocean area is adopted to strengthen the correlation between various nodes and avoid information loss. A dense prediction method with multiple times is adopted, and an encoder-decoder structure is constructed to improve the accuracy of this prediction task. The prediction structure is as follows: Figure 3 As shown.

[0067] Taking wave height prediction as an example, the data is first divided into multiple sub-inputs based on temporal information. Each sub-input contains environmental elements such as wind field characteristics and wave characteristics. After being split according to temporal information, the data is fed into a spatial correlation encoding network. Each sub-input is independent in time but has the same spatial structure. Spatial correlation encoding learns static spatial feature maps between various sea states and features through a multi-attention module, analyzing global ocean information at each moment, i.e., the correlation between nodes. Spatial correlation encoding is not directly decoded; its multiple output feature maps are aggregated to form temporal features. Then, each spatially correlated time series is used as input for temporal correlation encoding. The temporal correlation encoding network is highly similar to spatial correlation encoding, comprehensively learning the correlation between various temporal features through a temporal autoencoder module. After spatiotemporal decoding, the temporal and spatial information in the original data is effectively utilized.

[0068] 3. Efficient training methods based on pre-training.

[0069] Model pre-training is performed on normal sea state data by leveraging the similarity of the physical mechanisms of normal and extreme sea states. Figure 4 This diagram illustrates the principle of pre-training, providing the model with a favorable initial state, reducing the feasible optimization region, effectively improving the training efficiency of extreme sea state prediction models, and solving the problem of efficient training of deep network models. The improvement in the loss curve after pre-training is shown below. Figure 5 As shown, Figure 5 The left image shows the image before adding pre-training. Figure 5 The right figure shows the results after adding and training.

[0070] The steps to save the training model parameters are as follows: Figure 6 As shown, the specific steps include:

[0071] S21. Obtain raw meteorological element data from the meteorological center;

[0072] S22. Collect samples of meteorological element data according to the time step;

[0073] S23. Divide the meteorological element samples into a training set and a validation set;

[0074] S24. Feed the training set into the neural network for training;

[0075] S25. Determine if the loss curve has converged; if so, check the prediction accuracy on the validation set; if not, adjust the training strategy and network structure.

[0076] S26. Determine whether the prediction accuracy can be further improved. If yes, optimize the network parameters; otherwise, save the network model and parameters.

[0077] In this embodiment of the invention, an extreme wave field dataset for the Northwest Pacific region (0°N-60°N, 100°E-180°E) is constructed by reanalyzing the meteorological database ERA5 and combining the spatial and temporal ranges of typhoons. With the powerful spatial and temporal information extraction capabilities of Transformer, the implicit and nonlinear information in the historical wave field is analyzed, and an effective prediction is established from historical meteorological elements (10m wind speed u component, 10m wind speed v component, mean wave direction, mean zero-crossing period, mean spectral peak period, and significant wave height) to significant wave height and mean zero-crossing period.

[0078] This invention provides a feature selection method based on marine environmental elements. First, a feature subset is generated from the full feature set. A branch boundary is added by increasing a relevance threshold to eliminate subsets that negatively impact the results. The feature subset is then evaluated using a relevance function; an excellent feature subset should contain features with a high degree of relevance to the target.

[0079] This invention provides a pre-training strategy for extreme sea state prediction. Based on the similarity between the causes of conventional and extreme waves, model pre-training is performed on conventional sea state data, providing the model with a good initialization state, reducing the feasible optimization region, and effectively improving the training efficiency of the extreme sea state prediction model.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A wave prediction method based on Transformer spatiotemporal inter-coding, characterized in that, include: A feature sample is constructed for the entire region, and the feature sample is then filtered. A wave prediction model based on a Transformer encoder and decoder structure is constructed and trained. The encoder is used to perform spatiotemporal inter-encode on the selected feature samples to extract compact semantic representations. The decoder is used to decode based on global spatiotemporal semantic information. The selected feature samples are spatiotemporally encoded using Transformer, including: dividing the data into multiple sub-inputs based on temporal information, each sub-input containing wind field and wave features; after splitting according to temporal information, the sub-inputs are fed into a spatial correlation encoding network, where each sub-input is independent in time but has the same spatial structure; spatial correlation encoding learns static spatial feature maps between various sea states and features through a multi-attention module, analyzing global ocean information at each moment, i.e., the correlation between nodes; spatial correlation encoding is not directly decoded, but its multiple output feature maps are aggregated to form temporal features; then each spatially correlated time series is used as input for temporal correlation encoding, and the temporal correlation encoding network is highly similar to spatial correlation encoding, comprehensively learning the correlation between various temporal features through a temporal autoencoder module; After spatiotemporal decoding, the temporal and spatial information in the original data is effectively utilized; The sea state characteristics of the predicted nodes at each time point are obtained through temporal and spatial sampling.

2. The wave prediction method based on Transformer spatiotemporal intercoding according to claim 1, characterized in that, Filtering the feature samples includes: Determine the current feature set and initialize the importance of the current features; Determine the target feature y, train the model with y, and test the importance of each feature; Update the importance of each feature; Determine if the feature set is too large; if so, delete features with low importance from the current feature set; if not, retain the current feature set.

3. The wave prediction method based on Transformer spatiotemporal intercoding according to claim 1, characterized in that, The selected features include: mean wave direction, zero-crossing period, spectral peak period, significant wave height, and wind speed u and v components.

4. The wave prediction method based on Transformer spatiotemporal intercoding according to claim 1, characterized in that, Filtering the feature samples includes: A subset of features is generated from the full set of features. Based on the exhaustive subset of features, a branch boundary is added by increasing a relevance threshold to separate out the subsets that have been confirmed to yield no good results. The feature subset is evaluated using a correlation function. A good feature subset should contain features that are highly relevant to the target. The correlation coefficients of each feature subset with the target feature are compared with the stopping criterion. If the evaluation result is better than the stopping criterion, the process stops; otherwise, the next set of feature subsets is generated and feature selection continues.

5. The wave prediction method based on Transformer spatiotemporal intercoding according to claim 1, characterized in that, Training a wave prediction model based on a Transformer encoder and decoder structure includes: The model is pre-trained on normal sea state data by taking advantage of the similarity of the physical mechanisms of normal and extreme sea states, and the training model parameters are saved. The model parameters were fine-tuned using extreme sea state data.

6. The wave prediction method based on Transformer spatiotemporal intercoding according to claim 5, characterized in that, Save the parameters of the trained model, including: Raw meteorological element data are obtained from the meteorological center; Meteorological element data are sampled according to time steps; The meteorological element samples were divided into training and validation sets; The training set is fed into the neural network for training; Determine if the loss curve has converged; if so, check the prediction accuracy on the validation set; if not, adjust the training strategy and network structure. Determine if the prediction accuracy can be further improved. If so, optimize the network parameters; otherwise, save the network model and parameters.

7. The wave prediction method based on Transformer spatiotemporal intercoding according to claim 1, characterized in that, Sea state characteristics include wave field characteristics and wind field characteristics, with wave height and wave zero-cycle being the prediction targets.

Images