Time-frequency cooperative significant wave height prediction method and system based on cross attention and adaptive weighting

Abstract

The application discloses a time-frequency cooperative significant wave height prediction method and system based on cross attention and adaptive weighting, and the method specifically comprises the following steps: acquiring hourly historical observation data of an observation station; constructing an ATF-CAN model based on cross attention and adaptive weighting, wherein the model comprises a learnable frequency domain decomposition module, a multi-path encoder group, a two-stage cross attention module, an adaptive weighting fusion module and a position embedding decoder; training the ATF-CAN model, verifying the trained ATF-CAN model by using a verification set; and predicting the significant wave height of a test set by using the trained ATF-CAN model. The application improves the generalization prediction ability of the model under multiple sea conditions.

Description

Technical Field

[0001] This invention belongs to the field of marine meteorological technology, and in particular relates to a time-frequency collaborative effective wave height prediction method and system based on cross-attention and adaptive weighting. Background Technology

[0002] Significant wave height is a core parameter characterizing the severity of sea states, and its accurate prediction is of great guiding significance for marine engineering safety, coastal protection, and wave energy development. Extending the lead time of significant wave height prediction has become an urgent need in extreme sea state early warning and marine resource planning, and improving the accuracy of medium- and long-term predictions has important theoretical value and application prospects.

[0003] Traditional effective wave height prediction relies primarily on numerical models, with representative methods including the nearshore wave simulation model SWAN, the third-generation wave model WAM, and the third-generation full-spectrum wave numerical model (WAVEWATCH Ⅲ, WW3). These models depend on high-resolution meteorological driving fields and large amounts of computational resources, resulting in long computation cycles, poor real-time performance, and difficulty in achieving ideal prediction accuracy under complex sea conditions due to the nonlinear and non-stationary characteristics of ocean waves.

[0004] In recent years, deep learning encoding and decoding methods have provided a new technical approach for predicting significant wave height. However, existing methods still have shortcomings: first, most methods directly extract features in the original time domain, ignoring the effective use of frequency domain information; second, there is a lack of interaction between time domain and frequency domain features, making it difficult to fully integrate high-frequency details and low-frequency trends.

[0005] Therefore, there is an urgent need to develop a high-precision medium- and long-term prediction method that comprehensively utilizes time and frequency information and has adaptive fusion capabilities. Summary of the Invention

[0006] Purpose of the Invention: This invention aims to provide a time-frequency collaborative method for predicting the effective wave height of ocean waves in the medium and long term based on cross-attention and adaptive weighting. It adopts an encoding and decoding framework to solve the problems of deep learning time series prediction methods ignoring the multi-scale periodic information of time series signals in the frequency domain, as well as the isolation between the time series dependencies extracted by the time domain encoder and the spectral structure information extracted by the frequency domain encoder. This results in the inability to utilize high-frequency fluctuation details to enhance the expressive power of time series features, and the inability to use time domain context to enrich the semantic information of frequency domain trend features, leading to insufficient information utilization.

[0007] Technical solution: This invention discloses a time-frequency collaborative effective wave height prediction method based on cross-attention and adaptive weighting, specifically as follows:

[0008] Hourly historical observation data from the observation station were acquired and used as a dataset. The dataset underwent quality control and preprocessing. Then, the dataset was divided into training, validation, and test sets.

[0009] An ATF-CAN model based on cross-attention and adaptive weighting was constructed. This model includes a learnable frequency domain decomposition module, a multi-channel encoder group, a two-level cross-attention module, an adaptive weighted fusion module, and a location embedding decoder; historical observation data was then used. The input is fed into the learnable frequency domain decomposition module, which will then... Decomposed into high-frequency components and low frequency components ,Will , as well as Input to a multi-channel encoder group, output to , as well as Corresponding coding features , as well as ;based on , as well as A two-level cross-attention module is used to obtain the temporal features enhanced with high-frequency information. and low-frequency features after time-domain context enhancement The adaptive weighted fusion module uses learnable fusion weight parameters. right and By performing fusion, fusion characteristics are obtained. The location embedding decoder will Decoding the Future Effective wave height prediction sequence for each time step;

[0010] The ATF-CAN model is trained, and the trained ATF-CAN model is validated and tested using a validation set and a test set;

[0011] The effective wave height is predicted using a trained ATF-CAN model.

[0012] Furthermore, the historical observation data includes significant wave height, main wave period, mean wave period, wind speed, and sea level pressure.

[0013] Furthermore, the quality control and preprocessing specifically involve: replacing outliers in historical observation data with missing values, deleting sample rows containing missing values, and then standardizing the data using the Z-score standardization method; finally, using a sliding window strategy to construct supervised learning samples from the standardized time series data.

[0014] Furthermore, the learnable frequency domain decomposition module decomposes the original sequence. Performing a Fast Fourier Transform along the time dimension converts the time-domain signal into a frequency-domain signal. Set the learnable parameter vector Where R represents the real number field, and L is The length of M is randomly sampled from a normal distribution N(0, 0.1²) with a mean of 0 and a standard deviation of 0.1 to obtain the initial value of M. Then, the Sigmoid activation function is used to activate M. Each element is mapped to the interval (0, 1) to obtain the frequency selection weight vector. ;Will Extending to the feature dimension, the high-frequency frequency domain representation is calculated using the following formula. Frequency domain representation of low frequencies :

[0015] ;

[0016] ;

[0017] in, For element-wise product;

[0018] Finally, and Perform the inverse fast Fourier transform separately:

[0019] ;

[0020] ;

[0021] in, For inverse fast Fourier transform, This indicates the operation of taking the real part.

[0022] Furthermore, the multi-channel encoder group includes a time-domain encoder and a frequency-domain encoder; both the time-domain encoder and the frequency-domain encoder employ a single-layer gated loop unit; the original sequence... Input to the time-domain encoder, obtain ;Will and The results are obtained by inputting the data into the frequency domain encoder respectively. as well as .

[0023] Furthermore, the two-level cross-attention module includes a first-level cross-attention module and a second-level cross-attention module; , as well as The original shape of each element is (B, d), where B is the batch size and d is the hidden dimension of the corresponding encoder. , as well as Add a dimension to the second dimension respectively, so that , as well as The shape becomes (B, 1, d);

[0024] The first-level cross-attention module uses As the query vector Q, Simultaneously serving as both the key vector K and the value vector V:

[0025] ;

[0026] );

[0027] ;

[0028] ;

[0029] in, For the first One point of attention, ; To keep track of the total number inside, It is the first The query vector projection matrix of each attention head. It is the first The key vector projection matrix of each attention head It is the first The projection matrix of the value vector of each attention head, For the first The output of each attention head, where T is the transpose. For each dimension of attention head, This indicates a splicing operation. For the output of multi-head attention, Let be the attention score matrix for the i-th attention head. To output the projection matrix;

[0030] The second-level cross-attention module uses As a query vector, As key and value vectors, they are obtained using multi-head attention computation. ;

[0031] Will and The shape is restored to (B, d) by the squeeze operation.

[0032] Furthermore, the adaptive weighted fusion module introduces learnable fusion weight parameters. Using the Sigmoid function to Constrained to the interval (0, 1), the fusion coefficient is obtained. The following formula is used for and To merge:

[0033] ;

[0034] in, This is a feature of fusion.

[0035] Furthermore, a learnable position embedding matrix P is introduced into the position embedding decoder. The learnable position embedding matrix P is initialized with small variance randomness. The forward propagation process of the position embedding decoder is as follows: ;

[0036] ;

[0037] ;

[0038] ;

[0039] in, To fuse features, the shape of h is (B, d), where B represents the batch size and d is the hidden dimension of the encoder. To obtain the size of the first dimension of h, To insert a new dimension of size 1 into the first dimension of h, so that the shape of h becomes (1, B, d), Indicates view operation, exist The expression in the middle indicates that the original dimension size remains unchanged, where This indicates a single-layer gated loop unit. Indicates a linear layer. This represents the final predicted sequence. For input to The temporal tensor, for The output tensor.

[0040] A system based on a time-frequency collaborative effective wave height prediction method using cross-attention and adaptive weighting includes:

[0041] The data acquisition and processing module is used to acquire hourly historical observation data from the observation station and use it as a dataset. The dataset is then subjected to quality control and preprocessing. The dataset is then divided into training, validation, and test sets.

[0042] The ATF-CAN model, based on cross-attention and adaptive weighting, includes a learnable frequency domain decomposition module, a multi-channel encoder group, a two-level cross-attention module, an adaptive weighting fusion module, and a position embedding decoder; it converts the original sequence... The input is fed into the learnable frequency domain decomposition module, which will then... Decomposed into high-frequency components and low frequency components ,Will , as well as Input to a multi-channel encoder group, output to , as well as Corresponding coding features , as well as ;based on , as well as A two-level cross-attention module is used to obtain the temporal features enhanced with high-frequency information. and low-frequency features after time-domain context enhancement The adaptive weighted fusion module uses learnable fusion weight parameters. right and By performing fusion, fusion characteristics are obtained. The location embedding decoder will Decoding the Future Effective wave height prediction sequence for each time step;

[0043] The training module is used to train the ATF-CAN model and to validate and test the trained ATF-CAN model using a validation set.

[0044] The prediction module is used to predict the effective wave height using a trained ATF-CAN model.

[0045] A computer device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the steps of the time-frequency co-operation effective wave height prediction method based on cross-attention and adaptive weighting.

[0046] Beneficial effects:

[0047] (1) This invention employs a learnable frequency domain decomposition module for adaptive frequency domain decomposition, replacing the traditional fixed window decomposition. It uses FFT to map the sequence to the frequency domain and then introduces learnable mask parameters, achieving soft-weighted separation of frequency components through the Sigmoid function. Traditional moving average decomposition or fixed bandpass filtering relies on manually preset window sizes or cutoff frequencies, which cannot adapt to the dynamic changes in wave spectra under different sea conditions. This invention uses FFT to map the sequence to the frequency domain and then introduces learnable mask parameters, achieving soft-weighted separation of frequency components through the Sigmoid function. During training, the mask parameters are automatically optimized through backpropagation, enabling the model to learn the optimal frequency partitioning strategy from the data, overcoming the problem of insufficient adaptability of fixed parameter decomposition methods to non-stationary wave signals.

[0048] (2) To address the problem in existing methods where the time-domain encoder and frequency-domain encoder extract features independently, resulting in the inability to collaboratively utilize high-frequency fluctuation details and low-frequency trend information of waves, this invention designs a two-level cross-attention mechanism to achieve bidirectional interaction of time and frequency information. The first level queries high-frequency components using time-domain information to inject fluctuation details into the time-series representation; the second level queries the enhanced time-domain representation using low-frequency information to integrate contextual semantics into the trend representation. This cascaded cross-design breaks down the information barrier between the time and frequency domains, enabling deep interaction and complementary enhancement of features from both domains, significantly improving the model's spatiotemporal fusion capability for multi-scale wave features.

[0049] (3) This invention introduces a learnable parameter λ and combines it with a Sigmoid activation function, enabling the model to automatically learn the optimal fusion ratio between time-domain and frequency-domain signals during training, thus achieving adaptive weighted fusion. Dynamic adaptive adjustment of the fusion strategy is achieved under different marine environmental conditions, overcoming the limitations of fixed fusion methods and improving the model's generalization prediction capability under multiple sea state conditions.

[0050] (4) To address the problem of gradual error accumulation and prediction curve degradation caused by the lack of temporal position information in traditional GRU decoders during long sequence prediction, this invention incorporates a learnable position embedding matrix into the decoder to enhance long sequence decoding. This allows the decoder to perceive its relative temporal position at each step of the prediction sequence (step 1, step 24, or step 48), thereby dynamically adjusting the decoding strategy based on positional differences. This design effectively alleviates the error accumulation effect of multi-step decoding in long sequence prediction, ensuring that the prediction curve maintains a consistent fluctuation trend with the true wave height over a long time span. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the overall method of the present invention;

[0052] Figure 2 ATF-CAN model architecture diagram;

[0053] Figure 3 This is a diagram of the cross-attention module structure.

[0054] Figure 4 This is a diagram of the adaptive weighted fusion module. Detailed Implementation

[0055] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0056] like Figure 1 As shown, this invention proposes a time-frequency collaborative method for predicting the effective wave height of ocean waves in the medium to long term, which integrates cross-attention and adaptive weighting strategies, and includes the following steps:

[0057] (1) Hourly observation data of NDBC 46042 buoy station from 2019 to 2023 were obtained with a time resolution of 1h. Five features were selected: significant wave height (WVHT), dominant wave period (DPD), average wave period (APD), wind speed (WSPD), and sea level pressure (PRES). After removing outliers (in this embodiment, outliers such as 99, 999, and 9999 were considered outliers) and missing values, Z-score standardization was performed (statistics were obtained only from the training set). A sliding window sample was constructed with 96h as the input window and 48 and 96h as the prediction windows. The training set, validation set, and test set were divided into three sets in a 7:1:2 ratio according to time sequence.

[0058] (2) Construct the ATF-CAN model, a time-frequency collaborative method for predicting the effective wave height of ocean waves in the medium and long term based on cross-attention and adaptive weighting. The ATF-CAN model includes a learnable frequency domain decomposition module, a multi-channel encoder group, a two-level cross-attention module, an adaptive weighted fusion module, and a position embedding decoder. Among them, the learnable frequency domain decomposition module is used to separate the input sequence into high-frequency components and low-frequency components; the multi-channel encoder group includes a time-domain encoder and a frequency-domain encoder (sharing weights, encoding high-frequency and low-frequency respectively), both of which use gated recurrent units (GRU); the two-level cross-attention module includes a first-level cross-attention and a second-level cross-attention, which are used to realize bidirectional information interaction between time-domain features and frequency-domain features; the adaptive weighted fusion module introduces a learnable fusion weight parameter λ, which is mapped to the (0, 1) interval through the Sigmoid function, and performs adaptive weighted fusion of the time-domain path and the frequency-domain path; the position embedding decoder replaces the traditional all-zero input with a learnable position embedding matrix, and generates a multi-step prediction sequence by gradually decoding through GRU.

[0059] (3) The MSE (Mean Squared Error) loss function and Adam optimizer are used for training, combined with ReduceLROnPlateau (a learning rate decay scheduler based on the validation platform) for learning rate decay and Early Stopping to prevent overfitting. MSE, MAE (Mean Absolute Error), and R² (R-Squared coefficient of determination) are used as evaluation metrics. The predicted results are de-standardized to obtain the true wave height value.

[0060] Step (1) includes the following steps:

[0061] (1.1) Obtain buoy wave significant wave height data from NDBC 46042 buoy station. The selected marine environmental features include significant wave height (WVHT), dominant wave period (DPD), average wave period (APD), wind speed (WSPD), and sea level pressure (PRES), with a time resolution of 1 hour. Five consecutive years of observation records were selected as the original dataset.

[0062] (1.2) Perform quality control and preprocessing on the raw data. First, replace the outlier markers (such as missing test identifiers like 99, 999, 9999) in the data with missing values. Subsequently, rows containing missing values ​​are deleted to ensure the integrity and continuity of the input data. Finally, Z-score normalization is applied to standardize each feature, eliminating the adverse effects of differences in units and orders of magnitude between different features on model training. The Z-score normalization formula is as follows:

[0063] ;

[0064] in, The original data values, The feature mean calculated on the training set. The standard deviation of the features calculated on the training set. This is the standardized data.

[0065] (1.3) A sliding window strategy is used to construct supervised learning samples from the standardized time series data. The input sequence length is set to L (historical observation window), and the prediction length is set to H (future prediction window; in this embodiment, H is a long-term series). For a time series of length N, starting from the first time step, L consecutive time steps are taken as input features X, and the effective wave height values ​​of the next H time steps are taken as the prediction target Y. The sliding window moves one time step at a time, generating a total of NL-H+1 supervised learning samples. The shape of each input sample is (L, C), where C=5 is the number of features; the shape of each target sample is (H, 1), meaning only the effective wave height is predicted.

[0066] like Figure 2 As shown, step (2) includes the following steps:

[0067] (2.1) Learnable frequency domain decomposition module

[0068] Traditional time series decomposition methods (such as moving average filtering or fixed bandpass filtering) rely on manually preset window sizes or cutoff frequencies, and cannot adaptively adjust to the dynamic changes in wave spectra under different sea conditions. Therefore, this module maps the sequence to the frequency domain using Fourier transform and introduces learnable mask parameters to achieve data-driven soft frequency domain decomposition. The specific process is as follows:

[0069] Frequency domain transformation. For the input sequence... Performing a Fast Fourier Transform (FFT) along the time dimension converts the time-domain signal into a frequency-domain representation. According to the properties of the Fourier transform, a length of... The real number sequence is obtained after FFT. Each frequency component is an independent frequency component, and each frequency component corresponds to a sine wave with a specific period. For example, the first frequency point represents the component that completes one oscillation within one period (96h), the fourth frequency point represents the component with a period of 24h (corresponding to a diurnal tide), the eighth frequency point represents the component with a period of 12h (corresponding to a semi-diurnal tide), and so on. Each frequency component consists of a real part and an imaginary part, which encode the amplitude and phase information of the sine wave at that frequency, respectively.

[0070] Learnable frequency selection weights. Introducing a learnable parameter vector. ,Right now It is a vector of length 49, where each element corresponds to a frequency position. The initial values ​​are randomly sampled from a normal distribution N(0, 0.1²) with a mean of 0 and a standard deviation of 0.1. Then, the Sigmoid activation function is applied to... Each element is mapped to the interval (0, 1), resulting in the frequency selection weight vector:

[0071] ;

[0072] in, For the Sigmoid function, This represents the weighting coefficient by which each frequency component is assigned to the low-frequency path.

[0073] Weight vector The physical meaning is: for the first Each frequency component has a weight. This indicates the proportion of that frequency component entering the low-frequency path. Specifically, if... A value approaching 1 indicates that the frequency component almost completely enters the low-frequency path; if Approaching 0 indicates that the frequency component has almost completely entered the high-frequency path; if Taking the median value (such as 0.5) indicates that the frequency component is simultaneously allocated to two paths, each accounting for half. This mechanism is fundamentally different from the traditional one-size-fits-all hard division (such as classifying everything below the cutoff frequency as low frequency and everything above it as high frequency), and is a continuous and differentiable soft separation.

[0074] Spectrum-weighted separation. Utilizing weight vectors. For the original spectrum Perform soft separation. In the frequency domain, the weight vector... After extending to the feature dimension, an element-wise product (Hadamard product, denoted as ⊙) is performed with the original spectrum to obtain the frequency domain representations of low and high frequencies:

[0075] ;

[0076] ;

[0077] In the low-frequency path, frequencies with high weights are fully preserved, while those with low weights are suppressed; in the high-frequency path, frequencies with low weights are fully preserved, while those with high weights are suppressed.

[0078] It is important to note here that... and The weights are complementary—the sum of the proportion of a frequency component retained in the low-frequency path and the proportion retained in the high-frequency path is always 1. This means that the information of the original signal is not lost during the decomposition process: the two paths combined completely preserve all the information of the original spectrum.

[0079] Time-domain restoration. (The following is a list of steps, not a direct translation.) and Perform an inverse fast Fourier transform (IFFT) to restore the weighted spectrum back to the time domain, and take the real part of the complex result:

[0080] ;

[0081] ;

[0082] in, and All are real tensors, with shapes identical to the original input. The values ​​are the same (B, L, C). Here, B represents the batch size, i.e., the number of samples fed into the model for training each time. The above describes the calculation methods for low-frequency and high-frequency trend sequences. To extract the real part, the imaginary part is discarded, resulting in a purely real number.

[0083] End-to-end learning. As trainable parameters of the model, they are continuously updated during training via backpropagation. The model automatically adjusts the weights of each frequency position based on the loss gradient of the prediction task—for frequency components useful for wave height prediction, the corresponding weights are optimized in a direction beneficial to the task; for components useless for prediction, the weights are shifted in a direction of suppression. Throughout the process, there is no need to manually specify which frequencies are low-frequency or high-frequency; the model autonomously discovers the optimal frequency partitioning strategy from the data. If the tidal cycle (12h and 24h) is most critical for effective wave height prediction, The weights of the corresponding frequency positions may tend to be higher (entering the low-frequency path and retaining its complete information) or lower (entering the high-frequency path and being processed along with other detailed fluctuations) after training, which is entirely determined by the model automatically end-to-end based on the prediction requirements.

[0084] (2.2) Multi-channel GRU encoder group

[0085] After frequency domain decomposition, the original sequence High-frequency components and low frequency components The three information streams are fed into the encoder group for feature encoding. The encoder group includes two configurations:

[0086] Time-domain encoder (enc_time): Receives the raw input sequence. The aim is to extract comprehensive temporal dependencies containing all frequency components from a complete time-series signal. A single-layer gated recurrent unit (GRU) is used, with the hidden dimension set to d=64. Through its reset and update gate mechanisms, the GRU can adaptively control the retention and forgetting of historical information, effectively mitigating the gradient vanishing problem in long sequence modeling.

[0087] Frequency domain encoder (enc_freq): This encoder is shared twice—once for the high-frequency components. Encoding yields high-frequency representation For low-frequency components Encoding yields low-frequency representation The frequency domain encoder also employs a single-layer GRU, with the same structure and parameter configuration as the time domain encoder, but with independent parameters. This design offers three advantages: first, it reduces the number of model parameters, lowering the risk of overfitting; second, it forces the encoder to learn feature extraction patterns applicable to both high-frequency and low-frequency components, improving feature generalization; and third, the high-frequency and low-frequency encoding results reside in the same semantic space, facilitating subsequent cross-attention operations.

[0088] GRU through its reset gate and the update gate The adaptive control information flow is represented by the following formula:

[0089]

[0090]

[0091]

[0092]

[0093] in, It is a sigmoid activation function; To reset the gate weight matrix, the hidden state from the previous time step is... With current input After splicing, a linear transformation is performed, and the amount of historical information retained is determined after Sigmoid activation. To update the gate weight matrix, the concatenated [ , A linear transformation is performed, and after activation by the Sigmoid function, the ratio of the old to the new state is controlled—a high value retains historical information, while a low value accepts new information. Given the candidate hidden state weight matrix, the concatenated matrix... (in The gate filter (which has been reset) undergoes a linear transformation, and after activation, candidate states are generated. , The hidden state of the previous time step stores the historical information of the sequence up to time step t-1; This is the input for the current moment.

[0094] The two encoders take the hidden state of the last time step of the GRU as their output, with a shape of (B, d): , Let be the hidden state tensor of GRU at all time steps and all layers, with shape (num_layers (number of layers), B, d); Take the hidden state of the last layer at the last moment, with shape (B, d).

[0095] The semantics output by the encoder are as follows:

[0096] The temporal-domain synthesis representation encodes the complete temporal dependency information of the original sequence;

[0097] High-frequency representation encodes the short-term fluctuation and noise characteristics in wave signals;

[0098] Low frequency indicates that the long-term trend and tidal information in the wave signal are encoded.

[0099] (2.3) Two-level cross-attention module

[0100] like Figure 3 As shown, the time-domain encoder and the frequency-domain encoder operate independently, and the output is... Although derived from the same input sequence, they encode features from different domains. To achieve deep fusion of time-domain and frequency-domain information, this invention designs a two-level cross-attention mechanism, enabling bidirectional information exchange and enhancement between the features of the two domains.

[0101] First-level cross-attention is represented in the time domain. The query vector is represented by high frequency. Given a key vector and a value vector, the time-domain representation is enhanced using high-frequency fluctuation details, resulting in... Second-level cross-attention is represented by low frequency. For the query, in an enhanced time-domain representation Using the key and value, the time-domain context information is incorporated into the low-frequency representation to obtain... Cross-attention employs a multi-head attention mechanism (h=2), as shown in the following formula:

[0102]

[0103]

[0104]

[0105] in, It is the query vector projection matrix of the i-th attention head. It is the key vector projection matrix of the i-th attention head. It is the projection matrix of the value vector of the i-th attention head. This represents the i-th attention head. For the output of the i-th attention head, It is the spliced ​​output projection, the output projection matrix of multi-head attention, which fuses multi-head information. The two-level cross-attention design enables bidirectional information exchange between high-frequency details and low-frequency trends through the enhanced temporal domain representation.

[0106] First-level cross-attention (CrossAttn1): Represented in the time domain As a query vector, it is represented by high frequency. Simultaneously serving as both a key and a value vector, this operation essentially involves using time-domain information to query the most relevant patterns within high-frequency components, thereby injecting high-frequency fluctuation details into the time-domain representation. The calculation process for multi-head attention is as follows:

[0107] ;

[0108] );

[0109] ;

[0110] ;

[0111] in, For each dimension of attention head, The number of attention heads (h=2 in this invention). Divide by The purpose is to perform scaling to prevent the softmax gradient from vanishing due to excessively large inner product values. and After adding a second dimension, input cross-attention (changing the shape to (B, 1, d)), and then restore the output to (B, d) using a squeeze operation. The output of the first-level cross-attention is denoted as... This represents the time-domain representation after high-frequency information enhancement. Let be the attention score matrix for the i-th attention head.

[0112] Second-level cross-attention (CrossAttn2): Represented by low frequency. As a query vector, in an enhanced time-domain representation As key and value vectors, the essence of this operation is to use low-frequency trend information to query the relevant contextual patterns in the enhanced time-domain representation, thereby integrating the enhanced time-domain contextual information into the low-frequency representation. The calculation process is the same as the first-level cross-attention, and the output is denoted as... , representing the low-frequency representation after time-domain context enhancement.

[0113] The two-level cross-attention mechanism creates an information flow path from the time domain to high frequencies and then to low frequencies: high-frequency details are first enhanced in the time domain representation, and the enhanced time domain representation then enriches the low-frequency trends. This cascaded design allows the model to progressively fuse feature information at different granularities.

[0114] (2.4) Adaptive weighted fusion module

[0115] like Figure 4 As shown, after two levels of cross-attention, two enhanced representations are obtained. and The traditional method is to directly add the two together. The implicit assumption is that time-domain and frequency-domain information always contribute equally to the prediction. However, this assumption does not hold under different sea state conditions.

[0116] To enable the model to adaptively adjust the fusion strategy under different sea conditions, this invention introduces a learnable fusion weight parameter. It is initialized to 0.5. The fusion coefficient is obtained by constraining it to the interval (0, 1) using the Sigmoid function. The final fusion is represented as:

[0117]

[0118] in, The weights of the time-domain path. When As the value approaches 1, the model becomes more reliant on the time-domain path enhanced by high frequency. It is suitable for sea states where wind and waves dominate and short-term fluctuation information is crucial; when As the frequency approaches zero, the model becomes more reliant on the frequency domain path enhanced by the time domain. This method is suitable for sea states dominated by swells and where low-frequency trend information is crucial. λ is a learnable scalar parameter that is automatically optimized through backpropagation during model training, eliminating the need for manual setting and enabling the model to adaptively adjust to different data distributions.

[0119] (2.5) Location Embedding Decoder

[0120] The decoder is responsible for merging the representations Decoding the Future The effective wave height prediction sequence at each time step. Traditional decoders use all-zero vectors as input, lacking temporal location information during decoding, causing the model to be unable to distinguish the positional differences between different prediction steps.

[0121] This invention introduces a learnable position embedding matrix into the decoder. Each row of this matrix corresponds to a time step in the prediction sequence, encoding the relative position information of that step within the prediction sequence. The learnable position embedding matrix P is initialized with small variance (mean 0, standard deviation σ = 0.02) to ensure that the position signals tend to zero in the early stages of training. This ensures that the decoder first uses the semantic information of the fused representation for prediction, and then gradually learns the position differences at different time steps. The selection of the initial standard deviation follows the open-source implementation conventions of the Transformer series models, and experiments have verified that it can guarantee stable convergence of training on this task. It is optimized along with other model parameters during training.

[0122] The forward propagation process of the decoder is as follows:

[0123]

[0124]

[0125]

[0126]

[0127] in, for Size of the first dimension The shape is (B, d) = (batch size, 64), so The current batch contains 32 samples. ,but . For view operations, copy a tensor multiple times along a certain dimension without consuming new memory. The position embedding is extended from (1, 48, 64) to (32, 48, 64), and each sample in the batch uses the same position encoding. exist The hidden dimension is kept unchanged. The GRU decoder uses a single-layer structure, with the hidden dimension being the same as that of the encoder. The input to the decoder is no longer a vector of all zeros, but a positional embedding rich in positional information, so that the GRU knows which future time step it is predicting at each decoding step. express pass The operation adjusts the state from (B, d) to (1, B, d) as the initial hidden state of the GRU. The output of the GRU at each time step is mapped to a scalar wave height value through a fully connected Linear layer. After the first step of decoding, the final predicted sequence is obtained. , Indicates a linear layer. For input to The temporal tensor, for The output tensor.

[0128] Step (3) includes the following steps:

[0129] (3.1) Model training configuration. Mean Squared Error (MSE) is used as the loss function:

[0130]

[0131] The optimizer used is Adam, and the initial learning rate is set to... The weight decay factor is set to To apply Regularization. The learning rate scheduling strategy uses ReduceLROnPlateau, monitoring the validation set. If the learning rate does not decrease for 5 consecutive epochs, it will be decayed to 0.5 times the current rate. Mixed precision training is enabled during training to reduce GPU memory usage and accelerate computation. An Early Stopping strategy is adopted, with a patience value set to 10 epochs: if the validation set... If the lowest record is not broken for 10 consecutive epochs, training will be terminated early, and the model parameters that performed best on the validation set will be restored. The maximum number of training epochs is set to 200.

[0132] (3.2) Model Evaluation Metrics. Four quantitative metrics are used to comprehensively evaluate model performance:

[0133] Mean square error ( ): This measures the mean squared deviation between the predicted and actual values, imposing a higher penalty on larger errors. The formula is:

[0134]

[0135] Mean absolute error ( ): Measures the average absolute deviation between predicted and actual values, and is insensitive to outliers. The formula is:

[0136]

[0137] Coefficient of determination ( ): Measures the extent to which the model explains the variance of the target variable, with values ​​ranging from -∞ to 1. 1 indicates perfect prediction, 0 indicates that the prediction is equivalent to directly predicting the mean, and a negative value indicates that the prediction is better than the mean prediction.

[0138]

[0139] in, For the true value, For predicted values, The mean of the true values. This represents the total number of test samples. The value ranges from (-∞, 1], and the closer it is to 1, the stronger the model's ability to explain the wave height variance.

[0140] (3.3) Destandardization of Prediction Results. Since the model training used data standardized by Z-scores, the predicted values ​​output by the model are also standardized. To obtain the effective wave height prediction values ​​with true physical dimensions, the model output needs to be destandardized:

[0141]

[0142] in, The standardized predicted values ​​output by the model. The standard deviation of the effective wave height in the training set. The mean of the effective wave heights in the training set. This is the predicted effective wave height after destandardization.

[0143] This embodiment compares the model of this invention with seven deep learning models, including: Transformer, DLinear, iTransformer, ModernTCN, SegRNN, SOFTS, and TimeMixer. All models were trained and evaluated under the same data partitioning and experimental conditions. The MSE, MAE, and R² results on the test set are shown in Table 1.

[0144] Table 1. Comparison of test set results for 8 models

[0145]

[0146] As can be seen from the table above:

[0147] (1) In the 48-hour prediction task, the ATF-CAN model proposed in this invention achieved the best performance in all evaluation indicators (MSE=0.2467m, MAE=0.3256m, R²=0.8360), the second best model SOFTS (MSE=0.2477m, MAE=0.3262m, R²=0.8350), and the classic linear model DLinear (MSE=0.2569). The traditional Transformer model achieved the worst performance, with an MSE value of 0.2877m.

[0148] (2) On the prediction task with a longer time scale of 96 hours, the ATF-CAN model still maintains the best performance (MSE=0.2903m, MAE=0.3721m, R²=0.8293), which is superior to SOFTS (MSE=0.2952m), DLinear (MSE=0.3009m) and Transformer (MSE=0.3349m). At the same time, when comparing the index results of the 48-hour prediction step, the performance degradation of each model is similar. The experimental results show that ATF-CAN can more effectively capture the multi-scale time-frequency characteristics of wave signals through learnable frequency domain decomposition and two-level cross-attention mechanism.

[0149] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present invention will not describe the various possible combinations separately.

Claims

1. A time-frequency collaborative effective wave height prediction method based on cross-attention and adaptive weighting, characterized in that, Specifically: Hourly historical observation data from the observation station were acquired and used as a dataset. The dataset underwent quality control and preprocessing. Then, the dataset was divided into training, validation, and test sets. An ATF-CAN model based on cross-attention and adaptive weighting was constructed. This model includes a learnable frequency domain decomposition module, a multi-channel encoder group, a two-level cross-attention module, an adaptive weighted fusion module, and a location embedding decoder; historical observation data was then used. The input is fed into the learnable frequency domain decomposition module, which will then... Decomposed into high-frequency components and low frequency components ,Will , as well as Input to a multi-channel encoder group, output to... , as well as Corresponding coding features , as well as ;based on , as well as A two-level cross-attention module is used to obtain the temporal features enhanced with high-frequency information. and low-frequency features after time-domain context enhancement ; The adaptive weighted fusion module uses learnable fusion weight parameters. right and By performing fusion, fusion characteristics are obtained. The location embedding decoder will Decoding the Future Effective wave height prediction sequence for each time step; The ATF-CAN model is trained, and the trained ATF-CAN model is validated and tested using a validation set and a test set; The effective wave height is predicted using a trained ATF-CAN model.

2. The time-frequency collaborative effective wave height prediction method based on cross-attention and adaptive weighting according to claim 1, characterized in that, The historical observation data includes significant wave height, main wave period, average wave period, wind speed, and sea level pressure.

3. The time-frequency collaborative effective wave height prediction method based on cross-attention and adaptive weighting according to claim 1, characterized in that, The quality control and preprocessing are as follows: outliers in historical observation data are uniformly replaced with missing values, sample rows containing missing values ​​are deleted, and then the Z-score standardization method is used for standardization. Finally, a sliding window strategy is used to construct supervised learning samples from the standardized time series data.

4. The time-frequency collaborative effective wave height prediction method based on cross-attention and adaptive weighting according to claim 1, characterized in that, The learnable frequency domain decomposition module decomposes the original sequence. Perform a Fast Fourier Transform along the time dimension to convert the time-domain signal into a frequency-domain signal. ; Set learnable parameter vector Where R represents the real number field, and L is The length of M is randomly sampled from a normal distribution N(0, 0.1²) with a mean of 0 and a standard deviation of 0.1 to obtain the initial value of M. Then, the Sigmoid activation function is used to activate M. Each element is mapped to the interval (0, 1) to obtain the frequency selection weight vector. ;Will Extending to the feature dimension, the high-frequency frequency domain representation is calculated using the following formula. Frequency domain representation of low frequencies : ； ； in, For element-wise product; Finally, and Perform the inverse fast Fourier transform separately: ； ； in, For inverse fast Fourier transform, This indicates the operation of taking the real part.

5. The time-frequency collaborative effective wave height prediction method based on cross-attention and adaptive weighting according to claim 1, characterized in that, The multi-channel encoder group includes a time-domain encoder and a frequency-domain encoder; both the time-domain encoder and the frequency-domain encoder employ a single-layer gated loop unit; the original sequence is... Input to the time-domain encoder, obtain ;Will and The results are obtained by inputting the data into the frequency domain encoder respectively. as well as .

6. The time-frequency collaborative effective wave height prediction method based on cross-attention and adaptive weighting according to claim 5, characterized in that, The two-level cross-attention module includes a first-level cross-attention module and a second-level cross-attention module; , as well as The original shape of each element is (B, d), where B is the batch size and d is the hidden dimension of the corresponding encoder. , as well as Add a dimension to the second dimension respectively, so that , as well as The shape becomes (B, 1, d); The first-level cross-attention module uses As the query vector Q, Simultaneously serving as both the key vector K and the value vector V: ； )； ； ； in, For the first One point of attention, ; To keep track of the total number inside, It is the first The query vector projection matrix of each attention head It is the first The key vector projection matrix of each attention head It is the first The projection matrix of the value vector of each attention head, For the first The output of each attention head, where T is the transpose. For each dimension of attention head, This indicates a splicing operation. For the output of multi-head attention, Let be the attention score matrix for the i-th attention head. To output the projection matrix; The second-level cross-attention module uses As a query vector, As key and value vectors, they are obtained using multi-head attention computation. ; Will and The shape is restored to (B, d) by squeezing the dimensions.

7. The time-frequency collaborative effective wave height prediction method based on cross-attention and adaptive weighting according to claim 1, characterized in that, The adaptive weighted fusion module introduces learnable fusion weight parameters. Using the Sigmoid function to Constrained to the interval (0, 1), the fusion coefficient is obtained. The following formula is used for and To merge: ； in, This is a feature of fusion.

8. The time-frequency collaborative effective wave height prediction method based on cross-attention and adaptive weighting according to claim 1, characterized in that, The location embedding decoder introduces a learnable location embedding matrix P, which is initialized with small variance randomness. The forward propagation process of the location embedding decoder is as follows: ; ； ； ； in, To fuse features, the shape of h is (B, d), where B represents the batch size and d is the hidden dimension of the encoder. To obtain the size of the first dimension of h, To insert a new dimension of size 1 into the first dimension of h, such that the shape of h becomes (1, B, d), Indicates view operation, exist The expression in the middle indicates that the original dimension size remains unchanged, where This indicates a single-layer gated loop unit. Indicates a linear layer. This represents the final predicted sequence. For input to The temporal tensor, for The output tensor.

9. A system applied to the time-frequency collaborative effective wave height prediction method based on cross-attention and adaptive weighting as described in claim 1, characterized in that, include: The data acquisition and processing module is used to acquire hourly historical observation data from the observation station and use it as a dataset for quality control and preprocessing. The dataset is then divided into training, validation, and test sets. The ATF-CAN model, based on cross-attention and adaptive weighting, includes a learnable frequency domain decomposition module, a multi-channel encoder group, a two-level cross-attention module, an adaptive weighting fusion module, and a position embedding decoder; it converts the original sequence... The input is fed into the learnable frequency domain decomposition module, which will then... Decomposed into high-frequency components and low frequency components ,Will , as well as Input to a multi-channel encoder group, output to... , as well as Corresponding coding features , as well as ; based on , as well as A two-level cross-attention module is used to obtain the temporal features enhanced with high-frequency information. and low-frequency features after time-domain context enhancement The adaptive weighted fusion module uses learnable fusion weight parameters. right and By performing fusion, fusion characteristics are obtained. ; The position embedding decoder will Decoding the Future Effective wave height prediction sequence for each time step; The training module is used to train the ATF-CAN model and to validate and test the trained ATF-CAN model using a validation set. The prediction module is used to predict the effective wave height using a trained ATF-CAN model.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the time-frequency collaborative effective wave height prediction method based on cross-attention and adaptive weighting as described in any one of claims 1 to 8.

Images