A wind speed and direction prediction method based on improved TCN-LSTM

Abstract

The present invention relates to the field of wind speed and direction prediction, and discloses a wind speed and direction prediction method based on an improved TCN-LSTM. The method comprises the following steps: collecting real-time monitoring multidimensional meteorological data of offshore wind stations and performing preprocessing; performing feature fusion and feature enhancement processing on the preprocessed data, wherein the feature enhancement processing comprises spatial feature extraction, tensor reconstruction and time series modeling; inputting the data after feature fusion and feature enhancement processing into a wind speed and direction prediction model based on an improved TCN-LSTM hybrid neural network to predict the wind speed and direction at a future moment; the wind speed and direction prediction model comprises a spatiotemporal separation TCN network, a multi-head self-attention mechanism, an adaptive Dropblock mechanism, an LSTM network and a Bayesian optimization module; the method disclosed by the present invention can improve the accuracy, robustness and computational efficiency of wind power prediction in complex marine environments.

Description

Technical Field

[0001] The present invention relates to the field of wind speed and direction prediction, and in particular to a wind speed and direction prediction method based on an improved TCN-LSTM. Background Art

[0002] With the advancement of ocean observation and autonomous navigation technologies, unmanned sailboats, as zero-carbon intelligent ocean observation platforms, are becoming increasingly important tools for ocean exploration and long-term surface operations. Because their power systems primarily rely on wind energy, the ability to accurately predict wind speed and direction in real time is a key prerequisite for autonomous navigation and efficient route planning.

[0003] Existing wind speed and direction prediction methods primarily include physical models, statistical models, single deep learning models, and hybrid deep learning models. Among these, physical model-based numerical weather prediction (NWP) methods rely on complete atmospheric dynamics parameters and require high-performance computing resources, making them difficult to meet the real-time and resource-constrained prediction requirements of unmanned sailboats.

[0004] To improve forecasting efficiency and accuracy, researchers have proposed deep learning-based wind speed and direction modeling methods, such as LSTM (Long Short-Term Memory) and TCN (Temporal Convolutional Network). For example, LSTM can effectively extract the time-dependent features of wind speed sequences, but its ability to perceive the spatial gradients of wind direction changes is weak. While TCN expands the receptive field through dilated convolution, it lacks the ability to dynamically correct sudden changes in wind direction or phase drift, resulting in unstable forecasting accuracy.

[0005] Building on this foundation, some studies have attempted to integrate the spatiotemporal characteristics of wind speed and direction using hybrid models such as CNN-LSTM. However, these models exhibit significant performance degradation under extreme weather conditions, such as sudden changes in wind speed and wind direction disturbances, and generally lack robust mechanisms for addressing noise interference in ocean observation data. Furthermore, existing models often rely on traditional methods such as grid search for hyperparameters, which are computationally expensive and inefficient, making it difficult to achieve coordinated optimization of the prediction model structure and input window length.

[0006] In summary, although the existing deep learning hybrid model has integrated the spatiotemporal characteristics of wind speed and direction to a certain extent, it still has key problems such as insufficient spatiotemporal feature fusion, sensitivity to noise, and low efficiency of hyperparameter optimization. It cannot fully meet the high-precision and low-latency prediction requirements of unmanned sailboats in complex dynamic wind fields. Summary of the Invention

[0007] To solve the above technical problems, the present invention provides a wind speed and direction prediction method based on an improved TCN-LSTM, so as to improve the accuracy, robustness and computational efficiency of wind prediction in complex ocean environments.

[0008] To achieve the above object, the technical solution of the present invention is as follows:

[0009] A wind speed and direction prediction method based on an improved TCN-LSTM includes the following steps:

[0010] Step 1: Collect real-time multi-dimensional meteorological data from offshore wind stations and perform preprocessing;

[0011] Step 2: Perform feature fusion and feature enhancement processing on the preprocessed data, wherein the feature enhancement processing includes spatial feature extraction, tensor reconstruction and time series modeling;

[0012] Step 3: Input the data after feature fusion and feature enhancement into the wind speed and direction prediction model based on the improved TCN-LSTM hybrid neural network to predict the wind speed and direction at the future moment;

[0013] The wind speed and direction prediction model includes a time-space separation TCN network, a multi-head self-attention mechanism, an adaptive Dropblock mechanism, an LSTM network and a Bayesian optimization module;

[0014] The spatiotemporal separation TCN network captures multi-scale temporal features through dilated causal convolution, further enhances the spatiotemporal feature extraction capability through spatiotemporal decoupling convolution, and uses residual connections to construct cross-layer shortcuts.

[0015] The multi-head self-attention mechanism is used to model the dynamic context of features, enabling the model to capture global relevance;

[0016] The adaptive Dropblock mechanism achieves the synchronous discarding of adjacent features by generating a two-dimensional mask;

[0017] The LSTM network is a hierarchical interaction-enhanced double-layer LSTM network that achieves multi-scale temporal feature fusion at the cell state level through a cross-layer gated coupling mechanism.

[0018] The Bayesian optimization module adopts the Optuna framework and uses the TPE sampler and pruning strategy to optimize the hyperparameters of the wind speed and direction prediction model in two stages.

[0019] In the above solution, in step 1, the multidimensional meteorological data includes wind speed, wind direction, temperature, air pressure, humidity and rainfall.

[0020] In the above scheme, in step 1, the preprocessing method includes patching missing data, data standardization and wavelet denoising.

[0021] In the above solution, in step 2, the feature fusion includes sliding window statistical modeling, dynamic physical difference modeling and multi-scale feature fusion.

[0022] In the above solution, in step 2, the time series modeling adopts a two-layer LSTM network for time series modeling. The first layer of LSTM network is responsible for learning short-term dependencies, and the second layer of LSTM network further improves the modeling ability of long-term dependencies.

[0023] In the above scheme, in step 3, the mathematical expression of the causal convolution is:

[0024] ;

[0025] in, for tk The input sequence of time, is the convolution kernel weight, K is the convolution kernel size, k represents the index position in the convolution kernel, Represents the output at the current time t;

[0026] The time-space separation TCN network uses dilated causal convolution to expand the receptive field, and its mathematical form is:

[0027] ;

[0028] in, represents the value of the output sequence at time step s, is the input signal, Indicates the position of the current time step, f is a convolution kernel of length k, Represents the position index of the element in the convolution kernel, represents the convolution kernel i elements, Indicates that the input sequence is expanded according to the expansion rate d Adjusted elements, d is the expansion rate.

[0029] In the above scheme, in step 3, the activation process of the time-space separation TCN network is described as follows:

[0030] ;

[0031] ;

[0032] in, 、 is the weight matrix, is the bias, For the Tier jThe residual block is in time t The activation output, is a nonlinear activation function, For the Tier j The residual block is delayed s The output after time steps is For the Tier j The residual block is in time t The output, For the Tier j The residual block is in time t The final output, For error compensation, is the linear mapping matrix.

[0033] In the above scheme, in step 3, the adaptive Dropblock mechanism first initializes the drop probability , and then calculate the adjustment factor:

[0034] ;

[0035] Where, is the drop probability, is the block size, represents the adjustment factor, c is a constant;

[0036] Dynamically adjust according to the training progress t and calculate the dynamic drop probability:

[0037] ;

[0038] in, is the drop probability after dynamic adjustment at time t;

[0039] During training, adjust the block size according to the current step size t , and its update method is:

[0040] ;

[0041] in, is the initial block size, is the minimum block size, is the adjustment step size, T is the adjustment interval, represents the block size after adjustment at step t;

[0042] During the training process, according to the dynamic drop probability With the current block size , perform the Dropblock operation.

[0043] In the above scheme, in step 3, in the LSTM network, the input time series data is generated through the first layer of LSTM to generate the primary cell state and hidden state Afterwards, the upward gating channel After global average pooling compression, and combined with the historical hidden state of the second layer LSTM Splicing, input sigmoid function to generate gating weights , ;

[0044] Using learnable scaling factors Dynamically adjust the cross-layer information strength to enhance the features Injecting the second layer of LSTM to achieve fine-grained feature fusion, the cell state update formula is as follows:

[0045] ;

[0046] in, is the cell state of the second layer LSTM at time t, is the forget gate output of the second layer LSTM at time t, is the cell state of the second layer LSTM at time t-1, is the input gate output of the second layer LSTM at time t, The candidate cell state generated for the current time step, represents element-wise product;

[0047] At the same time, the second layer LSTM outputs the gate signal Mapping to feedback correction , superimposed on the original value of the first layer forget gate in the form of residual, and the corrected forget gate is generated after the output range is constrained by the truncation function , forming a closed-loop feedback control.

[0048] In the above solution, in step 3, the method for optimizing the hyperparameters of the wind speed and direction prediction model is as follows:

[0049] (1) Constructing hyperparameter space: Hyperparameters include learning rate, time window length, and hidden layer dimension;

[0050] (2) Formalize the hyperparameter optimization problem as follows:

[0051] ;

[0052] in, is the parameter vector, Θ is the parameter space, is a composite loss function, defined as follows:

[0053] ;

[0054] ;

[0055] ;

[0056] in, is the wind speed and wind direction error balance coefficient, , Predict losses for wind speed, Predict losses for wind direction, is the true wind speed of the i-th sample, is the predicted wind speed of the i-th sample, is the true wind direction of the i-th sample, is the predicted wind direction of the i-th sample, N is the number of samples, i represents the sample serial number;

[0057] Assume that the observation data set is , historical data is divided into high-quality sets according to the loss function and inferior sets ,in, is the γ quantile of the current test result;

[0058] Next, kernel density estimation is used to construct high-quality parameter distributions and inferior parameter distributions , KDE is the kernel density estimation function; parameter selection is based on the expected improvement criterion, namely:

[0059] ;

[0060] in, represents the positive constraint function, is the mathematical expectation, is the quality set threshold, is a high-quality parameter distribution, is a poorly parameterized distribution, is the expected improvement; the new candidate parameters are obtained by maximizing EI, that is:

[0061] ;

[0062] in, is the next solution selected in the t+1th iteration, To find the solution that maximizes EI(θ) among all possible θ;

[0063] When the test progress t / T ≤ 0.2, T is the total number of tests, set the maximum number of evaluations ; Optimize the mid-to-late stage to At the same time, an adaptive early stopping strategy is introduced to reduce computational costs:

[0064] set up is the historical optimal loss value of the t-th trial. When the following conditions are met in P consecutive training cycles, the current trial is terminated:

[0065] ;

[0066] in, is the path length of the individual in the previous round, is the path length corresponding to the historical optimal solution at the current time t, is the tolerance threshold.

[0067] Through the above technical solution, the wind speed and direction prediction method based on improved TCN-LSTM provided by the present invention has the following beneficial effects:

[0068] To enhance data robustness, the model of this invention combines meteorological and physical parameters to construct an enhanced feature set. A multi-layer convolutional structure extracts spatial features of wind speed and direction, which are then integrated with temporal features and meteorological and physical parameters to construct an enhanced feature set to improve the robustness of data representation. To capture long-term dependencies, a spatiotemporal separation TCN network is used, combined with a multi-head self-attention mechanism (MHSA) to dynamically associate global contextual information. Furthermore, to enhance the model's adaptability to complex ocean wind fields, an improved long short-term memory (LSTM) network is introduced during the feature fusion stage to improve its response to short-term dynamic changes.

[0069] For model optimization, a two-stage hyperparameter search strategy based on Bayesian optimization was employed. This strategy begins with a coarse-grained search followed by a refined search in areas of high performance, improving training efficiency and model generalization. Furthermore, the model employs adaptive DropBlock regularization technology to prevent overfitting and enhance training stability.

[0070] Compared with traditional wind speed and direction prediction methods, the present invention can more effectively capture the spatiotemporal dependence of wind speed and direction, especially when dealing with sudden meteorological changes and non-stationary wind field modeling in complex ocean environments, showing better prediction accuracy and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments or the prior art.

[0072] Figure 1 A schematic flow chart of a wind speed and direction prediction method based on an improved TCN-LSTM disclosed in an embodiment of the present invention;

[0073] Figure 2 This is the structure diagram of the multi-head self-attention mechanism;

[0074] Figure 3 This is the flow chart of the adaptive Dropblock mechanism;

[0075] Figure 4 This is the LSTM network structure diagram;

[0076] Figure 5 The wind speed prediction results of different models are shown in the figure;

[0077] Figure 6 Figure 2 shows the wind direction prediction results of different models. DETAILED DESCRIPTION

[0078] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0079] The present invention provides a wind speed and direction prediction method based on improved TCN-LSTM, such as Figure 1 As shown, the following steps are included:

[0080] Step 1: Collect real-time monitoring multi-dimensional meteorological data from offshore wind stations and perform preprocessing.

[0081] (1) Data collection: Multidimensional meteorological data from offshore wind stations (such as the MOTU EWS wind station provided by the New Zealand National Meteorological Database) are collected, including wind speed, wind direction, temperature, air pressure, humidity, and rainfall. The collected data have temporal continuity and physical constraints, and the raw data may have missing data, noise, and boundary effects. Therefore, data preprocessing is required. Preprocessing methods include patching missing data, data standardization, and wavelet denoising.

[0082] (2) Missing value processing: In order to maintain the continuity of time series data, an interpolation method based on the K-nearest neighbor (KNN) algorithm is used to fill in the missing data by calculating the Euclidean distance of each sample in the multidimensional feature space.

[0083] ;

[0084] in, p is the complete feature dimension involved in distance calculation (excluding the dimension where missing values ​​are located), For the i The sample in k The value of the dimension, Indicates the i samples and j The Euclidean distance of samples, For the j The sample ink The value of the dimension.

[0085] (3) Data standardization: Use the Z-score method to convert the original data into standardized data with a mean of 0 and a standard deviation of 1 to eliminate the dimensional differences between different physical quantities and ensure stability during model training.

[0086] ;

[0087] in, For the i The original data of the features, Represents the normalized i feature data, represents the standardized target variable, represents the original value of the target variable (wind speed or wind direction), represents the feature mean, represents the standard deviation, represents the mean of the target variable, represents the standard deviation of the target variable.

[0088] (4) Wavelet denoising: To eliminate measurement noise and data interference, the db4 wavelet basis is used for multi-scale decomposition. The number of decomposition layers L is dynamically calculated to meet the frequency band constraint:

[0089] ;

[0090] in, N is the signal length, is the wavelet filter length.

[0091] The signal is denoised by adaptive threshold and mirror symmetric extension compensation mechanism, while retaining the key features of the wind field. j Layer detail coefficient , combining Stein unbiased risk estimation with sliding standard deviation analysis to dynamically calculate the threshold:

[0092] ;

[0093] in, represents the length of detail coefficients of layer j, represents the estimated value of the noise standard deviation of the jth layer, which is obtained by the median estimate. Indicates the standard deviation of the first 5 points of the detail coefficient, Represents the adaptive threshold finally used in the soft threshold function.

[0094] For each layer of detail coefficients after wavelet decomposition , calculate the adaptive threshold Finally, a soft threshold function is used to perform coefficient compression processing, effectively suppressing high-frequency noise caused by environmental interference while retaining the key characteristics of wind field changes. Finally, the processed detail coefficients are combined with the original approximate coefficients to perform wavelet reconstruction, resulting in a time-series continuous wind speed and direction signal with minimal noise interference.

[0095] Step 2: Perform feature fusion and feature enhancement on the preprocessed data. Feature enhancement includes spatial feature extraction, tensor reconstruction, and time series modeling.

[0096] (1) Feature fusion

[0097] In order to improve the model's ability to model the nonlinear and time-varying features in wind speed and direction data, the present invention designs a feature fusion structure at the input layer. Through three steps of sliding window statistical modeling, dynamic physical differential modeling, and multi-scale feature fusion, the temporal feature expression is enriched step by step, realizing the layer-by-layer construction from the original observation data to high-dimensional dynamic features.

[0098] Level 1: Sliding window statistical modeling. , with a step size of 1) slides on the standardized original six-dimensional variables (wind speed, wind direction, air pressure, temperature, ship speed, and heading), and calculates five types of statistics for each time segment: mean, standard deviation, maximum, minimum, and median. This process produces dimensional local statistical characteristics, describing short-term trends, fluctuations and extreme value behaviors, which helps the model capture mutation and periodicity.

[0099] Level 2: Dynamic physical differential modeling. Based on meteorological physical mechanisms, multiple dynamic differential features are constructed, including: wind speed change rate ( ), wind direction angle difference (using sine and cosine or vector angle method to avoid angle jump), pressure gradient ( ), temperature gradient ( ), etc. If the data contains humidity and rainfall information, humidity / rainfall changes can be introduced as extended physical quantities. This process typically produces a number of differential dimensions (6–8) comparable to the number of original variables, which can be used to capture the physical dynamics of continuous change and improve the physical interpretability of the data.

[0100] The third level: multi-scale feature fusion. The original observation features (6 dimensions), sliding statistical features (30 dimensions) and physical dynamic features (such as 8 dimensions) are horizontally spliced ​​in the time step dimension to construct a To control computational cost and avoid overfitting, 1D convolution (or PCA) can be introduced after fusion for dimensionality reduction, extracting the most significant feature expressions (e.g., compressing to 32 dimensions), and aligning them with the time step sequence as the standard input format for the composite neural network.

[0101] Through the above three levels of processing, the original observation data is gradually transformed into high-dimensional time series features with statistical, physical and multi-scale expression capabilities, providing rich and structured input support for the subsequent spatial modeling and time series modeling of deep models, and realizing seamless connection from raw data to deep prediction.

[0102] (2) Feature enhancement

[0103] Feature enhancement also uses a three-level modular structure to improve the prediction accuracy of meteorological variables such as wind speed and direction. This method combines spatial feature extraction, tensor reconstruction, and time series modeling, using deep learning models for effective feature learning and time series prediction. The specific implementation is as follows:

[0104] Level 1: Spatial feature extraction: Feature extraction of meteorological data is performed through a multi-layer convolutional neural network. The input meteorological data includes wind speed, wind direction, temperature, air pressure, and humidity. Each layer of convolution operation increases the nonlinearity of the network through the ReLU activation function, thereby improving the network's ability to learn complex meteorological patterns. To ensure the stability of the model training process, this module adds a batch normalization operation after the convolution layer and introduces the Dropout technology in some layers for regularization to prevent overfitting. The main goal of this module is to extract representative local spatial features from the raw meteorological data, especially key information such as the spatial variation of wind speed and wind direction, the interaction effect of air pressure and humidity, and the sudden change trend of temperature.

[0105] Second level: Tensor reconstruction unit: This module converts the high-dimensional feature tensors output by the spatial feature extraction module into sequence data suitable for time series modeling. By performing tensor reconstruction on the extracted spatial features, a feature representation that combines both spatial and time series information is generated. This process effectively integrates the temporal variation patterns of meteorological data, providing efficient data input for subsequent time series modeling.

[0106] The reconstructed tensor data is used as input in sequence form for further analysis by the time series modeling module.

[0107] Level 3: Timing Modeling Module:

[0108] This module uses a two-layer long short-term memory (LSTM) network for time series modeling. LSTM effectively captures dependencies between long and short time spans through its unique gating mechanism. In this implementation, the first LSTM layer primarily learns short-term dependencies, while the second LSTM layer further enhances the modeling of long-term dependencies. This two-layer structure better handles the temporal variations of meteorological variables such as wind speed and direction, enhancing the model's ability to predict future weather data.

[0109] The input data is time series data processed by the tensor reconstruction module. After being processed by the two-layer LSTM network, the output is the predicted value of wind speed and wind direction at the future moment.

[0110] The integration of these three modules effectively improves the accuracy of forecasts for meteorological variables such as wind speed and direction, particularly in complex weather environments. Each module independently optimizes its functionality, while working closely together to significantly enhance overall forecasting.

[0111] Step three: Input the data after feature fusion and feature enhancement processing into the wind speed and direction prediction model based on the improved TCN-LSTM hybrid neural network to predict the wind speed and direction at future moments.

[0112] The wind speed and direction prediction model includes a spatiotemporal separation TCN network, a multi-head self-attention mechanism, an adaptive Dropblock mechanism, an LSTM network, and a Bayesian optimization module. The spatiotemporal separation TCN network extracts long-range dependencies, the multi-head self-attention mechanism enables cross-time feature fusion, the adaptive Dropblock mechanism performs deregularization, and the LSTM network models short-term dynamics. Bayesian optimization then dynamically searches for key hyperparameters, such as input sequence length and learning rate, to improve model performance.

[0113] The data flow is: input → TCN → multi-head self-attention → adaptive Dropblock → LSTM → output. This method integrates statistical, physical, and deep feature representations, achieving higher prediction accuracy and robustness in complex ocean wind fields than traditional methods.

[0114] (1) Time-space separation TCN network

[0115] The spatiotemporal separation TCN network captures multi-scale temporal features through dilated causal convolution, further enhances the spatiotemporal feature extraction capability through spatiotemporal decoupling convolution, and uses residual connections to construct cross-layer shortcuts.

[0116] The data flow of the spatiotemporal separation TCN network is causal dilated convolution → spatiotemporal decoupling → residual connection → output.

[0117] The temporal convolutional network is used to capture multi-scale temporal features through dilated causal convolution. The core of TCN lies in the causal convolution mechanism, which is mathematically expressed as:

[0118] ;

[0119] in, for tk The input sequence of time, is the convolution kernel weight, K is the convolution kernel size, k represents the index position in the convolution kernel, Represents the output at the current time t.

[0120] To maintain the temporal causality constraint, the sequence is padded on the left. Zero elements mean that the output depends only on the current and previous inputs, strictly preventing future information leakage. For example, when K = 3, each output is only associated with the current and previous two steps of data. This design not only conforms to the causal laws of physical systems but also accelerates model training through parallel computing.

[0121] To address the challenge of long-range dependency modeling, TCN uses dilated convolution to expand the receptive field. Its mathematical form is:

[0122] ;

[0123] in, represents the value of the output sequence at time step s, is the input signal, Indicates the position of the current time step, f is a convolution kernel of length k, Represents the position index of the element in the convolution kernel, represents the convolution kernel i elements, Indicates that the input sequence is expanded according to the expansion rate d Adjusted elements, d is the expansion rate. (When d = [1, 2, 4] and k = 3, the receptive field expands exponentially with the number of layers.) By dynamically adjusting the expansion rate, the model can capture dependency patterns at different time scales while keeping the total number of parameters constant.

[0124] To further enhance the ability of spatiotemporal feature extraction, this study designs a spatiotemporal decoupling convolution module:

[0125] ;

[0126] in, For spatial convolution, use Convolution extracts spatial correlations, Temporal convolution captures multi-scale temporal patterns through dilated convolution. This parallel architecture explicitly separates the spatiotemporal feature learning paths, resulting in higher parameter efficiency and interpretability than traditional joint convolutional methods.

[0127] Residual connections and dilated convolutions are used to prevent gradient vanishing when modeling long-range dependencies, while extracting local spatial associations through multi-level convolutional layers.

[0128] To alleviate the vanishing gradient problem in deep networks, TCN introduces residual connections to build cross-layer shortcuts. The residual module consists of the following components:

[0129] Dropout layer: randomly blocks some neurons to prevent overfitting;

[0130] ReLU activation: Enhance feature expression through nonlinear mapping;

[0131] Weight normalization: standardize the convolution kernel weights to accelerate training convergence;

[0132] Atrous Causal Convolution: Fusing dilation rate and causal constraints to model long-range temporal dependencies.

[0133] The activation process of TCN can be formally described as:

[0134] ;

[0135] ;

[0136] in, 、 is the weight matrix, is the bias, For the Tier j The residual block is in time t The activation output, is a nonlinear activation function, For the Tier j The residual block is delayed s The output after time steps is For the Tier j The residual block is in time t The output, For the Tier j The residual block is in time t The final output, For error compensation, is the linear mapping matrix.

[0137] (2) Multi-head self-attention mechanism

[0138] The multi-head self-attention mechanism is used to model the dynamic context of features, enabling the model to capture global correlations.

[0139] like Figure 2 As shown, Gaussian noise (σ=0.05) is injected into the input layer to generate anti-interference features. ; Sliding window (w=24) extracts local mean / variance and constructs time domain statistical features ; 3-layer decomposition based on Daubechies-4 wavelet to separate trend terms , periodic items and the residual Finally, multi-scale enhanced features are generated by channel splicing , achieving the coordinated optimization of noise robustness and multi-granularity temporal representation. Finally, the enhanced features are fused across scales through the multi-head attention module:

[0140] ;

[0141] in To enhance the input feature sequence, MultiHead represents the multi-head attention mechanism and LayerNorm is the layer normalization operation. In the multi-head self-attention module, the feature Z is first projected into multiple subspaces for parallel calculation. The specific calculation formula is:

[0142] Z is transformed linearly to obtain query, key, and value matrices:

[0143] ;

[0144] in, is the learned weight matrix, Represent query, key, and value matrices respectively.

[0145] Each attention head performs a dot product calculation on the query-key-value pair to obtain the attention weight:

[0146] ;

[0147] in, is the attention dimension, T represents the number of time steps, Indicates normalization.

[0148] The attention outputs in different subspaces are obtained by calculation of multiple attention heads and then concatenated:

[0149] ;

[0150] Where, is the number of attention heads, It is the output projection matrix, which is responsible for mapping the stitched results to the final output space.

[0151] After multi-head attention calculation, the final output Y is the recalibration result of the features, which represents the correlation pattern across time steps. The specific calculation is:

[0152] ;

[0153] (3) Adaptive Dropblock mechanism

[0154] To prevent overfitting of deep networks, an adaptive DropBlock mechanism is introduced to achieve synchronous discarding of adjacent features by generating a two-dimensional mask.

[0155] The probability of feature discarding is calculated using a formula, and adjacent features are discarded simultaneously through maximum pooling, thereby reducing the risk of overfitting. This mechanism draws on the idea of ​​convolution regularization and adapts to the spatiotemporal heterogeneity of offshore wind farms by dynamically adjusting the block size.

[0156] like Figure 3 As shown, in the adaptive Dropblock mechanism, the drop probability is first initialized , and then calculate the adjustment factor:

[0157] ;

[0158] Where, is the drop probability, is the block size, represents the adjustment factor, c is a constant;

[0159] Dynamically adjust according to the training progress t and calculate the dynamic drop probability:

[0160] ;

[0161] in, is the drop probability after dynamic adjustment at time t;

[0162] During training, adjust the block size according to the current step size t , and its update method is:

[0163] ;

[0164] in, is the initial block size, is the minimum block size, is the adjustment step size, T is the adjustment interval, represents the block size after adjustment at step t;

[0165] During the training process, according to the dynamic drop probability With the current block size , perform the Dropblock operation.

[0166] (4) LSTM network

[0167] like Figure 4 As shown in the figure, the LSTM network is a hierarchical interaction-enhanced two-layer LSTM network, which realizes the fusion of multi-scale temporal features at the cell state level through a cross-layer gated coupling mechanism;

[0168] The hierarchical interaction-enhanced two-layer LSTM network is characterized by achieving multi-scale temporal feature fusion at the cell state level through a cross-layer gated coupling mechanism. The specific implementation includes the following steps:

[0169] Input time series data generates primary cell state through the first layer of LSTM and hidden state Afterwards, the upward gating channel After global average pooling compression, and combined with the historical hidden state of the second layer LSTM Splicing, input sigmoid function to generate gating weights , ;

[0170] Using learnable scaling factors Dynamically adjust the cross-layer information strength to enhance the features Injecting the second layer of LSTM to achieve fine-grained feature fusion, the cell state update formula is as follows:

[0171] ;

[0172] in, is the cell state of the second layer LSTM at time t, is the forget gate output of the second layer LSTM at time t, is the cell state of the second layer LSTM at time t-1, is the input gate output of the second layer LSTM at time t, The candidate cell state generated for the current time step, represents element-wise product;

[0173] At the same time, the second layer LSTM outputs the gate signal Mapping to feedback correction , superimposed on the original value of the first layer forget gate in the form of residual, and the corrected forget gate is generated after the output range is constrained by the truncation function , forming a closed-loop feedback control.

[0174] (5) Bayesian optimization module

[0175] The Bayesian optimization module adopts the Optuna framework and uses the TPE (Tree-structured Parzen Estimator) sampler and pruning strategy to optimize the hyperparameters of the wind speed and direction prediction model in two stages.

[0176] The method for optimizing the hyperparameters of the wind speed and direction prediction model is as follows:

[0177] (1) Constructing hyperparameter space: Hyperparameters include learning rate, time window length, and hidden layer dimension;

[0178] (2) Formalize the hyperparameter optimization problem as follows:

[0179] ;

[0180] in, is the parameter vector, Θ is the parameter space, is a composite loss function, defined as follows:

[0181] ;

[0182] ;

[0183] ;

[0184] in, is the wind speed and wind direction error balance coefficient, , Predict losses for wind speed, Predict losses for wind direction, is the true wind speed of the i-th sample, is the predicted wind speed of the i-th sample, is the true wind direction of the i-th sample, is the predicted wind direction of the i-th sample, N is the number of samples, i represents the sample serial number;

[0185] Assume that the observation data set is , historical data is divided into high-quality sets according to the loss function and inferior sets ,in, is the γ quantile of the current test result;

[0186] Next, kernel density estimation is used to construct high-quality parameter distributions and inferior parameter distributions , KDE is the kernel density estimation function; parameter selection is based on the expected improvement criterion, namely:

[0187] ;

[0188] in, represents the positive constraint function, is the mathematical expectation, is the quality set threshold, is a high-quality parameter distribution, is a poorly parameterized distribution, is the expected improvement; the new candidate parameters are obtained by maximizing EI, that is:

[0189] ;

[0190] in, is the next solution selected in the t+1th iteration, To find the solution that maximizes EI(θ) among all possible θ;

[0191] When the test progress t / T ≤ 0.2, T is the total number of tests, set the maximum number of evaluations ; Optimize the mid-to-late stage to At the same time, an adaptive early stopping strategy is introduced to reduce computational costs:

[0192] set up is the historical optimal loss value of the t-th trial. When the following conditions are met in P consecutive training cycles, the current trial is terminated:

[0193] ;

[0194] in, is the path length of the individual in the previous round, is the path length corresponding to the historical optimal solution at the current time t, is the tolerance threshold. In this embodiment, P = 5, .

[0195] Adaptive learning rate scheduling and early stopping strategies shorten the training cycle and prevent overfitting, ensuring that the model converges quickly and stably.

[0196] (6) Model verification and evaluation

[0197] Experimental data and testing: Taking the data collected by the MOTU EWS wind station in New Zealand as an example, the data is divided into training set and test set according to the proportion;

[0198] Evaluation indicators: MAE, RMSE, determination coefficient ( ) Evaluate wind speed forecasts;

[0199] ;

[0200] ;

[0201] ;

[0202] in, and are the actual wind speed and the predicted wind speed, is the mean wind speed, and n is the total number of samples.

[0203] Wind direction is also evaluated using the periodic mean absolute error (MACE), periodic mean square error (CMSE), and directional accuracy (DA). For example, when the true wind direction is 355° and the predicted value is 5°, the traditional error calculation is 350°, while the periodic error is correctly identified as 10°. Furthermore, directional accuracy (DA) is introduced to measure the percentage of samples with an error of ≤10°, which directly reflects the practical value of the prediction.

[0204] ;

[0205] ;

[0206] ;

[0207] in, and is the actual and predicted wind direction angle, the indicative function The percentage of samples with an error less than 10°.

[0208] (7) Comparative experiment

[0209] The improved TCN-LSTM model is compared with various models such as BP, RNN, CNN, and traditional TCN-LSTM. Figure 5 and Figure 6 From the experimental results shown, it can be seen that the improved TCN-LSTM model of the present invention shows higher prediction accuracy and stronger robustness in the prediction of wind speed and wind direction.

[0210] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A wind speed and direction prediction method based on improved TCN-LSTM, characterized in that: The steps include: Step 1: Collect real-time multi-dimensional meteorological data from offshore wind stations and perform preprocessing; Step 2: Perform feature fusion and feature enhancement processing on the preprocessed data, wherein the feature enhancement processing includes spatial feature extraction, tensor reconstruction and time series modeling; Step 3: Input the data after feature fusion and feature enhancement into the wind speed and direction prediction model based on the improved TCN-LSTM hybrid neural network to predict the wind speed and direction at the future moment; The wind speed and direction prediction model includes a time-space separation TCN network, a multi-head self-attention mechanism, an adaptive Dropblock mechanism, an LSTM network and a Bayesian optimization module; The spatiotemporal separation TCN network captures multi-scale temporal features through dilated causal convolution, further enhances the spatiotemporal feature extraction capability through spatiotemporal decoupling convolution, and uses residual connections to construct cross-layer shortcuts. The multi-head self-attention mechanism is used to model the dynamic context of features, enabling the model to capture global relevance; The adaptive Dropblock mechanism achieves the synchronous discarding of adjacent features by generating a two-dimensional mask; The LSTM network is a hierarchical interaction-enhanced double-layer LSTM network that achieves multi-scale temporal feature fusion at the cell state level through a cross-layer gated coupling mechanism. The Bayesian optimization module adopts the Optuna framework and uses the TPE sampler and pruning strategy to optimize the hyperparameters of the wind speed and direction prediction model in two stages.

2. The wind speed and direction prediction method based on the improved TCN-LSTM according to claim 1 is characterized in that: In step 1, the multidimensional meteorological data includes wind speed, wind direction, temperature, air pressure, humidity and rainfall.

3. The wind speed and direction prediction method based on the improved TCN-LSTM according to claim 1 is characterized in that: In step 1, the preprocessing methods include patching missing data, data normalization, and wavelet denoising.

4. The wind speed and direction prediction method based on improved TCN-LSTM according to claim 1, characterized in that: In step 2, the feature fusion includes sliding window statistical modeling, dynamic physical difference modeling and multi-scale feature fusion.

5. The wind speed and direction prediction method based on improved TCN-LSTM according to claim 1, characterized in that: In step 2, the time series modeling adopts a two-layer LSTM network for time series modeling. The first layer of LSTM network is responsible for learning short-term dependencies, and the second layer of LSTM network further improves the modeling ability of long-term dependencies.

6. The wind speed and direction prediction method based on improved TCN-LSTM according to claim 1, characterized in that: In step 3, the mathematical expression of the causal convolution is: ； in, for tk The input sequence of time, is the convolution kernel weight, K is the convolution kernel size, k represents the index position in the convolution kernel, Represents the output at the current time t; The time-space separation TCN network uses dilated causal convolution to expand the receptive field, and its mathematical form is: ； in, represents the value of the output sequence at time step s, is the input signal, Indicates the position of the current time step, f is a convolution kernel of length k, Represents the position index of the element in the convolution kernel, represents the convolution kernel i elements, Indicates that the input sequence is expanded according to the expansion rate d Adjusted elements, d is the expansion rate.

7. The wind speed and direction prediction method based on improved TCN-LSTM according to claim 1, characterized in that: In step 3, the activation process of the time-space separation TCN network is described as follows: ； ； in, 、 is the weight matrix, is the bias, For the Tier j The residual block is in time t The activation output, is a nonlinear activation function, For the Tier j The residual block is delayed s The output after time steps is For the Tier j The residual block is in time t The output, For the Tier j The residual block is in time t The final output, For error compensation, is the linear mapping matrix.

8. The wind speed and direction prediction method based on improved TCN-LSTM according to claim 1, characterized in that: In step 3, in the adaptive Dropblock mechanism, the drop probability is first initialized , and then calculate the adjustment factor: ； Where, is the drop probability, is the block size, represents the adjustment factor, c is a constant; Dynamically adjust according to the training progress t and calculate the dynamic drop probability: ； in, is the drop probability after dynamic adjustment at time t; During training, adjust the block size according to the current step size t , and its update method is: ； in, is the initial block size, is the minimum block size, is the adjustment step size, T is the adjustment interval, represents the block size after adjustment at step t; During the training process, according to the dynamic drop probability With the current block size Perform a Dropblock operation.

9. The wind speed and direction prediction method based on improved TCN-LSTM according to claim 1, characterized in that: In step 3, in the LSTM network, the input time series data is generated through the first layer of LSTM to generate the primary cell state and hidden state Afterwards, the upward gating channel After global average pooling compression, and combined with the historical hidden state of the second layer LSTM Splicing, input sigmoid function to generate gating weights , ; Using learnable scaling factors Dynamically adjust the cross-layer information strength to enhance the features Injecting the second layer of LSTM to achieve fine-grained feature fusion, the cell state update formula is as follows: ； in, is the cell state of the second layer LSTM at time t, is the forget gate output of the second layer LSTM at time t, is the cell state of the second layer LSTM at time t-1, is the input gate output of the second layer LSTM at time t, The candidate cell state generated for the current time step, represents element-wise product; At the same time, the second layer LSTM outputs the gate signal Mapping to feedback correction , superimposed on the original value of the first layer forget gate in the form of residual, and the corrected forget gate is generated after the output range is constrained by the truncation function , forming a closed-loop feedback control.

10. The wind speed and direction prediction method based on improved TCN-LSTM according to claim 1, characterized in that: In step 3, the method for optimizing the hyperparameters of the wind speed and direction prediction model is as follows: (1) Constructing hyperparameter space: Hyperparameters include learning rate, time window length, and hidden layer dimension; (2) Formalize the hyperparameter optimization problem as follows: ； in, is the parameter vector, Θ is the parameter space, is a composite loss function, defined as follows: ； ； ； in, is the wind speed and wind direction error balance coefficient, , Predict losses for wind speed, Predict losses for wind direction, is the true wind speed of the i-th sample, is the predicted wind speed of the i-th sample, is the true wind direction of the i-th sample, is the predicted wind direction of the i-th sample, N is the number of samples, i represents the sample serial number; Assume that the observation data set is , historical data is divided into high-quality sets according to the loss function and inferior sets ,in, is the γ quantile of the current test result; Next, kernel density estimation is used to construct high-quality parameter distributions and inferior parameter distributions , KDE is the kernel density estimation function; parameter selection is based on the expected improvement criterion, namely: ； in, represents the positive constraint function, is the mathematical expectation, is the quality set threshold, is a high-quality parameter distribution, is a poorly parameterized distribution, is the expected improvement; the new candidate parameters are obtained by maximizing EI, that is: ； in, is the next solution selected in the t+1th iteration, To find the solution that maximizes EI(θ) among all possible θ; When the test progress t / T ≤ 0.2, T is the total number of tests, set the maximum number of evaluations ; Optimize the mid-to-late stage to At the same time, an adaptive early stopping strategy is introduced to reduce computational costs: set up is the historical optimal loss value of the t-th trial. When the following conditions are met in P consecutive training cycles, the current trial is terminated: ； in, is the path length of the individual in the previous round, is the path length corresponding to the historical optimal solution at the current time t, is the tolerance threshold.

Images