Wind power prediction method based on Temporal Fusion Transformer and EHO optimization algorithm

Abstract

The invention designs a wind power prediction method based on improved time sequence fusion Transform (ITFT). According to the method, a Mama module is adopted to replace a traditional LSTM encoder-decoder structure, so that the long sequence modeling capability is remarkably improved; designing a wind speed prediction network (WFN) to generate future wind speed prediction as auxiliary input of the decoder; the improved EHO algorithm is applied to carry out hyper-parameter optimization, and chaos initialization, a fitness-distance balance strategy and a hybrid variation mechanism are fused. According to the method, the technical bottlenecks of an existing wind power prediction method in the aspects of precision, efficiency and interpretability are solved. Characteristic importance quantitative analysis is realized through a variable selection network, and a credible decision basis is provided for power grid dispatching. The method is suitable for wind power plant short-term power prediction, and the renewable energy consumption capability can be remarkably improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of new energy power generation and artificial intelligence, and particularly relates to a wind farm short-term power prediction method based on an improved time sequence fusion Transformer (ITFT) architecture and an enhanced EHO algorithm, which is suitable for power system dispatching, wind farm operation and maintenance, and energy market transactions. BACKGROUND

[0003] The Transformer neural network is a model based on a self-attention mechanism. Compared with traditional recurrent neural networks and convolutional neural networks, the Transformer model has better parallel performance and shorter training time. The TemporalFusion Transformer (TFT) is an interpretable time series model based on the Transformer architecture, which has inherent interpretability compared with other black box models. The TFT quantifies feature contribution through a multi-head attention mechanism, effectively addressing this key limitation. The model can also distinguish between static and dynamic variables, further improving generalization ability. The application proposes a wind power prediction method based on the ITFT network, which introduces a wind speed prediction module (WFN) and an adaptive optimization algorithm to significantly improve computational efficiency while ensuring prediction accuracy, providing more reliable decision support for power grid dispatching. SUMMARY

[0004] The application aims to provide a wind power prediction method based on the ITFT combined with the EHO algorithm, which significantly improves the accuracy and efficiency of wind power prediction. The method fully considers the time characteristics and nonlinear features of wind power data, and combines the Mamba module and the wind speed prediction network to construct a high-precision wind power prediction model.

[0005] The wind power prediction method based on the improved time sequence fusion Transformer proposed by the application consists of the following four steps:

[0006] S1: Describe and define the wind power prediction task;

[0007] S2: Construct the ITFT model architecture for wind power prediction, replace the traditional LSTM encoder-decoder structure with the Mamba module; design the WFN module to generate future wind speed prediction values as auxiliary input for the decoder;

[0008] S3: Apply the EHO algorithm with fusion of chaotic initialization, fitness-distance balance (FDB) strategy, and hybrid mutation mechanism for model hyperparameter optimization;

[0009] S4: Verify the model performance based on actual wind turbine operation data and perform interpretability analysis.

[0010] S1: Traditional wind power prediction methods are usually based on wind turbine operation records and historical meteorological parameters such as wind speed, wind direction and temperature to construct a prediction model, but such methods often ignore key time series characteristics such as hourly cycle fluctuations, day-night change rules, monthly trend characteristics and seasonal time references. Specifically, the current weather conditions may be related to the weather patterns that produce similar renewable energy generation in the same period in history, and seasonal time references can significantly affect the wind power output characteristics through environmental parameter changes. More importantly, traditional methods usually cannot obtain future wind speed data, which is the most critical factor affecting wind power output. Therefore, the present application designs a lightweight module specifically for wind speed prediction, which can improve the prediction accuracy by using it as a known input variable in the future. In general, this prediction framework integrates three types of data sources: historical wind power data P t-n:t , known historical variables X t-n:t , and known variables R t-n:t+H that contain both history and future, to achieve accurate wind power prediction. From a mathematical point of view, this prediction task can be formally defined as: by learning a function mapping relationship F, using the historical observations of the previous n time steps, to predict the wind power output in the future H time intervals.

[0011] Based on the analysis of S1, the present application proposes a short-term wind power prediction model based on the ITFT architecture. First, the Mamba module is used to replace the LSTM encoder-decoder structure in the traditional TFT model to improve the modeling ability of the complex time series dependence of wind power data; second, the WFN module is designed to generate future wind speed as the decoder input, and a multi-task learning framework is constructed to further improve the prediction reliability. This model can effectively capture the spatio-temporal correlation characteristics between multiple variables, and the WFN module integrates historical data and future wind speed prediction to form a more complete decision support system. The model uses a multi-head attention mechanism and a variable selection network (VSN) to build an interpretable architecture that can quantitatively reveal the dynamic importance of input features, providing key insights for grid scheduling. Step S2 network ITFT design is specified as:

[0012] S2.1: Gene regulatory network

[0013] The inherent complex correlation between features and targets in wind power prediction requires the use of adaptive nonlinear processing methods. The gated residual network (GRN) addresses this challenge through its residual architecture, which can dynamically adjust the nonlinear transformation process based on input features. For the original input a and the context vector c, the operation process of GRN is formally defined as: GRN ω (a)=LayerNorm(a+GLUω (η1)) (1) η1=W 1,ω η2+b 1,ω (2) η2=ELU(W 2,ω a+W 3,ω c+b 2,ω (3) Where LayerNorm represents the standard layer normalization operation; w i and b i These correspond to the weights and bias parameters of each layer, respectively. ω represents the index of the weight sharing, where η1 and η2 ∈ R. d_model This represents the intermediate layer output. Based on the Gated Linear Unit (GLU) structure, TFT achieves dynamic control of the information flow by introducing dedicated gating components. This architectural innovation enables the model to selectively disable unnecessary components for specific datasets. The mathematical expression of the GLU mechanism is as follows: GLU ω (γ)=σ(W 4,ω γ+b 4,ω )⊙(W 5,ω γ+b 5,ω (4) Where W(·)∈R d_model×d_model and b(·)∈R d_model These represent the learnable weight matrix and bias vector, respectively. For the input features, σ(·) represents the Sigmoid activation function; ⊙ represents the Hadamard product (element-wise product); and d_model represents the dimension of the hidden state.

[0014] S2.2: Variable selection network.

[0015] To effectively utilize core variables in the input that play a crucial role in target prediction, this study employs a VSN network for variable selection. This network combines an attention mechanism with a gated residual network (GRN) to achieve dynamic optimization of input variables while ensuring model interpretability. This module effectively filters out unimportant features that may affect prediction performance, thereby significantly reducing the redundancy and noise interference of the input data. The mathematical expression of the variable selection network is as follows: Where, ξ t (j) Let represent the transformation of the j-th variable at time t. This is the flattened historical input sequence. Variable selection weights V xt The calculation process is as follows: Input [I] t With the external context vector cs After processing by a gated residual network (GRN), the weights of each variable are obtained by softmax normalization, where v (j) xt That is, the selection weight corresponding to the j-th variable.

[0016] S2.3: Mamba encoder / decoder.

[0017] The Mamba model is a typical state-space model (SSM) that innovatively integrates traditional SSM design principles with the multilayer perceptron module of the Transformer, constructing a selective state-space framework that combines simplicity and high performance. This model performs excellently on Transformer architectures of various sizes, especially exhibiting linear sequence length scaling characteristics in large-scale experimental data.

[0018] Sequence Modeling (SSM) represents an emerging paradigm in sequence modeling. Compared to the traditional Transformer architecture, this model exhibits significant computational efficiency advantages, especially in long sequence processing tasks. Mathematically, SSM describes the recursive process through a state representation controlled by first-order differential equations. Its core mechanism involves defining the input sequence x(t) ∈ R... D via the latent state h(t)∈R N Mapped to the output sequence y(t)∈R N Specifically, it is expressed by the following governing equations: h'(t)=Ah(t)+Bx(t) (8) y(t)=Ch(t) (9) Where A∈R NxN and B, C∈R NxD The model consists of a learnable parameter matrix. Through step discretization, the continuous state-space model is transformed into a discrete form: y t =Ch t (11) in From continuous form (A, B, C) to discrete form The transformation can be efficiently computed using a linear recursive method. Based on the SSM framework, the Structured State-Space Sequence (S4) model introduces a higher-order polynomial projection operator (HiPPO) for initialization. This method applies a specific structure to the state matrix A, significantly improving the model's ability to capture long-range dependencies in sequence data.

[0019] S2.4: Interpretable multi-head attention mechanism.

[0020] The TFT model improves the interpretability of time series prediction by learning long-range dependencies between different time steps through an enhanced self-attention mechanism. (Using query vectors...) With key vector Based on the correlation, the attention mechanism applies to value vectors Perform the following scaling calculation:

[0021] To enhance representation learning capabilities, the multi-head attention mechanism performs attention computation in parallel across different subspaces: MultiHead(Q,K,V)=[H1,...,H mH W H (13) In the formula and These represent the learnable projection weights of the query, key, and value in the current attention head, respectively. The outputs of all attention heads are integrated through linear transformation.

[0022] The TFT model improves upon traditional multi-head attention mechanisms through an innovative shared value vector architecture. This optimization paradigm uses the same value vector representation across all attention heads and fuses the weight distributions of different attention heads through additive aggregation, thereby optimizing computational efficiency without compromising temporal pattern recognition capabilities. InterpretableMultiHead(Q,K,V)=HW H (15) In the formula This represents the value vector weights shared by all attention heads. Used for the final linear mapping.

[0023] S2.5: Wind speed prediction module.

[0024] In time-series forecasting tasks, obtaining future information on key variables can significantly improve model performance. Therefore, this study employs an auxiliary prediction module to perform forward-looking calculations of wind speed required for wind power forecasting. This module uses historical wind speed data to predict future wind speeds and integrates it as an auxiliary input into the decoder architecture.

[0025] Future wind speed predictions are generated by a WFN network, which consists of two Mamba layers and a feedforward network. As a lightweight module, the WFN network effectively enhances the model's predictive capabilities without significantly increasing the number of parameters in the original model.

[0026] S2.6: Multi-task learning.

[0027] Wind speed has a decisive impact on wind power prediction, and obtaining future wind speed information can significantly improve model prediction performance. However, due to the chaotic nature of weather systems, it is impossible to know the precise future wind speed in advance in practical applications. Therefore, this paper designs a wind speed prediction module to estimate future wind speed values ​​and uses them as input to the ITFT model. To achieve high-precision wind power prediction, future wind speed prediction is introduced as an auxiliary task into the model framework: wind power prediction, as the primary task, is achieved by minimizing the quantile loss. In the formula, q represents the set of output quantiles. In this study, q is set to [0.1, 0.25, 0.5, 0.75, 0.9].

[0028] The auxiliary task loss uses the classic regression loss function L. aux λ is the weight that balances the primary and auxiliary tasks, and L is the multi-task joint loss of the ITFT model. IFTT As shown below: L IFTT =L pri +λL aux (20) L aux =||y t -y t || 2 (twenty one)

[0029] Hyperparameter selection is a crucial prerequisite during neural network training. To avoid suboptimal solutions caused by manual parameter tuning and improve selection efficiency, this study employs an improved EHO algorithm for hyperparameter tuning. The hyperparameters to be optimized include: auxiliary task weights, learning rate, hidden layer dimension, dropout rate, and the dimension of consecutive hidden layers. The objective function is set as the loss value of the validation set during training. The specific design steps of the EHO algorithm in S3 are as follows:

[0030] S3.1: Initialization of the EHO algorithm.

[0031] The EHO algorithm conceptualizes each hippo in the population as a candidate solution to the optimization problem. The position vector of each individual represents the potential value of the decision variable in the search space. The chaotic initial position is represented as follows: z i+1 =μz i (1-z i ) (twenty two) X i :x i,j =lb j +z i ·(ubj -lb j ),i=1,2,...,N,j=1,2,...,m (23) In the formula: X i The position of the i-th candidate solution is represented by zi, the chaotic variable is represented by , and the control parameter with a value range of [0,4] is represented by lb. j and ub j Let represent the lower and upper bounds of the j-th variable, respectively. N represents the size of the hippopotamus population, and m represents the dimension of the decision variable.

[0032] S3.2: FDB strategy.

[0033] The core mechanism of the FDB strategy lies in balancing the fitness and spatial distribution of individuals to enhance search efficiency and global optimization capability. The fitness value of the algorithm is 1 / loss of the model validation set. In the initial stage of the FDB strategy, the distance Di between each individual Xi and the best individual Xbest is calculated using Euclidean distance:

[0034] Normalized vector F i and Dp i The FDB score, substituted into the original fitness score, is determined by the following formula:

[0035] S3.3: Update the location of hippos in rivers or ponds (exploration phase).

[0036] The following equation describes the mathematical representation of the location of male hippopotamus members in a lake or pond within a group: in, D represents the position vector of the male hippopotamus. hippo The position representing the dominant hippo (the hippo individual with the best cost in the current iteration). Let be a vector randomly selected from the set {0,1}. I1 and I2 are integers between 1 and 2. Integrated average value MG i Derived from the uniform selection probability (x i The random subgroup members are obtained by combining the specified uniformly distributed random coefficients y1∈[0,1]. In the formula, This represents the location of female or immature hippos in the population. h1 and h2 are randomly selected values ​​or vectors from the five defined scenarios, and Q1 and Q2 are binary random variables taking values ​​of 0 or 1. r7 represents a random variable that follows a uniform distribution in [0,1]. Fi This represents the objective function value.

[0037] S3.4: Update the location of hippos in rivers or ponds (exploration phase).

[0038] The following equation represents the predator's position in the search space: in, Let [the vector be] a random vector in the interval [0,1]. Equation This represents the distance between the i-th hippopotamus and the predator. This represents the hippo's location when it encounters a predator. RS constitutes a random vector that follows either Lévy flight or Cauchy mutation.

[0039] S3.5: Hippos fleeing predators (development phase).

[0040] In this stage, the hippopotamus avoids predation by disengaging from the predator and fleeing to the nearest body of water (lake or pond). Mathematical modeling of this behavioral pattern constitutes the third stage of the algorithm, effectively enhancing its local search capabilities. To simulate this escape mechanism, the algorithm generates randomized locations in the vicinity of the hippopotamus's current location: in This represents the hippo's current position. S1 is a vector or scalar randomly selected from three predefined scenarios. Let r be a random vector in the interval [0,1]. 13 r is a random number ranging from 0 to 1. 12 The numbers are random numbers that follow a normal distribution. The formula for updating the hippopotamus's position is as follows:

[0041] Based on the wind power prediction network model established in steps S2 and S3, step S4 verifies the model performance using actual wind farm operation data and performs interpretability analysis, specifically including the following steps:

[0042] S4.1: Data preparation and preprocessing.

[0043] Collect at least one year of SCADA historical operation data from the target wind farm, including meteorological parameters such as wind speed, wind direction, temperature, and humidity, as well as actual power output; perform Min-Max normalization on the variables; and divide the training set, validation set, and test set into a 7:2:1 ratio.

[0044] S4.2: Model Training and Validation.

[0045] The ITFT-WFN model architecture constructed based on steps S1-S3 was used for training; a multi-task learning strategy was adopted, with the main task being power prediction and the auxiliary task being wind speed prediction; an early stop mechanism (patience=10) was set to prevent overfitting.

[0046] S4.3: Performance Evaluation.

[0047] Using NMAE, NRMSE, MAPE, ACC, R 2 Five indicators are used to assess the accuracy of predictions;

[0048] Compare the prediction performance with baseline models (including LSTM, GRU, and traditional Transformer); analyze the prediction performance in different seasons (spring, summer, autumn, and winter) and plot the results.

[0049] S4.4: Interpretability analysis.

[0050] The importance of each input feature is quantified using the VSN module; the attention weight distribution is analyzed to reveal the temporal patterns that the model focuses on; and the contribution of different lag time steps is visualized. Beneficial effects of this invention:

[0051] In response to the development needs of my country's new energy power system, this invention proposes a wind power prediction method based on an improved time-series fusion Transformer from the perspective of prediction algorithm innovation. This method provides technical support for grid dispatching decisions and wind farm optimization, thereby improving the safe and stable operation capability of high-proportion renewable energy power systems.

[0052] This invention combines the Mamba module and wind speed prediction network to predict wind power, which fully considers the spatiotemporal characteristics and nonlinear features of wind farm operation. It combines efficient time series modeling capabilities with a multi-task learning framework, enabling the established prediction model to achieve industry-leading accuracy.

[0053] This invention innovatively constructs the wind power prediction problem as a multivariate time-series prediction task that integrates historical data and future weather forecasts. It accurately captures the impact of key features through variable selection networks and attention mechanisms. Furthermore, the proposed improved Hippo optimization algorithm significantly enhances the efficiency of model parameter optimization, providing a reliable technical guarantee for real-time wind farm power prediction. Attached Figure Description

[0054] Figure 1 This is a block diagram of the ITFT-WFN network structure involved in this invention;

[0055] Figure 1The network can be mainly divided into a gene regulation module, a variable selection module, a Mamba encoder / decoder, an interpretable multi-head attention module, and a wind speed prediction module. The model input can be categorized into three main types: historical time-series information, future prior information, and prediction target information.

[0056] Figure 2 This is a flowchart illustrating the overall model training process involved in this invention.

[0057] Figure 2 The training of the model mainly includes two steps: data preprocessing and EHO optimization. Detailed Implementation

[0058] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0059] Please refer to the accompanying drawings in the instruction manual. Figure 1 , Figure 1 The diagram shows the ITFT-WFN network structure involved in this invention. This architecture integrates variables from historical time steps through a configurable lookback window to establish temporal dependencies. To achieve sufficient analytical depth while ensuring computational efficiency, the model uses historical data from 24 time-series points to predict wind power at the next time-series point.

[0060] Please refer to the accompanying drawings in the instruction manual. Figure 2 , Figure 2 This is a flowchart of the overall model training process involved in this invention. In the data preprocessing stage, after feature engineering, this invention selects the following variables as inputs to the encoder for past time features: historical wind power data, wind speed and direction at 100 meters height, temperature, relative humidity and dew point, and hourly / dayly / monthly time markers. The following variables are used as inputs to the decoder for future time moments: predicted wind speed and direction at 100 meters height, temperature, relative humidity and dew point, and hourly / dayly / monthly time markers. In the EHO optimization stage, the FDB score of the model test set is used as the fitness value of the optimization algorithm. The specific process is as follows: initialize EHO parameters, including population size and number of iterations; use chaotic initialization to determine the initial position of the population; calculate the individual fitness based on the FDB strategy and record the optimal individual; execute the three stages of the EHO algorithm sequentially; determine the termination condition to see if optimization is complete. This embodiment is based on a real-world operating wind turbine unit, and the method steps include:

[0061] S1: Describe and define the wind power forecasting task;

[0062] S2: Construct an ITFT model architecture for wind power prediction, using a Mamba module to replace the traditional LSTM encoder-decoder structure; design a WFN module to generate future wind speed prediction values ​​as auxiliary inputs to the decoder;

[0063] S3: The EHO algorithm, which integrates chaotic initialization, fitness-distance balance (FDB) strategy and hybrid mutation mechanism, is used to optimize the model hyperparameters;

[0064] S4: Verify the model performance based on actual wind turbine operating data and conduct interpretability analysis.

[0065] S1: Traditional wind power forecasting methods typically build forecasting models based on wind turbine operating records and historical meteorological parameters such as wind speed, wind direction, and temperature. However, these methods often overlook key temporal characteristics such as hourly cycle fluctuations, diurnal variation patterns, monthly trend characteristics, and seasonal time references. Specifically, current meteorological conditions may be correlated with weather patterns that generated similar renewable energy power generation in the same historical period, and seasonal time references can significantly affect wind power output characteristics through changes in environmental parameters. More importantly, traditional methods often fail to obtain future wind speed data, the most critical determinant affecting wind power output, during the forecasting process. To address this, this invention designs a lightweight module specifically for wind speed forecasting, using it as a known future input variable to improve forecast accuracy through multi-source data collaboration. Overall, this forecasting framework integrates three types of data sources: historical wind power data P... t-n:t Given the historical variable X t-n:t And the known variable R that includes both the past and the future. t-n:t+H This aims to achieve accurate wind power forecasting. From a mathematical perspective, this forecasting task can be formally defined as: by learning a functional mapping relationship F, using historical observations from the previous n time steps, predicting wind power output for the next H time intervals.

[0066] Based on the analysis in S1, this invention proposes a short-term wind power prediction model based on an ITFT architecture. First, a Mamba module is used to replace the LSTM encoder-decoder structure in the traditional TFT model to improve the modeling ability for complex temporal dependencies in wind power data. Second, a WFN module is designed to generate future wind speeds as decoder input, constructing a multi-task learning framework to further improve prediction reliability. This model effectively captures the spatiotemporal correlation characteristics between multiple variables, while the WFN module integrates historical data with future wind speed predictions to form a more complete decision support system. The interpretable architecture constructed through a multi-head attention mechanism and a variable selection network (VSN) can quantitatively reveal the dynamic importance changes of input features, providing key operational insights for grid dispatch. Step S2, the network ITFT design steps, are specified as follows:

[0067] S2.1: Gene Regulatory Network

[0068] The inherent complex correlation between features and targets in wind power forecasting necessitates the use of adaptive nonlinear processing methods. Gated Residual Networks (GRNs) address this challenge through their residual architecture, which dynamically adjusts the nonlinear transformation process based on input features. For the original input *a* and the context vector *c*, the GRN computation process is formally defined as follows: GRN ω (a) = LayerNorm(a + GLU) ω (η1)) (46) η1=W 1,ω η2+b 1,ω (47) η2=ELU(W 2,ω a+W 3,ω c+b 2,ω (48) Where LayerNorm represents the standard layer normalization operation; w i and b i These correspond to the weights and bias parameters of each layer, respectively. ω represents the index of the weight sharing, where η1 and η2 ∈ R. d_model This represents the intermediate layer output. Based on the Gated Linear Unit (GLU) structure, TFT achieves dynamic control of the information flow by introducing dedicated gating components. This architectural innovation enables the model to selectively disable unnecessary components for specific datasets. The mathematical expression of the GLU mechanism is as follows: GLU ω (γ)=σ(W 4,ω γ+b 4,ω )⊙(W 5,ω γ+b 5,ω (49) Where W(·)∈R d_model×d_model and b(·)∈R d_model These represent the learnable weight matrix and bias vector, respectively. For the input features, σ(·) represents the Sigmoid activation function; ⊙ represents the Hadamard product (element-wise product); and d_model represents the dimension of the hidden state.

[0069] S2.2: Variable selection network.

[0070] To effectively utilize core variables in the input that play a crucial role in target prediction, this study employs a VSN network for variable selection. This network combines an attention mechanism with a gated residual network (GRN) to achieve dynamic optimization of input variables while ensuring model interpretability. This module effectively filters out unimportant features that may affect prediction performance, thereby significantly reducing the redundancy and noise interference of the input data. The mathematical expression of the variable selection network is as follows: v xt=Softmax(GRN) vx ([Ι] t ,c s (50) ξ t (j) =GRN ξt (ξ t (j) (51) Where, ξ t (j) Let represent the transformation of the j-th variable at time t. This is the flattened historical input sequence. Variable selection weights V xt The calculation process is as follows: Input [I] t With the external context vector c s After processing by a gated residual network (GRN), the weights of each variable are obtained by softmax normalization, where v (j) xt That is, the selection weight corresponding to the j-th variable.

[0071] S2.3: Mamba encoder / decoder.

[0072] The Mamba model is a typical state-space model (SSM) that innovatively integrates traditional SSM design principles with the multilayer perceptron module of the Transformer, constructing a selective state-space framework that combines simplicity and high performance. This model performs excellently on Transformer architectures of various sizes, especially exhibiting linear sequence length scaling characteristics in large-scale experimental data.

[0073] Sequence Modeling (SSM) represents an emerging paradigm in sequence modeling. Compared to the traditional Transformer architecture, this model exhibits significant computational efficiency advantages, especially in long sequence processing tasks. Mathematically, SSM describes the recursive process through a state representation controlled by first-order differential equations. Its core mechanism involves defining the input sequence x(t) ∈ R... D via the latent state h(t)∈R N Mapped to the output sequence y(t)∈R N Specifically, it is expressed by the following governing equations: h'(t)=Ah(t)+Bx(t) (53) y(t)=Ch(t) (54) Where A∈R NxN and B, C∈R NxDThe model consists of a learnable parameter matrix. Through step discretization, the continuous state-space model is transformed into a discrete form: y t =Ch t (56) in From continuous form (A, B, C) to discrete form The transformation can be efficiently computed using a linear recursive method. Based on the SSM framework, the Structured State-Space Sequence (S4) model introduces a higher-order polynomial projection operator (HiPPO) for initialization. This method applies a specific structure to the state matrix A, significantly improving the model's ability to capture long-range dependencies in sequence data.

[0074] S2.4: Interpretable multi-head attention mechanism.

[0075] The TFT model improves the interpretability of time series prediction by learning long-range dependencies between different time steps through an enhanced self-attention mechanism. (Using query vectors...) With key vector Based on the correlation, the attention mechanism applies to value vectors Perform the following scaling calculation:

[0076] To enhance representation learning capabilities, the multi-head attention mechanism performs attention computation in parallel across different subspaces: MultiHead(Q,K,V)=[H1,...,H mH W H (58) In the formula and These represent the learnable projection weights of the query, key, and value in the current attention head, respectively. The outputs of all attention heads are integrated through linear transformation.

[0077] The TFT model improves upon traditional multi-head attention mechanisms through an innovative shared value vector architecture. This optimization paradigm uses the same value vector representation across all attention heads and fuses the weight distributions of different attention heads through additive aggregation, thereby optimizing computational efficiency without compromising temporal pattern recognition capabilities. InterpretableMultiHead(Q,K,V)=HW H (60) In the formula This represents the value vector weights shared by all attention heads. Used for the final linear mapping.

[0078] S2.5: Wind speed prediction module.

[0079] In time-series forecasting tasks, obtaining future information on key variables can significantly improve model performance. Therefore, this study employs an auxiliary prediction module to perform forward-looking calculations of wind speed required for wind power forecasting. This module uses historical wind speed data to predict future wind speeds and integrates it as an auxiliary input into the decoder architecture.

[0080] Future wind speed predictions are generated by a WFN network, which consists of two Mamba layers and a feedforward network. As a lightweight module, the WFN network effectively enhances the model's predictive capabilities without significantly increasing the number of parameters in the original model.

[0081] S2.6: Multi-task learning.

[0082] Wind speed has a decisive impact on wind power prediction, and obtaining future wind speed information can significantly improve model prediction performance. However, due to the chaotic nature of weather systems, it is impossible to know the precise future wind speed in advance in practical applications. Therefore, this paper designs a wind speed prediction module to estimate future wind speed values ​​and uses them as input to the ITFT model. To achieve high-precision wind power prediction, future wind speed prediction is introduced as an auxiliary task into the model framework: wind power prediction, as the primary task, is achieved by minimizing the quantile loss. In the formula, q represents the set of output quantiles. In this study, q is set to [0.1, 0.25, 0.5, 0.75, 0.9].

[0083] The auxiliary task loss uses the classic regression loss function L. aux λ is the weight that balances the primary and auxiliary tasks, and L is the multi-task joint loss of the ITFT model. IFTT As shown below: L IFTT =L pri +λL aux (65) L aux =||y t -y t || 2 (66)

[0084] Hyperparameter selection is a crucial prerequisite during neural network training. To avoid suboptimal solutions caused by manual parameter tuning and improve selection efficiency, this study employs an improved EHO algorithm for hyperparameter tuning. The hyperparameters to be optimized include: auxiliary task weights, learning rate, hidden layer dimension, dropout rate, and the dimension of consecutive hidden layers. The objective function is set as the loss value of the validation set during training. The specific design steps of the EHO algorithm in S3 are as follows:

[0085] S3.1: Initialization of the EHO algorithm.

[0086] The EHO algorithm conceptualizes each hippo in the population as a candidate solution to the optimization problem. The position vector of each individual represents the potential value of the decision variable in the search space. The chaotic initial position is represented as follows: z i+1 =μz i (1-z i (67) X i :x i,j =lb j +z i ·(ub j -lb j ),i=1,2,...,N,j=1,2,...,m (68) In the formula: X i The position of the i-th candidate solution is represented by zi, the chaotic variable is represented by , and the control parameter with a value range of [0,4] is represented by lb. j and ub j Let represent the lower and upper bounds of the j-th variable, respectively. N represents the size of the hippopotamus population, and m represents the dimension of the decision variable.

[0087] S3.2: FDB strategy.

[0088] The core mechanism of the FDB strategy lies in balancing the fitness and spatial distribution of individuals to enhance search efficiency and global optimization capability. The fitness value of the algorithm is 1 / loss of the model validation set. In the initial stage of the FDB strategy, the distance Di between each individual Xi and the best individual Xbest is calculated using Euclidean distance:

[0089] Normalized vector F i and Dp i The FDB score, substituted into the original fitness score, is determined by the following formula:

[0090] S3.3: Update the location of hippos in rivers or ponds (exploration phase).

[0091] The following equation describes the mathematical representation of the location of male hippopotamus members in a lake or pond within a group: in, D represents the position vector of the male hippopotamus. hippo The position representing the dominant hippo (the hippo individual with the best cost in the current iteration). Let be a vector randomly selected from the set {0,1}. I1 and I2 are integers between 1 and 2. Integrated average value MG i Derived from the uniform selection probability (x i The random subgroup members are obtained by combining the specified uniformly distributed random coefficients y1∈[0,1]. In the formula, This represents the location of female or immature hippos in the population. h1 and h2 are randomly selected values ​​or vectors from the five defined scenarios, and Q1 and Q2 are binary random variables taking values ​​of 0 or 1. r7 represents a random variable that follows a uniform distribution in [0,1]. F i This represents the objective function value.

[0092] S3.4: Update the location of hippos in rivers or ponds (exploration phase).

[0093] The following equation represents the predator's position in the search space: in, Let [the vector be] a random vector in the interval [0,1]. Equation This represents the distance between the i-th hippopotamus and the predator. This represents the hippo's location when it encounters a predator. RS constitutes a random vector that follows either Lévy flight or Cauchy mutation.

[0094] S3.5: Hippos fleeing predators (development phase).

[0095] In this stage, the hippopotamus avoids predation by disengaging from the predator and fleeing to the nearest body of water (lake or pond). Mathematical modeling of this behavioral pattern constitutes the third stage of the algorithm, effectively enhancing its local search capabilities. To simulate this escape mechanism, the algorithm generates randomized locations in the vicinity of the hippopotamus's current location: in This represents the hippo's current position. S1 is a vector or scalar randomly selected from three predefined scenarios. Let r be a random vector in the interval [0,1]. 13 r is a random number ranging from 0 to 1. 12 The numbers are random numbers that follow a normal distribution. The formula for updating the hippopotamus's position is as follows:

[0096] Based on the wind power prediction network model established in steps S2 and S3, step S4 verifies the model performance using actual wind farm operation data and performs interpretability analysis, specifically including the following steps:

[0097] S4.1: Data preparation and preprocessing.

[0098] Collect at least one year of SCADA historical operation data from the target wind farm, including meteorological parameters such as wind speed, wind direction, temperature, and humidity, as well as actual power output; perform Min-Max normalization on the variables; and divide the training set, validation set, and test set into a 7:2:1 ratio.

[0099] S4.2: Model Training and Validation.

[0100] The ITFT-WFN model architecture constructed based on steps S1-S3 was used for training; a multi-task learning strategy was adopted, with the main task being power prediction and the auxiliary task being wind speed prediction; an early stop mechanism (patience=10) was set to prevent overfitting.

[0101] S4.3: Performance Evaluation.

[0102] Using NMAE, NRMSE, MAPE, ACC, R 2 Five indicators are used to assess the accuracy of predictions;

[0103] Compare the prediction performance with baseline models (including LSTM, GRU, and traditional Transformer); analyze the prediction performance in different seasons (spring, summer, autumn, and winter) and plot the results.

[0104] S4.4: Interpretability analysis.

[0105] The importance of each input feature is quantified using the VSN module; the attention weight distribution is analyzed to reveal the temporal patterns that the model focuses on; and the contribution of different lag time steps is visualized.

Claims

1. A wind power prediction method based on Temporal Fusion Transformer and EHO optimization algorithm, characterized in that: Includes the following steps: S1: Describe and define the wind power forecasting task; S2: Construct an ITFT model architecture for wind power prediction, using a Mamba module to replace the traditional LSTM encoder-decoder structure; design a WFN module to generate future wind speed prediction values ​​as auxiliary inputs to the decoder; S3: The EHO algorithm, which integrates chaotic initialization, fitness-distance balance (FDB) strategy and hybrid mutation mechanism, is used to optimize the model hyperparameters; S4: Verify the model performance based on actual wind turbine operating data and conduct interpretability analysis.

2. The wind power prediction method combining Temporal Fusion Transformer according to claim 1, characterized in that: The description and definition of the wind power prediction task in step S1 is as follows: S1: Traditional wind power prediction methods typically build prediction models based on wind turbine operation records and historical meteorological parameters such as wind speed, wind direction, and temperature. However, these methods often neglect key temporal characteristics such as hourly cycle fluctuations, diurnal variation patterns, monthly trend characteristics, and seasonal time references. Specifically, current meteorological conditions may be correlated with weather patterns that produce similar renewable energy generation in the same historical period, and seasonal time references can significantly affect wind power output characteristics through changes in environmental parameters. More importantly, traditional methods often cannot obtain future wind speed data, the most critical and decisive factor affecting wind power output, during the prediction process. To address this, this invention designs a lightweight module specifically for wind speed prediction, using it as a known future input variable to improve prediction accuracy through multi-source data collaboration. Overall, this prediction framework integrates three types of data sources: historical wind power data P t-n:t Given the historical variable X t-n:t And the known variable R that includes both the past and the future. t-n:t+H This aims to achieve accurate wind power forecasting. From a mathematical perspective, this forecasting task can be formally defined as: by learning a functional mapping relationship F, using historical observations from the previous n time steps, predicting wind power output for the next H time intervals.

3. The wind power prediction method combining Temporal Fusion Transformer according to claim 1, characterized in that: Step S2, the network ITFT design steps, are specified as follows: S2.1: Gene Regulatory Network The inherent complex correlation between features and targets in wind power forecasting requires the use of adaptive nonlinear processing methods. Gated Residual Networks (GRNs) address this challenge through their residual architecture, which dynamically adjusts the nonlinear transformation process based on input features. For the original input *a* and the context vector *c*, the GRN's computation process is formally defined as follows: GRN ω (a)=LayerNorm(a+GLU ω (η1)) (1) η1=W 1,ω η2+b 1,ω (2) η2=ELU(W 2,ω a+W 3,ω c+b 2,ω ) (3) Where LayerNorm represents the standard layer normalization operation; w i and b i These correspond to the weights and bias parameters of each layer, respectively. ω represents the index of the weight sharing, where η1 and η2 ∈ R. d_model This represents the intermediate layer output. Based on the Gated Linear Unit (GLU) structure, TFT achieves dynamic control of the information flow by introducing dedicated gating components. This architectural innovation enables the model to selectively disable unnecessary components for specific datasets. The mathematical expression of the GLU mechanism is as follows: GLU ω (c)=σ(W 4,ω c+b 4,ω )⊙(W 5,ω c+b 5,ω ) (4) Where W(·)∈R d_model×d_model and b(·)∈R d_model These represent the learnable weight matrix and bias vector, respectively. For the input features, σ(·) represents the Sigmoid activation function; ⊙ represents the Hadamard product (element-wise product); and d_model represents the dimension of the hidden state. S2.2: Variable selection network. To effectively utilize core variables in the input that play a crucial role in target prediction, this study employs a VSN network for variable selection. This network combines an attention mechanism with a gated residual network (GRN) to achieve dynamic optimization of input variables while ensuring model interpretability. This module effectively filters out unimportant features that may affect prediction performance, thereby significantly reducing the redundancy and noise interference of the input data. The mathematical expression of the variable selection network is as follows: Where, ξ t (j) Let represent the transformation of the j-th variable at time t. This is the flattened historical input sequence. Variable selection weights V xt The calculation process is as follows: Input [I] t With the external context vector c s After processing by a gated residual network (GRN), the weights of each variable are obtained by softmax normalization, where v (j) xt That is, the selection weight corresponding to the j-th variable. S2.3: Mamba encoder / decoder. The Mamba model is a typical state-space model (SSM) that innovatively integrates traditional SSM design principles with the multilayer perceptron module of the Transformer, constructing a selective state-space framework that combines simplicity and high performance. This model performs excellently on Transformer architectures of various sizes, especially exhibiting linear sequence length scaling characteristics in large-scale experimental data. Sequence Modeling (SSM) represents an emerging paradigm in sequence modeling. Compared to the traditional Transformer architecture, this model exhibits significant computational efficiency advantages, especially in long sequence processing tasks. Mathematically, SSM describes the recursive process through a state representation controlled by first-order differential equations. Its core mechanism involves defining the input sequence x(t) ∈ R... D via the latent state h(t)∈R N Mapped to the output sequence y(t)∈R N Specifically, it is expressed by the following governing equations: h'(t)=Ah(t)+Bx(t) (8) y(t)=Ch(t) (9) Where A∈R NxN and B, C∈R NxD The model consists of a learnable parameter matrix. Through step discretization, the continuous state-space model is transformed into a discrete form: the t =Ch t (11) in From continuous form (A, B, C) to discrete form The transformation can be efficiently computed using a linear recursive method. Based on the SSM framework, the Structured State-Space Sequence (S4) model introduces a higher-order polynomial projection operator (HiPPO) for initialization. This method applies a specific structure to the state matrix A, significantly improving the model's ability to capture long-range dependencies in sequence data. S2.4: Interpretable multi-head attention mechanism. The TFT model improves the interpretability of time series prediction by learning long-range dependencies between different time steps through an enhanced self-attention mechanism. (Using query vectors...) With key vector Based on the correlation, the attention mechanism applies to value vectors Perform the following scaling calculation: To enhance representation learning capabilities, the multi-head attention mechanism performs attention computation in parallel across different subspaces: MultiHead(Q,K,V)=[H1,...,H mH ]W H (13) In the formula and These represent the learnable projection weights of the query, key, and value in the current attention head, respectively. The outputs of all attention heads are integrated through linear transformation. The TFT model improves upon traditional multi-head attention mechanisms through an innovative shared value vector architecture. This optimization paradigm uses the same value vector representation across all attention heads and fuses the weight distributions of different attention heads through additive aggregation, thereby optimizing computational efficiency without compromising temporal pattern recognition capabilities. InterpretableMultiHead(Q,K,V)=HW H (15) In the formula This represents the value vector weights shared by all attention heads. Used for the final linear mapping. S2.5: Wind speed prediction module. In time-series forecasting tasks, obtaining future information on key variables can significantly improve model performance. Therefore, this study employs an auxiliary prediction module to perform forward-looking calculations of wind speed required for wind power forecasting. This module uses historical wind speed data to predict future wind speeds and integrates it as an auxiliary input into the decoder architecture. Future wind speed predictions are generated by a WFN network, which consists of two Mamba layers and a feedforward network. As a lightweight module, the WFN network effectively enhances the model's predictive capabilities without significantly increasing the number of parameters in the original model. S2.6: Multi-task learning. Wind speed has a decisive impact on wind power prediction, and obtaining future wind speed information can significantly improve model prediction performance. However, due to the chaotic nature of weather systems, it is impossible to know the precise future wind speed in advance in practical applications. Therefore, this paper designs a wind speed prediction module to estimate future wind speed values ​​and uses them as input to the ITFT model. To achieve high-precision wind power prediction, future wind speed prediction is introduced as an auxiliary task into the model framework: wind power prediction, as the primary task, is achieved by minimizing the quantile loss. In the formula, q represents the set of output quantiles. In this study, q is set to [0.1, 0.25, 0.5, 0.75, 0.9]. The auxiliary task loss uses the classic regression loss function L. aux λ is the weight that balances the primary and auxiliary tasks, and L is the multi-task joint loss of the ITFT model. IFTT As shown below: THE IFTT =L pri +λL aux (20) L aux =||y t -and t || 2 (21) 4. The wind power prediction method combining Temporal Fusion Transformer according to claim 1, characterized in that: Step S3 of the model hyperparameter optimization using the EHO algorithm, which integrates chaotic initialization, fitness-distance balance (FDB) strategy, and hybrid mutation mechanism, is as follows: S3.1: Initialization of the EHO algorithm. The EHO algorithm conceptualizes each hippo in the population as a candidate solution to the optimization problem. The position vector of each individual represents the potential value of the decision variable in the search space. The chaotic initial position is represented as follows: z i+1 =μz i (1-z i ) (22) X i :x i,j =lb j +z i ·(ub j -lb j ),i=1,2,...,N,j=1,2,...,m (23) In the formula: X i The position of the i-th candidate solution is represented by zi, the chaotic variable is represented by , and the control parameter with a value range of [0,4] is represented by lb. j and ub j Let represent the lower and upper bounds of the j-th variable, respectively. N represents the size of the hippopotamus population, and m represents the dimension of the decision variable. S3.2: FDB strategy. The core mechanism of the FDB strategy lies in balancing the fitness and spatial distribution of individuals to enhance search efficiency and global optimization capability. The fitness value of the algorithm is 1 / loss of the model validation set. In the initial stage of the FDB strategy, the distance Di between each individual Xi and the best individual Xbest is calculated using Euclidean distance: Normalized vector F i and Dp i The FDB score, substituted into the original fitness score, is determined by the following formula: S3.3: Update the location of hippos in rivers or ponds (exploration phase). The following equation describes the mathematical representation of the location of male hippopotamus members in a lake or pond within a group: in, D represents the position vector of the male hippopotamus. hippo The position representing the dominant hippo (the hippo individual with the best cost in the current iteration). Let be a vector randomly selected from the set {0,1}. I1 and I2 are integers between 1 and 2. Integrated average value MG i Derived from the uniform selection probability (x i The random subgroup members are obtained by combining the specified uniformly distributed random coefficients y1∈[0,1]. In the formula, This represents the location of female or immature hippos in the population. h1 and h2 are randomly selected values ​​or vectors from the five defined scenarios, and Q1 and Q2 are binary random variables taking values ​​of 0 or 1. r7 represents a random variable that follows a uniform distribution in [0,1]. F i This represents the objective function value. S3.4: Update the location of hippos in rivers or ponds (exploration phase). The following equation represents the predator's position in the search space: in, Let [the vector be] a random vector in the interval [0,1]. Equation This represents the distance between the i-th hippopotamus and the predator. This indicates the location of the hippopotamus when it encounters a predator. Construct a random vector that follows Lévy flight or Cauchy mutation S3.5: Hippos fleeing predators (development phase). In this stage, the hippopotamus avoids predation by disengaging from the predator and fleeing to the nearest body of water (lake or pond). Mathematical modeling of this behavioral pattern constitutes the third stage of the algorithm, effectively enhancing its local search capabilities. To simulate this escape mechanism, the algorithm generates randomized locations in the vicinity of the hippopotamus's current location: in This represents the hippo's current position. S1 is a vector or scalar randomly selected from three predefined scenarios. Let r be a random vector in the interval [0,1]. 13 r is a random number ranging from 0 to 1. 12 The numbers are random numbers that follow a normal distribution. The formula for updating the hippopotamus's position is as follows: 。 5. The wind power prediction method combining Temporal Fusion Transformer according to claim 1, characterized in that: Based on the wind power prediction network model established in steps S2 and S3, step S4 verifies the model performance using actual wind farm operation data and performs interpretability analysis, specifically including the following steps: S4.1: Data preparation and preprocessing. Collect at least one year of SCADA historical operation data from the target wind farm, including meteorological parameters such as wind speed, wind direction, temperature, and humidity, as well as actual power output; perform Min-Max normalization on the variables; and divide the training set, validation set, and test set into a 7:2:1 ratio. S4.2: Model Training and Validation. The ITFT-WFN model architecture constructed based on steps S1-S3 was used for training; a multi-task learning strategy was adopted, with the main task being power prediction and the auxiliary task being wind speed prediction; an early stop mechanism (patience=10) was set to prevent overfitting. S4.3: Performance Evaluation. Using NMAE, NRMSE, MAPE, ACC, R 2 Five indicators are used to assess the accuracy of predictions; Compare the prediction performance with baseline models (including LSTM, GRU, and traditional Transformer); analyze the prediction performance in different seasons (spring, summer, autumn, and winter) and plot the results. S4.4: Interpretability analysis. The importance of each input feature is quantified using the VSN module; the attention weight distribution is analyzed to reveal the temporal patterns that the model focuses on; and the contribution of different lag time steps is visualized.

Images