A method for ultra-short-term wind power prediction based on a Transformer-LSTM fusion model

Abstract

This invention belongs to the field of new energy power prediction technology, and proposes an ultra-short-term wind power prediction method based on the Transformer-LSTM fusion model: The method involves acquiring the historical power sequence and multi-source covariate time-series data of the target wind farm, dividing them into training and testing sets; inputting the multi-source covariate time-series data from the training set into the LSTM local feature extraction branch to obtain local feature vectors, and inputting the historical power sequence of the target wind farm from the training set into the Transformer global dependency extraction branch to obtain global context feature vectors; dynamically fusing the local feature vectors and global context feature vectors through a cross-attention mechanism adjustable by learnable temperature parameters, and outputting the power prediction value for the next 0-4 hours through the Transformer-LSTM fusion model.

Description

Technical Field

[0001] This invention relates to the field of new energy power prediction technology, and in particular to a method for ultra-short-term wind power prediction based on a Transformer-LSTM fusion model. Background Technology

[0002] In recent years, wind power, as one of the most promising renewable energy sources, has seen its installed capacity grow rapidly. However, the intermittency and volatility of wind power pose significant challenges to grid dispatch, especially in scenarios with a high proportion of renewable energy connected to the grid. The accuracy of wind power forecasting directly affects grid stability and economic efficiency.

[0003] Currently, wind power prediction methods are mainly divided into two categories: physical methods and data-driven methods. (1) Physical methods are based on numerical weather prediction (NWP) and fluid dynamics models. They predict meteorological parameters such as wind speed and direction by solving atmospheric motion equations, and then calculate power generation by combining the wind turbine power curve. Typical examples include: mesoscale meteorological models (such as WRF and MM5): suitable for short-term (24-72 hours) forecasts, but with high computational complexity and requiring supercomputers; computational fluid dynamics (CFD) models: can simulate the microscale flow field of wind farms, but it is difficult to update data in real time and requires extremely high accuracy in terrain modeling.

[0004] The physical method has the advantage of strong interpretability, but it has the following inherent defects: large computational delay: NWP data update cycle is usually 6 to 12 hours, which cannot meet the needs of ultra-short-term forecasting; poor local adaptability: the wind speed prediction error in complex terrain (such as mountains and sea) can reach 20% to 30%; ignoring unit status: actual operating factors such as wind turbine aging and yaw error are not considered.

[0005] (2) Data-driven methods utilize historical operational data to construct statistical or machine learning models, mainly including: traditional statistical models such as ARIMA (autoregressive integral moving average) and Kalman filtering, which rely on linear assumptions and have limited ability to fit nonlinear wind power characteristics; shallow machine learning models such as support vector machines (SVM) and random forests, which require manual feature design and are difficult to capture long-term time-series dependencies; and deep learning methods such as LSTM, GRU, and Transformer, which can automatically extract time-series features and are gradually becoming mainstream technologies.

[0006] Despite the significant progress made by deep learning in wind power prediction, existing single models still have the following key problems: (1) Limitations of LSTM and its variants: LSTM alleviates the gradient vanishing problem of RNN through gating mechanism and performs well in ultra-short-term prediction, but its shortcomings are: limited local vision: hidden state can only pass information step by step, making it difficult to model long-term dependencies across cycles (such as daily cycle, seasonal cycle); insufficient utilization of covariates: weak ability to capture dynamic interaction relationships of multi-source heterogeneous data such as wind speed and temperature; poor parallelism: the temporal recursive structure leads to slow training speed and makes it difficult to meet real-time requirements.

[0007] (2) Applicability challenges of Transformer: Transformer has shown advantages in long sequence modeling with its self-attention mechanism, but it faces the following challenges in wind power prediction: Sensitivity to local fluctuations: The attention mechanism treats all time steps equally and is easily affected by noise such as sudden changes in wind speed; High computational resource consumption: The complexity of multi-head attention increases with the square of the sequence length, which requires high hardware; Missing temporal inductive bias: It is necessary to rely on position encoding to manually inject sequence information, which may introduce bias.

[0008] (3) Shortcomings of fusion methods: Current fusion strategies (such as feature splicing and weighted averaging) have two major defects: lack of interaction mechanism: no dynamic relationship between the target power sequence and covariates (such as the nonlinear effect of wind speed change on the power curve); information redundancy: simple splicing leads to feature dimension explosion, which reduces the model's generalization ability.

[0009] According to the State Grid's "Technical Specification for New Energy Power Prediction" (Q / GDW 12073-2021), ultra-short-term prediction (0~4 hours) must meet the following requirements: accuracy index: normalized RMSE (nRMSE) <15%; timeliness: prediction period ≤5 minutes, delay <10 seconds; robustness: adaptable to different climate regions (such as sandstorms, typhoons and other extreme weather).

[0010] However, existing technologies do not fully meet the above requirements: the nRMSE of a single LSTM model fluctuates by more than 25% under extreme weather conditions; the Transformer model is difficult to meet real-time requirements due to computational delay; and traditional fusion methods (such as CNN-LSTM) have an R² of only 0.85~0.88 in wind farm tests in Xinjiang Uygur Autonomous Region.

[0011] In summary, there is an urgent need for a new method for wind power prediction that can balance long-term reliance on modeling, dynamic interaction of multi-source data, and real-time calculation. Summary of the Invention

[0012] The purpose of this invention is to provide a method for ultra-short-term wind power prediction based on the Transformer-LSTM fusion model, which aims to solve the problems of insufficient modeling of long-term dependence of target power sequence, insufficient utilization of dynamic features of covariates, and lack of feature interaction mechanism in the existing technology.

[0013] The technical solution adopted by this invention to solve its technical problem is as follows: The method for ultra-short-term wind power prediction based on the Transformer-LSTM fusion model includes the following steps: Acquire the historical power sequence and multi-source covariate time series data of the target wind farm, and divide them into training set and test set. The multi-source covariate time series data includes at least meteorological data. The multi-source covariate time series data in the training set are input into the LSTM local feature extraction branch to obtain the local feature vector, and the target wind farm power history sequence in the training set is input into the Transformer global dependency extraction branch to obtain the global context feature vector. The local feature vector and the global context feature vector are dynamically fused through a cross-attention mechanism that can be adjusted by learnable temperature parameters, and the power prediction value for the next 0-4 hours is output through the Transformer-LSTM fusion model to complete the training of the Transformer-LSTM fusion model. The prediction results of the trained Transformer-LSTM fusion model are evaluated using the test set; The evaluated Transformer-LSTM fusion model was deployed on the edge computing node of the wind farm.

[0014] As a further optimization, the meteorological data includes measured wind speed, wind direction, temperature, air pressure, and humidity; The multi-source covariate time series data also includes remote sensing cloud top brightness temperature, radar combined reflectivity, wind turbine nacelle vibration, blade pitch angle, and rotational speed; Before inputting the multi-source covariate time series data from the training set into the LSTM local feature extraction branch, the process also includes prior screening through a variable selection network.

[0015] As a further optimization, after acquiring the historical power sequence and multi-source covariate time series data of the target wind farm, and before dividing them into training and testing sets, the method further includes: Standardize the historical power sequence and multi-source covariate time series data of the target wind farm; Training samples are generated using a sliding window, where the window length is 24, corresponding to 4 hours of historical data, and the step size is 6, corresponding to 1 hour of prediction. The training set and the test set are divided in a 7:3 ratio.

[0016] As a further optimization, the dynamic fusion of local feature vectors and global context feature vectors through a cross-attention mechanism adjustable by learnable temperature parameters refers to: The local feature vector is represented as H L And as the key-value vector, the global context feature vector is represented as H. T and used as a query vector; The attention weight matrix is ​​calculated so that the Transformer-LSTM fusion model dynamically adjusts the contribution ratio of multi-source covariate time series data and the historical power sequence of the target wind farm at different time steps. The attention weight matrix is ​​represented as follows: A The calculation formula is as follows: ; in For feature dimension, The dot product of the query vector and the key vector. For learnable temperature parameters; The similarity weights between the Key-Value vector and the Query vector are calculated using the attention weight matrix, and the weighted fusion feature F=A•V after dynamic fusion is obtained by calculating the attention weight matrix and the similarity weights, where V is the Value vector; The weighted fusion feature F after dynamic fusion is represented as F = Attention(Q = H). T K=H L V=H L , T ).

[0017] As a further optimization, the learnable temperature parameter T During the training phase, it is jointly optimized with the other parameters of the model to dynamically scale the attention weight matrix, especially in extreme weather scenarios. T Automatically reduce to amplify covariate weights.

[0018] As a further optimization, the LSTM local feature extraction branch adopts a single-layer unidirectional LSTM, with the number of hidden units = covariate dimension × 2~3, and the output shape is (batch, seq_len, d_l). The LSTM local feature extraction branch adopts a direct multi-step prediction mechanism, with the number of output layer nodes equal to the number of prediction steps, and uses the Adam optimizer with an initial learning rate of 0.001 and a batch size of 128.

[0019] As a further optimization, the Transformer global dependency extraction branch adopts a 3-layer sparse LogSparse self-attention encoder with 4 heads, an embedding dimension of 32, a feedforward dimension of 64, and an output shape of (batch, seq_len, d_t). The Transformer global dependency extraction branch uses sinusoidal position encoding to inject timing information.

[0020] As a further optimization, the cross-attention layer containing the learnable temperature parameter-adjustable cross-attention mechanism has an output dimension consistent with the LSTM local feature extraction branch.

[0021] As a further optimization, the Transformer-LSTM fusion model deployed on the edge computing nodes of the wind farm adopts an online incremental learning mechanism based on a sliding window. After the edge computing nodes of the wind farm collect new data for 168 hours, the sliding window incremental training is triggered. The position encoding layer of the Transformer global dependency extraction branch and the hidden state mapping weights of the LSTM local feature extraction branch are kept unchanged. Only the parameters of the cross attention layer and the prediction head are updated. During the update process, the learning rate is 1e-4 and the number of training steps is ≤50.

[0022] As a further optimization, the Transformer-LSTM fusion model deployed on the edge computing nodes of the wind farm supports GPU parallel acceleration, with a single prediction latency of less than 5 seconds.

[0023] The beneficial effects of this invention are as follows: By combining the Transformer-LSTM fusion architecture with the cross-attention mechanism, significant advantages are achieved in several aspects: In terms of prediction accuracy, compared with a single Transformer or LSTM model, this invention reduces MSE by 28.3%~40.8%, stabilizes RMSE below 0.265, and increases the coefficient of determination R² to 0.952. Especially in high-fluctuation scenarios, the error volatility is controlled within 15%, effectively solving the shortcomings of traditional methods in terms of long-term dependence and insufficient modeling of local feature collaboration; In terms of computational efficiency, through GPU parallel acceleration and sliding window streaming processing, the single prediction latency is less than 5 seconds, fully meeting the real-time requirements of power grid dispatch for ultra-short-term prediction; In terms of engineering applicability, This invention can more accurately capture wind power ramp-up / drop characteristics in complex scenarios such as sudden wind speed changes and rapid weather changes, reducing prediction errors and helping dispatch centers to take orderly power curtailment and backup dispatch measures earlier. Furthermore, the model of this invention supports online learning to adapt to changes in the operating status of wind farms and seamlessly integrates with existing SCADA systems through standardized interfaces. In actual deployment in wind farms in Xinjiang Uygur Autonomous Region, it has reduced the wind curtailment rate by 2.3 percentage points, and the prediction error of the fusion model has been reduced, making the dispatch strategy more accurate. In addition, the architecture has both scalability and generalization capabilities. It can be compressed into a lightweight model and deployed on edge devices, and can also be adapted to diverse scenarios such as offshore wind power and photovoltaic power plants by adjusting the input dimensions, providing reliable technical support for high-proportion renewable energy grid connection. Attached Figure Description

[0024] Figure 1 This is a flowchart of the wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the LSTM unit in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the LSTM prediction model in Embodiment 2 of the present invention; Figure 4 This is a schematic diagram of the Transformer deep learning model in Embodiment 2 of the present invention; Figure 5 This is a schematic diagram of the Transformer-LSTM deep learning model in Embodiment 2 of the present invention; Figure 6 This is a diagram showing the specific parameters of the Transformer-LSTM deep learning model in Embodiment 2 of the present invention; Figure 7 This is a schematic diagram showing the memory usage size when the embedding dimension is 16 in Embodiment 2 of the present invention; Figure 8This is a schematic diagram of GPU utilization when the embedding dimension is 16 in Embodiment 2 of the present invention; Figure 9 This is a schematic diagram showing the memory usage size when the embedding dimension is 32 in Embodiment 2 of the present invention; Figure 10 This is a schematic diagram of GPU utilization when the embedding dimension is 32 in Embodiment 2 of the present invention; Figure 11 This is a schematic diagram showing the memory usage size when the embedding dimension is 64 in Embodiment 2 of the present invention; Figure 12 This is a schematic diagram of GPU utilization when the embedding dimension is 64 in Embodiment 2 of the present invention; Figure 13 The image is fitted to the prediction result of the Transformer-LSTM deep learning in Embodiment 2 of the present invention; Figure 14 This is a comparison chart of the prediction results of the Transformer-LSTM deep learning model in Embodiment 2 of the present invention and the evaluation metrics of other models. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0026] Example 1

[0027] This embodiment provides a method for ultra-short-term wind power prediction based on a Transformer-LSTM fusion model. See the flowchart below. Figure 1 The method may include the following steps: S1. Obtain the historical power sequence and multi-source covariate time series data of the target wind farm, and divide them into training set and test set. The multi-source covariate time series data includes at least meteorological data. S2. Input the multi-source covariate time series data in the training set into the LSTM local feature extraction branch to obtain the local feature vector, and input the target wind farm power history sequence in the training set into the Transformer global dependency extraction branch to obtain the global context feature vector. S3. The local feature vector and the global context feature vector are dynamically fused through a cross-attention mechanism that can be adjusted by learnable temperature parameters, and the power prediction value for the next 0-4 hours is output through the Transformer-LSTM fusion model to complete the training of the Transformer-LSTM fusion model. S4. Evaluate the prediction results of the trained Transformer-LSTM fusion model using the test set; S5. Deploy the evaluated Transformer-LSTM fusion model on the edge computing node of the wind farm.

[0028] In this embodiment, a three-layer cascaded heterogeneous deep learning architecture can be used to combine the local temporal modeling capability of LSTM and the global dependency capture advantage of Transformer, and a cross-attention mechanism can be introduced to achieve dynamic feature fusion, which significantly improves prediction accuracy and real-time performance.

[0029] The core architecture in this embodiment may include a covariate dynamic feature extraction layer (LSTM local feature extraction branch), a target sequence global modeling layer (Transformer global dependency extraction branch), and a dynamic interaction fusion layer (cross-attention mechanism). In the covariate dynamic feature extraction layer, an LSTM network is used to process multi-dimensional covariate time-series data such as wind speed, wind direction, temperature, air pressure, and humidity. LSTM adaptively captures local temporal dynamics and nonlinear relationships through its unique gating mechanism (forget gate, input gate, output gate), and outputs hidden features with a dimension 2-3 times that of the input variables. Therefore, it can effectively solve the problem of insufficient fitting ability of traditional statistical models for nonlinear time-series features, while avoiding gradient vanishing or exploding phenomena.

[0030] In practical applications, multi-source covariate time series data should at least include meteorological data, which should include measured wind speed, wind direction, temperature, air pressure, humidity, and grid point meteorological elements output by numerical weather prediction models. In addition, multi-source covariate time series data can also include remote sensing cloud top brightness temperature, radar combined reflectivity, wind turbine nacelle vibration, pitch angle, and speed, among which wind turbine nacelle vibration, pitch angle, and speed are SCADA variables.

[0031] To enhance the robustness of the Transformer-LSTM fusion model and improve its efficiency in utilizing multi-source heterogeneous information, this embodiment includes a prior screening process using a variable selection network (VSN) before inputting the multi-source covariate time-series data from the training set into the LSTM local feature extraction branch. Here, the multi-source covariate time-series data is first screened by the VSN before being fed into the LSTM branch. This process, which automatically evaluates the contribution of each variable to the prediction target through a gating mechanism and dynamically selects key features for input into the LSTM network, effectively reduces redundant information interference and enhances the robustness and interpretability of the Transformer-LSTM fusion model under complex conditions.

[0032] It should be noted that this embodiment also optimizes the model training and inference processes. During data preprocessing, the power sequence and covariates are standardized, and a sliding window mechanism is used to generate training samples (window length 24, stride 6). Model training uses the Adam optimizer with an initial learning rate of 0.0001, a batch size of 128, 150 training epochs, and mean squared error (MSE) as the loss function. Experiments show that this configuration avoids overfitting while ensuring convergence speed. During inference, the model supports GPU parallel acceleration (such as NVIDIA GTX1050), with a single prediction latency of less than 5 seconds, fully meeting the real-time requirements of ultra-short-term prediction.

[0033] Furthermore, the Transformer-LSTM fusion model requires training and testing of the fusion model during the training process. Therefore, after obtaining the historical power sequence and multi-source covariate time series data of the target wind farm, and before dividing them into training and testing sets, the following steps are also required: Standardize the historical power sequence and multi-source covariate time series data of the target wind farm; Training samples are generated using a sliding window, where the window length is 24, corresponding to 4 hours of historical data, and the step size is 6, corresponding to 1 hour of prediction. The training set and the test set are divided in a 7:3 ratio.

[0034] This embodiment was validated using measured data from a wind farm in Xinjiang Uygur Autonomous Region. The dataset contains sampling data for the entire year of 2023 (at 10-minute intervals), divided into training and test sets in a 7:3 ratio. Comparative experiments show that the fusion model in this embodiment significantly outperforms the single Transformer (MSE 0.099) and LSTM (MSE 0.120) models in terms of MSE (0.071), RMSE (0.265), and R² (0.952). In particular, under the scenario of sudden wind speed changes, the prediction error volatility of this embodiment is less than 15%, demonstrating stronger robustness.

[0035] In practical applications, the Transformer encoder is used to process the power history sequence in the global modeling layer of the target sequence. A multi-head self-attention mechanism is used to parse long-term dependencies across time steps. The Transformer's embedding dimension is set to 32, the encoder has 3 layers, each containing 4 attention heads, and the feedforward network has a dimension of 64. Furthermore, sinusoidal position encoding injects temporal information into the input sequence, compensating for the Transformer's inherent lack of temporal awareness. Compared to LSTM, the Transformer can simultaneously focus on the relationship between any two time steps in the sequence, thus more effectively capturing long-term patterns such as daily and seasonal cycles.

[0036] In this embodiment, the dynamic interactive fusion layer achieves deep fusion of the wind farm power sequence (target sequence) and the meteorological covariate sequence through a cross-attention mechanism. Specifically, the historical power sequence is first input into the Transformer branch to extract global contextual features, denoted as H. T Then, covariates such as wind speed, wind direction, temperature, and humidity are input into the LSTM branch to extract local dynamic features, denoted as H. L Subsequently, with H T As a query vector, using H L As the key-value vector, calculate the attention weight matrix: ; in For feature dimension, The dot product of the query vector and the key vector. As a learnable temperature parameter, the model can dynamically adjust the contribution ratio of covariates and power sequences at different time steps through this weight matrix, and output a weighted fusion feature F=A•V. This fusion method not only preserves the uniqueness of each data mode, but also establishes a nonlinear dynamic correlation between the target power and meteorological elements, overcoming the information redundancy problem caused by traditional splicing or weighted averaging methods.

[0037] Thanks to the cross-attention mechanism, this model can more accurately capture wind power ramp-up / drop characteristics in complex scenarios such as sudden wind speed changes and rapid weather changes, reducing prediction bias and thus helping the dispatch center to take orderly power curtailment and reserve dispatch measures earlier. In actual deployment at a wind farm in Xinjiang Uygur Autonomous Region, the prediction error of this fusion model was reduced, making the dispatch strategy more accurate and ultimately reducing the wind curtailment rate by about 2.3 percentage points.

[0038] It should be noted that, in this embodiment, the learnable temperature parameter T During the training phase, it is jointly optimized with the other parameters of the model to dynamically scale the attention weight matrix, especially in extreme weather scenarios. T Automatically reduce to amplify covariate weights.

[0039] Therefore, in this embodiment, the LSTM local feature extraction branch uses a single-layer unidirectional LSTM with the number of hidden units equal to the covariate dimension multiplied by 2 to 3, and an output shape of (batch, seq_len, d_l). The LSTM local feature extraction branch employs a direct multi-step prediction mechanism, with the number of output layer nodes equal to the number of prediction steps, and uses the Adam optimizer with an initial learning rate of 0.001 and a batch size of 128. The Transformer global dependency extraction branch uses a 3-layer sparse LogSparse self-attention encoder with 4 heads, an embedding dimension of 32, a feedforward dimension of 64, and an output shape of (batch, seq_len, d_t). The Transformer global dependency extraction branch uses sinusoidal positional encoding to inject temporal information. The cross-attention layer, which houses the learnable temperature parameter adjustment cross-attention mechanism, has the same output dimension as the LSTM local feature extraction branch, enabling end-to-end training.

[0040] To enhance the adaptability of the Transformer-LSTM fusion model to changes in wind farm operating conditions, this embodiment employs an online incremental learning mechanism based on a sliding window. This mechanism is automatically triggered after every 168 hours of accumulated new data and uses a partial parameter freezing strategy: the mapping weights between the Transformer's position encoding layer and the LSTM's hidden state remain unchanged, while only the cross-attention layer and prediction head are fine-tuned. The training process uses a low learning rate (1e-4) and keeps the number of training steps below 50, ensuring that the model quickly adapts to recent changes in wind power output characteristics without overfitting to new data. The entire update process is completed locally on the edge nodes, supporting hot switching with the old model without interrupting the prediction service, achieving true "learning while running."

[0041] Furthermore, this embodiment also supports an online learning mechanism. By periodically (e.g., weekly) fine-tuning the model parameters with the latest data, it can adapt to dynamic factors such as wind farm equipment aging and seasonal changes. In addition, the model can be compressed into a lightweight version using knowledge distillation technology and deployed on edge devices (such as wind turbine PLCs) to achieve real-time prediction at the edge. These extended functions significantly improve the engineering applicability of this invention. In this embodiment, the Transformer-LSTM fusion model deployed on the edge computing nodes of the wind farm supports GPU parallel acceleration. In a GPU (NVIDIA GTX1050 and above) environment, its single prediction latency is less than 5 seconds.

[0042] Example 2 Based on Example 1, this example provides a method for ultra-short-term wind power prediction based on the Transformer-LSTM fusion model, which further explains the method based on the Transformer-LSTM fusion model.

[0043] This embodiment's Transformer-LSTM hybrid model combines the advantages of both models. Transformer possesses a self-attention mechanism to capture global dependencies, demonstrating strong ability to model long-range temporal relationships. Furthermore, it is loop-free, allowing for parallel computation and fast training and inference. LSTM's gating mechanism effectively preserves or forgets information, excelling at characterizing local temporal dynamics and short-term fluctuations. LSTM is also robust to vanishing / exploding gradients, making it suitable for modeling continuous small-step sequences. In this Transformer-LSTM hybrid model, the LSTM first extracts local temporal features and retains long-term memory, while the Transformer then performs global self-attention association on the LSTM output, considering both local details and global trends. The complementary advantages of both models improve prediction accuracy, reduce errors, and maintain high computational efficiency, making it suitable for time series prediction tasks with multi-scale characteristics, such as wind power forecasting.

[0044] See appendix Figure 2This diagram illustrates a single neuron in an LSTM (Long Short-Term Memory) network. The LSTM memory unit is the fundamental building block of the LSTM network, particularly well-suited for processing and predicting time-series data because it effectively captures long-term dependencies. The LSTM memory unit contains three gates: a forget gate, an input gate, and an output gate, each controlling the flow of information. The forget gate determines which information needs to be discarded from the cell state. It receives the previous hidden state and the current input, and calculates a value between 0 and 1 using a sigmoid function, representing the proportion of each state value to be forgotten. The input gate determines which new information needs to be added to the cell state. It also receives the previous hidden state and the current input, and calculates which values ​​need to be updated using a sigmoid function, while simultaneously generating a new candidate value vector using a tanh function for state updates. Cell state updates are the process of forgetting old information and adding new information; the updated cell state combines the outputs of the forget and input gates. The output gate determines what to output. Based on the current cell state and the previous hidden state, it uses a sigmoid function and a tanh function to determine the final output. This structure enables LSTM to effectively process and predict important events in time series, even those separated by multiple time steps. By controlling the flow of information, LSTM can learn long-term dependencies, which is crucial for many sequence prediction tasks.

[0045] See appendix Figure 3 This demonstrates the structure of a typical LSTM (Long Short-Term Memory) neural network, which consists of an input layer, two hidden layers, and an output layer. In the input layer, the network receives data at a series of time steps. x 1 、x 2 、…、x 24 These data are fed into LSTM units in the hidden layer. Each unit consists of an input gate, a forget gate, and an output gate, which work together to process and remember key information in the sequence data. Each LSTM unit in the hidden layer contains a cell state. and a hidden state This state updates over time and can capture long-term dependencies. Cell state Responsible for storing long-term memories, while hiding the state. The output features of the current time step are then carried and passed to the unit at the next time step. The transfer of information within and between layers is controlled by the input gate, forget gate, and output gate, allowing the network to selectively retain or forget historical information. The output of the hidden layer... The information is passed to the output layer, which then generates the final prediction result based on this information. y1 、y 2 、…y 24 The figure also illustrates the cell state update process within each LSTM unit, showing how information flows and is processed within the network. Furthermore, the network's hidden layer number and dimensionality are scalable, allowing adjustments based on task complexity and data characteristics to further enhance the model's learning ability and prediction accuracy. The overall structure allows the LSTM network to effectively process sequence data, enabling it to excel in tasks such as time series prediction and natural language processing.

[0046] See appendix Figure 4 This paper demonstrates the structure of the Transformer neural network. In this embodiment, the Transformer model is a deep learning architecture based on an attention mechanism, consisting of an encoder and a decoder. Both parts contain multiple stacked layers, each composed of a multi-head attention mechanism and a feedforward network, with residual connections and layer normalization after each sub-layer. The encoder first converts the input sequence into a vector representation through input embedding and positional encoding, and then uses the multi-head attention mechanism to capture the dependencies within the sequence. The decoder adds a masked multi-head attention mechanism on top of the encoder to ensure that future information is not seen when generating the output sequence, while processing the target sequence through output embedding and positional encoding. In the decoder, the multi-head attention mechanism also allows the model to pay attention to the encoder's output to combine information from the input sequence. Finally, the decoder's output generates the predicted probability distribution for each position through linear layers and softmax layers. This structure enables the Transformer to efficiently process sequential data, capture long-range dependencies, and support efficient parallel computation.

[0047] See appendix Figure 5 This document illustrates the structural diagram of the Transformer-LSTM fusion model in this embodiment. This model combines the self-attention mechanism of the Transformer with the temporal processing capabilities of the LSTM. This embodiment primarily focuses on improvements in both model structure and fusion mechanism. First, a parallel dual-branch architecture is adopted in the model structure: the power history sequence is processed by the Transformer branch to extract global contextual features across time steps, while covariates (wind speed, wind direction, temperature, humidity, etc.) are processed by the LSTM branch to extract local dynamic features. The two branches operate independently without interference, effectively preserving their respective modal advantages and avoiding information loss and computational delays caused by a sequential structure. Second, a cross-attention mechanism is introduced during the fusion process, using the Transformer output as the query and the LSTM output as the key-value pair, through a similarity weight matrix: ; By achieving feature-level adaptive weighted aggregation and dynamically adjusting the contribution ratio of power and meteorological elements at different times, a fused feature F=A•V is obtained. This mechanism not only establishes a dynamic nonlinear mapping between power series and meteorological covariates, overcoming the problems of lack of interaction and redundancy in traditional splicing or weighted averaging, but also takes into account the ability of LSTM to characterize short-term upslopes and local fluctuations and the ability of Transformer to capture long-term trends. Therefore, it can solve the pain points of single models in modeling long-term and short-term dependencies and insufficient utilization of multi-source data. While maintaining real-time performance, it improves prediction accuracy and robustness, especially under complex conditions such as sudden changes in wind speed, it can more accurately reflect the trend of power change, providing reliable support for refined grid scheduling and reducing wind curtailment rate.

[0048] In this embodiment, the model processes historical power data through the Transformer global dependency extraction branch, utilizing its self-attention mechanism to capture dependencies at different time steps in the sequence, and enhancing the understanding of long-term dependencies and temporal information through multi-head attention and positional encoding. Simultaneously, the LSTM local feature extraction branch processes covariate data, including wind speed and temperature, capturing short-term dynamic changes and local temporal dependencies of the covariates through its gating mechanism. The cross-attention mechanism in the model dynamically fuses the global context features of the Transformer with the local dynamic features of the LSTM to adaptively adjust the contribution ratio of covariates and the power sequence at different time steps. Finally, the model outputs power prediction values ​​for the next 0-4 hours through a fully connected layer. The entire model uses mean squared error (MSE) as the loss function and optimizes parameters using the Adam optimizer to achieve high-precision prediction of wind power in the ultra-short term, meeting the real-time optimization and control requirements of the power grid dispatching system.

[0049] See appendix Figure 6 The diagram lists the key training parameters and their settings for the Transformer-LSTM fusion model, including a batch size of 128 (meaning 128 samples will be processed per training iteration), a learning rate of 0.0001 (controlling the step size for weight updates), 150 training epochs (meaning the model will be trained 150 times on the entire dataset), and a loss function using mean squared error (MSE) to evaluate the difference between predicted and actual values. Furthermore, the diagram indicates the embedding dimension (32) and number of layers (3) of the Transformer module, and the hidden layer dimension (32) and number of layers (3) of the LSTM module. These parameters collectively determine the model's capacity and learning ability, affecting its performance in extracting and predicting features from sequential data. The number of training iterations was initially chosen based on project experience in deep learning; neural networks typically reach their peak performance around 150 iterations. See also... Figures 7-12Other parameters, such as the embedding dimension and number of layers, were selected based on multiple comparative experiments on a cloud server with an NVIDIA RTX 4090D (24 GB) graphics card: when the embedding dimension increased from 16 to 32, the validation set RMSE decreased by about 7%, and further increasing it to 64 only decreased it by another 1.6%, but the memory usage almost doubled; when the number of layers increased from 2 to 3, the RMSE decreased by 5.2%, while 4 layers increased it by 1.3% due to overfitting. At the same time, the hidden layer dimension and number of layers of LSTM were kept consistent with the embedding dimension and number of layers of Transformer, respectively, and were set to 32 and 3 layers. Taking into account model performance, training efficiency, and hardware memory limits, this combination was finally determined to have good training stability and convergence speed while ensuring prediction accuracy.

[0050] See appendix Figure 13 The figure shows the fitted image after prediction by the Transformer-LSTM fusion model in this embodiment of the invention, illustrating the fit of the prediction results on the test set. The blue line represents the actual data, and the red line represents the model's prediction results. As can be seen from the figure, the model performs excellently in capturing the overall trend of the data; the predicted line closely matches the actual line in most areas, showing a good fit. Furthermore, the correlation coefficient between the model and the actual data is as high as 0.953. This statistical indicator further confirms the high linear correlation between the model's predictions and the actual observations, indicating that the model has high prediction accuracy.

[0051] See appendix Figure 14 This paper demonstrates a comparison of the prediction results of the Transformer-LSTM deep learning model in this embodiment with the evaluation metrics of other models. The Transformer-LSTM fusion model in this embodiment significantly outperforms the single Transformer and LSTM in terms of MSE, RMSE, and R²: its MSE is reduced to 0.071, a further reduction of 28% compared to Transformer and 41% compared to LSTM; RMSE is only 0.265, a reduction of 16% and 23% respectively; and R² is as high as 0.952, an improvement of about 5-7 percentage points, meaning that the model can explain more than 95% of the power variation. This result verifies that by combining the local dynamic features of LSTM and the global dependency features of Transformer through cross-attention parallelism, the fusion structure has the ability to perceive both short-term fluctuations and long-term trends, maintaining lower errors and higher fitting accuracy even in high-noise and highly nonlinear wind power scenarios, providing a more accurate solution for ultra-short-term power prediction.

[0052] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for ultra-short-term wind power prediction based on a Transformer-LSTM fusion model, characterized in that, Includes the following steps: The historical power sequence and multi-source covariate time series data of the target wind farm are obtained and divided into training set and test set. The multi-source covariate time series data includes at least meteorological data, remote sensing cloud top brightness temperature, radar combined reflectivity, wind turbine nacelle vibration, pitch angle and rotational speed. Multi-source covariate time-series data from the training set are input into the LSTM local feature extraction branch to obtain local feature vectors. The target wind farm power history sequence from the training set is input into the Transformer global dependency extraction branch to obtain global context feature vectors. The contribution of each variable to the prediction target is automatically evaluated through a gating mechanism, and key features are dynamically selected and input into the LSTM local feature extraction branch. The Transformer global dependency extraction branch uses a 3-layer sparse LogSparse self-attention encoder with 4 heads, an embedding dimension of 32, a feedforward dimension of 64, and an output shape of (batch, seq_len, d_t). The Transformer global dependency extraction branch uses sinusoidal position coding to inject time-series information. A cross-attention mechanism with learnable temperature parameters is used to dynamically fuse local and global contextual feature vectors. The resulting power predictions for the next 0-4 hours are then output through a Transformer-LSTM fusion model, completing the training of the model. Specifically, the historical power sequence is input into the Transformer branch to extract global contextual features, denoted as H. T Multi-source covariate time-series data is input into the LSTM local feature extraction branch to extract local dynamic features, denoted as H. L , with H T As a query vector, using H L As the Key-Value vector, the attention weight matrix A is calculated. Using the attention weight matrix, the similarity weight between the Key-Value vector and the Query vector is calculated. Finally, the dynamically fused weighted fusion feature F = A⋅V is obtained by using the attention weight matrix and the similarity weight, where V is the Value vector. The prediction results of the trained Transformer-LSTM fusion model are evaluated using the test set; The evaluated Transformer-LSTM fusion model was deployed on the edge computing node of the wind farm.

2. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, The meteorological data includes measured wind speed, wind direction, temperature, air pressure, and humidity; Before inputting the multi-source covariate time series data from the training set into the LSTM local feature extraction branch, the process also includes prior screening through a variable selection network.

3. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, After acquiring the historical power sequence and multi-source covariate time series data of the target wind farm, and before dividing them into training and testing sets, the process also includes: Standardize the historical power sequence and multi-source covariate time series data of the target wind farm; Training samples are generated using a sliding window, where the window length is 24, corresponding to 4 hours of historical data, and the step size is 6, corresponding to 1 hour of prediction. The training set and the test set are divided in a 7:3 ratio.

4. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, The dynamic fusion of local feature vectors and global context feature vectors through a cross-attention mechanism adjustable by learnable temperature parameters refers to: The Transformer-LSTM fusion model dynamically adjusts the contribution ratio of multi-source covariate time series data and the historical power sequence of the target wind farm at different time steps through the attention weight matrix. The attention weight matrix is ​​represented as follows: A The calculation formula is as follows: ； in For feature dimension, The dot product of the query vector and the key vector. For learnable temperature parameters; The weighted fusion feature F after dynamic fusion is represented as F = Attention(Q = H). T K=H L V=H L , T ).

5. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 4, characterized in that, The learnable temperature parameter T During the training phase, it is jointly optimized with the other parameters of the model to dynamically scale the attention weight matrix, especially in extreme weather scenarios. T Automatically reduce to amplify covariate weights.

6. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, The LSTM local feature extraction branch uses a single-layer unidirectional LSTM, with the number of hidden units = covariate dimension × 2~3, and the output shape is (batch, seq_len, d_l). The LSTM local feature extraction branch adopts a direct multi-step prediction mechanism, with the number of output layer nodes equal to the number of prediction steps, and uses the Adam optimizer with an initial learning rate of 0.001 and a batch size of 128.

7. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, The cross-attention layer containing the learnable temperature parameter-adjustable cross-attention mechanism has an output dimension consistent with the LSTM local feature extraction branch.

8. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, The Transformer-LSTM fusion model deployed on the edge computing nodes of the wind farm adopts an online incremental learning mechanism based on a sliding window. After the edge computing nodes of the wind farm collect 168 hours of new data, the sliding window incremental training is triggered. The position encoding layer of the Transformer global dependency extraction branch and the hidden state mapping weights of the LSTM local feature extraction branch are kept unchanged. Only the parameters of the cross attention layer and the prediction head are updated. During the update process, the learning rate is 1e-4 and the number of training steps is ≤50.

9. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to any one of claims 1-8, characterized in that, The Transformer-LSTM fusion model deployed on the edge computing nodes of wind farms supports GPU parallel acceleration, with a single prediction latency of less than 5 seconds.

Images