Wind power ultra-short-term prediction method based on Transform-LSTM fusion model

Abstract

The invention belongs to the technical field of new energy power prediction, and provides a wind power ultra-short-term prediction method based on a Transform-LSTM fusion model, and the method comprises the steps: obtaining a power historical sequence and multi-source covariable time series data of a target wind power plant, and dividing the power historical sequence and the multi-source covariable time series data into a training set and a test set; inputting multi-source covariable time series data in the training set into an LSTM local feature extraction branch to obtain a local feature vector, and inputting a target wind power plant power historical sequence in the training set into a Transform global dependency extraction branch to obtain a global context feature vector; and carrying out dynamic fusion on the local feature vector and the global context feature vector through a cross attention mechanism capable of learning temperature parameter adjustment, and outputting a power prediction value in the next 0-4 hours through a Transform-LSTM fusion model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of new energy power prediction, in particular to a wind power ultra-short-term prediction method based on a Transformer-LSTM fusion model. BACKGROUND

[0002] In recent years, as one of the most potential renewable energy sources, the installed capacity of wind power has been growing rapidly. However, the intermittency and volatility of wind power have brought great challenges to power grid dispatching, especially in the scenario of high proportion of renewable energy grid connection, the accuracy of wind power prediction directly affects the stability and economy of the power grid.

[0003] At present, wind power prediction methods mainly include physical methods and data-driven methods: (1) Physical methods are based on numerical weather prediction (NWP) and fluid mechanics models, which predict meteorological parameters such as wind speed and direction by solving atmospheric motion equations, and then calculate power generation by combining wind turbine power curves. Typical representatives include: mesoscale meteorological models (such as WRF, MM5): suitable for short-term (24~72 hours) prediction, but with high computational complexity, requiring supercomputers; computational fluid dynamics (CFD) model: can simulate microscale flow field of wind farm, but difficult to update data in real time, and requires high accuracy of terrain modeling.

[0004] The advantages of physical methods are strong interpretability, but they have the following inherent defects: large calculation delay: NWP data update period is usually 6~12 hours, which cannot meet the demand of ultra-short-term prediction; poor local adaptability: the prediction error of wind speed in complex terrain (such as mountains, sea) can reach 20%~30%; ignoring unit state: without considering factors such as wind turbine aging and yaw error in actual operation.

[0005] (2) Data-driven methods use historical operation data to build statistical or machine learning models, mainly including: traditional statistical models: such as ARIMA (autoregressive integrated moving average), Kalman filter, etc., which rely on linear assumptions and have limited fitting ability for nonlinear wind power characteristics; shallow machine learning models: such as support vector machine (SVM), random forest, etc., which require manual feature design and are difficult to capture long-term temporal dependencies; deep learning methods: such as LSTM, GRU, Transformer, etc., which can automatically extract temporal features and have gradually become the mainstream technology.

[0006] Although deep learning has made significant progress in wind power prediction, existing single models still have the following key problems: (1) Limitations of LSTM and its variants: LSTM alleviates the gradient vanishing problem of RNN through gating mechanisms, and performs well in ultra-short-term prediction. However, it has the following defects: limited local view: hidden states can only pass information step by step, making it difficult to model long-term dependencies across cycles (such as daily and seasonal cycles); insufficient utilization of covariates: weak ability to capture dynamic interactions of multi-source heterogeneous data such as wind speed and temperature; poor parallelism: the time series recursive structure leads to slow training speed, making it difficult to meet real-time requirements.

[0007] (2) Challenges of applicability of Transformer: Transformer has advantages in long sequence modeling with self-attention mechanism, but faces the following challenges in wind power prediction: sensitivity to local fluctuations: attention mechanism treats all time steps equally, which is easily disturbed by wind speed mutations and other noises; high computational resource consumption: multi-head attention complexity grows with the square of sequence length, requiring high hardware requirements; lack of inductive bias: needs to rely on position encoding to artificially inject sequential information, which may introduce bias.

[0008] (3) Deficiencies of fusion methods: current fusion strategies (such as feature concatenation and weighted averaging) have two major defects: lack of interaction mechanism: no dynamic association between target power sequence and covariates (such as the nonlinear impact of wind speed changes on power curves); information redundancy: simple concatenation leads to an explosion of feature dimensions, which actually reduces the model's generalization ability.

[0009] According to the State Grid "New Energy Power Prediction Technical Specification" (Q / GDW 12073-2021), ultra-short-term prediction (0-4 hours) needs to meet the following requirements: accuracy index: normalized RMSE (nRMSE) <15%; timeliness: prediction period ≤5 minutes, delay <10 seconds; robustness: adapt to different climate regions (such as extreme weather such as sandstorms and typhoons).

[0010] However, existing technologies do not fully meet the above requirements: single LSTM model has nRMSE fluctuations exceeding 25% in extreme weather; Transformer model is difficult to meet real-time requirements due to computational delay; traditional fusion methods (such as CNN-LSTM) have R² of only 0.85-0.88 in Xinjiang Uygur Autonomous Region wind farm tests.

[0011] In summary, there is an urgent need for a new wind power prediction method that can balance long-term dependency modeling, dynamic interaction of multi-source data, and real-time computation. SUMMARY

[0012] The application aims to provide a wind power ultra-short-term prediction method based on a Transformer-LSTM fusion model, aiming to solve the problems of insufficient long-term dependence modeling of target power sequence, insufficient utilization of dynamic characteristics of covariates, and missing feature interaction mechanism in the prior art.

[0013] The application solves the technical problems by adopting the technical solutions of: The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model comprises the following steps: Obtain a target wind farm power historical sequence and multi-source covariate time series data, and divide them into a training set and a test set, wherein the multi-source covariate time series data at least comprises meteorological data; Input the multi-source covariate time series data in the training set into an LSTM local feature extraction branch to obtain a local feature vector, and input the target wind farm power historical sequence in the training set into a Transformer global dependence extraction branch to obtain a global context feature vector; Dynamically fuse the local feature vector and the global context feature vector through a cross-attention mechanism adjusted by a learnable temperature parameter, and output a power prediction value for 0-4 hours in the future through the Transformer-LSTM fusion model to complete training of the Transformer-LSTM fusion model; Evaluate the prediction result of the trained Transformer-LSTM fusion model by using the test set; Deploy the evaluated Transformer-LSTM fusion model on a wind farm edge computing node.

[0014] As a further optimization, the meteorological data comprises measured wind speed, wind direction, temperature, air pressure, and humidity; The multi-source covariate time series data further comprises remote sensing cloud top brightness temperature, radar combined reflectivity, wind turbine cabin vibration, pitch angle, and rotating speed; Before inputting the multi-source covariate time series data in the training set into the LSTM local feature extraction branch, it further comprises prior screening through a variable selection network.

[0015] As a further optimization, after obtaining the target wind farm power historical sequence and the multi-source covariate time series data, and before dividing them into the training set and the test set, it further comprises: Standardize the target wind farm power historical sequence and the multi-source covariate time series data; Generate training samples by using a sliding window, wherein the window length is 24, corresponding to 4 hours of historical data, and the step length is 6, corresponding to 1 hour of prediction; Divide the training set and the test set in a ratio of 7:3.

[0016] As a further optimization, the cross-attention mechanism adjusted by the learnable temperature parameter dynamically fuses the local feature vector and the global context feature vector, which means: The local feature vector is represented as H L , and the global context feature vector is represented as H T as a Query vector; The attention weight matrix is calculated, and the Transformer-LSTM fusion model dynamically adjusts the contribution proportion of the multi-source covariate time series data and the target wind farm power historical sequence at different time steps through the attention weight matrix, and the attention weight matrix is represented as A The calculation formula is: ; Wherein is the feature dimension, is the dot product of the Query vector and the Key vector, is the learnable temperature parameter; The similarity weight of the Key-Value vector and the Query vector is calculated through the attention weight matrix, and the weighted fusion feature F=A•V after dynamic fusion is calculated through the attention weight matrix and the similarity weight, and V is the Value vector; The weighted fusion feature F after dynamic fusion is represented as F=Attention(Q=H T ,K=H L ,V=H L , T ).

[0017] As a further optimization, the learnable temperature parameter T is optimized jointly with the rest of the model parameters in the training phase, which is used to dynamically scale the attention weight matrix, and in the extreme weather scenario, T it is automatically reduced to amplify the covariate weight.

[0018] As a further optimization, the LSTM local feature extraction branch adopts a single-layer one-way LSTM, the number of hidden units = covariate dimension x 2~3, and the output shape is (batch, seq_len, d_l). The LSTM local feature extraction branch adopts a direct multi-step prediction mechanism, the number of output layer nodes is equal to the prediction step number, and the Adam optimizer is used, the initial learning rate is 0.001, and the batch size is 128.

[0019] As a further optimization, the Transformer global dependency extraction branch adopts a 3-layer sparse LogSparse self-attention encoder with 4 heads, an embedding dimension of 32, a feed-forward dimension of 64, and an output shape of (batch, seq_len, d_t). The Transformer global dependency extraction branch uses sinusoidal position encoding to inject temporal information.

[0020] As a further optimization, the cross-attention layer where the learnable temperature parameter adjusted cross-attention mechanism is located has an output dimension consistent with the LSTM local feature extraction branch.

[0021] As a further optimization, the Transformer-LSTM fusion model deployed on the wind farm edge computing node adopts a sliding window based online incremental learning mechanism. When the wind farm edge computing node collects 168 hours of new data, it triggers a sliding window incremental training. The position encoding layer of the Transformer global dependency extraction branch and the hidden state mapping weight of the LSTM local feature extraction branch are kept unchanged, and only the cross-attention layer and the prediction head parameters are updated. During the update process, the learning rate is 1e-4, and the training step is ≤50.

[0022] As a further optimization, the Transformer-LSTM fusion model deployed on the wind farm edge computing node supports GPU parallel acceleration, and its single prediction delay is less than 5 seconds.

[0023] The beneficial effects of the present application are: through the combination of the Transformer-LSTM fusion architecture and the cross attention mechanism, the following advantages are realized: in terms of prediction accuracy, compared with single Transformer or LSTM model, the present application reduces the MSE by 28.3%~40.8%, the RMSE is stable below 0.265, the determination coefficient R² is increased to 0.952, especially in the high volatility scenario, the error fluctuation rate is controlled within 15%, effectively solving the defects of the traditional method in the long-term dependence and the lack of local feature collaborative modeling; in terms of calculation efficiency, through GPU parallel acceleration and sliding window streaming processing, the single prediction delay is less than 5 seconds, which fully meets the real-time requirement of power grid dispatching on ultra-short-term prediction; in terms of engineering applicability, the present application can more accurately capture the wind power climbing / dropping characteristics in complex scenarios such as wind speed mutation and meteorological sudden change, reduce the prediction deviation, so as to help the dispatching center to take the ordered limiting and standby scheduling measures earlier, and the model of the present application supports online learning to adapt to the change of wind farm operation state, and is seamlessly connected with the existing SCADA system through the standardized interface, which has reduced the wind curtailment rate by 2.3 percentage points in the actual deployment of wind farms in Xinjiang Uygur Autonomous Region, and the prediction error of the fusion model is reduced, so that the dispatching strategy is more accurate; in addition, the architecture has expansion and generalization ability, which can be compressed into a lightweight model and deployed on edge devices, and can also be adapted to offshore wind power, photovoltaic power station and other multiple scenarios by adjusting the input dimension, providing reliable technical support for high proportion of renewable energy grid connection. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The flow chart of the wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model in the embodiment one of the present application is shown in the figure. Figure 2 The structure diagram of the LSTM unit in the embodiment two of the present application is shown in the figure. Figure 3 The structure diagram of the LSTM prediction model in the embodiment two of the present application is shown in the figure. Figure 4 The structure diagram of the Transformer deep learning model in the embodiment two of the present application is shown in the figure. Figure 5 The structure diagram of the Transformer-LSTM deep learning model in the embodiment two of the present application is shown in the figure. Figure 6 The specific parameter diagram of the Transformer-LSTM deep learning model in the embodiment two of the present application is shown in the figure. Figure 7 The memory usage size diagram when the embedding dimension is 16 in the embodiment two of the present application is shown in the figure. Figure 8For the GPU usage rate diagram when the embedding dimension is 16 in the second embodiment of the present application; Figure 9 For the GPU usage rate diagram when the embedding dimension is 32 in the second embodiment of the present application; Figure 10 For the GPU usage rate diagram when the embedding dimension is 32 in the second embodiment of the present application; Figure 11 For the GPU usage rate diagram when the embedding dimension is 64 in the second embodiment of the present application; Figure 12 For the GPU usage rate diagram when the embedding dimension is 64 in the second embodiment of the present application; Figure 13 For the image fitting the prediction result of the Transformer-LSTM deep learning model in the second embodiment of the present application; Figure 14 For the image comparing the evaluation index values of the prediction result of the Transformer-LSTM deep learning model in the second embodiment of the present application and other models. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0026] Embodiment one

[0027] The present embodiment provides a wind power ultra-short-term prediction method based on a Transformer-LSTM fusion model, and the flowchart is shown in Figure 1 The method can include the following steps: S1, obtaining a target wind farm power history sequence and multi-source covariant time series data, and dividing them into a training set and a test set, wherein the multi-source covariant time series data at least includes meteorological data; S2, inputting the multi-source covariant time series data in the training set into an LSTM local feature extraction branch to obtain a local feature vector, and inputting the target wind farm power history sequence in the training set into a Transformer global dependence extraction branch to obtain a global context feature vector; S3, dynamically fuse the local feature vector and the global context feature vector through the cross-attention mechanism adjusted by the learnable temperature parameter, and output the power prediction value of 0-4 hours in the future through the Transformer-LSTM fusion model to complete the training of the Transformer-LSTM fusion model; S4, evaluate the prediction results of the trained Transformer-LSTM fusion model using the test set; S5, deploy the evaluated Transformer-LSTM fusion model on the edge computing node of the wind farm.

[0028] In this embodiment, the three-layer cascaded heterogeneous deep learning architecture can be used to combine the local time series modeling capability of LSTM and the global dependence capturing advantage of Transformer, and introduce a cross-attention mechanism to realize dynamic feature fusion, which significantly improves the prediction accuracy and real-time performance.

[0029] The core architecture in this embodiment can include a covariate dynamic feature extraction layer (LSTM local feature extraction branch), a target sequence global modeling layer (Transformer global dependence extraction branch), and a dynamic interaction fusion layer (cross-attention mechanism). In the covariate dynamic feature extraction layer, LSTM network is used to process multi-dimensional covariate time series data such as wind speed, wind direction, temperature, air pressure, and humidity. LSTM can adaptively capture local time dynamics and nonlinear relationships through its unique gating mechanism (forget gate, input gate, and output gate), and the output dimension is 2-3 times the hidden features of the input variables. Therefore, it can effectively solve the problem of insufficient fitting ability of traditional statistical models for nonlinear time series features, while avoiding the gradient vanishing or explosion phenomenon.

[0030] In actual application, the multi-source covariate time series data should at least include meteorological data, which at least includes measured wind speed, wind direction, temperature, air pressure, humidity, and grid point meteorological elements output by numerical weather prediction models. In addition, the multi-source covariate time series data can also include remote sensing cloud top brightness temperature, radar combined reflectivity, wind turbine nacelle vibration, pitch angle, and rotational speed, etc. Among them, the wind turbine nacelle vibration, pitch angle, and rotational speed are SCADA variables.

[0031] In order to improve the robustness of the Transformer-LSTM fusion model and improve the utilization efficiency of the model for these multi-source heterogeneous information, in the embodiment, before the multi-source covariate time series data in the training set is input into the LSTM local feature extraction branch, it also needs to include: prior screening through the variable selection network. Here, the multi-source covariate time series data is sent into the LSTM branch after prior screening by the variable selection network (VSN). Here, since the contribution of each variable to the prediction target can be automatically evaluated through the gating mechanism, the key feature input into the LSTM network is dynamically selected, which effectively reduces the interference of redundant information and enhances the robustness and interpretability of the Transformer-LSTM fusion model under complex working conditions.

[0032] It should be noted that the model training and inference process is also optimized in the embodiment. In the data preprocessing stage, the power sequence and covariates are standardized, and a sliding window mechanism is used to generate training samples (window length 24, step length 6). The model training uses the Adam optimizer, the initial learning rate is set to 0.0001, the batch size is 128, the training rounds are 150 rounds, and the loss function uses mean square error (MSE). Experiments show that this configuration ensures the convergence speed while avoiding overfitting. In the inference stage, the model supports GPU parallel acceleration (such as NVIDIA GTX1050), and the single prediction delay is less than 5 seconds, fully meeting the real-time requirements of ultra-short-term prediction.

[0033] Moreover, based on the Transformer-LSTM fusion model, the training and testing of the fusion model need to be performed during the training process, therefore, after obtaining the target wind farm power historical sequence and multi-source covariate time series data, and before dividing them into a training set and a test set, it also needs to include: standardizing the target wind farm power historical sequence and multi-source covariate time series data; generating training samples using a sliding window, wherein the window length is 24, corresponding to 4 hours of historical data, and the step length is 6, corresponding to 1 hour of prediction; dividing the training set and the test set in a ratio of 7:3.

[0034] The embodiment is verified on the measured data of a wind farm in Xinjiang Uygur Autonomous Region. The data set contains 2023 annual sampling data (10-minute interval), which is divided into training set and test set according to the ratio of 7:3. Through comparative experiments, it can be obtained that the fusion model of the embodiment is significantly better than single Transformer (MSE 0.099) and LSTM (MSE 0.120) model in terms of MSE (0.071), RMSE (0.265) and R² (0.952) indexes, especially in the wind speed mutation scene, the prediction error fluctuation rate of the embodiment is lower than 15%, which shows stronger robustness.

[0035] In the specific application process, in the target sequence global modeling layer, the Transformer encoder is used to process the power history sequence, the long-term dependence relationship across time steps is analyzed through the multi-head self-attention mechanism, the embedding dimension of the Transformer is set to 32, the number of encoder layers is 3, each layer contains 4 attention heads, and the feedforward network dimension is 64. In addition, the time sequence information is injected into the input sequence through the sine position encoding, which makes up for the defect that the Transformer itself lacks time sequence perception ability. Compared with LSTM, Transformer can simultaneously focus on the relationship between any two time steps in the sequence, thereby more effectively capturing long-term patterns such as daily cycles and seasonal cycles.

[0036] For the dynamic interaction fusion layer in the embodiment, the layer realizes the deep fusion of the wind farm power sequence (target sequence) and the meteorological covariant sequence through the cross-attention mechanism. Specifically, first, the power history sequence is input into the Transformer branch to extract the global context features, denoted as H T , and then the wind speed, wind direction, temperature, humidity and other covariants are input into the LSTM branch to extract the local dynamic features, denoted as H L . Subsequently, H T is taken as the Query vector, and H L is taken as the Key-Value vector, and the attention weight matrix is calculated: ; wherein is the feature dimension, is the dot product of the Query vector and the Key vector, is a learnable temperature parameter. Through the weight matrix, the model can dynamically adjust the contribution proportion of the covariants and the power sequence at different time steps, and output the weighted fusion features F=A•V. This fusion method not only retains the uniqueness of each data modality, but also establishes the nonlinear dynamic correlation between the target power and the meteorological elements, overcoming the information redundancy problem caused by the traditional splicing or weighted average method.

[0037] Benefiting from the cross-attention mechanism, the model can more accurately capture the wind power climbing / dropping characteristics in complex scenarios such as sudden wind speed changes and sudden weather changes, reduce prediction bias, and help the dispatch center take orderly dispatching and standby scheduling measures earlier. In the actual deployment of a wind farm in Xinjiang Uygur Autonomous Region, the prediction error of the fusion model is reduced, making the dispatching strategy more accurate, and ultimately reducing the curtailment rate by about 2.3 percentage points.

[0038] It should be noted that in the embodiment, the learnable temperature parameter T In the training phase, the temperature parameter is optimized jointly with the rest of the model parameters, and is used to dynamically scale the attention weight matrix, so that in extreme weather scenarios, T It is automatically reduced to amplify the weight of the covariate.

[0039] Therefore, in the embodiment, the LSTM local feature extraction branch adopts a single-layer one-way LSTM, the number of hidden units = covariate dimension x 2~3, and the output shape is (batch, seq_len, d_l); the LSTM local feature extraction branch adopts a direct multi-step prediction mechanism, the number of output layer nodes is equal to the prediction step number, and the Adam optimizer is used, the initial learning rate is 0.001, and the batch size is 128. The Transformer global dependence extraction branch adopts a 3-layer sparse LogSparse self-attention encoder, the number of heads is 4, the embedding dimension is 32, the feedforward dimension is 64, and the output shape is (batch, seq_len, d_t); the Transformer global dependence extraction branch uses sinusoidal position encoding to inject timing information. The cross-attention layer where the learnable temperature parameter adjustment cross-attention mechanism is located has the same output dimension as the LSTM local feature extraction branch, and can realize end-to-end training.

[0040] To enhance the adaptability of the Transformer-LSTM fusion model to the changes in the operating state of the wind farm, the embodiment can adopt an online incremental learning mechanism based on a sliding window. The mechanism is automatically triggered after accumulating 168 hours of new data, and adopts a partial parameter freezing strategy: the position encoding layer of the Transformer and the hidden state mapping weight of the LSTM are kept unchanged, and only the cross-attention layer and the prediction head are fine-tuned. A lower learning rate (1e-4) is used in the training process, and the number of training steps is controlled within 50 steps, ensuring that the model does not overfit the new data while quickly adapting to recent changes in wind power output characteristics. The entire updating process is completed locally on the edge node, supports hot switching with the old model, does not need to interrupt the prediction service, and realizes true "running while learning".

[0041] In addition, the embodiment can also support an online learning mechanism. By fine-tuning the model parameters with the latest data on a regular basis (e.g., every week), the model can adapt to dynamic factors such as aging of wind farm equipment and seasonal changes. In addition, the model can be compressed into a lightweight version through knowledge distillation technology and deployed on edge devices (e.g., wind turbine PLCs) to realize end-side real-time prediction. These extended functions significantly improve the engineering applicability of the present application. In the embodiment, the Transformer-LSTM fusion model deployed on the edge computing node of the wind farm supports GPU parallel acceleration, and its single prediction delay is less than 5 seconds in the GPU (NVIDIA GTX1050 and above) environment.

[0042] Embodiment Two Based on the embodiment one, the embodiment provides a wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model. The method further describes the Transformer-LSTM fusion model.

[0043] The Transformer-LSTM hybrid model prediction of the embodiment combines the advantages of both models. The Transformer has the advantages of capturing global dependencies with self-attention mechanisms and modeling long-range temporal relationships. It also has no loop structure, can be calculated in parallel, and has fast training and reasoning speed. The gating mechanism of the LSTM effectively preserves or forgets information, is good at depicting local temporal dynamics and short-term fluctuations, and is robust to gradient vanishing / explosion, suitable for continuous small-step sequence modeling. In the Transformer-LSTM hybrid model combined by the two models in the embodiment, the LSTM first extracts local temporal features and retains long-term memory, and the Transformer then makes global self-attention correlation on the LSTM output, taking into account local details and global trends. The two models complement each other, improve prediction accuracy, reduce errors, and maintain high computational efficiency, and are suitable for time series prediction tasks such as wind power with multi-scale features.

[0044] Referring to the accompanying drawings Figure 2, which demonstrates a single neuron of LSTM (Long Short-Term Memory) network. LSTM memory cell is the basic building block in LSTM network, which is particularly suitable for processing and predicting time series-based data as it can effectively capture long-term dependency information. LSTM memory cell contains three gates: forget gate, input gate, and output gate, each of which controls the flow of information. The forget gate decides which information needs to be discarded from the cell state, it receives the previous hidden state and current input, and computes a value between 0 and 1 through a sigmoid function, representing the proportion of each state value that needs to be forgotten. The input gate decides which new information needs to be added to the cell state, it also receives the previous hidden state and current input, and decides which values need to be updated through a sigmoid function, while using a tanh function to generate a new candidate value vector for state update. The cell state update is the process of forgetting old information and adding new information, and the updated cell state combines the outputs of the forget gate and input gate. The output gate decides what to output, which is based on the current cell state and the previous hidden state, through a sigmoid function and a tanh function to determine the final output. This structure enables LSTM to effectively handle and predict important events in time series, even if these events are separated by multiple time steps. By controlling the flow of information, LSTM can learn long-term dependencies, which is crucial for many sequence prediction tasks.

[0045] Referring to the accompanying drawings Figure 3 , which demonstrates a typical LSTM (Long Short-Term Memory) neural network structure, consisting of an input layer, two hidden layers, and an output layer. In the input layer, the network receives a series of time steps of data x 1 、x 2 、…、x 24 These data are fed into the LSTM cells in the hidden layer, each consisting of an input gate, a forget gate, and an output gate, which work together to process and remember key information in sequence data. Each LSTM cell in the hidden layer contains a cell state and a hidden state , which updates over time and can capture long-term dependencies. The cell state is responsible for storing long-term memory, while the hidden state carries the output features of the current time step and passes them to the cells of the next time step. The transfer of information within and between layers is controlled by the input gate, forget gate, and output gate, allowing the network to selectively retain or forget historical information. The output of the hidden layer is passed to the output layer, which generates the final prediction result y1 、 y 2 、…y 24 The cell state update process within each LSTM unit is also annotated, showing how information flows and is processed within the network. Additionally, the network's hidden layer depth and dimensionality are scalable, allowing adjustments based on task complexity and data characteristics to further enhance learning ability and prediction accuracy. The entire structure allows the LSTM network to effectively handle sequential data, making it excel in tasks such as time series prediction, natural language processing, and more.

[0046] Referring to FIG. 6, Figure 4 The structure of the Transformer neural network is shown in FIG. 6. In this embodiment, the Transformer model is an attention-based deep learning architecture composed of an encoder and a decoder. Both parts contain multiple stacked layers, each consisting of a multi-head attention mechanism and a feed-forward network, with residual connections and layer normalization after each sub-layer. The encoder first converts the input sequence into vector representations through input embedding and positional encoding, then captures the internal dependencies within the sequence using the multi-head attention mechanism. The decoder adds a masked multi-head attention mechanism to the encoder to ensure that future information is not seen when generating the output sequence, while processing the target sequence through output embedding and positional encoding. In the decoder, the multi-head attention mechanism also allows the model to focus on the encoder's output to incorporate information from the input sequence. Finally, the decoder's output generates a prediction probability distribution for each position through a linear layer and a Softmax layer. This structure enables the Transformer to effectively handle sequential data, capture long-range dependencies, and support efficient parallel computation.

[0047] Referring to FIG. 6, Figure 5 The structure of the Transformer-LSTM fusion model in this embodiment is shown in FIG. 6. This model combines the self-attention mechanism of Transformer and the temporal processing capability of LSTM. In this embodiment, the main improvements are reflected in the dual improvements of model structure and fusion mechanism. First, in terms of model structure, a parallel dual-branch architecture is adopted: the power history sequence is processed through the Transformer branch to extract global context features across time steps, while the covariates (wind speed, wind direction, temperature, humidity, etc.) are processed through the LSTM branch to extract local dynamic features. The two branches run independently and do not interfere with each other, effectively preserving their respective modal advantages and avoiding information loss and computational delay caused by serial structures. Second, in the fusion process, a cross-attention mechanism is introduced, with the Transformer output as Query and the LSTM output as Key-Value. The similarity weight matrix is calculated as follows: ; The feature-level adaptive weighting aggregation is implemented to dynamically adjust the contribution proportion of power and meteorological elements at different times, and the fused feature F=A•V is obtained. This mechanism not only establishes a dynamic nonlinear mapping between the power sequence and the meteorological covariates, and overcomes the problem of lack of interaction and easy redundancy of traditional splicing or weighted average, but also takes into account the ability of LSTM to describe short-term climbing and local fluctuations and the ability of Transformer to capture long-term trends, so as to solve the shortcomings of single model in long and short-term dependence modeling and insufficient use of multi-source data, improve the prediction accuracy and robustness while maintaining real-time, and more accurately reflect the power change trend under complex working conditions such as wind speed mutation, thereby providing reliable support for fine scheduling of power grid and reducing the wind curtailment rate.

[0048] In the embodiment, the model processes the historical power data through the Transformer global dependence extraction branch, captures the dependence relationship between different time steps in the sequence by using the self-attention mechanism, and enhances the understanding of long-term dependence and time sequence information by multi-head attention and position encoding. At the same time, the LSTM local feature extraction branch is responsible for processing covariate data including wind speed, temperature, etc., and captures the short-term dynamic change and local time dependence of the covariates through the gating mechanism. The cross-attention mechanism in the model dynamically fuses the global context features of Transformer and the local dynamic features of LSTM to adaptively adjust the contribution proportion of covariates and power sequence at different time steps, and finally outputs the power prediction value of 0-4 hours in the future through the full connection layer. The entire model uses mean square error (MSE) as the loss function, and the parameters are optimized by Adam optimizer to realize high-precision prediction of wind power ultra-short term, and meet the real-time optimization control demand of power grid scheduling system.

[0049] Referring to FIG. 1, the model is composed of two branches: the Transformer global dependence extraction branch and the LSTM local feature extraction branch. Figure 6, which lists the key training parameters of the Transformer-LSTM fusion model and their settings, including a batch size of 128, meaning that 128 samples will be processed at a time; a learning rate of 0.0001, which controls the step size of model weight updates; 150 training epochs, indicating that the model will be trained on the entire dataset for 150 times; and a loss function of mean squared error (MSE), used to evaluate the difference between predicted and actual values. In addition, the figure also indicates the embedding dimension of 32 and the number of layers of 3 for the Transformer module, as well as the hidden layer dimension of 32 and the number of layers of 3 for the LSTM module, which together determine the capacity and learning ability of the model, affecting the extraction of sequence data features and the prediction performance of the model. First, based on the training times selected based on deep learning project experience, the training times of neural networks generally reach a peak around 150 times. See Figures 7-12 , other parameters such as embedding dimension and number of layers are selected based on multiple sets of comparative experiments on a cloud server NVIDIA RTX 4090D (24 GB) graphics card: when the embedding dimension increases from 16 to 32, the validation set RMSE decreases by about 7%, and continues to increase to 64, only decreases by 1.6%, but the video memory occupancy almost doubles; the number of layers increases from 2 to 3, the RMSE decreases by 5.2%, and 4 layers increase by 1.3% due to overfitting, while the hidden layer dimension and the number of layers of LSTM are unified with the embedding dimension and the number of layers of Transformer, taking 32 and 3 layers. Considering the model performance, training efficiency and hardware memory upper limit, the combination finally determines that it has good training stability and convergence speed while ensuring prediction accuracy.

[0050] See attached Figure 13 , which is the fitting image after prediction by the Transformer-LSTM fusion model in the embodiment of the present application, showing the fitting of the test set prediction results in this prediction, where the blue line represents the actual data and the red line represents the prediction results of the model. From the figure, it can be seen that the model performs well in capturing the overall trend of the data, and the prediction line closely fits the actual line in most areas, showing a good fitting trend. In addition, the correlation coefficient between the model and the actual data is as high as 0.953, which further confirms that there is a high linear correlation between the model prediction and the actual observation value, indicating that the model has high prediction accuracy.

[0051] See attached Figure 14The prediction result of the Transformer-LSTM deep learning model of the embodiment of the present application and the evaluation index value of other models are shown, and the Transformer-LSTM fusion model of the embodiment is significantly better than single Transformer and LSTM in three indexes of MSE, RMSE and R²: the MSE is reduced to 0.071, which is reduced by 28% compared with Transformer and reduced by 41% compared with LSTM; the RMSE is only 0.265, which is reduced by 16% and 23% respectively; the R² is as high as 0.952, which is increased by about 5-7 percentage points, which means that the model can explain more than 95% of the power change. The result verifies that after the cross-attention is combined with the local dynamic characteristics of LSTM and the global dependence characteristics of Transformer, the fusion structure has the perception ability of short-term fluctuation and long-term trend, and still maintains lower error and higher fitting degree in the high-noise and strong nonlinear scene of wind power, thereby providing a more accurate solution for ultra-short-term power prediction.

[0052] The above merely illustrates the preferred embodiments of the present application and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A wind power ultra-short-term prediction method based on a Transformer-LSTM fusion model, characterized in that, The method comprises the following steps: obtaining a target wind farm power history sequence and multi-source covariate time series data, and dividing the multi-source covariate time series data into a training set and a test set, wherein the multi-source covariate time series data at least comprises meteorological data; inputting the multi-source covariate time series data in the training set into an LSTM local feature extraction branch to obtain a local feature vector, and inputting the target wind farm power history sequence in the training set into a Transformer global dependence extraction branch to obtain a global context feature vector; dynamically fusing the local feature vector and the global context feature vector through a cross-attention mechanism adjusted by a learnable temperature parameter, and outputting a power prediction value for 0-4 hours in the future through a Transformer-LSTM fusion model to complete training of the Transformer-LSTM fusion model; evaluating the prediction result of the trained Transformer-LSTM fusion model by using the test set; deploying the evaluated Transformer-LSTM fusion model on a wind farm edge computing node.

2. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, The meteorological data comprises measured wind speed, wind direction, temperature, air pressure and humidity; The multi-source covariate time series data further comprises remote sensing cloud top brightness temperature, radar combined reflectivity, wind turbine cabin vibration, pitch angle and rotating speed; Before inputting the multi-source covariate time series data in the training set into the LSTM local feature extraction branch, the method further comprises:

3. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, performing prior screening through a variable selection network. After obtaining the target wind farm power history sequence and the multi-source covariate time series data, and before dividing the target wind farm power history sequence and the multi-source covariate time series data into the training set and the test set, the method further comprises: performing standardization processing on the target wind farm power history sequence and the multi-source covariate time series data; generating training samples by using a sliding window, wherein the window length is 24, corresponding to 4 hours of historical data, and the step length is 6, corresponding to 1 hour of prediction; 4. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, dividing the training set and the test set in a ratio of 7:

3. The local feature vector is denoted as H L The global context feature vector is denoted as H T and as Query vector; The attention weight matrix is calculated, and the Transformer-LSTM fusion model dynamically adjusts the contribution proportion of the multi-source covariant time series data and the historical sequence of the target wind farm power at different time steps through the attention weight matrix, and the attention weight matrix is represented as A The calculation formula is: ； wherein is the feature dimension, is the dot product of Query vector and Key vector, is a learnable temperature parameter; The dynamically fusing the local feature vector and the global context feature vector through the cross-attention mechanism adjusted by the learnable temperature parameter is that: The weighted fusion feature F after dynamic fusion is represented as F = Attention(Q = H T , K = H L , V = H L , T ).

5. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 4, characterized in that, the learnable temperature parameter T In the training phase, jointly optimize with the rest of the model parameters for dynamically scaling the attention weight matrix in extreme weather scenarios, T Automatic reduction to amplify the covariate weights.

6. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, the similarity weight of the Key-Value vector and the Query vector is calculated through an attention weight matrix, and the weighted fusion feature F=A•V after dynamic fusion is calculated through the attention weight matrix and the similarity weight, wherein V is a Value vector; The LSTM local feature extraction branch adopts a single-layer one-way LSTM, the number of hidden units is equal to the product of the covariate dimension and 2-3, and the output shape is (batch, seq_len, d_l); 7. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, The LSTM local feature extraction branch adopts a direct multi-step prediction mechanism, the number of output layer nodes is equal to the prediction step number, and an Adam optimizer is used, with an initial learning rate of 0.001 and a batch size of 128. The Transformer global dependence extraction branch adopts a 3-layer sparse LogSparse self-attention encoder, the number of heads is 4, the embedding dimension is 32, the feedforward dimension is 64, and the output shape is (batch, seq_len, d_t). The Transformer global dependence extraction branch uses sinusoidal position encoding to inject timing information.

8. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, The cross-attention layer where the learnable temperature parameter adjusted cross-attention mechanism is located has an output dimension consistent with the LSTM local feature extraction branch.

9. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to claim 1, characterized in that, The Transformer-LSTM fusion model deployed on the wind farm edge computing node adopts a sliding window based online incremental learning mechanism. When the wind farm edge computing node collects 168 hours of new data, it triggers the sliding window incremental training. The position encoding layer of the Transformer global dependence extraction branch and the hidden state mapping weight of the LSTM local feature extraction branch are kept unchanged, and only the cross-attention layer and the prediction head parameters are updated. During the updating process, the learning rate is 1e-4, and the training step number is less than or equal to 50.

10. The wind power ultra-short-term prediction method based on the Transformer-LSTM fusion model according to any one of claims 1-9, characterized in that, The Transformer-LSTM fusion model deployed on the wind farm edge computing node supports GPU parallel acceleration, and the single prediction delay is less than 5 seconds.

Images