Wind power output probability prediction method based on time sequence prompt words and multi-dimensional attention

Abstract

The present application belongs to the technical field of wind power generation, and specifically relates to a wind power output probability prediction method based on time sequence prompt words and multi-dimensional attention. In view of the fact that the existing wind power output probability prediction method is difficult to balance the prediction accuracy of conventional scenes and extreme scenes, the present application adopts the following technical scheme: a wind power output probability prediction method, comprising: obtaining historical output time series of a wind farm and corresponding meteorological covariate prediction data of a time period; training a basic model to construct a time sequence prompt word calculation function; constructing a deep neural network, the deep neural network comprising a covariate embedding function and a time embedding function, and extracting multi-dimensional correlation features through a cascaded covariate decoder and time decoder based on a Transformer structure; at the same time, setting zero padding bits for injecting time sequence prompt words in the deep neural network input for the to-be-predicted time period; training the deep neural network; and outputting wind power output probability prediction.

Description

Technical Field

[0001] This invention belongs to the field of wind power generation technology, specifically involving a wind power output probability prediction method based on time-series cue words and multidimensional attention. Background Technology

[0002] Wind power output forecasting is one of the key technologies for ensuring the safe and economical operation of a high-proportion renewable energy power system. Most deep learning models used for wind power output probability forecasting are direct extensions of point prediction models, adding multiple quantile output nodes to the output layer and using a quantile loss function for training. While these basic models (such as Long Short-Term Memory (LSTM) and Temporal Convolutional Networks (TCN)) demonstrate some predictive ability under normal weather conditions, their limitations become glaringly apparent in extreme weather scenarios. In extreme wind speeds (such as typhoons and hurricanes), wind turbines are forced to shut down due to self-protection mechanisms, causing output to plummet to zero, exhibiting a strong nonlinear characteristic of "increased wind speed, sharp decrease in output." Because such samples are extremely rare in historical data, the model training process, aiming to minimize global prediction error, tends to fit the conventional patterns with a high sample ratio, completely failing to capture this anomalous and sparse shutdown characteristic. This ultimately leads to severe prediction bias in extreme scenarios, posing a significant risk to the safe operation of the power grid.

[0003] In recent years, the Transformer model has shown great potential in the field of time series forecasting due to its powerful sequence modeling and long-range dependency capture capabilities. It has also been applied to time series forecasting tasks and extended to probabilistic forecasting forms to quantify the uncertainty of wind power output.

[0004] Existing methods for predicting wind power output probabilistically using the Transformer model rely on the core idea of ​​using the Transformer as a powerful feature extractor to automatically learn the complex mapping relationship between historical output sequences and meteorological covariates to future output through a self-attention mechanism. However, this method is essentially still a purely data-driven "black box" modeling paradigm, failing to deeply integrate the physical mechanisms and operational laws of wind power output into the model design. Consequently, it cannot adaptively adjust the predicted distribution when the physical environment undergoes sudden changes, leading to significant deviations between the predicted and actual wind power output.

[0005] Specifically, wind power output sequences exhibit significant multidimensional correlation characteristics—they possess both autocorrelation in the time dimension and strong coupling with meteorological covariates. Furthermore, output behavior under extreme wind speeds is dominated by external physical rules (such as turbine protection shutdowns) rather than simple historical data statistical patterns. However, existing Transformer models, when predicting wind power output probabilities, treat the time and covariate dimensions separately and lack mechanisms to integrate external prior information. This forces the model to learn these complex, multi-layered physical laws implicitly from scratch. This disconnect between the physical mechanisms and the model structure leads to low model learning efficiency and inherent biases.

[0006] For conventional scenarios, because the model fails to design a collaborative attention mechanism for time and covariate dimensions, it is difficult to achieve the optimal state when extracting the spatiotemporal evolution and meteorological coupling characteristics of wind power output, which limits the potential for further improvement in the accuracy of basic probability prediction.

[0007] More seriously, for extreme scenarios, the proportion of extreme shutdown samples in the training set is extremely low, and their anomalous pattern of "increased wind speed and sharp decrease in output" fundamentally conflicts with the trend of "higher wind speed and greater output" in regular samples. The gradient descent optimization process, which aims to minimize the global prediction error, will drive the model to prioritize fitting the dominant regular samples, completely ignoring the sparse characteristics of extreme samples. This causes the model to fail to correctly infer shutdown behavior when faced with extreme wind speeds, resulting in an output that deviates significantly from the actual prediction range (e.g., predicting full power generation when the actual output is zero).

[0008] In summary, existing methods for predicting wind power output probabilistics using the Transformer model suffer from the following irreconcilable drawbacks: 1. There is a bottleneck in the prediction accuracy of conventional scenarios: the failure to fully explore multidimensional correlation information limits the upper limit of the model's performance on the overall dataset.

[0009] 2. Severe inaccuracy in extreme scenario predictions: It is unable to capture low-probability anomalous nonlinear characteristics, resulting in huge prediction biases and range shifts under extreme operating conditions, posing a direct risk to the safe operation of the power grid.

[0010] Furthermore, other prediction methods specifically designed for extreme scenarios often fail to accurately predict common scenarios. Summary of the Invention

[0011] This invention addresses the shortcomings of existing wind power output probability prediction methods, which struggle to balance prediction accuracy in both conventional and extreme scenarios. It provides a wind power output probability prediction method based on a time-series prompt word-assisted multidimensional Transformer, deeply integrating the physical mechanisms and operational experience of wind power output into the model design. This fundamentally resolves the contradiction between the accuracy bottleneck in conventional scenarios and the inaccuracy in extreme scenarios, thereby improving both accuracy and reliability.

[0012] To achieve the above objectives, the present invention adopts the following technical solution: a wind power output probability prediction method based on temporal cue words and multidimensional attention, comprising: S1. Obtain the historical power output time series of the wind farm and the meteorological covariate forecast data for the corresponding time period, construct input-output sample pairs, and divide them into training set, validation set and test set according to time order; S2. Train the basic model based on the training set data, statistically analyze the distribution of prediction error of the basic model under different wind speeds and determine the shutdown wind speed threshold, calculate the mean and maximum value of prediction error in each wind speed interval for samples exceeding the shutdown wind speed threshold, and construct a time-series prompt word calculation function that associates the wind speed forecast value with the mean and maximum value of the prediction error. S3. Construct a deep neural network, which includes a covariate embedding function and a temporal embedding function, respectively mapping the input sequence to the feature spaces of the covariate dimension and the temporal dimension. Multidimensional correlation features are extracted through a cascaded covariate decoder and temporal decoder based on the Transformer structure. At the same time, zero-padding bits are set in the input of the deep neural network for the time period to be predicted to inject time-series cue words. S4. Perform end-to-end training on the deep neural network. During the training process, enable the deep neural network to learn the mapping relationship from historical output and meteorological covariates to the quantiles of future output. Adjust the hyperparameters through the validation set until the prediction error of the deep neural network reaches the expected level. S5. Input the acquired historical power output sequence, future weather forecast data, and time-series cue words calculated based on the forecast wind speed and time-series cue words into the trained deep neural network for forward inference to obtain the quantile prediction results for future times, which are used as the wind power output probability prediction output.

[0013] As an improvement, in S1, the meteorological covariate forecast data includes wind speed forecast values ​​at different altitudes; In S1, according to the preset historical time window length L and prediction window length S Perform sliding window slicing; preprocess the historical power output time series of the wind farm before performing sliding window slicing. In S1, the input of each sample contains a length of L The historical output sequence and length are L +S The meteorological covariate sequence is output as the corresponding S Predicted output sequence at time step.

[0014] As an improvement, in S2, the base model is a multilayer perceptron; In S2, without introducing time-series cue words, a base model is trained on the training set to obtain a baseline prediction model, which is then used to predict the wind speed forecast value for each training set sample. Construct a prediction error with wind power output prediction error. Compared with wind speed forecast values The jointly distributed dataset; In S2, the wind speed forecast values ​​in the training set are statistically analyzed. The historical distribution of its first The wind speed value corresponding to the quantile is used as the shutdown wind speed threshold. For samples with wind speeds exceeding this threshold, the mean prediction error within each wind speed interval is further calculated. and maximum value , respectively, as the median of the wind power output to be predicted and the lower bound quantile of the wind power output to be predicted The baseline correction amount; In S2, the process of constructing the timing cue word calculation function includes: when the wind speed forecast value... Time, timing prompt words ;when At that time, for the time sequence cue word corresponding to the median For the timing cue words corresponding to the lower bound quantile ,in This is an adjustable hyperparameter used to control the magnitude of timing cue word correction.

[0015] As an improvement, in S3, the historical output time series is copied several times to obtain a multiquantile historical sequence copy; the unpredictable portion of the target sequence is padded with zeros to obtain a zero-padded sequence aligned with the time dimension of the meteorological covariate sequence; the multiquantile historical sequence copy, the zero-padded sequence, and the meteorological covariate sequence are concatenated along the feature dimension to form the input of the deep neural network. .

[0016] As an improvement, in S3, in the zero-filling bits of the time period to be predicted, a function is calculated based on the constructed time-series cue words for each quantile. q and forecast wind speed Generate timing prompt words This is then added to the zero-padding sequence of the corresponding quantile to obtain the deep neural network input containing external prior guidance. When the timing cue word is zero, the deep neural network degenerates into a regular prediction mode. When a non-zero cue word is injected, the output of the deep neural network tends to approach the guiding signal, thus achieving adaptive correction of the prediction distribution in extreme scenarios. The timing prompts and historical output time series are processed using the same embedding parameters.

[0017] As an improvement, in S3, the covariate embedding function: for the input tensor The covariate dimension is reconstructed, and then the last dimension is increased to the deep neural network dimension through linear mapping. , obtain the input of the covariate decoder It is used to capture the coupling relationship between different quantiles and between meteorological covariates and power output; In S3, the time embedding function: by setting the Patch length and sliding step size , input tensor Divide the data into multiple patches along the time dimension; then transform the last dimension using a linear mapping. , obtain the time decoder input It is used to extract the local evolution pattern of wind power output sequence in the time dimension.

[0018] As an improvement, in S3, each prediction module of the deep neural network consists of a covariate decoder and a temporal decoder connected in sequence, both of which are decoders based on the Transformer structure. For covariate decoders: As input, through stacking The decoder layer of the layer calculates the attention weights between different quantiles and meteorological factors along the covariate dimension, and outputs the updated hidden state. Each decoder layer contains one multi-head attention mechanism computation and one feedforward neural network computation, and is connected through residual connections and layer normalization modules to stabilize training. For the time decoder: For input, also through The decoder layer of the layer captures the dependencies between patches at different times in the time dimension and outputs the updated hidden state. Each decoder layer contains one multi-head attention mechanism computation and one feedforward neural network computation, and is connected through residual connections and layer normalization modules to stabilize training.

[0019] As an improvement, in S3, multiple prediction modules mentioned above are stacked through residual connections and batch normalization to form a deep network: Output of each module As the first Input of each module, number of stacks Adjusted according to task complexity, after the last module, the hidden state is transformed into the target sequence through a linear output mapping layer. Predicted value at the quantile This completes the end-to-end mapping from input to probability prediction results.

[0020] As an improvement, in S4, quantile loss is used as the objective function. For the preset Q quantiles to be predicted, the quantile loss function is defined as follows:

[0021]

[0022] In the formula: The overall loss; N This represents the number of training samples; For the first i The actual wind power output of each sample; For the first i Wind power output of a sample q Quantile predictions; for q The quantile loss calculation function, where For the characteristic function, when The value is 1 if the condition is met, otherwise it is 0. In S4, an adaptive momentum estimation optimizer is used to perform end-to-end gradient updates of the deep neural network parameters: setting the initial learning rate, batch size, and number of training epochs; during training, an early stopping mechanism is used to monitor changes in quantile loss on the validation set, and if continuous... E l If the loss on the validation set does not decrease, training should be stopped to prevent overfitting. In S4, after training is completed, the various probability prediction metrics of the deep neural network are evaluated on the test set. If the accuracy of each metric meets the requirements, the application phase begins; otherwise, the hyperparameters are adjusted and the network is retrained.

[0023] As an improvement, in S5, for the time to be predicted, the length is obtained as follows: L The historical wind power output sequence and length are S Future weather forecast data, based on forecast wind speed sequence The corresponding timing cue words are calculated using the constructed timing cue word calculation function. This information is then injected into the zero-padding bits of the time period to be predicted in the input of the deep neural network. The complete deep neural network input, consisting of historical output sequences, meteorological covariate sequences, and injected time-series cue words, is fed into the trained deep neural network for forward inference. Through the collaborative computation of the covariate decoder and the time decoder, the future...S At that moment Q The quantile prediction results are used as the wind power output probability prediction output.

[0024] The beneficial effects of the wind power output probability prediction method of this invention are as follows: It innovatively proposes a time-series cue word mechanism to introduce external prior information (such as wind turbine protection shutdown, extreme weather warnings, etc.) into a deep neural network. This mechanism can adaptively adjust the prediction distribution under extreme operating conditions without updating parameters, effectively solving the prediction offset problem of traditional methods in extreme wind speed scenarios. For time-series cue words in wind turbine shutdown scenarios, it transforms the physical prior of wind turbine shutdown into a quantified numerical guidance signal, providing an interpretable and interventionist external knowledge injection path for the deep neural network. Furthermore, it collaboratively designs a time-dimensional attention mechanism and a covariate-dimensional attention mechanism, respectively used to extract the autocorrelation features of the wind power output sequence over time and its correlation with meteorological data. The deep coupling between variables and the cascading fusion of two-dimensional attention modules enable the full mining of multidimensional spatiotemporal correlation information of new energy sequences, fundamentally improving the accuracy of basic probability prediction. A fast calculation method is proposed to directly calculate the "future output multiquantile probability prediction result" from "historical output sequence, meteorological covariate sequence, and external prior information". With a deep neural network of collaborative multidimensional attention Transformer as the core and combined with the temporal prompt word mechanism, the complex high-dimensional nonlinear temporal modeling problem and the extreme scenario small sample learning problem are uniformly transformed into a single efficient feedforward neural network inference. While ensuring the prediction accuracy in conventional scenarios, the prediction reliability in extreme scenarios is significantly improved. Attached Figure Description

[0025] Figure 1 This is a flowchart of the wind power output probability prediction method according to Embodiment 1 of the present invention.

[0026] Figure 2 This is a diagram of the deep neural network architecture used in the wind power output probability prediction method of Embodiment 1 of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be explained and described below. However, the following embodiments are only preferred embodiments of the present invention and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of the present invention.

[0028] See Figure 1 The wind power output probability prediction method of this invention includes: S1. Obtain the historical power output time series of the wind farm and the meteorological covariate forecast data for the corresponding time period, construct input-output sample pairs, and divide them into training set, validation set and test set according to time order; S2. Train the basic model based on the training set data, statistically analyze the distribution of prediction error of the basic model under different wind speeds and determine the shutdown wind speed threshold, calculate the mean and maximum value of prediction error in each wind speed interval for samples exceeding the shutdown wind speed threshold, and construct a time-series prompt word calculation function that associates the wind speed forecast value with the mean and maximum value of the prediction error. S3. Construct a deep neural network, which includes a covariate embedding function and a temporal embedding function, respectively mapping the input sequence to the feature spaces of the covariate dimension and the temporal dimension. Multidimensional correlation features are extracted through a cascaded covariate decoder and temporal decoder based on the Transformer structure. At the same time, zero-padding bits are set in the input of the deep neural network for the time period to be predicted to inject time-series cue words. S4. Perform end-to-end training on the deep neural network. During the training process, enable the deep neural network to learn the mapping relationship from historical output and meteorological covariates to the quantiles of future output. Adjust the hyperparameters through the validation set until the prediction error of the deep neural network reaches the expected level. S5. Input the acquired historical power output sequence, future weather forecast data, and time-series cue words calculated based on the forecast wind speed and time-series cue words into the trained deep neural network for forward inference to obtain the quantile prediction results for future times, which are used as the wind power output probability prediction output.

[0029] The wind power output probability prediction method of this invention addresses the challenge of learning from historical data for low-probability events such as wind turbine shutdowns in extreme wind speed scenarios. It innovatively introduces a time-series cue word mechanism. This mechanism quantifies external physical priors (such as wind turbine protection shutdown rules and historical prediction error distributions) into guiding signals that can be injected into a deep neural network. These signals are embedded using zero-filling bits in the time period to be predicted, allowing the deep neural network to adaptively adjust the prediction distribution under extreme conditions without updating its parameters. This design overcomes the fitting defects of purely data-driven models to sparse samples, ensuring a reliable envelope of the prediction interval to the actual output in extreme scenarios while avoiding damage to generalization performance in conventional scenarios. It fundamentally solves the problem of traditional methods in... To address the problem of excessive prediction errors in extreme scenarios, a collaborative attention mechanism integrating time and covariate dimensions was designed to address the deep correlation between wind power output in both time evolution and meteorological coupling. This mechanism enables deep neural networks to capture time-series autocorrelation features and complex nonlinear coupling relationships between multiple meteorological factors, and achieves full interaction of multidimensional information through cascaded fusion. Compared to traditional Transformers that focus only on single-dimensional attention calculations, the multidimensional attention Transformer structure of deep neural networks in this embodiment can more comprehensively uncover the inherent patterns of new energy sequences, thereby significantly improving the basic accuracy of probabilistic predictions in conventional scenarios and providing a more reliable decision-making basis for scheduling operations.

[0030] Example 1 See Figure 1 The wind power output probability prediction method of Embodiment 1 of the present invention mainly includes several main steps: data preparation, feature calculation, model construction, model training and model application.

[0031] Data preparation: Obtain historical power output sequences of wind farms and meteorological covariate forecast data for corresponding time periods (including wind speed, wind direction, etc. at different altitudes), perform sliding window slicing according to the preset historical time window length L and prediction window length S, construct input-output sample pairs, and divide them into training set, validation set and test set in chronological order.

[0032] Feature calculation: Based on the training set data, the basic model is trained without introducing time-series cue words. The distribution of prediction error of the model under different wind speeds is statistically analyzed to determine the shutdown wind speed threshold. The mean and maximum value of the prediction error in each wind speed interval are calculated. Based on this index, a time-series cue word calculation function is constructed.

[0033] Model Construction: A deep neural network based on a multidimensional Transformer is constructed. This network includes a covariate embedding function and a temporal embedding function, which map the input sequence to the feature spaces of the covariate dimension and the temporal dimension, respectively. Multidimensional correlation features are extracted through a cascaded covariate decoder and temporal decoder. At the same time, zero-padding bits are set in the input of the deep neural network for the time period to be predicted, which are used to inject the temporal cue words output by the constructed temporal cue word calculation function to realize the guidance of external prior information.

[0034] Model training: Using quantile loss as the objective function, the Adam optimizer is used to train the deep neural network end-to-end. During the training process, the deep neural network learns the mapping relationship from historical output and meteorological covariates to future output quantiles, and the hyperparameters are adjusted through the validation set until the prediction error of the deep neural network reaches the expected level.

[0035] Model Application: For the time to be predicted, historical power output sequences and future weather forecast data are obtained. Time-series prompts are calculated based on the forecast wind speed and injected into the input of a deep neural network. The processed input is then fed into a trained deep neural network for forward inference to obtain the future... S The Q quantile prediction results at time point Q are used as the wind power output probability prediction output.

[0036] This embodiment uses wind turbine shutdown under extreme wind speed scenarios as an example. The sudden drop in output caused by wind turbine shutdown under extreme wind speeds is used as a representative nonlinear feature, and an external prior information is introduced using a time-series cue word mechanism for adaptive correction. The prediction offset correction method for other extreme operating conditions is the same as that for the wind turbine shutdown scenario.

[0037] During the data preparation phase, historical power output sequences and corresponding meteorological covariate forecast data for selected wind farms are obtained from the target dataset. These meteorological covariates include surface... h 1 and h In this embodiment, the predicted wind speed value at altitude 2 is selected. h 1 = 10m h 2 = 100m.

[0038] In the data preparation stage, the raw data (historical power output sequence and meteorological covariate forecast data for the corresponding time period) are preprocessed. Linear interpolation is used to fill in missing values ​​caused by equipment failure or communication interruption. The variables are normalized based on the Z-score standardization method to eliminate the influence of different units on the training of deep neural networks. During the data preparation phase, set the historical window length. L Prediction window length SA sliding window approach is used to construct input-output sample pairs from continuous time series data. Each sample's input contains data of length [length missing]. L The historical output sequence and length are L + S Meteorological covariate sequence (future) S The length of the meteorological covariate sequence is obtained from meteorological forecast data, and the output is the corresponding... S The predicted output sequence at each time step; all samples are divided into training, validation, and test sets in chronological order to ensure the reasonableness and generalization ability of the evaluation results. In this embodiment, the following is selected: L = 72h (i.e., 3 days of history). S = 24h (i.e., the next day).

[0039] In the feature calculation stage, without introducing time-series cue words, a basic time-series prediction model is trained on the training set to obtain a baseline prediction model. The baseline prediction model is then used to predict the wind speed forecast value for each training set sample. Error in wind power output prediction A joint distribution dataset of prediction errors and wind speed forecast values ​​is constructed. In this embodiment, the basic time series prediction model is a multilayer perceptron (MLP).

[0040] In the feature calculation stage, the statistical characteristics of the prediction error changing with wind speed are analyzed to determine the shutdown wind speed threshold: statistical training set. The historical distribution of its first The wind speed value corresponding to the quantile is used as the shutdown wind speed threshold. For samples with wind speeds exceeding this threshold, the mean prediction error within each wind speed interval is further calculated. and maximum value , respectively, as the median quantiles of the wind power output to be predicted and the lower bound quantile of the wind power output to be predicted The baseline correction amount. In the embodiment, select... , , ; In the feature calculation stage, a time-series cue word calculation function is constructed based on statistical results: when the wind speed forecast value... Time, timing prompt words ;when At that time, for the time sequence cue word corresponding to the median, For the timing cue words corresponding to the lower bound quantile, ,in This is an adjustable hyperparameter used to control the magnitude of timing cue word correction. In this embodiment, the following is selected: .

[0041] During the model building phase, the architecture of the deep neural network based on the multidimensional attention Transformer is constructed as follows: Figure 2 As shown, its core feature lies in the collaborative design of attention mechanisms along the time and covariate dimensions, and the innovative introduction of a time-series cue word injection module. First, the model maps historical wind power output and meteorological data to the feature space through covariate embedding and time embedding functions, respectively. Then, multiple prediction modules are stacked, each containing a covariate decoder and a time decoder. The former captures the coupling relationship between power output and meteorological elements along the covariate dimension, while the latter extracts the evolutionary pattern of the sequence along the time dimension. The two are processed alternately through residual connections and layer normalization to achieve deep modeling of multidimensional correlations. On this basis, the model introduces time-series cue words into the input layer. When the forecast wind speed exceeds the shutdown threshold, a cue signal is generated based on the mean and maximum values ​​of historical prediction errors and the zero-filling bits of the period to be predicted are replaced, thereby guiding the prediction distribution to adaptively shift towards extreme shutdown scenarios. Finally, the probability prediction results of multiple quantiles are obtained through the linear output layer.

[0042] The Transformer model, first proposed by Vaswani et al. in 2017, is a deep neural network architecture based on a self-attention mechanism. Originally designed for Natural Language Processing (NLP) tasks such as machine translation and text generation, it quickly replaced traditional Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks, becoming the mainstream model in NLP. A typical Transformer model consists of stacked encoder and decoder components. The encoder comprises multiple identical layers, each containing two sub-layers: a multi-head self-attention sub-layer and a feedforward neural network sub-layer. Each sub-layer is followed by residual connections and layer normalization. The decoder structure is similar to the encoder, but an additional encoder-decoder attention sub-layer is introduced above the self-attention sub-layer to focus on information from the encoder output. To preserve sequence order, the Transformer also introduces positional encoding to add positional signals to each input position. The general Transformer model still has limitations in power system time series forecasting tasks, especially wind power output probability forecasting, such as limitations in extracting multidimensional correlation features, insufficient ability to capture nonlinear features under extreme scenarios, and lack of an effective mechanism for integrating external prior information.

[0043] The construction process of a deep neural network includes the following five parts.

[0044] (1) Deep Neural Network Input Construction: To simultaneously utilize historical power output sequences and meteorological covariate sequences, the historical power output time series... copyQ This process yields copies of the multiquantile historical sequences; zero-filling is then applied to the unpredicted portion of the target sequence to obtain a zero-filled sequence aligned with the time dimension of the meteorological covariate sequence. ; Combine multiquantile historical series copies, zero-filled series, and meteorological covariate series Concatenating along the feature dimensions forms the input tensor of the deep neural network. ,in C This represents the number of meteorological covariates. This tensor simultaneously contains historical time-series information of the target sequence, meteorological covariate information, and zero-filling placeholders for the period to be predicted, providing a unified input representation for subsequent multidimensional attention Transformer calculations and time-series cue word injection. It indicates that it belongs to.

[0045] (2) Timing cue injection: In the zero-filling bits of the time period to be predicted, the function is calculated based on the constructed timing cue, and for each quantile... q and forecast wind speed Generate timing prompt words This is then added to the zero-padding sequence of the corresponding quantile to obtain the deep neural network input containing external prior guidance. When the time-series prompt word is zero, the deep neural network degenerates into a regular prediction mode. When a non-zero prompt word is injected, the output of the deep neural network tends to approach the guiding signal, realizing adaptive correction of the prediction distribution in extreme scenarios. To ensure the uniformity of the feature space, the time-series prompt words and the historical output time series are processed using the same embedding parameters.

[0046] (3) Multidimensional embedding function design: To handle the correlation features of different dimensions respectively, covariate embedding function and time embedding function are designed respectively.

[0047] Covariate Embedding Function : Input tensor The covariate dimension is reconstructed as Then, the last dimension is increased to the dimension of a deep neural network through a linear mapping. , obtain the input of the covariate decoder It is used to capture the coupling relationship between different quantiles and between meteorological covariates and power output.

[0048] Time embedding function By setting the Patch length and sliding step size , input tensor Divide into multiple patches along the time dimension to obtain ,in This represents the number of patches; then, through linear mapping, the last dimension is transformed into... , obtain the time decoder input It is used to extract the local evolution pattern of wind power output sequence in the time dimension.

[0049] (4) Construction of Multidimensional Attention Transformer Prediction Module: Each prediction module consists of a sequentially connected covariate decoder and a temporal decoder, both of which are decoders based on the Transformer structure. Different decoders are connected through residual connections and batch normalization modules to ensure stable training.

[0050] Covariate decoder: As input, through stacking The decoder layer of the layer calculates the attention weights between different quantiles and meteorological factors along the covariate dimension, and outputs the updated hidden state. Each decoder layer includes one multi-head attention mechanism computation and one feedforward neural network computation, connected via residual connections and layer normalization modules for stable training. This process enhances the ability of deep neural networks to express multivariate coupled features.

[0051] Time decoder: with For input, also through The layer decoder layer captures the dependencies between patches at different times in the time dimension and outputs the updated hidden state. Each decoder layer includes one multi-head attention mechanism computation and one feedforward neural network computation, which are connected through residual connections and layer normalization modules to ensure stable training. This process can effectively characterize the temporal evolution pattern of wind power output.

[0052] (5) Construction of a deep neural network based on prediction modules: Multiple prediction modules described above are stacked through residual connections and batch normalization to form a deep neural network. Specifically, the first... Output of each module As the first Input of each module, number of stacks Adjustments are made flexibly based on task complexity. Finally, after the last module, the hidden state is transformed into the target sequence through a linear output mapping layer. Predicted value at the quantile This completes the end-to-end mapping from input to probability prediction results. Deep neural networks, through a collaborative multi-dimensional attention Transformer, fully exploit the spatiotemporal correlation of wind power output and flexibly inject external priors via temporal cue words, significantly improving the probability prediction accuracy in both conventional and extreme scenarios. In this embodiment, [the following is selected] , , , , , .

[0053] During the model training phase, firstly, quantile loss is used as the objective function for model training. For a preset of Q quantiles to be predicted, the quantile loss function is defined as follows:

[0054]

[0055] In the formula: The overall loss; N This represents the number of training samples; For the first i The actual wind power output of each sample; For the first i Wind power output of a sample q Quantile predictions; for q The quantile loss calculation function, where For the characteristic function, when The value is 1 if the condition is met, and 0 otherwise. In this embodiment, the deep neural network predicts the 0.05, 0.5, and 0.95 quantiles of wind power output.

[0056] Then, the Adam (adaptive momentum estimation) optimizer is used to perform end-to-end gradient updates on the deep neural network parameters, and the initial learning rate is set. The value is 0.001, and the batch size is... B Number of training rounds E During training, an early stopping mechanism is used to monitor changes in quantile loss on the validation set. If continuous... E l If the loss on the validation set does not decrease, training is stopped to prevent overfitting. In this embodiment, the loss is taken as... , , , .

[0057] Finally, after training is complete, the various probabilistic prediction metrics of the deep neural network are evaluated on the test set. These metrics may include quantile loss, Winkler score, root mean square error, and mean absolute error. If the accuracy of each metric meets the requirements, the application phase can proceed; otherwise, hyperparameters such as the number of network layers and model dimensions need to be adjusted and the network retrained.

[0058] The model application process includes: (1) For the time to be predicted, obtain the length as L The historical wind power output sequence and length are S Future weather forecast data, based on forecast wind speed sequence The corresponding timing cue words are calculated using the constructed timing cue word calculation function. And inject it into the zero-padding bits of the time period to be predicted in the input of the deep neural network; (2) Inputting a complete deep neural network consisting of historical output sequences, meteorological covariate sequences, and injected time-series cue words into a pre-trained multidimensional Transformer model for forward inference, and obtaining future results through the collaborative computation of the covariate decoder and the time decoder. S At that moment Q The quantile prediction results are used as the wind power output probability prediction output to provide decision support for power system dispatch and operation, including information on uncertainties.

[0059] This embodiment designs a dedicated device that includes data preparation, temporal cue word construction, neural network construction, model training, and probabilistic prediction applications. It deeply integrates the multidimensional attention Transformer with the temporal cue word mechanism, achieving a unified approach of high-precision prediction in conventional scenarios and adaptive correction in extreme scenarios. This effectively solves the problem of prediction failure under extreme wind speeds in traditional methods, significantly improving the reliability and practicality of wind power output probability prediction. Existing methods such as LSTM, STGCN, and TSMixer, lacking the construction and injection of temporal cue words, struggle to capture wind power output characteristics under extreme wind speeds during training. In a comparison, the method proposed in this embodiment reduces the quantile loss by 3.6%, 2.5%, and 11.13%, respectively, compared to the aforementioned existing methods in day-ahead wind power probability prediction tasks.

[0060] Instead of using the deep neural network based on multidimensional attention Transformer proposed in this embodiment, other existing time series prediction models are selected as the basic architecture, such as STGCN (Spatiotemporal Graph Convolutional Network), LSTM (Long Short-Term Memory Neural Network), and TSMixer (Temporal Hybrid Prediction Model Based on Fully Connected Layers), and a probability output module is added to adapt to the probability prediction task. These existing models have been verified to have good performance in conventional time series prediction tasks. However, these solutions still have the following shortcomings: First, most of these existing models focus on feature extraction in the time dimension and lack the ability to specifically model the deep coupling relationship in the covariate dimension, making it difficult to fully explore the nonlinear interaction information between wind power output and meteorological factors; second, these existing models are all pure data-driven architectures and lack a mechanism to effectively integrate external prior information (such as wind turbine shutdown rules), still facing the dilemma of small sample learning under extreme wind speed scenarios, and unable to fundamentally solve the problem of abnormal prediction bias of "increased wind speed and sharp decrease in output"; in addition, the interpretability of these existing models is poor, and operators find it difficult to understand the prediction basis of the model under extreme operating conditions, and cannot correct the model output through external intervention.

[0061] Instead of using the specific method of injecting time-series cue words into the zero-padding positions of the predicted time period as in this embodiment, the time-series cue words are treated as a static feature and concatenated with other meteorological covariates and historical output data as a regular input dimension of the model. Simultaneously, the model automatically learns the mapping relationship between the cue words and the prediction results through training. This alternative approach also has significant drawbacks: First, treating the cue words as regular feature input means that the cue words and historical sequences have equal status in the model processing flow, failing to reflect their "guiding" role in extreme scenarios. The model may treat them as ordinary covariates, making it difficult to effectively distinguish between regular and extreme patterns. Second, this approach requires the cue words to be defined in all samples. However, in regular samples without extreme weather, the cue words lack physical meaning, and forced definitions will introduce noise, interfering with the model's learning of regular patterns.

[0062] In contrast, this embodiment injects the prompt word into the zero-filling position of the time period to be predicted and sets it to zero when there are no extreme scenarios, thus achieving a precise intervention effect of "guiding only when needed and not interfering in normal times".

[0063] The improvements and effects of the wind power output probability prediction method in this embodiment are mainly reflected in: (1) Temporal cue word mechanism. An innovative temporal cue word mechanism is proposed to introduce external prior information (such as wind turbine protection shutdown, extreme weather warnings, etc.) into deep neural networks. The core of this mechanism is: Prompt word design method: Based on the historical prediction error distribution, differentiated time-series prompt words are constructed for different quantiles (such as median and lower bound quantile) to quantitatively characterize the output offset features under extreme scenarios; Prompt word injection method: Utilize the zero-padding bits of the time period to be predicted in the input of the deep neural network to directly inject the time-series prompt words as external guiding signals, and process them with the same embedding parameters as the historical observation sequence to ensure the uniformity of the feature space.

[0064] This mechanism transforms the physical prior of wind turbine shutdown into a quantified numerical guidance signal, providing an interpretable and interventionist external knowledge injection path for deep neural networks. This enables deep neural networks to adaptively adjust the prediction distribution under extreme conditions without updating parameters, effectively solving the prediction offset problem of traditional methods in extreme wind speed scenarios.

[0065] (2) Deep Neural Network Structure with Collaborative Multidimensional Transformer. A deep neural network structure with clear physical meaning was constructed for wind power output probability prediction. The core of this network lies in the collaborative design of a time-dimensional attention mechanism and a covariate-dimensional attention mechanism, which are used to extract the autocorrelation features of the wind power output sequence over time and the deep coupling features with meteorological covariates, respectively. The two attention modules are fused in a cascaded manner, realizing the full mining of multidimensional spatiotemporal correlation information of new energy sequences, fundamentally improving the accuracy of basic probability prediction, unlike the traditional general Transformer model that only focuses on a single-dimensional attention.

[0066] (3) A method for predicting wind power output probability in extreme scenarios based on time-series cue words. This method provides a rapid calculation method that directly transforms "historical output sequence, meteorological covariate sequence, and external prior information" into "future output multi-quantile probability prediction results." Using the aforementioned collaborative multidimensional attention Transformer deep neural network as its core, combined with a time-series cue word mechanism, it unifies the complex high-dimensional nonlinear time-series modeling problem and the extreme scenario few-sample learning problem into a single, efficient feedforward neural network inference. This significantly improves the prediction reliability in extreme scenarios while maintaining the prediction accuracy for conventional scenarios.

[0067] This invention also proposes a wind power output probability prediction system based on temporal cue words and multidimensional attention, used to implement the aforementioned wind power output probability prediction method. The wind power output probability prediction system includes: The data preparation module is used to acquire historical power output sequences of wind farms and meteorological covariate forecast data for corresponding time periods, including wind speed and direction at different altitudes. It performs missing value imputation and normalization preprocessing on the raw data, according to a preset historical window length. L and prediction window length S Slide window slicing is used to construct input-output sample pairs, which are then divided into training, validation, and test sets in chronological order. The timing cue word construction module is used to train a basic timing prediction model based on training set data without introducing timing cue words, statistically analyze the prediction error distribution of the model under different wind speeds, and determine the shutdown wind speed threshold. And calculate the mean of the prediction error within each wind speed range. and maximum value We constructed a time-series cue word calculation function to provide external prior guidance for predictions in subsequent extreme scenarios; The neural network building module is used to build deep neural networks based on multidimensional Transformers. This module specifically includes: Model input building blocks: replicating historical output time series The input tensor is then filled with zeros during the period to be predicted and concatenated with the meteorological covariate sequence to obtain a unified input tensor. The timing cue injection unit injects timing cue words of the corresponding quantiles into the zero-filling positions of the time period to be predicted, based on the constructed cue word function. This enables guidance from external prior information. Multidimensional Embedded Function Unit: Contains covariate embedded functions and time embedding function The input is mapped to the feature spaces of the covariate dimension and the time dimension, respectively. Multidimensional Transformer prediction module unit: It consists of a cascaded covariate decoder and a temporal decoder, which extract deep correlation features in the covariate dimension and the temporal dimension through a multi-head attention mechanism, respectively; Model stacking unit: Multiple prediction modules are stacked through residual connections and batch normalization, and finally the future prediction is obtained through a linear output layer. S Moment Q Quantile prediction results.

[0068] The model training module is used to train the constructed multidimensional Transformer model end-to-end using quantile loss as the objective function and the Adam optimizer. It sets the initial learning rate, batch size, and number of training epochs, and obtains the optimal model parameters by monitoring the early stopping mechanism through the validation set.

[0069] The probabilistic prediction application module is used to load the trained model, receive historical power output sequences and future weather forecast data for the time to be predicted, and then determine the power output based on the predicted wind speed sequence and the shutdown threshold. The comparison results are used to generate time-series cue words corresponding to the quantiles using the cue word function and inject them into the model input. Then, forward inference calculations are performed to output the future... S Moment Q The quantile prediction results are used as the probability prediction values ​​for wind power output, providing uncertainty quantification support for power system dispatch and operation.

[0070] This invention also discloses a computer device, including a processor and a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by the processor, it implements the aforementioned wind power output probability prediction method.

[0071] This invention also discloses a computer-readable storage medium storing a computer program thereon, which, when executed, implements the aforementioned wind power output probability prediction method.

[0072] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art should understand that the present invention includes, but is not limited to, the content described in the above specific embodiments. Any modifications that do not depart from the functional and structural principles of the present invention will be included within the scope of the claims.

Claims

1. A wind power output probability prediction method based on temporal cue words and multidimensional attention, characterized in that: include: S1. Obtain the historical power output time series of the wind farm and the meteorological covariate forecast data for the corresponding time period, construct input-output sample pairs, and divide them into training set, validation set and test set according to time order; S2. Train the basic model based on the training set data, statistically analyze the distribution of prediction error of the basic model under different wind speeds and determine the shutdown wind speed threshold, calculate the mean and maximum value of prediction error in each wind speed interval for samples exceeding the shutdown wind speed threshold, and construct a time-series prompt word calculation function that associates the wind speed forecast value with the mean and maximum value of the prediction error. S3. Construct a deep neural network, which includes a covariate embedding function and a temporal embedding function, respectively mapping the input sequence to the feature spaces of the covariate dimension and the temporal dimension. Multidimensional correlation features are extracted through a cascaded covariate decoder and temporal decoder based on the Transformer structure. At the same time, zero-padding bits are set in the input of the deep neural network for the time period to be predicted to inject time-series cue words. S4. Perform end-to-end training on the deep neural network. During the training process, enable the deep neural network to learn the mapping relationship from historical output and meteorological covariates to the quantiles of future output. Adjust the hyperparameters through the validation set until the prediction error of the deep neural network reaches the expected level. S5. Input the acquired historical power output sequence, future weather forecast data, and time-series cue words calculated based on the forecast wind speed and time-series cue words into the trained deep neural network for forward inference to obtain the quantile prediction results for future times, which are used as the wind power output probability prediction output.

2. The wind power output probability prediction method based on temporal cue words and multidimensional attention as described in claim 1, characterized in that: In S1, the meteorological covariate forecast data includes wind speed forecast values ​​at different altitudes; In S1, according to the preset historical time window length L and prediction window length S Perform sliding window slicing; preprocess the historical power output time series of the wind farm before performing sliding window slicing. In S1, the input of each sample contains a length of L The historical output sequence and length are L + S The meteorological covariate sequence is output as the corresponding S Predicted output sequence at time step.

3. The wind power output probability prediction method based on temporal cue words and multidimensional attention according to claim 1, characterized in that: In S2, the basic model is a multilayer perceptron; In S2, without introducing time-series cue words, a base model is trained on the training set to obtain a baseline prediction model, which is then used to predict the wind speed forecast value for each training set sample. Construct a prediction error with wind power output prediction error. Compared with wind speed forecast values The jointly distributed dataset; In S2, the wind speed forecast values ​​in the training set are statistically analyzed. The historical distribution of its first The wind speed value corresponding to the quantile is used as the shutdown wind speed threshold. ; For samples with wind speeds exceeding this threshold, the mean of the prediction error within each wind speed interval is further calculated. and maximum value , respectively, as the median of the wind power output to be predicted and the lower bound quantile of the wind power output to be predicted The baseline correction amount; In S2, the process of constructing the timing cue word calculation function includes: when the wind speed forecast value... Time, timing prompt words ;when At that time, for the time sequence cue word corresponding to the median For the timing cue words corresponding to the lower bound quantile ,in This is an adjustable hyperparameter used to control the magnitude of timing cue word correction.

4. The wind power output probability prediction method based on temporal cue words and multidimensional attention according to claim 1, characterized in that: In S3, the historical output time series is copied several times to obtain a multiquantile historical sequence copy; the unpredictable portion of the target sequence is padded with zeros to obtain a zero-padded sequence aligned with the time dimension of the meteorological covariate sequence; the multiquantile historical sequence copy, the zero-padded sequence, and the meteorological covariate sequence are concatenated along the feature dimension to form the input of the deep neural network. .

5. The wind power output probability prediction method based on temporal cue words and multidimensional attention according to claim 4, characterized in that: In S3, within the zero-filling bits of the time period to be predicted, a calculation function is performed based on the constructed time-series cue words for each quantile. q and forecast wind speed Generate timing prompt words This is then added to the zero-padded sequence of the corresponding quantile to obtain the input containing external prior guidance. ; When the time-order cue word is zero, the deep neural network degenerates into the regular prediction mode. When a non-zero cue word is injected, the output of the deep neural network tends to approach the guiding signal, thus achieving adaptive correction of the prediction distribution in extreme scenarios. The timing prompts and historical output time series are processed using the same embedding parameters.

6. The wind power output probability prediction method based on temporal cue words and multidimensional attention according to claim 5, characterized in that: In S3, the covariate embedding function: for the input The covariate dimension is reconstructed, and then the last dimension is increased to the deep neural network dimension through linear mapping. The input to the covariate decoder is obtained. It is used to capture the coupling relationship between different quantiles and between meteorological covariates and power output; In S3, the time embedding function: by setting the patch length and sliding step size , will input Divide the data into multiple patches along the time dimension; then transform the last dimension using a linear mapping. , obtain the time decoder input It is used to extract the local evolution pattern of wind power output sequence in the time dimension.

7. The wind power output probability prediction method based on temporal cue words and multidimensional attention according to claim 6, characterized in that: In S3, each prediction module of the deep neural network consists of a covariate decoder and a temporal decoder connected in sequence, both of which are decoders based on the Transformer structure. For covariate decoders: As input, through stacking The decoder layer of the layer calculates the attention weights between different quantiles and meteorological factors along the covariate dimension, and outputs the updated hidden state. Each decoder layer contains one multi-head attention mechanism computation and one feedforward neural network computation, and is connected through residual connections and layer normalization modules to stabilize training. For the time decoder: For input, also through The decoder layer of the layer captures the dependencies between patches at different times in the time dimension and outputs the updated hidden state. Each decoder layer contains one multi-head attention mechanism computation and one feedforward neural network computation, and is connected through residual connections and layer normalization modules to stabilize training.

8. The wind power output probability prediction method based on temporal cue words and multidimensional attention according to claim 7, characterized in that: In S3, multiple prediction modules are stacked through residual connections and batch normalization to form a deep network: Output of each module As the first Input of each module, number of stacks Adjusted according to task complexity, after the last module, the hidden state is transformed into the target sequence through a linear output mapping layer. Predicted value at the quantile This completes the end-to-end mapping from input to probability prediction results.

9. The wind power output probability prediction method based on temporal cue words and multidimensional attention according to claim 1, characterized in that: In S4, the quantile loss is used as the objective function. For the preset Q quantiles to be predicted, the quantile loss function is defined as follows: In the formula: The overall loss; N This represents the number of training samples; For the first i The actual wind power output of each sample; For the first i Wind power output of a sample q Quantile predictions; for q The quantile loss calculation function, where For the characteristic function, when The value is 1 if the condition is met, otherwise it is 0. In S4, an adaptive momentum estimation optimizer is used to perform end-to-end gradient updates on the parameters of the deep neural network: setting the initial learning rate, batch size, and number of training rounds; During training, an early stopping mechanism is used to monitor changes in quantile loss on the validation set. If continuous... E l If the loss on the validation set does not decrease, training should be stopped to prevent overfitting. In S4, after training is completed, the various probability prediction metrics of the deep neural network are evaluated on the test set. If the accuracy of each metric meets the requirements, the application phase begins; otherwise, the hyperparameters are adjusted and the network is retrained.

10. The wind power output probability prediction method based on temporal cue words and multidimensional attention according to claim 1, characterized in that: In S5, for the time to be predicted, the length is obtained as follows: L The historical wind power output sequence and length are S Future weather forecast data, based on forecast wind speed sequence The corresponding timing cue words are calculated using the constructed timing cue word calculation function. This information is then injected into the zero-padding bits of the time period to be predicted in the input of the deep neural network. The complete deep neural network input, consisting of historical output sequences, meteorological covariate sequences, and injected time-series cue words, is fed into the trained deep neural network for forward inference. Through the collaborative computation of the covariate decoder and the time decoder, the future... S At that moment Q The quantile prediction results are used as the wind power output probability prediction output.

Images