Wind power prediction method and device based on ild and co-attention fusion
By employing learnable iterative decomposition and common attention fusion methods, the problems of insufficient collaborative optimization between decomposition and prediction targets and inadequate information utilization in wind power forecasting are solved, achieving higher accuracy in wind power forecasting, adapting to different wind fields and time periods, and improving the local fitting ability of multi-step forecasting.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTHEAST UNIV
- Filing Date
- 2026-03-24
- Publication Date
- 2026-06-09
AI Technical Summary
In existing wind power forecasting methods, the decomposition process and the forecasting objective are difficult to optimize in a coordinated manner. They fail to explicitly distinguish between endogenous and exogenous variables and future meteorological information, resulting in insufficient forecasting accuracy. Furthermore, the segment-level representation is too smooth and fails to effectively characterize the relationship between multi-scale time series and multi-source features.
We employ a learnable iterative decomposition of ILD and a co-attention fusion method. By adaptively decomposing endogenous and exogenous variables and future meteorological information, we preserve temporal details and achieve cross-modal alignment and interaction through joint modeling using a multi-scale decoder.
It significantly improves the accuracy of wind power prediction, adapts to different wind fields and time periods, makes full use of meteorological information, alleviates the oversmoothing problem caused by segment compression, enhances the multi-source time series characterization capability, and improves the local fitting capability of multi-step prediction.
Smart Images
Figure CN121903404B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of wind power prediction technology based on deep learning, and in particular to a wind power prediction method and apparatus based on the fusion of ILD and common attention. Background Technology
[0002] Wind power generation exhibits strong randomness and volatility, making accurate forecasting crucial for grid dispatch and absorption. While wind power forecasting methods have seen some development, they still face the following challenges:
[0003] (1) The decomposition method is difficult to optimize in conjunction with the prediction target:
[0004] Existing technologies mostly employ fixed moving averages, wavelet decomposition, or variational mode decomposition (VMD) to decompose power sequences. These methods have two main drawbacks: First, the decomposition process is decoupled from the subsequent prediction model; decomposition parameters (such as the number of modes, wavelet basis and layers, and moving average window in VMD) cannot participate in end-to-end training and cannot be automatically adjusted based on prediction loss. Second, the quality of the decomposition results often depends on manually set or heuristic criteria (such as envelope entropy and correlation coefficient), making it difficult to obtain the optimal decomposition directly consistent with the multi-step prediction objective. Furthermore, the optimal decomposition form varies for different wind fields and time periods, and fixed or prior decompositions lack adaptability to non-stationarity and have limited generalization ability.
[0005] (2) Endogenous and exogenous variables are not explicitly distinguished from and integrated with future numerical weather prediction (NWP):
[0006] From a physical perspective, wind power forecasting involves three types of information: endogenous variables (the power sequence itself, reflecting historical output and inertia), exogenous variables (historical weather conditions, such as wind speed, wind direction, and temperature, which have synchronous or lagging correlations with power), and future NWP (future weather conditions, directly constraining future power). Existing methods simply concatenate the above information in the feature dimension and input it into the model. This approach neither explicitly distinguishes the different roles of the three in causal relationships and interpretability, nor performs cross-modal alignment and interactive modeling of the historical segment representation and the future NWP segment representation. This results in insufficient utilization of future weather information and hinders understanding the model's decision-making from the physical chain of historical power—historical weather—future weather.
[0007] (3) Segment-level representation is too smooth: After the Patch-based Transformer compresses the long sequence into segment-level representation, it is easy to lose point-level temporal details, resulting in overly smooth prediction curves and insufficient local fitting ability of multi-step prediction.
[0008] (4) Time and channel dependencies are not jointly modeled: If segment-level time dependencies and dependencies between multiple channels such as power / exogenous are handled separately, it is difficult to characterize the relationship between multi-scale time series and multi-source features, which affects the prediction accuracy. Summary of the Invention
[0009] To address the shortcomings of existing technologies, this invention provides a wind power prediction method and apparatus based on ILD and common attention fusion, which solves the technical problems of poor prediction accuracy caused by the difficulty of co-optimizing the decomposition method with the prediction target, insufficient utilization of future meteorological information, and failure to strengthen the relationship between multi-scale time series and multi-source features.
[0010] The technical solution adopted in this invention is as follows:
[0011] This invention provides a method based on ILD and co-attention fusion, comprising:
[0012] Obtain the historical multivariate sequence and future NWP sequence of the unit; preprocess the historical multivariate sequence and split it into a power sequence and an exogenous feature sequence;
[0013] The power sequence is subjected to learnable iterative decomposition to obtain the decomposed power components, and the decomposition and reconstruction loss is calculated to assist training.
[0014] The decomposed power components, the exogenous feature sequence, and the future NWP sequence are represented by a ResBiLSTM module to obtain corresponding point-level representations.
[0015] The last part of the power point level representation and the exogenous point level representation H Step by step, add the initial NWP point-level representations together in the time dimension. H Step by step, point-level temporal features are obtained; H To predict the number of steps, the point-level temporal features are input into the point-level detail extraction module, and the output is directly input into the temporal detail branch of the multi-scale decoder for decoding.
[0016] The power point-level representation and the exogenous point-level representation are respectively patched and stacked in the channel dimension to form a multi-channel segment representation; the future NWP point-level representation is patched to obtain a future NWP segment representation with the same number of segments as the multi-channel segment representation.
[0017] The multi-channel segment representation is independently encoded by channel using autoregressive attention, and then updated multi-channel segment representation is obtained through two-stage attention.
[0018] The updated multi-channel segment representation is fused into a historical segment representation, and the historical segment representation is fused with the future NWP segment representation through bidirectional cross-attention and gating to obtain a fused segment representation.
[0019] Based on the fused segment representation, coarse and fine predictions are obtained through the cross-attention of the learnable query of the coarse-scale branch and fine-scale branch of the multi-scale decoder and the fused segment representation. These predictions are then weighted and summed with the output based on the temporal detail branch, and finally added to the power value of the last moment in the power sequence in the time dimension broadcast to obtain the future multi-step power prediction results.
[0020] The preferred technical solution is:
[0021] The execution module for learnable iterative decomposition includes multiple learnable decomposition layers, including learnable one-dimensional convolutional kernels, Softmax normalization layers, and reflection-filled one-dimensional convolutional layers.
[0022] Learnable iterative decomposition, including:
[0023] The power sequence is used as the input to the first learning decomposition layer;
[0024] Each learning decomposition layer, after reflection filling of the current input, performs one-dimensional convolution using a learnable one-dimensional convolution kernel normalized by the Softmax function to obtain the trend component of the layer, and then subtracts the trend component from the current input to obtain the seasonal component.
[0025] The seasonal components of the next and previous layers are used as inputs to continue iterating. After completing the set number of iterations, the trend components of each layer are summed to obtain the total trend. The total trend and the seasonal components of each layer are concatenated in the channel dimension to obtain the decomposed component matrix. The reconstruction loss is calculated based on the sum of the total trend and the last layer of seasonal components and the mean square error of the power sequence, which is used to assist training.
[0026] The trend component represents a low-frequency, slowly varying component, and the seasonal component represents a high-frequency, fluctuating component.
[0027] The process of fusing the historical segment representation and the future NWP segment representation through bidirectional cross-attention and gating to obtain the fused segment representation includes:
[0028] A bidirectional cross-attention layer is used, with the historical segment representation as the query and the future NWP segment representation as the key and value, respectively, and with the future NWP segment representation as the query and the historical segment representation as the key and value, to obtain the updated historical segment representation and future NWP segment representation;
[0029] The updated historical segment representation and the future NWP segment representation are concatenated in the channel dimension and then passed through a linear layer and a Sigmoid function to obtain a gate vector. The updated historical segment representation and the future NWP segment representation are then weighted and summed according to the gate vector to obtain an intermediate fused representation.
[0030] The intermediate fusion representation is normalized by the layer and fed forward network layer, and the fusion segment representation is obtained through residual connection.
[0031] The exogenous feature sequence includes historical NWP sequences that are contemporaneous with the historical window.
[0032] The updated multi-channel segment representation obtained through two-stage attention includes:
[0033] First, a bidirectional attention sequence is captured between the multi-channel segment representation and the learnable router. Then, self-attention and feedforward are performed on the channel dimension to restore the shape and obtain the output.
[0034] The coarse-scale branch is represented by the fusion segment as the key and value, and the learnable coarse-scale query is the query. After cross-attention and linear mapping, it is interpolated to the prediction step size to obtain the coarse-scale prediction sequence.
[0035] The fine-scale branch is represented by the fusion segment as a key and value, and the learnable fine-scale query is the query. Through cross attention and linear mapping, a fine-scale prediction sequence is obtained.
[0036] The temporal detail branch uses the point-level temporal detail features output by the point-level detail extraction module as keys and values, and the learnable detail query as the query. Through cross attention, one-dimensional convolution, and learnable scalar, the detail correction amount is obtained.
[0037] The learnable coarse-scale query represents the query proposed by the decoder to the fusion segment representation at a coarse time step; the learnable fine-scale query represents the query proposed by the decoder to the fusion segment representation for each prediction step; and the learnable detail query represents the query proposed by the decoder to point-level temporal detail features for each prediction step.
[0038] The autoregressive attention encoding includes:
[0039] The multi-channel segment representation is subjected to a causal multi-head self-attention mechanism to capture temporal dependencies independently of each channel, and a nonlinear feature transformation is performed through a feedforward network.
[0040] The present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the wind power prediction method.
[0041] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the wind power prediction method.
[0042] The present invention also provides a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the wind power prediction method.
[0043] The technical solution of the present invention can achieve at least some of the following beneficial effects:
[0044] This invention, based on adaptive learnable decomposition in synergy with the prediction target, explicitly distinguishes and fuses endogenous / exogenous and future NWPs, fully integrates historical and meteorological information, preserves temporal details, and jointly models time and channel dependencies. In wind field comparison experiments, its prediction accuracy surpasses that of many existing methods, significantly improving prediction accuracy. Specifically, it has the following advantages:
[0045] Learnable iterative decomposition: The decomposition kernel weights are learned from the data, adapting to different wind fields and time periods, which is superior to fixed moving average or wavelet decomposition; the multi-scale kernel takes into account both trends and multiple periods, and the reconstruction loss improves the interpretability of the decomposition and the stability of training.
[0046] Cross-attention fusion: The historical segment representation and the future NWP segment representation are fused through bidirectional cross-attention and gating to achieve cross-modal alignment, making full use of meteorological information, which is superior to simple splicing or unidirectional cross-attention.
[0047] Temporal detail injection: Preserve point-level proximal information that has not been segmented and explicitly model it through independent branches at the decoding end, thereby reducing the oversmoothing caused by patching and improving the local fitting ability of multi-step prediction.
[0048] Two-stage attention: Cross-Time and Cross-Dimension jointly model time-dependent relationships and power / exogenous channel relationships, enhancing the ability to represent multi-source time series.
[0049] Multi-scale decoding: Coarse, fine, and point-level detail decoding in parallel, taking into account both global trends and local details, achieving simplicity, ease of training, and high prediction accuracy.
[0050] Through improvements and collaboration among various modules, the overall performance of the model was optimized. Experiments show that the present invention performs better than existing models.
[0051] Other features and advantages of the invention will be set forth in the following description or may be learned by practicing the invention. Attached Figure Description
[0052] Figure 1 This is an overall flowchart of the prediction method according to an embodiment of the present invention.
[0053] Figure 2 This is a schematic diagram of the Learnable Iterative Decomposition (ILD) module in an embodiment of the present invention.
[0054] Figure 3 This is a schematic diagram of the common attention fusion module in an embodiment of the present invention.
[0055] Figure 4 This is a schematic diagram of the multi-scale decoder structure according to an embodiment of the present invention.
[0056] Figure 5 This is a comparative structure of the average MSE plots for each prediction step using different models for a specific target wind field.
[0057] Figure 6 This is a graph showing the prediction results of typical power output downslope for a target wind farm over 16 consecutive steps, according to an embodiment of the present invention.
[0058] Figure 7 This is a graph showing the typical power output prediction results for a target wind farm over a continuous 16-step uphill slope, according to an embodiment of the present invention.
[0059] Figure 8 This is a graph showing the prediction results of a typical gradual change in power output for a target wind farm over 16 consecutive steps, according to an embodiment of the present invention. Detailed Implementation
[0060] The specific embodiments of the present invention are described below with reference to the accompanying drawings.
[0061] This application provides a wind power prediction method based on ILD (Iterative Learnable Decomposition) and co-attention fusion. The wind power prediction method is formed by training and applying the wind power prediction model DeCoFormer (Decomposition Co-attention Transformer).
[0062] See Figure 1 Based on the architecture and training of the wind power prediction model DeCoFormer, the workflow of the prediction method in this embodiment includes:
[0063] S1. Obtain the historical multivariate sequence and future NWP sequence of the unit; preprocess the historical multivariate sequence and split it into a power sequence and an exogenous feature sequence.
[0064] In this embodiment, the historical multivariate sequence includes a power sequence and exogenous feature sequences (wind speed, wind direction, temperature, etc.). Preferably, the exogenous feature sequences include historical NWP sequences contemporaneous with the historical window. The preprocessing includes standardization and wind direction sine / cosine coding.
[0065] S2. Perform learnable iterative decomposition on the power sequence to obtain the decomposed power components, and calculate the decomposition and reconstruction loss to assist training.
[0066] In this embodiment, the execution module for learnable iterative decomposition is described below. Figure 2As shown, it includes multiple learnable decomposition layers, each of which includes a learnable one-dimensional convolutional kernel, a softmax normalization layer, and a reflection-filled one-dimensional convolutional layer. Based on this, the learnable iterative decomposition includes:
[0067] The power sequence As input to the first learning decomposition layer; For batch size, The length of the historical sequence;
[0068] Each learning decomposition layer fills the current input with reflection, then performs one-dimensional convolution using a learnable one-dimensional convolution kernel normalized by the Softmax function to obtain the trend component of that layer. The current input is then subtracted from the trend component to obtain the seasonal component.
[0069] The seasonal components of the next and previous layers are used as inputs to continue iterating. After completing the set number of iterations, the trend components of each layer are summed to obtain the total trend. The total trend and the seasonal components of each layer are concatenated in the channel dimension to obtain the decomposed component matrix. The reconstruction loss is calculated based on the sum of the total trend and the last layer of seasonal components and the mean square error of the power sequence, which is used to assist training.
[0070] Wherein, the trend component represents a low-frequency, slowly varying component, and the seasonal component represents a high-frequency, fluctuating component. i ( Trend components of the output of the learnable decomposition layer Seasonal Quantity for:
[0071]
[0072] In the formula, , Learnable one-dimensional convolution kernels , For the length of the core;
[0073] General trend Seasonal proportions of each layer The last dimension is concatenated to form the decomposition component matrix. ,in The auxiliary reconstruction loss is:
[0074]
[0075] Predicting losses For the model output, multi-step power prediction With real future power The mean square error (MSE) is:
[0076]
[0077] Total loss during training is ,in It is a constant, such as 0.05.
[0078] S3. The decomposed power components, the exogenous feature sequence, and the future NWP sequence are processed by the ResBiLSTM module to obtain the corresponding point-level representations.
[0079] The ResBiLSTM module used in this embodiment is a module based on Bidirectional Long Short-Term Memory (BiLSTM) network with residual connection method: the output of BiLSTM is linearly projected, layer normalized and dropped out and then added to the linear projection of the input to obtain the final output, so as to retain the input information, alleviate gradient flow and improve the ability to model sequence data.
[0080] In this embodiment, the decomposition component matrix output in step S2 Exogenous feature sequences Future NWP Series The input is processed by the corresponding feature extraction unit in the ResBiLSTM module. In each feature extraction unit, the input is passed through a Bidirectional Long Short-Term Memory (BiLSTM) network, linear projection, and layer normalization, and then a residual connection is made with the linear projection of the input to obtain the point-level representation of that path. Finally, the power point-level representation and the exogenous point-level representation are obtained. And future NWP point-level representation .
[0081] The formula for calculating the residual BiLSTM output is as follows:
[0082]
[0083] in, Map the bidirectional hidden dimension as , This is the residual projection from input to output.
[0084] S4. The final power point level representation and exogenous point level representation H Step by step, add the initial NWP point-level representations together in the time dimension. H Step 1, obtain point-level temporal features , H To predict the number of steps, To hide dimensions in the model; the point-level temporal features are input into the Temporal Detail Injection (TDI) module, and the output is directly input into the temporal detail branch of the multi-scale decoder.Figure 1 Decode the output at the midpoint scale.
[0085] This step aims to concatenate the point-level representations obtained in S3 into a tensor and directly input this tensor into the temporal detail branch of the multi-scale decoder without segment-level compression, so as to preserve the near-endpoint information without segment compression.
[0086] S5. Patch embedding of the power point-level representation and exogenous point-level representation obtained in S3, and stacking them in the channel dimension to form a multi-channel segment representation; Patch embedding of the future NWP point-level representation obtained in S3 to obtain a future NWP segment representation with the same number of segments as the multi-channel segment representation.
[0087] This step aims to process the point-level representation obtained from S3 into a segment representation.
[0088] In this embodiment, the point-level representation sequence is divided into segments of fixed length. Cut into segment, number of segments Each segment, after being flattened, undergoes a linear layer and learnable position encoding to obtain a segment-level representation. On the historical side, the power point-level representation and the exogenous point-level representation are processed by patch embedding to obtain... The two are stacked in the channel dimension to form a multi-channel segment representation. On the future side, the point-level representation of future NWPs is also based on segment length. Perform patch embedding to obtain the future NWP segment representation. , The number of segments corresponding to the future NWP sequence, and Consistent with or according to actual settings. Here, Not with , Stacked as Multichannel tensors.
[0089] S6. Perform autoregressive attention encoding on the multi-channel segment representation independently for each channel, and then obtain the updated multi-channel segment representation through two-stage attention.
[0090] In this embodiment, the autoregressive attention encoding includes:
[0091] The multi-channel segment representation Causal multi-head self-attention is performed independently according to the channel. It captures temporal dependencies and performs nonlinear feature transformation through a feedforward network (LN).
[0092] Specifically, for ( For the number of channels, such as power / exogenous, flattened along the channel dimension. For each channel independently, perform causal MHA+FFN (Pre-LN) according to the above formula to obtain the processed tensor and restore its shape. The segment sequences of each channel are normalized and summed after being compared with residuals. The output within each channel is as follows: This information is used as input for the subsequent two stages of attention, and the calculation formula is as follows:
[0093]
[0094] In the formula, the feedforward network Two-layer linear mapping and GELU activation are used:
[0095] .
[0096] The standard calculation for multi-head attention MHA is as follows:
[0097]
[0098] In the formula, For the first The linear projection matrix of the size, For output projection, For each head, the key / query dimension. Causal multi-head attention. Add an extra upper triangle to the dot product similarity matrix. The mask is used to prevent future views from being viewed.
[0099] The Two-Stage Attention Layer (TSA) is an innovative attention mechanism architecture designed to enhance a model's understanding and execution capabilities for complex tasks through a hierarchical processing approach, particularly excelling in handling multivariate time series (MTS) data. The core idea of TSA is a hierarchical processing model that prioritizes time over variables. First, it independently models the time dimension, capturing the dependencies between different time periods within the same variable. Then, it aligns and interacts with variables to capture the dependencies between different variables. This hierarchical approach aligns with human cognitive patterns and is more consistent with real-world data structures.
[0100] In this embodiment, the updated multi-channel segment representation is obtained through two-stage attention, including: first, performing bidirectional attention (Cross-Time) between the multi-channel segment representation and the learnable router to capture the temporal relationship, and then performing self-attention (Cross-Dimension) and feedforward on the channel dimension to restore the shape and obtain the output.
[0101] Specifically, in the Cross-Time phase, the router vectors are first learned. For Query, Segment Sequence Multi-head attention update the router for key / value pairs, then use segment sequences. For the query and the updated router as the key / value pair, a multi-head attention update segment representation is performed. The calculation formula is as follows:
[0102]
[0103] In the Cross-Dimension phase, Self-attention and feedforward are performed on the channel dimension, and the TSA output is obtained after shape recovery. Let... ( The number of channels, usually First, rearrange it into a channel-priority shape for cross-channel attention:
[0104]
[0105] right In the channel dimension (length is) Perform multi-head self-attention and feedforward (Pre-LN) transformation on the )
[0106]
[0107] Then restore the result to a segment-level multichannel shape:
[0108]
[0109] This is the output of Cross-Dimension, i.e., the updated multi-channel segment representation.
[0110] S7. The updated multi-channel segment representation is fused into a historical segment representation through channels, and the historical segment representation and the future NWP segment representation are fused together through bidirectional cross-attention and gating to obtain a fused segment representation.
[0111] As a specific method, the updated multi-channel segment representation is fused into a historical segment representation through channel fusion, including: after splicing the updated multi-channel segment representation in the channel dimension, it is merged with the activation and LayerNorm through a linear layer to form a historical segment representation.
[0112] As a preferred method, see Figure 3 The execution module for bidirectional cross-attention and gating fusion includes a bidirectional cross-attention layer, a gating computation layer, and a feedforward network layer. The process of fusing the historical segment representation and the future NWP segment representation through bidirectional cross-attention and gating to obtain the fused segment representation includes:
[0113] S71. A bidirectional cross-attention layer is used, with each segment representing a historical period. For querying future NWP segments Represented as key and value, in the form of a future NWP segment. For querying, historical segment representation Using keys and values, we obtain the updated history segment representation. and future NWP segment The calculation formula is as follows:
[0114]
[0115]
[0116] In the formula, For LayerNorm, For the attention of multiple parties;
[0117] S72. The updated historical segment representation and the future NWP segment representation are concatenated in the channel dimension and then passed through a linear layer and a Sigmoid function to obtain a gated vector:
[0118]
[0119] in, It is Sigmoid;
[0120] S73. The updated historical segment representation and the future NWP segment representation are weighted and summed according to the gate vector to obtain the intermediate fused representation:
[0121]
[0122] S74. The intermediate fused representation is normalized and fed forward network layer, and then the fused segment representation is obtained through residual connection:
[0123] .
[0124] S8. Based on the fused segment representation, coarse and fine predictions are obtained through the cross-attention of the learnable queries of the coarse-scale and fine-scale branches of the multi-scale decoder and the fused segment representation. These predictions are then weighted and summed with the output based on the temporal detail branch, and finally added to the power value at the last moment in the power sequence broadcast in the time dimension to obtain the future multi-step power prediction results. The coarse and fine predictions, as well as the temporal detail prediction, can cover both global trends and local details.
[0125] The structure of the multi-scale decoder described in this embodiment is shown in [reference]. Figure 4 It includes coarse-scale branching, fine-scale branching, point-scale (temporal detail) branching, and three-way summation;
[0126] The multi-scale decoder fuses two types of input features: the first type is the fused segment representation. The first category is obtained by fusing segment-level encoding and co-attention; the second category is the point-level temporal features obtained in step S4. .
[0127] Specifically, the coarse-scale branch is represented by the fusion segment. Key-value, learnable coarse-scale query For the query, after cross-attention and linear mapping, the result is interpolated to the prediction step size to obtain a coarse-scale prediction sequence. .
[0128] Specifically, the learnable coarse-scale query represents the query proposed by the decoder to the fusion segment representation at a coarse time step. Each coarse-scale query corresponds to a coarse time step; this query is then compared with the fusion segment representation. The attention weights are obtained by calculating the similarity of the keys, and then... The values are weighted and summed to extract information related to the coarse step from the segment representation. A scalar prediction for the coarse step is obtained through linear mapping, and then linearly interpolated to the prediction step size. To obtain the coarse-scale prediction sequence .
[0129] The fine-scale branches are represented by the fusion segments. Key-value, learnable fine-scale query For the query, a fine-scale prediction sequence is obtained through cross-attention and linear mapping. .
[0130] In this context, the learnable fine-scale query representation on the decoding side represents the proposed query for each prediction step towards the fusion segment. There is a one-to-one correspondence between fine-scale queries and prediction steps; through each query pair... By performing cross-attention on the keys and values, we can directly obtain the fine-scale representation of each step and linearly map it to a scalar to obtain the fine-scale prediction sequence.
[0131] The temporal detail branch uses the point-level temporal detail features output by the point-level detail extraction module as keys and values, and the learnable detail query. For the query, cross-attention, one-dimensional convolution, and a learnable scalar are applied. , obtain detailed correction amount .
[0132] Here, the learnable detail query represents the query proposed by the decoder for each prediction step, targeting point-level temporal detail features. Each detail query corresponds one-to-one with a prediction step; it uses point-level temporal detail features... For keys and values, cross-attention is performed on each query to extract near-endpoint information, and the output is convolved with a learnable scalar. Get detailed correction amount The expression is as follows:
[0133]
[0134] The three queries mentioned above are all learnable parameters of the model, determined during model initialization and training, and are not calculated based on the current input sample. Specifically: during model initialization, the coarse-scale query, fine-scale query, and detail query are respectively defined by shape. , , storage( , , (See below) The query is initialized using a small-variance normal distribution; it is updated via backpropagation during the training phase. During forward computation, the stored query is copied in batch dimension... ( The number of queries in each branch serves as the input for the multi-head cross-attention query of the corresponding branch; the key and value of each branch are encoded by the output of that branch ( or )supply.
[0135] The three queries have defined ranges in shape and dimension, but no pre-defined numerical constraints; they are learned through training. Shape and dimension (consistent with the notation above): To predict the number of steps, Hidden dimensions for the model, (for batch size)
[0136] Learnable coarse-scale query: stored in shape as coarse time steps (For example hour ).
[0137] Learnable fine-scale query: stored in shape as .
[0138] Learnable details query: Stored in shape as .
[0139] Forward expansion is performed in batch dimension. Hidden dimensions of multi-headed attention and predicted step size Consistent.
[0140] The implementation does not impose pruning or normalization constraints on the values of each query; the initialization uses a small variance normal distribution (e.g., mean 0, variance 0). After training, the values are determined by gradient updates, specifically:
[0141]
[0142] The final prediction output of the multi-scale decoder in this embodiment is as follows:
[0143]
[0144] In the formula, The power value (shape) of the power sequence at the last moment. Broadcast to time dimension (Then added together), serving as the prediction baseline bias. The sum of the four terms element-wise yields the future... Step power pre .
[0145] As a specific method, the historical sequence length in this embodiment 192 is a good starting point. The future weather forecast NWP can be set at 192, representing the predicted steps. 16 is acceptable, segment length 16 is possible, number of segments The number of iterations for learnable iterative decomposition. The number of routers for two-stage attention can be 2, and the kernel length sequence can be [25, 13]; A value of 5 is acceptable; model hidden dimension 256 is a suitable value. During training, the total loss is a weighted sum of the pre-prediction loss and the decomposition / reconstruction loss (e.g., ...). The parameters are updated using gradient descent and backpropagation.
[0146] As a specific method, the training of the wind power prediction model DeCoFormer in this embodiment includes collecting relevant wind farm data and weather forecast data, performing data preprocessing and dataset partitioning (dividing into training set, validation set, and test set), and using the partitioned dataset to train the model. Figure 1 The model shown is trained, validated, and tested on a set to obtain its optimal parameters. The trained DeCoFormer can then be used to predict wind power output for the target task.
[0147] This embodiment also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the wind power prediction method.
[0148] This embodiment also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the wind power prediction method.
[0149] This embodiment also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the wind power prediction method.
[0150] The effectiveness of the prediction method in this embodiment is further verified through comparative experiments.
[0151] On a target wind field, the DeCoFormer prediction method of this embodiment was compared with the prediction methods of various existing models (BiLSTM, PatchTST, Informer, Crossformer, Leddam), and ablation experiments were conducted to verify the contribution of each module in the model of this embodiment. Specific evaluation metrics included MSE, MAE, RMSE, NMAE, NRMSE, and R. 2 Both the predicted and actual values are in units of MW, which are restored to the original units through inverse model transformation.
[0152] The prediction results obtained by the method in this embodiment can be found in [reference]. Figures 6 to 8 ,Depend on Figure 6 It can be seen that during the typical power output decline period of the target wind farm, the method of this embodiment can effectively predict the power in the next 16 steps, and the error is low in this scenario; Figure 7 It can be seen that during typical power output uphill periods, this method can accurately track the power increase trend and has good prediction results; from Figure 8 It can be seen that in typical scenarios of gradually changing (small fluctuations) power output, this method can well characterize subtle changes in power, and the prediction matches the actual value well. In summary, under different power output conditions such as typical downhill, uphill, and gradually changing power output, the method in this embodiment shows good prediction performance and low prediction error.
[0153] The comparison results between this embodiment model and various existing models can be found in [link to relevant documentation]. Figure 5 , as well as Tables 1 and 2.
[0154] Table 1 Comparison of different models for a target wind field
[0155]
[0156] Depend on Figure 5 As shown in Table 1, the method of this embodiment (DeCoFormer) achieves the following results in this wind farm: MSE, MAE, RMSE, NMAE, NRMSE, and R... 2 All are superior to other comparative models in the table; the NRMSE of the method in this embodiment is reduced to 0.1101, and R 2 Increased to 0.8714.
[0157] Table 2 Ablation Experiment Results
[0158]
[0159] As shown in Table 2, removing any of the following modules—ILD (Learnable Iterative Decomposition), CoAttn (Channel Merging and Co-Attention Fusion), TSA (Two-Stage Attention), and TDI (Point-Level Temporal Detail Input)—leads to a performance decrease, indicating that each module is beneficial to prediction.
[0160] It will be understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A wind power prediction method based on ILD and common attention fusion, characterized in that, include: Obtain the historical multivariate sequence and future NWP sequence of the unit; The historical multivariate sequence is preprocessed and split into a power sequence and an exogenous feature sequence; The power sequence is subjected to learnable iterative decomposition to obtain the decomposed power components, and the decomposition and reconstruction loss is calculated to assist training. The decomposed power components, the exogenous feature sequence, and the future NWP sequence are represented by a ResBiLSTM module to obtain corresponding point-level representations. The last part of the power point level representation and the exogenous point level representation H Step by step, add the initial NWP point-level representations together in the time dimension. H Step by step, point-level temporal features are obtained; H To predict the number of steps, the point-level temporal features are input into the point-level detail extraction module, and the output is directly input into the temporal detail branch of the multi-scale decoder for decoding. The power point-level representation and the exogenous point-level representation are respectively patched and stacked in the channel dimension to form a multi-channel segment representation; the future NWP point-level representation is patched to obtain a future NWP segment representation with the same number of segments as the multi-channel segment representation. The multi-channel segment representation is independently encoded by channel using autoregressive attention, and then updated multi-channel segment representation is obtained through two-stage attention. The updated multi-channel segment representation is fused into a historical segment representation, and the historical segment representation is fused with the future NWP segment representation through bidirectional cross-attention and gating to obtain a fused segment representation. Based on the fusion segment representation, coarse and fine predictions are obtained through the cross-attention of the learnable query of the coarse-scale branch and fine-scale branch of the multi-scale decoder and the fusion segment representation. These predictions are then weighted and summed with the output based on the temporal detail branch, and finally added to the power value of the last moment in the power sequence in the time dimension broadcast to obtain the future multi-step power prediction results. The coarse-scale branch is represented by the fusion segment as the key and value, and the learnable coarse-scale query is the query. After cross-attention and linear mapping, it is interpolated to the prediction step size to obtain the coarse-scale prediction sequence. The fine-scale branch is represented by the fusion segment as a key and value, and the learnable fine-scale query is the query. Through cross attention and linear mapping, a fine-scale prediction sequence is obtained. The temporal detail branch uses the point-level temporal detail features output by the point-level detail extraction module as keys and values, and the learnable detail query as the query. Through cross attention, one-dimensional convolution, and learnable scalar, the detail correction amount is obtained. The learnable coarse-scale query represents the query proposed by the decoder to the fusion segment representation at a coarse time step; the learnable fine-scale query represents the query proposed by the decoder to the fusion segment representation for each prediction step; and the learnable detail query represents the query proposed by the decoder to point-level temporal detail features for each prediction step.
2. The method according to claim 1, characterized in that, The execution module for learnable iterative decomposition includes multiple learnable decomposition layers, including learnable one-dimensional convolutional kernels, Softmax normalization layers, and reflection-filled one-dimensional convolutional layers. Learnable iterative decomposition, including: The power sequence is used as the input to the first learning decomposition layer; Each learning decomposition layer, after reflection filling of the current input, performs one-dimensional convolution using a learnable one-dimensional convolution kernel normalized by the Softmax function to obtain the trend component of the layer, and then subtracts the trend component from the current input to obtain the seasonal component. The seasonal components of the next and previous layers are used as inputs to continue iterating. After completing the set number of iterations, the trend components of each layer are summed to obtain the total trend. The total trend and the seasonal components of each layer are concatenated in the channel dimension to obtain the decomposed component matrix. The reconstruction loss is calculated based on the sum of the total trend and the last layer of seasonal components and the mean square error of the power sequence, which is used to assist training. The trend component represents a low-frequency, slowly varying component, and the seasonal component represents a high-frequency, fluctuating component.
3. The method according to claim 1, characterized in that, The process of fusing the historical segment representation and the future NWP segment representation through bidirectional cross-attention and gating to obtain the fused segment representation includes: A bidirectional cross-attention layer is used, with the historical segment representation as the query and the future NWP segment representation as the key and value, respectively, and with the future NWP segment representation as the query and the historical segment representation as the key and value, to obtain the updated historical segment representation and future NWP segment representation; The updated historical segment representation and the future NWP segment representation are concatenated in the channel dimension and then passed through a linear layer and a Sigmoid function to obtain a gate vector. The updated historical segment representation and the future NWP segment representation are then weighted and summed according to the gate vector to obtain an intermediate fused representation. The intermediate fusion representation is normalized by the layer and fed forward network layer, and the fusion segment representation is obtained through residual connection.
4. The method according to claim 1, characterized in that, The exogenous feature sequence includes historical NWP sequences that are contemporaneous with the historical window.
5. The method according to claim 1, characterized in that, The updated multi-channel segment representation obtained through two-stage attention includes: First, a bidirectional attention sequence is captured between the multi-channel segment representation and the learnable router. Then, self-attention and feedforward are performed on the channel dimension to restore the shape and obtain the output.
6. The method according to claim 1, characterized in that, The autoregressive attention encoding includes: The multi-channel segment representation is subjected to a causal multi-head self-attention mechanism to capture temporal dependencies independently of each channel, and a nonlinear feature transformation is performed through a feedforward network.
7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 6.