Small sample wind power prediction method fusing gradient collaboration and double alignment

Abstract

The invention relates to the technical field of wind power prediction, and particularly discloses a gradient synergy and double alignment fused small sample wind power prediction method, which comprises the following steps: designing a Fourier enhanced Transform shared feature extractor to extract general feature representation with periodic perception from wind power time sequence data of a multi-source domain and a target domain; then, constructing a hybrid domain adaptive module, and realizing implicit and explicit dual alignment of feature distribution of a source domain and a target domain through a plurality of adversarial domain classifiers arranged in parallel and multi-core maximum mean difference measurement; and finally, introducing a gradient projection algorithm, carrying out collaborative optimization on conflict gradients of the prediction task and the domain adaptation task in a back propagation process, and eliminating gradient conflicts in multi-task learning. The method effectively improves the feature extraction capability, domain adaptability and optimization stability of the model in a small sample scene, and remarkably improves the prediction precision and robustness in a cross-domain wind power prediction task.

Description

Technical Field

[0001] This invention relates to the field of wind power prediction technology, specifically a small-sample wind power prediction method that integrates gradient coordination and dual alignment. Background Technology

[0002] Wind energy, as a clean and renewable energy source, plays a vital role in the global energy transition. However, wind energy itself exhibits significant spatiotemporal nonstationarity and stochastic fluctuations, posing challenges to power grid operation and dispatch. Accurate wind power prediction (WPF) is a key means to improve grid reliability and economic dispatch. Data-driven methods, represented by deep learning, especially sequence modeling methods based on Transformers, have become one of the research hotspots and main methods in the field of wind power prediction due to their powerful representation capabilities and parallel computing advantages. However, the good performance of deep learning methods usually depends on a large amount of high-quality training data. In the scenario of newly built or data-scarce wind farms, insufficient samples can lead to model overfitting and a significant decrease in generalization ability, thereby affecting the accuracy of wind power prediction.

[0003] To alleviate the problem of small sample sizes, transfer learning (TL) has been introduced. It transfers knowledge learned from a data-rich source domain, such as historically operating existing wind farms, to a target domain, such as newly built wind farms. However, existing transfer strategies have several shortcomings: First, the common "pre-training-fine-tuning" paradigm is prone to overfitting and "catastrophic forgetting" when the target domain samples are extremely scarce. Moreover, this paradigm does not fundamentally solve the problem of data feature distribution drift between the source and target domains. Second, the standard Transformer has a spectral bias, meaning that the model tends to learn the low-frequency components of the signal first. Therefore, it is not able to capture the multi-scale periodicity, such as daily periodicity and seasonal periodicity, that exists simultaneously in the wind power sequence, which affects the prediction accuracy. In addition, existing multi-source domain adaptation (MDA) methods mostly rely on a single alignment strategy, which makes it difficult to achieve comprehensive and efficient alignment of different types of distribution differences.

[0004] Cross-domain wind power prediction is essentially a multi-task learning problem. It is necessary to optimize prediction performance and achieve inter-domain alignment. The currently commonly used simple loss weighting strategy ignores the interaction and potential conflict between gradients between tasks. When the gradient directions are opposite, the parameter updates will cancel each other out, resulting in unstable training and impairing the final performance.

[0005] Therefore, there is an urgent need to propose a new technical solution that can take into account multi-scale periodic capture, achieve full-domain alignment, and solve multi-task gradient conflicts through collaborative optimization, so as to improve the robustness and generalization ability of wind power prediction in small sample scenarios. Summary of the Invention

[0006] The purpose of this invention is to provide a small-sample wind power prediction method that integrates gradient coordination and dual alignment to solve the problems of spectral bias, insufficient multi-source domain alignment, and gradient conflict in multi-task learning of the standard Transformer in the above-mentioned background technology.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a small-sample wind power prediction method integrating gradient coordination and dual alignment, comprising the following steps: Step 1: Acquire wind power time series data for at least one source domain (S) and one target domain (T), and perform data preprocessing. The data includes power, wind speed, temperature, etc. Preprocessing includes using the sliding window method to construct supervised learning samples from the time series data into input sequence X and prediction target y, and standardizing the data. Step 2: Use a shared Fourier-enhanced Transformer feature extractor Gf to extract features from the data in all domains to obtain high-dimensional feature representations. ,like Figure 2 As shown, the extractor uses a Fourier-enhanced feedforward network (FAN) module, which takes the input features... The data is projected onto the periodic processing path through two parallel, non-weight-sharing linear layers. Aperiodic processing path ; ; ; in, The weight matrix for a linear layer with non-shared weights for a periodic processing path; This is the bias vector of the linear layer in the periodic processing path; The weight matrix of a linear layer with non-shared weights for aperiodic processing paths; This is the bias vector of the linear layer in the non-periodic processing path.

[0008] Through the aforementioned parallel linear projection operations, this invention achieves frequency domain decoupling of input features. Compared to traditional standard processing methods, this design can separate the periodic components implicit in wind power data, such as diurnal and seasonal variations, from non-periodic trend components, such as abrupt changes and random fluctuations. This overcomes the spectral bias defect of the standard Transformer model, which tends to favor low-frequency information, and significantly enhances the model's accuracy in capturing the complex time-varying characteristics of wind power.

[0009] Furthermore, the periodic processing path of the FAN module affects the features. Applying sin and cos activation functions, non-periodic processing paths are used to modify features. Apply the GeLU activation function; ; ; The outputs of the two paths are concatenated and then fused through an output linear layer to obtain the output of the FAN module: ; To overcome the spectral bias of the standard Transformer and improve the ability to perceive the periodicity of wind power sequences; Step 3: Represent the features Input a power regressor composed of a multilayer perceptron (MLP). In the process, the predicted power value is obtained. And calculate the regression loss. Mean squared error (MSE) loss is typically used: ; Where D represents the training dataset; Step 4: Construct a dual alignment domain adaptive module to align the feature distributions of the source and target domains. This module combines implicit and explicit alignment. (1) Implicit alignment based on adversarial training: establish an independent binary domain classifier for each source-target domain pair (Si,T). The feature extractor is improved by using a gradient inversion layer (GRL). Learning generates domain-indistinguishable features, while classifiers... Then, we should try our best to distinguish the sources of features and combat losses. Loss for all m independent adversarial tasks Mean: ; Among them, the Si-th domain classifier loss Using the binary cross-entropy definition: ; in, Represents the mathematical expectation. Indicates input sample Sampled from the first Data distribution of each source domain , Indicates input sample Data distribution sampled from the target domain ; The output of the feature extractor For the first The output probabilities of each domain classifier.

[0010] (2) Explicit alignment based on metric learning: Multi-kernel maximum mean difference (MK-MMD) is used as a non-parametric statistical metric to directly calculate and narrow the mean embedding distance of each source domain and target domain feature distribution in the regenerating kernel Hilbert space. MMD loss The maximum mean difference between the multi-kernel features of all m source and target domain feature distributions. Mean: ; Step 5: Apply the gradient collaborative optimization strategy (PCGrad) to the regression loss generated in Steps 3 and 4. (Main mission), combating losses and MMD loss (Auxiliary task) relative to shared feature extractor parameter The gradient is processed, such as Figure 3 As shown, this strategy resolves gradient conflicts between multiple tasks in the following way: (a) Calculate the gradient of the main task and auxiliary task gradient , .

[0011] (b) Conflict Detection: Determine whether the gradient of the auxiliary task conflicts with the gradient of the main task by using the vector dot product. If If so, it is determined to be a conflict.

[0012] (c) Gradient projection: If a conflict exists, the gradient of the conflicting auxiliary task will be projected. Orthogonal projection onto the gradient of the main task On the normal plane, we obtain: ; This operation eliminates the negative interference of auxiliary tasks on the main task, while the main task gradient... It remains unchanged.

[0013] (d) Synthetic gradient: ; Step 6: Based on the gradient synthesized in Step 5 Update the shared feature extractor parameters Meanwhile, according to step 3 Update power regressor The parameters, and according to step 4 Update all domain classifiers The parameters.

[0014] Compared with existing technologies, the beneficial effects of this invention are: the small-sample wind power prediction method that integrates gradient coordination and dual alignment: 1. Design of a Fourier-enhanced Transformer Shared Feature Network To enhance periodicity perception, the model is improved by modifying the feedforward network of the standard Transformer and introducing periodic activation functions (sin, cos), enabling it to have an endogenous periodicity perception bias. This allows the model to capture and decouple multi-scale periodic patterns in wind power sequences more efficiently, providing high-quality feature representations for prediction. 2. A dual alignment domain adaptation framework is constructed, integrating implicit alignment using adversarial methods and explicit alignment using multi-kernel maximum mean difference (MMD), to achieve comprehensive domain alignment. This framework considers both macroscopic separability and microscopic statistical properties, achieving more comprehensive and robust domain adaptation than single methods, and significantly enhancing the effectiveness of cross-domain knowledge transfer. 3. The gradient co-optimization mechanism PCGrad is introduced. By orthogonally projecting the gradients of conflicting auxiliary tasks (neighborhood alignment) during backpropagation, the negative interference of these gradients on the main task (power prediction) is eliminated, thus solving the gradient conflict problem in multi-task learning and ensuring the stability and efficiency of the training process. Attached Figure Description

[0015] Figure 1 This is a general framework diagram of the DAFT-PG invention; Figure 2 This is a schematic diagram of the Fourier Enhancement Transformer (FT) coding layer structure of the present invention; Figure 3 This is a schematic diagram of the gradient conflict detection and PCGrad processing between tasks in this invention; Figure 4 This is a t-SNE feature distribution diagram of each source domain and target domain before and after training according to the present invention; Figure 5 This is a comparison chart of the 15-minute wind power generation prediction trends of various models in this invention; Figure 6 This is a comparison chart of the 1-hour wind power generation prediction trends of various models in this invention; Figure 7 This is a comparison chart of the 2-hour wind power generation prediction trends for each model in this invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figures 1-7This invention provides a technical solution: a small-sample wind power prediction method that integrates gradient collaboration and dual alignment; Example 1 The actual wind energy data studied in this embodiment comes from the New Energy Laboratory (NREL), which includes four wind farms with different geographical locations and climate conditions. Three of them are used as source domains (S1, S2, S3), and one is used as target domain (T). All samples and features are used in the experiment.

[0018] Step 1: Data Acquisition and Division (1) Data acquisition: Data from source domains S1, S2, S3 and target domain T. Each dataset contains three key physical quantities: output power (MW), wind speed at 100 meters (m / s), and air temperature at 2 meters (K); (2) Data partitioning: The target domain T uses only a small amount of data (e.g., 7739 samples, approximately 26.8 days) for training, and another 2547 samples (approximately 8.8 days) are used as the validation set and the test set. The source domains S1, S2, and S3 use all the data (105071 samples each, approximately 365 days) as the training set. The sample information for each source domain and target domain is shown in Table 1 below.

[0019] Table 1: Sample Information for Each Source and Target Domain

[0020] Step 2: Data Preprocessing (1) Data normalization: To eliminate the influence of dimensions and ensure scale consistency, the source and target domain data are normalized. To strictly avoid future information leakage, the normalization parameter is only fitted on the union of all source and target domain training sets, and then all data are transformed using this scale; (2) Construction of supervised learning samples: The time series data is converted into supervised learning samples using a sliding window. The input sequence length is set to 48, and the single-point power value is predicted at the third time step in the future, i.e., 15 minutes in advance. In this case, the prediction performance is evaluated at 1 hour and 2 hours in advance.

[0021] Step 3: Fourier Enhanced Transformer Feature Extractor ( The core of this invention is a shared feature extractor. It is built on the Transformer encoder, but its standard feedforward network (FFN) is replaced by a Fourier enhanced feedforward network (FAN) to solve the spectral bias problem of the standard Transformer. The processing flow of the FAN module is as follows: (1) Feature decomposition: decompose the input features The data is projected onto a periodic processing path through two parallel linear layers. Aperiodic processing path : ; ; (2) Parallel activation: for periodic parts Applying sin and cos activation functions; for the aperiodic part Applying the GeLU activation function: ; ; (3) Information fusion: The activated features of the two parts are concatenated and fused through an output linear layer: ; (4) Prediction output: Feature extractor Output features Feed into the power regressor (A multilayer perceptron (MLP)) obtains the predicted value. The loss function for this basic prediction task uses the mean squared error (MSE): ; Step 4: Construction of the hybrid domain dual alignment module. To solve the "distribution drift" between the source domain and the target domain, this invention adopts a dual alignment strategy that integrates adversarial implicit and metric explicit methods. (1) Implicit alignment is learned based on adversarial domain invariant feature learning, using gradient inversion layer (GRL) and a set of independent domain discriminators. For each source-target pair ( (T) Establish a dedicated binary domain classifier Its adversarial loss is defined using the binary cross-entropy definition: ; in, Represents the mathematical expectation. Indicates input sample Sampled from the first Data distribution of each source domain , Indicates input sample Data distribution sampled from the target domain ; The output of the feature extractor For the first The output probabilities of each domain classifier.

[0022] Total combat losses The average loss of all m independent adversarial tasks: ; (2) Explicit distribution alignment based on MMD metric: To supplement the shortcomings of adversarial training which only focuses on macroscopic separability, multi-kernel maximum mean difference (MK-MMD) is introduced to reduce the distribution gap in statistical properties. MMD loss The calculation is as follows: ; The MMD kernel function uses a set of Gaussian kernels with different bandwidths. =[0.01,0.1,1.0,10.0,100.0]; Step 5: Co-optimization of gradients (PCGrad) and model training. This invention is essentially a multi-task learning problem that requires simultaneous optimization. (Main Task) and (Auxiliary Task). To prevent gradient conflicts between tasks, i.e., negative dot products of gradient vectors, this invention introduces the gradient projection (PCGrad) algorithm. This strategy employs an asymmetric approach, i.e., the gradient of the main task... Always remain constant, when the gradient of the auxiliary task ( or )and When a conflict occurs The gradient of the auxiliary task is orthogonally projected onto the normal plane of the gradient of the main task to eliminate its conflicting components. ; in, The gradient for the auxiliary task after projection correction. The gradient vector of the main task Norm; finally, the synthesized gradient is used. Update the shared feature extractor The parameters.

[0023] Model hyperparameter configuration: The optimal hyperparameter configuration obtained through the Bayesian optimization framework (Optuna) is shown in Table 2; Table 2: Model Parameter Settings

[0024] Step 6: Performance Evaluation This invention compared the following baseline models: SVR, LSTM, Transformer, FEDformer, PatchTST, TimeMixer, TimeKAN, and two transfer learning benchmarks: Target-Only (training only in the target domain) and Fine-Tuning (fine-tuning after pre-training). Three internationally recognized metrics were used to evaluate model performance: Mean Absolute Error (MAE): ; Root Mean Square Error (RMSE): ; Fit coefficient (R²): ; The experiment tested the prediction performance of various prediction methods and models in three scenarios: 15 minutes, 1 hour and 2 hours in advance. The results are shown in Table 3. Table 3: Performance comparison of each model on the target domain test set:

[0025] The following conclusions can be drawn from Table 3: The DAFT-PG model proposed in this invention achieves the best performance in all prediction durations and evaluation metrics; For example, in the 15-minute prediction task, compared with the second-best performing Fine-Tuning model, DAFT-PG reduced MAE and RMSE by 10.3% and 6.3%, respectively. In the 2-hour prediction task, the performance advantage of DAFT-PG remained robust, with MAE and RMSE reduced by 9.8% and 7.5%, respectively, compared with Fine-Tuning. This fully demonstrates the advancement and robustness of the proposed method in small-sample cross-domain wind power prediction tasks.

[0026] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A small sample wind power prediction method fusing gradient synergy and double alignment, characterized in that, The method comprises the following steps: Step 1: Obtain wind power time series data of at least one source domain (S) and one target domain (T), the data comprising power, wind speed and air temperature characteristics, and perform sliding window sampling and normalization preprocessing on the data; Step 2: using a shared Fourier-augmented Transformer feature extractor Feature extraction on data of all domains to get high-dimensional feature representation ; the feature extractor contains a Fourier-augmented feedforward network (FAN) module that projects the input features into parallel periodic and aperiodic processing paths; wherein the periodic processing path applies and activation functions to capture high-frequency periodic features, and the aperiodic processing path applies activation functions to capture aperiodic features, and finally the outputs of the two paths are concatenated and fused through a linear layer to obtain the feature representation ; Step 3: input the feature representation f into a power regressor to obtain a power prediction value and compute the gradient of the regression loss ; Step 4: Construct a double alignment domain adaptive module to align the feature distribution of the source domain and the target domain, the module comprising: (1) Implicit alignment based on adversarial training, that is, using a domain classifier D i and gradient reversal layer (GRL) to adversarially train the source domain and the target domain, and calculate the adversarial loss ; (2) The explicit alignment based on the multi-kernel maximum mean discrepancy (MK-MMD) calculates the MMD loss of the feature distribution of the source domain and the target domain ; Step 5: The regression loss generated in Step 3 and Step 4 is processed by using the gradient cooperative optimization strategy PCGrad , the adversarial loss , and the MMD loss with respect to the gradient of the shared feature extractor to solve the gradient conflict among multiple tasks; Step 6: Update the shared feature extractor according to the processed gradient and the parameters of the power regressor .

2. The method according to claim 1, characterized in that: the Fourier-augmented Transformer feature extractor in step 2 includes a Fourier-augmented feed-forward network (FAN) module that decomposes input features into periodic and aperiodic processing paths.

3. The method according to claim 2, characterized in that: The periodic processing path of the FAN module passes through linear layers After projection, apply and activation function to obtain periodic features ; the aperiodic processing path passes through linear layers After projection, apply activation function to obtain aperiodic features ; and the outputs of the two paths are spliced and fused through an output linear layer to obtain: ； wherein, is the final output of the Fourier-augmented feedforward network module; x is the input feature of the Fourier-augmented feedforward network module, which is fed into the periodic processing path and the aperiodic processing path, respectively; is a concatenation operation that concatenates the periodic feature and the aperiodic feature together in the feature dimension; is the weight matrix of the output linear layer, and is the bias vector of the output linear layer, and is the corresponding bias term of the linear transformation 4. The method according to claim 1, characterized in that: the regression loss in step 3 calculated using mean squared error (MSE): ； where (x, y) is a supervised learning sample sampled from the training dataset D, x is the input sequence feature of the sample, is the true power value, is the predicted power value.

5. The method according to claim 1, characterized in that: The gradient reversal layer (GRL) in step 4 is an identity transformation when the network is forward propagated, and in back propagation, the gradient from the domain classifier Di loss is multiplied by a negative coefficient , which is passed to the shared feature extractor , thus achieving a feature extractor that maximizes the domain classifier loss objective.

6. The method according to claim 1, characterized in that: The adversarial loss in step 4 For all Individual adversarial task losses Mean: ； wherein the first field classifier loss is defined using binary cross-entropy: ； wherein, denotes the mathematical expectation; denotes an input sample sampled from the data distribution of the th source domain ; denotes an input sample sampled from the data distribution of the target domain ; is the output feature of the feature extractor; is the output probability of the th domain classifier.

7. The method according to claim 6, characterized in that: The MMD loss in step 4 For all The multi-kernel maximum mean discrepancy of the feature distribution of each source domain Si and the target domain T The mean: ； where The computation is performed using a set of Gaussian kernels with different bandwidths, denote the data representing the ith source domain, denote the data representing the target domain, is a function that takes these data as input and outputs their high-dimensional feature representation, is the feature distribution of the source domain data after the feature extractor, while is the feature distribution of the target domain data after the same feature extractor.

8. The method according to claim 1, characterized in that: The processing process of the gradient collaborative optimization strategy (PCGrad) in the step 5 comprises: (a) computing a main task gradient and a side task gradient , wherein, is a gradient operator, is a parameter of the shared feature extractor , and are loss weight hyperparameters; (b) Check whether there is a conflict between the auxiliary task gradient and the main task gradient.

9. The method according to claim 8, characterized in that: The conflict is determined by computing the dot product of the gradient vectors, e.g. where is the regression loss according to step 3 with respect to the shared feature extractor parameters the resulting main task gradient, is the adversarial loss according to step 4 with respect to the shared feature extractor parameters the resulting auxiliary task gradient, and the conflict is determined if the dot product is less than 0.

10. The method according to claim 9, characterized in that: When there is a conflict, the auxiliary task gradient that is in conflict is projected orthogonally onto the normal plane of the main task gradient to eliminate its negative interference component, while the main task gradient remains unchanged. , represents or ) is projected orthogonally onto the normal plane of the main task gradient ( ) to eliminate its negative interference component, while the main task gradient ( ) remains unchanged; The projection calculation formula is: ； wherein, is the projected modified auxiliary task gradient, is the norm of the main task gradient vector norm; Finalizing for updating shared feature extractor Synthetic gradients is: ； wherein, and respectively represent the adversarial loss gradient and MMD loss gradient the modified gradient obtained after the projection calculation process; The gradient after the synthesis updating the shared feature extractor After training, the loss function of the model gradually converges, and the wind power prediction on the target domain test set is realized.

Images