A method for ultra-short-term wind power prediction based on an improved loss function and a patched temporal Transformer network

Abstract

This invention discloses an ultra-short-term wind power prediction method based on an improved loss function and a patched time-series Transformer network, belonging to the field of wind power prediction technology. The method includes the following steps: detecting and cleaning power data; calculating the correlation between meteorological and power data, and extracting features highly correlated with wind power; processing the features highly correlated with wind power using average pooling to extract the trend component, and then subtracting the trend component from the original sequence to obtain the fluctuation component; establishing Patch TST models for the trend component and the fluctuation component respectively, introducing a channel independence strategy, and adding the prediction results of the two components to reconstruct the final prediction value; optimizing the parameters of the Patch TST model using a multivariate nonlinear loss function; and verifying the effectiveness of wind power prediction. Compared with other models, the method proposed in this invention can achieve optimal prediction in multi-step ultra-short-term wind power prediction, effectively improving the reliability of wind power prediction.

Description

Technical Field

[0001] This invention relates to the field of wind power prediction technology, and in particular to an ultra-short-term wind power prediction method based on an improved loss function and a patched time-series Transformer network. Background Technology

[0002] Wind energy itself is characterized by fluctuations and randomness, which poses a significant threat to the safe and stable operation of the power system. Therefore, accurate wind power forecasting has become a critical issue that urgently needs to be addressed.

[0003] Ultra-short-term wind power forecasting is a type of wind power forecasting with a time interval of 15 minutes. It is typically used to predict wind power for the next 0-4 hours. Early forecasting models often used traditional models such as the Autoregressive Moving Average (ARIMA) model. These models, through recursive methods, can only capture the linear relationship between data. As data complexity and prediction step size increase, the prediction results often tend to converge to the mean, making it difficult to meet the needs of complex application scenarios. With the development of artificial intelligence technology, AI-based prediction models have gradually become a research hotspot, such as Recurrent Neural Networks (RNNs), Long Short-Term Memory Neural Networks (LSTMs), and Gated Recurrent Unit Networks (GRUs). However, the structure of these models is difficult to effectively capture the temporal dependencies between output sequences. Furthermore, they are prone to model instability and catastrophic forgetting of long sequences during training, limiting further improvements in prediction accuracy.

[0004] To date, most ultra-short-term wind power forecasts have focused on single-step forecasting, while research on multi-step forecasting is relatively limited. Compared to single-step forecasting, multi-step forecasting can better reflect power change trends and fluctuations over a future period, and has stronger guiding significance for the formulation of grid dispatching strategies.

[0005] In the field of natural language processing, typical machine translation problems are similar to multi-step prediction research, especially in the application of encoder-decoder (sequence-to-sequence, Seq2Seq) models. Since Google proposed the Transformer network in 2017, this network, with its encoder-decoder architecture and multi-head attention mechanism, theoretically possesses a longer global perspective and has been applied to the field of power prediction in recent years. However, the original Transformer model, designed for processing natural language, overemphasizes global semantics while neglecting the control of local features in power prediction. Simply relying on the original model is insufficient to meet the needs of wind power prediction. Therefore, enhancing the model's ability to extract local features is particularly urgent to address the challenges of intermittent energy prediction. Furthermore, existing time series prediction models, when processing multi-dimensional inputs, often mix data from different dimensions in the same channel, which can easily lead to cross-contamination of information, thereby affecting the model's accurate prediction of specific dimensional features and reducing the model's robustness. Therefore, it is imperative to adopt reasonable methods to separate the input channels of different features, reduce mutual interference between features, and enable the model to handle more complex data.

[0006] Furthermore, the prediction accuracy of a neural network model is closely related to the loss function used. Although root mean square error (MSE) and mean absolute error (MAE) are widely used loss functions, they are not optimal choices when dealing with the dynamic changes of complex time series, especially when shape perception is required. Traditional loss functions only focus on pointwise errors and rarely consider the temporal correlation of the sequence. This makes them prone to propagating incorrect gradients when dealing with local abrupt changes and drastic variations in the sequence, leading to abnormal model training. For example, MSE is prone to gradient explosion when dealing with outliers (noise) due to its squared error amplification property, which in turn affects the model's training performance. While some advanced loss functions, such as approximate dynamic time warping and DILATE, focus on shape perception, their noise suppression capabilities are still not ideal.

[0007] Therefore, a more accurate prediction method for wind power is needed, based on an improved loss function and a patched temporal Transformer network. Summary of the Invention

[0008] The purpose of this invention is to propose an ultra-short-term wind power prediction method based on an improved loss function and a patched time-series Transformer network, comprising the following steps:

[0009] An image anomaly detection and cleaning algorithm for wind power data is used to detect and clean the power data;

[0010] The correlation between meteorological data and power data is calculated using Pearson correlation coefficient or MIC correlation coefficient, and features with high correlation to wind power are extracted.

[0011] The average pooling operation is used to process the features that are highly correlated with wind power to extract the trend component. Then, the fluctuation component is obtained by subtracting the trend component from the original sequence.

[0012] Patch TST models for trend and volatility components are established separately. A channel-independent strategy is introduced, and the prediction results of the two components are added together to reconstruct the final prediction value.

[0013] The parameters of the Patch TST model are optimized using a multivariate nonlinear loss function;

[0014] The effects of patch block structure, channel independence strategy and multivariate nonlinear loss function on the prediction results are compared and analyzed to verify the effectiveness of wind power prediction.

[0015] The image anomaly detection and cleaning algorithm for wind power data specifically includes:

[0016] The wind power data is pre-cleaned to remove obvious outliers. The pre-cleaned data is then used to generate a binary image. Finally, the maximum contour is extracted from the binary image using erosion and dilation operations with different structuring elements b.

[0017] Calculate the Hu value for each binary image;

[0018] Compare the dissimilarity between the 7 Hu moments of each binary image and the Hu moments of the reference WPC binary image, select the binary image with the smallest dissimilarity of the 7 Hu moments, and use it for image contour extraction;

[0019] Mark the data outside the outline, establish a mapping relationship between normal data and the image, and filter out all normal data.

[0020] The process of building the Patch TST model is as follows:

[0021] After normalizing the input sequence, it is divided into several potentially overlapping patches, with each patch sequence considered as a unit. These patches are then processed on their respective independent channels to preserve local features in the time series. After positional encoding, the patch sequence is input into the Transformer encoding layer. The encoding layer uses a multi-head attention mechanism to capture the global dependencies of the time series from multiple dimensions. Finally, after flattening, the output vector of the Transformer is input into the linear prediction head to generate the final prediction result.

[0022] Multivariate nonlinear loss functions include:

[0023] Amplitude translation invariance loss function

[0024]

[0025] in, and These are the i-th predicted value and the i-th actual value, respectively, and T is the length of the time series;

[0026] Phase translation invariance loss function

[0027]

[0028] in, Represents the Fourier transform, ||·|| p It is L p Norm;

[0029] Smooth quadratic loss function

[0030]

[0031] Where c is a hyperparameter used to balance the noise filtering effect; α is used to balance the weights of different loss terms;

[0032] Weighted combination

[0033] L combined =λ1·L amplitude +λ2·L phase +λ3·L SQL

[0034] Wherein, λ1, λ2 and λ3 are weighting parameters used to balance the contributions of different loss terms.

[0035] The beneficial effects of this invention are as follows:

[0036] This invention enhances the local feature extraction capability and robustness of the original Transformer model by introducing a patch structure and a channel independence strategy on the traditional Transformer model; the designed multivariate nonlinear loss function enhances shape perception capability and reduces the impact of noise; compared with other models, it can achieve optimal prediction of ultra-short-term wind power multi-step prediction, which can effectively improve the reliability of wind power prediction. Attached Figure Description

[0037] Figure 1 This is a flowchart of the wind power prediction process of this invention;

[0038] Figure 2 It is a normal data extraction process based on images;

[0039] Figure 3 This is a schematic diagram of the Patch TST architecture;

[0040] Figure 4 It is a scaled dot product attention structure diagram;

[0041] Figure 5 This is a diagram illustrating the multi-head attention mechanism;

[0042] Figure 6 It is a correlation heatmap;

[0043] Figure 7 These are the prediction results from different models (4-step prediction);

[0044] Figure 8 This is a box plot of the error distribution of the 4-step prediction results. Detailed Implementation

[0045] This invention proposes an ultra-short-term wind power prediction method based on an improved loss function and a patched temporal Transformer network. The invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0046] Figure 1 This is a flowchart of the wind power prediction process of this invention. The process includes the following steps: data preprocessing, feature processing, Patch TST model prediction, and model optimization. Details are as follows:

[0047] S101 data preprocessing consists of three parts: feature selection, data cleaning, and missing value imputation.

[0048] S101.1 Feature selection utilizes the Pearson correlation coefficient (PCC) or maximum mutual information coefficient (MIC) to screen out several meteorological features in historical wind power data that are most correlated with wind power:

[0049]

[0050] In the formula: a and b represent two types of characteristic variables, a m b m ... and F represents the mean of the two classes of samples. PCC F represents the inter-sample PCC coefficient; MIC Let be the inter-sample MIC, n1 and n2 be the number of grid distributions, B be the 0.6 power of the data size, p(a,b) be the joint probability between sample variables, and I(a,b) be the mutual information.

[0051] S101.2 data cleaning targets power data and employs an anomaly detection and cleaning algorithm based on wind power images. This algorithm consists of three parts: data pre-cleaning, normal data extraction, and data labeling. Data pre-cleaning filters out outlier data points where the wind speed is greater than the cut-in wind speed but the power is less than 0.

[0052] Normal data extraction involves converting observational data from normal wind turbines into a binary image, while outlier data points are manually identified and removed by experts. After removing outliers, the remaining normal data is used to generate a reference wind power curve (WPC) image, which serves as the benchmark for subsequent data extraction and evaluation.

[0053] Furthermore, the erosion and dilation operations in mathematical morphology (MMO) are used to extract the main body of the binary image. The calculation method for the erosion operation is shown in (4), and the calculation method for the dilation operation is shown in (5):

[0054]

[0055] Considering that erosion and dilation operations are affected by the shape and size of the structuring element b, the Hu moment dissimilarity is used to measure the difference between the extracted image subject and the reference WPC image subject. The Hu moment is calculated from the linear combination of the image center moments; therefore, the image center moments must be calculated first to calculate the Hu moment. The formula for calculating the image center moment is shown in (6); and, in order to eliminate the influence of image scaling on the calculation of the center moment, the center moment is normalized, and the calculation formula is shown in (7).

[0056]

[0057] In the formula, μ pq Let γ denote the p+q order central moment of a binary image f(x,y) of size M×N; γ = 1 + (p+q) / 2, p+q = 2, 3, ...

[0058] Figure 2 This is a normal data extraction process based on images. The previously pre-cleaned wind power data is used to generate a binary image of WPC. First, different sizes of structuring elements n are used to perform erosion and dilation operations to extract the main body. At the same time, in order to reduce complexity, the extracted main body is further simplified to the maximum contour. Then, the Hu moment value of the image is calculated by formula (8), and the overall dissimilarity between the WPC binary image contour extracted under different structuring elements and the reference WPC binary image contour is calculated by formula (9). By comparing the maximum value of these dissimilarity values, the maximum difference in shape features between the two images is measured.

[0059] m k =sign(I k)·log(I k ),k=1...7 (8)

[0060]

[0061] In the formula, m k Let m be the k-th Hu moment of the image, and D(a,b) represent the maximum dissimilarity among the 7 Hu moments of images a and b. a,k and m b,k Let represent the k-th Hu moment of images a and b, respectively.

[0062] Finally, the optimal size n of the structuring element is determined according to equation (10). best The main part extracted through this structural element represents the normal data in the wind power data.

[0063] n best =minD(a n b) (10)

[0064] In the formula, n = 2, ..., n e ;n e It is the maximum size of the structuring element.

[0065] After extracting normal data, data labeling is performed. The main task of data labeling is to identify and label abnormal data points outside the main outline, and at the same time establish a mapping relationship between wind power points and image pixels. Through this mapping relationship, normal and abnormal data are labeled. The scaling parameters between pixels and wind power points are determined according to formula (11), while formula (12) is based on the data mapping algorithm to label and filter normal and abnormal wind power data of SCADA.

[0066]

[0067] In the formula, x max =max{x|f(x,y)=1}, x min =min{x|f(x,y)=1},y max =max{y|f(x,y)=1},y min =min{y|f(x,y)=1},v max and v min P represents the maximum and minimum wind speeds, respectively. max and P min These represent the maximum and minimum wind forces, respectively, and the z-th pixel in the image (x z y z The z-th wind farm point (v) corresponds to z P z ).

[0068] S101.3 addresses the issue of missing values ​​in the extracted data by using cubic spline interpolation to interpolate the missing data and ensure temporal continuity. Cubic spline interpolation is a method that ensures the smoothness and continuity of interpolation by fitting a piecewise cubic polynomial. The interpolation formula is as follows:

[0069] S l (x)=a l +b l (xx l )+c l (xx l ) 2 +d l (xx l ) 3 (13)

[0070] Among them, S i (x) is the cubic interpolation polynomial corresponding to the l-th segment. l b l c l d l These are the coefficients related to the interpolation polynomial of the l-th segment, x l It is a node in the l-th interpolation interval.

[0071] S102 feature processing involves decomposing the time series data into feature components, extracting the trend component and the residual component. By analyzing wind power data, the original time series X is decomposed into two parts: the trend component Xt. s and residual component X r Its decomposition formula is as follows:

[0072] X = X r +X s (14)

[0073] Among them, X s X r These represent the extracted trend component and residual component, respectively.

[0074] To extract the trend component from the raw data, average pooling (Avg Pooling) was used. The specific calculation formula is as follows:

[0075]

[0076] Where w represents the size of the pooling window, which determines the number of time points included in each average calculation; X s [α] represents the value of the trend component at position α after pooling.

[0077] To maintain the original sequence length during pooling, a padding technique is used. The padding length p is calculated based on the pooling window size.

[0078]

[0079] The length of the padded sequence remains consistent with that of the original sequence, thus avoiding the problem of sequence length shortening caused by pooling operations.

[0080] S103 Patch TST architecture such as Figure 3 As shown, the model structure consists of three parts: a sequence construction layer, a Transformer encoding layer, and a prediction head. The sequence construction layer mainly involves three processes: input normalization, patch partitioning, and channel independence. During prediction, prediction models for the trend component and the residual component are established separately. After input sequence normalization, it is divided into several potentially overlapping patches, with each patch sequence considered as a unit. These patch data are then processed on each independent channel to preserve local features in the time series. After positional encoding, these patch sequences are input into the Transformer encoding layer. The encoding layer uses a multi-head attention mechanism to capture the global dependencies of the time series from a multi-dimensional perspective, improving the model's ability to perceive global information in the data. Finally, after flattening, the Transformer's output vector is input into the linear prediction head to generate the final prediction result.

[0081] In the S103.1 sequence construction layer, patch partitioning is achieved by dividing the time series into several windows, which can be overlapping or non-overlapping. Let the time series length be L, the patch length be P, and the length of the non-overlapping region between two consecutive patches be S. The expression for the number of projected patch sequences N is:

[0082]

[0083] in This indicates rounding down. Additionally, to ensure the generated patch covers the entire time series and has a consistent length, the original time series is padded. The padding method involves using the last value x of the time series... L Repeat S times and add to the end of the time series.

[0084] Channel-independent strategies focus on the prediction of a single feature channel to better learn the periodicity and volatility of a single feature without being disturbed by noise from other feature channels.

[0085] The S103.2 Transformer coding layer mainly consists of positional encoding, multi-head attention mechanism, residual connections and normalization, and feedforward neural network.

[0086] To enable the model to perceive the positional information of the input data, positional encoding (PE) is introduced in the Transformer encoding layer. The generation formulas for positional encoding are shown in equations (18) and (19).

[0087]

[0088] In the formula: P(·) is the position encoding operation, Pos is the position index, and 2i pe This indicates that the time series position is even, 2i pe +1 indicates that the time position is odd, and d is the feature dimension of the input data.

[0089] The generated position encoding vectors are combined one by one to form the position encoding matrix W. Pos Each column of the matrix corresponds to a position in the sequence, and the length of each column is d, meaning the matrix has dimensions d×N, where N is the length of the input sequence. The formula is as follows:

[0090] W pos =(PE(0) PE(1) … PE(N-1)) (20)

[0091] Next, the input patch sequence x p (t) Through the mapping matrix W P ∈R d×P Perform a linear projection onto a space of dimension d to obtain the feature projection, which is then compared with the position encoding matrix W. Pos ∈R d×N Adding them together yields a patch sequence x that includes the chronological order. d (t) The calculation formula is as follows:

[0092]

[0093] In the formula, W P Let x be the mapping matrix. p (t) For the t-th patch input, x d (t) This represents the feature projection result of the t-th patch.

[0094] The multi-head attention mechanism consists of multiple scaled dot-product attention modules.

[0095] Figure 4This is a scaled dot product attention structure diagram. Scaled dot product self-attention obtains attention scores by calculating the dot product between the query vector (Query, Q), key vector (Key, K), and value vector (Value, V). These scores represent the relevance of a given element in the sequence to other elements. First, a patch sequence containing time order is... Transpose, and you get dimension R. N×d The sequence. Then, through three linear mapping matrices W Q W K W V Projecting these vectors onto each other yields three vectors: Q, K, and V.

[0096]

[0097] In the formula, W Q W K ∈R d×dk W V ∈R d×d Q, K∈R N×dk V∈R N×d .

[0098] Based on this, by calculating Q and K T The original attention score is obtained by multiplying the values ​​between the two numbers. However, this score needs to be scaled (usually divided by 1 / 2). Then, the scaled scores are subjected to Softmax normalization activation processing to obtain attention weight coefficients with a sum of 1.

[0099] Finally, these weight coefficients are multiplied by V to obtain a weighted value, and the sum of the weighted values ​​is used as the output corresponding to the current query.

[0100]

[0101] Where Attention∈R N×d d k This represents the scaling factor.

[0102] Figure 5 This is a schematic diagram of a multi-head attention mechanism; the multi-head attention mechanism uses h independent scaling dot product self-attention calculations in parallel to process a patch sequence containing temporal order. Transpose, and then pass through different linear mapping matrices respectively. and Projecting is performed to obtain the query, key, and value vectors under each attention head. These vectors are then fed into the scaled dot product self-attention system for parallel computation.

[0103]

[0104] Where j represents the j-th attention head, and These are the linear mapping matrices for the j-th attention head.

[0105] Next, the outputs of all attention heads are concatenated and linearly transformed using a linear mapping matrix in a fully connected layer to obtain the final output:

[0106] MultiHead=Concat(head1,...,head h )U msa (25)

[0107] In the formula, U msa Let h be the linear mapping matrix, and h represent the number of scaling dot product self-attention heads. In the multi-head self-attention mechanism, by using multiple self-attention modules in parallel, the model can capture more features in different subspaces, improving its ability to perceive global dependencies and local features of sequences.

[0108] S103.3 Residual connection and normalization structure connects the various sub-modules through addition and normalization operations.

[0109] The S103.4 feedforward neural network (FFN) structure consists of two linear transformations and a nonlinear activation function, calculated as follows:

[0110] F(x)=max(0,xW1+b1)W2+b2 (26)

[0111] In the formula: F(·) is the feedforward neural network function; W1 and b1 are the parameters of the first linear transformation; W2 and b2 are the parameters of the second linear transformation.

[0112] The S104 model optimization employs a multivariate nonlinear loss function to optimize the parameters of the established Patch TST model. This loss function consists of three parts: amplitude translation invariant loss, phase translation invariant loss, and smooth quadratic loss. The multivariate nonlinear loss function comprises the following three parts: amplitude translation invariant loss, phase translation invariant loss, and smooth quadratic loss function.

[0113] S104.1 Amplitude shift invariance loss enhances the model's sensitivity to changes in the amplitude of the time series by calculating the normalized error between the predicted and actual sequences. The formula is:

[0114]

[0115] in, and These are the i-th predicted value and the i-th actual value, respectively, and T is the length of the time series.

[0116] S104.2 Phase Shift Invariance Loss captures phase changes in a time series using Fourier transform. This loss term maps the sequence from the time domain to the frequency domain to capture these phase changes. Its loss function formula is as follows:

[0117]

[0118] in, Represents the Fourier transform, ||·|| p It is L p Norm.

[0119] The S104.3 smooth quadratic loss function, combined with a rational quadratic kernel function and absolute error, effectively improves the model's robustness to noisy data. Its formula is as follows:

[0120]

[0121] Where c is a hyperparameter used to balance the noise filtering effect; α is used to balance the weights of different loss terms, with α set to 0.2 and c set to 0.08.

[0122] Finally, the three loss functions are combined using weighted summation to form a comprehensive multivariate nonlinear loss function. The formula is:

[0123] L combined =λ1·L amplitude +λ2·L phase +λ3·L SQL (30)

[0124] Wherein, λ1, λ2, and λ3 are weighting parameters used to balance the contributions of different loss terms. Here, based on experimental results, they are taken as 0.3, 0.3, and 0.4, respectively.

[0125] To improve prediction accuracy, the prediction steps constructed in this embodiment are as follows:

[0126] 1) Data Preprocessing: Meteorological data from historical wind power data are filtered using Pearson correlation coefficient or MIC correlation coefficient to identify several meteorological features most correlated with wind power. Power data is first pre-cleaned to remove obvious abnormal data. Then, based on the wind power image detection and cleaning algorithm, binary images are generated from the pre-cleaned power data. The maximum contour is extracted from the binary images under different structuring elements b using equations (4) and (5). The Hu values ​​of each binary image are compared using equation (8). Then, the dissimilarity between the seven Hu moments of each binary image and the Hu moments of the reference WPC binary image is compared using equations (9) and (10). The binary image with the smallest maximum dissimilarity among the seven Hu moments is selected. This process ensures that the extracted binary image is most similar to the reference WPC binary image in terms of shape features. After comparing the results, the main body of the binary image is extracted using the corresponding structuring elements, and abnormal data outside the main body is filtered out. After extracting normal data, the cleaned data is labeled, and the mapping relationship between normal data and images is established using equation (11). Then, the normal data is labeled using equation (12) to filter out all normal data. In the case of missing values ​​in the extracted data, cubic spline interpolation is used to interpolate the missing data to ensure time continuity.

[0127] 2) Feature Processing: To better capture the trends and fluctuations of the time series, average pooling is used to smooth the screened meteorological series and the interpolated power series, removing short-term noise and extracting a smoother trend component. Then, the fluctuation component is obtained by subtracting the extracted trend component from the original series. Simultaneously, to maintain consistent series length after decomposition, boundary padding is used to fill the gaps in the series.

[0128] 3) Patch TST Model Prediction: During prediction, prediction models for the trend component and the residual component are established separately. In order to accelerate the convergence speed of the model and eliminate the influence of different scales on model training, the time series after feature processing is normalized. The sequence is divided into several patches using Equation (17). After the patch division, the channel-independent strategy is adopted to process the input data of different meteorological features separately to avoid mutual interference between features. Then, the position code of the sequence is generated using Equations (18) and (19). The position code is processed into a position code matrix using Equation (20). The position information of the time series is embedded into the input sequence using Equation (21) so that the model can capture the sequential information of the time step. Then, the embedded patch sequence is sent into the multi-head attention mechanism. Each attention head calculates the attention score from different angles according to Equations (24)-(25) to capture global dependencies. Afterwards, the output and input of each attention module are connected via residual connections to prevent gradient vanishing. The output after residual connection is normalized to accelerate the convergence speed of training and ensure model stability. The normalized output is further transformed nonlinearly by a feedforward neural network according to equation (26) to enhance the model's learning of complex features. Then, the output sequence passes through a residual connection and normalization layer, and the final output is flattened to reduce the multidimensional features to low-dimensional vectors. After projection through the fully connected layer in the linear head connection, the predicted values ​​of the relevant components are obtained. Based on the prediction models established for the two components above, the prediction results are added together to obtain the final predicted value.

[0129] 4) Model Optimization: To enable the model to better perceive changes in sequence shape and resist noise interference in wind power prediction, a multivariate nonlinear loss function was designed to optimize the parameters in Patch TST. This loss function consists of three parts: amplitude translation invariant loss, phase translation invariant loss, and smooth quadratic loss. The amplitude translation invariant loss is established using Equation (27), the phase translation invariant loss using Equation (28), and the smooth quadratic loss using Equation (29). These three losses are used to jointly optimize the model through linear weighting in Equation (30). The entire optimization process of the loss function is implemented through backpropagation and gradient descent algorithms to guide the parameter updates of the model.

[0130] To illustrate the effects of this invention, the following numerical examples are provided:

[0131] Measured data for a wind farm were collected from January to December 2019. The dataset includes wind power output and local meteorological data, with a sampling interval of 15 minutes. Specific data information is shown in Table 1.

[0132] Table 1 Scene Settings

[0133]

[0134] The total wind power dataset includes not only wind power output but also multi-dimensional meteorological data, including wind speed, wind direction, temperature, and air pressure. Too many input features not only make it difficult to fit the model during training but also reduce training efficiency. The feature correlation heatmap is shown below. Figure 6 As shown.

[0135] Based on the correlation analysis results in Table 2, we consider using the wind speed, historical power, air pressure, and temperature data with the highest correlation as input features for the model.

[0136] Table 2. Correlation Analysis Results

[0137]

[0138]

[0139] Considering the impact of different data units and sizes on the speed of parameter updates during deep learning backpropagation, to ensure rapid model convergence, it is necessary to use the minimax normalization formula to normalize the data to the interval (0, 1):

[0140]

[0141] In the formula: x i,feature This represents the i-th actual value of a certain feature in the original data. Let x be the normalized value of a certain feature in the original data; min(x) i,feature ) represents the minimum value among a certain type of feature in the original data; max(x) feature ) represents the maximum value of a certain type of feature in the original data.

[0142] The preprocessed data features include wind speed (at the 10m wind tower), wind speed (at the 30m wind tower), wind speed (at the hub), wind direction (at the 10m wind tower), and wind power. The experimental dataset is divided into training, validation, and test sets in an 8:1:1 ratio and input into the model. A fixed random seed is used in the experiment, and the Adam optimizer is employed with an initial learning rate of 0.001.

[0143] The comparison models employed were the widely used TCN-LSTM, seq2seq, Transformer, and Dlinear models for time series prediction, with a loss function of MSE, a learning rate of 0.001, a dropout of 0.2, and 50 epochs. The seq2seq model used an encoder-decoder structure, where both the encoder and decoder employed LSTM. The embedding dimension and the number of network layers were known empirical parameters. To obtain other model parameters, a Bayesian optimization search was first performed on the basic prediction parameters of the model, where the batch size N... batch ={32,48,64}, time step input L seq ={8,16,32}. Then, after fixing the basic parameters, the model network parameters are searched again, where the number of attention heads N is... head ={8,16,32}, the number of neurons in the hidden layer N hidden ={128,256,512}, where N is the number of coding layers. layers ={2,3,4}. The parameter settings for the same network were completely identical in the experiment. The final model parameters are shown in Table 3.

[0144] Table 3 Model Parameter Configuration Table

[0145]

[0146]

[0147] To prevent overfitting and improve training efficiency, an early stopping strategy for model training is introduced during the training phase, which means that training is stopped when the loss no longer decreases after 5 consecutive training iterations.

[0148] In terms of predictive indicator evaluation, the MAE indicator e is used. MAE Root Mean Square Error (RMSE) index e RMSE and the Mean Absolute Percentage Error (MAPE) index e MAPE .

[0149]

[0150] In the formula, and Y h Let N be the predicted value and the actual value of the h-th test sample, respectively; test This represents the total number of test samples.

[0151] To verify the model's performance, the following experiments were conducted:

[0152] 1) Model Comparison Experiment

[0153] Table 4 Comparison of multi-step prediction (4 steps) results from various models

[0154]

[0155] Comparing the prediction metrics in Table 4 reveals significant differences among the various prediction models. The Patch TST model proposed in this invention demonstrates a substantial improvement over the comparative models in all error metrics. Due to the lack of complex feature extraction and enhancement, the performance of traditional machine learning TCN-LSTM is far inferior to other deep learning models. While Attention-Seq2Seq, employing an encoder-decoder structure, introduces an attention mechanism that can transmit richer hidden information to the decoder, the cyclical structure, relying on the output of the previous step at each time step, amplifies subsequent errors due to errors in the previous step, resulting in larger prediction errors. In contrast, the Transformer-based model, with its self-attention mechanism, allows the model to focus on the entire sequence at each time step, reducing error accumulation. Finally, the Patch TST model, building upon the Transformer model, uses local patching to represent features, replacing single-point feature extraction, thus improving the ability to extract local features and achieving better prediction results. (The average e is not provided in the original text.) MAE Taking the index as an example, the prediction error of the Patch TST model is 1.481MW, compared to the average e MAE The second-best performing Dlinear model saw a 5.1% reduction, compared to a 16.1% decrease for the Transformer. Other metrics also showed some improvement, with specific reductions or increases as follows: e RMSE The indicators decreased by 0.6%, 5.6%, 26.6%, and 37.4% respectively. MAPE The indicators decreased by 78.9%, 62%, 49.6%, and 23.2%, respectively.

[0156] Figure 7 The results are shown in the predictions from different models (4-step prediction). Curves of the prediction results from different models are plotted to more intuitively demonstrate the prediction performance of the proposed method. The comparison shows that despite the large fluctuations in wind power, the prediction results of the proposed Patch TST model closely match the actual values ​​and can maintain accurate fitting within a relatively stable range. Furthermore, the model does not exhibit the drastic changes seen in other models at the turning points of power fluctuations, indicating that the model has better fitting performance and stability.

[0157] To more accurately and intuitively demonstrate the advantages of the proposed model in wind power prediction compared to other models, 20% of the error results were randomly selected from the prediction results of all test sets for box plot analysis, such as... Figure 8As shown in the box plot, the line in the middle of the box represents the median, reflecting the median value of the error. The lower the median, the smaller the overall prediction error of the model. The upper and lower boundaries of the box represent the first and third quartiles (Q1 and Q3). The smaller the IQR(Q3-Q1), the more concentrated the error distribution. Independent points represent outliers. The fewer outliers, the better the stability of the model's predictions. Overall, Patch TST outperforms other models in predicting all indicators.

[0158] 2) Loss Function Experiment

[0159] To verify the effectiveness of the loss function in multi-step prediction, multi-step prediction experiments at different scales were conducted. A fixed random seed was used in each experiment, and the average value was taken after seven trials. The experimental results are recorded in Tables 5 and 6.

[0160] Table 5 Comparison of MAE Indicators in Multi-Step Prediction Results

[0161]

[0162]

[0163] Table 6 Comparison of RMSE indices for multi-step prediction results

[0164]

[0165] As can be seen from Tables 5 and 6, the new loss function effectively improves various error metrics when performing multi-step predictions at different time scales. Especially in 3-hour predictions, the average e MAE Compared to traditional MAE and Mean Squared Error (MSE) loss functions, the metrics decreased by 1.9% and 1.2% respectively, with average e RMSE The metrics decreased by 1.9% and 2.3%, respectively. While multi-output prediction strategies can capture correlations between sequences, the model's need for local and global information weights varies depending on the prediction step size. Therefore, by adjusting the loss function accordingly, the data fit can still be improved to varying degrees.

[0166] 3) Model ablation experiment

[0167] To verify the effectiveness of each module in Patch TST, an ablation experiment was conducted, removing the modules one by one. Specifically, Patch TST... 1 Without using a new loss function, Patch TST 2 Without adopting a new loss function and channel-independent strategy, Patch TST 3Without employing the new loss function and patching structure, TST does not use the new loss function, patching structure, or channel-independent strategy. The ablation experiment results are shown in Table 7. Table 7 shows that using the new loss function, patching, and independent channel structure resulted in lower average e... MAE All metrics decreased by 6.5%. Compared to the basic TST model, the addition of the patching module and channel independence module resulted in an average reduction in e. RMSE and e MAPE The indicators decreased by 1.4% and 33.03%, respectively.

[0168] Table 7 Ablation Experiment Results

[0169]

[0170] Compared with other models, the method proposed in this invention can achieve optimal prediction of ultra-short-term wind power multi-step prediction, effectively improving the reliability of wind power prediction.

Claims

1. A method for ultra-short-term wind power prediction based on an improved loss function and a patched temporal Transformer network, characterized in that, Includes the following steps: An image anomaly detection and cleaning algorithm for wind power data is used to detect and clean the power data; The correlation between meteorological data and power data is calculated using Pearson correlation coefficient or MIC correlation coefficient, and features with high correlation to wind power are extracted; the meteorological data includes wind speed, wind direction, temperature, and air pressure; The average pooling operation is used to process the features that are highly correlated with wind power to extract the trend component. Then, the fluctuation component is obtained by subtracting the trend component from the original sequence. Patch TST models for trend and volatility components are established separately. A channel-independent strategy is introduced, and the prediction results of the two components are added together to reconstruct the final prediction value. The process of establishing the Patch TST model is as follows: After normalizing the input sequence, it is divided into several potentially overlapping patches, with each patch sequence considered as a unit; then the patches are processed on each independent channel to preserve the local features in the time series. After positional encoding, the patch sequence is input into the Transformer encoding layer; the encoding layer uses a multi-head attention mechanism to capture the global dependencies of the time series from multiple dimensions. Finally, after flattening, the output vector of the Transformer is input into the linear prediction head to generate the final prediction result; The parameters of the Patch TST model are optimized using a multivariate nonlinear loss function; The multivariate nonlinear loss function includes: Amplitude translation invariance loss function in, and These are the i-th predicted value and the i-th actual value, respectively, where T is the length of the time series; Phase translation invariance loss function in, Represents the Fourier transform, ||·|| p It is L p Norm; Smooth quadratic loss function Where c is a hyperparameter used to balance the noise filtering effect; α is used to balance the weights of different loss terms; y pred It is the predicted value, y true This is the actual value; Weighted combination Wherein, λ1, λ2 and λ3 are weighting parameters used to balance the contributions of different loss terms; The effects of patch block structure, channel independence strategy and multivariate nonlinear loss function on the prediction results are compared and analyzed to verify the effectiveness of wind power prediction.

2. The ultra-short-term wind power prediction method based on an improved loss function and a patched temporal Transformer network as described in claim 1, characterized in that, The image anomaly detection and cleaning algorithm for wind power data specifically includes: The wind power data is pre-cleaned to remove obvious outliers. The pre-cleaned data is then used to generate a binary image. Finally, the maximum contour is extracted from the binary image using erosion and dilation operations with different structuring elements b. Calculate the Hu value for each binary image; Compare the dissimilarity between the 7 Hu moments of each binary image and the Hu moments of the reference WPC binary image, select the binary image with the smallest dissimilarity of the 7 Hu moments, and use it for image contour extraction; Mark the data outside the outline, establish a mapping relationship between normal data and the image, and filter out all normal data.

Images