Ultra-short-term wind power prediction method based on improved loss function and patch time sequence Transform network

Abstract

The invention discloses an ultra-short-term wind power prediction method based on an improved loss function and a patch time sequence Transform network, and belongs to the technical field of wind power prediction. Comprising the following steps: detecting and cleaning power data; correlation calculation is carried out on the meteorological data and the power data, and features with high correlation with the wind power are extracted; utilizing average pooling operation to process the characteristics with high correlation with the wind power, extracting a trend component, and then utilizing an original sequence to subtract the trend component to obtain a fluctuation component; respectively establishing Patch TST models of a trend component and a fluctuation component, introducing a channel independent strategy, and adding and reconstructing two component prediction results to obtain a final prediction value; a multivariate nonlinear loss function is adopted to optimize parameters of the Patch TST model; and verifying the effectiveness of wind power prediction. Compared with other models, the method provided by the invention can realize the optimal multi-step prediction of the ultra-short-term wind power, and effectively improves the reliability of wind power prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of wind power prediction, and in particular to an ultra-short-term wind power prediction method based on an improved loss function and a patch time series Transformer network. Background Art

[0002] Wind energy itself has the characteristics of volatility and randomness, which brings major hidden dangers to the safe and stable operation of the power system. Therefore, accurate wind power forecasting has become a key issue that needs to be solved urgently.

[0003] Ultra-short-term wind power forecasting is a type of wind power forecasting. Its time interval is 15 minutes. It is usually used to predict the wind power of 0 to 4 hours in the future. In the early days, traditional forecasting models such as the autoregressive moving average (ARIMA) model were often used. Such models can only capture the linear relationship between data through recursive methods. With the increase of data complexity and the increase of prediction step length, the prediction results tend to the mean, which is difficult to meet the needs of complex application scenarios. With the development of artificial intelligence technology, prediction models based on artificial intelligence have gradually become a research hotspot, such as recurrent neural networks (RNN), long short-term memory neural networks (LSTM), and gated recurrent unit networks (GRU). However, the structure of such models is difficult to effectively capture the temporal dependency between output sequences. At the same time, the model is prone to instability and catastrophic forgetting of long sequences during training, which limits the further improvement of prediction accuracy.

[0004] So far, most ultra-short-term wind power forecasts focus on single-step forecasts, while there are few studies on multi-step forecasts. Compared with single-step forecasts, multi-step forecasts can better reflect the power change trend and fluctuation in the future, and have a stronger guiding significance for the formulation of power grid dispatching strategies.

[0005] In the field of natural language processing, typical machine translation problems are similar to the research on multi-step prediction, especially in the application of encoder-decoder (sequence-to-sequence, Seq2Seq) models. Since the Google team proposed the Transformer network in 2017, the network has theoretically had a longer global vision with its encoder-decoder architecture and multi-head attention mechanism, and has been applied to the field of power prediction in recent years. However, when the original Transformer model was designed to process natural language, it overemphasized global semantics and ignored the control of local features in power prediction. Simply relying on the original model is difficult to meet the needs of wind power prediction. Therefore, it is particularly urgent to enhance the model's ability to extract local features in order to meet the challenges of intermittent energy prediction. In addition, when processing multi-dimensional inputs, existing time series prediction models often mix data of different dimensions in the same channel, which is easy to cause information cross-contamination, thereby affecting the model's accurate prediction of specific dimensional features and reducing the robustness of the model. Therefore, it is imperative to adopt a reasonable way to separate the input channels of different features, reduce the mutual interference between features, and enable the model to have the ability to process more complex data.

[0006] In addition, the prediction accuracy of the neural network model is closely related to the loss function adopted. Although the root mean square error (MSE) and the mean absolute error (MAE) are widely used loss functions, they are not the best choice when dealing with the dynamic changes of complex time series, especially when the shape of the time series needs to be perceived. Traditional loss functions only focus on point-by-point errors and rarely consider the temporal correlation of the sequence, which makes them prone to passing wrong gradients when dealing with local mutations and drastic changes in the sequence, resulting in model training anomalies. For example, when dealing with outliers (noise), MSE is prone to gradient explosion due to its square amplification error characteristics, which in turn affects the training effect of the model. Although some advanced loss functions such as approximate dynamic time warping and DILATE focus on shape perception, their ability to suppress noise is still not ideal.

[0007] Therefore, an ultra-short-term wind power prediction method based on an improved loss function and a patch-series Transformer network is needed to make a more accurate prediction of wind power. Summary of the invention

[0008] The purpose of the present invention is to propose an ultra-short-term wind power prediction method based on an improved loss function and a patch time series Transformer network, comprising the following steps:

[0009] The image anomaly detection and cleaning algorithm of wind power data is used to detect and clean the power data;

[0010] The correlation between meteorological data and power data is calculated by Pearson correlation coefficient or MIC correlation coefficient, and features with high correlation with wind power are extracted;

[0011] The average pooling operation is used to process the features with high correlation with wind power, and the trend component is extracted. Then, the fluctuation component is obtained by subtracting the trend component from the original sequence.

[0012] The Patch TST models of trend component and volatility component are established respectively, and the channel independence strategy is introduced to add the prediction results of the two components to reconstruct the final prediction value;

[0013] The parameters of the Patch TST model are optimized using a multivariate nonlinear loss function;

[0014] The effects of patch block structure, channel independence strategy and multivariate nonlinear loss function on the prediction results are compared and analyzed to verify the effectiveness of wind power prediction.

[0015] The image anomaly detection and cleaning algorithm of wind power data is used to detect and clean the power data, including:

[0016] Pre-clean the wind power data, remove obvious abnormal data, generate a binary image from the pre-cleaned data, and then use erosion and dilation operations with different structural elements b to extract the maximum contour of the binary image;

[0017] Calculate the Hu value of each binary image;

[0018] Compare the dissimilarity between the 7 Hu moments of each binary image and the Hu moment of the reference WPC binary image, select the 7 binary images with the smallest Hu moment dissimilarity, and use them to extract image contours;

[0019] The data outside the contour is marked, and a mapping relationship between normal data and images is established to filter out all normal data.

[0020] The establishment process of the Patch TST model is as follows:

[0021] After normalization, the input sequence is divided into several possibly overlapping patches, and each patch sequence is regarded as a unit. The above patches are then processed on each independent channel to retain the local features in the time series. After position encoding, the patch sequence is input into the Transformer encoding layer. The encoding layer captures the global dependencies of the time series from a multi-dimensional perspective through a multi-head attention mechanism. Finally, after flattening, the output vector of the Transformer is input into the linear prediction head to generate the final prediction result.

[0022] Multivariate nonlinear loss functions include:

[0023] Amplitude translation invariance loss function

[0024]

[0025] in, and are the i-th predicted value and the i-th actual value respectively, and T is the length of the time series;

[0026] Phase shift invariance loss function

[0027]

[0028] in, represents Fourier transform, ||·|| p YesL p norm;

[0029] Smooth quadratic loss function

[0030]

[0031] Among them, c is a hyperparameter used to balance the noise filtering effect; α is used to balance the weights of different loss terms;

[0032] Weighted combination

[0033] L combined =λ1·L amplitude +λ2·L phase +λ3·L SQL

[0034] Among them, λ1, λ2, and λ3 are weight parameters used to balance the contribution of different loss terms.

[0035] The beneficial effects of the present invention are:

[0036] The present invention enhances the local feature extraction capability and robustness of the original Transformer by introducing the Patch structure and channel independence strategy on the traditional Transformer model. The designed multivariate nonlinear loss function enhances the shape perception capability and reduces the influence of noise. Compared with other models, it can achieve the optimal prediction of ultra-short-term wind power multi-step prediction, which can effectively improve the reliability of wind power prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of wind power prediction of the present invention;

[0038] Figure 2 It is a normal image-based data extraction process;

[0039] Figure 3 This is a schematic diagram of the Patch TST architecture;

[0040] Figure 4 It is a scaled dot product attention structure diagram;

[0041] Figure 5 It is a schematic diagram of the multi-head attention mechanism;

[0042] Figure 6 is a correlation heat map;

[0043] Figure 7 It is the prediction result of different models (4-step prediction);

[0044] Figure 8 It is a box plot of the error distribution of the 4-step prediction results. DETAILED DESCRIPTION

[0045] The present invention proposes an ultra-short-term wind power prediction method based on an improved loss function and a patch timing Transformer network. The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0046] Figure 1 This is the wind power prediction flow chart of the present invention. The process includes the following steps: data preprocessing, feature processing, Patch TST model prediction and model optimization. The details are as follows:

[0047] S101 data preprocessing consists of three parts: feature selection, data cleaning, and missing value filling.

[0048] S101.1 Feature selection uses the Pearson correlation coefficient (PCC) or the maximum mutual information coefficient (MIC) to screen out several meteorological features that are most correlated with wind power in wind power historical data:

[0049]

[0050] In the formula: a and b represent two types of characteristic variables, a m , b m are the values ​​of variables a and b of the mth sample, n is the total number of samples of the feature, and is the mean of the two types of samples, F PCC is the PCC coefficient between samples; F MIC is the MIC between samples, n1 and n2 are the number of grid distributions, the size of B is 0.6 of the data volume, p(a,b) is the joint probability between sample variables, and I(a,b) is the mutual information.

[0051] S101.2 data cleaning is aimed at power data, and adopts an abnormal detection and cleaning algorithm based on wind power images. This algorithm is divided into three parts: data pre-cleaning, normal data extraction and data marking. Data pre-cleaning is to filter out abnormal data points in the data where the wind speed is greater than the cut-in wind speed but the power is less than 0.

[0052] Normal data extraction is to convert the observation data of normal wind turbines into binary images, and the abnormal data points are manually identified and removed by experts. After removing the abnormal data, the remaining normal data is used to generate a reference wind power curve (WPC) image, which serves as the benchmark for subsequent data extraction and evaluation.

[0053] And the main part of the binary image is extracted by using the erosion and dilation operations in mathematical morphology (MMO). The calculation method of the erosion operation is shown in (4), and the calculation method of the dilation operation is shown in (5):

[0054]

[0055] Considering that the erosion and dilation operations are affected by the shape and size of the structural element b, the Hu moment dissimilarity is used to measure the difference between the extracted image body and the reference WPC image body. The calculation of the Hu moment comes from the linear combination of the image center moment. Therefore, to calculate the Hu moment, the image center moment must be calculated first. The calculation formula for the image center distance is shown in (6); and in order to eliminate the influence of image scaling on the calculation of the center moment, the center moment is normalized, and the calculation formula is shown in (7).

[0056]

[0057] In the formula, μ pq Represents the p+q order central moment of a binary image f(x,y) of size M×N; γ=1+(p+q) / 2, p+q=2,3,…

[0058] Figure 2 It is a normal data extraction process based on images; the previously pre-cleaned wind power data is used to generate a binary image of WPC, and the main part is first extracted by erosion and dilation operations using structural elements n of different sizes. At the same time, in order to reduce complexity, the extracted main part is further simplified to the maximum contour. Then, the Hu moment value of the image is calculated by formula (8), and the overall dissimilarity between the WPC binary image contour extracted under different structural elements and the reference WPC binary image contour is calculated by formula (9). By comparing the maximum value of these dissimilarities, the maximum difference in shape features between the two images is measured.

[0059] m k =sign(I k)·log(I k ),k=1...7 (8)

[0060]

[0061] In the formula, m k is the kth Hu moment of the image, D(a,b) represents the maximum dissimilarity among the 7 Hu moments of image a and image b, m a,k and m b,k Represent the kth Hu moment of image a and image b respectively.

[0062] Finally, the optimal size n of the structural element is determined according to formula (10): best ,The main part extracted by this structural element represents the normal data in the wind power data.

[0063] n best =minD(a n ,b) (10)

[0064] Where n = 2, ..., n e ;n e is the maximum size of the structural element.

[0065] After extracting normal data, data labeling is performed. The main task of data labeling is to identify and label abnormal data points outside the main contour, and at the same time establish a mapping relationship between wind power points and image pixels. Through this mapping relationship, normal and abnormal data are labeled. The scaling parameters between pixels and wind power points are completed according to formula (11), while formula (12) labels and filters the normal and abnormal wind power data of SCADA based on the data mapping algorithm.

[0066]

[0067] In the formula, x max =max{x|f(x,y)=1},x min =min{x|f(x,y)=1},y max =max{y|f(x,y)=1},y min =min{y|f(x,y)=1},v max and v min Represent the maximum and minimum wind speeds, P max and P min Represent the maximum and minimum wind speeds respectively. The zth pixel in the image (x z ,y z ) corresponds to the zth wind power point (v z , P z ).

[0068] S101.3 faces the situation of missing values ​​in the extracted data, and uses cubic spline interpolation to interpolate the missing data to ensure time continuity. Cubic spline interpolation is a method of ensuring the smoothness and continuity of interpolation by fitting a piecewise cubic polynomial. The interpolation formula is as follows:

[0069] S l (x) = a l +b l (xx l )+c l (xx l ) 2 +d l (xx l ) 3 (13)

[0070] Among them, S i (x) is the cubic interpolation polynomial corresponding to the lth segment. l , b l , c l , d l are the coefficients associated with the interpolation polynomial for the first segment, x l It is the node of the lth interpolation interval.

[0071] S102 feature processing is to decompose the time series into features and extract the trend component and residual component respectively. By analyzing the wind power data, the original time series X is decomposed into two parts: the trend component X s and the residual component X r The decomposition formula is as follows:

[0072] X=X r +X s (14)

[0073] Among them, X s , X r Represent the extracted trend component and residual component respectively.

[0074] In order to extract the trend component from the original data, the average pooling method is used. The specific calculation formula is:

[0075]

[0076] Where w represents the size of the pooling window, which determines the number of time points included in each average calculation; X s [α] represents the value of the trend component at position α after pooling.

[0077] In order to keep the length of the original sequence unchanged during the pooling process, the border padding technique is adopted. The padding length p is calculated according to the pooling window size:

[0078]

[0079] The length of the padded sequence remains consistent with the original sequence, thus avoiding the problem of shortening the sequence length caused by the pooling operation.

[0080] S103 Patch TST architecture Figure 3 As shown in the figure, the model structure consists of three parts: sequence construction layer, Transformer encoding layer and prediction head. The sequence construction layer mainly has three processes: input normalization, patch division and channel independence. When predicting, the prediction model of the trend component and the prediction model of the residual component are established respectively; after the input sequence is normalized, it is divided into several patches that may overlap, and each patch sequence is regarded as a unit. Then these patch data are processed on each independent channel to retain the local features in the time series. After position encoding, these patch sequences are input into the Transformer encoding layer. The encoding layer captures the global dependencies of the time series from a multi-dimensional perspective through the multi-head attention mechanism, and improves the model's perception of the global information of the data. Finally, after flattening, the output vector of the Transformer is input into the linear prediction head to generate the final prediction result.

[0081] In the S103.1 sequence construction layer, patch partitioning is achieved by splitting the time series into several windows, which can be overlapping or non-overlapping. Assuming the time series length is L, the patch length is P, and the length of the non-overlapping region between two consecutive patches is S, the expression for the number of projected patch sequences N is

[0082]

[0083] in Indicates rounding down. At the same time, in order to ensure that the generated patch covers the entire time series and has the same length, the original time series is padded. The padding method is to replace the last value x of the time series with L Repeat S times and add to the end of the time series.

[0084] The channel-independent strategy focuses on the prediction of a single feature channel to better learn the periodicity and volatility of a single feature without being disturbed by the noise of other feature channels.

[0085] S103.2 The Transformer encoding layer is mainly composed of position encoding, multi-head attention mechanism, residual connection and normalization, and feedforward neural network.

[0086] In order to enable the model to perceive the position information of the input data, positional encoding (PE) is introduced in the Transformer encoding layer. The generation formula of positional encoding is shown in Equation (18) and Equation (19).

[0087]

[0088] Where: P(·) is the position encoding operation, Pos is the position index, 2i pe Indicates that the time series position is an even number, 2i pe +1 indicates that the time position is an odd number, and d is the feature dimension of the input data.

[0089] The generated position encoding vectors are combined one by one to form the position encoding matrix W Pos Each column of the matrix corresponds to a position in the sequence, and the length of each column is d, that is, the dimension of the matrix is ​​d×N, where N is the length of the input sequence. The formula is as follows:

[0090] W pos =(PE(0) PE(1) … PE(N-1)) (20)

[0091] Next, the input patch sequence x p (t) By mapping the matrix W P ∈R d×P Perform linear projection and map it to a space of dimension d to obtain feature projection, which is then combined with the position encoding matrix W Pos ∈R d×N Add together to get the patch sequence x containing the time order d (t) , the calculation formula is as follows:

[0092]

[0093] Where W P is the mapping matrix, x p (t) is the t-th patch input, x d (t) is the feature projection result of the t-th patch.

[0094] The multi-head attention mechanism consists of multiple scaled dot-product self-attention modules.

[0095] Figure 4is a diagram of the scaled dot product attention structure; scaled dot product self-attention obtains attention scores by calculating the dot product between the query vector (Query, Q), the key vector (Key, K) and the value vector (Value, V). These scores represent the relevance of an element in the sequence to other elements. First, the patch sequence containing the time order Transpose, and get the dimension R N×d Then, through three linear mapping matrices W Q , W K , W V , project them respectively to get three vectors Q, K and V:

[0096]

[0097] Where W Q , W K ∈R d×dk , W V ∈R d×d , Q, K∈R N×dk , V∈R N×d .

[0098] On this basis, by calculating Q and K T The original attention score is obtained by the dot product between . However, this score needs to be scaled (usually divided by ). Then the scaled scores are subjected to Softmax normalization activation to obtain the attention weight coefficients with a sum of 1.

[0099] Finally, dot product these weight coefficients with V to obtain weighted values, and the sum of the weighted values ​​is used as the output corresponding to the current query.

[0100]

[0101] Among them, Attention∈R N×d , d k Represents the scaling factor.

[0102] Figure 5 is a schematic diagram of the multi-head attention mechanism; the multi-head attention mechanism converts a patch sequence containing time order into a sequence of time-ordered patches by parallelizing h independent scaled dot product self-attention calculations. Transpose, and then pass through different linear mapping matrices and Projection is performed to obtain the query, key, and value vectors under each attention head. These vectors are sent to the scaled dot product self-attention for parallel calculation:

[0103]

[0104] Among them, j represents the jth attention head, and are the linear mapping matrices of the j-th attention head respectively.

[0105] Next, the outputs of all attention heads are concatenated and linearly transformed through a linear mapping matrix in a fully connected layer to obtain the final output:

[0106] MultiHead=Concat(head1,...,head h ) msa (25)

[0107] Where U msa is a linear mapping matrix, and h represents the number of scaled dot product self-attention heads. In the multi-head self-attention mechanism, by using multiple self-attention modules in parallel, the model can capture more features in different subspaces and improve the perception of global dependencies and local features of the sequence.

[0108] S103.3 The residual connection and normalization structure connects each sub-module through addition and normalization operations.

[0109] The S103.4 feedforward neural network (FFN) structure consists of two linear transformations and a nonlinear activation function. The calculation formula is as follows:

[0110] F(x)=max(0,xW1+b1)W2+b2 (26)

[0111] Where: F(·) is the feedforward neural network function; W1 and b1 are the parameters of the first linear transformation; W2 and b2 are the parameters of the second linear transformation.

[0112] S104 model optimization optimizes the parameters in the established Patch TST model by designing a multivariate nonlinear loss function, which consists of three parts: amplitude shift invariance loss, phase shift invariance loss, and smooth quadratic loss. The multivariate nonlinear loss function consists of the following three parts: amplitude shift invariance loss, phase shift invariance loss, and smooth quadratic loss function.

[0113] S104.1 Amplitude translation invariance loss is to enhance the model's sensitivity to changes in the amplitude of the time series by calculating the normalized error between the predicted sequence and the actual sequence. Its formula is:

[0114]

[0115] in, and are the i-th predicted value and the i-th actual value respectively, and T is the length of the time series.

[0116] S104.2 phase shift invariance loss captures the phase change in the time series through Fourier transform, and maps the sequence from the time domain to the frequency domain through this loss term to capture the phase change. Its loss function formula is:

[0117]

[0118] in, represents Fourier transform, ||·|| p YesL p Norm.

[0119] The S104.3 smooth quadratic loss function combines the rational quadratic kernel function and absolute error to effectively improve the model's anti-interference ability when facing noisy data. The formula is as follows:

[0120]

[0121] Among them, c is a hyperparameter used to balance the noise filtering effect; α is used to balance the weights of different loss items, α is 0.2; c is 0.08.

[0122] Finally, the above three loss functions are weighted and combined to form a comprehensive multivariate nonlinear loss function. The formula is:

[0123] L combined =λ1·L amplitude +λ2·L phase +λ3·L SQL (30)

[0124] Among them, λ1, λ2 and λ3 are weight parameters used to balance the contribution of different loss terms. Here, they are taken as 0.3, 0.3 and 0.4 according to the experimental results.

[0125] To improve the prediction accuracy, the prediction steps constructed in this embodiment are as follows:

[0126] 1) Data preprocessing: The meteorological data in the historical wind power data is screened out through the Pearson correlation coefficient or the MIC correlation coefficient to select several meteorological features that are most correlated with wind power; the power data is first pre-cleaned to remove obvious abnormal data, and then based on the wind power image detection and cleaning algorithm, the pre-cleaned power data is first generated into a binary image, and then the maximum contour of the binary image under different structural elements b is extracted using equations (4) and (5), and the Hu values ​​of each binary image are compared using equations (8). Then, the dissimilarity between the 7 Hu moments of each binary image and the Hu moments of the reference WPC binary image is compared using equations (9) and (10), and the binary image with the smallest maximum dissimilarity among the 7 Hu moments is selected. This process is used to ensure that the extracted binary image is closest to the reference WPC binary image in terms of shape features. After the comparison results are obtained, the main part of the binary image is extracted using the corresponding structural element to screen out abnormal data outside the main part. After the normal data is extracted, the cleaned data is marked, and the mapping relationship between normal data and image is established using formula (11). Then, the normal data is marked using formula (12) to screen out all normal data. In the face of missing values ​​in the extracted data, cubic spline interpolation is used to interpolate the missing data to ensure time continuity.

[0127] 2) Feature processing: In order to better capture the trend and fluctuation of the time series, the average pooling operation is used to smooth the filtered meteorological series and the interpolated power series, remove the short-term noise of the series, extract a smoother trend component, and then subtract the extracted trend component from the original series to obtain the fluctuation component. At the same time, in order to keep the length of the decomposed series consistent, the boundary filling technology is used to fill the series.

[0128] 3) Patch TST model prediction: During prediction, the prediction models of trend component and residual component are established respectively; in order to speed up the convergence of the model and eliminate the influence of different dimensions on model training, the time series after feature processing is normalized, and the sequence is divided into several patches using formula (17). After the patch is divided, the channel independence strategy is adopted to process the input data of different meteorological characteristics separately to avoid mutual interference between features. Then, the position encoding of the sequence is generated using formula (18) and formula (19), and the position encoding is processed into a position encoding matrix using formula (20). The position information of the time series is embedded into the input sequence through formula (21), so that the model can capture the sequential information of the time step. Then, the embedded patch sequence is sent to the multi-head attention mechanism, and each attention head calculates the attention scores of different angles according to formula (24)-formula (25) to capture the global dependency. After that, the output and input of each attention module are connected through residual connection to prevent the gradient from disappearing. The output after residual connection is normalized to speed up the convergence of training and ensure the stability of the model. The normalized output is further transformed by the feedforward neural network according to formula (26) to enhance the model's learning of complex features. After that, the output sequence passes through a layer of residual connection and normalization layer, and the final output is flattened to reduce the multi-dimensional features to low-dimensional vectors. After projection through the fully connected layer in the linear head connection, the prediction value of the relevant component is obtained. Based on the prediction models established for the previous two components, the prediction results are added to obtain the final prediction value.

[0129] 4) Model optimization: In order to make the model better perceive the sequence shape changes in wind power prediction and resist noise interference, a multivariate nonlinear loss function is designed to optimize the parameters in Patch TST. The loss function consists of three parts, namely amplitude shift invariance loss, phase shift invariance loss and smooth quadratic loss. The amplitude shift invariance loss is established by formula (27), the phase shift invariance loss is established by formula (28), and the smooth quadratic loss is established by formula (29). These three losses are jointly optimized by linear weighting in formula (30). The optimization process of the entire loss function is implemented through back propagation and gradient descent algorithms to guide the parameter update changes of the model.

[0130] In order to illustrate the effect of the present invention, the following example analysis is performed:

[0131] The measured data of a wind farm were collected from January to December 2019. The data set includes wind power and local meteorological data, with a sampling interval of 15 minutes. The specific information of the data is shown in Table 1.

[0132] Table 1 Scene settings

[0133]

[0134] The total wind power data set contains not only wind power, but also multi-dimensional meteorological data, including wind speed, wind direction, temperature, and air pressure. Too many input features will not only make it difficult to fit the model during training, but also reduce the efficiency of model training. The correlation heat map of the features is shown in Figure 2. Figure 6 shown.

[0135] According to the correlation analysis results in Table 2, the wind speed, historical power, air pressure and temperature data with the highest correlation are considered as the input features of the model.

[0136] Table 2 Correlation analysis results

[0137]

[0138]

[0139] Considering the dimension of the data and the speed of parameter update in the deep learning back propagation process due to different data sizes, in order to ensure that the model can converge quickly, it is necessary to use the maximum-minimum normalization formula to normalize the data to the interval (0, 1):

[0140]

[0141] Where: x i,feature is the actual value of a certain type of feature in the original data; is the normalized value of the i-th feature of a certain type in the original data; min(x i,feature ) is the minimum value of a certain type of feature in the original data; max(x feature ) is the maximum value of a certain type of feature in the original data.

[0142] The preprocessed data features include wind speed (at the 10m wind tower), wind speed (at the 30m wind tower), wind speed (at the hub), wind direction (at the 10m wind tower) and wind power. The experimental data set is divided into training set, validation set and test set according to the ratio of 8:1:1 and input into the model. The experiment uses a fixed random seed, and the optimizer uses the Adam optimizer, with the initial value of the learning rate being 0.001.

[0143] The comparison model adopts the TCN-LSTM model, seq2seq model, Transformer model and Dlinear model which are widely used in time series prediction. The loss function is MSE, the learning rate is 0.001, the dropout is 0.2, and the epoch is 50. The seq2seq uses the encoder and decoder structure, in which the encoder and decoder both use LSTM. The embedding dimension and the number of network layers are known empirical parameters. To obtain other model parameters, the basic prediction parameters of the model are first searched by Bayesian optimization, where the batch number N batch ={32,48,64}, time step input L seq ={8,16,32}. Then, after fixing the basic parameters, the model network parameters are searched again, where the number of attention heads N head = {8, 16, 32}, the number of neurons in the hidden layer N hidden = {128, 256, 512}, the number of coding layers N layers ={2,3,4}. The parameters of the same network in the experiment are exactly the same. The final model parameters are shown in Table 3.

[0144] Table 3 Model parameter configuration table

[0145]

[0146]

[0147] In order to prevent overfitting and improve training efficiency, an early stopping strategy for model training was introduced during the training phase, that is, training was stopped when the loss did not decrease after five consecutive trainings.

[0148] In terms of prediction index evaluation, the MAE index is used MAE , Root Mean Square Error (RMSE) indicator RMSE And the mean absolute percentage error (MAPE) indicator e MAPE .

[0149]

[0150] In the formula, and Y h are the predicted value and true value of the hth test sample respectively; N test is the total number of test samples.

[0151] To verify the performance of the model, the following experiments are conducted:

[0152] 1) Model comparison experiment

[0153] Table 4 Comparison of multi-step prediction results (4 steps) of various models

[0154]

[0155] By comparing the prediction indicators in Table 4, it can be found that there are obvious differences in the various indicators of several prediction models. Among them, the Patch TST model proposed in the present invention shows a significant improvement in various error indicators compared with the comparison models. Due to the lack of complex feature extraction and feature enhancement, the performance of traditional machine learning TCN-LSTM is far inferior to other deep learning models; Attention-Seq2Seq, which adopts an encoder-decoder structure, introduces an attention mechanism, which can pass richer hidden information to the decoder. However, since the decoder depends on the output of the previous step at each moment, if the output of the previous step has an error, the subsequent errors will gradually amplify. This loop structure makes the prediction error larger; and the Transformer-based model, because of the self-attention mechanism, allows the model to pay attention to the entire sequence at each moment, reducing the accumulation of errors; finally, the Patch TST model, based on the Transformer model, uses local patches to represent features, replacing the single-point feature extraction method, improving the ability of local feature extraction, and thus achieving better prediction results. Taking the average e MAE Taking the Patch TST model as an example, the prediction error is 1.481MW, which is lower than the average e MAE The Dlinear model with the second best performance was reduced by 5.1%, and compared to the Transformer, it was reduced by 16.1%. Other indicators also improved to a certain extent, and the specific reduction or improvement ratios are as follows: RMSE The indicators decreased by 0.6%, 5.6%, 26.6% and 37.4% respectively. MAPE The indicators decreased by 78.9%, 62%, 49.6% and 23.2% respectively.

[0156] Figure 7 The following are the prediction results of different models (4-step prediction); the prediction result curves of different models are drawn to more intuitively show the prediction effect of the proposed method. By comparison, it can be seen that although the wind power fluctuation is large, the prediction results of the proposed Patch TST model are closely matched with the true value, and can continue to fit accurately in a relatively stable range. In addition, the model does not show the drastic changes that other models have at the turning point of power fluctuation, indicating that the model has better fitting effect and stability.

[0157] In order to more accurately and intuitively reflect the advantages of the model proposed in the present invention over other models in wind power prediction, 20% of the error results are randomly selected from the prediction results of all test sets for box plot analysis, such as Figure 8As shown. The line in the middle of the box in the box plot represents the median, reflecting the median value of the error. The lower the median, the smaller the overall prediction error of the model. The upper and lower boundaries of the box represent the first and third quartiles (Q1 and Q3). The smaller the IQR (Q3-Q1), the more concentrated the error distribution. Independent points represent outliers. The fewer outliers, the better the stability of the model prediction. Overall, Patch TST's prediction results for all indicators are better than other models.

[0158] 2) Loss function experiment

[0159] In order to verify the effectiveness of the loss function in multi-step prediction, multi-step prediction experiments of different scales were conducted. The experimental process also used a fixed random seed and the average value was taken from 7 experiments. The experimental results are recorded in Tables 5 and 6.

[0160] Table 5 Comparison of MAE indicators of multi-step prediction results

[0161]

[0162]

[0163] Table 6 Comparison of RMSE indicators of multi-step prediction results

[0164]

[0165] It can be seen from Tables 5 and 6 that the new loss function can effectively improve various error indicators when performing multi-step predictions at different time scales. Especially in the 3h prediction, the average e MAE Compared with the traditional MAE and mean square error (MSE) loss functions, the indicators are reduced by 1.9% and 1.2% respectively, and the average RMSE The indicators dropped by 1.9% and 2.3% respectively. The multi-output prediction strategy itself can capture the correlation between sequences, but in fact, for different prediction steps, the model has different requirements for the weights of local and global information. Therefore, after making corresponding adjustments using the new loss function, the fitting effect of the data can still be improved to varying degrees.

[0166] 3) Model ablation experiment

[0167] In order to verify the effectiveness of each module of Patch TST, an ablation experiment was conducted to remove the modules one by one. 1 Without using a new loss function, Patch TST 2 Without adopting new loss functions and channel-independent strategies, Patch TST 3No new loss function and patch block structure are used, while TST does not use new loss function, patch structure and channel independence strategy. The ablation experiment results are shown in Table 7. As can be seen from Table 7, the use of the new loss function, patch block and independent channel structure has a significant impact on the average e MAE The indicators are reduced by 6.5%. Compared with the basic TST model, after adding the patch block module and channel independence module, the average e RMSE and e MAPE The indicators decreased by 1.4% and 33.03% respectively.

[0168] Table 7 Ablation experiment results

[0169]

[0170] Compared with other models, the method proposed in the present invention can achieve the optimal prediction of ultra-short-term wind power multi-step prediction, and effectively improve the reliability of wind power prediction.

Claims

1. An ultra-short-term wind power prediction method based on an improved loss function and a patched temporal Transformer network, characterized in that: The following steps are involved: The image anomaly detection and cleaning algorithm of wind power data is used to detect and clean the power data; The correlation between meteorological data and power data is calculated by Pearson correlation coefficient or MIC correlation coefficient, and features with high correlation with wind power are extracted; The average pooling operation is used to process the features with high correlation with wind power, and the trend component is extracted. Then, the fluctuation component is obtained by subtracting the trend component from the original sequence. The Patch TST models of trend component and volatility component are established respectively, and the channel independence strategy is introduced to add the prediction results of the two components and reconstruct the final prediction value; The parameters of the Patch TST model are optimized using a multivariate nonlinear loss function; The effects of patch block structure, channel independence strategy and multivariate nonlinear loss function on the prediction results are compared and analyzed to verify the effectiveness of wind power prediction.

2. The ultra-short-term wind power prediction method based on the improved loss function and patch time series Transformer network according to claim 1 is characterized in that: The image anomaly detection and cleaning algorithm of wind power data is used to detect and clean the power data, specifically including: Pre-clean the wind power data, remove obvious abnormal data, generate a binary image from the pre-cleaned data, and then use erosion and dilation operations with different structural elements b to extract the maximum contour of the binary image; Calculate the Hu value of each binary image; Compare the dissimilarity between the 7 Hu moments of each binary image and the Hu moments of the reference WPC binary image, select the 7 binary images with the smallest Hu moment dissimilarity, and use them to extract image contours; The data outside the contour is marked, and a mapping relationship between normal data and images is established to filter out all normal data.

3. The ultra-short-term wind power prediction method based on improved loss function and patch time series Transformer network according to claim 1 is characterized in that: The establishment process of the Patch TST model is as follows: After normalizing the input sequence, it is divided into several possibly overlapping patches, and each patch sequence is regarded as a unit. Then the above patches are processed on each independent channel to preserve the local features in the time series. After position encoding, the patch sequence is input into the Transformer encoding layer; the encoding layer captures the global dependencies of the time series from a multi-dimensional perspective through a multi-head attention mechanism; finally, after flattening, the Transformer output vector is input into the linear prediction head to generate the final prediction result.

4. The ultra-short-term wind power prediction method based on improved loss function and patch time series Transformer network according to claim 1 is characterized in that: The multivariate nonlinear loss function includes: Amplitude translation invariance loss function in, and are the i-th predicted value and the i-th actual value respectively, and T is the length of the time series; Phase shift invariance loss function in, represents Fourier transform, ||·|| p YesL p norm; Smooth quadratic loss function Among them, c is a hyperparameter used to balance the noise filtering effect; α is used to balance the weights of different loss terms; Weighted combination L combined =λ1·L amplitude +λ2·L phase +λ3·L SQL Among them, λ1, λ2, and λ3 are weight parameters used to balance the contribution of different loss terms.

Images