CNN-Transform-based short-term wind power prediction method

Abstract

The invention discloses a short-term wind power prediction method based on CNN-Transform, and the method comprises the steps: detecting an abnormal value in original wind power data through employing a box plot code, and carrying out the processing of the abnormal value through employing a spline interpolation method, and obtaining a preprocessed wind power data set; constructing a hybrid model comprising two encoders and a decoding layer; the two encoders are respectively a single convolutional layer of the CNN module and an encoder of the Transform module, and the decoding layer uses a single-layer neural network to replace an original decoder of the Transform module; performing model training on the hybrid model by using the preprocessed wind power data set; and inputting real-time wind power data into the trained hybrid model to obtain a wind power prediction value. According to the method, the abnormal value which influences model training and prediction in the original data set can be effectively detected, and on the premise that complex time sequence features and space features are effectively captured at the same time to make full use of diversity information in wind power data, the increase of model complexity is reduced to the maximum extent.

Description

Technical Field

[0001] The present invention relates to the field of wind power prediction, and in particular to, but not limited to, a short-term wind power prediction method based on CNN-Transformer. Background Art

[0002] Existing wind power forecasting methods mainly use deep learning technology. This method uses the power of algorithms to learn from data, providing a more detailed and adaptive forecasting method. Deep learning models such as convolutional neural networks (CNN), recurrent neural networks (RNN), long short-term memory networks (LSTM) and Transformer are good at capturing complex patterns and time dependencies in wind power data.

[0003] CNN can effectively extract local features from wind power data by using the convolutional layers in its nodes. LSTM can effectively utilize the extracted features and remember information for a long time, thereby capturing long-term dependencies that are critical for accurate prediction. Based on LSTM, scholars have innovated the structure of LSTM to form a bidirectional long short-term memory network BiLSTM, which further improves the prediction process by considering forward and reverse sequence data, thereby enhancing the model's ability to understand and predict the temporal dynamics of wind power. GRU is a deep learning method that simplifies the formation of model structure based on LSTM. The Transformer model was originally used in the field of natural language processing (NLP). Its core is the self-attention mechanism, which can effectively capture long-term dependencies. In recent years, Transformer has been introduced into the field of time series prediction to address the limitations of traditional neural networks in dealing with long-term dependencies. Using the self-attention mechanism of the Transformer model, the time-varying characteristics of wind power data can be captured globally, thereby improving prediction accuracy.

[0004] Although the above model methods have shown some excellent performance in wind power forecasting, they still have the following problems:

[0005] (1) LSTM, GRU, BiLSTM, and CNN usually rely on fixed network structures or fixed time steps when modeling complex temporal dependencies, and therefore perform poorly when dealing with frequently changing wind power data.

[0006] (2) Although Transformer effectively makes up for the shortcomings of CNN, LSTM, and GRU in processing complex time series data in the task of wind power prediction with its flexible self-attention mechanism, efficient parallel processing capability, global feature extraction capability, and excellent ability to capture long-distance dependencies, Transformer ignores the extraction of local features in wind power data, which makes Transformer unable to fully extract local features.

[0007] (3) In addition, due to the large number of parameters of Transformer, the time spent on model training and prediction by Transformer is greatly increased.

[0008] (4) The above models are all single models. When faced with the complex time series characteristics and spatial characteristics of wind power data, these single models cannot effectively capture these characteristics at the same time. In addition, since single models cannot fully utilize the diverse information in wind power data, the prediction accuracy of single models is not high. Summary of the invention

[0009] In view of this, an embodiment of the present invention provides a short-term wind power prediction method based on CNN-Transformer, which at least solves the problem that a single model cannot fully utilize the diverse information in wind power data, resulting in low prediction accuracy of the single model.

[0010] The technical solutions of the embodiments of the present invention are as follows:

[0011] An embodiment of the present invention provides a short-term wind power prediction method based on CNN-Transformer, the method comprising:

[0012] A box plot code is used to detect outliers in the original wind power data, and the outliers are processed using a spline interpolation method to obtain a preprocessed wind power data set; wherein the spline interpolation method fits data points through a polynomial function to generate a smooth curve; a hybrid model including two encoders and a decoding layer is constructed; the two encoders are a single convolutional layer of a CNN module and an encoder of a Transformer module, and the one decoding layer uses a single-layer neural network to replace the original decoder of the Transformer; the preprocessed wind power data set is used to train the hybrid model; and real-time wind power data is input into the trained hybrid model to obtain a wind power prediction value.

[0013] Furthermore, the use of the box plot code to detect abnormal values ​​in the original wind power data and the use of the spline interpolation method to process the abnormal values ​​includes: drawing a box plot for the original wind power data, marking the data outside the box in the box plot as abnormal values; extracting all normal values ​​from the original wind power data, using the index of the normal value as the independent variable and the observed value of the normal value as the dependent variable, and constructing an interpolation model through the spline interpolation method; for the abnormal value, calculating the predicted value at the corresponding index position according to the interpolation model, and replacing the abnormal value with the predicted value; after completing the abnormal value replacement, performing data visualization detection again to verify the interpolation result.

[0014] Furthermore, the preprocessed wind power data set includes 9 features, namely wind speed, wind direction, ambient temperature, instrument temperature, turbine direction, absolute air pressure at measurement point 1, absolute air pressure at measurement point 2, absolute air pressure at measurement point 3, and power.

[0015] Furthermore, the use of the preprocessed wind power data set to perform model training on the real-time hybrid model includes: in a single convolution layer of the CNN module, performing a one-dimensional convolution operation after shape conversion of the input preprocessed wind power data set, and converting the extracted local features back to a shape consistent with the input data through a sorting operation after ReLU activation; obtaining a query vector, a key vector, and a value vector from the result output by the CNN module through three linear transformations and then inputting them into the encoder of the Transformer module; using Dropout in the encoder of the Transformer module after processing through a self-attention mechanism and a feedforward neural network to prevent model overfitting; selecting the representation h of the last time step in the output of the encoder of the Transformer module T , and the h T The prediction result is obtained by decoding the single-layer neural network; a learning loss is constructed based on the prediction result and the true value corresponding to the preprocessed wind power data set to iteratively train the parameters of the two encoders and one decoding layer until convergence.

[0016] Furthermore, the method further comprises: using MSE, RMSE, MAE, R 2 A model evaluation is performed on the trained hybrid model.

[0017] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0018] In the data processing stage, the embodiment of the present invention proposes to use a box plot method to detect outliers in the original data, and use a spline interpolation method to replace outliers. In the model design stage, it is proposed to combine a convolutional layer of CNN with Transformer. On the basis of minimizing the increase in model complexity, this model retains the ability of Transformer to capture long-term dependencies, while increasing the ability to extract local features of wind power data, and can make full use of the diversity information in wind power data to improve prediction accuracy. In addition, in order to reduce the model complexity of the hybrid model, this method proposes to use a single-layer neural network as the decoder of the hybrid model, replacing the original decoder structure of Transformer. The number of model parameters is greatly reduced, so that the training and prediction time of the wind power prediction method is greatly reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0020] Figure 1 A schematic diagram of a flow chart of a short-term wind power prediction method based on CNN-Transformer provided in an embodiment of the present invention;

[0021] Figure 2 A logical architecture diagram of a short-term wind power prediction method based on CNN-Transformer provided in an embodiment of the present invention;

[0022] Figure 3 A box plot corresponding to a wind power data set provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0024] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0025] It should be pointed out that the terms "first\second\third" involved in the embodiments of the present invention are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present invention described here can be implemented in an order other than that illustrated or described here.

[0026] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the field to which the embodiments of the present invention belong. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0027] Figure 1 A flow chart of a short-term wind power prediction method based on CNN-Transformer provided in an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method comprises at least the following steps:

[0028] Step S110 , using a box plot code to detect outliers in the original wind power data, and using a spline interpolation method to process the outliers to obtain a preprocessed wind power data set.

[0029] Here, outliers in wind power data are usually caused by sensor failure, data transmission problems, extreme weather conditions, or human input errors. These outliers often deviate from normal values, such as Figure 2 As shown in the line graph of the original wind power data in the data processing part, it is shown as a sudden change of a certain data point in the line graph. The existence of outliers will reduce the training effect and prediction accuracy of the model.

[0030] The box plot can intuitively show the concentration trend and dispersion of data, and help identify outliers in wind power data. After the anomaly is identified, spline interpolation is used to process the outlier. The spline interpolation method fits the data points through a polynomial function to generate a smooth curve to capture the trend of the data.

[0031] Step S120, constructing a hybrid model including two encoders and one decoding layer; the two encoders are respectively a single convolutional layer of the CNN module and an encoder of the Transformer module, and the one decoding layer uses a single-layer neural network to replace the original decoder of the Transformer.

[0032] Here, if Figure 2 As shown in the module 2 part of the embodiment of the present invention, the hybrid model constructed by the embodiment of the present invention combines a convolutional layer of the CNN module with the Transformer, which retains the ability of the Transformer to capture long-term dependencies while minimizing the increase in model complexity, and at the same time increases the ability to extract local features of wind power data. In addition, in order to reduce the model complexity of the hybrid model, the method proposes to use a single-layer neural network as the decoder of the hybrid model, replacing the original decoder structure of the Transformer.

[0033] Step S130: Perform model training on the hybrid model using the preprocessed wind power data set.

[0034] Here, first, the preprocessed wind power data set is input into the CNN module for one-dimensional convolution processing to extract local and spatial features, and then the encoder of the Transformer module is used to capture the long-term dependencies in the data. Finally, the prediction results are output through the decoding layer, the loss function is calculated, and the parameters are optimized through back propagation to improve the prediction accuracy.

[0035] Step S140: input the real-time wind power data into the trained hybrid model to obtain a wind power prediction value.

[0036] Here, real-time wind power data refers to relevant data such as numerical weather forecasts collected in real time. The trained hybrid model is applied to the actual wind power prediction task, and the representation of the last time step is selected from the output of the Transformer encoder layer. The output of the last time step is decoded by a single-layer neural network to obtain the wind power prediction value corresponding to the real-time wind power data. The model can effectively extract local features and capture global dependencies, and finally output the prediction results.

[0037] In the data processing stage, the embodiment of the present invention proposes to use a box plot method to detect outliers in the original data, and use a spline interpolation method to replace outliers. In the model design stage, it is proposed to combine a convolutional layer of CNN with Transformer. On the basis of minimizing the increase in model complexity, this model retains the ability of Transformer to capture long-term dependencies, while increasing the ability to extract local features of wind power data, and can make full use of the diversity information in wind power data to improve prediction accuracy. In addition, in order to reduce the model complexity of the hybrid model, this method proposes to use a single-layer neural network as the decoder of the hybrid model, replacing the original decoder structure of Transformer. The number of model parameters is greatly reduced, so that the training and prediction time of the wind power prediction method is greatly reduced.

[0038] In some embodiments, the above step S110 further includes the following steps:

[0039] S111, drawing a box plot for the original wind power data, and marking the data outside the box in the box plot as an abnormal value;

[0040] S112, extracting all normal values ​​in the original wind power data, taking the index of the normal value as an independent variable and the observed value of the normal value as a dependent variable, and constructing an interpolation model by a spline interpolation method;

[0041] S113, for the abnormal value, calculating a predicted value at a corresponding index position according to the interpolation model, and replacing the abnormal value with the predicted value;

[0042] S114, after the outlier replacement is completed, data visualization detection is performed again to verify the interpolation result.

[0043] Here, the box plot describes the distribution of data through a five-number summary method: median, first quartile (Q1), third quartile (Q3), upper bound, and lower bound. The box of the box plot is defined by Q1 and Q3, and its length represents the middle 50% range of the data, that is, the interquartile range (IQR). Outliers are usually defined as data points that exceed the upper bound (Q3+1.5*IQR) or are lower than the lower bound (Q1-1.5*IQR). First, outliers in the data are detected by drawing a box plot. The outliers will be drawn outside the box in the box plot, such as Figure 3 Shown in the upper middle part.

[0044] Then after identifying the outliers, extract all the data points of the normal values. These normal values ​​will be used to build the interpolation model. The interpolation model is built by the spline interpolation method using the index of the normal values ​​as the independent variable (x) and the observed value of the normal value as the dependent variable (y).

[0045] For each value marked as abnormal, the predicted value is calculated at the corresponding index position according to the interpolation model. The predicted value replaces the original abnormal value to obtain a more reasonable new value that conforms to the data distribution. After the abnormal value replacement is completed, the data visualization is performed again to check whether the abnormal value replacement is completed, such as Figure 3 It can be seen that after the wind power data is processed by the proposed method, there are no more deviated data and drastic data points.

[0046] In some embodiments, the preprocessed wind power data set includes 9 features, namely wind speed, wind direction, ambient temperature, instrument temperature, turbine direction, absolute air pressure at measurement point 1, absolute air pressure at measurement point 2, absolute air pressure at measurement point 3, and power.

[0047] Here, the polynomial functions are established as follows:

[0048] Wind speed: S Wspd,i (x) = a Wspd,i +b Wspd,i (xx i )+c Wspd,i (xx i ) 2 +d Wspd,i (xx i ) 3 ; S Wspd,i (x) is the wind speed characteristic, a Wspd,i is a constant, b Wspd,i 、c Wspd,i d Wspd,i are the linear term coefficient, the quadratic term coefficient, and the cubic term coefficient respectively;

[0049] Wind direction: S Wdir,i (x) = a Wdir,i +b Wdir,i (xx i )+c Wdir,i (xx i ) 2 +d Wdir,i (xx i ) 3 ; S Wdir,i (x) is the wind direction characteristic, a Wdir,i is a constant, b Wdir,i 、c Wdir,i d Wdir,i are the linear term coefficient, the quadratic term coefficient, and the cubic term coefficient respectively;

[0050] Ambient temperature: S Etmp,i (x) = a Etmp,i +b Etmp,i (xx i)+c Etmp,i (xx i ) 2 +d Etmp,i (xx i ) 3 ; S Etmp,i (x) is the environmental characteristic, a Etmp,i is a constant, b Etmp,i 、c Etmp,i d Etmp,i are the linear term coefficient, the quadratic term coefficient, and the cubic term coefficient respectively;

[0051] Instrument temperature: S Itmp,i (x) = a Itmp,i +b Itmp,i (xx i )+c Itmp,i (xx i ) 2 +d Itmp,i (xx i ) 3 ; S Itmp,i (x) is the instrument temperature characteristic, a Itmp,i is a constant, b Itmp,i 、c Itmp,i d Itmp,i are the linear term coefficient, the quadratic term coefficient, and the cubic term coefficient respectively;

[0052] Engine direction: S Ndir,i (x) = a Ndir,i +b Ndir,i (xx i )+c Ndir,i (xx i ) 2 +d Ndir,i (xx i ) 3 ; S Ndir,i (x) is the turbine direction characteristic, a Ndir,i is a constant, b Ndir,i 、c Ndir,i d Ndir,i are the linear term coefficient, the quadratic term coefficient, and the cubic term coefficient respectively;

[0053] Absolute air pressure 1: S Pab1,i (x) = a Pab1,i +b Pab1,i (xx i )+c Pab1,i (xx i ) 2 +d Pab1,i (xx i ) 3 ; S Pab1,i(x) is the absolute air pressure at test point 1, a Pab1,i is a constant, b Pab1,i 、c Pab1,i d Pab1,i are the linear term coefficient, the quadratic term coefficient, and the cubic term coefficient respectively;

[0054] Absolute air pressure 2: S Pab2,i (x) = a Pab2,i +b Pab2,i (xx i )+c Pab2,i (xx i ) 2 +d Pab2,i (xx i ) 3 ; S Pab2,i (x) is the absolute air pressure at test point 1, a Pab2,i is a constant, b Pab2,i 、c Pab2,i d Pab2,i are the linear term coefficient, the quadratic term coefficient, and the cubic term coefficient respectively;

[0055] Absolute air pressure 3: S Pab3,i (x) = a Pab3,i +b Pab3,i (xx i )+c Pab3,i (xx i ) 2 +d Pab3,i (xx i ) 3 ; S Pab3,i (x) is the absolute air pressure at test point 1, a Pab3,i is a constant, b Pab3,i 、c Pab3,i d Pab3,i are the linear term coefficient, the quadratic term coefficient, and the cubic term coefficient respectively;

[0056] Power: S Patv,i (x) = a Patv,i +b Patv,i (xx i )+c Patv,i (xx i ) 2 +d Patv,i (xx i ) 3 ; S Patv,i (x) is the power characteristic, a Patv,i is a constant, b Patv,i 、c Patv,i d Patv,i are the linear term coefficient, quadratic term coefficient, and cubic term coefficient respectively.

[0057] In some embodiments, the above step S130 “using the pre-processed wind power data set to perform model training on the real-time hybrid model” can be implemented by the following steps:

[0058] S131, in a single convolution layer of the CNN module, performing a one-dimensional convolution operation on the input preprocessed wind power data set after shape conversion, and converting the extracted local features back to a shape consistent with the input data through a sorting operation after ReLU activation;

[0059] S132, the result output by the CNN module is transformed into a query vector, a key vector, and a value vector through three linear transformations, and then input into the encoder of the Transformer module;

[0060] S133, in the encoder of the Transformer module, Dropout is used after self-attention mechanism and feed-forward neural network processing to prevent model overfitting;

[0061] S134, select the representation h of the last time step in the output of the encoder of the Transformer module T , and the h T The prediction result is obtained by decoding the single-layer neural network;

[0062] S135, constructing a learning loss based on the prediction result and the true value corresponding to the preprocessed wind power data set to iteratively train the parameters of the two encoders and one decoding layer until convergence.

[0063] Here, if Figure 2 As shown in the module 2 section, in the CNN module, the input data is processed by a one-dimensional convolution layer. First, the input data is represented as a matrix X with a shape of (N, T, C), where N is the batch size, T is the number of time steps, and C is the feature dimension of each time step. In order to adapt to the one-dimensional convolution operation, the embodiment of the present invention converts the shape of the input data from (N, T, C) to (N, C, T). Next, a convolution operation is performed. For a one-dimensional convolution, the input signal is x(t) and the convolution kernel is w. The formula for the output y(t) of the convolution operation is:

[0064]

[0065] Where x(t) is the value of the input sequence at time step t, w(i) is the weight of the convolution kernel at position i, K is the size of the convolution kernel, and y(t) is the output of the convolution operation at time step t. The output after the convolution operation is nonlinearly transformed by the ReLU activation function f(x) = max(0,x). Finally, the output of the convolution layer is activated by ReLU and then converted back to (N, T, C) through a sorting operation for input into the Transformer module.

[0066] The Transformer encoder layer consists of a multi-head self-attention mechanism and a feed-forward neural network. The input data is the data X output from the CNN module, and its shape is (N, T, C). In order to adapt to the input shape of the Transformer, the input X first undergoes three linear transformations to obtain the query vector Q, key vector K, and value vector V:

[0067] Q=XW Q ,K=XW K ,V=XW V ;

[0068] Among them, W Q , W K , W V is the weight matrix of the linear transformation.

[0069] Next, the model is implemented by formula The attention score is calculated to capture the global dependencies and comprehensively consider the entire information of the input sequence. k is the dimension of the key vector (Key). For the multi-head attention mechanism, the formula is:

[0070] MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O ;

[0071] The calculation formula for each head is in is the linear transformation weight of each head, W O is the output weight matrix.

[0072] The feedforward neural network in the Transformer encoder layer consists of two linear transformations and a nonlinear activation function (ReLU), which can be expressed as FFN(x)=ReLU(xW1+b1)W2+b2.

[0073] Among them, W1, b1, W2, b2 are the weights and biases of the feedforward neural network. To ensure stability, layer normalization is applied, and its formula is LayerNorm(x+Sublayer(x)), where Sublayer(x) is the output of the self-attention mechanism and the feedforward neural network.

[0074] Dropout is used after the self-attention mechanism and feedforward neural network to prevent the model from overfitting. The formula is as follows:

[0075]

[0076] Among them, p is the probability of dropout.

[0077] In the previous Transformer-based prediction model, the prediction result is output through the original Transformer decoder, but the original Transformer decoder will increase the complexity of the model and increase the training and prediction time of the model. To solve this problem, this method uses a single-layer neural network as a new decoder to replace the original Transformer decoder. This innovation will greatly reduce the number of model parameters and reduce the model training and prediction time. The specific steps for predicting power output are as follows:

[0078] Select the representation h of the last time step from the output of the Transformer encoder layer T The output of the last time step is decoded by a single-layer neural network to obtain the prediction result. The formula is:

[0079] y=h T W+b;

[0080] where h T is the representation of the last time step, W and b are the weight and bias of the single-layer neural network, and y is the final prediction result. Through these steps and formulas, the hybrid model can effectively extract local features and capture global dependencies, and finally output the prediction result.

[0081] In some embodiments, the method further comprises: using MSE, RMSE, MAE, R 2 A model evaluation is performed on the trained hybrid model.

[0082] Here, in order to verify the performance superiority of the wind power prediction method proposed in the present invention, on the same wind power data set processed by the box plot, the embodiment of the present invention designed multiple groups of comparative experiments, including comparison with the current mainstream single models (LSTM, BiLSTM, GRU, Transformer), comparison with existing combination models (CNN-GRU, CNN-LSTM, CNN-BiLSTM), and comparison with CNN-Transformer without replacing the decoder.

[0083] 1) Comparative test with a single model:

[0084] The experimental results are shown in Table 1. In comparison with a single model, Transformer has the smallest prediction error in a single model, proving the correctness of using Transformer as the main prediction method in the present invention. At the same time, it can be seen that the wind power prediction method based on CNN-Transformer proposed in the present invention is superior to several other currently mainstream single prediction methods and has the lowest prediction error.

[0085] Table 1 Error comparison between the single model baseline and the method proposed in this invention

[0086] Comparison Models MSE RMSE MAE <![CDATA[R 2 ]]> BiLSTM 0.0035 0.0591 0.0401 0.9487 GRU 0.0011 0.0337 0.0197 0.9689 LSTM 0.0011 0.0327 0.0213 0.9707 Transformer 0.0010 0.0316 0.0312 0.9812 Hybrid Model of the Present Invention 0.0003 0.0168 0.0125 0.9923

[0087] On the basis of retaining the excellent performance of Transformer, a single convolutional layer in CNN is introduced to further enhance the prediction performance of Transformer and make up for the problem of insufficient local feature extraction ability of Transformer. The proposed model method has a mean square error (MSE) of 0.0003, a root mean square error (RMSE) of 0.0168, a mean absolute error (MAE) of 0.0125, and a determination coefficient R 2 Reached 0.9923.

[0088] 2) Comparison with existing combination models:

[0089] Table 2 shows the evaluation results of all baseline models constructed using the combined model and the CNN-Transformer based hybrid model proposed in the present invention. It can be seen that the hybrid model proposed in the present invention has achieved the best results in terms of prediction accuracy and error.

[0090] Table 2 Error comparison between the combined model baseline and the proposed method

[0091] Comparison Models MSE RMSE MAE <![CDATA[R 2 ]]> CNN-GRU 0.0023 0.0479 0.0301 0.9664 CNN-BiLSTM 0.0011 0.0329 0.0223 0.9703 CNN-LSTM 0.0007 0.0269 0.0180 0.9802 Hybrid Model of the Present Invention 0.0003 0.0168 0.0125 0.9923

[0092] In comparison with the existing combined models, the proposed method still has the lowest prediction error, proving that the proposed method still has certain performance advantages in the hybrid prediction method. In addition, combining Table 1 and Table 2, it can be seen that the introduction of CNN further enhances the local feature extraction capability of Transformer, making up for the inherent defects of Transformer.

[0093] It is worth noting that, unlike other hybrid models built on CNN, which require the introduction of multiple layers of CNN to achieve the effect of improving model accuracy, this method only introduces a single-layer convolutional neural network CNN to achieve the same prediction effect as introducing multiple layers of CNN. Since only a single-layer CNN is introduced, this hybrid method minimizes the increase in model complexity.

[0094] 3) Comparison with CNN-Transformer, which replaces the original decoder:

[0095] The huge number of parameters in Transformer greatly increases the training and prediction time of the prediction model. To address this problem, this method modifies the decoder based on the encoder and decoder of the traditional Transformer, and uses a simple single-layer neural network to replace the original decoder structure of the Transformer. The single-layer neural network is used to directly map the prediction results, which greatly reduces the number of parameters in the model and greatly reduces the training and prediction time of the prediction method. The experimental results are shown in Table 3.

[0096] Table 3 Comparison of different decoders

[0097]

[0098] It can be seen that using a single-layer neural network instead of the Transformer decoder structure greatly reduces the complexity of the model and reduces the prediction and training time of the model. The training time of the model is reduced from the original 1105.36s to 574.33s, a reduction of 48%. The prediction time is reduced from the original 0.40s to 0.23s, a reduction of 42.5%.

[0099] In order to verify the detection of outliers by box plots and the influence of replacing outliers by spline interpolation on the prediction accuracy of the model, a comparative test was conducted on the data processing method based on the method proposed in the present invention, and the test results are shown in Table 4:

[0100] Table 4 Comparison of model performance before and after data processing

[0101] Dataset MSE RMSE MAE <![CDATA[R 2 ]]> Raw wind power data 0.0036 0.0598 0.0400 0.9474 Box plot data 0.0003 0.0168 0.0125 0.9923

[0102] It can be seen that using box plots to detect outliers and using spline interpolation to replace outliers can effectively improve the training effect and prediction accuracy of the prediction model. The unprocessed original wind power data was input into the proposed model method for training, and the error experimental results obtained were MSE of 0.0036, RMSE of 0.0598, MAE of 0.0400, and R 2 It is 0.9474.

[0103] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0104] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0105] In the several embodiments provided by the present invention, it should be understood that the disclosed methods can be implemented in other ways. The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments. The features disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0106] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A short-term wind power prediction method based on CNN-Transformer, characterized in that: include: Using a box plot code to detect outliers in the original wind power data, and using a spline interpolation method to process the outliers to obtain a preprocessed wind power data set; wherein the spline interpolation method fits data points through a polynomial function to generate a smooth curve; Construct a hybrid model including two encoders and one decoding layer; the two encoders are a single convolutional layer of the CNN module and an encoder of the Transformer module, and the one decoding layer uses a single-layer neural network to replace the original decoder of the Transformer; Using the preprocessed wind power data set to perform model training on the hybrid model; The real-time wind power data is input into the trained hybrid model to obtain the wind power prediction value.

2. The method according to claim 1, characterized in that The method of using the box plot code to detect abnormal values ​​in the original wind power data and using the spline interpolation method to process the abnormal values ​​includes: Draw a box plot for the original wind power data, and mark the data outside the box in the box plot as abnormal values; Extracting all normal values ​​from the original wind power data, taking the index of the normal value as an independent variable and the observed value of the normal value as a dependent variable, and constructing an interpolation model by a spline interpolation method; For the abnormal value, a predicted value is calculated at a corresponding index position according to the interpolation model, and the abnormal value is replaced by the predicted value; After the outlier replacement is completed, data visualization is performed again to verify the interpolation results.

3. The method according to claim 1, characterized in that The preprocessed wind power data set contains 9 features, namely wind speed, wind direction, ambient temperature, instrument temperature, turbine direction, absolute air pressure at measurement point 1, absolute air pressure at measurement point 2, absolute air pressure at measurement point 3, and power.

4. The method according to claim 1, characterized in that The method of using the preprocessed wind power data set to perform model training on the real-time hybrid model includes: In a single convolutional layer of the CNN module, the input preprocessed wind power data set is transformed in shape and then subjected to a one-dimensional convolution operation, and the extracted local features are activated by ReLU and then transformed back to a shape consistent with the input data through a sorting operation; The output of the CNN module is transformed into query vector, key vector, and value vector through three linear transformations and then input into the encoder of the Transformer module. In the encoder of the Transformer module, Dropout is used after processing by the self-attention mechanism and feedforward neural network to prevent the model from overfitting; Select the representation h of the last time step in the output of the encoder of the Transformer module T , and the h T The prediction result is obtained by decoding the single-layer neural network; A learning loss is constructed based on the prediction result and the true value corresponding to the preprocessed wind power data set to iteratively train the parameters of the two encoders and one decoding layer until convergence.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: The mean square error (MSE), standard error (RMSE), mean absolute error (MAE), and determination coefficient (R) are used respectively. 2 A model evaluation is performed on the trained hybrid model.

Images