A double-channel wind power prediction method integrated with squeeze-and-excitation attention mechanism

Abstract

The scheme discloses a double-channel wind power prediction method integrated with extrusion excitation attention mechanism, proposes wind power prediction based on multivariables, simultaneously considers endogenous variables and exogenous variables, effectively improves the performance of a model, and improves the prediction precision of the model; the prediction model mixes a CNN model and a GRU model, input data are processed by the two models in parallel, and SE attention mechanisms are introduced into the CNN model and the GRUB model respectively to give different probability weights to hidden states, so that the time sequence characteristic demand of wind power prediction can be met, overfitting problems can be avoided under the condition of multivariable input parameters, the model can more accurately predict the change of wind power under the time sequence of multivariable parameters, and the reasonable scheduling of power plants on electricity consumption is facilitated.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of wind power prediction, and proposes a method for predicting wind power generation by a hybrid model of a dual-channel convolutional neural network (CNN) and a gate control recurrent unit (GRU) integrated with a squeeze and excitation (SE) attention mechanism. BACKGROUND

[0002] In many countries around the world, renewable energy is gradually replacing traditional energy, mainly because fossil fuels and nuclear fuels will cause irreversible environmental damage, while the cost of renewable technology continues to decrease. Wind energy, as a renewable energy source, has the advantages of low cost and small environmental impact, and is an extremely high-quality clean energy. However, due to the uncertainty and variability of wind power generation, the integration of wind power generation and the power system faces several challenges. For example, short-term power generation scheduling of the power system must provide reliable estimates to obtain the best choice of dispatch units and the best power generation level of conventional units, but due to the greater influence of environmental factors on wind power generation, incorrect predictions can increase the cost of system power generation, and even may endanger the planning and economic operation of the power market, as well as the safety and reliability of the power system. Therefore, prediction tools that accurately describe and predict the behavior of wind power generation are essential to maintain the economy and reliability of power supply. Predicting the power of wind power generation can help equipment adjust power scheduling plans in a timely manner, develop appropriate power transmission schemes, and reduce equipment overload.

[0003] In recent years, more complex technologies in the field of artificial intelligence have been popularized. From the machine learning community, we can find suggestions for using different models, such as support vector machines, random forests, clustering, fuzzy logic, and artificial neural networks.

[0004] The prediction of wind power generation has obvious time series characteristics. At present, most power prediction methods are based on time series data sets for training and prediction. Therefore, when selecting a model architecture, a neural network that can dynamically model time series is usually selected. Although CNN network has very good application ability in other fields, CNN network is more suitable for capturing short-term local dependence, and its prediction ability for long sequence time is very limited. RNN recurrent neural network is better at capturing long-term dependencies and can dynamically model time series, but due to its own architectural limitations, the model needs a long training time. GRU (Gated Recurrent Unit) is also called gated recurrent unit structure, which is a variant of RNN. It can effectively capture the semantic association between long sequences, optimize the internal structure of RNN, alleviate the gradient vanishing or explosion phenomenon, and save the model training time. Therefore, many technologies currently use the GRU model for prediction, such as a wind power ultra-short-term power prediction method using a self-attention mechanism and a bilinear fusion [application number: CN202110199424.5] disclosed in a Chinese patent. However, first, at present, when predicting wind power generation, it is based on single variable (wind speed and wind direction) prediction. According to the applicant's research, adding the characteristics of the phenomenon studied can improve the model performance and improve the prediction accuracy of the model, so it is necessary to provide a power prediction scheme based on multiple variable parameters. Second, although the GRU model has good prediction ability for time series based on single variables, the prediction of time series based on multiple variables is not satisfactory, and the model is prone to overfitting. SUMMARY

[0005] The purpose of the present application is to solve the above problems, and to provide a dual-channel wind power prediction method integrating an extrusion excitation attention mechanism.

[0006] In order to achieve the above purpose, the following technical solutions are adopted in the present application:

[0007] A dual-channel wind power prediction method integrating an extrusion excitation attention mechanism, comprising:

[0008] A wind power prediction model based on dual channels is constructed, which includes a CNN network based on an attention mechanism and a GRU network based on an attention mechanism;

[0009] Prepare a data set composed of time series including power generation, component temperature, environmental temperature and wind speed; component temperature, environmental temperature and wind speed as input variables, and power generation as output variable, i.e. the true result corresponding to the prediction result;

[0010] Train the wind power prediction model using the data set:

[0011] The data set is respectively input into the CNN network based on attention mechanism and the GRU network based on attention mechanism;

[0012] The data Y output by the CNN network C and the data Y output by the GRU network G After splicing in the corresponding dimension, the full connection layer is input, and the full connection layer outputs the prediction result based on the spliced input result;

[0013] The wind power prediction model is updated according to the prediction result and the corresponding real result based on the loss function, and the above training steps are repeated until the model converges or reaches the maximum number of steps.

[0014] In the above-mentioned double-channel wind power prediction model prediction method integrating extrusion and excitation attention mechanism, the CNN network based on attention mechanism includes a CNN model and a first attention module;

[0015] The CNN model extracts features from the input time sequence and outputs the result H C ;

[0016] The first attention module excites and empowers the result H C output by the CNN model and outputs Yc.

[0017] In the above-mentioned double-channel wind power prediction model prediction method integrating extrusion and excitation attention mechanism, the constructed CNN model includes two convolution layers and one pooling layer, and uses a ReLU activation function for activation;

[0018] The output result of the CNN model is represented as:

[0019]

[0020] P1=max(C1)+b2

[0021]

[0022] In the formula, C1 and C2 represent the outputs of convolution layer 1 and convolution layer 2 respectively; P1 represents the output of the pooling layer; X is the input data; W1 and W2 are weight matrices; b1, b2 and b3 are bias coefficients.

[0023] In the above-mentioned double-channel wind power prediction model prediction method integrating extrusion and excitation attention mechanism, before the first attention module excites and empowers the result H C , the result H C is first data transformed and data compressed:

[0024] Data transformation: F tr :H C →U C =[uc1 ,u c2 ,…,u cN ],in: u cn It is the nth feature map obtained after calculating the input data;

[0025] Data compression: U C Generated by compression using spatial dimensions H×W The nth element is calculated using the following formula:

[0026] In the above-mentioned dual-channel wind power prediction model prediction method with integrated squeezing-incentivized attention mechanism, the first attention module performs the excitation and weighting process as follows:

[0027] S c =σ c (w2δ c (w1z c ))

[0028] In the formula: S c It is the feature map weight vector given by the first channel attention module, σ c and δ c Let w1 and w2 represent the Sigmoid and ReLU activation functions of the first channel attention module, respectively. Let z be the coefficients of the fully connected layers controlling model complexity. c =[z c1 , z c2 ,...,z cN ];

[0029] Subsequently, based on the obtained feature map vector u cn With weighting coefficient S cn ,S c =[S c1 ,S c2 ,...,S cN The final output value Y of the first attention module is obtained by multiplying the n=1,2,…,N numbers. cn Y c =[Y c1 ,Y c1 ,...Y cN ], n=1,2,…,N.

[0030] In the above-mentioned dual-channel wind power prediction model prediction method with integrated squeeze-excitation attention mechanism, the GRU network based on the attention mechanism includes a GRU model and a second attention module.

[0031] The GRU model extracts the internal variation patterns of the input time series data and outputs the result H. G ;

[0032] The second attention module stimulates and empowers the result H G output by the GRU model G .

[0033] In the double-channel wind power prediction model prediction method integrating the extrusion stimulation attention mechanism described above, the GRU model extracts the internal change rule of the data through the following formula:

[0034] H G = GRU (X t-1 , X t )

[0035] In the formula, X t-1 and X t are the inputs at the t-1th moment and the tth moment respectively.

[0036] In the double-channel wind power prediction model prediction method integrating the extrusion stimulation attention mechanism described above, before the second attention module stimulates and empowers the result H G , the result H G is first subjected to data transformation and data compression:

[0037] Data transformation: F tr : H G → U G = [u G1 , u G2 , …, u Gn ], wherein:

[0038] Data compression: U G is compressed by the spatial size HxW to generate wherein the nth element is calculated by the following formula:

[0039] In the double-channel wind power prediction model prediction method integrating the extrusion stimulation attention mechanism described above, the stimulation and empowerment process of the second attention module is as follows:

[0040] s G = σ G (w2δ G (w1z G ))

[0041] In the formula, s G is the feature map weight vector given by the second channel attention module, σ G and δ G represent the Sigmoid and ReLU activation functions respectively, w1 and w2 are the full connection layer coefficients for controlling the complexity of the model, and zG = [z G1 , z G2 ,..., z GN ], n = 1, 2,..., N;

[0042] Subsequently, the obtained feature map weight vector u Gn is multiplied with the weight coefficient S Gn , S G = [S G1 , S G2 ,..., S GN ], n = 1, 2,..., N to calculate the final output value Y of the second attention module Gn , Y G = [Y G1 , Y G1 ,... Y GN ], n = 1, 2,..., N.

[0043] In the above-mentioned double-channel wind power prediction model prediction method integrating extrusion excitation attention mechanism, the collected data is preprocessed as a data set for model training, and the preprocessing process is: the data is standardized,

[0044] In the formula, x' is the data after standardization, x is the original data without standardization, mu is the mean of the data, and sigma is the standard deviation of the data.

[0045] The loss function used by the wind power prediction model is mean square error:

[0046]

[0047] In the formula, y i represents the true result, represents the prediction result output by the wind power prediction model, and k represents the sample number.

[0048] The advantages of the present application are:

[0049] 1. The wind power prediction based on multivariate variables is proposed, which considers endogenous variables and exogenous variables, effectively improves the performance of the model, and improves the prediction accuracy of the model.

[0050] 2. The prediction model combines CNN and GRU models, with the two models processing the input data in parallel. The SE attention mechanism is introduced into the CNN and GRU models respectively to assign different probability weights to the hidden states. This not only meets the time series feature requirements of wind power generation prediction, but also ensures that overfitting is not a problem under multivariate input parameters. This allows the model to more accurately predict changes in wind power generation under multivariate parameter time series, which is beneficial for power plants to make reasonable scheduling of electricity consumption. Attached Figure Description

[0051] Figure 1 This is a flowchart of the model for the dual-channel wind power prediction method integrating the squeezing excitation attention mechanism of the present invention.

[0052] Figure 2 This is a comparison chart of prediction results and actual data in the dual-channel wind power prediction method integrating the squeeze-excitation attention mechanism of this invention. Detailed Implementation

[0053] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0054] like Figure 1 As shown, this scheme provides a dual-channel wind power prediction method integrating a squeeze-excitation attention mechanism, which includes the following steps:

[0055] 1) Obtaining parameter datasets: The operating status of wind turbine generators is affected by various environmental factors. Under normal operating conditions, parameters such as power generation, component temperatures, ambient temperature, and wind speed are measured and collected to form a time series.

[0056] 2) Standardize the parameter dataset using the following formula:

[0057]

[0058] In the formula, x′ is the standardized data, x is the unstandardized original data, μ is the mean of the data, and σ is the standard deviation of the data.

[0059] The standardized dataset is used to train the wind power prediction model. Training the wind power prediction model can actually be divided into two processes: the training process and the testing process. Therefore, the dataset is divided as follows: in this embodiment, the dataset is divided into a test set and a training set in a ratio of 3:17; when put into use, technicians can also divide it in other ratios based on experience.

[0060] 3) Establish and train a parallel model, as follows:

[0061] Building a CNN model: The CNN model is mainly used to extract features from the input time series. In order to preserve more data fluctuation information, the internal structure is constructed as two convolutional layers, one pooling layer, and the ReLU activation function is selected for activation. Among them, the output feature H c is expressed as:

[0062]

[0063] P1 = max(C1) + b2

[0064]

[0065] where C1 and C2 represent the outputs of convolutional layer 1 and convolutional layer 2, respectively; P1 represents the output of the pooling layer; X is the input data; W1 and W2 are weight matrices; b1, b2, and b3 are bias coefficients.

[0066] Building a GRU model: While inputting data X into the CNN model, the original input data X is simultaneously inputted into the GRU model for learning to extract the internal change rule of the data. Let the output of the GRU model be H G , whose formula is as follows:

[0067] H G = GRU(X t-1 , X t )

[0068] where X t-1 and X t are the inputs at the t-1th and tth moments, respectively.

[0069] 4) Attention mechanism processing

[0070] Data transformation: Transform the data of the CNN network channel, and transform the data H C processed by the CNN model:

[0071] F tr : H C → U C = [u c1 , u c2 , …, u cn ]

[0072] where: u cn is the nth feature map obtained by calculating the input data.

[0073] Transform the data of the GRU network channel, and transform the data H G processed by the GRU model:

[0074] Ftr :H G →U G =[u G1 ,u G2 ,…,u Gn ]

[0075] where: u Gn is the n-th feature map of the input data after calculation.

[0076] Data compression: In order to compress the global information, the global average pool is used to generate channel information. Generally, U C , U G are compressed by spatial dimensions HxW to generate The n-th element in the CNN network is calculated by the following formula:

[0077]

[0078] The n-th element in the GRU network is calculated by the following formula:

[0079]

[0080] Data excitation and empowerment: In order to fully compress the information in the operation to capture the correlation between channels, the CNN network and the GRU network respectively select the following formula for activation:

[0081] S c =σ c (w2δ c (w1z c ))

[0082] S G =σ G (w2δ G (w1z G ))

[0083] In the formula: S c is the feature map weight vector given by the attention module of the CNN network, here referred to as the first channel attention module, σ c and δ c represent the Sigmoid and ReLU activation functions of the first channel attention module respectively; S G is the feature map weight vector given by the attention module of the GRU network, here referred to as the second channel attention module, σ G and δ G represent the Sigmoid and ReLU activation functions respectively. w1 and w2 are the fully connected layer coefficients that control the complexity of the model, z c =[z c1 , z c2..., z cN ]z G = [z G1 , z G2 ..., z GN ], n = 1, 2,..., N.

[0084] Finally, based on the obtained feature map vector u cn and weight coefficient S cn , S c = [S c1 , S c2 ..., S cN ], n = 1, 2,..., N, the final output value Y cn of the first attention module is calculated, Y c = [Y c1 , Y c1 ,...Y cN ], n = 1, 2,..., N. Based on the obtained feature map weight vector u Gn and weight coefficient S Gn , S G = [S G1 , S G2 ..., S GN ], n = 1, 2,..., N, the final output value Y Gn of the second attention module is calculated, Y G = [Y G1 , Y G1 ,...Y GN ], n = 1, 2,..., N.

[0085] 5) Splicing processing

[0086] The data Y C output by the CNN channel and the data Y G output by the GRU model are spliced in the corresponding dimensions to obtain a new data matrix Y i , which is input into the full connection layer of the wind power prediction model.

[0087] 6) Weight update

[0088] The loss function used by the discharge prediction model is the mean square error:

[0089]

[0090] In the formula, y i represents the true result, The prediction result represents the output of the wind power prediction model, k represents the number of samples. Based on the foregoing loss function, the wind power prediction model is updated according to the deviation of the prediction result and the corresponding true result, and the above training steps are repeated until the model converges or reaches the maximum number of steps. The gradient descent method can be further preferred to optimize the parameters in the network for updating the wind power prediction model.

[0091] In order to better embody the advantages of the present scheme, the three prediction models of LSTM, GRU and SE-CNN-GRU which have been fully trained are tested, and the three prediction models are used to predict the wind turbine power generation power respectively, and the mean square error is used as the evaluation index

[0092]

[0093] The smaller the value of RMSE, the better the prediction effect of the model.

[0094] Table 1 gives the prediction results of the three models, and it can be clearly seen from the comparison results that the CNN-GRU model based on attention mechanism has better prediction effect. This fully shows that the CNN-GRU based on attention mechanism is a feasible and excellent method for wind power prediction.

[0095] Table 1 Comparison of wind turbine power prediction results

[0096]

[0097] The method uses the SE-CNN-GRU model formed by connecting the convolutional neural network and the gate control recurrent unit in parallel and adding the attention mechanism to predict the wind power generation power, which can accurately and quickly predict the change of the wind power generation power and is conducive to the reasonable scheduling of power plants.

[0098] The specific embodiments described herein are merely illustrative of the spirit of the present application. Those skilled in the art of the present application can make various modifications or supplements to the described specific embodiments or use similar ways to replace them, but will not deviate from the spirit of the present application or exceed the scope defined by the appended claims.

Claims

1. A dual-channel wind power prediction method integrated with squeeze-and-excitation attention mechanism, characterized in that, The application relates to a wind power prediction model based on a double-channel structure. The wind power prediction model comprises a CNN network based on an attention mechanism and a GRU network based on an attention mechanism. A data set is prepared, which comprises a time sequence of power generation, component temperature, ambient temperature and wind speed. The wind power prediction model is trained using the data set. The data set is input into the CNN network based on the attention mechanism and the GRU network based on the attention mechanism. Data Y output by the CNN network C Data Y output by the GRU network G After splicing in the corresponding dimensions, the full connection layer is input, and the full connection layer outputs a prediction result based on the spliced input result; The wind power prediction model is updated according to the predicted result and the corresponding real result based on a loss function, and the training steps are repeated until the model converges or the maximum number of steps is reached. The CNN network based on the attention mechanism comprises a CNN model and a first attention module. The CNN model comprises two convolution layers and one pooling layer, and is activated by using a ReLU activation function. The output result of the CNN model is represented as follows: In the formula, C1 and C2 represent the outputs of the first convolution layer and the second convolution layer respectively; P1 represents the output of the pooling layer; X is input data; W1 and W2 are weight matrices; b1, b2 and b3 are bias coefficients. The CNN model extracts features of the input time series and outputs a result H C ; To the result H C Data transformation and data compression are performed: Data transformation: wherein: , , is the i-th feature map of the input data after computation, n ;​ Data compression: By spatial size H × W Compression is performed to generate where the n first element is calculated by the following formula: ; The first attention module processes the results H C Incentivize and empower, output Yc: wherein: is a feature map weight vector given by the first channel attention module, and represent the Sigmoid and ReLU activation functions of the first channel attention module, respectively, and are fully connected layer coefficients that control the model complexity, respectively, ; Subsequently, based on the obtained feature map vector With weighting coefficients The final output value of the first attention module is obtained by multiplication. .

2. The integrated squeeze-and-excitation attention mechanism dual-channel wind power prediction method according to claim 1, characterized in that, The GRU network based on the attention mechanism comprises a GRU model and a second attention module. The GRU model extracts the internal change rule of data based on the time series of the input, and outputs the result H G ; The second attention module outputs a result H G Y after the incentive and empowerment G .

3. The integrated squeeze-and-excitation attention mechanism dual-channel wind power prediction method according to claim 2, characterized in that, The GRU model extracts the internal change rule of data through the following formula: In the formula, X t-1 and X t The first t -1 time and the t Input at any moment.

4. The integrated squeeze-and-excitation attention mechanism dual-channel wind power prediction method according to claim 3, characterized in that, The second attention module on the result H G Data transformation and data compression are performed on the result H G before the incentive and empowerment Data transformation: wherein: , , is the i-th feature map obtained after computation of the input data, n ​ Data compression: U G By spatial dimensions H × W Compression is performed to generate where the first n element is calculated by the following formula: .

5. The integrated squeeze-and-excitation attention mechanism dual-channel wind power prediction method according to claim 4, characterized in that, The excitation and weighting process of the second attention module is as follows: wherein: is a feature map weight vector given by the second channel attention module, and denote Sigmoid and ReLU activation functions, respectively, and are fully connected layer coefficients that control the model complexity, ; Subsequently, based on the obtained feature map weight vector and the weight coefficient are multiplied to calculate the final output value of the second attention module .

6. The integrated squeeze-to-excite attention mechanism dual-channel wind power prediction method according to any one of claims 1-5, characterized in that, The collected data is preprocessed as a data set for model training, and the preprocessing process is to standardize the data, In the formula, is the data after standardization processing, x is the original data without standardization processing, is the mean of the data, is the standard deviation of the data; The loss function used by the wind power prediction model is mean square error. wherein y i denotes the real outcome, denotes the predicted outcome output by the wind power prediction model, k denotes the number of samples.

Images