A wind power prediction method based on multi-scale attention mechanism

By introducing a multi-scale attention mechanism and a hybrid cyclic neural network model of bidirectional GRU layer in the prediction of wind power, the problem of failure to effectively capture timing correlation characteristics in the prior art is solved, and the prediction accuracy of wind power is significantly improved.

CN113642254BActive Publication Date: 2024-11-05NAT UNIV OF DEFENSE TECH
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202111021126.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-01
Publication Date
2024-11-05
Estimated Expiration
2041-09-01

AI Technical Summary

Technical Problem

The existing wind power prediction methods fail to effectively capture the timing correlation characteristics, resulting in poor prediction accuracy.

Method used

Adopt a hybrid recurrent neural network model based on multi-scale attention mechanism, and by introducing multi-scale attention layer and bidirectional GRU layer, combined with convolutional operations, different scale information in historical meteorological data are captured to enhance the accuracy of the model's prediction of wind power generation power sex.

Benefits of technology

It improves the accuracy of wind power generation power prediction, can collect historical information more effectively from different scales, distinguish the impact of different input elements on the prediction results, and enhances the accuracy of prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113642254B_ABST
    Figure CN113642254B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for predicting wind power generation based on a multi-scale attention mechanism, comprising the following steps: constructing a hybrid recurrent neural network model based on a multi-scale attention mechanism; training the hybrid recurrent neural network model using training set data, wherein the training set data includes influencing factor data and known wind power generation data; inputting the test set data into the trained hybrid recurrent neural network model to calculate and obtain the predicted value of wind power generation. The present invention comprehensively considers multiple factors affecting wind power generation from multiple aspects, introduces an attention mechanism into a convolutional recurrent GRU network, collects historical meteorological data information at different scales for multiple times for convolution, and achieves better results in predicting wind power generation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of wind power generation, and in particular relates to a wind power generation power prediction method based on a multi-scale attention mechanism. Background Art

[0002] As wind power technology matures, the capacity of wind power units and the scale of grid-connected wind farms continue to expand, and the proportion of wind power in the total power generation of the power system is also increasing year by year. The penetration power of wind farms continues to increase, and a series of problems brought to the power system are becoming increasingly prominent, seriously threatening the safe, stable, economical and reliable operation of the power system. Timely and accurate prediction of wind power can significantly enhance the safety, stability, economy and controllability of the power system. In recent years, many scholars have conducted extensive research on wind power prediction methods. Some literature has reviewed various wind power prediction methods and summarized the classification of wind power prediction methods, but has not introduced the combined prediction method and regional prediction method in detail, and has not clearly given the advantages and disadvantages of each method and the scope of application, and there are still certain limitations.

[0003] At present, the prediction accuracy of wind power generation is still insufficient. In summary, there are four different categories of methods: regression-based methods, support vector machine (SVM, SVR)-based methods, recurrent neural network (LSTM, GRU)-based methods, and ensemble learning-based methods. The regression-based prediction method is to establish a regression equation between these variables and the predicted value by analyzing the correlation between weather, meteorological and power generation data. SVM is a generalized linear classifier. It tries to find a hyperplane to divide the samples into different categories. The principle of segmentation adopts the principle of maximizing the interval between different classes, and finally transforms the original problem into a convex quadratic programming problem. Ensemble learning methods (ELM) are widely used in classification and regression tasks. ELM uses different classification or regression methods (fuzzy logic, decision tree, Boosting and random forest, etc.) to capture the distribution of the original training samples, construct multiple different classifiers, and linearly combine these classifiers to obtain a more powerful final decision-making ability. Recently, some scholars have adopted methods based on recurrent neural networks (RNN, LSTM, GRU, Bi-LSTM, Bi-LSTM, etc.) to predict power generation and achieved good results. However, both the traditional machine learning methods and the deep learning methods that have become popular in recent years have not captured the correlation characteristics between various factors in terms of time series, but have focused on certain features, resulting in less than ideal prediction accuracy. Summary of the invention

[0004] In view of this, to solve the problem of accurate prediction of wind power generation, the purpose of the present invention is to provide a wind power generation prediction method based on a multi-scale attention mechanism. The method comprehensively considers multiple factors affecting wind power generation from multiple aspects, introduces the attention mechanism into the convolutional recurrent GRU network, and performs convolution on historical meteorological data information collected multiple times at different scales, so as to achieve good results in the prediction of wind power generation.

[0005] Based on the above purpose, a wind power generation prediction method based on a multi-scale attention mechanism is proposed, including the following steps:

[0006] Step 1, construct a hybrid recurrent neural network model based on a multi-scale attention mechanism;

[0007] Step 2, use the training set data to train the hybrid recurrent neural network model, and the training set data includes influencing factor data and known wind power generation data;

[0008] Step 3, input the test set data into the trained hybrid recurrent neural network model, and calculate the predicted value of the wind power generation.

[0009] Specifically, the hybrid neural network model includes a first convolutional layer, a first bidirectional GRU layer, a first multi-scale attention layer, a second bidirectional GRU layer, a second multi-scale attention layer, a second convolutional layer, and a fully connected layer. Each layer is sequentially connected in order. The output of the first convolutional layer and the output of the first bidirectional GRU layer are connected and used as the input of the first multi-scale attention layer and the second multi-scale attention layer at the same time. The bidirectional GRU layer is formed by paralleling a forward GRU model and a backward GRU model to form a bidirectional structure. The bidirectional GRU layer outputs two merged GRU signals. The output layer of the fully connected layer is 1. The input of the first convolutional layer in the hybrid neural network model is the influencing factor data sequence, and the output of the fully connected layer is the wind power generation value.

[0010] Specifically, for the hybrid neural network model x0,...,x T is the influencing factor sequence data, (y0,...,y K ), K < T is the known wind power generation value, (y K+1 ,...,y T ) is the wind power generation value to be predicted, is the corresponding estimated value. The input is x0,...,x T , y0,...,y K variables, and input them into the hybrid neural network model in turn to start training. The loss function uses the standard normalized MSE, and the activation function uses the Relu function.

[0011] Specifically, the analytical expression of the hybrid neural network model is as follows:

[0012]

[0013]

[0014]

[0015]

[0016]

[0017]

[0018] C 4 t =η 2 ([C 2 t ,C 3 t ])

[0019]

[0020] Among them, x t is the input of the model at time t, η 1 (·) and η 2 (·) is two convolution operations, [·,·] is a merge connection operation, MutiScalAttention(·,ScaleNum1) and MutiScalAttention(·,ScaleNum2) are attention collection operations of two scales, ScaleNum1 and ScaleNum2. A bidirectional GRU layer is required as input before each attention layer, and a series structure is adopted between them. The specific process of fusion convolution is as follows:

[0021] The first convolutional layer η 1 (x t ) accepts sequence data x t The input is It also serves as the input of the first and second multi-scale attention layers;

[0022] is the output of the first bidirectional GRU layer, Indicates that the output of the forward GRU With the backward GRU output Perform a merge join; is to multiply the weight vector by the first bidirectional GRU layer. And add the offset vector Result;

[0023] Will With η 1 (x t ) Merge into P t 1 , as the input of the first multi-scale attention layer;

[0024] is the output of the attention layer of the first multi-scale attention layer with scale ScaleNum1, connected to the second bidirectional GRU layer;

[0025] is the output of the second bidirectional GRU layer, Indicates that the output of the forward GRU With the backward GRU output Perform a merge join; is to multiply the weight vector by the second bidirectional GRU layer And add the offset vector Result;

[0026] By analogy, we can get Then, the results of the two multi-scale attention models are fused through a convolution operation, and the outputs of different scales are screened and identified, so that the scale information that is more important to the target can be retained, and the output is obtained. After a full connection operation, the output O is obtained t ;

[0027] in, and All of them are acquired through learning and training.

[0028] Specifically, an improved multi-scale attention model is adopted in the first multi-scale attention layer and the second multi-scale attention layer, and the structure is as follows: t =Dense((W d ·x' t +B d ), units = ScaleNum), f t =1D-expand_dims(d t ), o t =attention(f t ), the multi-scale attention model is divided into two steps. First, the scale information is generated by using the output unit of the fully connected layer at the input variable. The scale size is represented by ScaleNum, so as to preprocess the input model into a two-dimensional vector d with a fixed scale. t; Then use the one-dimensional expansion mechanism 1D-expand_dims to convert the two-dimensional vector d t Expanded into a three-dimensional vector f t , so that the multi-scale attention model can expand and recognize the input information at a fixed scale, x' t is the input variable, W d and B d is the parameter vector to be learned, Dense() represents the fully connected operation, o t =attention(f t ), including the following two steps: a t =V a Softmax(W a f t T +B a ), Among them, f t T Yes t The transpose of V a , W a and B a is the parameter vector that needs to be learned. The first step is to output a through SoftMax t , the first t Transpose and multiply the input f t As the input for the next step, W b and B b is the learning parameter of the fully connected layer output, and the model process of the multi-scale attention layer is recorded as o t =MutiScalAttention(f t ,ScaleNum).

[0029] The bidirectional GRU layer is a bidirectional structure formed by connecting the forward GRU and the backward GRU models. The output of the first layer forget gate in the forward GRU is: 1 t =σ(W 1 f [H 1 t-1 ,x” t ]+B 1 f ),σ(x)=1 / (1+e -x ), in the forward GRU update gate, the first output is: z 1 t =σ(W 1 z [H 1 t-1 ,x” t ]+B 1z ), the second output is: Similarly, the corresponding first output of the backward GRU is: z 2 t =σ(W 2 z [H 2 t-1 ,x” t ]+B 2 z ), the second output: Intermediate output of the forward GRU And the intermediate output of the backward GRU By aggregating the intermediate outputs, we get the output Indicates that the output of the forward GRU With the backward GRU output H t 2 Merge the connection as the output of the bidirectional GRU layer x” t is the input of the bidirectional GRU layer, [W 1 f ,B 1 f ],[W 1 Z ,B 1 Z ],[W 1 h ,B 1 h ] is the forward GRU model parameter, [W 2 f ,B 2 f ],[W 2 Z ,B 2 Z ],[W 2 h ,B 2 h ] is the reverse GRU model parameter, [W 12 o ,B 12 o ] are the output layer parameters.

[0030] Preferably, the convolutional layer is a 1-dimensional convolutional network.

[0031] Specifically, the influencing factor data include wind speed, wind direction, temperature, humidity, air pressure, hourly rainfall, wind speed extreme difference, wind speed standard deviation, wind direction extreme difference, wind direction standard deviation, temperature extreme difference, temperature standard deviation, humidity extreme difference, humidity standard deviation, air pressure extreme difference and air pressure standard deviation.

[0032] The hybrid recurrent neural network model in the method of the present invention mainly includes a multi-scale attention layer, a bidirectional GRU layer and a fully connected layer. The multi-scale attention layer is introduced into the recurrent neural network, and the improved multiple attention units are connected in series to distribute the attention mechanism at different scales, so that the model can collect historical information from different scales and distinguish the influence of different input elements on the prediction results. The bidirectional GRU layer can more fully obtain the context information of the sequence data on the basis of the GRU. The whole model adopts a convolution structure to fuse the recognition outputs of the attention mechanisms of different scales, and screens and identifies the outputs of different scales through convolution connections, so that the scale information that is more important to the target can be retained, thereby obtaining better accuracy for the prediction of wind power generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of the process of the present invention;

[0034] Figure 2 is a data processing flow chart of the method of the present invention;

[0035] Figure 3 Schematic diagram of a 1-dimensional convolutional network in the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] When using historical meteorological data to predict wind power generation, it is necessary to analyze the factors that may affect wind power. Research has shown that it is at least related to historical wind speed, wind direction, temperature, humidity, air pressure, wind speed difference, wind speed standard deviation and wind direction standard deviation. However, historical meteorological data should be viewed from a dynamic perspective. Historical data is time series data based on dates, so when dealing with meteorological attribute data, it is necessary to analyze certain trends. Using this trend to predict future wind power generation is far more accurate than using meteorological attribute data of a certain independent date in history. In addition, in the process of analyzing trends, data that are more recent in time have greater reference significance than data that are farther away in time.

[0038] like Figure 1 As shown, a wind power prediction method based on a multi-scale attention mechanism includes the following steps:

[0039] Step 1, construct a hybrid recurrent neural network model based on a multi-scale attention mechanism;

[0040] Step 2, use the training set data to train the hybrid recurrent neural network model, where the training set data includes influencing factor data and known wind power generation power data;

[0041] Step 3, input the test set data into the trained hybrid recurrent neural network model, and calculate the predicted value of the wind power generation power.

[0042] Specifically, the hybrid neural network model includes a first convolutional layer, a first bidirectional GRU layer, a first multi-scale attention layer, a second bidirectional GRU layer, a second multi-scale attention layer, a second convolutional layer, and a fully connected layer. Each layer is sequentially connected in order. The output of the first convolutional layer and the output of the first bidirectional GRU layer are connected and simultaneously used as the input of the first multi-scale attention layer and the second multi-scale attention layer. The bidirectional GRU layer is formed by paralleling a forward GRU model and a backward GRU model to form a bidirectional structure. The bidirectional GRU layer outputs two merged GRU signals. The output layer of the fully connected layer is 1. The input of the first convolutional layer in the hybrid neural network model is the influencing factor data sequence, and the output of the fully connected layer is the wind power generation power value.

[0043] Specifically, for the hybrid neural network model x0,...,x T is the influencing factor sequence data, (y0,...,y K ), K < T is the known wind power generation power value, (y K+1 ,...,y T ) is the wind power generation power value to be predicted, is the corresponding estimated value, and the input is x0,...,x T ,y0,...,y K variables, and input them into the hybrid neural network model in sequence to start training. The loss function uses the standard normalized MSE, and the activation function uses the Relu function.

[0044] Specifically, the analytical expression of the hybrid neural network model is as follows:

[0045]

[0046]

[0047]

[0048]

[0049]

[0050]

[0051] C 4 t =η 2 ([C 2 t ,C 3 t ])

[0052]

[0053] Among them, x t is the input of the model at time t, η 1 (·) and η 2 (·) is two convolution operations, [·,·] is a merge connection operation, MutiScalAttention(·,ScaleNum1) and MutiScalAttention(·,ScaleNum2) are attention collection operations of two scales, ScaleNum1 and ScaleNum2. A bidirectional GRU layer is required as input before each attention layer, and a series structure is adopted between them. The specific process of fusion convolution is as follows:

[0054] The first convolutional layer η 1 (x t ) accepts sequence data x t The input is It also serves as the input of the first and second multi-scale attention layers;

[0055] is the output of the first bidirectional GRU layer, Indicates that the output of the forward GRU With the backward GRU output Perform a merge join; is to multiply the weight vector by the first bidirectional GRU layer And add the offset vector Result;

[0056] Will With η 1 (x t ) Merge into P t 1 , as the input of the first multi-scale attention layer;

[0057] is the output of the attention layer of the first multi-scale attention layer with scale ScaleNum1, connected to the second bidirectional GRU layer;

[0058] is the output of the second bidirectional GRU layer, Indicates that the output of the forward GRU With the backward GRU output Perform a merge join; is to multiply the weight vector by the second bidirectional GRU layer And add the offset vector Result;

[0059] By analogy, we can get Then, the results of the two multi-scale attention models are fused through a convolution operation, and the outputs of different scales are screened and identified, so that the scale information that is more important to the target can be retained, and the output is obtained. After a full connection operation, the output O is obtained t ; The data processing flow of the method of the present invention is as follows Figure 2 As shown;

[0060] in, and All of them are acquired through learning and training.

[0061] Specifically, an improved multi-scale attention model is adopted in the first multi-scale attention layer and the second multi-scale attention layer, and the structure is as follows: t =Dense((W d ·x' t +B d ), units = ScaleNum), f t =1D-expand_dims(d t ), o t =attention(f t ), the multi-scale attention model is divided into two steps. First, the scale information is generated by using the output unit of the fully connected layer at the input variable. The scale size is represented by ScaleNum, so as to preprocess the input model into a two-dimensional vector d with a fixed scale. t ; Then use the one-dimensional expansion mechanism 1D-expand_dims to convert the two-dimensional vector d t Expanded into a three-dimensional vector f t , so that the multi-scale attention model can expand and recognize the input information at a fixed scale, x' t is the input variable, W d and B d is the parameter vector to be learned, Dense() represents the fully connected operation, o t =attention(f t ), including the following two steps: at =V a Softmax(W a f t T +B a ), Among them, f t T Yes t The transpose of V a , W a and B a is the parameter vector that needs to be learned. The first step is to output a through SoftMax t , the first t Transpose and multiply the input f t As the input for the next step, W b and B b is the learning parameter of the fully connected layer output, and the model process of the multi-scale attention layer is recorded as o t =MutiScalAttention(f t ,ScaleNum).

[0062] The bidirectional GRU layer is a bidirectional structure formed by connecting the forward GRU and the backward GRU models. The output of the first layer forget gate in the forward GRU is: 1 t =σ(W 1 f [H 1 t-1 ,x” t ]+B 1 f ),σ(x)=1 / (1+e -x ), f 1 t is a 2D intermediate matrix of NumberofGRUCells×20, where NumberofGRUCells represents the number of internal neural units in the forward GRU model, and x t is a 2D matrix data with a dimension of Windowsize×20, where Windowsize is the length of the input data; in the forward GRU update gate, the first output is: z 1 t =σ(W 1 z [H 1 t-1 ,x” t ]+B 1 z ), z 1 t The data format is the same as f 1 tLikewise, the second output is: Similarly, the corresponding first output of the backward GRU is: z 2 t =σ(W 2 z [H 2 t-1 ,x” t ]+B 2 z ), the second output: Intermediate output of the forward GRU And the intermediate output of the backward GRU By aggregating the intermediate outputs, we get the output Indicates that the output of the forward GRU With the backward GRU output Merge the connection as the output of the bidirectional GRU layer x” t is the input of the bidirectional GRU layer, [W 1 f ,B 1 f ],[W 1 Z ,B 1 Z ],[W 1 h ,B 1 h ] is the forward GRU model parameter, [W 2 f ,B 2 f ],[W 2 Z ,B 2 Z ],[W 2 h ,B 2 h ] is the reverse GRU model parameter, [W 12 o ,B 12 o ] are the output layer parameters. It is a 2-dimensional intermediate matrix of NumberofGRUCells×20.

[0063] Preferably, the convolution layer is a 1D convolutional network. Convolutional neural networks generally include 1D convolution, 2D convolution and 3D convolutional networks. One-dimensional convolutional networks are mainly used for sequence data, such as audio data, equipment maintenance sampling data, etc. Two-dimensional convolution is mainly used for image processing, such as image classification, target recognition, image segmentation, etc. Three-dimensional convolutional networks are mainly used for video processing, such as medical imaging videos, motion detection, etc. In this embodiment, time series data is mainly analyzed, and the results of 1D convolutional networks are used. Typical 1D convolutional network results are as follows: Figure 3 As shown in FIG. 1 , it contains a one-dimensional convolution kernel vector, a filter size k=4, and a convolution interval (dilation factors) d=1.

[0064] For the element s that currently needs to be convolved, the mathematical expression of the 1D convolution operation is:

[0065] Where f(i) represents the convolution kernel function, X s-d·i It means taking sample values ​​with an interval of d forward.

[0066] Specifically, the influencing factor data include wind speed, wind direction, temperature, humidity, air pressure, hourly rainfall, wind speed extreme difference, wind speed standard deviation, wind direction extreme difference, wind direction standard deviation, temperature extreme difference, temperature standard deviation, humidity extreme difference, humidity standard deviation, air pressure extreme difference and air pressure standard deviation.

[0067] In order to better demonstrate the beneficial effects of the present invention, an embodiment is provided to demonstrate the entire method flow. The data in the embodiment is originally from the China Meteorological Network, with a total of 4000 sets of data for 400 days, one set of data is collected every hour, and ten sets of data are collected every day during the day, and each set of data includes data of five influencing factors: wind speed, wind direction, temperature, humidity and air pressure at the time of collection.

[0068] Through ten sets of data every day, the data of wind speed influencing factors are calculated, namely, the values ​​of fifteen influencing factors, including mean wind speed, mean wind direction, mean temperature, mean humidity, mean air pressure, wind speed range, wind speed standard deviation, wind direction range, wind direction standard deviation, temperature range, temperature standard deviation, humidity range, humidity standard deviation, air pressure range and air pressure standard deviation. The mean is the average value of the data collected at ten moments of a certain influencing factor, the range is the difference between the maximum and minimum values ​​of the data collected at ten moments of a certain influencing factor, and the standard deviation is the square root of the arithmetic mean of the squares of the deviations of the data collected at ten moments of a certain influencing factor from its mean.

[0069] The 4000 groups of data are transformed into 400 groups of data of wind speed influencing factors. The data of the wind speed influencing factors are preprocessed, and the Pearson correlation analysis method is used to perform correlation analysis on the aforementioned influencing factors and the wind speed to be predicted, and 7 influencing factors with the greatest correlation are obtained. The 7 influencing factors are respectively the mean wind speed, the extreme difference of wind speed, the standard deviation of wind speed, the mean wind direction, the mean temperature, the mean humidity and the extreme difference of humidity. Therefore, the data of these 7 influencing factors are selected as the values ​​of the input variables and normalized.

[0070] The experimental background adopted in this embodiment is: the computer is mainly configured as: Pentium (R) Dual-core 3.06 CPU, 8G RAM memory.

[0071] Effect evaluation: The parameters used in this embodiment to evaluate the performance of the method include RMSE, MAE, MAPE and CC:

[0072] RMSE (Root Mean Square Error) is calculated as follows:

[0073]

[0074] MAE (Mean Absolutely Error) is calculated as follows:

[0075]

[0076] MAPE (Mean Absolutely Percentage Error) is calculated as follows:

[0077]

[0078] CCV (correlation coefficient value), correlation coefficient, calculated as:

[0079]

[0080] It should be noted that RMSE, MAE and MAPE are all measures of prediction errors. The smaller the value, the more accurate it is. The CCV parameter represents the correlation coefficient of the two sequence quantities. The larger the value, the more correlated the two sequence data are and the better the prediction effect.

[0081]

[0082] It can be seen from the results that the method of the present invention has better accuracy than other recurrent neural network models.

[0083] It can be seen from the content of the invention and the embodiments that the hybrid recurrent neural network model in the method of the present invention mainly includes a multi-scale attention layer, a bidirectional GRU layer and a fully connected layer. The multi-scale attention layer is introduced into the recurrent neural network, and the improved multiple attention units are connected in series and in skip connection. The attention mechanism is distributed at different scales, so that the model can collect historical information from different scales and distinguish the influence of different input elements on the prediction results. The bidirectional GRU layer can more fully obtain the context information of the sequence data on the basis of GRU. The entire model adopts a convolutional structure to fuse the recognition outputs of the attention mechanisms of different scales, and screens and identifies the outputs of different scales through convolutional connections, so that those scale information that is more important to the target can be retained, thereby obtaining better accuracy for the prediction of wind power generation.

Claims

1. A wind power prediction method based on a multi-scale attention mechanism, characterized in that: The following steps are involved: Step 1: construct a hybrid recurrent neural network model based on a multi-scale attention mechanism; Step 2, training the hybrid recurrent neural network model using training set data, wherein the training set data includes influencing factor data and known wind power generation data; Step 3: Input the test set data into the trained hybrid recurrent neural network model to calculate the predicted value of wind power generation; The hybrid recurrent neural network model includes a first convolutional layer, a first bidirectional GRU layer, a first multi-scale attention layer, a second bidirectional GRU layer, a second multi-scale attention layer, a second convolutional layer and a fully connected layer. The layers are connected in sequence. The output of the first convolutional layer and the output of the first bidirectional GRU layer are connected and used as the input of the first multi-scale attention layer and the second multi-scale attention layer at the same time. The bidirectional GRU layer is a forward GRU model and a backward GRU model in parallel to form a bidirectional structure. The bidirectional GRU layer outputs two merged GRU signals. The output layer of the fully connected layer is 1. The input of the first convolutional layer in the hybrid recurrent neural network model is an influencing factor data sequence, and the output of the fully connected layer is a wind power value. The analytical expression of the hybrid recurrent neural network model is as follows: C 4 t =η 2 ([C 2 t ,C 3 t ]) Among them, x t is the input of the model at time t, η 1 (·) and η 2 (·) is two convolution operations, [·,·] is a merge connection operation, MutiScalAttention(·,ScaleNum1) and MutiScalAttention(·,ScaleNum2) are attention collection operations at two scales, ScaleNum1 and ScaleNum2. The specific process of fused convolution is as follows: The first convolutional layer η 1 (x t ) accepts sequence data x t The input is It also serves as the input of the first and second multi-scale attention layers; is the output of the first bidirectional GRU layer, Indicates that the output of the forward GRU With the backward GRU output Perform a merge join; is to multiply the weight vector by the first bidirectional GRU layer And add the offset vector Result; Will With η 1 (x t ) Merge into P t 1 , as the input of the first multi-scale attention layer; is the output of the attention layer of the first multi-scale attention layer with scale ScaleNum1, connected to the second bidirectional GRU layer; is the output of the second bidirectional GRU layer, Indicates that the output of the forward GRU With the backward GRU output Perform a merge join; is to multiply the weight vector by the second bidirectional GRU layer And add the offset vector Result; By analogy, we can get Then, the results of the two multi-scale attention models are fused through a convolution operation, and the outputs of different scales are screened and identified, so that the scale information that is more important to the target can be retained, and the output is obtained. After a full connection operation, the output O is obtained t ; in, and All of them are acquired through learning and training.

2. According to claim 1, a wind power prediction method based on a multi-scale attention mechanism is characterized in that: For the described hybrid recurrent neural network model, x0,...,x T is the sequence data of influencing factors, (y0,...,y K ), K < T is the known wind power generation value, (y K+1 ,...,y T ) is the wind power generation value to be predicted, is the corresponding estimated value, and the inputs are x0,...,x T ,y0,...,y K variables. They are sequentially input into the described hybrid recurrent neural network model for training. The loss function uses the standard normalized MSE, and the activation function uses the Relu function.

3. The wind power prediction method based on multi-scale attention mechanism according to claim 2 is characterized in that: An improved multi-scale attention model is adopted in the first multi-scale attention layer and the second multi-scale attention layer, and the structure is as follows: t =Dense((W d σx' t +B d ), units = ScaleNum), f t =1D-expand_dims(d t ), o t =attention(f t ), the multi-scale attention model is divided into two steps. First, the scale information is generated by using the output unit of the fully connected layer at the input variable. The scale size is represented by ScaleNum, so as to preprocess the input model into a two-dimensional vector d with a fixed scale. t ; Then use the one-dimensional expansion mechanism 1D-expand_dims to convert the two-dimensional vector d t Expanded into a three-dimensional vector f t , so that the multi-scale attention model can expand and recognize the input information at a fixed scale, x' t is the input variable, W d and B d is the parameter vector to be learned, Dense() represents the fully connected operation, o t =attention(f t ), including the following two steps: a t =V a Softmax(W a f t T +B a ), Among them, f t T Yes t The transpose of V a , W a and B a is the parameter vector that needs to be learned. The first step is to output a through SoftMax t , the first t Transpose and multiply the input f t As the input for the next step, W b and B b is the learning parameter of the fully connected layer output, and the model process of the multi-scale attention layer is recorded as o t =MutiScalAttention(f t ,ScaleNum).

4. The wind power prediction method based on multi-scale attention mechanism according to claim 1 is characterized in that: The bidirectional GRU layer is a bidirectional structure formed by connecting the forward GRU and the backward GRU models. The output of the first layer forget gate in the forward GRU is: 1 t =σ(W 1 f [H 1 t-1 ,x” t ]+B 1 f ),σ(x)=1 / (1+e -x ), f 1 t is a 2D intermediate matrix of NumberofGRUCells×20, where NumberofGRUCells represents the number of internal neural units in the forward GRU model, and x t is a 2D matrix data with a dimension of Windowsize×20, where Windowsize is the length of the input data; in the forward GRU update gate, the first output is: z 1 t =σ(W 1 z [H 1 t-1 ,x” t ]+B 1 z ), z 1 t The data format is the same as f 1 t Likewise, the second output is: Similarly, the corresponding first output of the backward GRU is: z 2 t =σ(W 2 z [H 2 t-1 ,x” t ]+B 2 z ), the second output: Intermediate output of the forward GRU And the intermediate output of the backward GRU By aggregating the intermediate outputs, we get the output Indicates that the output of the forward GRU With the backward GRU output Merge and connect as the output y of the bidirectional GRU layer t =W 12 o ·[H t 1 ,H t 2 ]+B 12 o , x” t is the input of the bidirectional GRU layer, [W 1 f ,B 1 f ],[W 1 Z ,B 1 Z ],[W 1 h ,B 1 h ] is the forward GRU model parameter, [W 2 f ,B 2 f ],[W 2 Z ,B 2 Z ],[W 2 h ,B 2 h ] is the reverse GRU model parameter, [W 12 o ,B 12 o ] is the output layer parameter, It is a 2-dimensional intermediate matrix of NumberofGRUCells×20.

5. The wind power prediction method based on multi-scale attention mechanism according to claim 1 is characterized in that: The convolutional layers are all 1-dimensional convolutional networks.

6. The wind power prediction method based on multi-scale attention mechanism according to claim 1 is characterized in that: The influencing factor data include wind speed, wind direction, temperature, humidity, air pressure, hourly rainfall, wind speed extreme, wind speed standard deviation, wind direction extreme, wind direction standard deviation, temperature extreme, temperature standard deviation, humidity extreme, humidity standard deviation, air pressure extreme and air pressure standard deviation.

Citation Information

Patent Citations

  • Sequence data prediction system of novel multi-scale attention mechanism

    CN110442846A

  • PM2.5 data prediction method based on multiple fusion convolution GRU

    CN111242369A

  • Short-term wind power prediction method based on multi-channel convolutional neural network and time convolutional network

    CN112365040A