Wind power prediction method fusing multiple attention mechanisms

Abstract

The invention discloses a wind power prediction method fusing multiple attention mechanisms. The method comprises the following steps: S1, collecting wind power feature data and meteorological feature data of a wind power plant; s2, performing data cleaning on the feature data, performing meteorological factor correlation analysis by using a Pearson coefficient, and constructing a data set; s3, dividing the data set into a training set, a verification set and a test set; s4, constructing a CNN-BiLSTM wind power prediction model fused with a multi-attention mechanism; s5, using an IGWO-CNN-BiLSTM prediction model based on a nonlinear convergence factor and the whole process Cauchy variation to obtain a multi-attention mechanism fused IGWO-CNN-BiLSTM prediction model; and S6, predicting the test set, and evaluating the effectiveness of the model. According to the method, a multi-head attention mechanism is introduced on the basis of CNN-BiLSTM, a nonlinear convergence factor and whole-process Cauchy variation are introduced for optimization to obtain IGWO, and the prediction precision of wind power is improved.

Description

Technical Field

[0001] The present invention relates to the field of wind power generation technology, and in particular to a wind power prediction method integrating a multi-head attention mechanism. Background Art

[0002] As a clean energy with broad application prospects, wind power generation has become a hot direction for energy transformation. Unlike traditional thermal power generation, the efficiency of wind power generation is inevitably restricted by environmental factors such as weather changes and seasonal changes, and the power generation capacity is intermittent and highly volatile. The prediction of wind power can be divided into point prediction, interval prediction, probability prediction and scenario prediction methods according to the time and space scale and prediction form. Point prediction is mainly based on numerical weather forecasting technology and wind power conversion model to accurately predict wind power at a certain moment in the future. Interval prediction and probability prediction provide certain prediction uncertainty information, which increases the reliability and practicality of the prediction. Scenario prediction provides a variety of possible wind power changes for wind farm operation and power system scheduling.

[0003] Factors that affect the accuracy of wind power prediction include meteorological conditions, geographical location, wind farm layout and model algorithms, among which the rational use of meteorological information can significantly improve the accuracy of prediction. In recent years, domestic and foreign research has combined cutting-edge technologies such as artificial intelligence and big data analysis to further optimize the structure and parameters of the prediction model, making useful attempts to improve prediction accuracy and solve the problem of wind power volatility. For example, the existing technology uses a long short-term memory network (Long Short-Term Memory, LSTM) with a gating mechanism to effectively maintain a long-term dependency for wind power prediction, but LSTM has the problem of difficulty in obtaining multiple feature space connections. At the same time, LSTM and its related improved deformation algorithms also have a large parameter range and it is difficult to determine the optimal parameter value. The local feature extraction capability of the convolutional neural network (CNN) is organically combined with the LSTM with a long-term dependency capture capability, which can effectively mine the local and overall temporal dependency of the signal, but the influence of important time steps is not fully considered. Therefore, under conditions such as transitional weather, when wind power fluctuates frequently, there is still a large prediction error. Summary of the invention

[0004] In order to solve the problems of large wind power prediction errors and difficulty in optimizing prediction model parameters, the present invention proposes a wind power prediction method integrating a multi-head attention mechanism to solve the above problems.

[0005] The present invention proposes a wind power prediction method integrating a multi-head attention mechanism, comprising the following steps:

[0006] S1. Collect wind power characteristic data and meteorological characteristic data of wind farms;

[0007] S2, cleaning the characteristic data obtained in S1, and using the Pearson coefficient to perform correlation analysis of meteorological factors to construct a data set;

[0008] S3, normalizing each feature data in the data set obtained in S2 to obtain a normalized data set, and dividing the normalized data set into a training set, a validation set, and a test set;

[0009] S4. Construct a CNN-BiLSTM wind power prediction model integrating multi-head attention mechanism;

[0010] S5. Use the improved gray wolf algorithm based on nonlinear convergence factor and full-process Cauchy mutation to optimize the parameters of the model constructed in S4, update the model when the maximum number of iterations of the improved gray wolf algorithm is reached, and obtain the final prediction model of CNN-BiLSTM fused with multi-head attention mechanism optimized by the improved gray wolf algorithm;

[0011] S6. Use the CNN-BiLSTM final prediction model that integrates the multi-head attention mechanism after being optimized by the improved gray wolf algorithm to predict the test set, obtain the prediction results, and evaluate the effectiveness of the model through evaluation indicators.

[0012] Preferably, the data cleaning comprises the following steps:

[0013] Check whether there is any feature data that exceeds the maximum or minimum value of the feature. If so, the corresponding outliers need to be removed, and spline interpolation is used to interpolate the missing data to ensure continuity in the time dimension.

[0014] Preferably, the use of the Pearson coefficient to perform meteorological factor correlation analysis includes analyzing wind speed data at a height of 70 meters in the wind power characteristic data and the meteorological characteristic data.

[0015] Preferably, the calculation formula for the normalization process is:

[0016]

[0017] Among them, x is any feature data, x max is the rated maximum value of x, min is the nominal minimum value of x.

[0018] Preferably, S4 comprises the following steps:

[0019] S41, S41, construct the CNN layer. First, construct a convolution kernel of size 5 and a one-dimensional convolution layer with a step size of 1 to extract local information of feature data. Then use the Relu function to perform nonlinear changes on the feature data extracted by the convolution layer. Finally, construct a maximum pooling layer with a window size of 2.

[0020] S42. Build a BiLSTM layer. Build a bidirectional LSTM after the CNN layer to extract the long-term dependency of feature data and connect it to the output of a fully connected layer. Bidirectional LSTM is a bidirectional structure based on LSTM, which can mine information more comprehensively.

[0021] S43. Build a multi-head attention mechanism.

[0022] Preferably, the calculation formula corresponding to the LSTM is expressed as:

[0023] f t =σ(W f ·[h t-1 , x t ]+b f );

[0024] i t =σ(W i ·[h t-1 , x t ]+b i );

[0025] o t =σ(W o ·[h t-1 , x t ]+b o );

[0026]

[0027] h t =o t ·tanh(c t );

[0028] Among them, f t is the value of the forget gate, i t is the value of the input gate, o t is the output of the output gate, x t is the input value at the current time t, σ represents the sigma activation function, W f is the weight of the forget gate, W i is the weight of the input gate, W o is the weight of the output gate, W c is the weight of the candidate memory state unit, h t-1 is the output state value of the previous cell, ht is the output state value of the current cell, is the candidate memory state value, c t is the memory state value, tanh represents the tanh activation function, b f is the forget gate bias term, b i is the input gate bias term, b o is the output gate bias term, b c is the bias term of the candidate memory unit.

[0029] Preferably, the calculation formula corresponding to the multi-head attention mechanism is expressed as:

[0030] MultiHeadAttention(Q,K,V)=Concat(head1,...,head n )W 0 ;

[0031] Among them, Q is the query matrix, K is the key matrix, V is the value matrix, Concat represents the concatenation of multi-head attention calculation results, head represents the single-head attention calculation result, and W 0 is the linear layer transformation matrix of the multi-head attention mechanism;

[0032] head i The calculation process is as follows:

[0033] head i =Attention(QW i Q , KW i K , VW i V );

[0034] Among them, W i Q is the transformation matrix of Q corresponding to different attention heads, W i K is the transformation matrix corresponding to different attention heads of K, V is the transformation matrix corresponding to different attention heads;

[0035] The calculation process of the attention mechanism is as follows:

[0036]

[0037] Among them, Q i Different representation spaces for Q, from Q through W i Q The transformation matrix is ​​transformed, K i For K different representation spaces, from K through W i KThe transformation matrix is ​​transformed, V i Different representation spaces for V, from V through W i V The transformation matrix is ​​transformed, softmax is the normalization function, d k is the dimension of the key and T represents the matrix transpose.

[0038] Preferably, the parameters optimized in S5 include the batch number Batch_size of the input data, the number of neurons N in the BiLSTM layer, and the model learning rate Lr. In terms of the impact on the accuracy of the prediction results, the optimal number of neurons N corresponding to different Batch_size parameters is often different, and different combinations of the two parameters have a more significant impact on the results.

[0039] Preferably, S5 comprises the following steps:

[0040] S51, initialize the CNN-BiLSTM wind power prediction model integrated with the multi-head attention mechanism and its training times, the total number of wolves in the wolf pack of the gray wolf algorithm, and the maximum iteration number K of the gray wolf algorithm, and use a uniformly distributed random number to initialize the position of each wolf in the wolf pack;

[0041] Randomly select α wolf, β wolf, and δ wolf. The different dimensions of the wolves in the wolf pack represent a parameter to be optimized in the model. Each wolf is composed of three dimensions: the batch number Batch_size of the input data, the number of neurons N in the BiLSTM layer, and the model learning rate Lr. The expression is as follows:

[0042] X = [Batch_size, N, Lr];

[0043] S52, using the mean square error of the validation set as the fitness function, calculate the fitness values ​​of all wolves in the wolf pack, select the best solution of the three best fitness values ​​as α wolf, the second best solution as β wolf, and the third best solution as δ wolf, the process is as follows:

[0044]

[0045] Among them, yi is the real value of wind power, is the predicted value of wind power, M is the number of calculated wind power points;

[0046] S53, surround the prey, update a, A, B parameters, the specific process is as follows:

[0047] D=|BX p (t)-X(t)|;

[0048] X(t+1)=X p (t)-AD;

[0049] Where D is the distance vector between the prey and the current wolf, X(t) is the position vector of the gray wolf at the current time t, and X p (t) is the position vector of the prey at the current time t, X(t+1) is the updated position vector of the current gray wolf, A and B are random coefficient vectors, and their calculation method is:

[0050] a=2.3-2.3(e(r / K)-1) / (e-1);

[0051] A = 2ar1-a;

[0052] B = 2r2;

[0053] Where a is the nonlinear convergence factor, K is the maximum number of iterations of the Grey Wolf Algorithm, r is the current number of iterations, and r1 and r2 are vectors of random numbers between 0 and 1.

[0054] S54, update the position of the gray wolf, as follows:

[0055] D α =|B1X α -X|;

[0056] D β =|B2X β -X|;

[0057] D δ =|B3X δ -X|;

[0058] X1=|X α -A1D α |;

[0059] X2=|X β -A2D β |;

[0060] X3=|X δ -A3D δ |;

[0061]

[0062] Among them, X α is the position vector of wolf α, X β is the position vector of β wolf, X δ is the position vector of δ wolf, D α is the distance vector between wolf α and other wolves, D β is the distance vector between wolf β and other wolves, D δis the distance vector between wolf δ and other wolves, X1 represents the distance vector of the remaining wolves approaching wolf α, X2 represents the distance vector of the remaining wolves approaching wolf β, X3 represents the distance vector of the remaining wolves approaching wolf δ, A1, A2, A3, B1, B2, B3 are random coefficient vectors;

[0063] S55. Use the whole process Cauchy mutation to disturb the positions of the three decision wolves α wolf, β wolf and δ wolf. The specific process is as follows:

[0064]

[0065] Where f(η) is the probability density function of the Cauchy distribution, η is a random variable, X i (t) is the current position vector of one of the decision wolves α, β, and δ, X′ i (t) is the new position vector of the decision wolf after Cauchy mutation, Cauthy(0,1) is the standard Cauchy distribution, that is, the random number generated by the Cauchy distribution with a center value of O and a scale parameter of 1, and fit is the fitness function.

[0066] Introducing the Cauchy mutation can effectively make the wolf pack jump out of the local optimum. In order to make the Cauchy mutation have a transition trend when the wolf pack transitions from global search to local development, and fully search the solution space, a proportional factor is added to the Cauchy mutation. K is the maximum number of iterations, r is the current number of iterations. When the current number of iterations r is small, small disturbances are mainly maintained so that the three decision wolves can find possible better solutions within a smaller range of their own positions. The small-scale disturbances in the early stage can more comprehensively search the surrounding solution space. As the current number of iterations r increases, it is easier to generate larger disturbances. The disturbances throughout the whole process reduce the possibility of the algorithm falling into the local optimum.

[0067] S56. Determine whether the maximum number of iterations K is reached. If so, end the model update and obtain the CNN-BiLSTM final prediction model that integrates the multi-head attention mechanism and is optimized by the improved gray wolf algorithm. If the maximum number of iterations K is not reached, return to S52.

[0068] Preferably, the evaluation indicators include mean square error, root mean square error, mean absolute error and determination coefficient.

[0069] Beneficial effects of the present invention:

[0070] (1) Based on the use of CNN-BiLSTM to mine data patterns and their impact on future data, the present invention introduces a multi-head attention mechanism to strengthen attention to important time steps, thereby more comprehensively mining data patterns and differences.

[0071] (2) The present invention introduces a nonlinear convergence factor and a full-process Cauchy variation to optimize GWO to obtain IGWO, which can seek the optimal parameter combination in the case of a multi-parameter high-dimensional combination model, effectively improving the prediction accuracy of wind power. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a flow chart of a wind power prediction method integrating a multi-head attention mechanism according to an embodiment of the present invention;

[0073] Figure 2 This is a schematic diagram of the structure of a CNN-BiLSTM wind power prediction model integrating a multi-head attention mechanism according to an embodiment of the present invention;

[0074] Figure 3 A flow chart of optimizing a model using IGWO according to an embodiment of the present invention;

[0075] Figure 4 This is a comparison chart of prediction results of different models in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and examples.

[0077] The present application embodiment discloses a wind power prediction method integrating a multi-head attention mechanism, such as Figure 1 As shown, the following steps are included:

[0078] S1. Collect wind power characteristic data and meteorological characteristic data of the wind farm. For example, in this embodiment, wind power operation data and meteorological data of a wind farm in the northwest in the first quarter of 2024 are collected.

[0079] S2. Perform data cleaning on the characteristic data obtained in S1, including inspection and screening, elimination of erroneous data, interpolation and correction, to check whether there is characteristic data that exceeds the maximum or minimum value of the characteristic. If so, the corresponding abnormal values ​​need to be eliminated, and spline interpolation is used to interpolate the missing data to ensure continuity in the time dimension. Then use the Pearson coefficient to perform correlation analysis on meteorological factors, including analysis of the 70-meter height wind speed data based on wind power characteristic data and meteorological characteristic data to construct a data set.

[0080] S3, normalize each feature data in the data set obtained in S2 to obtain a normalized data set, thereby reducing the difficulty of model training. The calculation formula for normalization is:

[0081]

[0082] Among them, x is any feature data, xmax is the rated maximum value of x, min is the rated minimum value of x. In this embodiment, the maximum rated wind speed x of the wind turbines in the wind farm collected is used. max is 20m / s, the minimum rated wind speed is x min is 0m / s.

[0083] The normalized data set is divided into training set, validation set and test set in a ratio of 6:2:2. The training set is used to build CNN-BiLSTM training with multi-head attention mechanism; the purpose of dividing the validation set is to establish MSE as the fitness function so as to objectively verify the effectiveness and generalization of the model; the test set is used to test the model optimized by IGWO. At the same time, data preprocessing is performed, using the sliding window algorithm, and the measured data is used as a time point every 15 minutes. According to the model with 8 hours, i.e. 32 points of input and 1 hour of prediction, i.e. 4 data points of output, the training set, test set and test set are divided according to the input and target values.

[0084] S4. Construct Figure 2 The CNN-BiLSTM wind power prediction model with multi-head attention mechanism is shown. Figure 2 In the equation, x1,...,x n To arrange the power and meteorological characteristic data in chronological order, the model includes: a convolutional layer including a Conv1d convolutional network with a convolution kernel size of 5 and a step size of 1; a Relu function is used after Conv1d to perform nonlinear changes on the extracted data; the output of the Relu function is sent to a maximum pooling layer maxpooling with a window size of 2; then a BiLSTM layer is connected, which is a bidirectional network based on LSTM; followed by a fully connected layer FC, and a multi-head attention layer with 4 attention heads is connected after the fully connected layer FC; finally, a fully connected layer FC is connected for the output of the final result.

[0085] S41, construct CNN layer, first construct a convolution kernel of size 5 and a one-dimensional convolution layer with a step size of 1, get Conv1d, which is used to extract local information of feature data, and then use Rel u The function performs nonlinear changes on the feature data extracted by the convolution layer, and finally constructs a maximum pooling layer with a window size of 2;

[0086] S42. Build a BiLSTM layer. Build a bidirectional LSTM after the CNN layer to extract the long-term dependency of feature data and connect it to the output of a fully connected layer. Bidirectional LSTM is a bidirectional structure based on LSTM, which can mine information more comprehensively.

[0087] The calculation formula corresponding to LSTM is expressed as:

[0088] f t =σ(W f ·[h t-1 , x t ]+b f );

[0089] i t =σ(W i ·[h t-1 , x t ]+b i );

[0090] o t =σ(W o ·[h t-1 , x t ]+b o );

[0091]

[0092] h t =o t ·tanh(c t );

[0093] Among them, f t is the value of the forget gate, i t is the value of the input gate, o t is the output of the output gate, x t is the input value at the current time t, σ represents the sigma activation function, W f is the weight of the forget gate, W i is the weight of the input gate, W o is the weight of the output gate, W c is the weight of the candidate memory state unit, h t-1 is the output state value of the previous cell, h t is the output state value of the current cell, is the candidate memory state value, c t is the memory state value, tanh represents the tanh activation function, b f is the forget gate bias term, b i is the input gate bias term, b o is the output gate bias term, b c is the bias term of the candidate memory unit.

[0094] S43. Build a multi-head attention mechanism.

[0095] The matrix transformation is shown in the following formula, using a matrix such as W 0 Perform a linear transformation on the data, and the calculation formula corresponding to the multi-head attention mechanism is expressed as:

[0096] MultiHeadAttention(Q,K,V)=Concat(head1,...,head n )W 0 ;

[0097] Among them, Q is the query matrix, K is the key matrix, V is the value matrix, Concat represents the concatenation of multi-head attention calculation results, head represents the single-head attention calculation result, and W 0 is the linear layer transformation matrix of the multi-head attention mechanism.

[0098] head i The calculation process is as follows:

[0099] head i =Attention(QW i Q , KW i K , VW i V );

[0100] Among them, W i Q is the transformation matrix of Q corresponding to different attention heads, W i K is the transformation matrix corresponding to different attention heads, W i V V is the transformation matrix corresponding to different attention heads.

[0101] The calculation process of the attention mechanism is as follows:

[0102]

[0103] Among them, Q i Different representation spaces for Q, from Q through W i Q The transformation matrix is ​​transformed, K i For K different representation spaces, from K through W i K The transformation matrix is ​​transformed, V i Different representation spaces for V, from V through W i V The transformation matrix is ​​transformed, softmax is the normalization function, d k is the dimension of the key and T represents the matrix transpose.

[0104] S5, construct an improved gray wolf algorithm (IGWO) for optimizing model parameters. The optimization parameters include the batch number Batch_size of input data, the number of neurons N of the BiLSTM layer and the model learning rate Lr, a total of three dimensions. In the present embodiment, the batch number Batch_size optimization range of input data is set to [256, 520], and the number of neurons N optimization range is [256, 520]. From the perspective of the impact on the accuracy of the prediction results, the optimal number of neurons N corresponding to different Batch_size parameters is often different. The different combinations of the two parameters have a more significant impact on the results. The model learning rate Lr optimization range is [0.001, 0.01]. The parameters of the model constructed by S4 are optimized using the improved gray wolf algorithm based on nonlinear convergence factor and full-process Cauchy variation. When the maximum number of iterations of the improved gray wolf algorithm is reached, the model is updated to obtain the final prediction model of the CNN-BiLSTM fusion multi-head attention mechanism optimized by the improved gray wolf algorithm.

[0105] The classic gray wolf algorithm (GWO) is a group bionic algorithm proposed based on the hierarchy and hunting methods of gray wolf groups. There is a strict social dominance hierarchy in the gray wolf group, which can be divided into α wolf, β wolf, δ wolf, and ω wolf according to their different status. Among them, the α wolf makes decisions and leads the wolf pack's predation and other behaviors, the β wolf obeys the α wolf and assists it in making decisions and dominating the wolf pack, the δ wolf is dominated by the α wolf and the β wolf, and dominates the remaining levels of wolves at the same time, and the actions of the ω wolf are dominated by the first three types of wolves, which are the most common wolves. The embodiment of the present application further improves the classic GWO algorithm and proposes an improved gray wolf algorithm IGWO. In the optimization process, the convergence factor is nonlinearly reduced as the number of iterations increases to better balance the global search and local development and utilization of the whole process Cauchy mutation perturbation optimization, which can fully search the solution space and avoid falling into the local optimum. It has the characteristics of few parameters and easy implementation.

[0106] like Figure 3 As shown in Figure 2, the parameter optimization process of IGWO is as follows:

[0107] S51. Initialize the CNN-BiLSTM wind power prediction model integrating the multi-head attention mechanism and its training times, the total number of wolves in the gray wolf algorithm wolf pack, and the maximum number of iterations K of the gray wolf algorithm, and use uniformly distributed random numbers to initialize the positions of each wolf in the wolf pack.

[0108] Randomly select α wolf, β wolf, and δ wolf. The different dimensions of the wolves in the wolf pack represent a parameter to be optimized in the model. Each wolf is composed of three dimensions: the batch number Batch_size of the input data, the number of neurons N in the BiLSTM layer, and the model learning rate Lr. The expression is as follows:

[0109] X = [Batch_size, N, Lr];

[0110] S52, using the mean square error of the validation set as the fitness function, calculate the fitness values ​​of all wolves in the wolf pack, select the best solution of the three best fitness values ​​as α wolf, the second best solution as β wolf, and the third best solution as δ wolf, the process is as follows:

[0111]

[0112] Among them, y i is the real value of wind power, is the predicted value of wind power, M is the number of calculated wind power points;

[0113] S53, surround the prey, update a, A, B parameters, the specific process is as follows:

[0114] D=|BX p (t)-X(t)|;

[0115] X(t+1)=X p (t)-AD;

[0116] Where D is the distance vector between the prey and the current wolf, X(t) is the position vector of the gray wolf at the current time t, and X p (t) is the position vector of the prey at the current time t, X(t+1) is the updated position vector of the current gray wolf, A and B are random coefficient vectors, and their calculation method is:

[0117] a=2.3-2.3(e(r / K)-1) / (e-1);

[0118] A = 2ar1-a;

[0119] B = 2r2;

[0120] Among them, a is a nonlinear convergence factor. The nonlinear convergence factor proposed in the embodiment of the present application can be reduced to 0 more quickly in the later stage, which can accelerate the convergence of the algorithm. K is the maximum number of iterations of the gray wolf algorithm, and r is the current number of iterations. As r increases, a will decrease nonlinearly from 2.3 to 0. It can be seen from the above formula that the coefficient vector A is affected by a, and each dimension takes a value in the range of [-a, a]. As a decreases nonlinearly from 2.3, it will affect the coefficient vector A. When the coefficient vector |A|≥1 transitions to |A|<1, the wolf pack transitions from global search to local development. The use of nonlinear convergence factors can better balance global search and local development, and accelerate the convergence speed of the algorithm in the later stage. r1 and r2 are vectors composed of random numbers between 0 and 1.

[0121] S54, update the position of the gray wolf, as follows:

[0122] D α =|B1X α-X|;

[0123] D β =|B2X β -X|;

[0124] D δ =|B3X δ -X|;

[0125] X1=|X α -A1D α |;

[0126] X2=|X β -A2D β |;

[0127] X3=|X δ -A3D δ |;

[0128]

[0129] Among them, X α is the position vector of wolf α, X β is the position vector of β wolf, X δ is the position vector of δ wolf, D α is the distance vector between wolf α and other wolves, D β is the distance vector between wolf β and other wolves, D δ is the distance vector between wolf δ and other wolves, X1 represents the distance vector of the remaining wolves approaching wolf α, X2 represents the distance vector of the remaining wolves approaching wolf β, X3 represents the distance vector of the remaining wolves approaching wolf δ, and A1, A2, A3, B1, B2, and B3 are random coefficient vectors.

[0130] S55. Use the whole process Cauchy mutation to disturb the positions of the three decision wolves α wolf, β wolf and δ wolf. The specific process is as follows:

[0131]

[0132] Where f(η) is the probability density function of the Cauchy distribution, η is a random variable, X i (t) is the current position vector of one of the decision wolves α, β, and δ, X′ i (t) is the new position vector of the decision wolf after Cauchy mutation, and Cauthy(0,1) is a standard Cauchy distribution, that is, a random number generated by a Cauchy distribution with a center value of 0 and a scale parameter of 1. i The calculation formula of (t+1) is greedily updated, and fit is the fitness function.

[0133] From the calculation formula of the Cauchy one-dimensional standard probability density function f(η), it can be seen that it has a higher probability density function near the current wolf position, and is widely distributed at both ends, which can generate disturbances near the current wolf position. At the same time, due to its wide distribution at both ends, it can produce a larger variation range, thereby reducing the possibility of the gray wolf algorithm falling into the local optimum. Since the nonlinear reduction process of the gray wolf algorithm a will affect the value range of the coefficient vector A, when |A| < 1, the wolf pack will enter the local development stage, and when |A| ≥ 1, the wolf pack is in the global search stage. Therefore, the introduction of the Cauchy mutation can enable the wolf pack to effectively jump out of the local optimum. In order to make the Cauchy mutation have a transition trend in the disturbance change when the wolf pack transitions from global search to local development, and fully search the solution space, a proportional factor is added to the Cauchy mutation. K is the maximum number of iterations, r is the current number of iterations. When the current number of iterations r is small, small disturbances are mainly maintained so that the three decision wolves can find possible better solutions within a smaller range of their own positions. The small-scale disturbances in the early stage can more comprehensively search the surrounding solution space. As the current number of iterations r increases, it is easier to generate larger disturbances. The disturbances throughout the whole process reduce the possibility of the algorithm falling into the local optimum.

[0134] S56. Determine whether the maximum number of iterations K is reached. If so, end the model update and obtain the CNN-BiLSTM final prediction model that integrates the multi-head attention mechanism and is optimized by the improved gray wolf algorithm. If the maximum number of iterations K is not reached, return to S52.

[0135] S6. Use the CNN-BiLSTM final prediction model optimized by the improved gray wolf algorithm and integrated with the multi-head attention mechanism to predict the test set, obtain the prediction results, and evaluate the effectiveness of the model through evaluation indicators. The evaluation indicators include mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE) and determination coefficient R 2 , the calculation formula is as follows:

[0136]

[0137]

[0138] in, is the average value of the real value of wind power.

[0139] In a specific embodiment, the prediction model proposed in the embodiment of the present application is compared with other models using evaluation indicators, and the results obtained are shown in Table 1 and Figure 4 As shown:

[0140] Table 1 Comparison results of model error indicators

[0141]

[0142] From Table 1 and Figure 4 It can be seen that the MSE, RMSE and MAE values ​​of the prediction results of the IGWO-CNN-BiLSTM model fused with the multi-head attention mechanism proposed in the present invention are lower than those of the LSTM and CNN-LSTM prediction results. The MSE, RMSE and MAE values ​​of the CNN-LSTM model are reduced by 16.4%, 8.52% and 16.6% compared with the single LSTM, indicating that using CNN as the front end of LSTM to obtain multiple feature space connections can further explore the regular characteristics of the data, so that the prediction accuracy of CNN-LSTM is improved to a certain extent compared with LSTM. The IGWO-CNN-BiLSTM model combines the advantages of CNN-LSTM and LSTM, and the BiLSTM network can establish the temporal relationship between weather factors and corresponding wind power in a bidirectional manner. From the prediction results of the model of the present invention, the introduction of the multi-head attention mechanism can more comprehensively evaluate the differences and importance between data. Its MSE, RMSE and MAE values ​​are reduced by 65.8%, 41.6% and 49.3% respectively compared with the single LSTM, and by 59.0%, 36.1% and 39.2% respectively compared with CNN-LSTM. R of the prediction results of IGWO-CNN-BiLSTM model 2 The value is the highest, which also shows that the CNN-BiLSTM model that introduces both IGWO and multi-head attention mechanism can further reduce the prediction error and is more robust.

[0143] The present invention aims at the difficulty of single LSTM to deeply mine the time series law of wind power characteristic data with strong instability and volatility, and the inability of CNN-LSTM to comprehensively evaluate the importance of characteristic data at different time steps. The bidirectional LSTM structure is used to perform bidirectional modeling on sequence data, and the CNN-BiLSTM wind power prediction model integrating the multi-head attention mechanism fully mines the time series law of meteorological factors and their corresponding wind power output, and mines important time steps. Specifically, the learning ability of BiLSTM for time series is used to extract the time series feature information of wind power signals, and a CNN link is added to the front end of BiLSTM to further strengthen the extraction of local features. The introduction of the multi-head attention mechanism can give corresponding weights according to the criticality of the feature information, thereby effectively capturing the important time series features of wind power data. This combined wind power prediction model based on multivariate modules has the disadvantages of large parameter combination quantity and high setting freedom. Therefore, the present invention proposes a gray wolf algorithm that introduces a nonlinear convergence factor and a full-process Cauchy mutation to optimize the input data batch size, the number of hidden layer nodes and the learning rate. The present invention can improve the model's mining of data differences and important time steps, and can more conveniently optimize model parameters.

[0144] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A wind power prediction method integrating multi-head attention mechanism, characterized in that: The following steps are involved: S1. Collect wind power characteristic data and meteorological characteristic data of wind farms; S2, cleaning the characteristic data obtained in S1, and using the Pearson coefficient to perform correlation analysis of meteorological factors to construct a data set; S3, normalizing each feature data in the data set obtained in S2 to obtain a normalized data set, and dividing the normalized data set into a training set, a validation set, and a test set; S4. Construct a CNN-BiLSTM wind power prediction model integrating multi-head attention mechanism; S5. Use the improved gray wolf algorithm based on nonlinear convergence factor and full-process Cauchy mutation to optimize the parameters of the model constructed in S4, update the model when the maximum number of iterations of the improved gray wolf algorithm is reached, and obtain the IGWO-CNN-BiLSTM prediction model optimized by the improved gray wolf algorithm and integrated with the multi-head attention mechanism; S6. Use the IGWO-CNN-BiLSTM prediction model that integrates the multi-head attention mechanism after being optimized by the improved gray wolf algorithm to predict the test set, obtain the prediction results, and evaluate the effectiveness of the model through evaluation indicators.

2. The wind power prediction method integrating multi-head attention mechanism according to claim 1 is characterized in that: The data cleaning comprises the following steps: Check whether there is any feature data that exceeds the maximum or minimum value of the feature. If so, the corresponding outliers need to be removed, and spline interpolation is used to interpolate the missing data to ensure continuity in the time dimension.

3. The wind power prediction method integrating multi-head attention mechanism according to claim 2 is characterized in that: The use of the Pearson coefficient to perform meteorological factor correlation analysis includes analyzing wind speed data at a height of 70 meters in wind power characteristic data and meteorological characteristic data.

4. The wind power prediction method integrating multi-head attention mechanism according to claim 3 is characterized in that: The calculation formula for the normalization process is: Among them, x is any feature data, x max is the rated maximum value of x, min is the nominal minimum value of x.

5. The wind power prediction method integrating multi-head attention mechanism according to claim 4 is characterized in that: The S4 comprises the following steps: S41, construct the CNN layer. First, construct a convolution kernel of size 5 and a one-dimensional convolution layer with a step size of 1 to extract local information of feature data. Then use the Relu function to perform nonlinear changes on the feature data extracted by the convolution layer. Finally, construct a maximum pooling layer with a window size of 2. S42, build a BiLSTM layer, build a bidirectional LSTM after the CNN layer to extract the long-term dependency of feature data and connect the output of a fully connected layer; S43. Build a multi-head attention mechanism.

6. The wind power prediction method integrating multi-head attention mechanism according to claim 5 is characterized in that: The calculation formula corresponding to the LSTM is expressed as: f t =σ(W f ·[h t-1 ,x t ]+b f ); i t =σ(W i ·[h t-1 ,x t ]+b i ); the t =σ(W o ·[h t-1 ,x t ]+b o ); h t =o t ·tanh(c t ); Among them, f t is the value of the forget gate, i t is the value of the input gate, o t is the output of the output gate, x t is the input value at the current time t, σ represents the sigma activation function, W f is the weight of the forget gate, W i is the weight of the input gate, W o is the weight of the output gate, W c is the weight of the candidate memory state unit, h t-1 is the output state value of the previous cell, h t is the output state value of the current cell, is the candidate memory state value, c t is the memory state value, tanh represents the tanh activation function, b f is the forget gate bias term, b i is the input gate bias term, b o is the output gate bias term, b c is the bias term of the candidate memory unit.

7. The wind power prediction method integrating multi-head attention mechanism according to claim 6 is characterized in that: The calculation formula corresponding to the multi-head attention mechanism is expressed as: MultiHeadAttention(Q,K,V)=Concat(head1,...,head n )W 0 ; Among them, Q is the query matrix, K is the key matrix, V is the value matrix, Concat means concatenating the multi-head attention calculation results, head is the single-head attention calculation result, and W 0 is the linear layer transformation matrix of the multi-head attention mechanism; head i The calculation process is as follows: head i =Attention(QW i Q ,KW i K ,VW i V ); Among them, Wi i Q is the transformation matrix of Q corresponding to different attention heads, W i K is the transformation matrix corresponding to different attention heads, W i V V is the transformation matrix corresponding to different attention heads; The calculation process of the attention mechanism is as follows: Among them, Q i Different representation spaces for Q, from Q through W i Q The transformation matrix is ​​transformed, K i For K different representation spaces, from K through W i K The transformation matrix is ​​transformed, V i Different representation spaces for V, from V through W i V The transformation matrix is ​​transformed, softmax is the normalization function, d k is the dimension of the key and T represents the matrix transpose.

8. The wind power prediction method integrating multi-head attention mechanism according to claim 7 is characterized in that: The parameters optimized in S5 include the batch number Batch_size of the input data, the number of neurons N in the BiLSTM layer, and the model learning rate Lr.

9. The wind power prediction method integrating multi-head attention mechanism according to claim 8 is characterized in that: The S5 comprises the following steps: S51, initialize the CNN-BiLSTM wind power prediction model integrated with the multi-head attention mechanism and its training times, the total number of wolves in the wolf pack of the gray wolf algorithm, and the maximum iteration number K of the gray wolf algorithm, and use a uniformly distributed random number to initialize the position of each wolf in the wolf pack; Randomly select α wolf, β wolf, and δ wolf. The different dimensions of the wolves in the wolf pack represent a parameter to be optimized in the model. Each wolf is composed of three dimensions: the batch number Batch_size of the input data, the number of neurons N in the BiLSTM layer, and the model learning rate Lr. The expression is as follows: X = [Batch_size, N, Lr]; S52, using the mean square error of the validation set as the fitness function, calculate the fitness values ​​of all wolves in the wolf pack, select the best solution of the three best fitness values ​​as α wolf, the second best solution as β wolf, and the third best solution as δ wolf, the process is as follows: Among them, y i is the real value of wind power, is the predicted value of wind power, M is the number of calculated wind power points; S53, surround the prey, update a, A, B parameters, the specific process is as follows: D=|BX p (t)-X(t)| X(t+1)=X p (t)-AD; Where D is the distance vector between the prey and the current wolf, X(t) is the position vector of the gray wolf at the current time t, and X p (t) is the position vector of the prey at the current time t, X(t+1) is the updated position vector of the current gray wolf, A and B are random coefficient vectors, and their calculation method is: a=2.3-2.3(e (r / K) -1) / (e-1); A = 2ar1-a; B＝2r2 Where a is the nonlinear convergence factor, K is the maximum number of iterations of the Grey Wolf Algorithm, r is the current number of iterations, and r1 and r2 are vectors of random numbers between 0 and 1. S54, update the position of the gray wolf, as follows: D α =|B1X α -X|; D β =|B2X β -X|; D δ =|B3X δ -X|; X1=|X α -A1D α |; X2=|X β -A2D β | <h2 style=";text-align:left;direction:ltr">X3=|X<h2 style=";text-align:left;direction:ltr"> δ <h2 style=";text-align:left;direction:ltr"> -A3D<h2 style=";text-align:left;direction:ltr"> δ <h2 style=";text-align:left;direction:ltr"> |; Among them, X α is the position vector of wolf α, X β is the position vector of β wolf, X δ is the position vector of δ wolf, D α is the distance vector between wolf α and other wolves, D β is the distance vector between wolf β and other wolves, D δ is the distance vector between wolf δ and other wolves, X1 represents the distance vector of the remaining wolves approaching wolf α, X2 represents the distance vector of the remaining wolves approaching wolf β, X3 represents the distance vector of the remaining wolves approaching wolf δ, A1, A2, A3, B1, B2, B3 are random coefficient vectors; S55. Use the whole process Cauchy mutation to disturb the positions of the three decision wolves α wolf, β wolf and δ wolf. The specific process is as follows: Where f(η) is the probability density function of the Cauchy distribution, η is a random variable, X i (t) is the current position vector of one of the decision wolves α, β, and δ, X′ i (t) is the new position vector of the decision wolf after Cauchy mutation, Cauthy(0,1) is the standard Cauchy distribution, and fit is the fitness function; S56. Determine whether the maximum number of iterations K is reached. If so, end the update and obtain the final prediction model of CNN-BiLSTM that integrates the multi-head attention mechanism and is optimized by the improved gray wolf algorithm. If the maximum number of iterations K is not reached, return to S52.

10. The wind power prediction method integrating multi-head attention mechanism according to claim 9 is characterized in that: The evaluation indicators include mean square error, root mean square error, mean absolute error and determination coefficient.

Images