Ultra-short-term wind power forecasting method based on temporal convolutional network

By combining the ultra-short-term wind power prediction method of time convolution network and channel convolution attention module, the problem of manual feature extraction and the shortcomings of deep learning methods in the prior art in feature selection and training stability are solved, and wind power is realized High accuracy and training stability for ultra-short-term predictions.

CN114861533BActive Publication Date: 2024-11-08SOUTHEAST UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210447856.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-26
Publication Date
2024-11-08
Estimated Expiration
2042-04-26

AI Technical Summary

Technical Problem

In the ultra-short-term prediction of wind power, there is a problem that artificial feature extraction workload is large, subjective and difficult to dig deep-level information in the ultra-short-term prediction of wind power. In addition, deep learning methods have defects in feature selection and training stability, and cannot achieve the level of feature. Difference between sex and difference.

Method used

The wind power ultra-short-term prediction method based on the time convolution network (TCN) and the channel convolution attention module is adopted. Time series features are automatically extracted through the time convolution network, and the channel convolution attention module is used to weight the features to establish inputs. The mapping relationship between data and predicted data realizes adaptive feature extraction and high-precision prediction.

Benefits of technology

It achieves high accuracy of ultra-short-term prediction of wind power, has fewer parameters, and is stable in training, which improves the prediction performance of the time convolution network, and does not significantly reduce the model training speed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114861533B_ABST
    Figure CN114861533B_ABST
Patent Text Reader

Abstract

The present invention relates to a wind power ultra-short-term prediction method based on a temporal convolutional network, comprising the following steps: collecting actual power generation data within a certain period of time and preprocessing the data; dividing the processed data into samples, and dividing them into a training set, a validation set and a test set; inputting the training set and the validation set into a TCN-CCA network for model training; the TCN-CCA network is composed of a temporal convolutional network and a channel convolutional attention module, the channel convolutional attention module is arranged between the last temporal convolutional module and the fully connected layer, and more weights are assigned to the key input features in the time dimension by calculating the probability distribution of attention; the test set is input into the trained TCN-CCA network to obtain the wind power ultra-short-term prediction value. The present invention realizes the automatic feature extraction of time series data features, and is applied to the wind power ultra-short-term prediction task, with higher accuracy, fewer training parameters and stable training.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of wind turbine power prediction, and in particular to an ultra-short-term prediction method for wind power based on a temporal convolutional network. Background Art

[0002] As a renewable energy source, wind power has been vigorously developed in recent years. However, due to its volatility and intermittency, large-scale grid connection has brought severe challenges to the safe operation and dispatch of the power grid. Therefore, it is of great practical significance to study the accurate prediction technology of wind turbine power generation. Traditional wind power prediction has physical model-based methods and statistical analysis methods. The prediction is made by establishing the relationship between power generation and factors such as terrain, wind speed, and wake effect. However, due to the numerous and complex factors affecting wind power, it is difficult to carry out effective modeling. In recent years, with the continuous deepening of machine learning research, diagnostic methods using statistical analysis have made great progress. For example, by extracting the maximum, mean, variance and other features of wind speed, as well as seasonal and periodic features, they are input into support vector machines, random forests and other models for prediction. However, manual feature extraction is labor-intensive, subjective and difficult to mine deep information. It has achieved good results in short-term power prediction, but the effect is poor in ultra-short-term power prediction. In order to overcome the limitations brought by manual feature extraction, methods based on deep learning have been widely used. For example, the use of long-short-term memory neural networks and recurrent gating units to achieve ultra-short-term prediction of wind power has achieved good results. This type of method does not require manual feature extraction and can automatically learn feature extraction without a lot of prior knowledge. However, there are still defects in feature selection and training stability, and it is impossible to distinguish the hierarchy and differences of the extracted features. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention provides a wind power ultra-short-term prediction method based on a temporal convolutional network, aiming to achieve high-precision wind power ultra-short-term prediction by adaptive feature extraction and weighting.

[0004] The technical solution adopted by the present invention is as follows:

[0005] A wind power ultra-short-term prediction method based on a temporal convolutional network comprises the following steps:

[0006] S1. Collect the actual power generation data within a certain period of time, process the outliers, fill in missing values ​​and normalize the data;

[0007] S2. Divide the processed data into samples, take the power generation several hours before each moment as input data x, and take the power generation 4 hours in the future as prediction data y, take (x, y) as a sample and construct a data set, and then divide it into training set, verification set and test set;

[0008] S3, input the samples of the training set and the validation set into the TCN-CCA network, establish a mapping relationship between the input data and the predicted data, and update the parameters. When the objective function meets the iteration termination condition, the update is stopped to complete the network training;

[0009] The TCN-CCA network is composed of a time convolution network and a channel convolution attention module. The time convolution network includes multiple time convolution modules and a fully connected layer. The channel convolution attention module is arranged between the last time convolution module and the fully connected layer. The channel convolution attention module assigns more weights to the key input features in the time dimension by calculating the probability distribution of attention. The fully connected layer is used to integrate the features in the channel dimension and output the predicted value of wind power.

[0010] S4. Input the samples of the test set into the trained TCN-CCA network to obtain the ultra-short-term prediction value of wind power.

[0011] Further technical solutions are:

[0012] The mathematical model of the channel convolution attention module is as follows:

[0013]

[0014] Where t represents the time, 1≤t≤T, Cov() represents the convolution operation, and F t represents the multiple channel features corresponding to the tth moment extracted by the temporal convolutional network, p t F t The result of the convolution calculation, P is the set of convolution calculation results corresponding to each moment, M(F) is the attention probability matrix obtained by the softmax function of the set P, and its size is T×1;

[0015] Through the mathematical model, the channel convolution attention module first performs a convolution operation on the original feature matrix F in the channel dimension, with a convolution kernel size of 1 and an output of a T×1 matrix. The convolution operation adaptively increases the value of the moment containing important information and reduces the value of the moment containing only minor information through continuous iterative learning, and then calculates the attention probability matrix M(F), and then multiplies M(F) with the corresponding bits of the original feature matrix F to obtain the weighted feature matrix F′, that is, This is achieved by applying greater weights to features of important moments and smaller weights to features of minor moments.

[0016] The temporal convolutional network includes four temporal convolutional modules. The input data and output data of each temporal convolutional module are of equal length. The training is performed by minimizing the loss of both as the objective function. Each temporal convolutional module includes two convolutional layers and sets normalization, ReLU layer, Dropout layer and residual connection. The mathematical model of the temporal convolutional network is as follows:

[0017]

[0018] In the formula, {x i , i=1,2,…,T} is the input sequence, is the prediction sequence, {y i , i = 1, 2, …, T} is the output sequence, L(·) is the loss function between the predicted sequence and the output sequence, and f(·) represents the temporal convolutional network model.

[0019] The loss function uses the mean square error loss function, and the iteration is stopped when the loss function values ​​of the training set and the validation set are small and tend to be stable.

[0020] The beneficial effects of the present invention are as follows:

[0021] The present invention designs a TCN-CCA network suitable for ultra-short-term wind power prediction, realizes the automatic feature extraction of time series data features, can be better applied to ultra-short-term wind power prediction tasks, has higher accuracy, fewer training parameters, and more stable training.

[0022] The present invention combines the temporal convolutional network and the channel convolutional attention module, and the two have good adaptability. The channel convolutional attention module more comprehensively calculates the feature importance in the time dimension and weights it, thereby improving the prediction performance of the temporal convolutional network. At the same time, it does not significantly reduce the model training speed and is effectively applied to ultra-short-term wind power prediction.

[0023] Other features and advantages of the present invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention.

[0025] Figure 2 It is a line chart of the original power data collected by the embodiment of the present invention.

[0026] Figure 3 This is a data line chart after the original data of the embodiment of the present invention is processed.

[0027] Figure 4 Schematic diagram of the TCN-CCA network structure of an embodiment of the present invention.

[0028] Figure 5 It is a training set loss function diagram of the TCN-CCA network in the training process of an embodiment of the present invention.

[0029] Figure 6 This is a graph of the verification set accuracy of the TCN-CCA network during the training process of an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The specific implementation of the present invention is described below with reference to the accompanying drawings.

[0031] This embodiment proposes a wind power ultra-short-term prediction method based on a temporal convolutional network, see Figure 1 , including the following steps:

[0032] S1. Collect the actual power generation data of a wind farm from January 1, 2020 to December 31, 2020, with an interval of 15 minutes and a rated power generation of 48MW. Preprocess the data: outlier processing, missing value filling, and normalization, which specifically includes the following steps:

[0033] Step S11: Power data less than 0, greater than rated power, or wind speed of 0 and power greater than 3MW are regarded as abnormal values; data less than 0 is replaced with 0, data greater than rated power is replaced with rated power, and data with wind speed of 0 and power greater than 3MW are linearly interpolated.

[0034] Step S12: Perform missing value processing on the data after outlier processing in step S11. If the missing value ratio in a day is greater than one-third, delete the whole day data. If the missing value ratio in a day is less than one-third, perform linear interpolation.

[0035] Step S13: Divide the power data processed in step S12 by the rated power to achieve data normalization.

[0036] like Figure 2 and Figure 3 As shown in the figure, they are line graphs before and after data preprocessing. It can be clearly seen that after data preprocessing, there are no outliers or missing values ​​in the data, and time series data with a time interval of 15 minutes per day is formed. The value range is between 0-1, and there are a total of 96 data points per day, which can be used for model training.

[0037] S2. Divide the processed data into samples, with the power generation in the previous 24 hours of each moment as the input data x, whose dimension is 96*1, and the power generation in the next 4 hours of the moment as the predicted data y, whose dimension is 16*1. (x, y) is taken as a sample and a data set is constructed, with a total of 6778 samples, which are then divided into training set, validation set and test set in a ratio of 6:1:3.

[0038] S3, input the samples of the training set and the validation set into the TCN-CCA network, establish a mapping relationship between the input data and the predicted data, and update the parameters in the network. When the objective function meets the iteration termination condition, stop updating and complete the network training.

[0039] like Figure 4 As shown, the TCN-CCA network of this embodiment is composed of a temporal convolutional network (TCN) and a channel convolutional attention module (Channel-Convolutional Attention), and the temporal convolutional network includes multiple temporal convolutional modules and a fully connected layer; the channel convolutional attention module is arranged between the last temporal convolutional module and the fully connected layer.

[0040] Among them, the temporal convolution network includes four temporal convolution modules. The input data and output data of each temporal convolution module are of equal length. The training is performed by minimizing the loss of both as the objective function. Each temporal convolution module includes two convolution layers and sets normalization, ReLU layer, Dropout layer and residual connection. The mathematical model of the temporal convolution network is as follows:

[0041]

[0042] In the formula, {x i , i=1,2,…,T} is the input sequence, is the prediction sequence, {y i , i = 1, 2, …, T} is the output sequence, L(·) is the loss function between the predicted sequence and the output sequence, and f(·) represents the temporal convolutional network model.

[0043] The temporal convolutional network uses causal convolution to realize the causality between sequences, that is, the predicted data at a certain moment is only related to the data at the current moment and before that moment, and has nothing to do with the data after that moment. The expression of causal convolution is as follows:

[0044]

[0045] In the above formula, 1≤i≤T, f(·) represents the temporal convolutional network model.

[0046] The temporal convolutional network introduces dilated convolution and residual connection strategies to improve the expression ability of temporal features, realize parallel computing, and avoid gradient vanishing and explosion problems in the training process.

[0047] The dilated convolution can obtain a larger receptive field to solve the problem of long sequence prediction. For a one-dimensional sequence x∈R n and convolution kernel f:{0, 1,…, k-1}→R, the dilated convolution operation on the sequence element s is defined as:

[0048]

[0049] In the above formula, d is the expansion coefficient, k is the size of the convolution kernel, and the subscript (sd·i) is the sequence number of the previous layer element corresponding to the i-th element f(i) of the convolution kernel when the s-th element is dilated and convolved.

[0050] From the above formula, we can see that increasing the dilated convolution coefficient and increasing the convolution kernel size can increase the TCN receptive field size. The TCN i-th layer network dilation coefficient is usually set to d = O (2 i ) to ensure that the deep network can obtain a sufficiently large receptive field of the sequence.

[0051] Among them, the channel convolution attention module assigns more weights to the key input features in the time dimension by calculating the probability distribution of attention, and the fully connected layer is used to integrate the features in the channel dimension and output the predicted value of wind power.

[0052] The mathematical model of the channel convolution attention module of this embodiment is as follows:

[0053]

[0054] Where t represents the time, 1≤t≤T, Cov() represents the convolution operation, and F t represents the multiple channel features corresponding to the tth moment extracted by the temporal convolutional network, p t F t The result of the convolution calculation, P is the set of convolution calculation results corresponding to each moment, M(F) is the attention probability matrix obtained by the softmax function of the set P, and its size is T×1;

[0055] Through the above mathematical model, the channel convolution attention module first performs a convolution operation on the original feature matrix F in the channel dimension. The convolution kernel size is 1, and the output is a T×1 matrix. The convolution operation adaptively increases the value of the moment containing important information and reduces the value of the moment containing only minor information through continuous iterative learning, and then calculates the attention probability matrix M(F), and then multiplies M(F) with the corresponding position of the original feature matrix F to obtain the weighted feature matrix F′, that is, This is achieved by applying greater weights to features of important moments and smaller weights to features of minor moments.

[0056] See also Figure 4 The input data first passes through four time convolution modules to extract multiple time series features. The input data and output data of each time convolution module are of equal length, and the number of channels is 20. As the number of layers of the time convolution module increases, the size of the dilated convolution increases exponentially by 2, so the obtained field of view is wider. The channel convolution attention module added after the last time convolution module weights the features in the time dimension, and finally integrates the features in the channel dimension through the fully connected layer to output the predicted value of wind power.

[0057] Specifically, the SGD optimizer is used during training, the batch size is 32, the Dropout is set to 0.2, the initial learning rate is 0.01, the loss function uses the mean square error loss function, the learning rate decay is set, and the update is stopped when the objective function meets the iteration termination condition.

[0058] like Figure 5 As shown, the loss function of the TCN-CCA network training set changes with the number of iterations. Figure 6 This figure shows how the accuracy of the validation set changes with the number of iterations. It can be seen that at the beginning of training, the loss function is large and the accuracy of the validation set is low. As the training progresses, the loss function decreases rapidly and the accuracy increases rapidly. When the number of iterations reaches 100, the loss function is basically stable at around 0.05 and the accuracy is stable at around 0.98.

[0059] S4: Input the samples of the test set into the trained TCN-CCA network to obtain the ultra-short-term prediction value of wind power.

[0060] In order to verify the performance of the TCN-CCA model proposed in this application, three deep learning models commonly used for ultra-short-term wind power prediction were used for comparison, namely LSTM, GRU and TCN. The same test data was input into the four network models respectively, and the predicted wind power values ​​were obtained and compared with the actual wind power to calculate the prediction accuracy of each model.

[0061] The TCN network consists of 5 residual block layers plus one fully connected layer, with a convolution kernel size of 5; the number of LSTM and GRU network layers is 2, and the number of hidden neurons is 30. The network iteration is set to 300 rounds, Dropout is set to 0.2, the SGD optimizer is used, the batch size is 32, the loss function is set to the mean square error loss function, the initial learning rate is 0.01, and the learning rate decay strategy is adopted.

[0062] Table 1 shows the accuracy of the TCN-CCA model and the three comparison models. It can be seen that in the ultra-short-term wind power prediction task, the four models all performed well, among which the LSTM and GRU recurrent neural network models can achieve an accuracy of more than 97% in the validation set and an accuracy of more than 96% in the test set.

[0063] Table 1

[0064]

[0065] This embodiment introduces a channel convolution attention module on the basis of the TCN model in the TCN-CCA model to achieve adaptive weighting of features in the time dimension, and the accuracy on the validation set is improved from 96.93% to 98.47%, and the accuracy on the test set is improved from 96.50 to 97.59%. In addition, in order to more comprehensively analyze the training status of each model, this experiment calculated the single-round time consumption during the training process, among which LSTM, TCN and TCN-CCA time consumption is basically the same, and GRU time consumption is slightly lower than the above three models. The above analysis can prove that TCN and TCN-CCA can be effectively applied to ultra-short-term wind power prediction, and the introduction of the attention mechanism improves the prediction performance of the temporal convolutional network without significantly reducing the model training speed.

[0066] Those skilled in the art can understand that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention is described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions recorded in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A wind power ultra-short-term prediction method based on a temporal convolutional network, characterized in that: The following steps are involved: S1. Collect the actual power generation data within a certain period of time, process the outliers, fill in missing values ​​and normalize the data; S2. Divide the processed data into samples, take the power generation several hours before each moment as input data x, and take the power generation 4 hours in the future as prediction data y, take (x, y) as a sample and construct a data set, and then divide it into training set, verification set and test set; S3, input the samples of the training set and the validation set into the TCN-CCA network, establish a mapping relationship between the input data and the predicted data, and update the parameters. When the objective function meets the iteration termination condition, the update is stopped to complete the network training; The TCN-CCA network is composed of a time convolution network and a channel convolution attention module. The time convolution network includes multiple time convolution modules and a fully connected layer. The channel convolution attention module is arranged between the last time convolution module and the fully connected layer. The channel convolution attention module assigns more weights to the key input features in the time dimension by calculating the probability distribution of attention. The fully connected layer is used to integrate the features in the channel dimension and output the predicted value of wind power. S4, inputting the samples of the test set into the trained TCN-CCA network to obtain an ultra-short-term prediction value of wind power; The mathematical model of the channel convolution attention module is as follows: Where t represents the time, 1≤t≤T, Cov() represents the convolution operation, and F t represents the multiple channel features corresponding to the tth moment extracted by the temporal convolutional network, p t F t The result of the convolution calculation, P is the set of convolution calculation results corresponding to each moment, M(F) is the attention probability matrix obtained by the softmax function of the set P, and its size is T×1; Through the mathematical model, the channel convolution attention module first performs a convolution operation on the original feature matrix F in the channel dimension, with a convolution kernel size of 1 and an output of a T×1 matrix. The convolution operation adaptively increases the value of the moment containing important information and reduces the value of the moment containing only minor information through continuous iterative learning, and then calculates the attention probability matrix M(F), and then multiplies M(F) with the corresponding bits of the original feature matrix F to obtain the weighted feature matrix F', that is, To achieve greater weights for features of important moments and less weights for features of minor moments; The temporal convolutional network includes four temporal convolutional modules. The input data and output data of each temporal convolutional module are of equal length. The training is performed by minimizing the loss of both as the objective function. Each temporal convolutional module includes two convolutional layers and sets normalization, ReLU layer, Dropout layer and residual connection. The mathematical model of the temporal convolutional network is as follows: In the formula, {x i ,i=1,2,…,T} is the input sequence, is the prediction sequence, {y i ,i=1,2,…,T} is the output sequence, L(·) is the loss function between the prediction sequence and the output sequence, and f(·) represents the temporal convolutional network model.

2. The ultra-short-term wind power prediction method based on a temporal convolutional network according to claim 1 is characterized in that: During training, the loss function uses the mean square error loss function, and the iteration is stopped when the loss function values ​​of the training set and the validation set are small and tend to be stable.