CNN-XGBoost power load prediction method based on attention mechanism

Abstract

The invention discloses a CNN-XGBoost power load prediction method based on an attention mechanism, and the method comprises the steps: obtaining related historical load data and feature data, carrying out the data correlation analysis, employing the CNN to construct a feature extraction module based on the advantages of the CNN in feature extraction, and carrying out the calculation of the data correlation. Aiming at the problem that a traditional convolutional neural network cannot distinguish whether the extracted features are effective for prediction, Attention is introduced, different weights are given to the features, the features with different output contribution degrees are distinguished, the extracted features are emphasized consciously, the relation between the extracted features and labels is explored, and an XGBoost algorithm is combined to determine whether the extracted features are effective for prediction or not. According to the method, the advantages of the CNN algorithm in the aspect of feature extraction and the advantages of the XGBoost algorithm of high nonlinear load prediction precision and high prediction convergence speed can be integrated, and the accuracy and stability of short-term load prediction of power consumption of the whole network are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power load prediction, and specifically relates to a power load prediction method of a CNN-XGBoost algorithm based on an attention mechanism. Background Art

[0002] With the continuous innovation and development of the power grid system and the substantial increase in the complexity of the power grid, the load requirements of people's production and life electricity over the years have led researchers to conduct more in-depth research on traditional power load forecasting technology, gradually developing into today's artificial intelligence forecasting field, and continuously improving its prediction accuracy and model performance. High-precision short-term power load forecasting optimizes power grid operation strategies and improves the reliable guarantee of power grid efficiency, which is of great significance to the stability analysis, safety assessment and real-time electricity price calculation of the power system.

[0003] At present, machine learning and deep learning methods have become popular methods for power load forecasting due to their outstanding performance in accuracy and computational efficiency. Taking advantage of CNN in the field of data mining, a load forecasting method based on a hybrid neural network of convolutional neural networks and gated recurrent units is proposed. The input data is clustered by the K-means method, and then the input data features are extracted by the CNN neural network. Finally, the prediction is made by the GRU neural network. The above method solves the problem that GRU needs to manually construct feature relationships, and has adaptive functions for a large number of non-structural and non-precise rules, and good generalization performance of prediction objects. However, it also has the disadvantages of complex optimization process, slow convergence speed or large error and low stability. The prediction accuracy of the XGBoost model is higher than that of the prediction models based on BP, LSTM, and GRU neural network algorithms, but power load data usually has obvious seasonality and periodicity. XGBoost is not good at capturing these seasonality and periodicity. However, the advantages of CNN-Attention feature extraction and XGBoost's fast convergence speed and high prediction accuracy can well solve the problems of slow convergence speed and prediction accuracy of load forecasting. Summary of the invention

[0004] The present invention mainly aims at the problems that the existing technology has slow convergence of load forecasting speed, no feature extraction in time and space dimensions for load forecasting, and unstable prediction accuracy due to power grid fluctuations, and proposes a power load forecasting method based on the CNN-XGBoost algorithm of the attention mechanism.

[0005] The above-mentioned problems of the present invention are mainly solved by the following technical solutions:

[0006] The power load forecasting method based on CNN-XGBoost of attention mechanism is characterized by comprising the following steps:

[0007] S1. Based on the historical operation information and load information of the power grid, a historical sample database is established using the correlation analysis method;

[0008] S2. Based on the historical sample database, the CNN algorithm combined with the attention mechanism is used to establish a feature extraction module for the entire power consumption in the region and extract the key features that affect the load data;

[0009] S3, based on the key features extracted in step S2 and the historical load information of the power grid in the historical sample database, the XGBoost method is used for training to generate an artificial intelligence prediction model for power consumption of the entire network;

[0010] S4. Based on the key features extracted in step S2, the key feature forecast data of the whole network's electricity consumption in the next one and three days is obtained at time intervals of 15 minutes. The key feature forecast data is used as input features, and the artificial intelligence model generated in step S3 is used to predict the load curve of the whole network's electricity consumption in the next one and three days.

[0011] The beneficial effects of the present invention are

[0012] (1) The present invention introduces the method of CNN and attention mechanism to obtain the key features of historical load information. Specifically, it solves the problem that the features of historical load information are complex and nonlinear and difficult to extract. In addition, the attention mechanism is used to overcome the problem that the traditional CNN model cannot distinguish whether the feature channels are important or not.

[0013] (2) The present invention uses the XGBoost algorithm as the algorithm model for load forecasting. The algorithm can control the complexity of the model to prevent overfitting and improve the convergence speed and convergence accuracy. In addition, the algorithm has a better prediction effect for the nonlinearity of the load;

[0014] (3) The present invention adopts a method combining CNN algorithm with XGBoost algorithm, and utilizes CNN combined with attention model as a load feature extraction module to strengthen the influence of key features on the output, and inputting features into the XGBoost model can further improve the prediction speed. The combination of the two improves the stability of prediction under power grid fluctuations. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of a method flow in an embodiment of the present invention.

[0016] Figure 2 It is a schematic diagram of a feature extraction module combining a CNN and an attention mechanism algorithm in an embodiment of the present invention.

[0017] Figure 3 It is an overall framework diagram of the method in the embodiment of the present invention. DETAILED DESCRIPTION

[0018] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0019] like Figure 1 As shown, a power load forecasting method based on the CNN-XGBoost algorithm of the attention mechanism includes the following steps:

[0020] S1. Based on the historical operation information and load information of the power grid, a historical sample database is established using the correlation analysis method;

[0021] S2. Based on the historical sample database, the CNN algorithm combined with the attention mechanism is used to establish a feature extraction module for the entire power consumption in the region and extract the key features that affect the load data;

[0022] S3, based on key features and historical load information, uses the XGBoost method for training to generate an artificial intelligence prediction model for power consumption across the entire network;

[0023] S4. Based on the key features, the key feature forecast data of the whole network's electricity consumption in the next one and three days is obtained at time intervals of 15 minutes. It is used as the input feature and the generated artificial intelligence model is used to predict the load curve of the whole network's electricity consumption in the next one and three days.

[0024] Step S1 is specifically as follows:

[0025] S11. Collect historical load information, as well as weather, date type (weekends, holidays, weekdays), temperature, and humidity information data of past dates, and perform data cleaning and data missing filling. Missing historical load information is filled with missing values ​​using the median method, and missing weather-related data is filled with missing values ​​using forward and backward filling methods;

[0026] S12. Use the Pearson correlation coefficient method to extract each feature and the load value of the corresponding time period to form a data pair of (feature value, load), and calculate the correlation coefficient between the feature data and the load data. If the calculation result is between (-0.5, 0.5), it is considered that the linear correlation between the feature and the load is weak. The features with weak correlation are eliminated, leaving the features with strong correlation, and these feature information and historical load information are saved in the database to form a historical information database.

[0027] Step S2 is specifically as follows:

[0028] S21, CNN convolution layer realizes feature extraction of input information through convolution kernel. The convolution kernel in CNN convolution layer moves and scans from left to right and then from top to bottom according to the preset sliding step until the entire feature map is fully input. Each convolution layer contains multiple convolution kernels to jointly operate on the input information and extract key features. The calculation formula of the convolution kernel is:

[0029]

[0030] in, is the jth output feature map of the lth convolutional layer, f() is the activation function, is the i-th input feature map of the l-1th layer, is the weight corresponding to the convolution kernel of the lth layer, * is the dot product,, It is the bias corresponding to the j-th feature map output by the l-th convolutional layer. l represents the network layer index, indicating that the current convolution is the l-th layer. i represents the index of the input feature map, and j represents the index of the output feature map.

[0031] S22. The role of the CNN pooling layer is to perform secondary selection on the features extracted in the convolutional layer, reduce the dimensionality of the features, retain the most significant features, thereby discarding redundant or irrelevant information and enhancing robustness. The role of the CNN fully connected layer is to flatten the multi-dimensional feature map output by the convolutional layer or pooling layer into a one-dimensional vector to further extract global information.

[0032] S23. Introduce the attention mechanism, input the feature map output by the CNN convolution layer into the attention mechanism, enhance the weight of the feature, use the channel attention network (SENet), automatically obtain the importance of each channel through learning, and then enhance the useful features according to the importance and suppress the features that are not very useful for the current task to improve the accuracy of model prediction. First, SENet performs global average pooling on the output feature map of the CNN convolution layer and calculates the global statistical features of each channel. After pooling, the channel becomes a single scalar. These scalars are fully connected at two levels, first compressing the features and then expanding them. The sigmoid function is used to calculate the weight of each channel and limit the weight to [0,1] to indicate the importance of the channel. Multiply this weight to the channel of the CNN convolution result as the input of the next level. By controlling the size of the weight, important features are enhanced and unimportant features are weakened, so that the extracted features are more directional. The sigmoid function is as follows:

[0033]

[0034] Where x represents the channel feature value generated by global average pooling and two-level full connection, representing the importance score of the channel. It is an intermediate variable output by the fully connected layer and is compressed by the sigmoid function. When the S(x) value is close to 1, it means that the importance of the channel is high, and when the S(x) value is close to 0, it means that the importance of the channel is low.

[0035] Step S3 is specifically as follows:

[0036] S31. The feature data and load information are used to form a test set and a training set, and the XGBoost method is used for training. The hyperparameters of XGBoost can be adjusted by using algorithms such as grid search or manual parameter adjustment methods to obtain the optimal tree depth, optimal number of iterations, and shrinkage step size on the XGBoost model training set.

[0037] S32, the characteristic data and load information are used to form a test set and a training set, and the XGBoost method is used for training. In order to evaluate the prediction accuracy of the model, the mean absolute percentage error M is selected. APE and the root mean square error R MSE As the evaluation criteria, their expressions are:

[0038]

[0039]

[0040] In the formula, n represents the total number of prediction results; r i and are the actual load value and predicted load value of the i-th sampling point respectively; M APE R is a measure of the quality of the model's prediction results. MSE Evaluate the accuracy of the forecast and be sensitive to large or small errors in the results; in power load forecasting, M APE and R MSE The smaller the value of is, the more accurate the load forecast result is;

[0041] Step 4 is as follows:

[0042] S41. With 15-minute time intervals, 24 hours a day, and 96 points, obtain forecast data of other key features corresponding to the future load moments of the entire network, such as temperature, humidity, whether it is a weekend, whether it is a holiday, etc. as input features.

[0043] S42, and the key features extracted by CNN form c i =[t i ,h i , k i1 , w i ...], where ci is the input feature predicted at time i, t i is the predicted temperature characteristic at time i, k i1 is the CNN feature extraction at time i, w i Whether the i moment is a weekend, holiday or vacation, the c of the 96 moments in a day i By inputting the generated XGBoost model, the load curve of the predicted day can be generated. According to the different dates to be predicted, prediction curves for different dates can be generated. If the load for three consecutive days needs to be predicted, it can be predicted day by day to obtain the prediction curves for the next three days.

[0044] The present invention introduces the method of CNN and attention mechanism to obtain the key features of historical load information. Specifically, it solves the problem that the features of historical load information are complex and nonlinear and difficult to extract, and uses the attention model to overcome the situation that the traditional CNN model cannot distinguish whether the feature channels are important or not.

[0045] The present invention uses the XGBoost algorithm as an algorithm model for load forecasting. The algorithm can control the complexity of the model to prevent overfitting and improve the convergence speed and convergence accuracy. In addition, the algorithm has a better prediction effect for the nonlinearity of the load.

[0046] The present invention adopts a method combining the CNN algorithm with the XGBoost algorithm, utilizes the combination of CNN and attention mechanism as a load feature extraction module, strengthens the influence of key features on the output, and inputs the features into the XGBoost model to further improve the prediction speed. The combination of the two improves the stability of prediction under power grid fluctuations.

[0047] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. A power load forecasting method based on CNN-XGBoost with attention mechanism, characterized by: The following steps are involved: S1. Based on the historical operation information and load information of the power grid, a historical sample database is established using the correlation analysis method; S2. Based on the historical sample database, the CNN algorithm combined with the attention mechanism is used to establish a feature extraction module for the entire power consumption in the region and extract the key features that affect the load data; S3, based on the key features extracted in step S2 and the historical load information of the power grid in the historical sample database, the XGBoost method is used for training to generate an artificial intelligence prediction model for power consumption of the entire network; S4. Based on the key features extracted in step S2, the key feature forecast data of the whole network's electricity consumption in the next one and three days is obtained at time intervals of 15 minutes. The key feature forecast data is used as input features, and the artificial intelligence model generated in step S3 is used to predict the load curve of the whole network's electricity consumption in the next one and three days.

2. The power load forecasting method based on the CNN-XGBoost algorithm based on the attention mechanism according to claim 1, characterized in that: Step S1 specifically comprises: S11. Collect historical load information, as well as weather, date type (weekends, holidays, weekdays), temperature, and humidity information data of past dates, and perform data cleaning and data missing filling. Missing historical load information is filled with missing values ​​using the median method, and missing weather-related data is filled with missing values ​​using forward and backward filling methods; S12. Use the Pearson correlation coefficient method to extract each feature and the load value of the corresponding time period to form a data pair of (feature value, load), and calculate the correlation coefficient between the feature data and the load data. If the calculation result is between (-0.5, 0.5), it is considered that the linear correlation between the feature and the load is weak. The features with weak correlation are eliminated, leaving the features with strong correlation, and these feature information and historical load information are saved in the database to form a historical information database.

3. The power load forecasting method based on the CNN-XGBoost algorithm based on the attention mechanism according to claim 1, characterized in that: Step S2 specifically includes: S21, CNN convolution layer realizes feature extraction of input information through convolution kernel. The convolution kernel in CNN convolution layer moves and scans from left to right and then from top to bottom according to the preset sliding step until the entire feature map is fully input. Each convolution layer contains multiple convolution kernels to jointly operate on the input information and extract key features; the operation formula of the convolution kernel is: in, is the jth output feature map of the lth convolutional layer, f() is the activation function, is the i-th input feature map of the l-1th layer, is the weight corresponding to the convolution kernel of the lth layer, * is the dot product,, The offset corresponding to the jth feature map output by the lth convolutional layer, l represents the network layer index, indicating that the current convolution is the lth layer, i represents the index of the input feature map, and j represents the index of the output feature map; S22. The role of the CNN pooling layer is to perform secondary selection on the features extracted in the convolutional layer, reduce the dimensionality of the features, retain the most significant features, thereby discarding redundant or irrelevant information and enhancing robustness. The role of the CNN fully connected layer is to flatten the multi-dimensional feature map output by the convolutional layer or pooling layer into a one-dimensional vector to further extract global information. S23. Introduce the attention mechanism, input the feature map output by the CNN convolution layer into the attention mechanism, enhance the weight of the feature, use the channel attention network (SENet), automatically obtain the importance of each channel through learning, and then enhance the useful features according to the importance and suppress the features that are not very useful for the current task to improve the accuracy of model prediction. First, SENet performs global average pooling on the output feature map of the CNN convolution layer and calculates the global statistical features of each channel. After pooling, the channel becomes a single scalar. These scalars are fully connected at two levels, first compressing the features and then expanding them. The sigmoid function is used to calculate the weight of each channel and limit the weight to [0,1] to indicate the importance of the channel. Multiply this weight to the channel of the CNN convolution result as the input of the next level. By controlling the size of the weight, important features are enhanced and unimportant features are weakened, so that the extracted features are more directional. The sigmoid function is as follows: Where x represents the channel feature value produced by global average pooling and two-level full connection, representing the importance score of the channel. It is an intermediate variable output by the fully connected layer and is compressed by the sigmoid function. When the S(x) value is close to 1, it means that the importance of the channel is high, and when the S(x) value is close to 0, it means that the importance of the channel is low.

4. The power load forecasting method based on the CNN-XGBoost algorithm based on the attention mechanism according to claim 1, characterized in that: Step S3 specifically comprises: S31, the characteristic data and load information are used to form a test set and a training set, and the XGBoost method is used for training. In order to evaluate the prediction accuracy of the model, the mean absolute percentage error M is selected. APE and the root mean square error R MSE As the evaluation criteria, their expressions are: In the formula, n represents the total number of prediction results; r i and are the actual load value and predicted load value of the i-th sampling point respectively; M APE R is a measure of the quality of the model's prediction results. MSE Evaluate the accuracy of the forecast and be sensitive to large or small errors in the results; in power load forecasting, M APE and R MSE The smaller the value of is, the more accurate the load forecast result is; S32, XGBoost hyperparameters are adjusted by grid search or manual parameter adjustment to obtain the optimal tree depth, optimal number of iterations, and shrinkage step size on the XGBoost model training set.

5. The power load forecasting method based on the CNN-XGBoost algorithm based on the attention mechanism according to claim 1, characterized in that: Step S4 specifically includes: S41, with 15-minute time intervals, 24 hours a day, and a total of 96 points, obtain forecast data of other key features corresponding to the future load moment of the entire network, such as temperature, humidity, whether it is a weekend, and whether it is a holiday as input features; S42, and the key features extracted by CNN form c i =[t i ,h i ,k i1 ,w i …], where c i is the input feature predicted at time i, t i is the predicted temperature characteristic at time i, k i1 is the CNN feature extraction at time i, w i Whether the i moment is a weekend, holiday or vacation, the c of the 96 moments of the day i By inputting the generated XGBoost model, the load curve of the predicted day can be generated. According to the different dates to be predicted, prediction curves for different dates can be generated. If the load for three consecutive days needs to be predicted, it can be predicted day by day to obtain the prediction curves for the next three days.

Images