Power load forecasting method based on attention mechanism of CNN-XGBoost

Abstract

The power load prediction method based on the attention mechanism of CNN-XGBoost obtains relevant historical load data and feature data, and performs data correlation analysis, on the basis of which, the advantages of CNN in feature extraction are utilized, a feature extraction module is constructed by using CNN, and in view of the problem that the traditional convolutional neural network cannot distinguish whether the extracted features are effective for prediction, the attention mechanism is introduced, different weights are given to the features, features with different contribution degrees to the output are distinguished, the extracted features are consciously focused on, the relationship between the extracted features and labels is explored, and then the extracted features are input into the XGBoost structure to perform short-term load prediction. The method can comprehensively utilize the advantages of the CNN algorithm in feature extraction and the advantages of the XGBoost algorithm in high prediction accuracy and fast prediction convergence speed for nonlinear load, and improve the accuracy and stability of the short-term load prediction of the whole network.

Description

Technical Field

[0001] This invention belongs to the field of electricity load forecasting technology, specifically relating to an electricity load forecasting method based on the CNN-XGBoost algorithm with an attention mechanism. Background Technology

[0002] With the continuous innovation and development of power grid systems and the significant increase in grid complexity, researchers have conducted in-depth studies on traditional power load forecasting techniques to meet the increasing electricity demands of people's production and daily life over the years. This has gradually evolved into the field of artificial intelligence forecasting, continuously improving its prediction accuracy and model performance. High-precision short-term power load forecasting is a reliable guarantee for optimizing power grid operation strategies and improving grid efficiency, and it is of great significance for power system stability analysis, security assessment, and real-time electricity price calculation.

[0003] Currently, machine learning and deep learning methods have become popular for electricity load forecasting due to their outstanding performance in both accuracy and computational efficiency. Leveraging the advantages of CNNs in data mining, a load forecasting method based on a hybrid neural network of convolutional neural networks and gated recurrent units (GRUs) is proposed. This method clusters the input data using K-means, extracts features from the input data using a CNN, and finally performs prediction using a GRU. This approach solves the problem of GRU requiring manual construction of feature relationships and exhibits adaptive capabilities for a large number of unstructured and imprecise patterns, as well as good generalization performance. However, it also suffers from drawbacks such as complex optimization processes, slow convergence speed, large errors, and low stability. The XGBoost model has higher prediction accuracy than prediction models based on BP, LSTM, and GRU neural network algorithms, but electricity load data typically exhibits significant seasonality and periodicity. XGBoost is not adept at capturing these seasonalities and periodicities. However, by utilizing the feature extraction advantages of CNN-Attention and the fast convergence speed and high prediction accuracy of XGBoost, the problems of slow convergence speed and high prediction accuracy in load forecasting can be effectively solved. Summary of the Invention

[0004] This invention addresses the problems of slow convergence speed, lack of feature extraction in the time and space dimensions of load forecasting, and unstable forecast accuracy due to power grid fluctuations in existing technologies. It proposes a power load forecasting method based on the CNN-XGBoost algorithm with an attention mechanism.

[0005] The above-mentioned problems of the present invention are mainly solved by the following technical solutions:

[0006] The CNN-XGBoost-based power load prediction method based on the attention mechanism is characterized by the following steps:

[0007] S1. Based on historical power grid operation information and load information, a historical sample database is established using correlation analysis methods;

[0008] S2. Based on historical sample databases, a feature extraction module for the entire power grid in the region is established using a CNN algorithm combined with an attention mechanism to extract key features that affect load data.

[0009] S3. Based on the key features extracted in step S2 and the historical load information of the power grid in the historical sample database, the XGBoost method is used for training to generate an artificial intelligence prediction model for the entire power grid.

[0010] S4. Based on the key features extracted in step S2, obtain key feature forecast data of the whole network's electricity consumption for the next day and three days at 15-minute intervals. Using these as input features, use the artificial intelligence model generated in step S3 to predict the load curve of the whole network's electricity consumption for the next day and three days.

[0011] The beneficial effects of this invention are

[0012] (1) This invention introduces a CNN and attention mechanism to obtain key features of historical load information. Specifically, it solves the problem that historical load information features are complex and nonlinear and difficult to extract. Furthermore, it uses the attention mechanism to overcome the situation that traditional CNN models cannot distinguish whether feature channels are important or not.

[0013] (2) The present invention uses the XGBoost algorithm as the algorithm model for load prediction. This algorithm can control the complexity of the model to prevent overfitting and improve the convergence speed and convergence accuracy. In addition, the algorithm has a better prediction effect for nonlinear loads.

[0014] (3) The present invention adopts a method combining CNN algorithm and XGBoost algorithm. It utilizes the combination of CNN and attention model as load feature extraction module to strengthen the influence of key features on output. Furthermore, inputting features into XGBoost model can improve prediction speed. Combining the two improves the stability of prediction under power grid fluctuation. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the method flow in an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of the feature extraction module combining CNN and attention mechanism algorithms in an embodiment of the present invention.

[0017] Figure 3 This is an overall framework diagram of the method in the embodiments of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.

[0019] like Figure 1 As shown, a power load forecasting method based on the attention mechanism CNN-XGBoost algorithm includes the following steps:

[0020] S1. Based on historical power grid operation information and load information, a historical sample database is established using correlation analysis methods;

[0021] S2. Based on historical sample databases, a feature extraction module for the entire power grid in the region is established using a CNN algorithm combined with an attention mechanism to extract key features that affect load data.

[0022] S3. Based on key features and historical load information, the XGBoost method is used for training to generate an artificial intelligence prediction model for the entire network's electricity consumption.

[0023] S4. Based on key features, acquire key feature forecast data of the entire network's electricity consumption for the next day and three days at 15-minute intervals. Use these as input features and the generated artificial intelligence model to predict the load curve of the entire network's electricity consumption for the next day and three days.

[0024] Step S1 is as follows:

[0025] S11. Collect historical load information, as well as weather, date type (weekend, holiday, weekday), temperature, and humidity information data for past dates, and perform data cleaning and data missing filling. Missing historical load information is filled using the median method, and missing weather-related data is filled using forward and backward filling methods.

[0026] S12. Using the Pearson correlation coefficient method, extract each feature and the corresponding load value for the time period to form a data pair of (feature value, load). Calculate the correlation coefficient between the feature data and the load data. If the calculation result is between (-0.5, 0.5), it is considered that the linear correlation between the feature and the load is weak. Remove the weakly correlated features and keep the strongly correlated features. Save these feature information and historical load information into the database to form a historical information database.

[0027] Step S2 is as follows:

[0028] S21. CNN convolutional layers extract features from the input information through convolutional kernels. The convolutional kernels in a CNN convolutional layer move and scan from left to right and then from top to bottom with a preset stride until the entire feature map is input. Typically, each convolutional layer contains multiple convolutional kernels that work together to process the input information and extract key features. The formula for the convolutional kernel operation is:

[0029]

[0030] in, Let f() be the j-th output feature map of the l-th convolutional layer. For the i-th input feature map of the (l-1)-th layer, Here are the weights corresponding to the l-th convolutional kernel, and * represents the dot product. This is the bias corresponding to the j-th feature map output by the l-th convolutional layer, where l represents the layer index of the network, i represents the index of the input feature map, and j represents the index of the output feature map.

[0031] S22. The role of the pooling layer in CNN is to perform secondary selection on the features extracted by the convolutional layer, reduce the dimensionality of the features, retain the most significant features, and discard redundant or irrelevant information to enhance robustness. The role of the fully connected layer in CNN is to flatten the multi-dimensional feature map output by the convolutional or pooling layer into a one-dimensional vector to further extract global information.

[0032] S23. An attention mechanism is introduced, inputting the feature maps output from the CNN convolutional layers into the attention mechanism to enhance feature weights. Channel Attention Network (SENet) is used, which automatically learns the importance of each channel and then enhances useful features while suppressing features less useful for the current task, thus improving the model's prediction accuracy. First, SENet performs global average pooling on the output feature maps of the CNN convolutional layers, calculating the global statistical features of each channel. After pooling, the channel becomes a single scalar. These scalars are then processed through two fully connected layers, first compressing the features and then expanding them. The sigmoid function is used to calculate the weight of each channel, limiting the weights to [0,1] to represent channel importance. This weight is multiplied by the channel of the CNN convolution result and used as the input for the next stage. By controlling the magnitude of the weights, important features are enhanced, while unimportant features are weakened, thus making the extracted features more targeted. The sigmoid function is as follows:

[0033]

[0034] Where x represents the channel feature value generated after global average pooling and two-stage fully connected layers, representing the channel importance score. It is an intermediate variable of the fully connected layer output and is compressed by the sigmoid function. When the value of S(x) is close to 1, it indicates that the channel is of high importance, and when the value of S(x) is close to 0, it indicates that the channel is of low importance.

[0035] Step S3 is as follows:

[0036] S31. Combine the feature data and load information to form a test set and a training set. Train the model using the XGBoost method. You can use algorithms such as grid search or manual parameter tuning to adjust the hyperparameters of XGBoost so that the optimal tree depth, optimal number of iterations, and shrinkage step size are obtained on the XGBoost model training set.

[0037] S32. The feature data and load information are used to form a test set and a training set. The XGBoost method is used for training. To evaluate the model's prediction accuracy, the mean absolute error M is selected. APE and root mean square error R MSE As evaluation criteria, their expressions are as follows:

[0038]

[0039]

[0040] In the formula, n represents the total number of prediction results; r i and M represents the actual load value and the predicted load value for the i-th sampling point, respectively; APE R is used to measure the quality of a model's predictions. MSE The accuracy of the forecast is evaluated, and it is sensitive to both extremely large and extremely small errors in the results; in power load forecasting, M APE and R MSE The smaller the value, the more accurate the load forecast result;

[0041] Step 4 specifically involves:

[0042] S41. At 15-minute intervals, for 24 hours a day, obtain forecast data of other key characteristics corresponding to the future load of the entire network's electricity consumption at 96 points, such as temperature, humidity, whether it is a weekend, whether it is a holiday, etc. as input features.

[0043] S42, together with the key features extracted by CNN, form c i =[t i h i k i1 w i ...], where ci Let t be the input feature predicted at time i. i Let k be the predicted temperature characteristic at time i. i1 To extract features for the CNN at time i, w i To determine whether time i is a weekend, holiday, or vacation, calculate c for each of the 96 times in a day. i Inputting the data into the generated XGBoost model will produce the load curve for that forecast day. Different forecast curves can be generated for different dates depending on the forecast date. If it is necessary to forecast the load for three consecutive days, forecasts can be made day by day to obtain the forecast curves for the following three days.

[0044] This invention introduces a CNN and attention mechanism to obtain key features of historical load information. Specifically, it solves the problem that historical load information features are complex and nonlinear, making them difficult to extract. Furthermore, it utilizes an attention model to overcome the inability of traditional CNN models to distinguish the importance of feature channels.

[0045] This invention utilizes the XGBoost algorithm as the algorithm model for load forecasting. This algorithm can control the complexity of the model to prevent overfitting and improve the convergence speed and convergence accuracy. In addition, this algorithm has better prediction performance for nonlinear loads.

[0046] This invention employs a method combining CNN and XGBoost algorithms. It utilizes a combination of CNN and attention mechanisms as a load feature extraction module, enhancing the impact of key features on the output. Furthermore, inputting these features into the XGBoost model improves prediction speed. Combining the two enhances the stability of predictions under power grid fluctuations.

[0047] The above description is only a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by those skilled in the art based on the content disclosed in the present invention should be included within the scope of protection set forth in the claims.

Claims

1. A power load prediction method based on attention mechanism CNN-XGBoost, characterized in that: Includes the following steps: S1. Based on historical power grid operation information and load information, a historical sample database is established using correlation analysis methods; S2. Based on historical sample database, a feature extraction module for regional power consumption is established using CNN algorithm combined with attention mechanism to extract key features affecting load data; Step S2 specifically involves: S21. The CNN convolutional layer extracts features from the input information through convolutional kernels. The convolutional kernels in the CNN convolutional layer move and scan from left to right and then from top to bottom according to a preset sliding stride until the entire feature map is completely input. Each convolutional layer contains multiple convolutional kernels that work together to process the input information and extract key features. The formula for operating on a convolution kernel is: in, Let f() be the j-th output feature map of the l-th convolutional layer. For the i-th input feature map of the (l-1)-th layer, Here are the weights corresponding to the l-th convolutional kernel, and * represents the dot product. This is the bias corresponding to the j-th feature map output by the l-th convolutional layer, where l represents the layer index of the network, i represents the index of the input feature map, and j represents the index of the output feature map. S22. The role of the pooling layer in CNN is to perform secondary selection on the features extracted by the convolutional layer, reduce the dimensionality of the features, retain the most significant features, thereby discarding redundant or irrelevant information and enhancing robustness. The role of the fully connected layer in CNN is to flatten the multi-dimensional feature map output by the convolutional layer or pooling layer into a one-dimensional vector, and further extract global information. S23. An attention mechanism is introduced, inputting the feature maps output from the CNN convolutional layers into the attention mechanism to enhance feature weights. Channel Attention Network (SENet) is used, which automatically learns the importance of each channel and then enhances useful features while suppressing features less useful for the current task, thus improving the model's prediction accuracy. First, SENet performs global average pooling on the output feature maps of the CNN convolutional layers, calculating the global statistical features of each channel. After pooling, the channel becomes a single scalar. These scalars are then processed through two fully connected layers, first compressing the features and then expanding them. The sigmoid function is used to calculate the weight of each channel, limiting the weights to [0,1] to represent channel importance. This weight is multiplied by the channel of the CNN convolution result and used as the input for the next stage. By controlling the magnitude of the weights, important features are enhanced, while unimportant features are weakened, thus making the extracted features more targeted. The sigmoid function is as follows: Where x represents the channel feature value produced after global average pooling and two-stage fully connected layers, and represents the channel importance score. It is an intermediate variable of the fully connected layer output and is compressed by the sigmoid function; when A value close to 1 indicates that the channel is of high importance. A value close to 0 indicates that the channel is of low importance; S3. Based on the key features extracted in step S2 and the historical load information of the power grid in the historical sample database, the XGBoost method is used for training to generate an artificial intelligence prediction model for the entire power grid. Step S3 specifically involves: S31. The feature data and load information are used to form a test set and a training set. The XGBoost method is used for training. To evaluate the model's prediction accuracy, the mean absolute error is selected. and root mean square error As evaluation criteria, their expressions are as follows: *100% In the formula, n represents the total number of prediction results; and These represent the actual load value and the predicted load value for the i-th sampling point, respectively. To measure the quality of a model's predictions, The accuracy of the forecast is evaluated, and it is sensitive to both extremely large and extremely small errors in the results; in power load forecasting, and The smaller the value, the more accurate the load forecast result; The hyperparameters of S32 and XGBoost are adjusted using grid search or manual parameter tuning to obtain the optimal tree depth, optimal number of iterations, and shrinkage step size on the XGBoost model training set. S4. Based on the key features extracted in step S2, obtain key feature forecast data of the whole network's electricity consumption for the next day and three days at 15-minute intervals. Using these as input features, use the artificial intelligence model generated in step S3 to predict the load curve of the whole network's electricity consumption for the next day and three days.

2. The power load prediction method based on attention mechanism CNN-XGBoost according to claim 1, characterized in that: Step S1 specifically involves: S11. Collect historical load information, as well as weather, date type, temperature, and humidity information data for past dates, and perform data cleaning and data missing filling. Missing historical load information is filled using the median method, and missing weather-related data is filled using forward and backward filling methods. S12. Using the Pearson correlation coefficient method, extract each feature and the corresponding load value for the time period to form a data pair of feature value and load. Calculate the correlation coefficient between the feature data and the load data. If the calculation result is between (-0.5, 0.5), it is considered that the linear correlation between the feature and the load is weak. Remove the weakly correlated features and keep the strongly correlated features. Save these feature information and historical load information into the database to form a historical information database.

3. The power load prediction method based on attention mechanism CNN-XGBoost according to claim 1, characterized in that: Step S4 specifically involves, S41. At 15-minute intervals, for a total of 96 points in 24 hours, obtain forecast data of other key characteristics corresponding to the future load of the entire network's electricity consumption, such as temperature, humidity, whether it is a weekend, and whether it is a holiday, as input features. S42, combined with the key features extracted by CNN, forms... ,in Let i be the input features predicted at time i. Let i represent the predicted temperature characteristics at time i. To extract features for the CNN at time i, To determine whether time i is a weekend, holiday, or vacation, consider the 96 times of a day. Inputting the data into the generated XGBoost model will produce the load curve for that forecast day. Different forecast curves can be generated for different dates depending on the forecast date. If it is necessary to forecast the load for three consecutive days, forecasts can be made day by day to obtain the forecast curves for the following three days.

Images