Load prediction method based on machine learning model XGBoost

Abstract

The invention discloses a load prediction method based on a machine learning model XGBoost, and relates to the technical field of load prediction, and the method comprises the following steps: S1, carrying out Min-Max normalization processing on historical load data: carrying out the linear mapping of the data to a range of 0-1 through employing a Min-Max normalization method; s2, climate influence factor selection: selecting key features related to a target variable by adopting a feature importance evaluation method of a random forest model, wherein the target variable comprises a power load; and S3, feature scaling processing: for the extreme temperature value, amplifying the temperature value in the interval through a feature scaling method. According to the combined model, extreme temperature values are processed through a feature scaling method, the sensitivity of the model to extreme temperature changes is enhanced, the prediction accuracy of the model to extreme events is improved, prediction errors are corrected through cooperative work of the time sequence model and the machine learning model, the prediction capacity of the model to complex events is enhanced, and the prediction accuracy of the model to extreme events is improved. And the prediction precision is higher.

Description

Technical Field

[0001] The present invention relates to the technical field of load forecasting, and in particular to a load forecasting method based on a machine learning model XGBoost. Background Art

[0002] In the current global context, climate change has become a key issue affecting the Earth's ecosystem, economic development and human well-being. With the rapid development of the economy and society, the interaction between human activities and the natural environment has become increasingly stronger, and the world is facing unprecedented environmental risks. As the average temperature rises, the frequency and intensity of extreme temperature events are also on the rise, which has a more serious impact on people's lives.

[0003] As the key to urban infrastructure, the robustness of the power system not only supports daily life and industrial activities, but also ensures the smooth operation of other key infrastructure such as transportation, communications and water supply. Extreme temperature events are usually random and unpredictable, which means that the time, duration and intensity of such events are difficult to accurately predict. In addition, the intensification of climate change has made extreme temperature events more frequent and severe, increasing the complexity of power system management. As electric energy gradually penetrates into various fields, power load forecasting has received more and more attention.

[0004] In the context of improving the robustness of the power system under extreme temperature events, the accuracy of power load forecasting is particularly important. Since power demand is volatile and unpredictable, short-term load forecasting can predict load change trends in advance and reasonably arrange power production and supply, thereby avoiding economic losses and waste of resources caused by power shortages or surpluses. At the same time, it can also provide necessary data support to help governments and power companies formulate and implement effective power consumption measures during peak power consumption periods or special periods to ensure the reliability and stability of the power system.

[0005] Traditional power load forecasting models can be mainly divided into two categories: time series models based on traditional statistics and machine learning. Time series forecasting models include autoregressive moving average, Prophet, etc. These models are widely recognized for their simplicity, maturity, fast training process and low human intervention. Machine learning models include random forests, XGBoost (Extreme Gradient Boosting), etc. These models have shown significant advantages in the forecasting field due to their excellent nonlinear mapping capabilities.

[0006] 1. Technical solution of the prior art 1 related to the field of the present invention

[0007] 1. Load forecasting method based on time series model Prophet

[0008] Among time series forecasting models, Prophet is a newer model developed by Facebook in 2017. The Prophet model uses a flexible framework that can achieve accurate forecasts of different time series through simple parameter adjustments.

[0009] The Prophet model has attracted widespread attention for its unique way of decomposing time series, dividing the time series into four main components: trend, seasonality, holiday effect, and noise. Among them, the trend represents the long-term trend of the time series, seasonality reflects the periodic change pattern, the holiday effect considers the impact on special dates or time periods, and noise captures unpredictable random changes. The Prophet model has the flexibility of both additive and multiplicative models, and the additive model and the multiplicative model each have unique advantages and disadvantages, and are suitable for different types of time series.

[0010] In the Prophet additive model, the forecast value of the time series is the sum of trend, seasonality and holiday effects, and then the error from the true value is calculated, that is:

[0011] yhat(t)=g(t)+s(t)+h(t) (1)

[0012] e=y(t)-yhat(t) (2)

[0013] In the formula, yhat(t) represents the predicted value of the Prophet model, g(t) represents the non-periodic changes in the time series; s(t) represents the periodic changes in the time series; h(t) represents abnormal activities in the time series, such as festivals, etc.; y(t) represents the true value, and e represents the abnormal error that is difficult to describe by the model.

[0014] There are three types of trend terms g(t), namely flat steady change, linear piecewise linear change and logistic logical change. The default setting is linear, and the formula is as follows:

[0015] g(t)=(k+α(t)δ)×t+(m+α(t) T γ) (3)

[0016] Where k represents the growth rate, δ represents the change in growth rate, m represents the bias parameter, γ represents the smoothing offset at the change point, and α(t) represents the indicator function.

[0017] The seasonal term s(t) can be fitted by Fourier series, and the formula is as follows:

[0018]

[0019] Where N is the total number of cycles, and T is the duration of a specific type of cycle. Where an and bn are the model parameters to be estimated.

[0020] The function of the holiday term h(t) is to reflect the abnormal changes in the model during fitting and prediction. In the absence of abnormalities, it is assumed to follow a normal distribution.

[0021] Disadvantages of the above prior art 1

[0022] As a newer time series model, the Prophet model can show excellent periodic forecasting performance, accurately capture the daily and weekly load fluctuation periodicity, and show good forecast fitting results when it is not an extreme value. However, the time series model is mainly an autoregression of time series data, and does not take into account external factors such as climate and economy. Therefore, it is difficult to accurately predict the maximum / minimum load extreme values ​​in the context of extreme temperature events.

[0023] Technical solution of the second prior art related to the present invention

[0024] 1. Load forecasting method based on machine learning model XGBoost

[0025] XGBoost is a powerful algorithm widely used in machine learning and data analysis. It was proposed by Dr. Tianqi Chen of the University of Washington and belongs to the gradient boosting tree family. It is an integrated learning algorithm used to handle various supervised learning tasks, including classification and regression problems.

[0026] The algorithm builds a model based on integrating multiple regression trees and approximates the negative gradient of the loss function with respect to the model parameters through a second-order Taylor expansion. In each round of iteration, the algorithm learns the negative gradient as the residual of the previous model. In addition, the algorithm adjusts the learning weights so that samples with lower accuracy in the previous round receive higher weights in this round, thereby enhancing the model's learning of these samples. Through multiple rounds of iteration, the model deviation is gradually reduced until the predetermined convergence condition is reached. The convergence condition is as follows:

[0027]

[0028] In the formula, is the model prediction value, i is the number of samples, k is the number of integrated regression trees, x i is the data sample, F is the set space formed by the regression tree, f k Represents one of the regression trees.

[0029] The XGBoost algorithm has shown remarkable performance in machine learning applications. The algorithm enhances the generalization ability of the model by introducing regularization terms and column sampling techniques. In the process of building decision trees, XGBoost uses a parallel split point selection mechanism to effectively improve the training efficiency. In addition, XGBoost uses tree models as base learners, which shows more efficient learning capabilities compared to traditional feature engineering methods.

[0030] Disadvantages of the above-mentioned prior art 2

[0031] The XGBoost model has better prediction results for time-varying mutations, but its shortcoming is that it fails to reflect the periodic characteristics of weekdays and weekends.

[0032] As mentioned above, we have designed a load forecasting method based on the machine learning model XGBoost to solve the above problems. Summary of the invention

[0033] The purpose of the present invention is to solve the shortcomings existing in the prior art and propose a load forecasting method based on the machine learning model XGBoost.

[0034] In order to achieve the above object, the present invention adopts the following technical solutions:

[0035] A load forecasting method based on a machine learning model XGBoost comprises the following steps:

[0036] Step S1, performing Min-Max normalization processing on historical load data: using the Min-Max normalization method to linearly map the data to a range of 0-1;

[0037] Step S2, climate influencing factor selection: using the feature importance evaluation method of the random forest model to select key features related to the target variable, the target variable includes power load;

[0038] Step S3, feature scaling processing: for extreme temperature values, amplify the temperature values ​​within the interval by a feature scaling method;

[0039] Step S4, the first prediction based on the Prophet model: Substitute the historical power load data y value into the Prophet model, use the Prophet model to predict the historical power load data, obtain the historical simulation prediction value yhat(t), and compare it with the true value y to calculate the prediction error e;

[0040] Step S5, error prediction of XGBoost model: The prediction error e of the Prophet model is used as the target of the XGBoost model, and regression analysis is performed using climate, time and other characteristics to obtain the prediction error value ehat. Parameter optimization uses GridSearchCV combined with manual adjustment;

[0041] Step S6, final prediction of the combined model: merge the error prediction value ehat obtained by the XGBoost model with the prediction value yhat of the Prophet model to obtain the final prediction result of the combined model.

[0042] 2. The load forecasting method based on the machine learning model XGBoost according to claim 1, characterized in that in step S1, the normalization formula is:

[0043]

[0044] In the formula, X normalized represents the normalized value, X represents the original value of the historical load data, max(X) and min(X) represent the maximum and minimum values ​​in the historical data, respectively.

[0045] 3. A load forecasting method based on a machine learning model XGBoost according to claim 1, characterized in that, in step S3, the extreme temperature interval is included in the interval [0.95, 1] ​​to highlight the impact of extreme temperature on power load, and the formula is as follows:

[0046]

[0047] In the formula, a and b are the lower and upper limits of the selected specific interval, Tamplified is the normalized temperature data, Multiplier is the characteristic amplification factor, Multiplier = 2, a = 0.95, b = 1, when the temperature value is within the specific interval, it is multiplied by the amplification factor, otherwise it remains unchanged.

[0048] 4. A load forecasting method based on the machine learning model XGBoost according to claim 1, characterized in that, in the model optimization of step S5, the parameter range of each parameter optimization is limited to four dimensions, and if a certain optimal parameter is at the boundary of the parameter range, the parameter range is reset.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. The combined model proposed in the present invention processes extreme temperature values ​​through a feature scaling method, which enhances the model's sensitivity to extreme temperature changes and improves the model's prediction accuracy for extreme events.

[0051] 2. By combining GridSearchCV with manually adjusted parameter optimization methods, the performance of the machine learning model is further improved, and the prediction accuracy of the machine learning model is improved.

[0052] 3. The combined model of the present invention shows stronger adaptability under extreme climate events. Compared with a single model, the combined model corrects the prediction error through the collaborative work of the time series model and the machine learning model, enhances the model's prediction ability for complex events, and has higher prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a workflow diagram of a load forecasting method based on the machine learning model XGBoost proposed in the present invention. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0055] In order to overcome the deficiencies in the prior art, the present invention proposes a power load forecasting method based on the “Prophet-XGBoost” combined model, which can effectively improve the accuracy and stability of load forecasting under extreme climate events.

[0056] The combined model is a method that integrates multiple models, aiming to combine the advantages of each model to improve the overall performance and accuracy of the prediction model.

[0057] The Prophet model makes predictions based on time series data, while the XGBoost prediction model makes predictions by considering the relationship between different influencing factors and the target value. Although the two methods in the background technology perform well in independent predictions, both methods have some limitations. The XGBoost prediction model does not fully consider the inherent time period characteristics of load data when dealing with power load forecasting. In contrast, although the Prophet model focuses on time series analysis, it fails to fully capture external factors that are highly correlated with power load.

[0058] In view of the limitations of the two methods in the background technology, the present invention aims to combine the advantages of the two algorithms and build a combined model based on two advanced algorithms, time series Prophet and machine learning XGBoost. Combining their complementary characteristics, it is expected to build a more accurate prediction model to achieve a more accurate prediction of power load under extreme temperature events.

[0059] Reference Figure 1 , the specific embodiments are as follows:

[0060] Step S1, data normalization: Normalization is a data preprocessing technology, which adjusts the data scale to map it to a standard distribution or a uniform scale range. Taking into account the differences in the dimensions and value ranges of different meteorological conditions, in order to eliminate the influence of the differences in the dimensions and value ranges of different meteorological conditions, the present invention adopts the Min-Max normalization method to linearly map the data to the range of 0-1. The formula is:

[0061]

[0062] In the formula, X normalized represents the normalized value, X represents the original value of the historical load data, max(X) and min(X) represent the maximum and minimum values ​​in the historical data, respectively.

[0063] Step S2, climate influencing factor selection: In a multivariate data set, multiple features jointly affect the target variable. However, the increase in the number of features not only has a negative impact on model construction, but also prolongs the model training time.

[0064] Based on this, considering that multiple features jointly affect the target variable, the present invention adopts the feature importance evaluation method of the random forest model to select features of climate data. By evaluating the impact of different climate factors on power load, the most influential features are selected for model training, reducing the negative impact of redundant features on model construction.

[0065] Step S3, feature scaling of influencing factors: In studying the impact of extreme temperature events on power load, considering the significant impact of extreme temperature on power load, a feature scaling method is adopted to process extreme temperature values. This method aims to enhance the sensitivity of the model to extreme temperature changes by adjusting the proportional range of temperature values, thereby improving the accuracy of predicting power load under extreme conditions.

[0066] Then, by setting the extreme temperature range, the temperature values ​​in the range are amplified to highlight the impact of extreme temperature on power load. The formula for feature scaling is:

[0067]

[0068] Where a and b are the lower and upper limits of the selected specific interval, and in this method, they are set to [0.95, 1]. Tamplified is the normalized temperature data, and Multiplier is the characteristic amplification factor, which is 2 in this method. If the temperature value is within the specific interval, it is multiplied by the amplification factor, otherwise it remains unchanged.

[0069] Step S4: Based on the Prophet model, the historical power load data y value is substituted to perform the first prediction, and the historical simulation prediction value yhat(t) is obtained, which is compared with the actual historical power load data y value to obtain the prediction error value e of the Prophet model.

[0070] Step S5, using the error value e as the prediction target of the XGBoost machine learning model, using the data such as climate and time point as the feature items of the model regression analysis, and obtaining the error prediction value ehat. At the same time, in this embodiment, a method combining manual adjustment on the basis of GridSearchCV is adopted. The parameter range of each parameter optimization is limited to four dimensions, and the optimal value and parameter setting range are observed. If a certain optimal parameter is at the boundary of the parameter range, the parameter range is reset to float around the current optimal value of the parameter, and then the next round of parameter tuning grid search is performed.

[0071] Step S6: Combine the error prediction value ehat obtained by the machine learning model and the yhat obtained by the time series model to obtain the final prediction result of the combined model.

[0072] In this embodiment, in order to quantitatively analyze the prediction effects of the Prophet model, the XGBoost model and the combined model from a data perspective, the present invention selects two evaluation indicators and introduces the power load prediction data under real extreme temperature events to evaluate the hourly power load prediction results of the invented combined model, the original machine learning model and the time series model during extreme high temperature events.

[0073] Mean Absolute Percentage Error (MAPE), the MAPE value measures the average percentage error between the actual value and the predicted value. The smaller the MAPE value, the more accurate the model's prediction. The formula is:

[0074]

[0075] In the formula, y i and are the true value and predicted value of the i-th sample respectively, and N is the number of samples.

[0076] Root Mean Square Error (RMSE), RMSE calculates the average square root difference between the actual observation value and the model prediction value. The smaller the RMSE value, the more accurate the model prediction. The formula is:

[0077]

[0078] In the formula, yi and are the true value and predicted value of the i-th sample respectively, and N is the number of samples.

[0079] The prediction index values ​​of the three models under extreme high temperature events are shown in Table 1.

[0080] Table 1 Prediction performance of different models

[0081]

[0082] The application of the "Prophet-XGBoost" combined model shows significant advantages in predicting power loads under extreme temperature events. The effect of this combined model is better than the two single models overall. The time series-based prediction model Prophet is suitable for processing data with strong periodicity and trends, and can effectively identify and adapt to seasonal changes. XGBoost, as a decision tree algorithm based on gradient boosting, can capture more subtle data fluctuations and potential patterns by learning the interactions between a large number of features.

[0083] In complex situations such as extreme temperature events, the "Prophet-XGBoost" combined model can more comprehensively capture and analyze the various factors that affect power load. Therefore, the use of the "Prophet-XGBoost" combined model not only improves the accuracy of predictions, but also increases the adaptability of the prediction model to extreme events. This is of great significance for formulating more effective power security plans and reducing the uncontrollability and blindness of plan selection, which helps the power system maintain stable and efficient operation in the face of extreme weather events.

[0084] Among the above schemes, the more specific ones are:

[0085] 1. In studying the impact of extreme temperature events on power load, taking into account the significant impact of extreme temperature on power load, the feature scaling method is adopted to process the extreme temperature values, which enhances the model's sensitivity to extreme temperature changes and improves the model's prediction accuracy for extreme events.

[0086] 2. In the parameter optimization process of the machine learning model, a method based on GridSearchCV combined with manual adjustment is adopted. The parameter range of each parameter optimization is limited to four dimensions, and the optimal value and parameter setting range are observed. If a certain optimal parameter is at the boundary of the parameter range, the parameter range is reset to float around the current optimal value of the parameter, and then the next round of parameter tuning grid search is carried out, which improves the prediction accuracy of the machine learning model.

[0087] 3. In the load forecasting framework based on the "Prophet-XGBoost" combined model, the machine learning model introduces other variables for secondary prediction based on the result of the first prediction of the time series model to correct the first error result. Through the collaborative work of the time series model and the machine learning model, the prediction error is corrected and the model's prediction ability for complex events is enhanced.

[0088] It should be noted that there are other alternatives to the above technical solutions that can also achieve the purpose of the invention. Alternative technical solutions are other feasible approaches besides the approach currently considered by the inventor.

[0089] (1) The "Prophet-XGBoost" combined model idea proposed in the present invention can introduce different sub-model combinations. The basic models used in the present invention respectively adopt the Prophet model in the time series model and the XGBoost model in the machine learning model, but the model framework is not limited to these two models and can be replaced separately. For example, the machine learning model can also consider the neural network model, support vector machine model, etc., and it is also expected to show good performance.

[0090] (2) Electricity demand forecast considering multiple types of composite indicators such as climate factors and economic factors. The idea of ​​constructing the model in the present invention is mainly focused on developing a load forecasting algorithm suitable for extreme temperature events, and focuses on climate factors in variable selection. However, the ability of the model algorithm itself to introduce variables is not limited to this. In combination with the proposed data processing and model building methods, a wider range of variable indicators such as social and economic factors can also be introduced for further expansion.

[0091] (3) When scaling the features, the present invention sets the feature amplification interval to [0.95, 1] ​​and the feature amplification factor to 2. This is mainly based on the determination method of the extreme temperature threshold in this paper and the adaptability to the data example. When using different threshold determination methods or data from different regions, this parameter can be appropriately modified.

[0092] The basic principles, main features and advantages of the present invention are shown and described above. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of the present invention is defined by the attached claims and their equivalents.

Claims

1. A load forecasting method based on the machine learning model XGBoost, characterized in that: The following steps are involved: Step S1, performing Min-Max normalization processing on historical load data: using the Min-Max normalization method to linearly map the data to a range of 0-1; Step S2, climate influencing factor selection: using the feature importance evaluation method of the random forest model to select key features related to the target variable, the target variable includes power load; Step S3, feature scaling processing: for extreme temperature values, amplify the temperature values ​​within the interval by a feature scaling method; Step S4, the first prediction based on the Prophet model: Substitute the historical power load data y value into the Prophet model, use the Prophet model to predict the historical power load data, obtain the historical simulation prediction value yhat(t), and compare it with the true value y to calculate the prediction error e; Step S5, error prediction of XGBoost model: The prediction error e of the Prophet model is used as the target of the XGBoost model, and regression analysis is performed using climate, time and other characteristics to obtain the prediction error value ehat. Parameter optimization uses GridSearchCV combined with manual adjustment; Step S6, final prediction of the combined model: merge the error prediction value ehat obtained by the XGBoost model with the prediction value yhat of the Prophet model to obtain the final prediction result of the combined model.

2. A load forecasting method based on the machine learning model XGBoost according to claim 1, characterized in that: In step S1, the normalization formula is: Where, X normalized represents the normalized value, X represents the original value of the historical load data, max(X) and min(X) represent the maximum and minimum values ​​in the historical data, respectively.

3. A load forecasting method based on the machine learning model XGBoost according to claim 1, characterized in that: In step S3, the extreme temperature interval is included in the interval [0.95, 1] ​​to highlight the impact of extreme temperature on power load. The formula is as follows: In the formula, a and b are the lower and upper limits of the selected specific interval, Tamplified is the normalized temperature data, Multiplier is the characteristic amplification factor, Multiplier = 2, a = 0.95, b = 1, when the temperature value is within the specific interval, it is multiplied by the amplification factor, otherwise it remains unchanged.

4. A load forecasting method based on the machine learning model XGBoost according to claim 1, characterized in that: In the model optimization of step S5, the parameter range of each parameter optimization is limited within four dimensions. If an optimal parameter is at the boundary of the parameter range, the parameter range is reset.

Images