A multi-task load prediction method and system based on an Autoformer model and an XGBoost model

Abstract

The application discloses a kind of multi-task load prediction method and system based on Autoformer model and XGBoost model, comprising the following steps: obtaining historical data, respectively extracting time characteristics, weather characteristics, population characteristics and resident load data, to construct historical time series data;Population quantity and resident load are predicted based on Autoformer model, to obtain population quantity prediction value and resident maximum load prediction value in a time period;The resident maximum load in the same time period is predicted based on XGboost model using time characteristics, weather characteristics and the population quantity prediction value predicted by Autoformer model;The resident maximum load predicted by Autoformer model and XGboost model is fused, to obtain final prediction result.The system includes feature extraction module, Autoformer model prediction module, XGboost model prediction module and prediction result fusion module;The application solves the problem that time series data cannot be effectively utilized for load prediction in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electric load prediction, and more particularly to a multi-task load prediction method and system based on an Autoformer model and an XGBoost model. BACKGROUND

[0002] Electricity consumption prediction is an important research field, and accurate circuit load consumption prediction can effectively help power companies to allocate and schedule power and improve resource utilization efficiency.

[0003] Electricity consumption prediction is a typical time series problem, which is influenced by multiple factors such as time, weather, holidays, population flow, and has the uncertainty and complexity of a typical complex system; especially in recent years, the population flow is limited, which brings greater challenges to electricity consumption prediction. For medium and long-term prediction, the larger prediction time span further magnifies the uncertainty of the system.

[0004] Generally, time series prediction methods can be divided into traditional statistical methods, machine learning-based methods, and deep learning-based methods. The representative algorithms of traditional statistics are ARMA model, ARIMA model, AR model, etc. The algorithm of traditional statistics can only learn time series features and cannot use additional features; machine learning-based algorithms can use more features and can learn the time dynamics of time series in a data-driven manner. XGBoost algorithm is a typical representative of machine learning algorithms, which has the advantages of fast training speed and high prediction accuracy, and has very good performance in classification and regression tasks, but XGBoost is a tree-based model, and the disadvantage of the tree-based model is that the extrapolation ability is weak, and for prediction problems, the predicted value will not exceed the range of the maximum and minimum values of the training set data. In recent years, deep learning algorithms such as RNN, LSTM, and Tranformer model have also been applied to time series prediction problems. Deep learning-based prediction algorithms have strong representation learning capabilities from large amounts of data, and the model has strong generalization ability. The Autoformer model uses autocorrelation mechanism instead of point attention mechanism, trying to further find the similarity of sub-sequences, and has more advantages in long-time time series data prediction. However, the main problem of deep learning-based prediction algorithms is the black box characteristics of the model, which cannot effectively utilize the time characteristics of time series data.

[0005] Therefore, how to provide a multi-task load prediction method and system based on Autoformer model and XGBoost model that can effectively utilize time series data prediction is a problem that those skilled in the art need to solve. SUMMARY

[0006] Therefore, the application provides a multi-task load prediction method and system based on an Autoformer model and an XGBoost model, which solves the problem that the prior art cannot effectively utilize time series data for load prediction.

[0007] To achieve the above object, the application adopts the following technical solutions:

[0008] A multi-task load prediction method based on an Autoformer model and an XGBoost model comprises the following steps:

[0009] Obtain historical data, extract time features, weather features, population features and resident load data respectively, and thus construct historical time series data;

[0010] Based on the Autoformer model, the population quantity and the resident load are predicted to obtain a population quantity prediction value and a resident maximum load prediction value in a time period;

[0011] Based on the XGboost model, the time features, the weather features and the population quantity prediction value predicted by the Autoformer model are used to predict the resident maximum load in the same time period;

[0012] The resident maximum loads predicted by the Autoformer model and the XGboost model are fused to obtain a final resident maximum load prediction result, and the final resident maximum load prediction result and the population quantity prediction value are output.

[0013] Preferably, the step of predicting the population quantity and the resident load based on the Autoformer model comprises:

[0014] S11. The historical time series data is denoted as x, and after x is sequentially decomposed, a seasonal term x des and a trend term x det are obtained.

[0015] S12. x is encoded by N Autoformer encoding layers of an Autoformer encoder, and final encoding data is output, wherein the encoding method of the lth layer is:

[0016]

[0017]

[0018] In the formula, the input of the encoding layer is is obtained by performing a convolution operation on x, _ is the trend part eliminated, represents the output of the lth encoding layer, denotes the i-th sequence decomposition at the l-th layer; FeedForward denotes the feedforward operation, and Auto-Correlation denotes the autocorrelation coefficient of the sequence;

[0019] S13. Decoding the trend term x det and the seasonal term x des by M decoding layers of the Autoformer decoder, and outputting final decoding data and where the decoding method at the l-th layer is:

[0020]

[0021]

[0022]

[0023]

[0024] wherein, denotes the output of the l-th layer, obtained by convolution of the seasonal term x des . denote the i-th sequence decomposition operation of the periodic term and the trend term at the l-th layer, respectively, and W l,i ,i∈{1,2,3} denote the projection of x

[0025] S14. Obtaining a prediction value and by a multi-layer perception (MLP) from the final decoding data

[0026] Preferably, the time feature is one-hot encoded, and the remaining features are max-min normalized, thereby constructing the historical time series data.

[0027] Preferably, for a discrete time series process {x t}, the autocorrelation coefficient is specifically calculated as:

[0028]

[0029] wherein L is the length of the sequence, and L approaches infinity.

[0030] Preferably, the variance uncertainty is used as a benchmark in the two tasks of resident load prediction and population quantity to weight the loss of different tasks, and the predicted resident load and population quantity are respectively denoted as y load and y pop, the prediction target is represented as p(y load ,y pop |f(x|θ)) and the loss function is:

[0031]

[0032] where θ is the learned parameter, and σ load and σ pop are the variances of the resident load and population, respectively, and θ

[0033] The specific content of predicting the maximum resident load in the same time period based on the XGboost model using the time features, the weather features, and the population prediction value predicted by the Autoformer model includes:

[0034] S21. Training an XGboost model based on feature data;

[0035] S22. Obtaining time features, weather features, and a population prediction value predicted by an Autoformer model for a to-be-predicted time period, performing maximum and minimum value normalization on the population prediction value and the weather features to obtain a test set;

[0036] S23. Performing prediction based on the XGboost training model and the features of the test set to obtain a resident load prediction value based on the XGboost model

[0037] The specific content of fusing the maximum resident loads predicted by the Autoformer model and the XGboost model includes:

[0038]

[0039] wherein, is the maximum resident load predicted by the Autoformer model, is the maximum resident load predicted by the XGboost model, and ε xgboost and ε autoformer represent the training errors of the XGboost and Autoformer, respectively.

[0040] A multi-task load prediction system based on an Autoformer model and an XGBoost model includes a feature extraction module, an Autoformer model prediction module, an XGboost model prediction module, and a prediction result fusion module.

[0041] The feature extraction module is configured to acquire historical data, and extract time features, weather features, population features and resident load data respectively, so as to construct historical time series data.

[0042] The Autoformer model prediction module is configured to predict the population and the resident load based on the Autoformer model, and obtain a population prediction value and a resident maximum load prediction value in a time period.

[0043] The XGboost model prediction module is configured to predict the resident maximum load in the same time period based on the XGboost model, the time features, the weather features and the population prediction value predicted by the Autoformer model.

[0044] The prediction result fusion module is configured to fuse the resident maximum loads predicted by the Autoformer model and the XGboost model, and obtain a final prediction result.

[0045] According to the above technical solution, compared with the prior art, the present application provides a multi-task load prediction method and system based on an Autoformer model and an XGBoost model, and has the following advantages:

[0046] (1) Based on the Autoformer model, the autocorrelation mechanism is used to replace the point connection attention mechanism, and the similarity between sequence levels is fully learned, so that the periodicity of time series can be coped with, and the accuracy of long-term prediction can be effectively improved.

[0047] (2) The XGBoost can automatically learn and predict the most relevant features of the task, and the model has strong interpretability.

[0048] (3) The loss function of the same variance uncertainty combination multi-task is used to simultaneously learn the population prediction task and the resident load task. In actual application scenarios, the construction of the quantitative population-load conduction prediction model can predict the peak value and time period of the holiday population inflow and load pressure before the major holidays, so as to provide quantitative support and experience reference for the key area holiday power supply work. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0050] Figure 1A flowchart of a multi-task load prediction method based on an Autoformer model and an XGBoost model is provided. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0052] The embodiments of the present application disclose a multi-task load prediction method based on an Autoformer model and an XGBoost model, as shown in Figure 1 The method comprises the following steps:

[0053] Obtain historical data, extract time features, weather features, population features and resident load data respectively, and construct historical time series data;

[0054] Based on the Autoformer model, the population quantity and the resident load are predicted to obtain a population quantity prediction value and a resident maximum load prediction value in a time period;

[0055] Based on the XGboost model, the time features, the weather features and the population quantity prediction value predicted by the Autoformer model are used to predict the resident maximum load in the same time period;

[0056] The resident maximum loads predicted by the Autoformer model and the XGboost model are fused to obtain a final resident maximum load prediction result, and the final resident maximum load prediction result and the population quantity prediction value are output.

[0057] In the present embodiment, the time features are: the current day is the first day of the year, the first week, the first month, the first quarter, whether it is a weekend, whether it is a holiday, the first year, etc.; the weather features are: the maximum temperature, the minimum temperature and whether it rains on the current day; the population features are: the population quantity of the current region every day; and the resident load data is: the maximum load of the residents in the current region.

[0058] x pop and x load are respectively the historical population quantity and the resident maximum load, and the predicted resident load data y load is a one-dimensional vector with a length of O. The present embodiment relates to medium and long term prediction, so O is generally between 7 and 14.

[0059] To further implement the above scheme, the steps of predicting the population quantity and the resident load based on the Autoformer model include:

[0060] S11. Historical time series data is recorded as x, and after sequence decomposition of x, seasonal term x des and trend term x det are obtained.

[0061] S12. Encode x through N Autoformer encoding layers of the Autoformer encoder, and output final encoding data wherein the encoding method of the lth layer is:

[0062]

[0063]

[0064] In the formula, the input of the encoding layer is is obtained by convolution operation on x, and _ is the trend part eliminated, represents the output of the lth encoding layer, represents the i-th sequence decomposition at the lth layer; FeedForward represents the feedforward operation, and Auto-Correlation is the autocorrelation coefficient of the sequence.

[0065] S13. Decode trend term x det and seasonal term x des through M decoding layers of the Autoformer decoder, and output final decoding data and wherein the decoding method of the lth layer is:

[0066]

[0067]

[0068]

[0069]

[0070] In the formula, represents the output of the lth layer, is obtained by convolution of seasonal term x des ; and respectively represent the i-th sequence decomposition operation of the periodic term and the trend term at the lth layer, W l,i , i ∈ {1, 2, 3} represent the projection of ;

[0071] S14. According to the final decoding data and The prediction value is obtained by a multi-layer perception (MLP)

[0072] In this embodiment, the input of the encoding layer is The convolution operation on x is performed by using the Conv1d interface of Pytorch, FeedForward represents the feedforward operation, and is obtained by using the Linear interface of Pytorch, is obtained by performing the convolution operation on x by using the Conv1d interface of Pytorch. des

[0073] In order to further implement the above scheme, the time characteristics are one-hot encoded, and the remaining characteristics are normalized by maximum and minimum, thereby constructing the historical time series data.

[0074] In order to further implement the above scheme, for a discrete time series process {x t}, the specific calculation method of the autocorrelation coefficient is as follows:

[0075]

[0076] In the formula, L is the length of the sequence, and L tends to infinity.

[0077] In this embodiment, the autocorrelation coefficient can be calculated according to the fast Fourier transform (FFT) interface and the inverse Fourier transform (irfft) interface of Pytorch.

[0078] In order to further implement the above scheme, the variance uncertainty is used as a benchmark in the resident load prediction and population quantity tasks to weight the losses of different tasks, and the predicted resident load and population quantity are respectively represented as y load and y pop , the prediction target is represented as p(y load , y pop |f(x|θ)), and the loss function is:

[0079]

[0080] Where θ is the learned parameter, and are the variances of the resident load and the population quantity, respectively, θ load and θ pop are the parameters to be learned for the resident load and the population quantity tasks, respectively.

[0081] ​To further implement the above scheme, the specific content of predicting the maximum resident load in the same time period based on the XGboost model using the time features, weather features, and the population quantity prediction value predicted by the Autoformer model includes:

[0082] S21. Training an XGboost model according to feature data;

[0083] S22. Obtaining time features, weather features, and a population quantity prediction value predicted by an Autoformer model for a to-be-predicted time period, performing maximum and minimum value normalization on the population quantity prediction value and the weather features to obtain a test set;

[0084] S23. Performing prediction according to the XGboost training model and the features of the test set to obtain a resident load prediction value based on the XGboost model

[0085] To further implement the above scheme, the specific content of fusing the maximum resident load predicted by the Autoformer model and the XGboost model includes:

[0086]

[0087] wherein, is the maximum resident load predicted by the Autoformer model, is the maximum resident load predicted by the XGboost model, and ε xgboost and ε autoformer respectively represent the training errors of the XGboost and the Autoformer.

[0088] In this implementation example, the number of layers of the Autoformer encoder is 2, the number of decoding layers is 1, and the dimension of the hidden layer is 512. The optimizer is ADAM, the learning rate is 0.0001, the size of the batch is 16, and the main parameters of the XGboost model are as follows: the maximum depth max_depth is 10, the learning rate learning_rate is 0.01, n_estimators=1100, the enhancer is gbtree, and the target learning function reg is linear.

[0089] A multi-task load prediction system based on an Autoformer model and an XGBoost model, comprising: a feature extraction module, an Autoformer model prediction module, an XGboost model prediction module, and a prediction result fusion module.

[0090] The feature extraction module is configured to acquire historical data, extract time features, meteorological features, population features and resident load data respectively, and construct historical time series data.

[0091] The Autoformer model prediction module is configured to predict the population and the resident load based on the Autoformer model, and obtain a population prediction value and a resident maximum load prediction value in a time period.

[0092] The XGboost model prediction module is configured to predict the resident maximum load in the same time period based on the XGboost model, time features, meteorological features and the population prediction value predicted by the Autoformer model.

[0093] The prediction result fusion module is configured to fuse the resident maximum loads predicted by the Autoformer model and the XGboost model, and obtain a final prediction result.

[0094] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0095] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-task load forecasting method based on an Autoformer model and an XGBoost model, characterized in that, The method comprises the following steps: obtaining historical data, extracting time features, weather features, population features and resident load data respectively, and constructing historical time series data; predicting the population quantity and the resident load based on an Autoformer model to obtain a population quantity prediction value and a resident maximum load prediction value in a time period; predicting the resident maximum load in the same time period based on an XGboost model using the time features, the weather features and the population quantity prediction value predicted by the Autoformer model; fusing the resident maximum load predicted by the Autoformer model and the XGboost model to obtain a final resident maximum load prediction result, and outputting the final resident maximum load prediction result and the population quantity prediction value.

2. The multi-task load forecasting method based on an Autoformer model and an XGBoost model according to claim 1, characterized in that, The step of predicting the population quantity and the resident load based on the Autoformer model comprises: S11. Let the historical time series data be x, and after performing a seasonal decomposition on x, obtain the seasonal component x des and the trend component x det ; S12. Encode x through N Autoformer encoding layers of the Autoformer encoder, and output final encoding data wherein the lth layer encoding method is: where the input of the encoding layer is is the trend component removed by the convolution operation on x, denotes the output of the lth encoding layer, denotes the i-th sequence decomposition at the lth layer; FeedForward denotes the feedforward operation, and Auto-Correlation denotes the autocorrelation coefficient of the sequence. S13. Decode the trend term x by M decoding layers of the Autoformer decoder det and the seasonal term x des and output the final decoded data and where the decoding method of the l-th layer is: wherein denotes the output of the l-th layer, denotes the seasonal term x des after convolution; denote the seasonal and trend term decomposition operation at the i-th iteration of the l-th layer, respectively, W l,i denote the projection of the projection of S14. Decoding the final data and Obtaining the prediction value by a multi-layer perceptron, MLP 3. The multi-task load forecasting method based on an Autoformer model and an XGBoost model according to claim 2, characterized in that, performing one-hot encoding processing on the time features, and performing maximum-minimum normalization processing on the remaining features to construct the historical time series data.

4. The multi-task load forecasting method based on an Autoformer model and an XGBoost model according to claim 2, characterized in that, For a discrete time series process {x t}, the autocorrelation coefficient is calculated as follows: In the formula, L is the length of the sequence, and L tends to infinity.

5. The multi-task load forecasting method based on an Autoformer model and an XGBoost model according to claim 1, characterized in that, Variance uncertainty is used as a baseline to weight the loss of different tasks in both the resident load prediction and the population number tasks, predicting the resident load and the population number as y load and y pop respectively, the prediction target is represented as p(y load , y pop | f(x | θ)), and the loss function is: where θ is a parameter to be learned, respectively the variances of the resident load and the population number, θ load and θ pop respectively the variances of the resident load and the population number, θ 6. The multi-task load forecasting method based on an Autoformer model and an XGBoost model according to claim 1, characterized in that, The specific content of predicting the resident maximum load in the same time period based on the XGboost model using the time features, the weather features and the population quantity prediction value predicted by the Autoformer model comprises: S21. Training an XGboost model according to feature data; S22. Obtaining time features, weather features and a population quantity prediction value predicted by an Autoformer model in a to-be-predicted time period, performing maximum-minimum normalization processing on the population quantity prediction value and the weather features to obtain a test set; S23. Perform prediction according to the XGboost trained model and the features of the test set, to obtain the resident load prediction value based on the XGboost model 7. The multi-task load forecasting method based on an Autoformer model and an XGBoost model according to claim 1, characterized in that, The specific content of fusing the resident maximum load predicted by the Autoformer model and the XGboost model comprises: where, is the predicted resident maximum load by the Autoformer model, is the predicted resident maximum load by the XGboost model, ε xgboost and ε autoformer denote the training errors of XGboost and Autoformer, respectively.

8. A multi-task load forecasting system based on an Autoformer model and an XGBoost model, characterized in that, It comprises: a feature extraction module, an Autoformer model prediction module, an XGboost model prediction module and a prediction result fusion module; The feature extraction module is configured to obtain historical data, extract time features, weather features, population features and resident load data respectively, and construct historical time series data; The Autoformer model prediction module is configured to predict the population quantity and the resident load based on an Autoformer model to obtain a population quantity prediction value and a resident maximum load prediction value in a time period; The XGboost model prediction module is configured to predict the resident maximum load in the same time period based on an XGboost model using the time features, the weather features and the population quantity prediction value predicted by the Autoformer model; The prediction result fusion module is configured to fuse the resident maximum load predicted by the Autoformer model and the XGboost model to obtain a final prediction result.

Images