Ultra-short-term load prediction method and system based on hybrid expert model

Abstract

The invention belongs to the technical field of information, and relates to an ultra-short-term load prediction method and system based on a hybrid expert model. The method comprises the following steps: constructing an expert model for learning an influence mode of a load period on a load, and naming the expert model as a time effect expert sub-model; constructing an expert model for learning the short-term fluctuation characteristics of the load, wherein the expert model is called a fluctuation effect expert sub-model; and integrating the outputs of the time effect expert sub-model and the fluctuation effect expert sub-model through a gating mechanism to obtain an ultra-short-term load prediction result. Compared with the current latest ultra-short-term load prediction method, the ultra-short-term load prediction model based on the hybrid expert model is better in prediction precision, shorter in overall training time and lower in cost, all the expert sub-models can be trained in parallel, and the prediction period is greatly shortened. According to the invention, the MOE-EL framework has high expandability, and the existing load prediction model can be quickly added into the MOE-EL framework, so that the ultra-short-term load prediction performance is improved.

Description

Technical Field

[0001] The invention belongs to the field of information technology, and in particular relates to an ultra-short-term load forecasting method and system based on a hybrid expert model. Background Art

[0002] The change in load is closely related to the historical load and is affected by many factors such as load composition, load period, meteorological factors, policies, and emergencies. Existing technologies use statistics, machine learning, and deep learning methods to analyze multiple influencing factors to achieve load forecasting.

[0003] Ultra-short-term load forecasting is a forecast of load within a very short time range in the future, usually a few minutes to a few hours, and has the characteristics of high timeliness, high precision, and online adaptability. The high timeliness is due to the short time span of the forecast. Ultra-short-term load forecasting needs to respond quickly to real-time changes in load, which places extremely high demands on the real-time data processing capabilities and fast prediction speed of the forecasting model. The high precision is due to the fact that ultra-short-term load fluctuations are affected by many factors, such as sudden weather changes, special events, user behavior patterns, etc., requiring the forecasting model to accurately capture these subtle and rapid changes, so there are extremely strict requirements on the forecasting accuracy. Online adaptability is due to the fact that the load shows significant dynamics and non-stationarity with time and external conditions. The forecasting model needs to have online learning and adaptation capabilities, and quickly and accurately update the model after continuously receiving the latest load data, adjust the model parameters and structure, and give the latest inference results in a short time.

[0004] The existing methods consider a variety of influencing factors to build large-scale and complex models. Training the model requires a lot of computing resources and time. Adjusting the model parameters and structure requires repeated attempts and optimization, which can easily lead to overfitting problems. Especially in the context of the rapid development of large models, researchers can capture and analyze more factors that affect the load, and the prediction model will become more complex, resulting in greater computational complexity, longer time, and higher costs for model training and updating. Increasingly complex prediction models are difficult to meet the needs of high timeliness, high accuracy, and real-time adjustment of ultra-short-term load forecasting. Summary of the invention

[0005] In view of the above problems, the present invention provides an ultra-short-term load forecasting method and system based on a hybrid expert model.

[0006] The technical solution adopted by the present invention is as follows:

[0007] An ultra-short-term load forecasting method based on a hybrid expert model comprises the following steps:

[0008] Construct an expert model to learn the load period's influence on the load pattern, called the time effect expert sub-model;

[0009] Construct an expert model of the short-term fluctuation characteristics of learning load, called the fluctuation effect expert sub-model;

[0010] The outputs of the time effect expert sub-model and the fluctuation effect expert sub-model are integrated through the gating mechanism to obtain the ultra-short-term load forecasting results.

[0011] Furthermore, the step of constructing the time effect expert sub-model includes:

[0012] Extract the date characteristics of the load, and classify the load according to the date characteristics of the load to obtain multiple load categories;

[0013] A training data set and prediction model are constructed for each load category, and model training is performed based on each load category to obtain a time effect expert sub-model.

[0014] Furthermore, the load is classified according to its date characteristics, and the load is divided into 9 load categories, namely, weekday load, weekend load, New Year's Day load, Spring Festival load, Qingming Festival load, Labor Day load, Dragon Boat Festival load, Mid-Autumn Festival load and National Day load.

[0015] Furthermore, for weekday loads and weekend loads, the Transformer model is used to construct a time effect expert sub-model; for New Year's Day loads, the KRLS model is used to construct a time effect expert sub-model; for Spring Festival loads, the SVR model is used to construct a time effect expert sub-model; for Qingming Festival loads, the KRLS model is used to construct a time effect expert sub-model; for Labor Day loads, the SVR model is used to construct a time effect expert sub-model; for Dragon Boat Festival loads and Mid-Autumn Festival loads, the KRLS model is used to construct a time effect expert sub-model; for National Day loads, the SVR model is used to construct a time effect expert sub-model.

[0016] Furthermore, the fluctuation effect expert sub-model is constructed by combining the LSTM meta-learner and the MLP base learner.

[0017] Furthermore, the training process of the fluctuation effect expert sub-model includes:

[0018] For the input load data, calculate the loss function value and loss function gradient value of the MLP basic learner;

[0019] Feed the loss function value and the loss function gradient value back to the LSTM meta-learner to update the cell state of the meta-learner;

[0020] Update the parameters of the base learner using the updated cell states of the meta-learner.

[0021] Furthermore, the outputs of the time effect expert sub-model and the fluctuation effect expert sub-model are integrated through the gating mechanism, including:

[0022] Based on the input load data, a softmax function is used to assign weights to each expert sub-model;

[0023] The expert sub-models corresponding to the highest K weights are selected, and the output results of the K expert sub-models are weighted to obtain the final load forecasting result.

[0024] An ultra-short-term load forecasting system based on a hybrid expert model comprises:

[0025] The time effect expert sub-model construction module is used to construct an expert model for learning the load period's influence pattern on the load, which is called the time effect expert sub-model;

[0026] The fluctuation effect expert sub-model building module is used to build an expert model of the short-term fluctuation characteristics of the learning load, which is called the fluctuation effect expert sub-model;

[0027] The gating mechanism module is used to integrate the outputs of the time effect expert sub-model and the fluctuation effect expert sub-model through the gating mechanism to obtain the ultra-short-term load forecasting results.

[0028] The ultra-short-term load forecasting technology based on the hybrid expert model provided by the present invention has the following advantages:

[0029] 1. Compared with the latest ultra-short-term load forecasting method, the ultra-short-term load forecasting model based on the hybrid expert model proposed in the present invention has better prediction accuracy.

[0030] 2. On the basis of ensuring the prediction performance, the overall training time of the framework is shorter and the cost is lower, and each expert sub-model can be trained in parallel, which greatly shortens the prediction cycle. Compared with the existing deep learning framework, the present invention adopts the idea of ​​ensemble learning. On the one hand, the complexity of each expert sub-model is low, and on the other hand, each expert sub-model can be trained in parallel, which shortens the training time and reduces the cost.

[0031] 3. The MOE-EL framework of the present invention has strong scalability, and the existing load forecasting model can be quickly added to the MOE-EL framework to improve the ultra-short-term load forecasting performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is an architectural diagram of the ultra-short-term load forecasting method based on the hybrid expert model.

[0033] Figure 2 It is the architecture diagram of the time effect expert sub-model. DETAILED DESCRIPTION

[0034] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below through specific embodiments and drawings.

[0035] The present invention proposes an ultra-short term load forecasting model based on mixture of experts (MOE-EL), which establishes expert models that are good at learning the influence of specific factors and scenarios on load, and integrates the outputs of multiple expert models through a gating mechanism to obtain the final load forecasting result. The overall process is as follows: Figure 1 As shown. Specifically, in view of the problem that the load presents different change patterns in different load periods, the existing method fuses the load data together to learn the load change pattern, which affects the accuracy of load forecasting. The present invention proposes an expert model that is good at learning the load period's impact pattern on the load, called the time effect expert sub-model. In view of the problem that the short-term fluctuation characteristics of the load are difficult to capture, it is proposed to establish a fluctuation effect expert sub-model based on the Meta-LSTM model. In addition, the MOE-EL framework supports the fusion of load forecasting models under specific factors and scenarios proposed by existing studies as expert sub-models.

[0036] The key technical points of the present invention are:

[0037] 1. An expert model that is good at learning the load period's influence on the load pattern is proposed, called the Time Effect Expert Sub-module (TExpertizer). First, the date characteristics of the load are extracted, and the load is classified into 9 load categories: weekday load, weekend load, New Year's Day load, Spring Festival load, Qingming Festival load, Labor Day load, Dragon Boat Festival load, Mid-Autumn Festival load and National Day load. Then, a training data set and prediction model for each load category are constructed, and an expert sub-model is trained based on each load category. Finally, a specific time effect expert sub-model and fluctuation effect expert sub-model are used to achieve ultra-short-term load prediction.

[0038] 2. An expert model that is good at learning short-term load fluctuation characteristics is proposed, called the Fluctuation Effect Expert Sub-module (FExpertizer). When receiving the latest load data, it learns the latest data characteristics based on the feature analysis of historical load data, initial parameters or other training parameters, and quickly updates the model to obtain the latest load inference results, achieving good prediction performance for the short-term load fluctuation characteristics.

[0039] 3. The gating model determines which expert sub-models to use to predict the input load data, and dynamically assigns tasks to different expert sub-models based on the characteristics of the load data, so as to optimize the learning and prediction effects of the entire model.

[0040] Examples of the present invention:

[0041] Example 1: Time Effect Expert Submodel

[0042] like Figure 2 As shown in the figure, for the power load data set (or other loads), the date characteristics of the load are first extracted, and the power load is classified accordingly. For the loads on weekdays and weekends, the training data set is large, and the Transformer model is used to build a time effect expert sub-model; for the New Year's Day load, the KRLS (kernel recursive least squares) model is used to build a time effect expert sub-model; for the Spring Festival load, the SVR (support vector regression) model is used to build a time effect expert sub-model; for the Qingming Festival load, the KRLS model is used to build a time effect expert sub-model; for the Labor Day load, the SVR model is used to build a time effect expert sub-model; for the Dragon Boat Festival load and the Mid-Autumn Festival load, the KRLS model is used to build a time effect expert sub-model; for the National Day load, the SVR model is used to build a time effect expert sub-model.

[0043] In the training phase of the time effect expert sub-model based on Transformer, the load data is input into the sub-model, and the sub-model obtains the output prediction result. The output of the sub-model is compared with the actual load, and the mean square error loss function is used to calculate the difference between the actual value and the predicted value of the load. The loss function is minimized by the stochastic gradient descent method to achieve multiple rounds of iterations. In the training phase of the time effect expert sub-model based on SVR, the initial values ​​of the penalty function, kernel function, etc. are set, and the sub-model is trained using the training set to search for the best support vector and corresponding weight. In the training phase of the time effect expert sub-model based on KRLS, the training data set is input into the sub-model, and the weight vector is updated according to the difference between the sub-model prediction value and the actual load value. The updated weight vector is used for multiple rounds of iterations.

[0044] Example 2: Volatility Effect Expert Submodel

[0045] The Meta-LSTM method uses Long Short-term Memory Neural Networks (LSTM) as a meta-learner and Convolutional Neural Networks (CNN) as a basic learner to achieve good image classification performance. By introducing the "gate" mechanism and cell state, LSTM can effectively capture and memorize the long-term dependency information in the sequence (a load sequence or time series formed by a load that changes dynamically over time), avoiding the gradient vanishing or gradient exploding problems that may occur in the traditional recurrent neural network during the training process, thereby greatly improving the accuracy of time series prediction. CNN networks are large in scale, have many parameters, and are highly complex, making it difficult to quickly update the model after receiving the latest data.

[0046] Therefore, a meta-learner implemented by LSTM (LSTM meta-learner for short) and a base learner implemented by multilayer perceptron (MLP for short) are combined to build a meta-learning model to realize the Fluctuation Effect Expert Sub-module (FExpertizer). MLP is a feedforward neural network with a simple structure and easy expansion. After receiving the latest data, it can quickly update the model and obtain the latest load inference results.

[0047] Training of the volatility effect expert sub-model:

[0048] In the training phase, for the input load data, first, the loss function value L of the MLP base learner is calculated t And the loss function gradient The method for calculating the loss function value is shown in formula (1). Where B represents the basic learner, θ t-1 represents the parameters of the basic learner, L represents the loss function, X t represents the input value of the load data in the dataset used to train the fluctuation effect expert sub-model, Y t Represents the output value of the load data in the dataset used to train the fluctuation effect expert sub-model. Secondly, the loss function value and the loss function gradient value are fed back to the LSTM meta-learner to update the cell state of the meta-learner. The calculation method is shown in formula (2). Where M represents the meta-learner and Θ represents the parameters of the meta-learner. Next, the updated cell state of the meta-learner is used to update the parameters of the base learner. The calculation method is shown in formula (3).

[0049] L t =L[B(X t θ t-1),Y t ] (1)

[0050]

[0051] θ t =c t (3)

[0052] Example 3: Gated Model

[0053] The Top-K gating mechanism is adopted to implement it. First, a softmax function is used to assign weights to each expert sub-model based on the input load data. The input load data can be processed by different expert sub-models with different weights. Then, the expert sub-models corresponding to the highest K weights are selected, and the output results of the K expert sub-models are weighted to obtain the final load forecasting result.

[0054] In order to demonstrate the effectiveness of the ultra-short-term load forecasting framework (MOE-EL) proposed in this paper, the MOE-EL framework is compared with the widely used and latest methods in ultra-short-term load forecasting, namely the ARIMA model, SVR model, BiLSTM model and CNN-GRU model.

[0055] As shown in Table 1, overall, the ARIMA model has the worst ultra-short-term prediction results for the three loads, the SVR model has a certain improvement in prediction effect, the BiLSTM model has a significantly better prediction effect than the above two models, the CNN-GRU model has a better effect than the first three models, and the MOE-EL in this paper has a significantly better prediction effect than other models.

[0056] Table 1. Comparison of prediction results of different models

[0057]

[0058] Another embodiment of the present invention provides an ultra-short-term load forecasting system based on a hybrid expert model, comprising:

[0059] The time effect expert sub-model construction module is used to construct an expert model for learning the load period's influence pattern on the load, which is called the time effect expert sub-model;

[0060] The fluctuation effect expert sub-model building module is used to build an expert model of the short-term fluctuation characteristics of the learning load, which is called the fluctuation effect expert sub-model;

[0061] The gating mechanism module is used to integrate the outputs of the time effect expert sub-model and the fluctuation effect expert sub-model through the gating mechanism to obtain the ultra-short-term load forecasting results.

[0062] The division of the above modules is only an example. In actual applications, the above functions can be assigned to different functional modules as needed to complete all or part of the functions described in the above method. The specific working process of each module can refer to the corresponding process in the above method embodiment, which will not be repeated here.

[0063] Another embodiment of the present invention provides a computer device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0064] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk), wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the steps of the method of the present invention are implemented.

[0065] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and implement it accordingly. It can be understood by those skilled in the art that various replacements, changes and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the contents disclosed in the embodiments of this specification, and the scope of protection of the present invention shall be subject to the scope defined in the claims.

Claims

1. An ultra-short-term load forecasting method based on a hybrid expert model, characterized in that: The following steps are involved: Construct an expert model to learn the load period's influence on the load pattern, called the time effect expert sub-model; Construct an expert model of the short-term fluctuation characteristics of learning load, called the fluctuation effect expert sub-model; The outputs of the time effect expert sub-model and the fluctuation effect expert sub-model are integrated through the gating mechanism to obtain the ultra-short-term load forecasting results.

2. The method according to claim 1, characterized in that The steps of constructing the time effect expert sub-model include: Extract the date characteristics of the load, and classify the load according to the date characteristics of the load to obtain multiple load categories; A training data set and prediction model are constructed for each load category, and model training is performed based on each load category to obtain a time effect expert sub-model.

3. The method according to claim 2, characterized in that The load is classified according to its date characteristics, and the load is divided into nine load categories, namely, weekday load, weekend load, New Year's Day load, Spring Festival load, Qingming Festival load, Labor Day load, Dragon Boat Festival load, Mid-Autumn Festival load and National Day load.

4. The method according to claim 3, characterized in that For weekday loads and weekend loads, the Transformer model is used to construct a time effect expert sub-model; for New Year's Day loads, the KRLS model is used to construct a time effect expert sub-model; for Spring Festival loads, the SVR model is used to construct a time effect expert sub-model; for Qingming Festival loads, the KRLS model is used to construct a time effect expert sub-model; for Labor Day loads, the SVR model is used to construct a time effect expert sub-model; for Dragon Boat Festival loads and Mid-Autumn Festival loads, the KRLS model is used to construct a time effect expert sub-model; for National Day loads, the SVR model is used to construct a time effect expert sub-model.

5. The method according to claim 1, characterized in that: The fluctuation effect expert sub-model is constructed by combining the LSTM meta-learner and the MLP base learner.

6. The method according to claim 5, characterized in that The training process of the fluctuation effect expert sub-model includes: For the input load data, calculate the loss function value and loss function gradient value of the MLP basic learner; Feed the loss function value and the loss function gradient value back to the LSTM meta-learner to update the cell state of the meta-learner; Update the parameters of the base learner using the updated cell states of the meta-learner.

7. The method according to claim 1, characterized in that The outputs of the time effect expert sub-model and the fluctuation effect expert sub-model are integrated through the gating mechanism, including: Based on the input load data, a softmax function is used to assign weights to each expert sub-model; The expert sub-models corresponding to the highest K weights are selected, and the output results of the K expert sub-models are weighted to obtain the final load forecasting result.

8. An ultra-short-term load forecasting system based on a hybrid expert model, characterized in that: include: The time effect expert sub-model construction module is used to construct an expert model for learning the load period's influence pattern on the load, which is called the time effect expert sub-model; The fluctuation effect expert sub-model building module is used to build an expert model of the short-term fluctuation characteristics of the learning load, which is called the fluctuation effect expert sub-model; The gating mechanism module is used to integrate the outputs of the time effect expert sub-model and the fluctuation effect expert sub-model through the gating mechanism to obtain the ultra-short-term load forecasting results.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 7 is implemented.

Images