A holiday load forecasting method based on ANN and transfer learning

Abstract

This invention discloses a holiday load prediction method based on ANN and transfer learning, comprising the following steps: performing feature engineering operations on the preprocessed dataset to form a full load dataset; extracting the holiday load dataset and the festival load dataset from the full load dataset; constructing a full data source model using an ANN, and training the full data source model using the full load training dataset; using the full data source model as a pre-trained model to train the festival / holiday load dataset, using the same parameters as the full data source model, to obtain the final festival / holiday load training model; predicting the load values ​​at each time point of the prediction day, obtaining the prediction results for the load at each time point of the prediction day. By using ANN to predict the holiday load data and the festival load data separately for holiday prediction, the characteristics of each holiday can be better learned, thereby improving the prediction accuracy.

Description

Technical Field

[0001] This invention belongs to the field of load forecasting technology, specifically relating to a holiday load forecasting method based on ANN and transfer learning. Background Technology

[0002] Electricity load forecasting is a crucial task for the power sector, serving as the foundation for ensuring a balance between electricity supply and demand and playing a vital role in promoting national economic development. Electricity load forecasting provides essential information for electricity planning and design, as well as for the operation and decision-making of power grid companies and the power sector.

[0003] The method for forecasting load during holidays differs from that of ordinary short-term load forecasting. This is because there is much less data available for holidays than for working days.

[0004] Holiday load forecasting is a challenging aspect of short-term load forecasting, prompting extensive research and the emergence of numerous methods. Load pattern classification systems based on data mining and genetic optimization algorithms (GA) have complex classification rules and are difficult to implement. Support vector machines (SVMs) in short-term load forecasting often suffer from inaccurate predictions due to insufficient holiday data. Methods based on similar weekends incorporate weekend data into the selection of similar days, choosing days similar to previous holidays from both weekend data and historical holidays; however, significant regional variations due to weather and geographical influences lead to large prediction deviations. Fuzzy neural networks, methods using curve patterns and load level patterns, correction of similar days based on morphological characteristics, point-to-point ratio methods, ratio smoothing methods, and point-by-point growth rate methods are also frequently used holiday load forecasting methods, but their accuracy remains unsatisfactory due to limited holiday data.

[0005] To address the issue of limited holiday data, a common solution is to incorporate weekend data into the holiday data set. However, weekend data has a higher load, and the shape of the load curve varies, resulting in less than ideal prediction accuracy. Furthermore, existing holiday prediction techniques exhibit unstable accuracy with significant fluctuations, leading to a decrease in the average accuracy for holidays, months, and years. Current techniques typically input all features together into the model for training without analyzing feature redundancy, ineffectiveness, or negative effects, resulting in reduced model training efficiency. They generally fail to consider the differences in load curve shape and load between holiday and festival data, directly using all data from the entire holiday period. Moreover, most existing techniques neglect the relationships between different holidays. Summary of the Invention

[0006] To overcome the problems in the prior art, this invention proposes a holiday load prediction method based on ANN (Artificial Neural Network) and transfer learning.

[0007] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0008] A holiday load forecasting method based on ANN and transfer learning includes the following steps:

[0009] Step 1: Obtain the weather forecast, holidays, and daily load, actual weather, and holiday information for the predicted date, as well as for each time point in the previous N years, to form the initial dataset;

[0010] Step 2: Preprocess the initial dataset to form a preprocessed dataset;

[0011] Step 3: Perform feature engineering on the preprocessed dataset to form the full load dataset, and extract the holiday load dataset and the holiday load dataset from the full load dataset;

[0012] Step 4: Use ANN to build a full data source model, and train the full data source model using the full load training dataset to obtain a trained full data source model;

[0013] Step 5: Use the trained full data source model as a pre-trained model to train the holiday load dataset using the same parameters as the full data source model to obtain the final holiday load training model; train the holiday load dataset using the same parameters as the full data source model to obtain the final holiday load training model.

[0014] Step 6: Predict the load values ​​at each time point on the forecast day to obtain the predicted load results at each time point on the forecast day.

[0015] Furthermore, in step 2, the preprocessing of the initial dataset includes: completing missing numerical data; correcting holiday data; and encoding the weather and holiday information attribute data.

[0016] Furthermore, in step 3, the feature engineering operation specifically includes:

[0017] Add the temperature features of the previous day: Calculate and add the maximum, minimum, and average temperatures of the day before the forecast date to enhance the meteorological impact of adjacent two days;

[0018] Add load characteristics from the previous day: Calculate and add the maximum, minimum, and average loads from the day before the forecast date to enhance the load impact between adjacent days;

[0019] Add load characteristics from the previous two days: Calculate and add the maximum, minimum, and average loads from the two days prior to the forecast date to enhance the load impact of adjacent three days.

[0020] Furthermore, in step 3, feature engineering also includes feature importance analysis, which is divided into all data feature importance analysis and holiday / holiday feature importance analysis. The all data feature importance analysis uses all load data to participate in feature importance analysis, while the holiday / holiday feature importance analysis uses holiday / holiday load data to perform feature importance analysis. Features that exceed the threshold or rank among the top few are used as inputs to the model for training.

[0021] Furthermore, in step 3, the feature importance analysis methods include, but are not limited to, Pearson coefficient, mutual information, grey relational analysis, and SHAP.

[0022] Furthermore, in step 3, after selecting appropriate features through feature importance analysis, the data is normalized to eliminate the dimensional differences between different features. The normalization methods include, but are not limited to, maximum-minimum normalization and mean normalization.

[0023] Furthermore, in step 4, the artificial neural network structure includes one input layer, two hidden layers, and one output layer.

[0024] Furthermore, step 4 also includes: determining whether to retrain the model based on the prediction accuracy of the day before the prediction date; adjusting the corresponding parameters and retraining the model for models with accuracy less than a threshold; and using models with accuracy reaching the threshold as source models for transfer learning in the training of the holiday model.

[0025] Furthermore, the accuracy formula is as follows:

[0026]

[0027] in,

[0028]

[0029] In the above formula, VALUE pred VALUE is the predicted value. true E is the true value. i The relative error of the prediction at each time point, where N is the number of prediction time points.

[0030] Furthermore, step 6 also includes: denormalizing the load forecast results to convert them into load forecast values ​​of the original load level.

[0031] Compared with the prior art, the present invention has the following technical effects:

[0032] (1) In view of the reasons that features are redundant, features have no effect on the prediction results, and features have a negative effect on the prediction results, this invention proposes to perform feature importance analysis before model training, reduce the number of features, and thus improve the training efficiency of the model.

[0033] (2) Considering the large difference in load between festival data and holiday data and the correlation between various holidays, festival load data and holiday load data are predicted by ANN respectively. This can better learn the characteristics of festivals and holidays, thereby improving the prediction accuracy.

[0034] (3) Since the split holiday load data and holiday load data are small samples, transfer learning is added to solve this problem, thereby improving the prediction accuracy. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall system structure of the present invention;

[0036] Figure 2 This is a schematic diagram of the ANN structure of the present invention;

[0037] Figure 3 This is a schematic diagram illustrating the transfer learning process of this invention;

[0038] Figure 4 This is a bar chart showing the Pearson coefficients for various features at different times during holidays according to the present invention.

[0039] Figure 5 This is a bar chart showing the mutual information of various features at different times during holidays according to the present invention;

[0040] Figure 6 This is a bar chart showing the gray relational degree of various features at different times during holidays in this invention.

[0041] Figure 7 This is a schematic diagram comparing holiday load images and festival load images according to the present invention. Detailed Implementation

[0042] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0043] Reference Figures 1-7 This invention addresses issues such as feature redundancy, features having no impact on prediction results, and features having a negative impact on prediction results by proposing a holiday load prediction method based on ANN and transfer learning, comprising the following steps:

[0044] Step 1: Obtain the weather forecast, holidays, and daily load, actual weather, and holiday information for the predicted date, and construct the initial dataset.

[0045] It is worth noting the difference between festivals and holidays. Taking the Qingming Festival holiday as an example, Qingming Festival itself is a festival, while other Qingming Festival holidays are holidays.

[0046] Step 2: Preprocess the initial dataset to form a preprocessed dataset; the data preprocessing steps mainly include the following aspects:

[0047] Missing value imputation: Filling in missing data due to reasons such as acquisition equipment, including but not limited to filling in numerical data and time-type data;

[0048] Add feature columns: Add holiday features through third-party packages or other holiday data, and add load information for each time point of the previous day, the maximum, minimum and average load information of the previous day, and the maximum, minimum and average load information of each day of the previous N years by calculation;

[0049] Discrete data encoding: Encoding discrete data such as holiday information into numerical data. For discrete data encoding, third-party packages can be used to convert character-based data into numerical data; for example, the `LabelEncoder` function in `sklearn.preprocessing` can directly achieve this effect.

[0050] Step 3: Perform feature engineering on the preprocessed dataset to form the full load dataset; divide the full load dataset into the holiday load dataset and the holiday load dataset;

[0051] Feature engineering is an indispensable part of machine learning and occupies a very important position in the field. The main function of feature engineering is to extract features from raw data to improve training effectiveness and performance.

[0052] The feature engineering operations of this invention mainly include:

[0053] Add the temperature features of the previous day: Calculate and add the maximum, minimum, and average temperatures of the day before the forecast date to enhance the meteorological impact of adjacent two days;

[0054] Add load characteristics from the previous day: Calculate and add the maximum, minimum, and average loads from the day before the forecast date to enhance the load impact between adjacent days;

[0055] Add load characteristics from the previous two days: Calculate and add the maximum, minimum, and average loads from the two days prior to the forecast date to enhance the load impact of adjacent three days.

[0056] The training efficiency of a model is often closely related to the number of features. Generally, the more features a model has, the lower its training efficiency and the longer its running time; conversely, the fewer features a model has, the higher its training efficiency and the shorter its running time. Among these features, some may be redundant, some may have a negative impact on the prediction results, or some may be completely irrelevant to the prediction results. However, fewer features are not always better; choosing a more concise combination of features without compromising accuracy is more important.

[0057] This invention addresses the aforementioned problems by performing feature importance analysis, reducing redundancy among various features. Through single or combined methods in feature importance analysis, it selects feature columns that have a significant impact on the prediction results and have less redundancy as input features for the model, thereby improving the efficiency and effectiveness of the prediction model.

[0058] The feature importance analysis employed in this invention includes, but is not limited to, Pearson coefficient, mutual information, grey relational analysis, the model's built-in feature importance analysis methods, and SHAP (SHapley Additive exPlanations). Using these individual methods or combinations thereof, features exceeding a threshold or ranking among the top features are selected as input to the model for training.

[0059] Feature importance analysis is divided into full data feature importance analysis and holiday / holiday feature importance analysis. Full data feature importance analysis uses all data to participate in feature importance analysis, while holiday / holiday feature importance analysis uses holiday / holiday data for feature importance analysis. Due to the different data used, the feature importance scores of the same feature are also different.

[0060] After selecting appropriate features through feature importance analysis, the data needs to be normalized to eliminate the dimensional differences between different features, thereby facilitating algorithm training. Normalization methods include, but are not limited to, max-min normalization and mean normalization.

[0061] The maximum-minimum normalization formula is as follows:

[0062]

[0063] In the formula, a i Here, 'a' is the value to be normalized. min Let a be the minimum value in the feature column containing the element 'a' in the data. max This represents the maximum value of the feature column containing 'a' in the data.

[0064] During the normalization process, it is necessary to record the maximum and minimum values ​​of the feature columns, and the subsequent prediction results need to be denormalized.

[0065] Step 4: Use ANN to build a full data source model, and train the full data source model using the full load training dataset to obtain a trained full data source model.

[0066] The full data source model serves as the source model for transfer learning. Transfer learning effectively reduces training time, enabling the training of high-performing models even with small sample sizes and reducing overfitting. The full data source model uses all training data, including holiday and weekday data, as training samples and employs an ANN network. The model is retrained based on its prediction accuracy for the day before the forecast date. Models with accuracy below a threshold are retrained with adjusted parameters. Models with accuracy reaching the threshold are used as the source model for transferring learning in the holiday model training.

[0067] Specifically, the precision formula is as follows:

[0068]

[0069]

[0070] In the above formula, VALUE pred VALUE is the predicted value. true E is the true value. i The relative error of the prediction for each time point is given, and N is the number of predicted time points. If each time point is 15 minutes, then the number of predicted time points N is 96.

[0071] Step 5: Use the trained full data source model as a pre-trained model to train the holiday load dataset using the same parameters as the full data source model to obtain the final holiday load training model; train the holiday load dataset using the same parameters as the full data source model to obtain the final holiday load training model.

[0072] Because there are significant differences in load volume and load graph shape between holiday data and regular holiday data, the holiday data is also split into holiday data and regular holiday data for training the holiday load model and the regular holiday load model, respectively. The training process for the holiday load model and the regular holiday load model is the same. The holiday / regular holiday model training process uses holiday / regular holiday data as training samples, and the pre-trained model uses the full data source model, and the parameters are still the same as those used when training the full data source model.

[0073] Step 6: Predict the load values ​​at each time point on the forecast day to obtain the predicted load results at each time point on the forecast day.

[0074] The prediction results are normalized values. Therefore, it is necessary to denormalize the prediction results to convert them into load prediction values ​​of the original load level. That is, the maximum and minimum values ​​of the feature column recorded in step 3 will also participate in the denormalization process.

[0075] This invention trains the model separately for holiday load data and daytime load data, which better avoids the negative impact between holidays and daytime loads. The ANN (Application Not Responsible) can also better explore the connections between different holidays, thereby improving the algorithm's prediction accuracy. In load data, holiday load data is relatively scarce compared to weekday data, clearly representing a small sample size. Direct prediction accuracy for small samples is generally low. To address the problem of limited holiday load data, transfer learning is a very effective method. Transfer learning can achieve high holiday prediction accuracy even with limited holiday load data. Current holiday prediction methods are mostly unstable, meaning the prediction accuracy fluctuates greatly between different holidays. For the power sector, stable and highly accurate prediction results are paramount. The ANN+transfer learning holiday prediction method used in this invention can predict holiday loads relatively stably, with an accuracy of around 90% for each holiday and an accuracy error generally not exceeding 10%.

[0076] Experimental results

[0077] To verify the feasibility and usability of this invention, several experiments were conducted, and the results are as follows:

[0078] Experiment 1: Feature Importance Analysis Experiment

[0079] 1) The Pearson coefficient values ​​for each feature are as follows:

[0080] Table 1. Pearson coefficient values ​​for various characteristics at different times during holidays.

[0081]

[0082]

[0083] The Pearson coefficient ranges from [-1, 1]. Since the Pearson coefficients at various time points are consistently high, close to 1, two time points are selected as representative features in the figure. (From Table 1 and...) Figure 4 As can be seen, regardless of whether the impact is positive or negative, the Pearson coefficient values ​​of features such as week information, whether it is a holiday (HOLIDAY), whether it is a weekday (GZR), and holiday type (HOLIDAY_TYPE) are lower than the pre-set threshold. This means that these features have a small impact on the result column, and these features can be excluded from the model training process.

[0084] 2) The mutual information values ​​of each feature are as follows:

[0085] Table 2 Mutual information values ​​of various features at different times during holidays.

[0086]

[0087]

[0088] From Table 2 and Figure 5 As can be seen, the mutual information values ​​of features such as whether it is a holiday (HOLIDAY), whether it is a weekday (GZR), whether it is a holiday day (FIRST_HOLIDAY), holiday type (HOLIDAY_TYPE), and the number of days until the Spring Festival (JL) are relatively small (less than the preset threshold). That is, the impact on the time point feature column is relatively small, and these features do not need to participate in the model training process.

[0089] 3) The grey relational values ​​of each feature are as follows:

[0090] Table 3. Grey Relation Values ​​of Various Features at Different Holiday Time Points

[0091]

[0092] From Table 3 and Figure 6 The data shows whether it is a holiday (HOLIDAY), whether it is a weekday (GZR), whether it is a holiday (FIRST_HOLIDAY), and the number of days until the Spring Festival (JL). The mutual information values ​​of these features are relatively small (less than the preset threshold), meaning they have a relatively small impact on the time point feature column. These features do not need to participate in the model training process.

[0093] Combining the three feature importance analysis methods mentioned above, the following can be excluded from model training: week information (week), whether it is a holiday (HOLIDAY), whether it is a weekday (GZR), whether it is a holiday day (FIRST_HOLIDAY), holiday type (HOLIDAY_TYPE), and the number of days until the Spring Festival (JL), thereby improving the efficiency of model training.

[0094] Table 4 Comparison of accuracy using all features versus simplified features

[0095]

[0096] Table 4 shows that using simplified features resulted in both increases and decreases in the accuracy of holiday load forecasting, but the increase in accuracy was greater than the decrease in feature accuracy. A lower feature importance score indicates that a smaller portion of the data is correlated with that feature, so for a small number of holidays, the prediction accuracy using all features is higher than that using simplified features. However, overall, using feature importance analysis is reasonable.

[0097] Experiment 2: The Necessity of Splitting Holiday Data into Holiday Load Data and Holiday Load Data

[0098] To improve the accuracy of prediction results, this invention splits holiday data into two parts: holiday load data and holiday load data, and conducts experiments on each part separately to compare the accuracy of using all holiday data versus using only the holiday load data. The conditions for using the holiday load data and holiday load data are: holiday load data is used when the predicted date is a holiday, and holiday load data is used when the predicted date is a public holiday.

[0099] from Figure 7 As can be seen, the load and load image shapes for the holiday April 30th and the holiday May 1st are completely different. Therefore, the holiday data and the holiday data can be separated and predicted separately for the holiday and the holiday.

[0100] Table 5 compares the accuracy of data using all holidays versus data using only holidays.

[0101]

[0102] As can be seen from Table 5, the prediction accuracy of the model trained using all holiday data is lower than that trained using holiday data alone. Therefore, it is reasonable and correct to divide all holiday data into holiday load data and holiday load data.

[0103] Experiment 3: The Impact of Transfer Learning on Experiments

[0104] Table 6 compares the prediction accuracy of ANN without transfer learning versus ANN with transfer learning.

[0105]

[0106] As can be seen from Table 6, using transfer learning can improve accuracy in most cases. Therefore, compared with simply using holiday and festival data, using transfer learning is undoubtedly a better choice.

[0107] Experiment 4: Comparison of ANN+transfer learning with other methods in this invention

[0108] Table 7 Comparison of prediction accuracy between ANN+ transfer learning and other methods

[0109]

[0110]

[0111] As shown in Table 7, the similar daily load correction method, the improved ratio smoothing method, and the Qingchuang patent + filtering method achieve high prediction results on some days, but their prediction accuracy also suddenly drops below 80%, or even below 70%, on other days. In contrast, the ANN + transfer learning method, with an appropriate learning rate, generally does not drop below 80% in accuracy, mostly remaining around 90%, with an accuracy error not exceeding 10%.

[0112] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A holiday load forecasting method based on ANN and transfer learning, characterized in that, The method comprises the following steps: Step 1: Obtain the weather forecast of the predicted day, holidays and the daily load of each time point in the previous N years, the real weather and holiday information to form an initial data set; Step 2: Preprocess the initial data set to form a preprocessed data set; Step 3: Perform feature engineering operations on the preprocessed data set to form an all-load data set, and extract a holiday load data set and a holiday load data set from the all-load data set; The feature engineering operation specifically comprises: Add the temperature features of the previous day: calculate and add the maximum, minimum and average temperature of the previous day of the predicted day to enhance the meteorological influence of the adjacent two days; Add the load features of the previous day: calculate and add the maximum, minimum and average load of the previous day of the predicted day to enhance the load influence of the adjacent two days; Add the load features of the previous two days: calculate and add the maximum, minimum and average load of the previous two days of the predicted day to enhance the load influence of the adjacent three days; Step 4: Construct an ANN model using all data sources, and train the all-data-source model using the all-load training data set to obtain a trained all-data-source model; determine whether to retrain the model according to the prediction accuracy of the previous day of the predicted day; adjust the corresponding parameters to retrain the model whose accuracy is less than the threshold value, and use the model whose accuracy reaches the threshold value as a source model for the training of a holiday model; Step 5: Use the trained all-data-source model as a pre-trained model to train the holiday load data set, and use the same parameters as the all-data-source model to obtain a final holiday load training model; train the holiday load data set using the same parameters as the all-data-source model to obtain a final holiday load training model; Step 6: Predict the load value of each time point of the predicted day to obtain the prediction result of the load of each time point of the predicted day.

2. The holiday load forecasting method based on ANN and transfer learning according to claim 1, characterized in that, In step 2, preprocessing the initial data set comprises: completing missing numerical data; correcting holiday data; and encoding type data of weather and holiday information attributes.

3. The holiday load forecasting method based on ANN and transfer learning according to claim 2, characterized in that, In step 3, the feature engineering further comprises feature importance analysis, which is divided into all-data feature importance analysis and holiday / holiday feature importance analysis. The all-data feature importance analysis uses all-load data to participate in feature importance analysis, and the holiday / holiday feature importance analysis uses holiday / holiday load data to participate in feature importance analysis. The features that exceed the threshold value or are in the top few are used as inputs of the model for training.

4. The holiday load forecasting method based on ANN and transfer learning according to claim 3, characterized in that, In step 3, the feature importance analysis method includes but is not limited to Pearson coefficient, mutual information, grey correlation degree analysis and SHAP.

5. The holiday load forecasting method based on ANN and transfer learning according to claim 4, characterized in that, In step 3, after selecting appropriate features through feature importance analysis, the data is normalized to eliminate the dimensional differences between different features. The normalization methods include but are not limited to maximum-minimum normalization and mean normalization.

6. The holiday load forecasting method based on ANN and transfer learning according to claim 1, characterized in that, In step 4, the ANN artificial neural network structure comprises one input layer, two hidden layers and one output layer.

7. The holiday load forecasting method based on ANN and transfer learning according to claim 1, characterized in that, The precision formula is as follows: wherein, In the above formula, VALUE pred is the predicted value, VALUE true is the true value, E i is the relative error of the prediction at each time point, and N is the number of prediction time points.

8. The holiday load forecasting method based on ANN and transfer learning according to claim 7, characterized in that, In step 6, the load prediction result is further normalized to convert it into a load prediction value at the original load level.

Images