Load probability forecasting method based on dual attention and quantile regression

Abstract

The present invention discloses a load probability forecasting method based on dual attention and quantile regression, and relates to the technical field of power systems. The prior art does not adequately characterize load uncertainty in power load forecasting, and the information loss in the feature extraction process affects the accuracy and generalization performance of the prediction model. The present invention includes the steps of: performing standardized modeling on multi-time scale load forecasting problems, considering the multi-dimensional correlation characteristics of power loads, and constructing a multi-time scale load forecasting framework based on sliding windows. Then, based on the dual attention mechanism, the autocorrelation characteristics of the power load in the time series dimension and the cross-correlation characteristics with its influencing factors in the feature dimension are mined, and the power load probability forecasting is realized by improving the quantile regression. The method of the present invention can comprehensively consider the time series autocorrelation characteristics and cross-correlation characteristics of the power load, realize the quantification of the random uncertainty of load fluctuations, and improve the reliability and accuracy of the prediction.

Description

Technical Field

[0001] The invention relates to the technical field of power systems, and in particular to a load probability prediction method based on double attention and quantile regression. Background Art

[0002] With the accelerated construction of new power systems, the proportion of flexible loads (electric vehicles, energy storage, distributed power sources, etc.) in power systems has increased significantly, the randomness and volatility of load changes have increased significantly, and the uncertainty of power load changes has brought huge challenges to the safe operation of power systems. Scientific prediction is the basis and guarantee for correct decision-making. How to characterize the randomness and uncertainty of load changes is the key to accurate prediction of power loads.

[0003] At present, there have been many studies on power load forecasting. The main prediction methods for power load are traditional prediction methods based on time series analysis and intelligent prediction methods based on artificial intelligence (including traditional machine learning and deep learning). Traditional prediction methods include Kalman filtering, differential integrated moving average autoregression, exponential smoothing, etc.; intelligent prediction methods include autoencoders, long short-term memory networks (LSTM), convolutional neural networks (CNN), time series convolutional neural networks, etc. Neural network models have strong nonlinear mapping capabilities and feature mining characteristics, so they are widely used in the field of load forecasting research. However, the above methods can only provide deterministic prediction results, and the uncertainty of load in new power systems is not sufficiently characterized. Or only the change law and characteristics of the load history sequence (autocorrelation characteristics) are considered, or only the correlation between the load and external influencing factors is considered, or the load history sequence and external influencing factors are directly input into the model without distinction. Feature information is missing during feature extraction, which affects the prediction accuracy and generalization performance of the prediction accuracy model. Summary of the invention

[0004] The technical problem to be solved and the technical task proposed by the present invention are to improve and perfect the existing technical solutions and provide a load probability forecasting method based on dual attention and quantile regression to improve the reliability and accuracy of the forecast. To this end, the present invention adopts the following technical solutions.

[0005] The load probability forecasting method based on dual attention and quantile regression includes the following steps:

[0006] 100) Standardize and model multi-time scale load data, consider the autocorrelation characteristics of load and the cross-correlation characteristics with external influencing factors, and build a load probability forecasting framework based on sliding windows;

[0007] 200) Based on the dual attention mechanism, the autocorrelation characteristics of power load in the time series dimension and the cross-correlation characteristics with its influencing factors in the feature dimension are extracted, and a quantile regression feedforward network is constructed to map the autocorrelation characteristics and cross-correlation characteristics to the load prediction quantile values. A load probability prediction model is constructed based on the sliding window prediction framework and the extracted multidimensional features. The constructed load probability prediction model is used for load probability prediction.

[0008] In the standardized modeling process, this technical solution fully considers the multi-dimensional correlation characteristics of power load, which enables the model to more accurately reflect the complex relationship between power load and various influencing factors, thereby improving the accuracy of prediction. By constructing a multi-time scale load forecasting framework based on sliding windows, it can simultaneously cope with the load forecasting needs of different time scales (such as ultra-short term, short term and medium- and long-term), enhancing the adaptability and versatility of the model. By using the dual attention mechanism, the autocorrelation and cross-correlation of the load are mined in the time series dimension and feature dimension respectively, and the attention weight is dynamically allocated, which can more effectively capture the changing law of power load and improve the performance of the prediction model. By improving the quantile regression method, the model can more accurately estimate the probability distribution of the load and provide a reliable prediction interval, which not only improves the prediction accuracy, but also provides uncertainty information of the load forecast, making the prediction results more credible.

[0009] This technical solution achieves high-precision and highly adaptable power load probability prediction by comprehensively considering multi-dimensional features, multiple time scales and dual attention mechanism, combined with improved quantile regression.

[0010] As a preferred technical means: the standardized modeling in step 100) includes the following steps:

[0011] 110) Incorporate multi-source heterogeneous data including load data, meteorological data, and socio-economic data into the multi-time scale load forecasting dataset to comprehensively consider the coupling relationship between autocorrelation characteristics and external factors;

[0012] 120) Encode various holidays and distinguish different types of electricity consumption days to consider the impact of holidays on power load;

[0013] 130) Use sine and cosine coding to uniquely map the monthly, daily, working day and moment time scales of the load, and characterize the autocorrelation characteristics of the power load at different time scales.

[0014] By incorporating multi-source heterogeneous data such as load data, meteorological data, and socio-economic data into the load forecasting dataset, the complex coupling relationship between the intrinsic autocorrelation characteristics of the power load and external influencing factors can be comprehensively considered, thereby improving the overall accuracy of the forecasting model. By encoding holidays and distinguishing different types of electricity consumption days, the model can better capture the special impact of holidays on the power load, significantly improve the load forecasting accuracy on special dates, and reduce the impact of abnormal data on the model. Using sine and cosine coding to uniquely map the load on the month, day, working day, and time effectively characterizes the autocorrelation characteristics of the power load at different time scales. This coding method can capture the periodic and trend changes of the load in the time dimension, and enhance the model's ability to predict load changes at different time scales.

[0015] This technical solution improves the accuracy and robustness of the load forecasting model by integrating multi-source data, considering the impact of holidays and fine-tuning time-scale coding, so that the model can more accurately reflect the complex changes in power load.

[0016] As a preferred technical means: in step 110), the multi-time scale load forecasting data set is represented as:

[0017]

[0018] in,

[0019] Where: t for t Multi-dimensional load characteristic vector at each moment; for t Moment auxiliary feature vector; for t The load of statistical caliber at different time dimensions at all times; N is the number of load samples; D is the characteristic dimension; for t Time stamp encoding vector; for t Meteorological characteristic vector at the time, including temperature, wind speed, and precipitation; for t Economic statistics indicators at all times; Code for holidays.

[0020] This technical solution introduces multi-dimensional feature vectors including load characteristics, auxiliary characteristics, meteorological characteristics, economic statistical indicators and holiday codes. The solution can fully capture various factors affecting power load. The feature vector at each moment covers the intrinsic characteristics of load changes and external influencing factors, making the prediction more comprehensive and accurate. This data set allows the processing of load data in different time dimensions (such as hours, days, months, etc.). By counting the load calibers at different time scales, the model can flexibly adapt to short-term, medium-term and long-term load forecasting needs, enhancing the applicability of the model. By integrating various high-dimensional feature vectors (such as meteorological characteristics and economic statistical indicators) into the load forecasting model, the solution can make full use of these high-dimensional information to improve the prediction accuracy of the model. At the same time, this integration method can also help the model capture the complex correlation between features and improve the understanding and prediction ability of load change trends. The time-scale coding vector can help the model better understand the load change characteristics in the time dimension, including periodic and trend changes, and enhance the prediction accuracy of the model at different time points. Especially when processing data at different time scales, it can better capture the impact of time on load. By specifically encoding holidays and incorporating auxiliary features, we can effectively address the special impact of holidays on power load and make the model's predictions on special dates more accurate. In addition, the introduction of auxiliary feature vectors also increases the flexibility of the prediction and can adapt to different load forecasting scenarios.

[0021] As a preferred technical means: Step 130) uses a sine-cosine encoding method to encode the time-scale sine-cosine of each time scale, and the expression is:

[0022] Where: M sin, t and M cos, t are the sine and cosine coded values ​​of the load monthly time scale respectively; D sin, t and D cos, t are the sine and cosine encoding values ​​of the load day time scale respectively; sin, t and cos, t are the sine and cosine encoding values ​​of the load working day time scale respectively; H sin, t and H cos, t They are the sine and cosine encoding values ​​at the load moment respectively; the unique mapping of time scales in various time dimensions is achieved through sine and cosine encoding.

[0023] The sine-cosine coding of this technical solution can well express the periodic characteristics of the time dimension. Since the time characteristics are essentially cyclical (such as 24 hours in a day and 12 months in a year), the sine-cosine coding can convert this cyclical characteristic into a continuous numerical representation, helping the model to better capture the periodic changes of power load at different time scales.

[0024] Traditional integer encoding may cause the model to misunderstand the continuity or distance of time features. For example, directly using integers to represent months (such as 1 to 12) will make the model think that the distance between months is linear and ignore its cyclicity. Sin-cosine encoding avoids these singularity problems by mapping the time scale to a circle, making it easier for the model to learn the true relationship between time features. Sin-cosine encoding can provide richer information for the model and enhance the model's perception of time features by mapping time features to two-dimensional space (i.e., sine and cosine values). It can not only retain the periodicity of time features, but also increase the expression ability of features, thereby improving prediction accuracy. Sin-cosine encoding is uniformly used for each time scale (such as month, day, weekday, and time), realizing the unique mapping of time scales. The model can process time features in a consistent encoding method, simplifying the feature engineering process and improving the generalization ability of the model. The values ​​generated by sin-cosine encoding change continuously on the time scale, avoiding the non-smoothness problem that may be introduced by discretization encoding. The smooth feature expression can better reflect the changing trend of load in the time dimension, making the model more sensitive to time features.

[0025] As a preferred technical means: the expression for constructing a load probability forecasting framework based on a sliding window is:

[0026]

[0027] in,

[0028] Where: is the input historical load characteristic vector, K bac is the length of the input historical load characteristic vector; A time stamp code for the load timestamp to be predicted; is the weather forecast feature vector at the time to be predicted; It is the economic forecast statistical indicator at the time to be predicted; Code special holidays at the time to be predicted; K step is the prediction step length, where K step =1 epoch single-step prediction; K ahe is the prediction step ahead; are the parameters of the multi-time scale load forecasting model; Represents the mapping from the above input features to load forecast values; Indicates the predicted value of the corresponding feature quantity.

[0029] The expression of this technical solution incorporates multi-dimensional features such as historical load characteristics, time characteristics, weather forecasts, economic statistical indicators, and holiday codes into the modeling process. By comprehensively considering these influencing factors, the model can more comprehensively capture the complex relationship that affects the power load and improve the accuracy of the prediction. The model adopts a sliding window mechanism, which can flexibly handle historical load characteristics (Kbac) and prediction steps (Kstep) of different time scales. It can not only handle single-step predictions, but also multi-step predictions, making the model more widely applicable in various practical application scenarios. By introducing time-scale coding, meteorological characteristics, economic characteristics, and holiday characteristics, the model can capture the potential laws and trends of load changes on different time scales, making the model more accurate in predicting future loads, especially in dealing with special situations such as holidays or abnormal weather. The model can make predictions more sensitively. By considering historical load characteristics and multiple external influencing factors at future moments (such as meteorology, economy, holidays, etc.), the model can perform well in short-term and long-term load forecasting, making the prediction results more accurate and reliable.

[0030] As a preferred technical means: construct a load probability prediction model, specifically including:

[0031] The multi-dimensional data is processed into a sliding window, and the load history autocorrelation characteristics and load synchronization correlation characteristics within the sliding window are used as model input characteristics;

[0032] As the sliding window moves, the power load is predicted in a rolling manner, taking into account the autocorrelation of load changes and the cross-correlation between the load and its influencing factors;

[0033] For ultra-short-term and short-term load forecasting, the resolution of each feature is 15 minutes. For low-resolution features, the super-resolution method is used to unify the resolution;

[0034] For medium- and long-term load forecasting, time-scale characteristics are encoded using the time-scale code of the corresponding time dimension. Meteorological characteristics are represented by average, maximum and minimum values. Economic factors are represented by statistical indicators of the corresponding time dimension. Holiday factors are represented by the number of holidays and holiday types.

[0035] Through the sliding window mechanism, the model can make rolling predictions based on the continuously updated real-time data, so that the model can dynamically adapt to the changes in load and respond to new data input more quickly, thereby improving the timeliness and accuracy of the prediction. This technical solution fully considers the historical autocorrelation characteristics of the load and the correlation between the load and the influencing factors, covering multi-dimensional characteristics such as load data, meteorological data, economic indicators and holidays. Combining the interconnected relationship of these characteristics, the model can more accurately capture the fluctuations and trends of the power load. This technical solution has formulated specific processing methods for load forecasts of different time scales (ultra-short-term, short-term, medium- and long-term). For example, ultra-short-term and short-term forecasts use 15-minute resolution, and low-resolution features are improved to a unified resolution through super-resolution methods; medium- and long-term forecasts use time-scale coding, statistical values ​​of meteorological characteristics and economic indicators, etc., which can flexibly adapt to the prediction needs of different time scales. The super-resolution method is used for low-resolution features, which effectively improves the resolution of the features, making the model more consistent when processing data of different time scales, thereby improving the accuracy of the overall prediction, especially when dealing with short-term fluctuations and long-term trends, the model can better capture key information. Through targeted feature processing and encoding, the feature redundancy of the model is reduced, making the model structure more concise. The statistical value representation methods such as average, maximum and minimum values ​​used in medium- and long-term load forecasting simplifies the feature dimension while ensuring the accuracy of the prediction, which helps the model to run efficiently and converge quickly.

[0036] As a preferred technical means: Step 200) specifically includes the following steps:

[0037] 210) In the feature dimension, based on the feature attention mechanism, query, key, and value mapping matrices are constructed respectively to calculate the correlation between each influencing factor and the load to be predicted, dynamically allocate attention weights according to the correlation strength, and extract key load correlation features;

[0038] 220) In the time series dimension, based on the time series attention mechanism, a query, key, and value mapping matrix is ​​constructed to calculate the load correlation between different time steps, and the attention weight is dynamically allocated according to the correlation strength to extract the load time series autocorrelation features;

[0039] 230) The autocorrelation features and cross-correlation features extracted above are input into the load probability prediction model to achieve the probability prediction of power load.

[0040] In this technical solution, step 200) can effectively extract key features related to power load forecasting by introducing feature attention mechanism and time series attention mechanism. The feature attention mechanism dynamically adjusts the weights of various influencing factors in the feature dimension to ensure that the model can capture the key features that have the greatest impact on load forecasting. The time series attention mechanism captures the load correlation between different time steps in the time series dimension and extracts the time series autocorrelation features of the load, so that the model has stronger explanatory power in the time dimension. Using the attention mechanism, the model can dynamically allocate weights according to the correlation strength between different features and time series, thereby focusing on features and time steps that are highly related to load forecasting. The dynamic weight allocation method avoids the limitation of traditional models that treat all input features equally, improves the sensitivity of the model to complex feature relationships, and thus improves the prediction accuracy. Through the improved quantile regression model, combined with the extracted autocorrelation features and cross-correlation features, a more refined power load probability prediction can be achieved. The improved quantile regression model can be trained more stably when the prediction error is close to 0, avoiding gradient oscillation and ensuring the convergence of the model and the stability of the prediction results. This solution can not only meet the needs of power load forecasting in ultra-short-term and short-term time scales, but also cope with long-span load forecasting problems by dynamically adjusting feature weights. The adaptability of multiple time scales enables the model to have high prediction accuracy at different time scales, meeting the diverse needs in practical applications. The combination of feature attention mechanism and time series attention mechanism enables the model to have a deeper understanding of the time series autocorrelation and feature intercorrelation of load data, thereby improving the prediction accuracy; at the same time, the improved quantile regression model can provide more reliable prediction results through probabilistic prediction, enhancing the credibility of power load forecasting.

[0041] As a preferred technical means: the load cross-correlation feature extraction process based on the attention mechanism is:

[0042]

[0043]

[0044] Where: q F,i is the cross-correlation query matrix i Item element; k F,i The mutual key matrix i Item element; v F,i is the cross-correlation matrix i Item element; F Q , F K , F V They are the query, key, and value mapping matrices of the feature attention mechanism respectively; x i For the i Influencing factors; load,i For the i The attention score of item factors on load; After normalization i The attention score of item factors on load; h t is the reconstructed feature vector obtained by weighting the feature dimension by attention;

[0045] The process of extracting load autocorrelation features based on the attention mechanism is:

[0046]

[0047]

[0048]

[0049] Where: q T,t is the autocorrelation query matrix t Item element; k T,t is the self-correlated key matrix t Item element; v T,t is the autocorrelation matrix t Item element; T Q , T K , T V They are the query, key, and value mapping matrices of the temporal attention mechanism respectively; for t 1 moment t 2. Attention score at the moment; is the corresponding t 1 element at the moment; is the corresponding t 2 Elements of the moment; After normalization t 1 moment t 2. Attention score at the moment; is the reconstructed feature vector obtained by attention weighting in the temporal dimension; Corresponding to the autocorrelation value matrix t 2 Elements of the moment.

[0050] Through the feature attention mechanism, the model can identify and dynamically assign weights between influencing factors, thereby extracting the inter-correlation features between different influencing factors. This technical solution can accurately capture the key factors in load forecasting and their mutual influence relationships, so that the model has a better understanding of the complex relationship between multi-dimensional features and improves the accuracy of prediction. The time series attention mechanism can effectively capture the autocorrelation of power load in time series by dynamically assigning weights at different moments. In load forecasting, the model focuses on the data points in historical moments that are most relevant to the current moment, thereby better reflecting the law of load changes over time and improving the prediction performance in the time series dimension. Whether in the feature dimension or the time series dimension, the attention mechanism can dynamically adjust the weight of the model according to the correlation strength between the feature and the time step. The dynamic adjustment enables the model to more flexibly deal with nonlinear relationships and time-varying features in the data, thereby improving the adaptability and robustness of the model. The attention mechanism reconstructs the feature vector by weighted reconstruction, so that the final output feature vector not only contains the information of the original feature, but also contains the correlation information with other features. It not only improves the model's understanding of complex feature relationships, but also enhances the interpretability of the prediction results, making it easier to analyze which factors play a key role in load forecasting. The feature reconstruction vector extracted by the attention mechanism can better reflect the key factors and time patterns in power load forecasting, improve the generalization ability of the model, and enable it to maintain high prediction accuracy and stability when facing different types of power load data. The attention mechanism can flexibly handle data from different sources and types, so that the model can still effectively extract key features and make accurate predictions when facing multi-source heterogeneous data.

[0051] As a preferred technical means: in step 230), based on the extracted power load autocorrelation characteristics and cross-correlation characteristics, the mapping of load autocorrelation characteristics and cross-correlation characteristics to load forecast quantile values ​​is realized through a quantile regression feedforward network, and the loss is minimized through a modified PL function, specifically including the following steps:

[0052] 231) Based on the autocorrelation features and cross-correlation features extracted by the dual attention mechanism, a quantile regression feedforward network is constructed and mapped to the corresponding load forecast quantile values;

[0053] 232) The pinball loss (PL) is used as the comprehensive evaluation index of the quantile regression model. The PL function is transformed based on the Huber function. The load probability prediction model is constructed with the goal of minimizing PL.

[0054] 233) The weights and bias parameters of the quantile regression feedforward network are trained by the gradient descent algorithm to make the model converge;

[0055] 234) According to the load forecast values ​​at different quantile levels α, the load center output forecast interval at the corresponding confidence level is obtained;

[0056] 235) Based on the prediction interval coverage deviation and the prediction interval standard width as evaluation indicators of the load probability prediction model, the smaller the prediction interval coverage deviation and the smaller the width, the higher the comprehensive performance of the load prediction model; when the evaluation indicator reaches the set threshold, it is considered that the final load probability prediction model is formed; the final prediction model is used for load probability prediction.

[0057] Through the quantile regression feedforward network, different quantiles of load forecasting are accurately modeled and mapped. Compared with the traditional point prediction method, quantile regression can simultaneously predict load values ​​at multiple confidence levels, provide more comprehensive load forecasting information for the power system, and improve the accuracy and reliability of the prediction. The modified pinball loss (PL) function is used as the evaluation index of the quantile regression model, and combined with the Huber function for optimization, which can effectively reduce the prediction error of the model and enhance the robustness. The design of the loss function makes the model more robust when dealing with outliers and reduces the risk of overfitting. The weights and bias parameters of the quantile regression feedforward network are trained by the gradient descent algorithm to ensure rapid convergence of the model. The gradient descent algorithm gradually reduces the model error during the optimization process, so that the model can effectively learn the complex relationship between the autocorrelation characteristics and the cross-correlation characteristics of the load, thereby improving the prediction accuracy. Through the prediction of different quantile levels, the load prediction interval at the corresponding confidence level is generated, which provides a more reliable decision-making basis for power dispatchers, can help better deal with the uncertainty in load forecasting, and provide support for risk management. The prediction interval coverage deviation and prediction interval standard width are used as evaluation indicators of the model, so that the model can achieve the best comprehensive performance. Through the quantile regression feedforward network, improved loss function and dynamic prediction interval generation, not only the accuracy and robustness of load forecasting are improved, but also a more reliable basis for decision-making in the power system is provided, which makes this method have significant advantages in practical applications.

[0058] As the preferred technical means: the load probability prediction model is:

[0059]

[0060] Where: is the quantile PL function below; T is the number of samples; n is the quantile number; QR , b QR are the weights and bias parameters of the quantile regression feedforward network; is the quantile Down t Forecasted load at the time;

[0061] The weights and bias parameters of the quantile regression feedforward network are trained using the gradient descent algorithm;

[0062] The PL function is modified based on the Huber function, and the modified PL function is:

[0063]

[0064] Where: d is the quadratic smoothing bandwidth hyperparameter;

[0065] According to different quantile levels α The load forecast value can obtain the load center output forecast interval under the corresponding confidence level I 1-α ( t ), the specific expression is:

[0066]

[0067] Where: 1- α is the prediction confidence level; 1- α Forecast confidence level t The output prediction range of distributed photovoltaic at any moment; , are the upper and lower quantile levels of the corresponding central prediction interval, respectively; is the quantile Down t Forecasted load at the time;

[0068] Load probability forecasting using prediction interval coverage deviation e cov,α and the standard width of the prediction interval oh cov,α As evaluation indicators:

[0069]

[0070]

[0071]

[0072]

[0073] Where: The confidence level is 1- α The width of the prediction interval for s cov,α is the prediction interval coverage; is the indicative function, when E Its value is 1 if it is true, otherwise it is 0; d α The score is used as a comprehensive performance evaluation indicator.

[0074] The quantile regression feedforward network is used for load forecasting, so that the model can calculate different quantile values ​​of load forecasting and provide forecast results at multiple confidence levels, which helps to model the uncertainty of load and provide more comprehensive forecasting information. By modifying the pinball loss (PL) function based on the Huber function, the loss function is more robust in dealing with outliers in load forecasting. The modified PL function has a smooth gradient change, which avoids the oscillation phenomenon when the load forecast error is close to 0, thereby improving the stability and convergence speed of model training. According to different quantile levels α, the load center output forecast interval at the corresponding confidence level is generated, providing the power system with forecast intervals with different confidence levels, so that decision makers can make appropriate decisions under different risk scenarios. The prediction interval coverage deviation and the prediction interval standard width are used as evaluation indicators to comprehensively evaluate the performance of the load forecasting model. The accuracy and reliability of the prediction interval are quantified by these two indicators, thereby optimizing the performance of the model. The modified PL function improves the robustness of the model in dealing with high volatility and extreme situations in load forecasting, avoids the gradient oscillation problem that may occur in the traditional PL function, and the model can perform load forecasting more stably through the improved loss function and training algorithm. By comprehensively using the improved PL function, quantile regression feedforward network and a variety of evaluation indicators, the comprehensive performance of the load forecasting model can be effectively optimized, making the load forecasting more accurate and practical.

[0075] Beneficial effects: This technical solution proposes a load probability forecasting method based on dual attention mechanism and quantile regression. This technical solution comprehensively considers the time series autocorrelation, feature cross-correlation and random uncertainty of power load, can effectively mine the correlation of power load in different dimensions, and use quantile regression to accurately characterize the uncertainty of load fluctuation, ensuring the high reliability and accuracy of the prediction results. The specific beneficial effects are as follows:

[0076] The present invention firstly conducts standardized modeling of the load forecasting problem, fully considers the multi-dimensional correlation characteristics of the power load, and constructs a multi-time scale load forecasting framework based on a sliding window. By introducing multi-source heterogeneous data, the power load rolling forecast at different time scales is realized, which can dynamically respond to changes in the power load.

[0077] This paper proposes a power load feature extraction method based on a dual attention mechanism, which can deeply explore the autocorrelation of power load in the time series dimension and the cross-correlation in the feature dimension. By dynamically allocating attention weights, the model can capture key features more accurately and improve the accuracy of load forecasting.

[0078] The present invention improves the convergence performance of the model in probability prediction by improving quantile regression. The improved quantile regression can better adapt to the complex fluctuations of power load, provide prediction results at multiple confidence levels, and effectively improve the reliability and accuracy of the prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a multi-time-scale load forecasting framework based on sliding window.

[0080] Figure 2 Schematic diagram of power load cross-correlation feature extraction based on attention mechanism.

[0081] Figure 3 It is a probabilistic forecasting framework for power load at multiple time scales.

[0082] Figure 4 It is the result of ultra-short-term power load probability forecast. DETAILED DESCRIPTION

[0083] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0084] The present invention proposes a multi-time scale load probability prediction method based on dual attention mechanism and quantile regression, comprising the following steps:

[0085] Step 1: Standardize the modeling of multi-time scale load forecasting problem, consider the autocorrelation characteristics of load and the cross-correlation characteristics with external influencing factors, and build a multi-time scale load forecasting framework based on sliding window;

[0086] Step 2: Based on the dual attention mechanism, the autocorrelation characteristics of the power load in the time series dimension and the cross-correlation characteristics with its influencing factors in the feature dimension are extracted, and a quantile regression feedforward network is constructed to map the autocorrelation characteristics and cross-correlation characteristics to the load prediction quantile values. A load probability prediction model is constructed based on the sliding window prediction framework and the extracted multidimensional features; the constructed load probability prediction model is used for load probability prediction.

[0087] In the above technical solution, step 1 performs standardized modeling on the multi-time scale load forecasting problem, considers the multi-dimensional correlation characteristics of the power load, and constructs a multi-time scale load forecasting framework based on a sliding window. The specific method is as follows:

[0088] Since the power load is affected by the autocorrelation characteristics and multiple external factors, the input data set of the multi-time scale load forecasting model needs to include multi-source heterogeneous data such as load data, meteorological data, and socio-economic data. Considering the coupling relationship of the above factors, a multi-time scale load forecasting data set is constructed as shown in formula (1).

[0089] (1)

[0090] (2)

[0091] Where: t for t Multi-dimensional load characteristic vector at each moment; for t Moment auxiliary feature vector; for t The load of statistical caliber in different time dimensions at all times; N is the number of load samples; D is the characteristic dimension; for t Time stamp encoding vector; for t Meteorological characteristic vector at the time, including temperature, wind speed, and precipitation; for t Economic statistics indicators at all times; Code for holidays.

[0092] The electricity consumption patterns of various types of power users on special holidays are significantly different from those on regular days. Therefore, various types of holidays are encoded to distinguish various types of electricity consumption days to consider the impact of holidays on power load. In addition, multi-time scale load changes show certain periodicity, including monthly periodicity, weekly periodicity and daily periodicity, that is, the loads on the same day of different months, the loads on the same working day of different weeks, and the loads at the same time of different days have certain similarities. In order to characterize the above characteristics of power loads under different time scales, the time scales at different times are encoded. Common feature encoding methods use non-continuous periodic cyclic sequences to represent time scales. For example, 24-hour time scales are represented by integers 0, 1, ..., 23, but this encoding method is discontinuous, and the model cannot accurately learn and reflect the real distance of the load time scale (the distance from 0 to 1 is not equal to the distance from 23 to 0 the next day). Therefore, the present invention proposes a sine and cosine encoding method to encode the time scale sine and cosine of each time scale. The specific expression is shown in formula (3).

[0093] (3)

[0094] Where: M sin, t andM cos, t are the sine and cosine coded values ​​of the load monthly time scale respectively; D sin, t and D cos, t are the sine and cosine encoding values ​​of the load day time scale respectively; sin, t and cos, t are the sine and cosine encoding values ​​of the load working day time scale respectively; H sin, t and H cos, t are the sine and cosine encoding values ​​at the load moment, respectively. Sine and cosine encoding is used to achieve unique mapping of time scales in various time dimensions.

[0095] The fluctuation of power load is the result of the joint action of multi-dimensional characteristics. On the one hand, the power load has certain autocorrelation characteristics with its historical load changes, including the random characteristics of short-time scale load changes, the periodic characteristics of medium-time scale load changes, and the trend characteristics of long-time scale load changes; on the other hand, the power load has certain cross-correlation characteristics with external meteorological and economic factors. Therefore, multi-time scale load forecasting needs to comprehensively consider the correlation characteristics of the above two aspects. The multi-time scale load forecasting model considering multi-dimensional characteristics can be expressed by formula (4):

[0096] (4)

[0097] (5)

[0098] Where: is the input historical load characteristic vector, K bac is the length of the input historical load characteristic vector; A time stamp code for the load timestamp to be predicted; is the weather forecast feature vector at the time to be predicted; It is the economic forecast statistical indicator at the time to be predicted; Encode special holidays at the time to be predicted; K step is the prediction step length, where K step = 1 epoch single-step prediction; K ahe is the prediction step-ahead; are the parameters of the multi-time scale load forecasting model; Represents the mapping from the above input features to load forecast values; Indicates the predicted value of the corresponding feature quantity.

[0099] In order to build a multi-time scale load rolling forecasting model considering multi-dimensional characteristics, the original data set is processed by sliding window. The schematic diagram of multi-dimensional data sliding window processing is shown in Figure 1 As shown. The load history autocorrelation characteristics and load synchronization correlation characteristics in the sliding window are used as the input characteristics of the load forecasting model. The autocorrelation of load changes and the mutual correlation between load and its influencing factors are comprehensively considered. The load forecasting model is used to predict the load as the sliding window moves. The above modeling method and feature engineering method can be extended to load forecasting problems of various time scales. For ultra-short-term and short-term load forecasting (i.e., load forecasting for the next 15 minutes to 24 hours and weekly load forecasting), the resolution of each feature is 15 minutes. Considering that the resolution of some features is low (for example, the resolution of holiday features is daily), in order to ensure the consistency of data resolution, the resolution of low-resolution features is unified by super-resolution method; for medium- and long-term load forecasting (including weekly load forecasting, monthly maximum load forecasting, monthly power forecasting, etc.), the time scale features are encoded by the time scale of the corresponding time dimension, the meteorological features are represented by the average, maximum and minimum values ​​of the meteorological elements under the corresponding time dimension, the economic factors are represented by the statistical indicators under the corresponding time dimension, and the holiday factors are represented by the number of holidays and the types of holidays contained under the corresponding time dimension.

[0100] Furthermore, step 2 extracts the autocorrelation characteristics of the power load in the time series dimension and the cross-correlation characteristics with its influencing factors in the feature dimension based on the dual attention mechanism, constructs a quantile regression feedforward network to map the autocorrelation characteristics and cross-correlation characteristics to the load forecast quantile value, and constructs a load probability forecasting model based on the sliding window prediction framework and the extracted multidimensional features. The specific method is as follows:

[0101] The load cross-correlation feature extraction process based on the attention mechanism is shown in Equations (6) to (8).

[0102] (6)

[0103] (7)

[0104] (8)

[0105] Where: q F,i is the cross-correlation query matrix i Item element; k F,i The mutual key matrix i Item element; v F,i is the cross-correlation matrix i Item element; F Q , F K , F V They are the query, key, and value mapping matrices of the feature attention mechanism respectively; x i For the i Influencing factors; load,i For the i The attention score of item factors on load; After normalization i The attention score of item factors on load; h t is the reconstructed feature vector obtained by attention weighting of the feature dimension.

[0106] Introducing the attention mechanism in the feature dimension can explore the correlation between each influencing factor and the load to be predicted, dynamically allocate attention weights according to the correlation strength, and extract key load correlation features. The schematic diagram of load correlation feature extraction based on the attention mechanism is shown in the figure. Figure 2 shown.

[0107] In the time series dimension, the power load and its historical load changes have certain autocorrelation characteristics, including the randomness, periodicity and trend of load fluctuations. The historical load sequence has different correlations with the load at the time to be predicted. The load correlations at similar times and on the same day type are strong, while the load correlations at far distances and on different day types are weak. Introducing the attention mechanism in the time series dimension can mine the correlation between the load characteristics of different time steps and the load at the time step to be predicted, dynamically allocate attention weights according to the correlation strength, and extract the load time series autocorrelation characteristics. The process of extracting load autocorrelation features based on the attention mechanism is shown in Equations (9) to (11).

[0108] (9)

[0109] (10)

[0110] (11)

[0111] Where: q T,t is the autocorrelation query matrix t Item element; k T,t is the self-correlated key matrix t Item element; v T,t is the autocorrelation matrix t Item element; T Q , T K , T V They are the query, key, and value mapping matrices of the temporal attention mechanism respectively; for t 1 moment t 2. Attention score at the moment; is the corresponding t 1 element at the moment; is the corresponding t 2 Elements of the moment; After normalization t 1 moment t 2. Attention score at the moment; is the reconstructed feature vector obtained by attention weighting in the temporal dimension; Corresponding to the autocorrelation value matrix t 2 Elements of the moment.

[0112] Based on the extraction of power load autocorrelation features and cross-correlation features based on the dual attention mechanism, the quantile regression feedforward network is used to map the load autocorrelation features and cross-correlation features to the load forecast quantile values. Figure 3 The multi-time scale load probability prediction architecture based on dual attention mechanism and quantile regression is shown in Figure 1. Since quantile prediction generally uses pinball loss (PL) as a comprehensive evaluation index to measure the overall performance of the probability prediction model, the proposed load probability prediction method based on quantile regression aims to minimize PL and construct a multi-time scale load probability prediction model as shown in formula (12).

[0113] (12)

[0114] Where: is the quantile PL function below; T is the number of samples; n is the quantile number; QR , b QR are the weights and bias parameters of the quantile regression feedforward network; is the quantile Down t Forecasted load at the time.

[0115] The weight and bias parameters of the quantile regression feedforward network are obtained through gradient descent algorithm training, as shown in formula (12). When the load forecast error drops to near 0, the PL piecewise function is not differentiable, and the gradient is prone to oscillation, resulting in poor convergence of the model training. Therefore, the PL function is modified based on the Huber function, and the modified PL function is shown in formula (13).

[0116] (13)

[0117] Where: d is the quadratic smoothing bandwidth hyperparameter.

[0118] The modified PL function according to formula (13) retains an asymmetric structure similar to the original PL function and is differentiable at both the 0 point and the segmentation point. When the error is close to 0, the gradient change is smoother and more stable. The load probability prediction model can be trained and converged based on the gradient descent algorithm.

[0119] According to different quantile levels α The load forecast value can obtain the load center output forecast interval under the corresponding confidence level I 1-α ( t ), the specific expression is shown in formula (14).

[0120] (14)

[0121] Where: 1- α is the prediction confidence level; 1- α Forecast confidence level t The output prediction range of distributed photovoltaic at any moment; , are the upper and lower quantile levels of the corresponding central prediction interval, respectively. For example, when α =0.05, , They are 0.975 and 0.025 respectively; is the quantile Down t Forecasted load at the time;

[0122] The load probability forecast often uses the prediction interval coverage deviation e cov,α and the standard width of the prediction interval oh cov,α As evaluation indicators:

[0123] (15)

[0124] (16)

[0125] (17)

[0126] (18)

[0127] Where: The confidence level is 1- α The width of the prediction interval for s cov,α is the prediction interval coverage; is the indicative function, when E Its value is 1 if true, otherwise 0. d α The score is used as a comprehensive performance evaluation indicator. The smaller the prediction interval coverage deviation is, the smaller the prediction interval width is. d α The higher the score, the higher the overall performance of the load forecasting model.

[0128] In order to verify the effectiveness of the proposed method, the load data, meteorological data and economic data of a certain area in Zhejiang Province in the past three years are used as the original data set. The economic data include regional GDP and industrial enterprises above designated size. The meteorological data include temperature, humidity, precipitation and weather type. All simulation experiments in this section are implemented based on Python3.8 and Pytorch framework, and the network model training is completed on Nvidia GeForce RTX 2080Ti GPU.

[0129] Ultra-short-term load forecasting (i.e., K step =1, K ahe =1) In the simulation experiment, the central probability interval prediction results with confidence levels of 0.95, 0.8, and 0.6 are obtained from the quantile prediction values ​​with quantiles of 0.025, 0.1, 0.2, 0.8, 0.9, and 0.975. The evaluation indicators of the prediction results are shown in Table 1.

[0130] Table 1 Evaluation results of ultra-short-term power load probability interval forecast

[0131]

[0132] It can be seen from Table 1 that when the prediction confidence level is 0.95, the prediction interval coverage of the probability interval prediction result is 96.2%, which is close to the nominal coverage of 95%. At other prediction confidence levels, the prediction interval coverage is close to the corresponding nominal coverage, which shows that the photovoltaic prediction interval obtained by the proposed method has a high prediction reliability. In addition, it can be seen from Table 1 that as the confidence level increases, the width of the prediction interval also increases accordingly. The prediction interval width when the confidence level is 0.8 is 0.17, which is smaller than the prediction interval when the confidence level is 0.9. The uncertainty is reduced but the prediction accuracy is reduced. The rolling prediction curves of power load at different confidence levels are as follows: Figure 4 shown.

[0133] Depend on Figure 4 It can be seen that the actual power load curve is basically surrounded by the prediction intervals at different confidence levels, and only a small number of load values ​​fall outside the prediction interval. This shows that the load probability prediction method based on the dual attention mechanism and quantile regression has high reliability and can accurately predict the change trend of power load. The proposed method can provide power load prediction intervals at different confidence levels. The photovoltaic output prediction interval with a higher confidence level is wider. As the confidence level increases, the uncertainty of the prediction results also increases. The power load prediction intervals at different confidence levels obtained based on the proposed method can provide reference and reference for power grid operators to formulate operation strategies at different risk levels.

[0134] The formulation of system operation scheduling plans at different time scales requires the power load forecast results at different forecast time scales as a basis. The load probability forecasting model based on the dual attention mechanism and quantile regression is used for short-term load forecasting ( K step =7×96, K ahe =96) simulation experiments were performed, and compared with the prediction models based on quantile regression neural network (QRNN), quantile regression convolutional neural network (QRCNN), quantile regression long short-term memory neural network (QRLSTM), and quantile regression convolutional neural network-long short-term memory (QRCNN-LSTM). The evaluation indicators of the prediction results at different confidence levels are shown in Table 2.

[0135] Table 2 Evaluation results of power load probability interval prediction

[0136]

[0137] As shown in Table 2, the power load forecasting effect based on QRNN and QRCNN is poor, the prediction interval coverage deviation is large, and the prediction reliability and accuracy are low. The power load forecasting model based on QRLSTM has a larger prediction interval width and lower prediction accuracy, but the prediction interval coverage deviation is small and the prediction reliability is high. The power load probability forecasting effect based on QRCNN-LSTM is better. While ensuring a narrow prediction interval, it achieves a higher prediction interval coverage rate and has a higher comprehensive score. Considering the reliability and accuracy of the power load probability forecasting model, the proposed method is d α The score is the highest, the prediction interval width at the 0.95 confidence level is 0.32, the accuracy is high, the prediction interval coverage deviation is 1.9%, and the reliability is high. d α The score reached 107.8, which is higher than other methods and has the best prediction effect. Compared with other methods, the power load forecasting method based on dual attention mechanism and quantile regression comprehensively considers the dual characteristics of autocorrelation and cross-correlation of power load, so the prediction effect is better. Compared with ultra-short-term power load forecasting, the various indicators of short-term power load forecasting have been reduced, mainly because ultra-short-term load changes have a certain degree of continuity and a small prediction step, while short-term load changes are more random, with more prediction steps and higher prediction difficulty.

[0138] The load probability prediction method based on dual attention and quantile regression shown above is a specific embodiment of the present invention, which has embodied the essential characteristics and progress of the present invention. It can be modified equivalently according to actual use needs under the guidance of the present invention, and all of them are within the protection scope of this scheme.

Claims

1. Load probability forecasting method based on dual attention and quantile regression, characterized in that: The following steps are involved: 100) Standardize and model multi-time scale load data, consider the autocorrelation characteristics of load and the cross-correlation characteristics with external influencing factors, and build a load probability forecasting framework based on sliding windows; 200) Based on the dual attention mechanism, the autocorrelation characteristics of power load in the time series dimension and the cross-correlation characteristics with its influencing factors in the feature dimension are extracted, and a quantile regression feedforward network is constructed to map the autocorrelation characteristics and cross-correlation characteristics to the load forecast quantile value, and a load probability forecasting model is constructed based on the sliding window prediction framework and the extracted multi-dimensional features; The constructed load probability prediction model is used to perform load probability prediction; Construct a load probability forecasting model, including: The multi-dimensional data is processed into a sliding window, and the load history autocorrelation characteristics and load synchronization correlation characteristics within the sliding window are used as model input characteristics; As the sliding window moves, the power load is predicted in a rolling manner, taking into account the autocorrelation of load changes and the cross-correlation between the load and its influencing factors; For ultra-short-term and short-term load forecasting, the resolution of each feature is 15 minutes. For low-resolution features, the super-resolution method is used to unify the resolution; For medium- and long-term load forecasting, time-scale features are encoded using the time-scale of the corresponding time dimension. Meteorological features are represented by average, maximum, and minimum values. Economic factors are represented by statistical indicators of the corresponding time dimension. Holiday factors are represented by holiday days and holiday types. Step 200) specifically includes the following steps: 210) In the feature dimension, based on the feature attention mechanism, query, key, and value mapping matrices are constructed respectively to calculate the correlation between each influencing factor and the load to be predicted, dynamically allocate attention weights according to the correlation strength, and extract key load correlation features; 220) In the time series dimension, based on the time series attention mechanism, a query, key, and value mapping matrix is ​​constructed to calculate the load correlation between different time steps, and the attention weight is dynamically allocated according to the correlation strength to extract the load time series autocorrelation features; 230) Input the autocorrelation features and cross-correlation features extracted above into the load probability prediction model to achieve the probability prediction of power load; In step 230), based on the extracted autocorrelation characteristics and cross-correlation characteristics of the power load, the mapping of the load autocorrelation characteristics and cross-correlation characteristics to the load forecast quantile value is realized through the quantile regression feedforward network, and the loss is minimized through the modified PL function, which specifically includes the following steps: 231) Based on the autocorrelation features and cross-correlation features extracted by the dual attention mechanism, a quantile regression feedforward network is constructed and mapped to the corresponding load forecast quantile values; 232) The pinball loss PL is used as the comprehensive evaluation index of the quantile regression model, the PL function is transformed based on the Huber function, and the load probability prediction model is constructed with the goal of minimizing PL; 233) The weights and bias parameters of the quantile regression feedforward network are trained by the gradient descent algorithm to make the model converge; 234) According to the load forecast values ​​at different quantile levels α, the load center output forecast interval at the corresponding confidence level is obtained; 235) Based on the prediction interval coverage deviation and the prediction interval standard width as the evaluation index of the load probability forecasting model, the smaller the prediction interval coverage deviation and the smaller the width, the higher the comprehensive performance of the load forecasting model; when the evaluation index reaches the set threshold, it is considered that the final multi-time scale load probability forecasting model is formed; the final multi-time scale load probability forecasting model is used for load probability forecasting; The load probability forecasting model is: Where: is the quantile PL function below; T is the number of samples; n is the quantile number; QR , b QR are the weights and bias parameters of the quantile regression feedforward network; is the quantile Down t Forecasted load at the time; The weights and bias parameters of the quantile regression feedforward network are trained using the gradient descent algorithm; The PL function is modified based on the Huber function, and the modified PL function is: Where: δ is the quadratic smoothing bandwidth hyperparameter; According to different quantile levels α The load forecast value is used to obtain the load center output forecast interval under the corresponding confidence level. I 1-α ( t ), the specific expression is: Where: 1- α is the prediction confidence level; 1- α Forecast confidence level t The output prediction range of distributed photovoltaic at any moment; , are the upper and lower quantile levels of the corresponding central prediction interval, respectively; is the quantile Down t Forecasted load at the time; Load probability forecasting using prediction interval coverage deviation ε cov,α and the standard width of the prediction interval ω cov,α As evaluation indicators: Where: The confidence level is 1- α The width of the prediction interval for σ cov,α is the prediction interval coverage; is the indicative function, when E Its value is 1 if it is true, otherwise it is 0; δ α The score is used as a comprehensive performance evaluation indicator.

2. The load probability forecasting method based on dual attention and quantile regression according to claim 1 is characterized in that: The standardized modeling in step 100) includes the following steps: 110) Incorporate multi-source heterogeneous data including load data, meteorological data, and socio-economic data into the multi-time scale load forecasting dataset to comprehensively consider the coupling relationship between autocorrelation characteristics and external factors; 120) Encode various holidays and distinguish different types of electricity consumption days to consider the impact of holidays on power load; 130) Use sine and cosine coding to uniquely map the monthly, daily, working day and moment time scales of the load, and characterize the autocorrelation characteristics of the power load at different time scales.

3. The load probability forecasting method based on dual attention and quantile regression according to claim 2 is characterized in that: In step 110), the multi-time scale load forecasting dataset is represented as: in, Where: t for t Multi-dimensional load characteristic vector at each moment; for t Moment auxiliary feature vector; for t The load of statistical caliber in different time dimensions at all times; N is the number of load samples; D is the characteristic dimension; for t Time stamp encoding vector; for t Meteorological characteristic vector at the moment, including temperature, wind speed, and precipitation; for t Economic statistics indicators at all times; Code for holidays.

4. The load probability forecasting method based on dual attention and quantile regression according to claim 3 is characterized in that: Step 130) uses the sine and cosine encoding method to encode the time scale sine and cosine of each time scale, and the expression is: Where: M sin, t and M cos, t are the sine and cosine coded values ​​of the load monthly time scale respectively; D sin, t and D cos, t are the sine and cosine encoding values ​​of the load day time scale respectively; W sin, t and W cos, t are the sine and cosine encoding values ​​of the load working day time scale respectively; H sin, t and H cos, t They are the sine and cosine encoding values ​​at the load moment respectively; the unique mapping of time scales in various time dimensions is achieved through sine and cosine encoding.

5. The load probability forecasting method based on dual attention and quantile regression according to claim 4 is characterized in that: The expression for constructing the load probability forecasting framework based on sliding window is: in, Where: is the input historical load characteristic vector, K bac is the length of the input historical load characteristic vector; A time stamp code for the load timestamp to be predicted; is the weather forecast feature vector at the time to be predicted; It is the economic forecast statistical indicator at the time to be predicted; Code special holidays at the time to be predicted; K step is the prediction step length, where K step =1 epoch single-step prediction; K ahe is the prediction step-ahead; are the parameters of the multi-time scale load forecasting model; Represents the mapping from the above input features to load forecast values; Indicates the predicted value of the corresponding feature quantity.

6. The load probability forecasting method based on dual attention and quantile regression according to claim 5 is characterized in that: The load cross-correlation feature extraction process based on the attention mechanism is: Where: q F,i is the cross-correlation query matrix i Item element; k F,i The mutual key matrix i Item element; v F,i is the cross-correlation matrix i Item element; F Q , F K , F V They are the query, key, and value mapping matrices of the feature attention mechanism respectively; x i For the i Items of influencing factors; load,i For the i The attention score of item factors on load; After normalization i The attention score of item factors on load; h t is the reconstructed feature vector obtained by weighting the feature dimension by attention; The process of extracting load autocorrelation features based on the attention mechanism is: Where: q T,t is the autocorrelation query matrix t Item element; k T,t is the self-correlated key matrix t Item element; v T,t is the autocorrelation matrix t Item element; T Q , T K , T V They are the query, key, and value mapping matrices of the temporal attention mechanism respectively; for t 1 moment t 2. Attention score at the moment; is the corresponding t 1 element at the moment; is the corresponding matrix in the self-correlation key matrix t 2 Elements of the moment; After normalization t 1 moment t 2. Attention score at the moment; is the reconstructed feature vector obtained by attention weighting in the temporal dimension; Corresponding to the autocorrelation value matrix t 2 Elements of the moment.

Images