A mid-term hourly load probability forecasting method based on time series fusion Transformer model

Abstract

A medium-term hourly load probability forecasting method based on a time series fusion Transformer model includes collecting load values ​​and related influencing factor data for multiple historical sample days in a certain area; classifying the input data and reconstructing the original load data into a panel data matrix to shorten the length of the time series, thereby reducing the complexity of the model; normalizing the input data and dividing the data set into a training set, a validation set, and a test set; setting model hyperparameters; establishing a time series fusion Transformer model; iteratively training the model based on the training set to determine the optimal weight and bias parameter set of the model; optimizing the optimal hyperparameters of the model based on the validation set, and retraining the prediction model based on the optimal hyperparameters; and verifying the model performance on the test set based on the trained prediction model. The probabilistic load forecasting method proposed in the present invention can provide power grid operators with more accurate and detailed load forecasting information in medium-term power optimization scheduling and market transactions.

Description

Technical Field

[0001] The present invention belongs to the technical field of power systems, and in particular to power load forecasting technology, and specifically to a medium-term hourly load probability forecasting method based on a time series fusion Transformer model. Background Art

[0002] With the modernization of society and economic development, people's lives are becoming increasingly dependent on electricity. Accurate load forecasting helps improve the utilization of power equipment and minimize energy waste. It also enables dispatchers to formulate appropriate grid operation and maintenance plans to reduce generation costs and promote power supply. Medium-term load forecasts, with a forecast range of one week to one year, provide essential reference for formulating future development plans, guiding power production, and mitigating market risks.

[0003] Currently, there are several challenges in medium-term load forecasting:

[0004] 1) With the increase of distributed renewable energy and the implementation of demand-side response, the uncertainty of renewable energy generation and the differences in individual user response behavior increase the complexity and uncertainty of load forecasting;

[0005] 2) As the forecast horizon lengthens, the available information decreases dramatically;

[0006] 3) The demand for medium-term load forecasting in power system planning is increasing, requiring longer forecast horizons and finer temporal resolution.

[0007] Current research on medium-term load forecasting primarily focuses on point forecasts, whereas probabilistic forecasting can provide more information to quantify the uncertainty of future loads. Current probabilistic load forecasting research primarily utilizes methods such as interval estimation, kernel density estimation, and quantile regression. Interval estimation involves predicting the upper and lower bounds of an interval given a certain confidence level. This method does not fully describe the probability distribution of the forecast results. Kernel density estimation can estimate the probability density function of the forecast value, but it relies on a large number of samples and is difficult to select the bandwidth. Quantile regression, on the other hand, directly explains the relationship between the response variable and the dependent variable at different quantiles and has become a hot topic in the literature on probabilistic load forecasting in recent years. However, the phenomenon of crossover between quantile predictions in traditional quantile regression can lead to irrational forecast results.

[0008] For example, Liu Xinzhi et al. proposed a medium-term load forecasting model based on an Attention-ResNet-LSTM network. This model introduces the attention mechanism into the LSTM model, assigns different weights to feature vectors, and then adds a ResNet network to calculate daily load forecasts. However, its temporal resolution is relatively coarse, and its results are presented in a deterministic form, which cannot quantify the uncertainty of load forecasts.

[0009] Therefore, the present invention proposes a medium-term hourly load probability forecasting method based on a time series fusion Transformer model. Summary of the Invention

[0010] The present invention mainly addresses the problems of coarse time resolution and load uncertainty quantification in the above-mentioned existing medium-term probabilistic load forecasting methods, and proposes a medium-term hourly load probability forecasting method based on the time series fusion Transformer model. The present invention refines the time resolution to hourly level and below, and improves the quantile crossing problem in the quantile probability forecasting method to better quantify the uncertainty of the load.

[0011] A medium-term hourly load probability forecasting method based on a time series fusion Transformer model includes the following steps:

[0012] Step 1: Collect hourly load power values ​​and related influencing factors for multiple historical sample days in the target area;

[0013] Step 2: Reconstruct the original load data into a panel data matrix and classify the input data into time-varying inputs and static covariate inputs (data labels corresponding to time points). Time-varying inputs are divided into known inputs (which can be obtained in advance, including calendar characteristics and predictable weather information) and observed inputs (which are only obtained at the current time step and are unknown in advance, including load);

[0014] Step 3: Normalize the input data and divide the dataset into training set, validation set, and test set;

[0015] Step 4: Set model hyperparameters, such as the number of hidden layer neurons, learning rate, encoder input sequence length, and decoder input sequence length.

[0016] Step 5: Establish a time series fusion Transformer model, including an input feature interpretation module and a multi-time step fusion module;

[0017] Step 6: Iteratively train the time series fusion Transformer model based on the training set according to the loss function, and use the conventional gradient descent algorithm to determine the optimal weight and bias parameter set of the model;

[0018] Step 7: Input the validation set samples into the trained time series fusion Transformer model, select the optimal hyperparameters of the model based on the error of the validation set, and retrain the prediction model based on the optimal hyperparameters;

[0019] Step 8: Based on the test set, use the trained time series fusion Transformer model to predict the power values ​​of the load at different quantiles at each moment in the next month, and denormalize the output prediction results to obtain the load prediction values ​​at different quantiles at each prediction moment.

[0020] In step 2, the purpose of reconstructing the original load data is to shorten the length of the time series to be processed by the model, thereby reducing the complexity of the model. Specifically, the original univariate time series [y1, y2, ..., y n×m ] is reconstructed into a panel data matrix Y∈R m×n , the formula is as follows:

[0021]

[0022] Where y represents the load value, n and m are the number of days and the number of time points in a day respectively, t represents the tth day, s represents the time point label, which is also the static covariate input of the model; the time-related input χ s,t =[Z s,t ,X s,t ] consists of two parts, the observed input Z s,t (available only at that time step and unknown in advance) and the known input X s,t (External variables, including calendar characteristics and predictable weather characteristics).

[0023] In step 5, the established temporal fusion Transformer model includes an encoder, a decoder, an input feature interpretation module, a gated recurrent unit (GRU) layer, a multi-time step fusion module, and a linear mapping module. Specifically, the encoder consists of an input feature interpretation module and two GRU layers, and the decoder also consists of an input feature interpretation module and two GRU layers, but their weights and bias parameters are not shared.

[0024] (1) The input feature interpretation module is used to explain the contribution of each input variable to the output, making the model more interpretable and alleviating the "black box" problem in deep learning. Its main formula is as follows:

[0025] γ=dropout(W2(ELU(W1s+b1))+b2)

[0026] cs=LayerNom(s+σ(W3γ+b3)⊙(W4γ+b4))

[0027] v t =softmax(dropout(W6(ELU(W5(X t ,cs)+b5))+b6))

[0028]

[0029] Among them, X t =[x 1,t ,x 2,t ,...,x k,t ] T is the input vector consisting of k features at time t, is the vector fed into other modules after being processed by the input feature interpretation module, cs is the context vector after the static covariate s is processed by the gated residual process (as shown above), γ is an intermediate variable, and σ(·) is the sigmoid activation function; v t is the weight vector corresponding to the input variable, representing the importance of each input feature; W * and b * are the weight matrix and bias vector of the corresponding layer, LayerNorm represents the standard layer normalization operation, ELU represents the exponential linear unit activation function, dropout and softmax represent the conventional dropout operation and softmax activation function, respectively.

[0030] (2) The GRU layer is used to process temporal information. It overcomes the vanishing gradient problem in recurrent neural networks (RNNs) by retaining previously useful information while discarding irrelevant information. Structurally, the GRU has two gates: an update gate and a reset gate. The reset gate determines how new input information is integrated with previous memory, while the update gate defines the amount of previous memory preserved to the current time step. Its core principles are as follows:

[0031] z t =σ(W z [h t-1 ,x t ])

[0032] r t =σ(W r [h t-1 ,x t ])

[0033]

[0034]

[0035] Among them, x t is the current input, z t and r t They are update gate output and reset gate output, h t and h t-1 Represent the hidden layer states at time t and time t-1 respectively, is an intermediate variable, W z and W r is the weight matrix corresponding to the gate, represents the Hadamard product, and tanh represents the conventional tanh activation function.

[0036] (3) The multi-time-step fusion module is used to learn the long-term dependencies between different time steps. It can weight the outputs of the encoder and decoder of all time steps before the current prediction time step according to the importance of different time steps. The weighted output is then used as the input of the decoder's current prediction time step, allowing the model to focus on more important information. Its main formula is as follows:

[0037] Attention(Q,K,V)=A(Q,K)V

[0038]

[0039]

[0040]

[0041] Among them, A(·) is the normalization function, n is the dimension of the vector corresponding to the K matrix, and h m is the number of heads, Q, K and V are obtained by multiplying the input matrix X with the corresponding weight matrix respectively. are the weight matrices of the h-th head of Q and K, respectively, W V is the weight matrix V shared by all heads, is an intermediate variable, is the weight matrix of the final linear mapping.

[0042] In step 6, the time series fusion Transformer model is iteratively trained based on the training set according to the loss function, and the optimal weight and bias parameter set of the model is determined using the conventional gradient descent algorithm. Specifically, based on the loss function, the weight and bias parameters are continuously updated using the conventional gradient descent algorithm. The loss function used in this method is a loss function that adds quantile constraints and prediction interval penalty terms. Its advantage is that it can construct a more compact prediction interval while avoiding quantile crossing. Specifically, assuming the probability point τ i The quantile loss function under is defined as loss(τ i ). In the commonly used quantile prediction, a series of probability points 0<τ1<···<τ i ···<τ r The average of all quantile losses under <1 is used as the joint quantile loss function As shown in the following formula:

[0043]

[0044]

[0045] in is the probability point τ i Next t The conditional quantile of , N is the number of samples, r is the number of quantiles, and a is the intermediate variable. According to the inherent properties of quantiles, for each probability point τ, y t The conditional quantiles must meet the following requirements: To improve the rationality of the prediction results, this paper adds a penalty term for violating the quantile constraint to the original quantile loss function to minimize the overlap between adjacent quantiles. However, once the quantile constraint is added, the prediction interval will inevitably become wider. To obtain a tighter prediction interval, a corresponding prediction interval penalty term is also introduced into the loss function. The final improved loss function L is as follows:

[0046]

[0047] The penalty coefficient W 1,i and b i It is derived from the relationship between quantile losses at different probability points (see the specific implementation for the derivation process); W2 is a scaling factor used to ensure that the two terms added in the brackets in the loss function have similar values, and c j =τ r+1-j -τ j This ensures that the prediction interval penalty is weighted more as the confidence interval increases.

[0048] Compared with the prior art, the present invention has the following technical effects:

[0049] 1) Based on the daily periodicity of load, the present invention reconstructs the original hourly load time series into a panel data matrix, and then uses the time points as static covariates in the forecasting model, shortening the length of the time series and reducing the computational cost while retaining the variability of daily loads.

[0050] 2) The present invention uses an input feature interpretation module to make the model more interpretable, and the contribution of each input variable to the output is obvious; the use of a multi-time step fusion module allows the model to focus on more valuable information, thereby improving the model's prediction accuracy;

[0051] 3) This paper uses a time series fusion Transformer model to perform probabilistic forecasting of mid-term hourly loads. It adds quantile constraints and prediction interval penalty terms to the quantile loss function, avoiding the overlap between quantile prediction values. This makes the prediction results more reasonable and the prediction intervals more compact.

[0052] 4) The medium-term hourly load probability forecasting method based on the time series fusion Transformer model proposed in this invention has finer time resolution and higher prediction accuracy, which can provide power grid operators with more accurate and detailed load forecast information in medium-term power optimization scheduling and market transactions. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0054] Figure 1 A schematic diagram describing the medium-term load time series forecasting problem;

[0055] Figure 2 This is the Transformer model diagram for temporal fusion;

[0056] Figure 3 It is a structural diagram of GRU;

[0057] Figure 4 Schematic diagram of the multi-time step fusion module;

[0058] Figure 5 This is the prediction interval chart for half a month in the test set of Example 1;

[0059] Figure 6 This is the prediction interval chart for half a month in the test set of Example 2;

[0060] Figure 7-12 For two examples X QS ,X CS ,X QCS Box plots of indicators;

[0061] Figure 13 This is a comparison chart of the Transformer model's one-day prediction curve in Example 1;

[0062] Figure 14-15 For X PINRW Comparison chart of indicators;

[0063] Figure 16 This is an error comparison chart for the prediction range of 1-30 days. DETAILED DESCRIPTION

[0064] like Figure 1 As shown, Figure 1 It is a description of the medium-term load time series forecasting problem. The purpose of the predictor f is to simultaneously predict the next T o The load at each time point of the day [y s,t ,...,y s,t+τ ,...,y s,t+To-1 ]. In a given load time series dataset, the time-related input feature χ s,t =[Z s,t ,X s,t ] includes two categories. The observed input Z s,t , which is only available at that time step and is unknown in advance (e.g. load). s,t , can be obtained in advance (for example, external variables, including calendar characteristics and predictable weather information). Then the load forecast value can be estimated by the forecast model with parameter set Θ:

[0065]

[0066] Where τ represents the number of days, s represents the time point label, and Θ represents the parameter set after optimization in step 6.

[0067] like Figure 2 As shown in Figure 2, the medium-term hourly load probability forecasting method based on the time series fusion Transformer model includes the following steps:

[0068] Step 1: Collect load data every 15 minutes from January 1, 2016 to June 30, 2017 for a certain region A and hourly load data from January 1, 2004 to December 31, 2009 for a certain region B. Region A is used as Example 1, and region B is used as Example 2. The characteristic descriptions of the two examples and the input and output of the model are shown in Table 1, where T t Represents the temperature at each time of the forecast day, R t represents the rainfall in M ​​partitions, S t Indicates the corresponding prediction time point, Y s,t Represents the historical load value of the model input; S eaIndicates the season type, 1 represents winter, 2 represents spring and autumn, and 3 represents summer; H oli Represents the holiday type, 0 represents a holiday, 1 represents a non-holiday; W eek Represents the day of the week, 1-7 represent Monday to Sunday respectively.

[0069] Table 1 Characteristic description of two examples

[0070]

[0071] Step 2: Reconstruct the original load data into a panel data matrix and classify the input data into time-varying inputs and static covariate inputs. Time-varying inputs are further divided into known inputs and observed inputs. The purpose of reconstructing the original load data is to shorten the length of the time series to be processed by the model. Specifically, the original univariate time series [y1, y2, ..., y n×m ] is reconstructed into a panel data matrix Y∈R m×n , the formula is as follows:

[0072]

[0073] Where y represents the load value, n and m are the number of days and the number of time points in a day respectively; s represents the time point label, which is the static covariate input of the model. s,t =[Z s,t ,X s,t ] consists of two parts, the observed input Z s,t (available only at that time step and unknown in advance) and the known input X s,t (External variables, including calendar characteristics and predictable weather characteristics).

[0074] Step 3: Preprocess the input data and use maximum and minimum normalization for continuous variables such as temperature, rainfall, and historical load data. The formula is as follows:

[0075]

[0076] Among them, x norm is the normalized data. For discrete variables such as season type and week type, sine-cosine cyclic encoder is used for encoding. The formula is as follows:

[0077] [sin(2πx / x max ),cos(2πx / x max )]

[0078] Among them, x represents the index number, x maxRepresents the maximum value of the index. When encoding seasons, x∈{1,2,3}; when encoding days of the week, x∈{1,.....,7}. For holidays, one-hot encoding is used, with holidays encoded as 0 and non-holidays as 1. The dataset is divided into training, validation, and test sets in an 8:1:1 ratio.

[0079] Step 4: Set the model hyperparameters, including the number of hidden layer neurons (80), learning rate (0.01), encoder input sequence length (60), decoder input sequence length (30), etc.

[0080] Step 5: Build a time series fusion Transformer model, such as Figure 2 As shown in Figure 1, the temporal fusion Transformer model includes an encoder, a decoder, an input feature interpretation module, a GRU layer, a multi-timestep fusion module, and a linear mapping module. Specifically, the encoder consists of an input feature interpretation module and two GRU layers, and the decoder also consists of an input feature interpretation module and two GRU layers, but their weights and bias parameters are not shared.

[0081] Step 5.1: Time-dependent input features χ s,t-Ti:t-1 and X s,t:t+To-1 are fed into two input feature interpretation modules with no shared parameters, such as Figure 2 The input feature explanation module is used to explain the contribution of each input variable to the output. is sent to the next module. Its main formula is as follows:

[0082] γ=dropout(W2(ELU(W1s+b1))+b2)

[0083] cs=LayerNom(s+σ(W3γ+b3)⊙(W4γ+b4))

[0084] v t =softmax(dropout(W6(ELU(W5(X t ,cs)+b5))+b6))

[0085]

[0086] Among them, X t =[x 1,t ,x 2,t ,...,x k,t ] T is the input vector consisting of k features at time t, cs is the context vector after the static covariate s is processed by the gated residual process, γ is the intermediate variable, σ(·) is the sigmoid activation function; v tis the weight vector corresponding to the input variable, representing the importance of each input feature; W * and b * are the weight matrix and bias vector of the corresponding layer, LayerNorm represents the standard layer normalization operation, and ELU represents the exponential linear unit activation function.

[0087] Step 5.2: Input transformed by the input feature interpretation module are fed into two layers of GRU to more effectively process temporal information in long-term time series. GRU overcomes the vanishing gradient problem in RNN by retaining previously useful information while discarding irrelevant information. Structurally, GRU has two gates: update gate and reset gate, as shown in Figure 3 As shown. The reset gate determines how the new input information is integrated with the previous memory, while the update gate defines the amount of previous memory preserved to the current time step. Its core principles are as follows:

[0088] z t =σ(W z [h t-1 ,x t ])

[0089] r t =σ(W r [h t-1 ,x t ])

[0090]

[0091]

[0092] Among them, x t is the current input, z t and r t They are update gate output and reset gate output, h t and h t-1 Represent the hidden layer states at time t and time t-1 respectively, is an intermediate variable, W z and W r is the weight matrix corresponding to the gate, represents the Hadamard product.

[0093] Step 5.3: The outputs of the last GRU layer in the encoder and decoder are integrated into the multi-time step fusion module. The multi-time step fusion module is used to learn the long-term relationship between different time steps. It can weight the outputs of the encoder and decoder of all time steps before the current prediction time step according to the importance of different time steps, such as Figure 4The weighted output is then used as the input to the decoder's current prediction time step, allowing the model to focus on more important information. The main formula is as follows:

[0094] Attention(Q,K,V)=A(Q,K)V

[0095]

[0096]

[0097]

[0098] Where A(·) is the normalization function, n is the dimension of the vector corresponding to the K matrix, Q, K, and V are obtained by multiplying the input matrix X with the corresponding weight matrix respectively. are the weight matrices of the h-th head of Q and K, respectively, W V is the weight matrix V shared by all heads, is the weight matrix of the final linear mapping.

[0099] Step 5.4: Get the quantile prediction result by linearly mapping the output of the multi-time step fusion module.

[0100] Step 6: Iteratively train the time series fusion Transformer model based on the training set, and continuously update the weights and bias parameters using the gradient descent algorithm according to the loss function, and finally determine the optimal weight and bias parameter set of the model. The loss function used in this invention is a loss function that adds quantile constraints and prediction interval penalty terms, which avoids quantile crossing and constructs a more compact prediction interval. Specifically, assuming that the probability point τ i The quantile loss function under is defined as loss(τ i ). In the commonly used quantile prediction, a series of probability points 0<τ1<···<τ i ···<τ r The average of all quantile losses under <1 is used as the joint quantile loss function, as shown in the following formula:

[0101]

[0102]

[0103] in is the probability point τ i Next t The conditional quantile of , N is the number of samples, r is the number of quantiles. According to the inherent properties of quantiles, for each probability point τ, y t The conditional quantiles must meet the following requirements: To improve the rationality of the results, a penalty term for violating the quantile constraint is added to the original quantile loss function to minimize quantile crossover between adjacent quantiles. However, once the quantile constraint is added, the prediction interval will inevitably become wider. To obtain a tighter prediction interval, a corresponding prediction interval penalty term is also introduced into the loss function. The final improved loss function is as follows:

[0104]

[0105] Where W2 is a scaling factor that ensures that the two terms in the loss function have similar values, and c j =τ r+1-j -τ j Used to ensure that as the confidence interval increases, the weight of the prediction interval penalty term becomes larger. Penalty coefficient W 1,i and b i It is obtained from the following analysis. Define Δloss(τ i )=loss(τ i )-loss(τ i-1 ), then according to the relationship between quantile losses at different probability points, the original quantile loss function can be transformed into

[0106]

[0107] when That is, the predicted quantiles of adjacent probabilities violate the constraint relationship, then:

[0108]

[0109] where θ = τ i -τ i-1 is the step size between two adjacent quantile points. When the quantile constraint is violated and τ i ≤0.5, loss will be reduced When τ i ≤0.5, loss will be reduced When τ i >0.5, loss will be reduced Obviously, the loss value reduced when violating the quantile constraint is unreasonable. Therefore, in order to compensate for the unreasonable loss drop, the penalty coefficient W is derived based on the analysis. 1,i and b i for:

[0110]

[0111]

[0112] Step 7: Input the validation set samples into the trained time series fusion Transformer model, select the optimal hyperparameters of the model based on the error of the validation set, and retrain the prediction model based on the optimal hyperparameters;

[0113] Step 8: Use the trained time series fusion Transformer model to make predictions based on the test set, and denormalize the output prediction results to obtain the load prediction values ​​at different quantiles at each prediction time.

[0114] Figure 5 and Figure 6 The 30-day-ahead forecast curves obtained by the time series fusion Transformer model in Example 1 and Example 2 were compared with the actual load curves within half a month of the test set. It can be seen that the median curve (solid line) representing the load point forecast results has a similar variation pattern to the actual load curve. The prediction interval can fully cover the actual load curve (dashed line) at most time points, and the prediction interval with lower confidence is well embedded in the prediction interval with higher confidence. In addition, the prediction interval is wider at the peak and narrower at the trough and transition (especially obvious in Example 2), which indicates that the uncertainty of peak load is relatively high, making prediction more difficult. In addition, Example 2 is mainly residential electricity consumption, and the load changes are more regular, while Example 1 is mainly industrial electricity consumption, with higher uncertainty and weaker periodic characteristics.

[0115] Table 2 Comparison of evaluation indicators and training time of each model

[0116]

[0117] Table 2 shows the evaluation index and training time of each model, and the best value is bold. Figure 7-12 、 Figure 14-15 In the figure, models 1-5 represent TFT, QRNN, TCN, Transformer, and time series fusion Transformer models respectively. Figure 7-12 Shown are the X values ​​for two examples. QS 、X CS and X QCS The box plot depicts the distribution of these evaluation metrics, with the narrower part shown as a local enlargement above the gray arrow. Figure 13 The following shows the daily prediction curve of the Transformer model in Example 1 at different quantiles. Figure 14-15 Shown are the prediction interval sharpness X at different confidence levels PINRW Curve. For X QS 、XCS and X QCS , Table 2 and Figure 7-12 This shows that the time series fusion Transformer model is better than other models. Figure 7-12 The position of the model is the lowest, with fewer outliers and a shorter interquartile range, indicating that it has the smallest error and more stable performance. The error of the QRNN model is larger, the interquartile range of the TFT and Transformer models is wider, and the TCN model has more outliers. The QRNN, TCN, TFT, and Transformer models using the original quantile loss function have higher X CS , which shows that it is not well justified. Figure 13 The two quantile prediction curves of the Transformer model are compared with the actual curve (gray thinner solid line), as shown in Figure 13 As shown in Figure 2, the quantile curve of the 0.55 probability point is smaller than the quantile curve of the 0.45 probability point at some time points (shaded area), which is unreasonable. In contrast, the time series fusion Transformer model can avoid the intersection between quantiles and improve the reliability of the model. In terms of prediction interval sharpness, Table 2 and Figure 14-15 It shows that the TCN model has the lowest confidence level at all levels, but its X CS The large error indicates that it sacrifices the rationality of quantiles. In terms of prediction interval coverage PICP, Table 2 shows that the time series fusion Transformer model and QRNN model have smaller X than other models. AD and X MD , indicating that the proportion of actual load values ​​falling within the prediction intervals of the time series fusion Transformer model and the QRNN model is more consistent with the corresponding confidence levels, and their prediction intervals are more reliable.

[0118] Figure 16 The following table shows the daily X values ​​for Case 1 at 50% and 90% confidence levels when the forecast range is 1 day to 30 days. QCS and X PINRW Indicator situation. Except for the temporary rise on the 15th to 19th day, X PINRW It shows a downward trend at both 50% and 90% confidence levels and reaches a minimum around the 15th day. This is because the model training is performed as a whole and the error has a time transfer effect, which makes the model have a lower error in the short and medium forecast range. On the contrary, since the long-term dependence of the time series is difficult to capture and the forecast error of weather characteristics increases, X QCS As the prediction horizon increases, X PINRW and X QCSThis is because the tighter the prediction interval is, the fewer points fall within it, and the lower the reliability of the model is. Therefore, as the prediction range increases, when the prediction interval width is narrower, X QCS The higher.

Claims

1. A medium-term hourly load probability forecasting method based on a time series fusion Transformer model, characterized by: It includes the following steps: Step 1: Collect hourly load power values ​​and related weather influencing factors data for multiple historical sample days in the target area; Step 2: Reconstruct the original load data including the univariate load time series data into a panel data matrix, and classify the input data into time-varying input and static covariate input. The time-varying input is divided into known input and observed input. Step 3: Normalize the input data and divide the dataset into training set, validation set, and test set; Step 4: Set the model hyperparameters, including the number of hidden layer neurons, learning rate, encoder input sequence length, and decoder input sequence length. Step 5: Establish a time series fusion Transformer model, including an input feature interpretation module and a multi-time step fusion module; Step 6: Iteratively train the time series fusion Transformer model based on the training set according to the loss function, and use the conventional gradient descent algorithm to determine the optimal weight and bias parameter set of the model; Step 7: Input the validation set samples into the trained time series fusion Transformer model, select the optimal hyperparameters of the model based on the error of the validation set, and retrain the prediction model based on the optimal hyperparameters; Step 8: Based on the test set, use the trained time series fusion Transformer model to predict the power values ​​of the load at different quantiles at each time in the next month. Denormalize the output prediction results to obtain the load prediction values ​​at different quantiles at each prediction time. The temporal fusion Transformer model established in step 5 mainly includes an input feature interpretation module, a gated recurrent unit layer, and a multi-time step fusion module; (1) The input feature interpretation module is used to explain the contribution of each input variable to the output. Its formula includes: γ=dropout(W2(ELU(W1s+b1))+b2) cs=LayerNom(s+σ(W3γ+b3)⊙(W4γ+b4)) v t =softmax(dropout(W6(ELU(W5(X t ,cs)+b5))+b6)) Among them, X t =[x 1,t ,x 2,t ,...,x k,t ] T is the input vector consisting of k features at time t, is the vector fed into other modules after being processed by the input feature interpretation module, cs is the context vector processed by the static covariate s through the gated residual process, γ is the intermediate variable, and σ(·) is the sigmoid activation function; v t is the weight vector corresponding to the input variable, representing the importance of each input feature; W * and b * are the weight matrix and bias vector of the corresponding layer, LayerNorm represents the standard layer normalization operation, ELU represents the exponential linear unit activation function, dropout and softmax represent the conventional dropout operation and softmax activation function respectively; The observed input after transformation by the input feature interpretation module Known Input They are fed into the two-layer GRU of the encoder and decoder respectively; The encoder consists of an input feature interpretation module and two layers of GRU, and the decoder also consists of an input feature interpretation module and two layers of GRU, but their weights and bias parameters are not shared; (2) The gated recurrent unit layer is used to process time information. Its formula includes: z t =σ(W z [h t-1 ,x t ]) r t =σ(W r [h t-1 ,x t ]) Among them, x t is the current input, z t and r t They are update gate output and reset gate output, h t and h t-1 Represent the hidden layer states at time t and time t-1 respectively, is an intermediate variable, W z and W r is the weight matrix corresponding to the gate, represents the Hadamard product, and tanh represents the conventional tanh activation function; (3) The multi-time-step fusion module is used to learn the long-term dependencies between different time steps. The multi-time-step fusion module weights the outputs of the encoder and decoder of all time steps before the current prediction time step according to the importance of different time steps. The formula includes: Attention(Q,K,V)=A(Q,K)V Among them, A(·) is the normalization function, n is the dimension of the vector corresponding to the K matrix, and h m is the number of heads, Q, K and V are obtained by multiplying the input matrix X with the corresponding weight matrix respectively. are the weight matrices of the hth head of Q and K, W V is the weight matrix V shared by all heads, is an intermediate variable, is the weight matrix of the final linear mapping; The quantile prediction results are obtained by linearly mapping the output of the multi-time step fusion module.

2. The method according to claim 1, characterized in that In step 2, the original univariate load time series [y1,y2,...,y n×m ] is reconstructed into a panel data matrix Y∈R m×n , the formula is as follows: Where y represents the load value, n and m are the number of days and the number of time points in a day respectively, t represents the tth day, s represents the time point label, which is also the static covariate input of the model; the time-related input χ s,t =[Z s,t ,X s,t ] consists of two parts, the observed input Z s,t and the known input X s,t .

3. The method according to claim 1, characterized in that In step 6, the temporal fusion Transformer model is iteratively trained based on the training set according to the loss function, and the conventional gradient descent algorithm is used to determine the optimal weight and bias parameter set of the model; Specifically, the loss function used is a loss function that adds quantile constraints and prediction interval penalty terms. Its advantage is that it avoids quantile crossing while constructing a more compact prediction interval; Assume probability point τ i The quantile loss function under i ); In the commonly used quantile prediction, a series of probability points 0<τ1<…<τ i …<τ r The average of all quantile losses under <1 is used as the joint quantile loss function l, as shown in the following formula: in is the probability point τ i The actual load value y at the next moment t t The conditional quantile of , N is the number of samples, r is the number of quantiles, a is the intermediate variable; according to the inherent properties of quantiles, for each probability point τ, y t The conditional quantiles of must satisfy the following requirements: In order to improve the rationality of the prediction results, a penalty term for violating the quantile constraint is added to the original quantile loss function to eliminate the intersection between adjacent quantiles as much as possible; in order to obtain a more compact prediction interval, the corresponding prediction interval penalty term is also introduced into the loss function; the final improved loss function L is as follows: Where W 1,i and b i is the penalty coefficient; W2 is a scaling factor used to ensure that the two terms added in the brackets in the loss function have similar values, and c j =τ r+1-j -τ j This ensures that the prediction interval penalty is weighted more as the confidence interval increases.

Images