A multi-energy load forecasting method considering multiple uncertainties

Abstract

The application discloses a multi-energy load forecasting method suitable for a comprehensive energy system and considering multiple uncertainties, and belongs to the technical field of energy load forecasting. The application uses a multi-task learning framework to perform multi-load forecasting, captures complex coupling relationships among electric, heat and hydrogen loads, and enhances the prediction accuracy of each prediction task. Uncertainty factors affecting prediction are considered, specifically including input uncertainty and model parameter uncertainty, and uncertainty modeling is performed on both, so as to quantify the uncertainty risk of load forecasting. In combination with an attention mechanism, the model can focus on important time step feature information and reduce unnecessary information interference. The multi-load forecasting method provided by the application can capture the coupling relationships among loads and realize accurate prediction of multi-energy loads under the condition of considering uncertainty factors.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of energy load prediction, and particularly relates to a multi-energy load prediction method considering multiple uncertainties. BACKGROUND

[0002] With the continuous increase of global pressure to combat climate change, optimizing energy structure has become an important trend. The demand for energy structure optimization has driven the integration of various energies such as electricity, heat and hydrogen, forming a highly coupled and collaborative comprehensive energy system, reducing dependence on single energy forms and improving energy utilization efficiency. For the electricity, heat and hydrogen comprehensive energy system, accurate load prediction can optimize system operation, improve resource utilization efficiency and reduce energy waste.

[0003] Traditional load prediction methods mostly use the method of independently predicting different energy loads. This method ignores the complex coupling relationship between multiple energy loads, for example, in the electricity, heat and hydrogen system, the power production process directly affects the supply of heat energy; in the hydrogen production process, the consumption of electricity is closely related to the production of hydrogen energy. These coupling relationships make the comprehensive energy system have high complexity and mutual dependence in operation. In addition, traditional load prediction often ignores the influence of uncertainty factors, including input uncertainty and model parameter uncertainty. Fluctuations in input characteristics (such as temperature changes) can cause cumulative prediction bias, reducing prediction accuracy; at the same time, fixed parameter estimation methods fail to consider the generalization of model parameters when facing new data, thereby affecting the accuracy and reliability of the prediction results. These uncertainty factors together lead to a decline in prediction performance. SUMMARY

[0004] The purpose of the present application is to propose a multi-energy load prediction method considering uncertainty in view of the above-mentioned problems, which quantifies the uncertainty of external factors and model parameters themselves while capturing the coupling relationship between multiple loads, to achieve more accurate electricity, heat and hydrogen load prediction.

[0005] To achieve the above-mentioned purpose, the present application specifically includes: analyzing the influence of different factors on load prediction, selecting the most significant influence characteristics, and modeling the uncertainty of input characteristics; building a multi-task learning long short-term memory network (MTL-LSTM) model based on Bayesian neural network (BNN), quantifying the uncertainty of model parameters, and learning the coupling relationship between multiple loads; combining attention mechanism (AM) to improve the prediction accuracy of the model.

[0006] The application provides the following steps:

[0007] Step 1: analyze the influence of different factors on the prediction of electricity, heat and hydrogen load, select the most significant influence characteristics, model the uncertainty of the selected external characteristics, and establish the probability distribution model thereof;

[0008] Step 2: construct a multi-task learning MTL-LSTM model based on BNN, quantify the uncertainty of the model parameters, and capture the coupling relationship among multiple loads through the multi-task learning framework;

[0009] Step 3: combine the attention mechanism to enhance the attention of the model to key time step characteristics, and improve the prediction accuracy of the model.

[0010] Further, in step 1, the Pearson correlation coefficient analysis method is used to analyze the correlation of characteristics such as temperature, humidity, precipitation, wind speed, illumination, holiday information, population size, energy policy, industrial activity, and the correlation and information measurement results of electricity, heat and hydrogen load, and the correlation and information measurement results of electricity, heat and hydrogen load, and finally temperature, humidity, illumination and wind speed are selected as the most significant external influence characteristics. The external characteristic data is used as the input of the subsequent prediction model, and the historical heat, electricity and hydrogen load data are also used as the input.

[0011] It is considered that the external characteristics such as temperature may have noise, prediction error and other uncertainty factors, and this uncertainty will be directly transmitted to the load prediction model, causing inaccurate prediction results. In order to measure the influence of this uncertainty, the uncertainty of the external characteristics (temperature, humidity, wind speed and illumination) is modeled. The external characteristics are regarded as random variables subject to normal distribution, the mean and variance of these characteristics are estimated as random variables, and the fluctuation range of the input data is effectively evaluated. Specifically as follows:

[0012]

[0013]

[0014]

[0015]

[0016] Wherein, T t+k , H t+k , W t+k , S t+k are the probability distribution of temperature, humidity, wind speed and illumination at time t+k, k is the time step, μ is the mean value of the corresponding characteristics, and σ 2 is the corresponding uncertainty variance. By sampling the sequence multiple times, such as sampling the temperature characteristics, the following can be obtained: The input sequence corresponding to the external feature at t+k moment, n is the sampling number.

[0017] Further, in step 2, the core idea of multi-task learning is to process multiple related tasks at the same time through sharing and specific network structure, multiple tasks share parameters in the same model, and then capture the common features of different tasks, improve the model learning efficiency and generalization ability. In the present application, the multi-task learning model is MTL-LSTM, which is used to capture the commonality and characteristics among the heat, electricity and hydrogen load prediction tasks. The model is divided into two parts: a shared part and a specific part. The network structure of the shared part is an LSTM network, which is used to process the shared information of the heat, electricity and hydrogen load prediction tasks. The specific part structure is an independent fully connected layer corresponding to the electricity, heat and hydrogen load respectively, which is used to learn the specific information of each prediction task. The shared layer is specifically represented as follows:

[0018]

[0019] h t =f(u t ,h t-1 ,θ sh )

[0020] Wherein, u t is the prediction input at t moment, is the electricity, heat and hydrogen load data at t moment, T t , H t , W t , S t is the temperature, humidity, wind speed and illumination data at t moment, is the temperature, humidity, wind speed and illumination data of the mth sampling at t+k moment, h t , h t-1 is the LSTM layer hidden state output at t moment and t-1 moment, f(·) is the shared layer LSTM network function, θ sh is the shared parameter of the LSTM layer for electricity, heat and hydrogen prediction tasks.

[0021] The shared layer output h t is mapped to each load prediction value through the specific task layer, i.e. the fully connected layer, and the specific task layer is specifically represented as follows:

[0022]

[0023] Wherein, is the prediction output of each load at t+1 moment, g(·) is the fully connected layer network function, θ i is the parameter related to the electricity, heat and hydrogen load prediction tasks in the fully connected layer.

[0024] Based on the principle of BNN, the parameter uncertainty in MTL-LSTM is modeled: the parameters in LSTM, such as weights and biases, are modeled as probability distribution Thus the uncertainty estimation of the model is introduced. During the training process, for each iteration, the model samples parameters θ from the posterior distribution sh , and uses the sampled parameters to perform forward propagation to calculate the load prediction values of multiple tasks. The loss function is defined as the negative log-likelihood function, which is specifically represented as follows:

[0025]

[0026]

[0027] wherein L i is the loss function of each prediction task, is the actual value at time t+1 of each prediction task, is the predicted value at time t+1 of each prediction task, is the variance of each prediction task, L total is the overall joint loss function, λ i is the weight coefficient corresponding to the task. Finally, the parameters are updated through error back propagation to obtain the approximate posterior probability distribution of the model parameters.

[0028] During the prediction process, the model samples from the posterior probability distribution of the model parameters obtained by training to obtain Using multiple sampled parameters for calculation, the mean and variance of the load prediction result are obtained. The specific representation is as follows:

[0029]

[0030]

[0031]

[0032]

[0033] wherein h is the hidden state output at time t of the jth sampling, is the predicted mean at time t+1 of each prediction task, is the jth sampling parameter of the shared layer, is the predicted value under the jth sampling parameter.

[0034] Further, in step 3, the LSTM layer as the shared layer generates the hidden state h tThese hidden states are the comprehensive representation of the historical input at the current time t, including the correlation among electricity, heat, hydrogen load and the influence of external features on the load. However, in a long time sequence, different time step features have different importance for the prediction task. The attention mechanism is introduced to assign different weights to different time steps of the input sequence, so that the model pays attention to the information more important for the prediction task, and thus improves the prediction performance of the model.

[0035] The attention mechanism calculates the correlation between the current time step t and the historical time step, and assigns weights to different time steps according to the correlation to generate comprehensive features. Specifically, for the hidden state h t of the given current time step and the hidden state {h1, h2, … h t-1} of all historical time steps, the attention mechanism calculates the attention score of the current hidden state and the historical hidden state, which is specifically represented as follows:

[0036] e t,i =F(h t ,h i )

[0037] Where e t,i is the attention score of the current time step t and the historical time step i, and F(·) is the attention scoring function.

[0038] The attention score is normalized using the softmax function to convert it into attention weight α t,i , so that the sum of different weights is 1, which is specifically represented as follows:

[0039]

[0040] Where the attention weight α t,i represents the importance of the model at time step i to the current load prediction task.

[0041] Finally, based on the attention weight α t,i , the weighted sum of all input sequences is obtained, and the weighted input is obtained, and the comprehensive feature c t containing historical information is obtained, which is specifically represented as follows:

[0042]

[0043] The context vector c t is combined with the current hidden state to generate a new hidden state instead of the original h t , which is specifically represented as follows:

[0044]

[0045] The new hidden state not only contains the feature information of the current time step, but also combines more important feature information in the historical time step, which is input into the specific task layer for the final load prediction, so that the model can more accurately predict the result.

[0046] In summary, due to the adoption of the above technical solutions, the present application has the following advantages:

[0047] (1) Multiple load predictions are performed using a multi-task learning framework, the complex coupling relationship between electric, thermal and hydrogen loads is captured through an MTL-LSTM model, shared information between different tasks is utilized, and the prediction accuracy of each prediction task is enhanced.

[0048] (2) Uncertainty factors are considered, including input uncertainty and model parameter uncertainty. The uncertainty of input features is modeled, the prediction deviation caused by noise, prediction error and other factors of input features (such as temperature, humidity) on electric, thermal and hydrogen load prediction is quantified, and the reliability of prediction is improved; based on the principle of BNN, the uncertainty of MTL-LSTM model parameters is modeled, the uncertainty risk of load prediction is quantified, and the generalization ability of the model is improved.

[0049] (3) AM helps the model focus on key time step features that have a greater impact on the future in load prediction by assigning weights, improves the attention ability of the model to important moments, reduces unnecessary information interference, and improves the prediction accuracy of the model. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a schematic diagram of the structure of the regional integrated energy system built by the embodiment of the present application;

[0051] Figure 2 is a schematic diagram of the structure of the MTL-LSTM prediction model built by the embodiment of the present application;

[0052] Figure 3 is a flowchart of quantifying the uncertainty of model parameters proposed by the embodiment of the present application;

[0053] Figure 4 is a flowchart of the overall prediction method proposed by the embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with embodiments and drawings.

[0055] The embodiment of the present application provides a method suitable for multi-energy load prediction, which first builds a regional integrated energy system model, such as Figure 1As shown. Analysis can be known, by electrolytic tank using electricity for water electrolysis can generate hydrogen; hydrogen through hydrogen fuel cell can not only be converted into electrical energy, but also will produce a large amount of heat energy, these heat energy can be used for heating or heating. In addition, external weather factors, such as light intensity, temperature, humidity, etc., directly affect the working efficiency of photovoltaic and wind power generation equipment, and further affect the generation of electrical energy and the conversion of other energy in the system. In the integrated energy system, external factors have an important influence on energy load, and there is a complex coupling relationship between different energy loads such as electricity, heat and hydrogen. This mutual influence will further exacerbate the load fluctuation of the system. Therefore, in order to realize more accurate load prediction, the coupling relationship between these external environmental factors and loads must be considered, and the relative influence of each factor must be analyzed.

[0056] As a possible implementation manner, the specific implementation steps of the present application include:

[0057] S1, analyze the influence of different factors on the prediction of electricity, heat, hydrogen and other multi-load, select the most significant influence characteristics, and model the uncertainty of the selected future time characteristics to establish the probability distribution model.

[0058] The correlation between the characteristics such as temperature, humidity, precipitation, wind speed, illumination, holiday information, population size, energy policy, industrial activity and the like is analyzed by Pearson correlation coefficient analysis method, which is specifically expressed as follows, taking temperature as an example:

[0059]

[0060] Wherein, γ is the Pearson correlation coefficient, T t and are the temperature and corresponding load observation values corresponding to t time, and are the average values of temperature and corresponding load.

[0061] By comparing the correlation and information measurement results of each characteristic and electricity, heat, hydrogen load and the correlation and information measurement results among electricity, heat and hydrogen load, temperature, humidity, illumination and wind speed are finally selected as the most significant external influence characteristics, which are used as the input of the subsequent prediction model. At the same time, considering that there is also a complex coupling relationship among multi-load, historical heat, electricity and hydrogen load data are also used as input.

[0062] Considering that external characteristics such as temperature may have noise, prediction error and other uncertainty factors, this uncertainty will be directly transmitted to the load prediction model, causing inaccurate prediction results. In order to measure the influence of this uncertainty, the uncertainty of external characteristics (temperature, humidity, wind speed and illumination) is modeled. The external characteristics are regarded as random variables subject to normal distribution, and the mean and variance of these characteristics are estimated as random variables, so as to effectively evaluate the fluctuation range of input data. It is specifically expressed as follows:

[0063]

[0064]

[0065]

[0066]

[0067] where T t+k , H t+k , W t+k , S t+k are the probability distribution of temperature, humidity, wind speed, and light at time t+k, k is the time step, μ is the mean value of the corresponding feature, σ 2 is the uncertainty variance of the corresponding feature. By sampling the sequence multiple times, such as sampling the temperature feature, the input sequence of the external feature at time t+k can be obtained, n is the number of samplings.

[0068] S2, through the multi-task learning model MTL-LSTM, the structure is as shown in Figure 2 , to capture the commonality and characteristics between the heat, electricity, and hydrogen load prediction tasks. The model is divided into two parts: a shared part and a specific part. The network structure of the shared part is an LSTM network, which is used to process the shared information of the heat, electricity, and hydrogen load prediction tasks; the specific part structure is an independent fully connected layer corresponding to the electricity, heat, and hydrogen load, which is used to learn the specific information of each prediction task. The shared layer function is specifically represented as follows:

[0069]

[0070] h t = f(u t , h t-1 , θ sh )

[0071] where u t is the prediction input at time t, is the electricity, heat, and hydrogen load data at time t, T t , H t , W t , S t are the temperature, humidity, wind speed, and light data at time t, is the mth sampling data of temperature, humidity, wind speed, and light at time t+k, h t , h t-1 are the LSTM layer hidden state outputs at time t and t-1, f(·) is the shared layer LSTM network function, θ sh is the LSTM layer shared parameter of the electricity, heat, and hydrogen prediction tasks.​

[0072] shared layer output h t , and the load prediction values are obtained through a specific task layer, i.e., a full connection layer, which is specifically represented as follows:

[0073]

[0074] wherein, is the load prediction output at t+1, g(·) is a full connection layer network function, θ i is a parameter related to the electric, heat and hydrogen load prediction tasks in the full connection layer.

[0075] Based on the BNN principle, the parameter uncertainty in the MTL-LSTM is modeled, as shown in Figure 3 wherein the weights and biases and other parameters in the LSTM are modeled as a probability distribution so as to introduce the uncertainty estimation of the model. During the training of the model, for each iteration, the model samples the parameters θ sh from the posterior distribution, and uses the sampled parameters to perform forward propagation to calculate the load prediction values of multiple tasks. The loss function is defined as a negative log-likelihood function, and is specifically represented as follows:

[0076]

[0077]

[0078] wherein L i is the loss function of each prediction task, L total is the overall joint loss function, λ i is a task weight coefficient, is the actual value at t+1 of each prediction task, is the predicted value at t+1 of each prediction task, is the variance of each prediction task. Finally, the parameters are updated through error back propagation to obtain the approximate posterior probability distribution of the model parameters.

[0079] During the prediction process, the model samples the posterior probability distribution of the model parameters obtained through training to obtain and uses multiple sampled parameters to calculate to obtain the mean and variance of the load prediction results. The specific representation is as follows:

[0080]

[0081]

[0082]

[0083]

[0084] where, is the hidden state output at time step t for the jth sample, is the predicted mean at time t+1 for each prediction task, is the jth sample parameter, is the predicted value for the jth sample parameter, and n is the number of samples.

[0085] S3, introduce attention mechanism, by assigning different weights to different time steps of the input sequence, so that the model pays attention to more important information for the prediction task, and thus improves the prediction performance of the model.

[0086] The attention mechanism calculates the correlation between the current time step t and the historical time step, and assigns weights to different time steps according to the correlation to generate comprehensive features. Specifically, for a given hidden state h t of the current time step and the hidden state {h1, h2, … h t-1} of all historical time steps, the attention mechanism calculates the attention score of the current hidden state and the historical hidden state, which is specifically expressed as follows:

[0087] e t,i = F(h t , h i )

[0088] where e t,i is the attention score of the current time step t and the historical time step i, and F(·) is the attention scoring function.

[0089] The softmax function is used to normalize the attention score into attention weight α t,i , so that the sum of different weights is 1, which is specifically expressed as follows:

[0090]

[0091] where the attention weight α t,i represents the importance of the model at time step i to the current load prediction task.

[0092] Finally, based on the attention weight α t,i , the weighted sum of all input sequences is obtained, and the weighted input is obtained, and the comprehensive feature c t containing historical information is obtained, which is specifically expressed as follows:

[0093]

[0094] The context vector c t is combined with the current hidden state to generate a new hidden state instead of the original ht , specifically as follows;

[0095]

[0096] The new hidden state not only contains the feature information of the current time step, but also combines the more important feature information in the historical time step, which is input into the specific task layer for the final load prediction, so that the model can more accurately predict the result.

[0097] The embodiment of the application realizes the process as shown in Figure 4 The input data uncertainty is modeled, sampled and input into the historical data to predict the model, and the prediction model is BNN+MTL-LSTM+AM. BNN is used to quantify the deviation of the model parameter uncertainty on the load prediction, and to enhance the robustness of the model. MTL-LSTM captures the complex coupling relationship between electric, thermal and hydrogen loads, and simultaneously uses the respective fully connected layers to learn the time sequence features of each load. The AM mechanism gives higher weight to the key time step features, focuses on the future load prediction, and improves the accuracy and reliability of the multi-energy load prediction.

[0098] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

[0099] The above only describes some embodiments of the present application. For those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. A method for multi-energy load forecasting considering multiple uncertainties, characterized in that, The method comprises the following steps: Step 1: analyze the influence of different factors on the prediction of electric, thermal and hydrogen load, select the most significant influence characteristics, model the uncertainty of the selected external characteristics, and establish the probability distribution model thereof; Step 2: construct a multi-task learning long short-term memory network MTL-LSTM (Multi-Task Learning Long Short-Term Memory) model based on a Bayesian neural network BNN (Bayesian Neural Network), quantify the uncertainty of the model parameters, and capture the coupling relationship among the multiple loads through the multi-task learning framework; in step 2, multi-task learning is to simultaneously process multiple related tasks through sharing and a specific network structure, and multiple tasks share parameters in the same model, thereby capturing the common characteristics of different tasks and improving the learning efficiency and generalization ability of the model; the multi-task learning model is MTL-LSTM, which is used to capture the commonality and characteristics among the thermal, electric and hydrogen load prediction tasks; the model is divided into two parts: a shared part and a specific part, the network structure of the shared part is an LSTM network, which is used to process the shared information of the thermal, electric and hydrogen multi-load prediction tasks; the specific part structure is an independent fully connected layer corresponding to each of the electric, thermal and hydrogen load, which is used to learn the specific information of each prediction task; the shared layer is specifically represented as follows: h t = f(u t ,h t-1 ,θ sh ) wherein, u t is the prediction input at time t, is the electricity, heat, hydrogen load data at time t, T t , H t , W t , S t is the temperature, humidity, wind speed, and light data at time t, is the mth sampling data of temperature, humidity, wind speed, and light at future time t+k, h t , h t-1 is the LSTM layer hidden state output at time t and t-1, f(·) is the shared layer LSTM network function, θ sh is the electricity, heat, and hydrogen prediction task shared parameter in the LSTM layer; shared layer output h t The load prediction value is obtained by mapping through the specific task layer, i.e. the full connection layer, and the specific task layer is specifically represented as follows: wherein, is the predicted output for each load at time t+1, g(·) is a fully connected layer network function, θ i is the parameter related to the electric, heat, and hydrogen load prediction tasks in the fully connected layer. Modeling the parameter uncertainty in MTL-LSTM based on BNN principle: modeling the parameters such as weights and biases in LSTM as probability distributions Thus introducing the uncertainty estimation of the model; during the training process, for each iteration, the model samples the parameters θ from the posterior distribution sh Using the sampled parameters to perform forward propagation, calculate the multi-task load prediction value, define the loss function as the negative log-likelihood function, which is specifically represented as follows: wherein, L i is the loss function of each prediction task, is the actual value at time t+1 for each prediction task, is the predicted value at time t+1 for each prediction task, is the variance of each prediction task, L total is the overall joint loss function, λ i is the weight coefficient corresponding to the task; finally, the parameters are updated through error back propagation to obtain the approximate posterior probability distribution of the model parameters; In the prediction process, the model obtains the posterior probability distribution of the model parameters obtained by training by sampling The mean and variance of the load prediction result are obtained by calculating the multiple sampling parameters, and are specifically represented as follows: wherein, is the hidden state output at time step t for the jth sample, is the prediction mean at time t+1 for each prediction task, is the shared layer parameter for the jth sample, is the prediction value for the jth sample parameter. Step 3: combine the attention mechanism to enhance the attention of the model to the key time step features and improve the prediction accuracy of the model.

2. The method of claim 1, wherein, In step 1, the Pearson correlation coefficient analysis method is used to analyze the correlation of temperature, humidity, precipitation, wind speed, illumination, holiday information, population size, energy policy and industrial activity characteristics, compare the correlation and information measurement results of each characteristic with the electric, thermal and hydrogen load, and the correlation among the electric, thermal and hydrogen load, and finally select temperature, humidity, illumination and wind speed as the most significant external influence characteristics; the external characteristic data is used as the input of the subsequent prediction model, and the historical thermal, electric and hydrogen load data is also used as the input considering the complex coupling relationship among the multiple loads; The uncertainty of the temperature, humidity, wind speed and illumination external characteristics is modeled, the external characteristics are regarded as random variables subject to normal distribution, the mean and variance of the characteristics are estimated as random variables, and the fluctuation range of the input data is effectively evaluated, and the specific process is as follows: where T t+k , H t+k , W t+k , S t+k are the probability distribution of temperature, humidity, wind speed, and light at time t+k, k is the time step, μ is the mean of the corresponding feature, σ 2 is the variance of the corresponding uncertainty; by sampling the sequence multiple times, the input sequence of the external features at time t+k can be obtained.

3. The method of claim 1, wherein, In step 3, the LSTM layer as a shared layer generates the hidden state h of each time step t These hidden states are the comprehensive representation of historical inputs at the current time t, including the correlation among electricity, heat, and hydrogen loads and the influence of external features on the load; however, in a long time series, different time step features have different importance for the prediction task. Therefore, an attention mechanism is introduced to assign different weights to different time steps of the input sequence, so that the model pays more attention to the information that is more important for the prediction task, thereby improving the prediction performance of the model. The attention mechanism calculates the correlation between the current time step t and the historical time steps, and assigns weights to different time steps according to the correlation to generate a comprehensive feature; specifically, for the hidden state h t of the given current time step and the hidden state {h1, h2, … h t-1 of all historical time steps, the attention mechanism calculates the attention score of the current hidden state and the historical hidden state, which is specifically expressed as follows: e t,i = F(h t ,h i ) where e t,i is the attention score of the current time step t and the historical time step i, and F(·) is the attention scoring function. The attention scores are normalized using a softmax function to convert them into attention weights a t,i such that the sum of the different weights is 1, expressed as follows: Attention weight a t,i represents the importance of the model to the current load forecasting task at time step i; Finally, based on the attention weight α t,i The weighted input is obtained by weighting and summing all input sequences, and the comprehensive feature c containing historical information is obtained t , which is specifically represented as follows: The context vector c t In combination with the current hidden state, a new hidden state is generated Instead of the original h t , is specifically expressed as follows; The new hidden state not only contains the feature information of the current time step, but also combines the more important feature information of the historical time steps, which is input into the specific task layer for the final load prediction, so that the model can more accurately predict the results.

Images