Multi-energy load prediction method considering multiple uncertainties

Abstract

The invention discloses a multi-energy load prediction method considering multiple uncertainties and suitable for an integrated energy system, and belongs to the technical field of energy load prediction. According to the method, a multi-task learning framework is used for multi-load prediction, the complex coupling relation among electricity, heat and hydrogen loads is captured, and the prediction precision of each prediction task is enhanced; considering the influence of uncertainty factors on prediction, specifically including input uncertainty and model parameter uncertainty, respectively performing uncertainty modeling on the input uncertainty and the model parameter uncertainty, and quantifying the uncertainty risk of the uncertainty factors on load prediction; and in combination with an attention mechanism, the model is helped to pay attention to important time step feature information, and unnecessary information interference is reduced. According to the multi-load prediction method provided by the invention, the coupling relationship between the loads can be captured under the condition that uncertainty factors are considered, and accurate prediction of the multi-energy loads is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of energy load forecasting, and in particular relates to a multi-energy load forecasting method considering multiple uncertainties. Background Art

[0002] As the global pressure to respond to climate change continues to increase, optimizing the energy structure has become an important trend. The demand for optimizing the energy structure has promoted the integration of multiple energy sources such as electricity, heat, and hydrogen, forming a highly coupled and coordinated integrated energy system, reducing dependence on a single energy form and improving energy efficiency. For the integrated energy system of electricity, heat, and hydrogen, accurate load forecasting can optimize system operation, improve resource utilization efficiency, and reduce energy waste.

[0003] Most traditional load forecasting methods adopt the method of independently forecasting different energy loads. This method ignores the complex coupling relationship between multiple energy loads. For example, in the electricity, heat, and hydrogen systems, the electricity production process will directly affect the supply of heat energy; in the hydrogen production process, the consumption of electricity is closely related to the production of hydrogen energy. These coupling relationships make the integrated energy system highly complex and interdependent in operation. In addition, traditional load forecasting often ignores the influence of uncertainty factors, including input uncertainty and model parameter uncertainty. Fluctuations in input characteristics (such as temperature changes) will lead to the accumulation of prediction bias and reduce prediction accuracy; at the same time, the fixed parameter estimation method fails to consider the generalization of model parameters when facing new data, thereby affecting the accuracy and reliability of the prediction results. These uncertain factors jointly lead to a decline in prediction performance. Summary of the invention

[0004] The purpose of the present invention is to propose a multi-energy load forecasting method taking uncertainty into account in response to the above-mentioned problems. By quantifying the uncertainty of external factors and model parameters themselves and capturing the coupling relationship between multiple loads, a more accurate prediction of electricity, heat and hydrogen loads can be achieved.

[0005] To achieve the above objectives, the present invention specifically includes: analyzing the impact of different factors on load forecasting, selecting the most significant influencing features, and modeling the uncertainty of input features; building a multi-task learning long short-term memory network (Multi-Task Learning Long Short-Term Memory, MTL-LSTM) model based on a Bayesian Neural Network (Bayesian Neural Network, BNN), quantifying the uncertainty of model parameters, and learning the coupling relationship between multiple loads; combining the attention mechanism (Attention Mechanism, AM) to improve the model prediction accuracy.

[0006] The present invention provides the following steps:

[0007] Step 1: Analyze the impact of different factors on the electricity, heat and hydrogen load forecast, select the most significant influencing features, perform uncertainty modeling on the selected external features, and establish their probability distribution model;

[0008] Step 2: Build a BNN-based multi-task learning MTL-LSTM model to quantify the uncertainty of model parameters and capture the coupling relationship between multiple loads through a multi-task learning framework;

[0009] Step 3: Combine the attention mechanism to enhance the model's attention to key time step features and improve the model's prediction accuracy.

[0010] Furthermore, in step 1, the Pearson correlation coefficient analysis method is used to conduct correlation analysis on characteristics such as temperature, humidity, precipitation, wind speed, light, holiday information, population size, energy policy, industrial activities, etc., and the correlation and information measurement results of each characteristic with electricity, heat, and hydrogen loads and between electricity, heat, and hydrogen loads are compared. Finally, temperature, humidity, light, and wind speed are selected as the most significant external influencing characteristics, and the external characteristic data are used as input for subsequent prediction models. At the same time, considering the complex coupling relationship between multiple loads, historical heat, electricity, and hydrogen load data are also used as input.

[0011] Considering that external characteristics such as temperature may have uncertainties such as noise and prediction errors, this uncertainty will be directly transmitted to the load forecasting model, resulting in inaccurate forecast results. In order to measure the impact of this uncertainty, uncertainty modeling is performed on external characteristics (temperature, humidity, wind speed, and light). External characteristics are regarded as random variables that obey a normal distribution, and the mean and variance of these characteristics are estimated as random variables, thereby effectively evaluating the fluctuation range of the input data. The details are as follows:

[0012]

[0013]

[0014]

[0015]

[0016] Among them, T t+k , H t+k , W t+k , S t+k are the probability distributions of temperature, humidity, wind speed, and light at the future time t+k, k is the time step, μ is the corresponding feature mean, σ 2 is the corresponding uncertainty variance. By sampling the sequence multiple times, such as sampling the temperature feature, we can get The input sequence corresponding to the external features at time t+k, where n is the number of sampling times.

[0017] Furthermore, in step 2, the core idea of ​​multi-task learning is to process multiple related tasks simultaneously through sharing and specific network structures. Multiple tasks share parameters in the same model, thereby capturing the common features of different tasks and improving the learning efficiency and generalization ability of the model. In the present invention, the multi-task learning model is MTL-LSTM, which is used to capture the commonalities and characteristics between heat, electricity, and hydrogen load prediction tasks. The model is divided into two parts: a shared part and a specific part. The network structure of the shared part is an LSTM network, which is used to process the shared information of heat, electricity, and hydrogen multi-load prediction tasks; the specific part structure is an independent fully connected layer corresponding to each electricity, heat, and hydrogen load, which is used to learn the proprietary information of each prediction task. The shared layer is specifically expressed as follows:

[0018]

[0019] h t =f(u t ,h t-1 ,θ sh )

[0020] Among them, u t is the predicted input at time t, is the electricity, heat and hydrogen load data at time t, T t , H t , W t , S t is the temperature, humidity, wind speed, and light data at time t, is the mth sampling data of temperature, humidity, wind speed, and light at time t+k in the future, h t 、h t-1 is the hidden state output of the LSTM layer at time t and time t-1, f(·) is the shared layer LSTM network function, θ sh Share parameters in LSTM layer for electricity, heat, and hydrogen prediction tasks.

[0021] The shared layer output h t , after being mapped through the specific task layer, i.e. the fully connected layer, the load forecast values ​​are obtained. The specific task layer is specifically expressed as follows:

[0022]

[0023] in, is the load forecast output at time t+1, g(·) is the fully connected layer network function, θ i These are the parameters related to the electricity, heat, and hydrogen load prediction tasks in the fully connected layer.

[0024] Modeling parameter uncertainty in MTL-LSTM based on the basic principle of BNN: By modeling parameters such as weights and biases in LSTM as probability distributions This introduces uncertainty estimation of the model. During the training process, for each iteration, the model samples the parameter θ from the posterior distribution. sh , use the sampled parameters for forward propagation to calculate the load forecast value of multiple tasks. The loss function is defined as the negative log-likelihood function, which is specifically expressed as follows:

[0025]

[0026]

[0027] Among them, L i is the loss function for each prediction task, is the actual value of each prediction task at time t+1, is the predicted value of each prediction task at time t+1, is the variance of each prediction task, L total is the overall joint loss function, λ i is the corresponding task weight coefficient. Finally, the parameters are updated through error back propagation to obtain the approximate posterior probability distribution of the model parameters.

[0028] During the prediction process, the model obtains the posterior probability distribution of the model parameters obtained by training by sampling The mean and variance of the load forecasting results are obtained by using multiple sampling parameters for calculation. The specific expression is as follows:

[0029]

[0030]

[0031]

[0032]

[0033] in, is the hidden state output at the jth sampling time step t, is the predicted mean value of each prediction task at time t+1, is the j-th sampling parameter of the shared layer, is the predicted value under the j-th sampling parameter.

[0034] Furthermore, in step 3, the LSTM layer is used as a shared layer to generate the hidden state h for each time step. t, these hidden states are comprehensive expressions of historical inputs at the current time t, including the correlation between electricity, heat, and hydrogen loads and the impact of external features on loads. However, in long time series, features at different time steps have different importance for the prediction task. The introduction of the attention mechanism assigns different weights to different time steps of the input sequence, allowing the model to focus on information that is more important to the prediction task, thereby improving the prediction performance of the model.

[0035] The attention mechanism calculates the correlation between the current time step t and the historical time steps, and assigns weights to different time steps according to the correlation to generate comprehensive features. Specifically, for a given current time step hidden state h t and the hidden states {h1,h2,…h t-1}, the attention mechanism calculates the attention scores of the current hidden state and the historical hidden state, which is specifically expressed as follows:

[0036] e t,i =F(h t ,h i )

[0037] Among them, e t,i is the attention score of the current time step t and the historical time step i, and F(·) is the attention scoring function.

[0038] Use the softmax function to normalize the attention score and convert it into attention weight α t,i , so that the sum of different weights is 1, which is specifically expressed as follows:

[0039]

[0040] Among them, the attention weight α t,i Indicates the importance of the model to the current load forecasting task at time step i.

[0041] Finally, based on the attention weight α t,i Perform weighted summation on all input sequences and obtain the weighted input to obtain the comprehensive feature c containing historical information t , specifically expressed as follows:

[0042]

[0043] The context vector c t Combined with the current hidden state to generate a new hidden state Replace the original h t , specifically expressed as follows;

[0044]

[0045] The new hidden state not only contains the feature information of the current time step, but also combines the more important feature information in the historical time step, which is input into the specific task layer for the final load forecasting, so that the model can predict the results more accurately.

[0046] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0047] (1) A multi-task learning framework is used for multi-load forecasting. The MTL-LSTM model is used to capture the complex coupling relationship between electricity, heat, and hydrogen loads. The shared information between different tasks is used to enhance the prediction accuracy of each prediction task.

[0048] (2) Consider uncertainty factors, including input uncertainty and model parameter uncertainty. Model the uncertainty of input features, quantify the prediction deviation of input features (such as temperature and humidity) due to noise, forecast error and other factors on electricity, heat and hydrogen load forecasting, and improve the reliability of the forecast; based on the BNN principle, model the uncertainty of MTL-LSTM model parameters, quantify the uncertainty risk of load forecasting, and improve the generalization ability of the model.

[0049] (3) AM helps the model focus on the key time step features that have a greater impact on the future in load forecasting by assigning weights, thereby improving the model's ability to pay attention to important moments, reducing unnecessary information interference, and improving the model's prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the structure of a regional integrated energy system constructed according to an embodiment of the present invention;

[0051] Figure 2 It is a schematic diagram of the structure of the MTL-LSTM prediction model built in an embodiment of the present invention;

[0052] Figure 3 is a flow chart of model parameter uncertainty quantification proposed in an embodiment of the present invention;

[0053] Figure 4 It is an overall prediction flow chart of the prediction method proposed in the embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further elaborated in detail below in combination with the implementation modes and the accompanying drawings.

[0055] The embodiment of the present invention provides a method suitable for multi-energy load forecasting. The method first builds a regional comprehensive energy system model, such as Figure 1As shown. Analysis shows that hydrogen can be generated by electrolyzing water with electricity in an electrolyzer; hydrogen can not only be converted into electrical energy through a hydrogen fuel cell, but also generate a large amount of heat energy at the same time, which can be used for heating or heating. In addition, external weather factors, such as light intensity, temperature, humidity, etc., directly affect the working efficiency of photovoltaic and wind power generation equipment, and thus have a significant impact on the generation of electricity and the conversion of other energy sources in the system. In an integrated energy system, external factors have an important impact on energy loads, and there is also a complex coupling relationship between different energy loads such as electricity, heat, and hydrogen. This mutual influence will further aggravate the load fluctuations of the system. Therefore, in order to achieve more accurate load forecasting, it is necessary to comprehensively consider the coupling relationship between these external environmental factors and loads and analyze the relative impact of each factor.

[0056] As a possible implementation method, the specific implementation steps of the present invention include:

[0057] S1. Analyze the impact of different factors on the prediction of multiple loads such as electricity, heat, and hydrogen, select the most significant influencing features, model the uncertainty of the selected future moment features, and establish its probability distribution model.

[0058] The Pearson correlation coefficient analysis method is used to conduct correlation analysis on characteristics such as temperature, humidity, precipitation, wind speed, light, holiday information, population size, energy policy, industrial activities, etc. The specific expression is as follows, taking temperature as an example:

[0059]

[0060] Among them, γ is the Pearson correlation coefficient, T t and is the temperature and load observation value corresponding to time t, and is the average value of temperature and corresponding load.

[0061] By comparing the correlation and information measurement results of each feature with electricity, heat, hydrogen loads, and among electricity, heat, and hydrogen loads, we finally selected temperature, humidity, light, and wind speed as the most significant external influencing features and used them as inputs for subsequent prediction models. At the same time, considering the complex coupling relationship between multiple loads, historical heat, electricity, and hydrogen load data were also used as inputs.

[0062] Considering that external characteristics such as temperature may have uncertainties such as noise and prediction errors, this uncertainty will be directly transmitted to the load forecasting model, resulting in inaccurate forecast results. In order to measure the impact of this uncertainty, uncertainty modeling is performed on external characteristics (temperature, humidity, wind speed, and light). External characteristics are regarded as random variables that obey a normal distribution, and the mean and variance of these characteristics are estimated as random variables, thereby effectively evaluating the fluctuation range of the input data. The specific expression is as follows:

[0063]

[0064]

[0065]

[0066]

[0067] Among them, T t+k , H t+k , W t+k , S t+k are the probability distributions of temperature, humidity, wind speed, and light at the future time t+k, k is the time step, μ is the corresponding feature mean, σ 2 is the corresponding uncertainty variance. By sampling the sequence multiple times, such as sampling the temperature feature, we can get The input sequence corresponding to the external features at time t+k, where n is the number of sampling times.

[0068] S2, through the multi-task learning model MTL-LSTM, the structure is as follows Figure 2 As shown, it is used to capture the commonalities and characteristics between heat, electricity, and hydrogen load forecasting tasks. The model is divided into two parts: a shared part and a specific part. The shared part network structure is an LSTM network, which is used to process the shared information of heat, electricity, and hydrogen multi-load forecasting tasks; the specific part structure is an independent fully connected layer corresponding to each electricity, heat, and hydrogen load, which is used to learn the proprietary information of each forecasting task. The shared layer function is specifically expressed as follows:

[0069]

[0070] h t =f(u t ,h t-1 ,θ sh )

[0071] Among them, u t is the predicted input at time t, is the electricity, heat and hydrogen load data at time t, T t , H t , W t , S t is the temperature, humidity, wind speed, and light data at time t, is the mth sampling data of temperature, humidity, wind speed, and light at time t+k, h t 、h t-1 is the hidden state output of the LSTM layer at time t and time t-1, f(·) is the shared layer LSTM network function, θ sh Share parameters in LSTM layer for electricity, heat, and hydrogen prediction tasks.

[0072] The shared layer output h t , after being mapped through the specific task layer, i.e. the fully connected layer, the load forecast values ​​are obtained. The specific task layer is specifically expressed as follows:

[0073]

[0074] in, is the load forecast output at time t+1, g(·) is the fully connected layer network function, θ i These are the parameters related to the electricity, heat, and hydrogen load prediction tasks in the fully connected layer.

[0075] Based on the BNN principle, the parameter uncertainty in MTL-LSTM is modeled, such as Figure 3 As shown in Figure 2, by modeling the parameters such as weights and biases in LSTM as probability distributions This introduces uncertainty estimation of the model. During the training process, for each iteration, the model samples the parameter θ from the posterior distribution. sh , use the sampled parameters for forward propagation to calculate the load forecast value of multiple tasks. The loss function is defined as the negative log-likelihood function, which is specifically expressed as follows:

[0076]

[0077]

[0078] Among them, L i is the loss function of each prediction task, L total is the overall joint loss function, λ i is the corresponding task weight coefficient, is the actual value of each prediction task at time t+1, is the predicted value of each prediction task at time t+1, is the variance of each prediction task. Finally, the parameters are updated through error back propagation to obtain the approximate posterior probability distribution of the model parameters.

[0079] During the prediction process, the model obtains the posterior probability distribution of the model parameters obtained by training by sampling The mean and variance of the load forecasting results are obtained by using multiple sampling parameters for calculation. The specific expression is as follows:

[0080]

[0081]

[0082]

[0083]

[0084] in, is the hidden state output at the jth sampling time step t, is the predicted mean value of each prediction task at time t+1, is the jth sampling parameter, is the predicted value under the j-th sampling parameter, and n is the number of sampling times.

[0085] S3. Introduce the attention mechanism, by assigning different weights to different time steps of the input sequence, so that the model focuses on the information that is more important to the prediction task, thereby improving the prediction performance of the model.

[0086] The attention mechanism calculates the correlation between the current time step t and the historical time steps, and assigns weights to different time steps according to the correlation to generate comprehensive features. Specifically, for a given current time step hidden state h t and the hidden states {h1,h2,…h t-1}, the attention mechanism calculates the attention scores of the current hidden state and the historical hidden state, which is specifically expressed as follows:

[0087] e t,i =F(h t ,h i )

[0088] Among them, e t,i is the attention score of the current time step t and the historical time step i, and F(·) is the attention scoring function.

[0089] Use the softmax function to normalize the attention score and convert it into attention weight α t,i , so that the sum of different weights is 1, which is specifically expressed as follows:

[0090]

[0091] Among them, the attention weight α t,i Indicates the importance of the model to the current load forecasting task at time step i.

[0092] Finally, based on the attention weight α t,i Perform weighted summation on all input sequences and obtain the weighted input to obtain the comprehensive feature c containing historical information t , specifically expressed as follows:

[0093]

[0094] The context vector c t Combined with the current hidden state to generate a new hidden state Replace the original ht , as follows;

[0095]

[0096] The new hidden state not only contains the feature information of the current time step, but also combines the more important feature information in the historical time step, which is input into the specific task layer for the final load forecasting, so that the model can predict the results more accurately.

[0097] In the embodiment of the present invention, the implementation process is as follows Figure 4 As shown in the figure, the uncertainty of input data is modeled, sampled and input into the prediction model with historical data. The prediction model is BNN+MTL-LSTM+AM. BNN is used to quantify the deviation caused by model parameter uncertainty on load prediction and enhance the robustness of the model; MTL-LSTM captures the complex coupling relationship between electricity, heat and hydrogen loads, and uses each fully connected layer to learn the time series characteristics of each load; AM mechanism focuses on the information that has a greater impact on future load prediction by giving higher weight to key time step features. The overall accuracy and reliability of multi-energy load prediction are improved.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0099] The above are only some embodiments of the present invention. For those skilled in the art, several modifications and improvements can be made without departing from the creative concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A multi-energy load forecasting method considering multiple uncertainties, characterized in that: The following steps are involved: Step 1: Analyze the impact of different factors on the electricity, heat and hydrogen load forecast, select the most significant influencing features, perform uncertainty modeling on the selected external features, and establish their probability distribution model; Step 2: Build a Multi-Task Learning Long Short-Term Memory (MTL-LSTM) model based on Bayesian Neural Network (BNN) to quantify the uncertainty of model parameters and capture the coupling relationship between multiple loads through a multi-task learning framework; Step 3: Combine the attention mechanism to enhance the model's attention to key time step features and improve the model's prediction accuracy.

2. The method according to claim 1, characterized in that: In step 1, the Pearson correlation coefficient analysis method is used to conduct correlation analysis on characteristics such as temperature, humidity, precipitation, wind speed, light, holiday information, population size, energy policy, and industrial activities. The correlation and information measurement results of each characteristic and the electricity, heat, and hydrogen loads, as well as the electricity, heat, and hydrogen loads are compared. Finally, temperature, humidity, light, and wind speed are selected as the most significant external influencing characteristics, and the external characteristic data are used as input for the subsequent prediction model. At the same time, considering the complex coupling relationship between multiple loads, the historical heat, electricity, and hydrogen load data are also used as input. Considering that external characteristics such as temperature may have uncertainties such as noise and prediction errors, this uncertainty will be directly transmitted to the load forecasting model, resulting in inaccurate forecast results. In order to measure the impact of this uncertainty, uncertainty modeling is performed on external characteristics (temperature, humidity, wind speed, and light). External characteristics are regarded as random variables that obey a normal distribution, and the mean and variance of these characteristics are estimated as random variables, thereby effectively evaluating the fluctuation range of the input data. The details are as follows: Among them, T t+k , H t+k , W t+k , S t+k are the probability distributions of temperature, humidity, wind speed, and light at the future time t+k, k is the time step, μ is the corresponding feature mean, σ 2 is the corresponding uncertainty variance. By sampling the sequence multiple times, such as sampling the temperature feature, we can get The input sequence corresponding to the external features at time t+k, where n is the number of sampling times.

3. The method according to claim 1, characterized in that: In step 2, the core idea of ​​multi-task learning is to process multiple related tasks simultaneously through sharing and specific network structures. Multiple tasks share parameters in the same model, thereby capturing the common features of different tasks and improving model learning efficiency and generalization ability. In the present invention, the multi-task learning model is MTL-LSTM, which is used to capture the commonalities and characteristics between heat, electricity, and hydrogen load prediction tasks. The model is divided into two parts: a shared part and a specific part. The network structure of the shared part is an LSTM network, which is used to process the shared information of heat, electricity, and hydrogen multi-load prediction tasks; the specific part structure is an independent fully connected layer corresponding to each electricity, heat, and hydrogen load, which is used to learn the proprietary information of each prediction task. The shared layer is specifically expressed as follows: h t =f(u t ,h t-1 ,θ sh ) Among them, u t is the predicted input at time t, is the electricity, heat and hydrogen load data at time t, T t , H t , W t , S t is the temperature, humidity, wind speed, and light data at time t, is the mth sampling data of temperature, humidity, wind speed, and light at time t+k in the future, h t 、h t-1 is the hidden state output of the LSTM layer at time t and time t-1, f(·) is the shared layer LSTM network function, θ sh Share parameters in LSTM layer for electricity, heat, and hydrogen prediction tasks. The shared layer output h t , after being mapped through the specific task layer, i.e. the fully connected layer, the load forecast values ​​are obtained. The specific task layer is specifically expressed as follows: in, is the load forecast output at time t+1, g(·) is the fully connected layer network function, θ i These are the parameters related to the electricity, heat, and hydrogen load prediction tasks in the fully connected layer. Modeling parameter uncertainty in MTL-LSTM based on the BNN principle: By modeling parameters such as weights and biases in LSTM as probability distributions This introduces uncertainty estimation of the model. During the training process, for each iteration, the model samples the parameter θ from the posterior distribution. sh , use the sampled parameters for forward propagation to calculate the load forecast value of multiple tasks. The loss function is defined as the negative log-likelihood function, which is specifically expressed as follows: Among them, L i is the loss function for each prediction task, is the actual value of each prediction task at time t+1, is the predicted value of each prediction task at time t+1, is the variance of each prediction task, L total is the overall joint loss function, λ i is the corresponding task weight coefficient. Finally, the parameters are updated through error back propagation to obtain the approximate posterior probability distribution of the model parameters. During the prediction process, the model obtains the posterior probability distribution of the model parameters obtained by training by sampling Use multiple sampling parameters to calculate and obtain the mean and variance of the load forecast results. The specific expression is as follows: in, is the hidden state output at the jth sampling time step t, is the predicted mean value of each prediction task at time t+1, is the j-th sampling parameter of the shared layer, is the predicted value under the j-th sampling parameter.

4. The method according to claim 1, characterized in that: In step 3, the LSTM layer is used as a shared layer to generate the hidden state h for each time step. t , these hidden states are comprehensive expressions of historical inputs at the current time t, including the correlation between electricity, heat, and hydrogen loads and the impact of external features on loads. However, in long time series, features at different time steps have different importance for the prediction task. The introduction of the attention mechanism assigns different weights to different time steps of the input sequence, allowing the model to focus on information that is more important to the prediction task, thereby improving the prediction performance of the model. The attention mechanism calculates the correlation between the current time step t and the historical time steps, and assigns weights to different time steps according to the correlation to generate comprehensive features. Specifically, for a given current time step hidden state h t and the hidden states {h1,h2,…h t-1 }, the attention mechanism calculates the attention scores of the current hidden state and the historical hidden state, which is specifically expressed as follows: e t,i =F(h t ,h i ) Among them, e t,i is the attention score of the current time step t and the historical time step i, and F(·) is the attention scoring function. Use the softmax function to normalize the attention score and convert it into attention weight α t,i , so that the sum of different weights is 1, which is specifically expressed as follows: Attention weight α t,i Indicates the importance of the model to the current load forecasting task at time step i. Finally, based on the attention weight α t,i Perform weighted summation on all input sequences and obtain the weighted input to obtain the comprehensive feature c containing historical information t , specifically expressed as follows: The context vector c t Combined with the current hidden state to generate a new hidden state Replace the original h t , specifically expressed as follows; The new hidden state not only contains the feature information of the current time step, but also combines the more important feature information in the historical time step, which is input into the specific task layer for the final load forecasting, so that the model can predict the results more accurately.

Images