A hydrogen energy system scheduling method with adaptive multi-step comprehensive vision

Abstract

The application discloses a hydrogen energy system scheduling method with self-adaptive multi-step comprehensive vision, comprising the following steps: 1. establishing a hydrogen energy comprehensive energy system model suitable for the MATD3 method, and describing the equipment operation characteristics; 2. analyzing the scheduling performance from three dimensions of equipment loss, load balance and carbon emission, and determining an optimization objective function; 3. using historical data sets, constructing a probability interval through GPR, and combining a Transformer-LSTM model to realize short-term deterministic multi-step prediction and long-term interval multi-step prediction; 4. based on the prediction results and historical data, using the MATD3 method to optimize the output of each module equipment of the hydrogen energy system and the energy storage behavior, and realizing the synchronous training of the MATD3 method and the prediction model through reward value transmission, so that the overall decision model has self-adaptive multi-step comprehensive vision, and the accuracy and flexibility of scheduling decisions are improved. Through multi-model coupling, the hydrogen energy system's uncertainty processing capability and scheduling optimization performance are enhanced, and the system operation performance is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of time series prediction and intelligent decision-making, in particular to a hydrogen energy system scheduling method with adaptive multi-step comprehensive vision combining deep learning prediction capability and deep reinforcement learning decision-making capability. BACKGROUND

[0002] With the rapid development of hydrogen energy technology and the transformation of energy structure, hydrogen energy has been widely recognized as one of the key elements in the future comprehensive energy system. Hydrogen energy system contains a complete industrial chain from production, storage, transportation to use, which has important significance in dealing with energy diversification and achieving carbon neutralization goal. However, the high uncertainty of renewable energy supply fluctuation and load random change makes it difficult for traditional fixed scheduling mode to adapt to the dynamic characteristics of hydrogen energy system, thereby affecting the efficiency and stability of the system.

[0003] Current hydrogen energy scheduling decision is usually based on historical production and demand data, and is predicted and scheduled by static modeling method. However, hydrogen supply and demand are affected by many complex factors such as weather conditions, system state, user demand, etc., making it difficult for traditional methods to make scheduling decisions to meet the flexible and variable demand. At the same time, the production, storage and demand of hydrogen energy have high nonlinear characteristics, especially in the scene of coupling of multiple energy forms (such as electricity, heat), the multi-scale variation law of hydrogen energy system is difficult to be effectively captured by a single traditional prediction method, resulting in low reliability and efficiency of system scheduling.

[0004] In order to improve the flexibility of hydrogen energy system scheduling decision, existing research begins to introduce deep learning method for energy supply and demand prediction. However, deep learning model can only provide prediction of future supply and demand trend, and cannot be directly used for actual scheduling decision, which limits the dynamic response capability of hydrogen energy system. Therefore, how to combine intelligent prediction and adaptive scheduling to cope with complex supply and demand fluctuations has become a key problem for efficient use of hydrogen energy system. SUMMARY

[0005] The present application overcomes the deficiencies in the prior art and provides a hydrogen energy system scheduling method with adaptive multi-step comprehensive vision, which can improve the supply and demand matching efficiency and new energy utilization rate of the hydrogen energy system while maintaining low-carbon operation of the system, thereby enhancing the uncertainty processing capability and scheduling decision accuracy and flexibility of the hydrogen energy system, and effectively improving the system operation performance.

[0006] To achieve the above purpose, the technical solution adopted by the present application is as follows:

[0007] The application discloses a hydrogen energy comprehensive energy system scheduling method with self-adaptive multi-step comprehensive view, and the hydrogen energy comprehensive energy system comprises the following modules: a renewable energy module, an electrolytic hydrogen production module, a fuel cell module, a heat pump electricity supply heat module, an electric energy storage module and a hydrogen energy storage module.

[0008] Step one, a simulation model of the hydrogen energy comprehensive energy system is established.

[0009] Step two, based on the simulation model, a scheduling performance evaluation index of the hydrogen energy comprehensive energy system is constructed.

[0010] Step three, a short-term deterministic multi-step prediction model and a long-term interval multi-step prediction model are trained.

[0011] Step four, based on the scheduling performance evaluation index and the two models in step three, a scheduling model is constructed and trained by using a multi-agent double-delay deep deterministic policy gradient algorithm, so that an optimal scheduling decision model is obtained, and a hydrogen energy comprehensive energy system scheduling scheme is determined.

[0012] The hydrogen energy comprehensive energy system scheduling method with self-adaptive multi-step comprehensive view has the following characteristics.

[0013] Step 1.1, a simulation model of the renewable energy module is established by using formula (1) :

[0014] [ p t RE , c t RE ] = D e v i c e RE ( a t RE , W t ) (1)

[0015] In formula (1), represents a decision action of the renewable energy module at t time; represents an electric power output of the renewable energy module at t time; represents equipment loss of the renewable energy module at t time; represents a meteorological environmental parameter affecting the renewable energy module at t time;

[0016] Step 1.2, a simulation model of the electrolytic hydrogen production module is established by using formula (2) :

[0017] [ p t EL , c t EL , η t EL ] = D e v i c e EL ( a t EL ) (2)

[0018] In formula (2), represents a decision action of the electrolytic hydrogen production module at t time; represents an electric power consumption of the electrolytic hydrogen production module at t time; represents equipment loss of the electrolytic hydrogen production module at t time; P (t) represents the equivalent power of hydrogen production amount of the electrolytic hydrogen production module at time t;

[0019] Step 1.3, establishing a simulation model of the fuel cell module by using formula (3)

[0020] [ p t FC , c t FC , h t FC , η t FC ] = D e v i c e FC ( a t FC ) (3)

[0021] In formula (3), P (t) represents the decision action of the fuel cell module at time t; P (t) represents the electric power output of the fuel cell module at time t; P (t) represents the equipment loss of the fuel cell module at time t; P (t) represents the thermal power output of hydrogen production amount of the fuel cell module at time t; P (t) represents the equivalent power of hydrogen consumption amount of the fuel cell module at time t;

[0022] Step 1.4, establishing a simulation model of the heat pump electric heat supply module by using formula (4)

[0023] [ p t HP , c t HP , h t HP ] = D e v i c e H P ( a t HP ) (4)

[0024] In formula (4), P (t) represents the decision action of the electric heat supply module at time t; P (t) represents the electric power consumption of the electric heat supply module at time t; P (t) represents the equipment loss of the electric heat supply module at time t; P (t) represents the thermal power output of the electric heat supply module at time t;

[0025] Step 1.5, establishing a simulation model of the electric energy storage module by using formula (5)

[0026] [ p t ES , c t ES , s t + 1 ES ] = D e v i c e E S ( a t ES , s t ES ) (5)

[0027] In formula (5), P (t) represents the decision action of the electric energy storage module at time t; P (t) represents the initial capacity of the electric energy storage module at time t; P (t) represents the electric power change amount of the electric energy storage module at time t, and a positive number represents output electric power and a negative number represents input electric power; P (t) represents the equipment loss of the electric energy storage module at time t; P (t) represents the initial capacity of the electric energy storage module at time t+1;

[0028] Step 1.6, establishing a simulation model of the hydrogen energy storage module by using formula (6) ​​​​

[0029] [ η t HS , c t HS , s t + 1 HS ] = D e v i c e H S ( a t HS , s t HS ) (6)

[0030] In formula (6), represents the decision action of the hydrogen energy storage module at time t; represents the initial capacity of the hydrogen energy storage module at time t; represents the equivalent hydrogen power change amount of the hydrogen energy storage module at time t, and a positive number represents output hydrogen power and a negative number represents input hydrogen power; represents the equipment loss of the hydrogen energy storage module at time t; represents the initial capacity of the hydrogen energy storage module at time t+1;

[0031] Step 1.7, establish the power balance constraints of the hydrogen energy comprehensive energy system by using formula (7)-(9), including: electric power balance constraint, thermal power balance constraint, hydrogen power balance constraint:

[0032] (7)

[0033] (8)

[0034] (9)

[0035] In formula (7)-(9), represents the power of the upper power grid at time t; represents the power of the upper heat grid at time t; represents the power of the upper hydrogen energy grid at time t; represents the electric load at time t; represents the thermal load at time t.

[0036] Further, the step two comprises:

[0037] Step 2.1, construct the loss percentage of all devices in the hydrogen energy comprehensive energy system at time t by using formula (10) :

[0038] (10)

[0039] In formula (10), represents the theoretical maximum loss of the hydrogen energy comprehensive energy system at any time;

[0040] Step 2.2, construct the new energy utilization rate of the hydrogen energy comprehensive energy system at time t by using formula (11) :

[0041] (11)

[0042] In equation (11), This represents the theoretical maximum output power of the renewable energy module at time t;

[0043] Step 2.3: Construct the carbon emission rate of the hydrogen energy integrated system at time t using equation (12). :

[0044] (12)

[0045] In equation (12), Indicates the carbon emission factor of the upstream power grid; Indicates the carbon emission factor of the upstream heating network; Indicates the carbon emission factor of the upper-level hydrogen energy network;

[0046] Step 2.4: Calculate the performance evaluation index of the hydrogen energy integrated system at time t using equation (13). :

[0047] (13).

[0048] Furthermore, step three includes:

[0049] Step 3.1: Obtain the historical data set of the hydrogen energy integrated energy system. ,in, Representing historical datasets The total sampling time; Represents calendar information sampled at time t; Represents the external parameters at time t; This represents a historical data sample at time t;

[0050] Step 3.2: Denote the target parameter at time t as... ; Get the time before time t 1 historical data and construct t- The parameters other than the target parameters up to time t-1 constitute the observation dataset. and by t- The target parameters from time -1 to t-1 constitute the target dataset. ,in, This represents historical data samples from time t to time i. Remove target parameters at time i The resulting sample;

[0051] Given the observation data consisting of parameters other than the target parameter at time t. Then, calculate the probability distribution function of the target parameters at time t according to equation (14). This includes: meteorological environmental parameters at time t. probability distribution function of the meteorological environment parameter , the electric load probability distribution function of the meteorological environment parameter , the thermal load probability distribution function of the meteorological environment parameter ;

[0052] (14)

[0053] In formula (14), represents a Gaussian process regression function;

[0054] Step 3.3, according to the probability distribution function of the meteorological environment parameter at time t , the electric load probability distribution function of the meteorological environment parameter , the thermal load probability distribution function of the meteorological environment parameter , the upper limit of the interval of the meteorological environment parameter at time t is obtained , the lower limit of the interval of the meteorological environment parameter at time t is obtained , the upper limit of the interval of the electric load at time t is obtained , the lower limit of the interval of the electric load at time t is obtained , the upper limit of the interval of the thermal load at time t is obtained , the lower limit of the interval of the thermal load at time t is obtained , and the historical data set is combined, so as to obtain a data set containing interval information ;

[0055] Step 3.4, the length of the time window is set to , and the data set containing interval information is processed by a sliding window, so as to obtain a time sequence segment of the historical data at time t , a short-term deterministic time sequence segment of the target parameter to be predicted at time t and a long-term interval time sequence segment of the target parameter to be predicted at time t , so as to obtain a time sequence segment set of the historical data , a short-term deterministic time sequence segment set of the target parameter to be predicted , and a long-term interval time sequence segment set of the target parameter to be predicted , wherein, represents the historical data sample at time j, represents the meteorological environment parameter at time j, represents the electric load at time j, represents the thermal load at time j, represents the upper limit of the interval of the meteorological environment parameter at time j, represents the lower limit of the interval of the meteorological environment parameter at time j,​ represents the upper bound of the electrical load interval at time j, represents the lower bound of the electrical load interval at time j, represents the upper bound of the thermal load interval at time j, represents the lower bound of the thermal load interval at time j;

[0056] Step 3.5, constructing a short-term deterministic multi-step prediction model based on a Transformer network and a long short-term memory network (LSTM) ; and taking and as the input and target output of respectively, so as to update the parameters of using formula (17) to obtain updated parameters :

[0057] (15)

[0058] In formula (15), represents the mean square error loss function of the short-term deterministic multi-step prediction model;

[0059] Step 3.6, obtaining the short-term deterministic multi-step prediction value at time t using formula (16) :

[0060] (16)

[0061] Step 3.7, constructing a long-term interval multi-step prediction model based on Transformer-LSTM ; and taking and as the input and target output of respectively, so as to update the parameters of using formula (17) to obtain updated parameters :

[0062] (17)

[0063] In formula (17), represents the mean square error loss function of the long-term interval multi-step prediction model;

[0064] obtaining the long-term interval multi-step prediction value at any time t using formula (18) :

[0065] (18).

[0066] Further, the step four comprises:

[0067] Step 4.1, assign the decision variables of time t to the three agents according to formula (19) and form the decision variable set of time t ;

[0068] (19)

[0069] In formula (19), represents the decision variable of the first agent agent1 at time t; represents the decision variable made by the second agent agent2 at time t; represents the decision variable made by the third agent at time t;

[0070] Step 4.2, construct the current state observation of the three agents at time t and input into the scheduling decision model for processing, and output the decision variable set at time t ;

[0071] Step 4.3, bring into step one and step two to obtain the performance evaluation index at time t and as the reward value of each agent at time t, while obtaining the initial capacity of the electric energy storage module at time t+1 and the initial capacity of the hydrogen energy storage module at time t+1 ;

[0072] Step 4.4, obtain the time sequence segment of the historical data at time t+1 , and input and for processing, respectively, to obtain the short-term deterministic time sequence segment prediction value at time t+1 and the long-term interval time sequence segment prediction value at time t+1 ; thus obtaining the state observation of the three agents at time t+1 ; record the four-tuple form data into the experience replay pool;

[0073] Step 4.5, extract a training batch set from the experience replay pool and input into the scheduling decision model for training, so as to update the parameters of the scheduling decision model constructed by formula (23) , wherein, represents the current state observation set of the three agents, represents the current action decision set of the three agents, represents the current reward value set of the three agents, This represents the set of state observations for the three agents at the next moment.

[0074] (20)

[0075] In equation (23), This indicates that the updated optimal parameters have been obtained. represents the target parameter of the scheduling decision model; m represents the agent number; This represents the set of action decisions for the three agents at the next moment. The Q-function of the m-th agent; This represents the policy function of the m-th agent; Indicates the discount factor; Noise representing the target strategy;

[0076] Step 4.6: Construct short-term deterministic multi-step prediction models at time t using equations (21) and (22) respectively. and long-term interval multi-step prediction model The updated formula for the mean squared error loss function is as follows:

[0077] (twenty one)

[0078] (twenty two)

[0079] In equation (22), This indicates a short-term deterministic multi-step prediction model. The updated mean squared error loss function, This represents a long-term interval multi-step prediction model at time t. The updated mean squared error loss function; Indicates weight;

[0080] Step 4.5: Follow the process in steps 3.5-4.6. , The scheduling decision model is iteratively updated until the maximum number of iterations is reached or the scheduling decision model tends to stabilize, thereby obtaining the optimal scheduling decision model. This model is used to output the scheduling actions of the renewable energy module, electrolysis hydrogen production module, fuel cell module, heat pump electric heating module, electric energy storage module, and hydrogen energy storage module, in order to determine the scheduling scheme of the integrated hydrogen energy system.

[0081] The present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor in executing the hydrogen energy system scheduling method, and the processor is configured to execute the program stored in the memory.

[0082] The computer readable storage medium stores a computer program, and when the computer program is run by a processor, the steps of the hydrogen energy system scheduling method are executed.

[0083] Compared with the prior art, the beneficial effects of the present application are reflected in the following aspects:

[0084] 1The present application is aimed at the uncertainty fluctuations in the hydrogen energy system, constructs time sequence segments of historical meteorological data and predicted targets, analyzes the interval characteristics of historical data using GPR technology, captures long-term trends and local fluctuation characteristics in the data using a Transformer-LSTM model, and predicts multi-step short-term deterministic fluctuation trends and long-term interval fluctuation trends of key data based on historical data, thereby giving a multi-step comprehensive view to scheduling decisions and improving decision accuracy.

[0085] 2The present application constructs a hydrogen energy system scheduling decision model through the MATD3 method, includes renewable energy power supply, electric energy storage, hydrogen energy storage and heating decision into the scheduling decision model, constructs coupled decisions in different action fields during operation through multi-agent decision grouping training, ensures the balance constraint of electricity, heat and hydrogen energy, and realizes scheduling decision optimization of the hydrogen energy system with the goal of improving the comprehensive performance of the system.

[0086] 3The present application couples deep learning methods and deep reinforcement learning methods, provides a multi-step comprehensive view of future renewable energy and load demand estimation for the Transformer-LSTM prediction model, and the MATD3 model is based on predicted data and historical data to make scheduling decisions, and the decision reward value is transmitted back to the Transformer-LSTM prediction model to optimize model parameters while optimizing the MATD3 model, thereby realizing deep coupling and synchronous training of the two models, giving the decision method an adaptive comprehensive view and improving the flexibility of the decision. BRIEF DESCRIPTION OF DRAWINGS

[0087] Figure 1 A flowchart of the hydrogen energy system scheduling decision;

[0088] Figure 2 A structure diagram of the Transformer-LSTM and MATD3 coupled model. DETAILED DESCRIPTION

[0089] In this embodiment, the hydrogen energy comprehensive energy system includes: a renewable energy module, an electrolytic hydrogen production module, a fuel cell module, a heat pump electricity supply heat module, an electric energy storage module, and a hydrogen energy storage module. In view of the uncertainty of the hydrogen energy system and the optimization problem of the complex system decision, a hydrogen energy system scheduling decision method with adaptive multi-step comprehensive vision is constructed. Firstly, a hydrogen energy comprehensive energy system model in the reinforcement learning operating environment is established to describe the operating characteristics of each sub-module of the system. Then, the scheduling performance is analyzed from three dimensions of equipment loss, load supply balance and carbon emission. Based on the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS), the three index dimensions are integrated to determine the optimization objective function. Next, based on the normalization processing, the Gaussian Process Regression (GPR) is used to obtain the probability interval form, the data is processed to obtain the time sequence segment form, the data is segmented to obtain the short-term deterministic time sequence data form and the long-term interval time sequence data form of the target output. The Transformer model and the Long Short Term Memory Network (LSTM) are used to construct a Transformer-LSTM combined prediction model to capture the long-term trend and local fluctuation characteristics of the data to realize multi-step comprehensive prediction of various key data. Finally, the Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3) model is used to make decisions on the energy scheduling behaviors of the system, such as electricity, heat, hydrogen and storage. The MATD3 model and the Transformer-LSTM are deeply coupled to make the overall decision model have an adaptive multi-step comprehensive vision, so as to improve the accuracy and flexibility of the decision with the highest comprehensive scheduling performance as the goal. The specific process is shown in Figure 1 The method is performed according to the following steps:

[0090] Step 1, establish a simulation model of the hydrogen energy comprehensive energy system;

[0091] Step 1.1, although different energy systems have different component modules, the difference is only in the description of the parameters and formula characteristics, and the functions of the general model are consistent. The simulation model of the renewable energy module is established by using formula (1) :

[0092] [ p t RE , c t RE ] = D e v i c e RE ( a t RE , W t ) (1)

[0093] In formula (1), represents the decision action of the renewable energy module at time t, which is determined by the subsequent reinforcement learning part in step 1; represents the electrical power output of the renewable energy module at time t; represents the equipment loss of the renewable energy module at time t; represents the meteorological environmental parameters affecting the renewable energy module at time t, which can include wind speed, temperature, and irradiance, etc. according to different actual requirements.

[0094] Step 1.2, establish a simulation model of the electrolytic hydrogen production module using formula (2) :

[0095] [ p t EL , c t EL , η t EL ] = D e v i c e EL ( a t EL ) (2)

[0096] In formula (2), represents the decision action of the electrolytic hydrogen production module at time t; represents the electrical power consumption of the electrolytic hydrogen production module at time t; represents the equipment loss of the electrolytic hydrogen production module at time t; represents the equivalent power of the hydrogen production amount of the electrolytic hydrogen production module at time t;

[0097] Step 1.3, establish a simulation model of the fuel cell module using formula (3) :

[0098] [ p t FC , c t FC , h t FC , η t FC ] = D e v i c e FC ( a t FC ) (3)

[0099] In formula (3), represents the decision action of the fuel cell module at time t; represents the electrical power output of the fuel cell module at time t; represents the equipment loss of the fuel cell module at time t; represents the thermal power output of the hydrogen production amount of the fuel cell module at time t; represents the equivalent power of the hydrogen consumption amount of the fuel cell module at time t.

[0100] Step 1.4, establish a simulation model of the heat pump electric heat supply module using formula (4) :

[0101] [ p t HP , c t HP , h t HP ] = D e v i c e H P ( a t HP ) (4)

[0102] In formula (4), represents the decision action of the electric heat supply module at time t; represents the electrical power consumption of the electric heat supply module at time t; represents the equipment loss of the electric heat supply module at time t; represents the thermal power output of the electric heat supply module at time t;

[0103] Step 1.5, establishing a simulation model of the electric energy storage module by using formula (5)

[0104] [ p t ES , c t ES , s t + 1 ES ] = D e v i c e E S ( a t ES , s t ES ) (5)

[0105] In formula (5), represents the decision action of the electric energy storage module at time t; represents the initial capacity of the electric energy storage module at time t; represents the electric power change amount of the electric energy storage module at time t, and a positive number represents output electric power and a negative number represents input electric power; represents the equipment loss of the electric energy storage module at time t; represents the initial capacity of the electric energy storage module at time t+1.

[0106] Step 1.6, establishing a simulation model of the hydrogen energy storage module by using formula (6)

[0107] [ η t HS , c t HS , s t + 1 HS ] = D e v i c e H S ( a t HS , s t HS ) (6)

[0108] In formula (6), represents the decision action of the hydrogen energy storage module at time t; represents the initial capacity of the hydrogen energy storage module at time t; represents the equivalent hydrogen power change amount of the hydrogen energy storage module at time t, and a positive number represents output hydrogen power and a negative number represents input hydrogen power; represents the equipment loss of the hydrogen energy storage module at time t; represents the initial capacity of the hydrogen energy storage module at time t+1.

[0109] Step 1.7, establishing power balance constraints of the hydrogen energy comprehensive energy system by using formula (7)-(9), including: electric power balance constraint, thermal power balance constraint, and hydrogen power balance constraint:

[0110] (7)

[0111] (8)

[0112] (9)

[0113] In formula (7)-(9), represents the power of the upper-level power grid at time t; represents the power of the upper-level heat grid at time t; ​​P(t) represents the power of the upper hydrogen energy network at time t; P(t) represents the power of the upper hydrogen energy network at time t; P(t) represents the power of the upper hydrogen energy network at time t; In addition to the above constraints, the output limit constraint, the ramping power constraint, the energy storage capacity constraint, etc. are also included according to the differences in device characteristics, which are all included in the Device model of each device and will not be described again; Deep reinforcement learning converts the decision-making problem into several key concepts to construct a Markov decision process, which includes actions, agents, environments, states, and rewards; All the above models are included in the concept of environment, which simulates the specific scene to which the decision scheme is applied.

[0114] Step two, define the performance evaluation index of the hydrogen energy comprehensive energy system model scheduling;

[0115] This part will evaluate the scheduling results of the system at any time from three dimensions; Since the deep reinforcement learning method is combined, the three indexes need to be integrated into a reward with practical meaning, corresponding to the objective function of decision optimization;

[0116] Step 2.1, use formula (10) to construct the loss percentage of all devices in the hydrogen energy comprehensive energy system at time t :

[0117] (10)

[0118] In formula (10), P(t) represents the theoretical maximum loss of the hydrogen energy comprehensive energy system at any time;

[0119] Step 2.2, use formula (11) to construct the new energy utilization rate of the hydrogen energy comprehensive energy system at time t :

[0120] (11)

[0121] In formula (11), P(t) represents the theoretical maximum output power of the renewable energy module at time t.

[0122] Step 2.3, use formula (12) to construct the carbon emission rate of the hydrogen energy comprehensive energy system at time t :

[0123] (12)

[0124] In formula (12), P(t) represents the carbon emission factor of the upper power grid; P(t) represents the carbon emission factor of the upper heat grid; P(t) represents the carbon emission factor of the upper hydrogen energy network;

[0125] Step 2.4. Calculate the performance evaluation index of the hydrogen energy comprehensive energy system at time t using formula (13) :

[0126] (13)

[0127] Here, the idea of TOPSIS method is adopted, assuming that the performance index is a vector in the three-dimensional space defined in 2.1-2.3, and the scheduling performance is best when the three-dimensional space tends to zero; in order to adapt to the characteristics of reinforcement learning to maximize the reward value, the negative value of the length of the three-dimensional vector is used as the final evaluation index.

[0128] Step three, train short-term deterministic multi-step prediction model and long-term interval multi-step prediction model;

[0129] Step 3.1, obtain the historical data set of the hydrogen energy comprehensive energy system , wherein, represents the total time of the historical data set ; represents the calendar information sampled at time t; represents the remaining external parameters at time t, which directly or indirectly affect the prediction target, and the specific parameter types are not fixed and can be selected as needed; represents the historical data sample at time t;

[0130] Step 3.2, GPR is one of the most commonly used methods for interval analysis due to its good nonlinear relationship capturing ability and uncertainty modeling advantage; it uses kernel function to construct the covariance matrix between data pairs, establishes a non-parametric model of input and target output for regression analysis; the target parameter at time t can be represented as ; obtain historical data before time t to construct the observation data set of other parameters except the target parameter at time t-1 to t-1 , , represents the new sample formed after removing the target parameter at time i from the historical data sample at time i before time t .

[0131] When the observation data of other parameters except the target parameter at time t is given , the probability distribution function of the target parameter at time t is calculated according to formula (14) , including: the probability distribution function of the meteorological environmental parameter at time t Electrical load probability distribution function Heat load probability distribution function ;

[0132] (14)

[0133] In equation (14), all data are derived from historical datasets. , This represents the Gaussian process regression function.

[0134] Step 3.3: Based on the meteorological environmental parameters at time t probability distribution function Electrical load probability distribution function Heat load probability distribution function The upper bound of the meteorological and environmental parameter interval at time t is obtained accordingly. The lower bound of the meteorological environmental parameter interval at time t The upper boundary of the electrical load interval at time t The lower boundary of the electrical load range at time t The upper limit of the heat load range at time t The lower limit of the heat load range at time t and with historical datasets By combining the data, a dataset containing interval information can be obtained. .

[0135] Step 3.4: Time series segments can contain more complete fluctuation characteristics, therefore both the input and the target output are organized into time series segments; time series segments of historical data are constructed based on sliding time windows, and the length of the time window is set to [value missing]. And for datasets containing interval information Perform sliding window processing to obtain time series segments of historical data at time t. Short-term deterministic time series segment of the target parameter to be predicted at time t and the long-term interval time series segment of the target parameter to be predicted at time t This yields a set of time-series fragments of historical data. A set of short-term deterministic time series segments of the target parameters to be predicted A set of long-term interval time series segments of the target parameter to be predicted. ,in, This represents a historical data sample at time j. Represents the meteorological environmental parameters at time j. This represents the electrical load at time j. This represents the heat load at time j. an upper bound of the meteorological environment parameter interval at time j, a lower bound of the meteorological environment parameter interval at time j, an upper bound of the electrical load interval at time j, a lower bound of the electrical load interval at time j, an upper bound of the thermal load interval at time j, a lower bound of the thermal load interval at time j.

[0136] Step 3.5, as shown in Figure 2 , the Transformer part includes multiple encoder and decoder layers; each encoder layer is composed of a multi-head self-attention mechanism and a feedforward neural network, with residual connection and normalization operation; the structure of the decoder layer is similar to that of the encoder, except that it also contains an additional multi-head self-attention layer to handle the dependency between the encoder output and the decoder input; the Transformer can effectively capture long-term trends and global features through the self-attention mechanism, thereby improving the accuracy of the prediction; the LSTM is good at processing sequence data with time dependence, and can remember the context information of a longer time step, accurately capturing local time series fluctuations; these two methods have their own advantages, and combined use can better capture complex patterns in time series data; a short-term deterministic multi-step prediction model based on the Transformer network and the long short-term memory network (LSTM) is constructed ; and and are taken as the input and target output of model training, respectively, so as to update the parameters of the short-term deterministic multi-step prediction model according to formula (17): , to obtain the updated parameters :

[0137] (15)

[0138] In formula (15), represents the mean square error loss function of the short-term deterministic multi-step prediction model.

[0139] Step 3.6, when there is a prediction demand, the short-term deterministic multi-step prediction value at time t is obtained by using formula (16) based on the updated parameters :

[0140] (16)

[0141] In formula (16), represents the observation input of the current time to be predicted t.

[0142] Step 3.7, constructing a long-term interval multi-step prediction model based on Transformer-LSTM ; and and as the input and target output of model training, respectively, to update the parameters of formula (17) to obtain the updated parameters :

[0143] (17)

[0144] In formula (17), represents the mean square error loss function of the long-term interval multi-step prediction model;

[0145] When there is a prediction demand, the long-term interval multi-step prediction value at any time t is obtained based on the updated parameters using formula (18) :

[0146] (18)

[0147] Step four, constructing and training the scheduling model using the multi-agent double-delay deep deterministic policy gradient algorithm;

[0148] Step 4.1, the agent represents the control center of the scheduling scheme, which will learn the decision strategy based on the actor neural network to determine the decision action, and the critic network will evaluate the decision based on the reward value and optimize the actor network parameters; all decision actions are numerical values between 0 and 1, which is conducive to efficient decision-making of reinforcement learning; a scheduling decision model is established using the multi-agent double-delay deep deterministic policy gradient algorithm (MATD3), and the decision variables at time t are allocated to three agents according to formula (19), and a decision variable set at time t is formed ;

[0149] (19)

[0150] In formula (19), represents the decision variable of the first agent agent1 at time t; represents the decision variable made by the second agent agent2 at time t; represents the decision variable made by the third agent at time t.

[0151] Step 4.2, each agent can obtain the observation from the system as the input of the agent to calculate the action decision, and in this method, the observations of the three agents are shared; the current state observation of the three agents at time t is constructed ​And input into the scheduling decision model for processing, and output the decision variable set obtained at time t .

[0152] Step 4.3, bring into step one and step two, obtain the performance evaluation index at time t And as the reward value of each agent at time t, while obtaining the initial capacity of the electric energy storage module at time t+1 And the initial capacity of the hydrogen energy storage module at time t+1 ;

[0153] Step 4.4, obtain the time sequence segment of historical data at time t+1 , and input into And for processing, respectively, to obtain the short-term deterministic time sequence segment prediction value at time t+1 And the long-term interval time sequence segment prediction value at time t+1 ; Thus, the state observation of the three agents at time t+1 ; Record the four-tuple form data Into the experience replay pool.

[0154] Step 4.5, execute the MATD3 model training process in time step order, set the training batch number as Before the data accumulation reaches , randomly generate decision variables; After meeting the batch requirement, extract the training batch set from the experience replay pool And input into the scheduling decision model for training, so as to use the update formula of the parameters of the scheduling decision model constructed by formula (23) , wherein Indicates the current state observation set of the three agents, Indicates the current action decision set of the three agents, Indicates the reward value set of the three agents at the current time, Indicates the next state observation set of the three agents;

[0155] (20)

[0156] In formula (23), Indicates the updated optimal parameter, Indicates the target parameter of the scheduling decision model; m indicates the agent number; Indicates the next action decision set of the three agents; Indicates the Q function of the mth agent; Indicates the policy function of the mth agent; Indicates the discount factor;​ Noise representing the target strategy.

[0157] Step 4.6: Construct short-term deterministic multi-step prediction models at time t using equations (21) and (22) respectively. and long-term interval multi-step prediction model The updated formula for the mean squared error loss function is as follows:

[0158] (twenty one)

[0159] (twenty two)

[0160] In equation (22), This indicates a short-term deterministic multi-step prediction model. The updated mean squared error loss function, This represents a long-term interval multi-step prediction model at time t. The updated mean squared error loss function; Indicates the weight.

[0161] Step 4.5: Follow the process in steps 3.5-4.6. , The scheduling decision model is iteratively updated until the maximum number of iterations is reached or the scheduling decision model tends to stabilize, thereby obtaining the optimal scheduling decision model. This model is used to output the scheduling actions of the renewable energy module, electrolysis hydrogen production module, fuel cell module, heat pump electric heating module, electric energy storage module, and hydrogen energy storage module, and to determine the scheduling scheme of the integrated hydrogen energy system.

[0162] In summary, this invention proposes a hydrogen energy system scheduling decision-making method that combines the predictive capabilities of deep learning with the decision-making capabilities of deep reinforcement learning. The aim is to predict the supply and demand of the hydrogen energy system by introducing deep learning, and then utilize reinforcement learning to achieve intelligent decision-making in complex supply and demand environments. This improves the supply and demand matching efficiency and renewable energy utilization rate of the hydrogen energy system while maintaining low-carbon operation. This method can not only dynamically adjust production and storage strategies based on environmental fluctuations, but also adjust the predictive tendency of future energy supply and demand based on decision feedback. It achieves efficient system operation and enhanced flexibility through an adaptive multi-step integrated perspective. In the interaction with scheduling decision feedback, the predictive model can provide an adaptive and flexible perspective on top of accurate predictions. This flexible perspective further optimizes scheduling decisions, thereby improving the overall system performance.

[0163] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the hydrogen energy system scheduling method, and the processor is configured to execute the program stored in the memory.

[0164] In this embodiment, a computer readable storage medium is provided, and a computer program is stored in the computer readable storage medium. The computer program is run by a processor to perform the steps of the hydrogen energy system scheduling method.

Claims

1. A method for scheduling a hydrogen integrated energy system with adaptive multi-step integrated vision, the hydrogen integrated energy system comprising: The renewable energy module, the electrolytic hydrogen production module, the fuel cell module, the heat pump electric heat supply module, the electric energy storage module and the hydrogen energy storage module are characterized in that the hydrogen energy comprehensive energy system scheduling method is performed according to the following steps: Step one, establishing a simulation model of the hydrogen energy comprehensive energy system; Step two, based on the simulation model, constructing a scheduling performance evaluation index of the hydrogen energy comprehensive energy system; Step three, training a short-term deterministic multi-step prediction model and a long-term interval multi-step prediction model; Step 3.1, obtaining a historical data set of the hydrogen energy integrated energy system wherein, denotes a historical data set denotes the total time instants sampled in the historical data set; denotes the calendar information sampled at time instant t; denotes the external parameters at time instant t; denotes the historical data sample at time instant t; denotes the meteorological environmental parameters affecting the renewable energy module at time instant t; denotes the electrical load at time instant t; denotes the thermal load at time instant t; Step 3.2, denote the target parameter at time t as ; obtain historical data at time t-1 and construct an observation data set composed of other parameters except the target parameter at time t-1 ; and a target data set composed of the target parameter at time t-1 , wherein denotes a historical data sample at time i before time t ; and a sample formed after removing the target parameter at time i ​​​​ Given the observation data consisting of parameters other than the target parameter at time t. Then, calculate the probability distribution function of the target parameters at time t according to equation (14). This includes: meteorological environmental parameters at time t. probability distribution function Electrical load probability distribution function Heat load probability distribution function ; (14) In formula (14), denotes a Gaussian process regression function; Step 3.3, obtaining the probability distribution function of the meteorological environmental parameter at time t , the probability distribution function of the electrical load , the probability distribution function of the thermal load , the upper bound of the meteorological environmental parameter interval at time t , the lower bound of the meteorological environmental parameter interval at time t , the upper bound of the electrical load interval at time t , the lower bound of the electrical load interval at time t , the upper bound of the thermal load interval at time t , the lower bound of the thermal load interval at time t , and combining the historical data set , thereby obtaining the data set containing interval information ;​​​ Step 3.4, set the length of the time window as and perform sliding window processing on the data set containing interval information to obtain the time sequence segment of historical data at time t , the short-term deterministic time sequence segment of the target parameter to be predicted at time t , and the long-term interval time sequence segment of the target parameter to be predicted at time t , thereby obtaining the time sequence segment set of historical data , the short-term deterministic time sequence segment set of the target parameter to be predicted , and the long-term interval time sequence segment set of the target parameter to be predicted , wherein represents the historical data sample at time j represents the meteorological environment parameter at time j represents the electrical load at time j represents the thermal load at time j represents the upper limit of the meteorological environment parameter interval at time j represents the lower limit of the meteorological environment parameter interval at time j represents the upper limit of the electrical load interval at time j represents the lower limit of the electrical load interval at time j represents the upper limit of the thermal load interval at time j represents the lower limit of the thermal load interval at time j Step 3.5, constructing a short-term deterministic multi-step prediction model based on a Transformer network and a long short-term memory network LSTM ; and and as the input and target output of , respectively, to update the parameters of using formula (17) to obtain the updated parameters : (15) In formula (15), denotes the mean square error loss function of the short-term deterministic multi-step prediction model; Step 3.

6. Obtain the short-term deterministic multi-step prediction value at time t using formula (16) : (16) Step 3.

7. Constructing the long-term interval-based multi-step prediction model based on Transformer-LSTM ; and and as the input and target output of , respectively, to update the parameters of , to obtain the updated parameters :​ (17) In formula (17), denotes the mean square error loss function of the long-term interval multi-step prediction model; The long-term interval multi-step prediction value at any time t is obtained by using formula (18) : (18) Step four, based on the scheduling performance evaluation index and the two models of step three, constructing and training a scheduling model by using a multi-agent double-delay deep deterministic policy gradient algorithm, so as to obtain an optimal scheduling decision model, thereby determining a hydrogen energy comprehensive energy system scheduling scheme.

2. The method of claim 1, wherein the method further comprises: The step one comprises: Step 1.

1. Establishing a simulation model of the renewable energy module using formula (1) : (1) In formula (1), denotes the decision action of the renewable energy module at time t; denotes the electrical power output of the renewable energy module at time t; denotes the device losses of the renewable energy module at time t; Step 1.

2. Establishing a simulation model of the electrolytic hydrogen production module with formula (2) : (2) In formula (2), represents the decision action of the electrolytic hydrogen production module at time t; represents the electrical power consumption of the electrolytic hydrogen production module at time t; represents the equipment loss of the electrolytic hydrogen production module at time t; represents the equivalent power of the hydrogen production amount of the electrolytic hydrogen production module at time t; Step 1.

3. Building a simulation model of the fuel cell module using equation (3) : (3) In formula (3), represents the decision action of the fuel cell module at time t; represents the electric power output of the fuel cell module at time t; represents the equipment loss of the fuel cell module at time t; represents the thermal power output of the hydrogen production amount of the fuel cell module at time t; represents the equivalent power of the hydrogen consumption amount of the fuel cell module at time t; Step 1.

4. Establishing a simulation model of the heat pump electric heat supply module with formula (4) : (4) In formula (4), denotes the decision action of the electric heat supply module at time t; denotes the electric power consumption of the electric heat supply module at time t; denotes the device losses of the electric heat supply module at time t; denotes the thermal power output of the electric heat supply module at time t; Step 1.5, building a simulation model of the electrical energy storage module using formula (5) : (5) In formula (5), represents the decision action of the electrical energy storage module at time t; represents the initial capacity of the electrical energy storage module at time t; represents the electrical power variation of the electrical energy storage module at time t, and a positive number represents output electrical power and a negative number represents input electrical power; represents the equipment loss of the electrical energy storage module at time t; represents the initial capacity of the electrical energy storage module at time t+1; Step 1.6, Establishing a simulation model of the hydrogen storage module using formula (6) : (6) In formula (6), represents the decision action of the hydrogen energy storage module at time t; represents the initial capacity of the hydrogen energy storage module at time t; represents the equivalent hydrogen power change amount of the hydrogen energy storage module at time t, and a positive number represents output hydrogen power and a negative number represents input hydrogen power; represents the equipment loss of the hydrogen energy storage module at time t; represents the initial capacity of the hydrogen energy storage module at time t+1; Step 1.7, establishing power balance constraints of the hydrogen energy comprehensive energy system by using formulas (7)-(9), including: an electric power balance constraint, a thermal power balance constraint, and a hydrogen power balance constraint; (7) (8) (9) In formula (7) - formula (9), represents the power of the upper-level power grid at time t; represents the power of the upper-level heat grid at time t; represents the power of the upper-level hydrogen energy grid at time t.

3. The method of claim 2, wherein, The step two comprises: Step 2.1, Constructing the percentage of loss of all devices in the hydrogen energy integrated energy system at time t using formula (10) : (10) In formula (10), represents the theoretical maximum loss of the hydrogen energy comprehensive energy system at any moment; Step 2.2, constructing the new energy utilization rate of the hydrogen energy comprehensive energy system at time t using formula (11) : (11) In formula (11), represents the theoretical maximum output power of the renewable energy module at time t; Step 2.3, constructing carbon emission rate of hydrogen energy comprehensive energy system at time t using formula (12) : (12) In formula (12), represents a carbon emission factor of the upper-level power grid; represents a carbon emission factor of the upper-level heat grid; represents a carbon emission factor of the upper-level hydrogen energy grid; Step 2.4, calculating the performance evaluation index of the hydrogen energy comprehensive energy system at time t using formula (13) : (13)。 4. The dispatching method of hydrogen energy integrated energy system with adaptive multi-step integrated vision according to claim 3, characterized in that, The step four comprises: Step 4.1, assign the decision variables of time t to the three agents according to formula (19) and form the decision variable set of time t ; (19) In formula (19), denotes the decision variable of the first agent agentl at time t; denotes the decision variable made by the second agent agent2 at time t; denotes the decision variable made by the third agent at time t; Step 4.2, construct the current time state observation of the three agents at time t and input into the scheduling decision model for processing, and output the decision variable set obtained at time t ; Step 4.3, bring into step one and step two, obtain the performance evaluation index at time t , and as the reward value of each agent at time t, while obtaining the initial capacity of the electric energy storage module at time t+1 , and the initial capacity of the hydrogen energy storage module at time t+1 ;​ Step 4.4, obtain the time sequence segment of the historical data at time t+1 , and input them respectively and for processing, and obtain the short-term deterministic time sequence segment prediction value at time t+1 and the long-term interval time sequence segment prediction value at time t+1 respectively; thereby obtaining the state observation of the three agents at time t+1 ; record the four-tuple form data into the experience replay pool; Step 4.5: Extract training batch set from the experience replay pool The parameters are then input into the scheduling decision model for training, thereby utilizing the parameters of the scheduling decision model constructed using equation (23). The update formula, where, This represents the set of observations of the current state of three agents. This represents the set of action decisions made by three agents at the current moment. This represents the set of reward values ​​for the three agents at the current moment. This represents the set of state observations for the three agents at the next moment. (20) In formula (23), denotes the updated optimal parameter, denotes the target parameter of the scheduling decision model; m denotes the agent number; denotes the next time action decision set of 3 agents; denotes the Q function of the mth agent; denotes the policy function of the mth agent; denotes the discount factor; denotes the noise of the target policy; Step 4.6, the updated mean square error loss function update formula of the short-term deterministic multi-step prediction model at time t constructed by formula (21) and formula (22) respectively and the long-term interval multi-step prediction model ​ (21) (22) In equation (22), This indicates a short-term deterministic multi-step prediction model. The updated mean squared error loss function, This represents a long-term interval multi-step prediction model at time t. The updated mean squared error loss function; Indicates weight; Step 4.5: Follow the process in steps 3.5-4.

6. , The scheduling decision model is iteratively updated until the maximum number of iterations is reached or the scheduling decision model tends to stabilize, thereby obtaining the optimal scheduling decision model. This model is used to output the scheduling actions of the renewable energy module, electrolysis hydrogen production module, fuel cell module, heat pump electric heating module, electric energy storage module, and hydrogen energy storage module, in order to determine the scheduling scheme of the integrated hydrogen energy system.

5. An electronic device comprising a memory and a processor, characterized in that The memory is used for storing a program supporting the processor to execute the hydrogen energy comprehensive energy system scheduling method of any one of claims 1-4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to perform the steps of the hydrogen energy comprehensive energy system scheduling method of any one of claims 1-4.

Images