Hydrogen energy system scheduling method with self-adaptive multi-step comprehensive view

Abstract

The invention discloses a hydrogen energy system scheduling method with a self-adaptive multi-step comprehensive view, which comprises the following steps of: 1, establishing a hydrogen energy comprehensive energy system model suitable for an MATD3 method, and describing equipment operation characteristics; 2, analyzing scheduling performance from three dimensions of equipment loss, load balance and carbon emission, and determining an optimization objective function; 3, a probability interval is constructed through a GPR by using a historical data set, and short-term deterministic multi-step prediction and long-term interval multi-step prediction are realized in combination with a Transform-LSTM model; and 4, based on a prediction result and historical data, optimizing output and energy storage behaviors of each module device of the hydrogen energy system by using an MATD3 method, and realizing synchronous training of the MATD3 method and the prediction model through reward value transmission, so that the whole decision model has a self-adaptive multi-step comprehensive view, and the accuracy and flexibility of a scheduling decision are improved. According to the method, through multi-model coupling, the uncertainty processing capability and the scheduling optimization performance of the hydrogen energy system are enhanced, and the system operation performance is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of time series prediction technology and intelligent decision-making technology, and specifically to a hydrogen energy system scheduling method with an adaptive multi-step comprehensive vision that combines deep learning prediction capability and deep reinforcement learning decision-making capability. Background Art

[0002] With the rapid development of hydrogen energy technology and the transformation of energy structure, hydrogen energy has been widely regarded as one of the key elements in the future integrated energy system. The hydrogen energy system includes a complete industrial chain from production, storage, transportation to use, which is of great significance in coping with energy diversification and achieving carbon neutrality goals. However, the high uncertainty of renewable energy supply fluctuations and random load changes makes it difficult for traditional fixed scheduling modes to adapt to the dynamic characteristics of hydrogen energy systems, which in turn affects the efficiency and stability of the system.

[0003] Current hydrogen energy scheduling decisions are usually based on historical production and demand data, and are predicted and scheduled through static modeling methods. However, hydrogen energy supply and demand are affected by many complex factors such as weather conditions, system status, and user demand, which makes it difficult for scheduling decisions made by traditional methods to cope with flexible and changing demands. At the same time, the production, storage, and demand of hydrogen energy have highly nonlinear characteristics. Especially in scenarios where multiple energy forms (such as electricity and heat) are coupled, the multi-scale variation laws of hydrogen energy systems are difficult to effectively capture through a single traditional prediction method, resulting in low reliability and efficiency of system scheduling.

[0004] In order to improve the flexibility of hydrogen energy system scheduling decisions, existing research has begun to introduce deep learning methods to predict energy supply and demand. However, deep learning models can only provide predictions of future supply and demand trends and cannot be directly used for actual scheduling decisions, which limits the dynamic response capabilities of hydrogen energy systems. Therefore, how to combine intelligent prediction and adaptive scheduling to cope with complex supply and demand fluctuations has become a key issue for the efficient use of hydrogen energy systems. Summary of the invention

[0005] In order to overcome the deficiencies in the above-mentioned prior art, the present invention provides a hydrogen energy system scheduling method with an adaptive multi-step comprehensive vision, so as to improve the supply and demand matching efficiency and new energy utilization rate of the hydrogen energy system while maintaining the low-carbon operation of the system, so as to enhance the uncertainty handling capability of the hydrogen energy system and the accuracy and flexibility of scheduling decisions, thereby effectively improving the system operation performance.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] The present invention provides a method for scheduling a hydrogen energy integrated energy system with an adaptive multi-step comprehensive vision. The hydrogen energy integrated energy system includes: a renewable energy module, an electrolysis hydrogen production module, a fuel cell module, a heat pump electric heating module, an electric energy storage module, and a hydrogen energy storage module. The characteristic of the method for scheduling the hydrogen energy integrated energy system is that the method is performed according to the following steps:

[0008] Step 1: Establish a simulation model of the hydrogen integrated energy system;

[0009] Step 2: Based on the simulation model, construct the dispatch performance evaluation index of the hydrogen energy integrated energy system;

[0010] Step 3: Train the short-term deterministic multi-step prediction model and the long-term interval multi-step prediction model;

[0011] Step 4. Based on the scheduling performance evaluation indicators and the two models in step 3, the scheduling model is constructed and trained using the multi-agent dual-delay deep deterministic policy gradient algorithm to obtain the optimal scheduling decision model to determine the scheduling plan for the hydrogen integrated energy system.

[0012] The method for scheduling a hydrogen energy integrated energy system with an adaptive multi-step integrated vision described in the present invention is also characterized in that the step 1 comprises:

[0013] Step 1.1: Use formula (1) to establish a simulation model of the renewable energy module :

[0014] [ p t RE , c t RE ] = D e v i c e RE ( a t RE , W t ) (1)

[0015] In formula (1), represents the decision action of the renewable energy module at time t; represents the electric power output of the renewable energy module at time t; represents the equipment loss of the renewable energy module at time t; represents the meteorological environment parameters affecting the renewable energy module at time t;

[0016] Step 1.2: Use formula (2) to establish a simulation model of the electrolysis hydrogen production module :

[0017] [ p t EL , c t EL , η t EL ] = D e v i c e EL ( a t EL ) (2)

[0018] In formula (2), Represents the decision-making action of the electrolytic hydrogen production module at time t; represents the electric power consumption of the electrolytic hydrogen production module at time t; Indicates the equipment loss of the electrolytic hydrogen production module at time t; It represents the equivalent power of hydrogen produced by the electrolytic hydrogen production module at time t;

[0019] Step 1.3: Use formula (3) to establish a simulation model of the fuel cell module :

[0020] [ p t FC , c t FC , h t FC , η t FC ] = D e v i c e FC ( a t FC ) (3)

[0021] In formula (3), represents the decision action of the fuel cell module at time t; represents the electric power output of the fuel cell module at time t; represents the equipment loss of the fuel cell module at time t; It represents the thermal power output of the hydrogen produced by the fuel cell module at time t; represents the equivalent power of the hydrogen consumption of the fuel cell module at time t;

[0022] Step 1.4: Use formula (4) to establish a simulation model of the heat pump electric heating module :

[0023] [ p t HP , c t HP , h t HP ] = D e v i c e H P ( a t HP ) (4)

[0024] In formula (4), Represents the decision action of the electric heating module at time t; represents the electric power consumption of the electric heating module at time t; Represents the equipment loss of the electric heating module at time t; Represents the thermal power output of the electric heating module at time t;

[0025] Step 1.5: Use formula (5) to establish a simulation model of the electric energy storage module :

[0026] [ p t ES , c t ES , s t + 1 ES ] = D e v i c e E S ( a t ES , s t ES ) (5)

[0027] In formula (5), Represents the decision action of the electric energy storage module at time t; represents the initial capacity of the energy storage module at time t; Indicates the change in electric power of the electric energy storage module at time t, with a positive number indicating output electric power and a negative number indicating input electric power; Represents the equipment loss of the electric energy storage module at time t; Represents the initial capacity of the energy storage module at time t+1;

[0028] Step 1.6: Use formula (6) to establish a simulation model of the hydrogen energy storage module :

[0029] [ η t HS , c t HS , s t + 1 HS ] = D e v i c e H S ( a t HS , s t HS ) (6)

[0030] In formula (6), Represents the decision action of the hydrogen energy storage module at time t; represents the initial capacity of the hydrogen energy storage module at time t; Indicates the change in equivalent hydrogen power of the hydrogen energy storage module at time t, with positive numbers representing output hydrogen power and negative numbers representing input hydrogen power; Indicates the equipment loss of the hydrogen energy storage module at time t; represents the initial capacity of the hydrogen energy storage module at time t+1;

[0031] Step 1.7: Use equations (7) to (9) to establish the power balance constraints of the hydrogen energy integrated energy system, including: electric power balance constraints, thermal power balance constraints, and hydrogen power balance constraints:

[0032] (7)

[0033] (8)

[0034] (9)

[0035] In formula (7) to formula (9), represents the power of the upper grid at time t; represents the power of the upper heating network at time t; represents the power of the upper hydrogen energy network at time t; represents the electrical load at time t; represents the heat load at time t.

[0036] Furthermore, the step 2 includes:

[0037] Step 2.1: Use formula (10) to construct the loss percentage of all equipment in the hydrogen energy integrated energy system at time t :

[0038] (10)

[0039] In formula (10), It represents the theoretical maximum loss of the hydrogen integrated energy system at any time;

[0040] Step 2.2: Use formula (11) to construct the new energy utilization rate of the hydrogen energy integrated energy system at time t: :

[0041] (11)

[0042] In formula (11), represents the theoretical maximum output power of the renewable energy module at time t;

[0043] Step 2.3: Use formula (12) to construct the carbon emission rate of the hydrogen energy integrated energy system at time t: :

[0044] (12)

[0045] In formula (12), represents the carbon emission factor of the parent power grid; represents the carbon emission factor of the parent heat network; represents the carbon emission factor of the parent hydrogen energy network;

[0046] Step 2.4: Calculate the performance evaluation index of the hydrogen energy integrated energy system at time t using formula (13): :

[0047] (13).

[0048] Furthermore, the step three includes:

[0049] Step 3.1: Obtain historical data set of hydrogen integrated energy system ,in, Represents a historical dataset The total time of sampling in Represents the calendar information sampled at time t; represents the external parameters at time t; Represents the historical data sample at time t;

[0050] Step 3.2: Record the target parameter at time t as ; Get the time before t Historical data and construct t- At time t-1, all parameters except the target parameters constitute the observation data set And by t- The target parameters from -1 to t-1 constitute the target data set ,in, Represents the historical data sample at time i before time t Remove the target parameter at time i The sample formed later;

[0051] When the observation data consisting of other parameters except the target parameter at time t is given When , the probability distribution function of the target parameter at time t is calculated according to formula (14): , including: meteorological environment parameters at time t The probability distribution function of , electrical load The probability distribution function of , heat load The probability distribution function of ;

[0052] (14)

[0053] In formula (14), represents the Gaussian process regression function;

[0054] Step 3.3: According to the meteorological environment parameters at time t The probability distribution function of , electrical load The probability distribution function of , heat load The probability distribution function of , and the upper bound of the meteorological environment parameter interval at time t is obtained accordingly , the lower bound of the meteorological environment parameter interval at time t , the upper limit of the electric load interval at time t , the lower limit of the electric load interval at time t , upper bound of heat load interval at time t , lower bound of heat load interval at time t , and with historical datasets Combine to obtain a data set containing interval information ;

[0055] Step 3.4: Set the length of the time window to , and for data sets containing interval information Perform sliding window processing to obtain the time series fragment of historical data at time t , the short-term deterministic time series fragment of the target parameter to be predicted at time t And the long-term interval time series fragment of the target parameter to be predicted at time t , thus obtaining a set of time series fragments of historical data , a set of short-term deterministic time series segments of the target parameters to be predicted , a set of long-term interval time series segments of the target parameter to be predicted ,in, represents the historical data sample at time j, represents the meteorological environment parameters at time j, represents the electrical load at time j, represents the heat load at time j, represents the upper bound of the meteorological environment parameter interval at time j, represents the lower bound of the meteorological environment parameter interval at time j, represents the upper limit of the electric load interval at time j, represents the lower bound of the electric load interval at time j, represents the upper bound of the heat load interval at time j, represents the lower bound of the heat load interval at time j;

[0056] Step 3.5: Build a short-term deterministic multi-step prediction model based on Transformer network and long short-term memory network LSTM ; and and As The input and target output of Parameters , get the updated parameters :

[0057] (15)

[0058] In formula (15), represents the mean squared error loss function of the short-term deterministic multi-step forecasting model;

[0059] Step 3.6: Use formula (16) to obtain the short-term deterministic multi-step forecast value at time t :

[0060] (16)

[0061] Step 3.7: Build a long-term interval multi-step prediction model based on Transformer-LSTM ; and and As The input and target output of Parameters , get the updated parameters :

[0062] (17)

[0063] In formula (17), Represents the mean square error loss function of the long-term interval multi-step forecast model;

[0064] Using formula (18), we can get the long-term interval multi-step forecast value at any time t: :

[0065] (18).

[0066] Furthermore, the step 4 includes:

[0067] Step 4.1: Allocate the decision variables at time t to the three agents according to formula (19) and form the decision variable set at time t ;

[0068] (19)

[0069] In formula (19), Represents the decision variable of the first agent agent1 at time t; represents the decision variable made by the second agent agent2 at time t; represents the decision variable made by the third agent at time t;

[0070] Step 4.2: Construct the current state observation of the three agents at time t And input into the scheduling decision model for processing, and output the decision variable set at time t ;

[0071] Step 4.3: Substitute into steps 1 and 2 to obtain the performance evaluation index at time t And as the reward value of each agent at time t, the initial capacity of the energy storage module at time t+1 is obtained and the initial capacity of the hydrogen energy storage module at time t+1 ;

[0072] Step 4.4: Get the time series fragment of historical data at time t+1 , and enter and The short-term deterministic time series segment prediction value at time t+1 is obtained accordingly. and the long-term interval time series segment prediction value at time t+1 ; Thus, the state observation of the three agents at time t+1 is obtained ; Record data in the form of four tuples To the experience replay pool;

[0073] Step 4.5: Extract a set of training batches from the experience replay pool And input into the scheduling decision model for training, so as to use the parameters of the scheduling decision model constructed by formula (23) The update formula is: represents the current state observation set of the three agents, represents the action decision set of the three agents at the current moment, Represents the set of reward values ​​of the three agents at the current moment, Represents the state observation set of the three agents at the next moment;

[0074] (20)

[0075] In formula (23), Indicates the updated optimal parameters. represents the target parameter of the scheduling decision model; m represents the agent number; Represents the action decision set of the three agents at the next moment; represents the Q function of the mth agent; represents the strategy function of the mth agent; represents the discount factor; represents the noise of the target policy;

[0076] Step 4.6: Use equations (21) and (22) to construct short-term deterministic multi-step prediction models at time t and long-term multi-step forecasting models The updated mean square error loss function update formula is:

[0077] (twenty one)

[0078] (twenty two)

[0079] In formula (22), Represents a short-term deterministic multi-step forecasting model The updated mean square error loss function is: Represents the long-term multi-step forecasting model at time t The updated mean square error loss function of ; represents weight;

[0080] Step 4.5: Follow the process from step 3.5 to step 4.6 to , The scheduling decision model is iteratively updated until the maximum number of iterations is reached or the scheduling decision model tends to be stable, thereby obtaining the optimal scheduling decision model, which is used to output the scheduling actions of the renewable energy module, electrolysis hydrogen production module, fuel cell module, heat pump electric heating module, electric energy storage module and hydrogen energy storage module to determine the scheduling plan of the hydrogen energy integrated energy system.

[0081] The electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the hydrogen energy system scheduling method, and the processor is configured to execute the program stored in the memory.

[0082] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the hydrogen energy system scheduling method when executed by a processor.

[0083] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0084] 1 Aiming at the uncertainty fluctuations in the hydrogen energy system, the present invention constructs time series segments of historical meteorological data and forecast targets, adopts GPR technology to analyze the interval characteristics of historical data, uses the Transformer-LSTM model to capture the long-term trend and local fluctuation characteristics in the data, and predicts the multi-step short-term deterministic fluctuation trend and long-term interval fluctuation trend of key data based on historical data, which provides a multi-step comprehensive vision for scheduling decisions and improves the accuracy of decision-making.

[0085] 2 The present invention constructs a hydrogen energy system scheduling decision model through the MATD3 method, incorporates renewable energy power supply, electric energy storage, hydrogen energy storage and heating supply decisions into the scheduling decision model, and constructs coupling decisions in different action areas during operation through multi-agent decision group training, ensuring the balance constraints of electricity, heat and hydrogen energy, with the goal of improving the comprehensive performance of the system, and realizing the scheduling decision optimization of the hydrogen energy system.

[0086] 3 The present invention couples the deep learning method and the deep reinforcement learning method, and provides the Transformer-LSTM prediction model with a multi-step comprehensive vision for future renewable energy and load demand estimation. The MATD3 model makes scheduling decisions based on predicted data and historical data, and the decision reward value will be passed back to the Transformer-LSTM prediction model to optimize the model parameters while optimizing the MATD3 model, thereby realizing the deep coupling and synchronous training of the two models, so that the decision method has an adaptive comprehensive vision and improves the flexibility of decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 A schematic diagram of the process of scheduling decision-making for hydrogen energy system;

[0088] Figure 2 Schematic diagram of the Transformer-LSTM and MATD3 coupling model structure. DETAILED DESCRIPTION

[0089] In this embodiment, the hydrogen energy integrated energy system includes: a renewable energy module, an electrolytic hydrogen production module, a fuel cell module, a heat pump electric heating module, an electric energy storage module, and a hydrogen energy storage module. In view of the uncertainty of the hydrogen energy system and the problem of decision optimization of complex systems, a hydrogen energy system scheduling decision method with an adaptive multi-step comprehensive vision is constructed. First, a hydrogen energy integrated energy system model in a reinforcement learning operating environment is established to describe the operating characteristics of each submodule of the system. Then, the scheduling performance is analyzed from three dimensions: equipment loss, load supply balance, and carbon emissions. Based on the ideal solution sorting method (Technique for Order Preference by Similarity to Ideal Solution, TOPSIS), the three indicator dimensions are integrated to determine the optimization objective function. Next, Gaussian Process Regression (GPR) is used to obtain the probability interval form on the basis of normalization, and the data is processed to obtain the time series fragment form, and the data is segmented to obtain the target output in the form of short-term deterministic time series data and long-term interval time series data. The Transformer model and the Long Short Term Memory Network (LSTM) model are used to build a Transformer-LSTM combined prediction model to capture the long-term trends and local fluctuation characteristics in the data to achieve multi-step comprehensive prediction of various key data. Finally, the Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3) model is used to make decisions on the system's electricity, heat, hydrogen, storage and other energy scheduling behaviors, and the MATD3 model and Transformer-LSTM are deeply coupled to give the overall decision-making model an adaptive multi-step comprehensive vision, with the goal of maximizing comprehensive scheduling performance to improve the accuracy and flexibility of decision-making. Specific processes, such as Figure 1 As shown, the method is performed in the following steps:

[0090] Step 1: Establish a simulation model of the hydrogen integrated energy system;

[0091] Step 1.1: Although different energy systems have different components, the difference is only in the description of parameters and formula characteristics, and the general model functions are consistent. Use formula (1) to establish a simulation model of the renewable energy module. :

[0092] [ p t RE , c t RE ] = D e v i c e RE ( a t RE , W t ) (1)

[0093] In formula (1), represents the decision action of the renewable energy module at time t. The specific values ​​of the decision actions in step 1 are determined by the subsequent reinforcement learning part; represents the electric power output of the renewable energy module at time t; represents the equipment loss of the renewable energy module at time t; It represents the meteorological environment parameters that affect the renewable energy module at time t, which may include parameters such as wind speed, temperature, and radiation intensity according to actual needs.

[0094] Step 1.2: Use formula (2) to establish a simulation model of the electrolysis hydrogen production module :

[0095] [ p t EL , c t EL , η t EL ] = D e v i c e EL ( a t EL ) (2)

[0096] In formula (2), Represents the decision-making action of the electrolytic hydrogen production module at time t; represents the electric power consumption of the electrolytic hydrogen production module at time t; Indicates the equipment loss of the electrolytic hydrogen production module at time t; It represents the equivalent power of hydrogen produced by the electrolytic hydrogen production module at time t;

[0097] Step 1.3: Use formula (3) to establish a simulation model of the fuel cell module :

[0098] [ p t FC , c t FC , h t FC , η t FC ] = D e v i c e FC ( a t FC ) (3)

[0099] In formula (3), represents the decision action of the fuel cell module at time t; represents the electric power output of the fuel cell module at time t; represents the equipment loss of the fuel cell module at time t; It represents the thermal power output of the hydrogen produced by the fuel cell module at time t; Represents the equivalent power of the hydrogen consumption of the fuel cell module at time t.

[0100] Step 1.4: Use formula (4) to establish a simulation model of the heat pump electric heating module :

[0101] [ p t HP , c t HP , h t HP ] = D e v i c e H P ( a t HP ) (4)

[0102] In formula (4), Represents the decision action of the electric heating module at time t; represents the electric power consumption of the electric heating module at time t; Represents the equipment loss of the electric heating module at time t; Represents the thermal power output of the electric heating module at time t;

[0103] Step 1.5: Use formula (5) to establish a simulation model of the electric energy storage module :

[0104] [ p t ES , c t ES , s t + 1 ES ] = D e v i c e E S ( a t ES , s t ES ) (5)

[0105] In formula (5), Represents the decision action of the electric energy storage module at time t; represents the initial capacity of the energy storage module at time t; Indicates the change in electric power of the electric energy storage module at time t, with a positive number indicating output electric power and a negative number indicating input electric power; Represents the equipment loss of the electric energy storage module at time t; Represents the initial capacity of the energy storage module at time t+1.

[0106] Step 1.6: Use formula (6) to establish a simulation model of the hydrogen energy storage module :

[0107] [ η t HS , c t HS , s t + 1 HS ] = D e v i c e H S ( a t HS , s t HS ) (6)

[0108] In formula (6), Represents the decision action of the hydrogen energy storage module at time t; represents the initial capacity of the hydrogen energy storage module at time t; Indicates the change in equivalent hydrogen power of the hydrogen energy storage module at time t, with positive numbers representing output hydrogen power and negative numbers representing input hydrogen power; Indicates the equipment loss of the hydrogen energy storage module at time t; Represents the initial capacity of the hydrogen energy storage module at time t+1.

[0109] Step 1.7: Use equations (7) to (9) to establish the power balance constraints of the hydrogen energy integrated energy system, including: electric power balance constraints, thermal power balance constraints, and hydrogen power balance constraints:

[0110] (7)

[0111] (8)

[0112] (9)

[0113] In formula (7) to formula (9), represents the power of the upper grid at time t; represents the power of the upper heating network at time t; represents the power of the upper hydrogen energy network at time t; represents the electrical load at time t; represents the heat load at time t; in addition to the above constraints, it also includes output limit constraints, climbing power constraints, energy storage capacity constraints, etc. according to the differences in equipment characteristics. They are all included in the Device model of each device by default and will not be repeated here; deep reinforcement learning will transform the decision-making problem into several key concepts to construct the Markov decision process. These concepts include action, agent, environment, state and reward; all the above models are included in the concept of environment, which is a simulation of the specific scenario in which the decision-making plan is applied.

[0114] Step 2: Define the scheduling performance evaluation indicators of the hydrogen integrated energy system model;

[0115] This part will evaluate the scheduling results of the system at any time from three dimensions. Due to the need to combine deep reinforcement learning methods, the three indicators need to be integrated into a reward with practical meaning, corresponding to the objective function of decision optimization.

[0116] Step 2.1: Use formula (10) to construct the loss percentage of all equipment in the hydrogen energy integrated energy system at time t :

[0117] (10)

[0118] In formula (10), It represents the theoretical maximum loss of the hydrogen integrated energy system at any time;

[0119] Step 2.2: Use formula (11) to construct the new energy utilization rate of the hydrogen energy integrated energy system at time t: :

[0120] (11)

[0121] In formula (11), Represents the theoretical maximum output power of the renewable energy module at time t.

[0122] Step 2.3: Use formula (12) to construct the carbon emission rate of the hydrogen energy integrated energy system at time t: :

[0123] (12)

[0124] In formula (12), represents the carbon emission factor of the parent power grid; represents the carbon emission factor of the parent heat network; represents the carbon emission factor of the parent hydrogen energy network;

[0125] Step 2.4: Calculate the performance evaluation index of the hydrogen energy integrated energy system at time t using formula (13): :

[0126] (13)

[0127] The idea of ​​TOPSIS method is adopted here. It is assumed that the performance pricing index is a vector in the three indicator spaces defined in 2.1-2.3. The scheduling performance is best when the three indicator dimensions tend to 0. In order to adapt to the characteristics of reinforcement learning to maximize the reward value, the negative value of the three-dimensional vector length is used as the final evaluation index.

[0128] Step 3: Train the short-term deterministic multi-step prediction model and the long-term interval multi-step prediction model;

[0129] Step 3.1: Obtain historical data set of hydrogen integrated energy system ,in, Represents a historical dataset The total time of sampling in Represents the calendar information sampled at time t; Represents other external parameters at time t. This type of data may directly or indirectly affect the prediction target. The specific parameter types are not fixed and can be selected according to needs. Represents the historical data sample at time t;

[0130] Step 3.2, GPR has become one of the most commonly used methods for interval analysis due to its good ability to capture nonlinear relationships and uncertainty modeling advantages; it uses kernel functions to construct the covariance matrix between data pairs and establishes a non-parametric model of input and target output for regression analysis; the target parameter at time t can be expressed as ; Get the time before t Historical data to build t- At time t-1, all parameters except the target parameters constitute the observation data set and by t- The target parameters from -1 to t-1 constitute the target data set , Represents the historical data sample at time i before time t Remove the target parameter at time i The new sample formed later.

[0131] When the observation data consisting of other parameters except the target parameter at time t is given When , the probability distribution function of the target parameter at time t is calculated according to formula (14): , including: meteorological environment parameters at time t The probability distribution function of , electrical load The probability distribution function of , heat load The probability distribution function of ;

[0132] (14)

[0133] In formula (14), all data are derived from historical data sets. , represents the Gaussian process regression function.

[0134] Step 3.3: According to the meteorological environment parameters at time t The probability distribution function of , electrical load The probability distribution function of , heat load The probability distribution function of , and the upper bound of the meteorological environment parameter interval at time t is obtained accordingly , the lower bound of the meteorological environment parameter interval at time t , the upper limit of the electric load interval at time t , the lower limit of the electric load interval at time t , upper bound of heat load interval at time t , lower bound of heat load interval at time t , and with historical datasets Combine to obtain a data set containing interval information .

[0135] Step 3.4: Time series segments can contain more complete fluctuation characteristics, so both the input and target output are organized into time series segments; time series segments of historical data are constructed based on sliding time windows, and the length of the time window is set to , and for data sets containing interval information Perform sliding window processing to obtain the time series fragment of historical data at time t , the short-term deterministic time series fragment of the target parameter to be predicted at time t And the long-term interval time series fragment of the target parameter to be predicted at time t , thus obtaining a set of time series fragments of historical data , a set of short-term deterministic time series segments of the target parameters to be predicted , a set of long-term interval time series segments of the target parameter to be predicted ,in, represents the historical data sample at time j, represents the meteorological environment parameters at time j, represents the electrical load at time j, represents the heat load at time j, represents the upper bound of the meteorological environment parameter interval at time j, represents the lower bound of the meteorological environment parameter interval at time j, represents the upper limit of the electric load interval at time j, represents the lower bound of the electric load interval at time j, represents the upper bound of the heat load interval at time j, Represents the lower bound of the heat load range at time j.

[0136] Step 3.5: Figure 2 As shown in the figure, the Transformer part includes multiple encoder and decoder layers; each encoder layer consists of a multi-head self-attention mechanism and a feedforward neural network, with residual connections and normalization operations; the structure of the decoder layer is similar to the encoder, except that it also contains an additional multi-head self-attention layer to handle the dependency between the encoder output and the decoder input; Transformer can effectively capture long-term trends and global features through the self-attention mechanism, thereby improving the accuracy of prediction; LSTM is good at processing time-dependent sequence data, can remember contextual information for longer time steps, and accurately capture local time series fluctuations; these two methods have their own advantages, and combined use can better capture complex patterns in time series data; Construct a short-term deterministic multi-step prediction model based on Transformer network and long short-term memory network (LSTM) ; and and As the input and target output of model training respectively, we can use formula (17) to update Parameters , get the updated parameters :

[0137] (15)

[0138] In formula (15), Represents the mean squared error loss function for the short-term deterministic multi-step forecasting model.

[0139] Step 3.6: When there is a forecast demand, based on the updated parameters Using formula (16), we can get the short-term deterministic multi-step forecast value at time t: :

[0140] (16)

[0141] In formula (16), Represents the observed input at the current time t to be predicted.

[0142] Step 3.7: Build a long-term interval multi-step prediction model based on Transformer-LSTM ; and and As the input and target output of model training respectively, we can use formula (17) to update Parameters , get the updated parameters :

[0143] (17)

[0144] In formula (17), Represents the mean square error loss function of the long-term interval multi-step forecast model;

[0145] When there is a forecast demand, based on the updated parameters Using formula (18), we can get the long-term interval multi-step forecast value at any time t: :

[0146] (18)

[0147] Step 4: Use the multi-agent double-delay deep deterministic policy gradient algorithm to build and train the scheduling model;

[0148] Step 4.1: The agent represents the control center that formulates the scheduling plan. It will learn the decision strategy based on the actor neural network to solve the decision action. The critic network will evaluate the decision based on the reward value and optimize the actor network parameters. All decision actions are values ​​between 0 and 1, which is conducive to efficient decision-making in reinforcement learning. The scheduling decision model is established using the multi-agent double-delay deep deterministic policy gradient algorithm (MATD3). According to formula (19), the decision variables at time t are assigned to the three agents, and the decision variable set at time t is formed. ;

[0149] (19)

[0150] In formula (19), Represents the decision variable of the first agent agent1 at time t; represents the decision variable made by the second agent agent2 at time t; represents the decision variable made by the third agent at time t.

[0151] Step 4.2: Each agent can obtain the observations from the system and use them as the input of the agent to calculate the action decision. In this method, the observations of the three agents are shared; construct the current state observations of the three agents at time t And input into the scheduling decision model for processing, and output the decision variable set at time t .

[0152] Step 4.3: Substitute into steps 1 and 2 to obtain the performance evaluation index at time t And as the reward value of each agent at time t, the initial capacity of the energy storage module at time t+1 is obtained and the initial capacity of the hydrogen energy storage module at time t+1 ;

[0153] Step 4.4: Get the time series fragment of historical data at time t+1 , and enter and The short-term deterministic time series segment prediction value at time t+1 is obtained accordingly. and the long-term interval time series segment prediction value at time t+1 ; Thus, the state observation of the three agents at time t+1 is obtained ; Record data in the form of four tuples To the experience replay pool.

[0154] Step 4.5: Execute the MATD3 model training process in the order of time steps, and set the number of training batches to , before data accumulation reaches Before, randomly generate decision variables; after meeting the batch requirements, extract the training batch set from the experience replay pool And input into the scheduling decision model for training, so as to use the parameters of the scheduling decision model constructed by formula (23) The update formula is: represents the current state observation set of the three agents, represents the action decision set of the three agents at the current moment, Represents the set of reward values ​​of the three agents at the current moment, Represents the state observation set of the three agents at the next moment;

[0155] (20)

[0156] In formula (23), Indicates the updated optimal parameters. represents the target parameter of the scheduling decision model; m represents the agent number; Represents the action decision set of the three agents at the next moment; represents the Q function of the mth agent; represents the strategy function of the mth agent; represents the discount factor; represents the noise of the target policy.

[0157] Step 4.6: Use equations (21) and (22) to construct short-term deterministic multi-step prediction models at time t and long-term multi-step forecasting models The updated mean square error loss function update formula is:

[0158] (twenty one)

[0159] (twenty two)

[0160] In formula (22), Represents a short-term deterministic multi-step forecasting model The updated mean square error loss function is: Represents the long-term multi-step forecasting model at time t The updated mean square error loss function of ; Represents weight.

[0161] Step 4.5: Follow the process from step 3.5 to step 4.6 to , The scheduling decision model is iteratively updated until the maximum number of iterations is reached or the scheduling decision model tends to be stable, so as to obtain the optimal scheduling decision model, which is used to output the scheduling actions of renewable energy modules, electrolysis hydrogen production modules, fuel cell modules, heat pump electric heating modules, electric energy storage modules and hydrogen energy storage modules, and determine the scheduling plan of the hydrogen energy integrated energy system.

[0162] In summary, the present invention proposes a hydrogen energy system scheduling decision method that combines deep learning prediction capabilities with deep reinforcement learning decision-making capabilities. The method aims to predict the supply and demand of the hydrogen energy system by introducing deep learning, and then use reinforcement learning to achieve intelligent decision-making in complex supply and demand environments, thereby maintaining the low-carbon operation of the system. The supply and demand matching efficiency and new energy utilization rate of the hydrogen energy system are improved. This method can not only dynamically adjust the production and storage strategies based on environmental fluctuations, but also adjust the prediction tendency of future energy supply and demand according to decision feedback, and realize efficient operation and flexibility improvement of the system through an adaptive multi-step comprehensive perspective. In the interaction with the scheduling decision feedback, the prediction model can provide an adaptive and flexible perspective on the basis of accurate prediction, and this flexible perspective will further optimize the scheduling decision, thereby improving the overall performance of the system.

[0163] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the hydrogen energy system scheduling method, and the processor is configured to execute the program stored in the memory.

[0164] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the hydrogen energy system scheduling method are executed.

Claims

1. A method for scheduling a hydrogen energy integrated energy system with an adaptive multi-step integrated vision, the hydrogen energy integrated energy system comprising: Renewable energy module, electrolysis hydrogen production module, fuel cell module, heat pump electric heating module, electric energy storage module, hydrogen energy storage module, characterized in that the hydrogen energy integrated energy system scheduling method is carried out according to the following steps: Step 1: Establish a simulation model of the hydrogen integrated energy system; Step 2: Based on the simulation model, construct the dispatch performance evaluation index of the hydrogen energy integrated energy system; Step 3: Train the short-term deterministic multi-step prediction model and the long-term interval multi-step prediction model; Step 4. Based on the scheduling performance evaluation indicators and the two models in step 3, the scheduling model is constructed and trained using the multi-agent dual-delay deep deterministic policy gradient algorithm to obtain the optimal scheduling decision model to determine the scheduling plan for the hydrogen integrated energy system.

2. A method for scheduling a hydrogen energy integrated energy system with an adaptive multi-step integrated vision according to claim 1, characterized in that: The step one comprises: Step 1.1: Use formula (1) to establish a simulation model of the renewable energy module : (1) In formula (1), represents the decision action of the renewable energy module at time t; represents the electric power output of the renewable energy module at time t; represents the equipment loss of the renewable energy module at time t; represents the meteorological environment parameters affecting the renewable energy module at time t; Step 1.2: Use formula (2) to establish a simulation model of the electrolysis hydrogen production module : (2) In formula (2), Represents the decision-making action of the electrolytic hydrogen production module at time t; represents the electric power consumption of the electrolytic hydrogen production module at time t; Indicates the equipment loss of the electrolytic hydrogen production module at time t; It represents the equivalent power of hydrogen produced by the electrolytic hydrogen production module at time t; Step 1.3: Use formula (3) to establish a simulation model of the fuel cell module : (3) In formula (3), represents the decision action of the fuel cell module at time t; represents the electric power output of the fuel cell module at time t; represents the equipment loss of the fuel cell module at time t; It represents the thermal power output of the hydrogen produced by the fuel cell module at time t; represents the equivalent power of the hydrogen consumption of the fuel cell module at time t; Step 1.4: Use formula (4) to establish a simulation model of the heat pump electric heating module : (4) In formula (4), Represents the decision action of the electric heating module at time t; represents the electric power consumption of the electric heating module at time t; Represents the equipment loss of the electric heating module at time t; Represents the thermal power output of the electric heating module at time t; Step 1.5: Use formula (5) to establish a simulation model of the electric energy storage module : (5) In formula (5), Represents the decision action of the electric energy storage module at time t; represents the initial capacity of the energy storage module at time t; Indicates the change in electric power of the electric energy storage module at time t, with a positive number indicating output electric power and a negative number indicating input electric power; Represents the equipment loss of the electric energy storage module at time t; Represents the initial capacity of the energy storage module at time t+1; Step 1.6: Use formula (6) to establish a simulation model of the hydrogen energy storage module : (6) In formula (6), Represents the decision action of the hydrogen energy storage module at time t; represents the initial capacity of the hydrogen energy storage module at time t; Indicates the change in equivalent hydrogen power of the hydrogen energy storage module at time t, with positive numbers representing output hydrogen power and negative numbers representing input hydrogen power; Indicates the equipment loss of the hydrogen energy storage module at time t; represents the initial capacity of the hydrogen energy storage module at time t+1; Step 1.7: Use equations (7) to (9) to establish the power balance constraints of the hydrogen energy integrated energy system, including: electric power balance constraints, thermal power balance constraints, and hydrogen power balance constraints: (7) (8) (9) In formula (7) to formula (9), represents the power of the upper grid at time t; represents the power of the upper heating network at time t; represents the power of the upper hydrogen energy network at time t; represents the electrical load at time t; represents the heat load at time t.

3. A method for scheduling a hydrogen energy integrated energy system with an adaptive multi-step integrated vision according to claim 2, characterized in that: The second step comprises: Step 2.1: Use formula (10) to construct the loss percentage of all equipment in the hydrogen energy integrated energy system at time t : (10) In formula (10), It represents the theoretical maximum loss of the hydrogen integrated energy system at any time; Step 2.2: Use formula (11) to construct the new energy utilization rate of the hydrogen energy integrated energy system at time t: : (11) In formula (11), represents the theoretical maximum output power of the renewable energy module at time t; Step 2.3: Use formula (12) to construct the carbon emission rate of the hydrogen energy integrated energy system at time t: : (12) In formula (12), represents the carbon emission factor of the parent power grid; represents the carbon emission factor of the parent heat network; represents the carbon emission factor of the parent hydrogen energy network; Step 2.4: Calculate the performance evaluation index of the hydrogen energy integrated energy system at time t using formula (13): : (13)。 4. A method for scheduling a hydrogen energy integrated energy system with an adaptive multi-step integrated vision according to claim 3, characterized in that: The step three comprises: Step 3.1: Obtain historical data set of hydrogen integrated energy system ,in, Represents a historical dataset The total time of sampling in Represents the calendar information sampled at time t; represents the external parameters at time t; Represents the historical data sample at time t; Step 3.2: Record the target parameter at time t as ; Get the time before t Historical data and construct t- At time t-1, all parameters except the target parameters constitute the observation data set And by t- The target parameters from -1 to t-1 constitute the target data set ,in, Represents the historical data sample at time i before time t Remove the target parameter at time i The sample formed later; When the observation data consisting of other parameters except the target parameter at time t is given When , the probability distribution function of the target parameter at time t is calculated according to formula (14): , including: meteorological environment parameters at time t The probability distribution function of , electrical load The probability distribution function of , heat load The probability distribution function of ; (14) In formula (14), represents the Gaussian process regression function; Step 3.3: According to the meteorological environment parameters at time t The probability distribution function of , electrical load The probability distribution function of , heat load The probability distribution function of , and the upper bound of the meteorological environment parameter interval at time t is obtained accordingly , the lower bound of the meteorological environment parameter interval at time t , the upper limit of the electric load interval at time t , the lower limit of the electric load interval at time t , upper bound of heat load interval at time t , lower bound of heat load interval at time t , and with historical datasets Combine to obtain a data set containing interval information ; Step 3.4: Set the length of the time window to , and for data sets containing interval information Perform sliding window processing to obtain the time series fragment of historical data at time t , the short-term deterministic time series fragment of the target parameter to be predicted at time t And the long-term interval time series fragment of the target parameter to be predicted at time t , thus obtaining a set of time series fragments of historical data , a set of short-term deterministic time series segments of the target parameters to be predicted , a set of long-term interval time series segments of the target parameter to be predicted ,in, represents the historical data sample at time j, represents the meteorological environment parameters at time j, represents the electrical load at time j, represents the heat load at time j, represents the upper bound of the meteorological environment parameter interval at time j, represents the lower bound of the meteorological environment parameter interval at time j, represents the upper limit of the electric load interval at time j, represents the lower bound of the electric load interval at time j, represents the upper bound of the heat load interval at time j, represents the lower bound of the heat load interval at time j; Step 3.5: Build a short-term deterministic multi-step prediction model based on Transformer network and long short-term memory network LSTM ; and and As The input and target output of Parameters , get the updated parameters : (15) In formula (15), represents the mean squared error loss function of the short-term deterministic multi-step forecasting model; Step 3.6: Use formula (16) to obtain the short-term deterministic multi-step forecast value at time t : (16) Step 3.7: Build a long-term interval multi-step prediction model based on Transformer-LSTM ; and and As The input and target output of Parameters , get the updated parameters : (17) In formula (17), Represents the mean square error loss function of the long-term interval multi-step forecast model; Using formula (18), we can get the long-term interval multi-step forecast value at any time t: : (18)。 5. A method for scheduling a hydrogen energy integrated energy system with an adaptive multi-step integrated vision according to claim 4, characterized in that: The fourth step comprises: Step 4.1: Allocate the decision variables at time t to the three agents according to formula (19) and form the decision variable set at time t ; (19) In formula (19), Represents the decision variable of the first agent agent1 at time t; represents the decision variable made by the second agent agent2 at time t; represents the decision variable made by the third agent at time t; Step 4.2: Construct the current state observation of the three agents at time t And input into the scheduling decision model for processing, and output the decision variable set at time t ; Step 4.3: Substitute into steps 1 and 2 to obtain the performance evaluation index at time t And as the reward value of each agent at time t, the initial capacity of the energy storage module at time t+1 is obtained and the initial capacity of the hydrogen energy storage module at time t+1 ; Step 4.4: Get the time series fragment of historical data at time t+1 , and enter and The short-term deterministic time series segment prediction value at time t+1 is obtained accordingly. and the long-term interval time series segment prediction value at time t+1 ; Thus, the state observation of the three agents at time t+1 is obtained ; Record data in the form of four tuples To the experience replay pool; Step 4.5: Extract a set of training batches from the experience replay pool And input into the scheduling decision model for training, so as to use the parameters of the scheduling decision model constructed by formula (23) The update formula is: represents the current state observation set of the three agents, represents the action decision set of the three agents at the current moment, represents the set of reward values ​​of the three agents at the current moment, Represents the state observation set of the three agents at the next moment; (20) In formula (23), Indicates the updated optimal parameters. represents the target parameter of the scheduling decision model; m represents the agent number; Represents the action decision set of the three agents at the next moment; represents the Q function of the mth agent; represents the strategy function of the mth agent; represents the discount factor; represents the noise of the target policy; Step 4.6: Use equations (21) and (22) to construct short-term deterministic multi-step prediction models at time t and long-term multi-step forecasting models The updated mean square error loss function update formula is: (21) (22) In formula (22), Represents a short-term deterministic multi-step forecasting model The updated mean square error loss function is: Represents the long-term multi-step forecasting model at time t The updated mean square error loss function of ; represents weight; Step 4.5: Follow the process from step 3.5 to step 4.6 to , The scheduling decision model is iteratively updated until the maximum number of iterations is reached or the scheduling decision model tends to be stable, thereby obtaining the optimal scheduling decision model, which is used to output the scheduling actions of the renewable energy module, electrolysis hydrogen production module, fuel cell module, heat pump electric heating module, electric energy storage module and hydrogen energy storage module to determine the scheduling plan of the hydrogen energy integrated energy system.

6. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the hydrogen energy system scheduling method described in any one of claims 1 to 5, and the processor is configured to execute the program stored in the memory.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the hydrogen energy system scheduling method according to any one of claims 1 to 5 are executed.

Images