Low-carbon scheduling method for comprehensive energy system containing hydrogen storage

Abstract

The invention relates to a low-carbon dispatching method for a hydrogen storage-containing comprehensive energy system, which is applied to the comprehensive energy system comprising a hydrogen energy storage unit, a battery energy storage unit, a power grid power supply unit and a clean energy power supply unit, and comprises the following steps of: constructing a unit time cost model of the hydrogen energy storage unit, the battery energy storage unit, the power grid power supply unit and the clean energy power supply unit; wherein carbon cost is introduced into the unit time cost model of the power grid power supply unit, and oxygen income and hydrogen leakage cost are introduced into the unit time cost model of the hydrogen energy storage unit; under the operation constraint of the integrated energy system, constructing an objective function based on unit time cost models of a hydrogen energy storage unit, a battery energy storage unit, a power grid power supply unit and a clean energy power supply unit; and under the operation constraint of the integrated energy system, the objective function is taken as a target, and step-by-step decision of the objective function is realized by using a reinforcement learning algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of low-carbon scheduling of integrated energy systems, and in particular to a low-carbon scheduling method for an integrated energy system containing hydrogen storage. Background Art

[0002] With the continuous changes in energy structure and demand, integrated energy systems have become a research hotspot. Integrated energy systems combine and optimize two or more energy supplies in a certain way to improve energy utilization efficiency, reduce energy consumption, and reduce environmental pollution.

[0003] Energy storage is an important component of the integrated energy system. Currently, there are mainly battery energy storage and hydrogen energy storage. In the integrated energy system scheduling process of existing technologies, there is little research on hydrogen energy storage, and most of them ignore the benefits of oxygen as a byproduct in the electrolysis process of hydrogen energy storage, and most of them also ignore the losses caused by hydrogen leakage in hydrogen energy storage. This makes the cost calculation of hydrogen energy storage often inaccurate to a certain extent. Scheduling cannot meet the actual situation, and the scheduling effect cannot achieve the optimal cost. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present invention provides a low-carbon scheduling method for an integrated energy system containing hydrogen storage.

[0005] In a first aspect, the present invention provides a low-carbon scheduling method for an integrated energy system containing hydrogen storage, which is applied to an integrated energy system including a hydrogen energy storage unit, a battery energy storage unit, a grid power supply unit, and a clean energy power supply unit, wherein the clean energy power supply unit adopts a photovoltaic power generation unit and a wind power generation unit, including:

[0006] Under the operation constraints of the integrated energy system, the unit time cost model of hydrogen energy storage unit, battery energy storage unit, grid power supply unit and clean energy power supply unit is constructed: Among them, the unit time cost model of the power supply unit of the power grid is Introducing carbon cost into the unit time cost model of hydrogen energy storage unit Oxygen benefits are introduced in the model, and hydrogen leakage costs are introduced by modeling hydrogen leakage;

[0007] Under the operation constraints of the integrated energy system, the objective function is constructed based on the unit time cost model of the hydrogen energy storage unit, battery energy storage unit, grid power supply unit, and clean energy power supply unit:

[0008]

[0009] Among them, the operation constraints of the integrated energy system include the operation constraints of the hydrogen energy storage unit, the operation constraints of the battery energy storage unit and the supply and demand power balance constraints;

[0010] Under the operation constraints of the integrated energy system, with the objective function as the goal, pre-trained intelligent agents are used for step-by-step scheduling. The intelligent agents include decision networks and evaluation networks, and are trained based on the reinforcement learning architecture.

[0011] Furthermore, the hydrogen energy storage unit includes an electrolyzer for producing hydrogen by electrolysis, a hydrogen storage tank and an oxygen storage tank for storing hydrogen and oxygen produced by the electrolyzer, and a hydrogen fuel cell connected to the hydrogen storage tank;

[0012] The operation process of the hydrogen energy storage unit includes the process of electrolysis to produce hydrogen and oxygen and storage and the process of converting the stored hydrogen into electrical energy. The unit time cost of the process of electrolysis to produce hydrogen and oxygen and storage is as follows:

[0013]

[0014] Among them, C E is the electrolytic cell cost, L E is the electrolytic cell life, C S is the cost of hydrogen storage tank, L S is the life of the hydrogen storage tank; P t EP is the electrolysis power of the electrolytic cell; is the maintenance cost per unit time of the electrolytic cell at time t, is the maintenance cost per unit time of the electrolytic cell at time t; C o,pr is the price of oxygen per unit mass sold, λ is the mass conversion coefficient of oxygen production; η1 represents the efficiency of hydrogen production by electrolyzer;

[0015] The cost per unit time of converting stored hydrogen into electricity is as follows:

[0016]

[0017] Among them, C F is the cost of hydrogen fuel cells, L F For the life of hydrogen fuel cells, is the maintenance cost per unit time of the hydrogen fuel cell at time t;

[0018] The unit time cost model of hydrogen energy storage unit is as follows:

[0019]

[0020] Among them, Λ is a binary parameter. When Λ is 1, it means that the hydrogen energy storage unit is running, and when Λ is 0, it means that the hydrogen energy storage unit is idle; Ω is a binary parameter. When Ω is 1, it means that the electrolyzer of the hydrogen energy storage unit is running, and when Ω is 0, it means that the hydrogen fuel cell of the hydrogen energy storage unit is running.

[0021] Furthermore, the operating constraints of the hydrogen energy storage unit include: the mass of the stored hydrogen follows the following equation:

[0022]

[0023] in, represents the mass of hydrogen stored at time t, represents the mass of hydrogen stored at time t+Δt, 0≤P t EP ≤P EP，max , the electrolytic power of the electrolytic cell is limited by the maximum electrolytic power P of the electrolytic cell EP，max , η2 represents the efficiency of hydrogen fuel cell consuming hydrogen to generate electricity, P t Hout is the output power of the hydrogen fuel cell; 0≤P t Hout ≤P Hout,max , the output power of the hydrogen fuel cell is limited by the maximum output power of the fuel cell; for any hydrogen energy storage unit, its electrolyzer and hydrogen fuel cell cannot work at the same time, satisfying P t EP P t Hout =0;

[0024] Indicates the amount of hydrogen leakage per unit time of the hydrogen energy storage unit, where: is the leakage related to the diffusion of hydrogen molecules, α1 represents the first adjustment coefficient, D is the diffusion coefficient, represents the concentration gradient; α2kPa is the leakage related to hydrogen molecule penetration, α2 represents the second adjustment coefficient, k is the permeability coefficient, and Pa is the hydrogen storage pressure; is the pore leakage, α3 is the third adjustment coefficient, S is a constant representing the leakage area, V H is the volume of hydrogen, ΔPa is the pressure difference between the inside and outside of the leak;

[0025] Due to the gas pressure and volume restrictions of the hydrogen storage tank, the hydrogen storage capacity of the hydrogen storage tank is limited. The mass of hydrogen stored at any time t satisfies the following constraints:

[0026] in, is the maximum hydrogen storage mass of the hydrogen storage tank.

[0027] Furthermore, the unit time cost model of the battery energy storage unit is as follows:

[0028]

[0029] in, represents the SOC of the battery energy storage unit at time t, Lc Battery energy storage power cycle life, C B is the hardware cost of the battery energy storage unit, D t is the depth of charge and discharge, E is the battery capacity, η3 represents the charging efficiency of the battery energy storage unit, and η4 represents the discharging efficiency of the battery energy storage unit.

[0030] Furthermore, the battery energy storage unit operation constraints include: the SOC of the battery energy storage unit follows the following equation:

[0031]

[0032] in, represents the SOC of the battery energy storage unit at time t; represents the SOC of the battery energy storage unit at time t+Δt; η3 represents the charging efficiency of the battery energy storage unit; P t ch is the charging power of the battery energy storage unit, 0≤P t ch ≤P ch，max , the charging power of the battery energy storage unit is limited by the maximum charging power P of the battery energy storage unit ch，max ; η4 represents the discharge efficiency of the battery energy storage unit; P t disch is the output power of the battery energy storage unit, 0≤P t disch ≤P disch,max , the output power of the battery energy storage unit is limited by the maximum output power P of the battery energy storage unit disch,max ; For any battery energy storage unit, it cannot be charged and discharged at the same time, satisfying P t ch P t disch =0;

[0033] During the operation of the battery energy storage unit, the SOC meets the following requirements:

[0034] Among them, SOC B,min is the minimum SOC of the battery energy storage unit, SOC B，max is the maximum SOC of the battery energy storage unit.

[0035] Furthermore, in the process of meeting load demand, carbon consumption mainly comes from power supply by the power grid. If carbon cost is introduced into the power supply cost of the power grid, the unit time cost model of the power grid is as follows:

[0036]

[0037] in, is the electricity purchase price at time t, At time t, the electricity price is, β is the carbon emission coefficient, βP t grid It represents the carbon emission per unit time. Γ is a binary parameter. When Γ is 1, it means obtaining electricity from the grid. When Γ is 0, it means selling electricity to the grid.

[0038] Furthermore, the integrated energy system needs to meet the power balance between supply and demand:

[0039] P t need +∑P t EP +∑P t ch -∑P t Hout -∑P t disch -P t grid -∑P t g =0;

[0040] Among them, P t need is the load power demand at time t, ∑P t ch is the sum of the charging powers of all battery energy storage units in the integrated energy system at time t; ∑P t EP is the sum of the electrolysis power of all hydrogen energy storage units in the integrated energy system at time t; ∑P t Hout is the output power sum of all hydrogen energy storage units in the integrated energy system at time t; ∑P t disch is the output power sum of all battery energy storage units in the integrated energy system at time t; P t grid is the power supply of the power grid at time t, when P t grid When the value is negative, it indicates that power is supplied to the grid; ∑P t g is the sum of the power supply powers of all clean energy power supply units in the integrated energy system at time t.

[0041] Furthermore, the unit time cost model of the clean energy power supply unit is as follows:

[0042]

[0043] Among them, C gx is the hardware cost of the clean energy power supply unit, L gx For the life of the clean energy power supply unit, The maintenance cost per unit time of the clean energy power supply unit. When x is 1, it represents a photovoltaic power generation unit, and when x is 2, it represents a wind power generation unit.

[0044] Furthermore, we build an intelligent agent: we use three layers of fully connected layers and mapping fully connected layers with a set number of hidden units to build a decision network as an actor and an evaluation network as a critic in reinforcement learning. The non-mapping fully connected layer uses the tanh activation function, and the mapping fully connected layer uses the relu activation function.

[0045] Constructing the state of the integrated energy system: The real-time output power P of each clean energy in the integrated energy system t g , Load demand power P t need , the amount of hydrogen stored in the hydrogen energy storage unit SOC of battery storage unit Electricity purchase price and electricity sales price As can be observed:

[0046] Constructing actions to control the operation of the integrated energy system: In the integrated energy system, the objective function is achieved by adjusting the energy storage unit, and the electrolysis power P of the electrolyzer of the hydrogen energy storage unit is t EP , the output power P of the hydrogen fuel cell t Hout , the charging power P of the battery energy storage unit t ch And the output power P of the battery energy storage unit t disch As an action: Action = (P t EP ,P t Hout ,P t ch ,P t disch );

[0047] Build Rewards:

[0048] Based on the unit time cost of each time step of the objective function, a decision penalty term is introduced to construct the reward for each step:

[0049]

[0050] Among them, κ is the proportionality coefficient, U is a constant, D(P t EP ,P t Hout ,P tch ,P t disch ) is a function to determine whether the action exceeds the constraint limit. When there is an action that exceeds the constraint limit, D(P t EP ,P t Hout ,P t ch ,P t disch ) takes the value of 1, otherwise, D(P t EP ,P t Hout ,P t ch ,P t disch ) takes the value of 0;

[0051] The reward is accumulated at each step:

[0052] R=∑ρr t , where ρ is the weighting coefficient;

[0053] Build experience pool: Get the status of the experience pool based on the number of operations of the integrated energy system.

[0054] Furthermore, the parameters of the decision network are initialized to obtain the decision network Π, the parameters of the evaluation network are initialized to obtain the evaluation network Q, and the target decision network Π' and the target evaluation network Q' are defined to be obtained through training;

[0055] Take the initialized Π and Q as the target: Q'←Q, Π'←Π;

[0056] Randomly select a batch of state experience data from the experience pool; input the current state into the network to obtain the current moment action, obtain the current step reward corresponding to the current moment action and the state combination at the next moment and add them to the experience pool; (Action t ,State t ,r t ,State t+1 );

[0057] Random batch sampling of experience data from the experience pool is provided to the target decision network Π', decision network Π, target evaluation network Q', and evaluation network Q; the target decision network Π' and decision network Π take the state as input and the action as output; the target evaluation network Q' and evaluation network Q take the state action as input and the evaluation as output;

[0058] The evaluation mean square error of the evaluation network Q and the target evaluation network Q' for each batch of empirical data is used as the evaluation network loss function, and the parameters of the evaluation network Q are updated using the gradient descent algorithm according to the evaluation network loss function;

[0059] Among them, the evaluation network loss is: L Q =RMS(Y,Q(State,Action));

[0060] Where, Y=(y1,y2,......,y Z ) is the output of the target evaluation network, Z is the batch size; Q (State, Action) is the output of the evaluation network, State, Action are the corresponding actions and states contained in each batch of experience data, and RMS () represents the mean square error;

[0061] in, is the single-step reward corresponding to the output of the k-th target evaluation network, ζ is the discount factor, The action output by the target decision network Π' according to the state at the next moment;

[0062] The average of the evaluation network Q's evaluation of the decision network Π is used as the loss function of the decision network, and the evaluation is maximized to make the decision optimal; where the loss function of the decision network is: L Π =E(Q(State,Π(State))), where E() represents averaging, and the parameters of the decision network Π are updated using the gradient descent algorithm according to the evaluation network loss function;

[0063] According to the learning rate lr, the target evaluation network Q' is regularly updated with the parameters of the evaluation network Q: Q'←lrQ+(1-lr)Q';

[0064] The target decision network Π' is updated periodically with the parameters of the decision network Π according to the learning rate lr: Π'←lrΠ+(1-lr)Π'.

[0065] In a second aspect, the present invention provides a low-carbon scheduling device for an integrated energy system containing hydrogen storage, comprising: at least one processing unit, the processing unit being connected to a storage unit via a bus unit, the storage unit storing a computer program, and when the computer program is executed by the processing unit, the low-carbon scheduling method for the integrated energy system containing hydrogen storage is implemented.

[0066] In a third aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the low-carbon scheduling method for an integrated energy system containing hydrogen storage is implemented as described above.

[0067] The above technical solution provided by the embodiment of the present invention has the following advantages compared with the prior art:

[0068] This application constructs a unit time cost model for hydrogen energy storage units, battery energy storage units, grid power supply units, and clean energy power supply units: Among them, the unit time cost model of the power supply unit of the power grid is Introducing carbon cost into the unit time cost model of hydrogen energy storage unit Oxygen revenue and hydrogen leakage cost are introduced; under the operation constraints of the integrated energy system, the objective function is constructed based on the unit time cost model of hydrogen energy storage unit, battery energy storage unit, grid power supply unit, and clean energy power supply unit: Under the operation constraints of the integrated energy system, the objective function is used as the goal, and the pre-trained intelligent agent is used for step-by-step scheduling. The intelligent agent includes a decision network and an evaluation network, and is trained based on a reinforcement learning architecture. This application is aimed at hydrogen energy storage units. By introducing oxygen revenue and hydrogen leakage costs, the objective function can better reflect the actual cost, and the reinforcement learning decision based on the objective function is more realistic and has better results. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0071] Figure 1 A flow chart of a low-carbon scheduling method for an integrated energy system containing hydrogen storage provided in an embodiment of the present invention;

[0072] Figure 2 A schematic diagram of an integrated energy system and a reinforcement learning architecture provided by an embodiment of the present invention;

[0073] Figure 3 A schematic diagram of a low-carbon scheduling device for an integrated energy system containing hydrogen storage provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0076] Example 1

[0077] like Figure 1 As shown, the technology of the present invention realizes a low-carbon scheduling method for an integrated energy system containing hydrogen storage, which is applied to an integrated energy system including a hydrogen energy storage unit, a battery energy storage unit, a grid power supply unit, and a clean energy power supply unit, wherein the clean energy power supply unit adopts a photovoltaic power generation unit and a wind power generation unit.

[0078] In the specific implementation process, a low-carbon scheduling method for an integrated energy system containing hydrogen storage includes: constructing a unit time cost model for hydrogen energy storage units, battery energy storage units, grid power supply units, and clean energy power supply units: Among them, the unit time cost model of the power supply unit of the power grid is Introducing carbon costs into

[0079] Under the operation constraints of the integrated energy system, the objective function is constructed based on the unit time cost model of the hydrogen energy storage unit, battery energy storage unit, grid power supply unit, and clean energy power supply unit:

[0080]

[0081] Among them, the operating constraints of the integrated energy system include hydrogen energy storage unit operating constraints, battery energy storage unit operating constraints and supply and demand power balance constraints.

[0082] The operating constraints of hydrogen energy storage units include:

[0083] The hydrogen energy storage unit includes an electrolyzer for producing hydrogen by electrolysis, a hydrogen storage tank and an oxygen storage tank for storing hydrogen and oxygen produced by the electrolyzer, and a hydrogen fuel cell connected to the hydrogen storage tank. The fuel cell of the hydrogen energy storage unit uses oxygen in the air as a reaction raw material and produces oxygen as a by-product. After the oxygen is produced, it can be continuously stored by replacing the oxygen storage tank, and the capacity of the oxygen storage tank can be considered infinite.

[0084] For hydrogen energy storage units, electrolyzers, hydrogen storage tanks and hydrogen fuel cells are participants in energy storage and release. During the energy storage and release process, the mass of hydrogen stored follows the following equation:

[0085]

[0086] in, represents the mass of hydrogen stored at time t, represents the mass of hydrogen stored at time t+Δt, η1 represents the efficiency of hydrogen production in the electrolyzer, P t EP is the electrolytic power of the electrolytic cell, 0≤P t EP ≤P EP，max , the electrolytic power of the electrolytic cell is limited by the maximum electrolytic power P of the electrolytic cell EP，max , η2 represents the efficiency of hydrogen fuel cell consuming hydrogen to generate electricity, P t Hout is the output power of the hydrogen fuel cell; 0≤P t Hout ≤P Hout,max The output power of a hydrogen fuel cell is limited by the maximum output power P of the fuel cell. Hout,max For any hydrogen energy storage unit, its electrolyzer and hydrogen fuel cell cannot work at the same time, satisfying P t EP P t Hout =0.

[0087] Indicates the amount of hydrogen leakage per unit time of the hydrogen energy storage unit, where: is the leakage related to the diffusion of hydrogen molecules, α1 represents the first adjustment coefficient, D is the diffusion coefficient, represents the concentration gradient; α2kPa is the leakage related to hydrogen molecule penetration, α2 represents the second adjustment coefficient, k is the permeability coefficient, and Pa is the hydrogen storage pressure; is the pore leakage, α3 is the third adjustment coefficient, S is a constant representing the leakage area, V H is the volume of hydrogen, and ΔPa is the pressure difference inside and outside the leakage. By modeling the hydrogen leakage, the hydrogen leakage cost is introduced into the subsequent cost calculation.

[0088] Due to the gas pressure and volume restrictions of the hydrogen storage tank, the hydrogen storage capacity of the hydrogen storage tank is limited. The mass of hydrogen stored at any time t satisfies the following constraints:

[0089] in, is the maximum hydrogen storage mass of the hydrogen storage tank.

[0090] The operation process of the hydrogen energy storage unit includes the process of electrolysis to produce hydrogen and oxygen and storage and the process of converting the stored hydrogen into electrical energy. The unit time cost of the process of electrolysis to produce hydrogen and oxygen and storage is as follows:

[0091]

[0092] Among them, C E is the electrolytic cell cost, L E is the electrolytic cell life, C S is the cost of hydrogen storage tank, L S is the life of the hydrogen storage tank; P t EP is the electrolysis power of the electrolytic cell; is the maintenance cost per unit time of the electrolytic cell at time t, is the maintenance cost per unit time of the electrolytic cell at time t; C o,pr is the price of oxygen per unit mass sold, λ is the mass conversion coefficient of oxygen production; η1 represents the efficiency of hydrogen production by electrolyzer; unit time cost model of hydrogen energy storage unit Introducing oxygen gains.

[0093] The cost per unit time of converting stored hydrogen into electricity is as follows:

[0094]

[0095] Among them, C F is the cost of hydrogen fuel cells, L F For the life of hydrogen fuel cells, is the maintenance cost per unit time of the hydrogen fuel cell at time t;

[0096] The unit time cost model of hydrogen energy storage unit is as follows:

[0097]

[0098] Among them, Λ is a binary parameter. When Λ is 1, it means that the hydrogen energy storage unit is running, and when Λ is 0, it means that the hydrogen energy storage unit is idle; Ω is a binary parameter. When Ω is 1, it means that the electrolyzer of the hydrogen energy storage unit is running, and when Ω is 0, it means that the hydrogen fuel cell of the hydrogen energy storage unit is running.

[0099] Battery energy storage unit operating constraints include:

[0100] For the battery energy storage unit, during its energy storage and release process, the SOC of the battery energy storage unit follows the following equation:

[0101]

[0102] in, represents the SOC of the battery energy storage unit at time t; represents the SOC of the battery energy storage unit at time t+Δt; η3 represents the charging efficiency of the battery energy storage unit; P t ch is the charging power of the battery energy storage unit, 0≤P tch ≤P ch，max , the charging power of the battery energy storage unit is limited by the maximum charging power P of the battery energy storage unit ch，max ; η4 represents the discharge efficiency of the battery energy storage unit; P t disch is the output power of the battery energy storage unit, 0≤P t disch ≤P disch,max , the output power of the battery energy storage unit is limited by the maximum output power P of the battery energy storage unit disch,max ; For any battery energy storage unit, it cannot be charged and discharged at the same time, satisfying P t ch P t disch =0.

[0103] During the operation of the battery energy storage unit, the SOC meets the following requirements:

[0104] Among them, SOC B,min is the minimum SOC of the battery energy storage unit, SOC B，max is the maximum SOC of the battery energy storage unit.

[0105] The unit time cost model of a battery energy storage unit is as follows:

[0106]

[0107] Among them, L c Battery energy storage power cycle life, C B is the hardware cost of the battery energy storage unit, D t is the depth of charge and discharge, and E is the battery capacity.

[0108] In the process of meeting load demand, carbon consumption mainly comes from power supply of the power grid. If carbon cost is introduced into the power supply cost of the power grid, the unit time cost model of the power grid is as follows:

[0109]

[0110] in, is the electricity purchase price at time t, At time t, the electricity price is, β is the carbon emission coefficient, βP t grid It represents the carbon emission per unit time. Γ is a binary parameter. When Γ is 1, it means obtaining electricity from the grid. When Γ is 0, it means selling electricity to the grid.

[0111] The unit time cost model of the clean energy power supply unit is as follows:

[0112]

[0113] Among them, C gx is the hardware cost of the clean energy power supply unit, L gx For the life of the clean energy power supply unit, The maintenance cost per unit time of the clean energy power supply unit. When x is 1, it represents a photovoltaic power generation unit, and when x is 2, it represents a wind power generation unit.

[0114] The integrated energy system needs to meet the supply and demand power balance:

[0115] P t need +∑P t EP +∑P t ch -∑P t Hout -∑P t disch -P t grid -∑P t g =0;

[0116] Among them, P t need is the load power demand at time t, ∑P t ch is the sum of the charging powers of all battery energy storage units in the integrated energy system at time t; ∑P t EP is the sum of the electrolysis power of all hydrogen energy storage units in the integrated energy system at time t; ∑P t Hout is the output power sum of all hydrogen energy storage units in the integrated energy system at time t; ∑P t disch is the output power sum of all battery energy storage units in the integrated energy system at time t; P t grid is the power supply of the power grid at time t, when P t grid When the value is negative, it indicates that power is supplied to the grid; ∑P t g is the sum of the power supply powers of all clean energy power supply units in the integrated energy system at time t.

[0117] Under the operation constraints of the integrated energy system, a pre-trained intelligent agent is used for step-by-step scheduling with the objective function as the goal. The intelligent agent includes a decision network and an evaluation network, and is trained based on a reinforcement learning architecture.

[0118] like Figure 2As shown, the intelligent agent is built: the decision network as the actor and the evaluation network as the critic in reinforcement learning are constructed through three layers of fully connected layers and mapping fully connected layers containing a set number of hidden units. The non-mapping fully connected layer uses the tanh activation function, and the mapping fully connected layer uses the relu activation function.

[0119] Constructing the state of the integrated energy system: The real-time output power P of each clean energy in the integrated energy system t g , Load demand power P t need , the amount of hydrogen stored in the hydrogen energy storage unit Battery energy storage unit Electricity purchase price Electricity sales price and the price per unit mass of oxygen sold as the observable state:

[0120] Oxygen sales are delayed, and this application will The approximate selling price of oxygen is:

[0121] Constructing actions to control the operation of the integrated energy system: In the integrated energy system, the objective function is achieved by adjusting the energy storage unit, and the electrolysis power P of the electrolyzer of the hydrogen energy storage unit is t EP , the output power P of the hydrogen fuel cell t Hout , the charging power P of the battery energy storage unit t ch And the output power P of the battery energy storage unit t disch As an action: Action t =(P t EP ,P t Hout ,P t ch ,P t disch ).

[0122] Construct rewards based on the objective function:

[0123] Based on the unit time cost of each time step of the objective function, a decision penalty term is introduced to construct the reward for each step:

[0124]

[0125] Among them, κ is the proportionality coefficient, U is a constant, D(P t EP ,Pt Hout ,P t ch ,P t disch ) is a function to determine whether the action exceeds the constraint limit. When there is an action that exceeds the constraint limit, D(P t EP ,P t Hout ,P t ch ,P t disch ) takes the value of 1, otherwise, D(P t EP ,P t Hout ,P t ch ,P t disch ) takes a value of 0.

[0126] The reward is accumulated at each step:

[0127] R=∑ρr t , where ρ is the weighting coefficient;

[0128] Build experience pool: Get the status of the experience pool based on the number of operations of the integrated energy system.

[0129] Initialize the parameters of the decision network to obtain the decision network Π, initialize the parameters of the evaluation network to obtain the evaluation network Q, and define the target decision network Π' and the target evaluation network Q' obtained through training;

[0130] Take the initialized Π and Q as the target: Q'←Q, Π'←Π;

[0131] Randomly select a batch of state experience data from the experience pool; input the current state into the network to obtain the current moment action, obtain the current step reward corresponding to the current moment action and the state combination at the next moment and add them to the experience pool; (Action t ,State t ,r t ,State t+1 );

[0132] Randomly sample experience data in batches from the experience pool (Action t ,State t ,r t ,State t+1) is provided to the target decision network Π', the decision network Π, the target evaluation network Q', and the evaluation network Q; the target decision network Π' and the decision network Π take the state as input and the action as output; the target evaluation network Q' and the evaluation network Q take the state action as input and the evaluation as output;

[0133] The evaluation mean square error of the evaluation network Q and the target evaluation network Q' for each batch of empirical data is used as the evaluation network loss function, and the parameters of the evaluation network Q are updated using the gradient descent algorithm according to the evaluation network loss function;

[0134] Among them, the evaluation network loss is: L Q =RMS(Y,Q(State,Action));

[0135] Where, Y=(y1,y2,......,y Z ) is the output of the target evaluation network, Z is the batch size; Q (State, Action) is the output of the evaluation network, State, Action are the corresponding actions and states contained in each batch of experience data, and RMS () represents the mean square error;

[0136] Among them, r t k is the single-step reward corresponding to the output of the k-th target evaluation network, ζ is the discount factor, The action output by the target decision network Π' according to the state at the next moment;

[0137] The average of the evaluation network Q's evaluation of the decision network Π is used as the loss function of the decision network, and the evaluation is maximized to make the decision optimal; where the loss function of the decision network is: L Π =E(Q(State,Π(State))), where E() represents averaging, and the parameters of the decision network Π are updated using the gradient descent algorithm according to the evaluation network loss function;

[0138] According to the learning rate lr, the target evaluation network Q' is regularly updated with the parameters of the evaluation network Q: Q'←lrQ+(1-lr)Q';

[0139] The target decision network Π' is updated periodically with the parameters of the decision network Π according to the learning rate lr: Π'←lrΠ+(1-lr)Π'.

[0140] Example 2

[0141] See also Figure 3As shown, an embodiment of the present invention provides a low-carbon scheduling device for an integrated energy system containing hydrogen storage, comprising: at least one processing unit, the processing unit is connected to a storage unit via a bus unit, the storage unit is a computer-readable storage medium, and can be used to store software programs, computer executable programs, and modules, such as the software programs, computer executable programs, and modules corresponding to a low-carbon scheduling method for an integrated energy system containing hydrogen storage in an embodiment of the present invention. The processing unit implements the above-mentioned low-carbon scheduling method for an integrated energy system containing hydrogen storage by running the software programs, computer executable programs, and modules stored in the storage unit, comprising:

[0142] Construct the unit time cost model of hydrogen energy storage unit, battery energy storage unit, grid power supply unit and clean energy power supply unit: Among them, the unit time cost model of the power supply unit of the power grid is Introducing carbon costs into

[0143] Under the operation constraints of the integrated energy system, the objective function is constructed based on the unit time cost model of the hydrogen energy storage unit, battery energy storage unit, grid power supply unit, and clean energy power supply unit:

[0144]

[0145] Among them, the operation constraints of the integrated energy system include the operation constraints of the hydrogen energy storage unit, the operation constraints of the battery energy storage unit and the supply and demand power balance constraints;

[0146] Under the operation constraints of the integrated energy system, a pre-trained intelligent agent is used for step-by-step scheduling with the objective function as the goal. The intelligent agent includes a decision network and an evaluation network, and is trained based on a reinforcement learning architecture.

[0147] Of course, the computer program stored in the storage unit of a low-carbon scheduling device for an integrated energy system containing hydrogen storage provided in an embodiment of the present invention is not limited to the operation of the method described above, but can also execute related operations in a low-carbon scheduling method for an integrated energy system containing hydrogen storage provided in any embodiment of the present invention.

[0148] Example 3

[0149] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed, the low-carbon scheduling method of the integrated energy system containing hydrogen storage is implemented, including:

[0150] Construct the unit time cost model of hydrogen energy storage unit, battery energy storage unit, grid power supply unit and clean energy power supply unit: Among them, the unit time cost model of the power supply unit of the power grid is Introducing carbon costs into

[0151] Under the operation constraints of the integrated energy system, the objective function is constructed based on the unit time cost model of the hydrogen energy storage unit, battery energy storage unit, grid power supply unit, and clean energy power supply unit:

[0152]

[0153] Among them, the operation constraints of the integrated energy system include the operation constraints of the hydrogen energy storage unit, the operation constraints of the battery energy storage unit and the supply and demand power balance constraints;

[0154] Under the operation constraints of the integrated energy system, a pre-trained intelligent agent is used for step-by-step scheduling with the objective function as the goal. The intelligent agent includes a decision network and an evaluation network, and is trained based on a reinforcement learning architecture.

[0155] A computer-readable storage medium provided in an embodiment of the present invention stores a computer program which is not limited to the method operations described above, but can also execute related operations in a low-carbon scheduling method for an integrated energy system containing hydrogen storage provided in any embodiment of the present invention.

[0156] In the embodiments provided by the present invention, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the structural embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, structures or units, which can be electrical, mechanical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0159] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A low-carbon dispatching method for an integrated energy system containing hydrogen storage, applied to an integrated energy system including a hydrogen energy storage unit, a battery energy storage unit, a grid power supply unit, and a clean energy power supply unit, wherein the clean energy power supply unit adopts a photovoltaic power generation unit and a wind power generation unit, characterized in that: include: Under the operation constraints of the integrated energy system, the unit time cost model of hydrogen energy storage unit, battery energy storage unit, grid power supply unit and clean energy power supply unit is constructed: Among them, the unit time cost model of the power supply unit of the power grid is Introducing carbon cost into the unit time cost model of hydrogen energy storage unit Oxygen benefits are introduced in the model, and hydrogen leakage costs are introduced by modeling hydrogen leakage; Under the operation constraints of the integrated energy system, the objective function is constructed based on the unit time cost model of the hydrogen energy storage unit, battery energy storage unit, grid power supply unit, and clean energy power supply unit: Among them, the operation constraints of the integrated energy system include the operation constraints of the hydrogen energy storage unit, the operation constraints of the battery energy storage unit and the supply and demand power balance constraints; Under the operation constraints of the integrated energy system, a pre-trained intelligent agent is used for step-by-step scheduling with the objective function as the goal. The intelligent agent includes a decision network and an evaluation network, and is trained based on a reinforcement learning architecture.

2. The low-carbon scheduling method for an integrated energy system containing hydrogen storage according to claim 1, characterized in that: The hydrogen energy storage unit includes an electrolyzer for producing hydrogen by electrolysis, a hydrogen storage tank and an oxygen storage tank for storing hydrogen and oxygen produced by the electrolyzer, and a hydrogen fuel cell connected to the hydrogen storage tank; The operation process of the hydrogen energy storage unit includes the process of electrolysis to produce hydrogen and oxygen and storage and the process of converting the stored hydrogen into electrical energy. The unit time cost of the process of electrolysis to produce hydrogen and oxygen and storage is as follows: Among them, C E is the electrolytic cell cost, L E is the electrolytic cell life, C S is the cost of hydrogen storage tank, L S is the life of the hydrogen storage tank; P t EP is the electrolysis power of the electrolytic cell; is the maintenance cost per unit time of the electrolytic cell at time t, is the maintenance cost per unit time of the electrolytic cell at time t; C o,pr is the price of oxygen per unit mass sold, λ is the mass conversion coefficient of oxygen production; η1 represents the efficiency of hydrogen production by electrolyzer; The cost per unit time of converting stored hydrogen into electricity is as follows: Among them, C F is the cost of hydrogen fuel cells, L F For the life of hydrogen fuel cells, is the maintenance cost per unit time of the hydrogen fuel cell at time t; The unit time cost model of hydrogen energy storage unit is as follows: Among them, Λ is a binary parameter. When Λ is 1, it means that the hydrogen energy storage unit is running, and when Λ is 0, it means that the hydrogen energy storage unit is idle; Ω is a binary parameter. When Ω is 1, it means that the electrolyzer of the hydrogen energy storage unit is running, and when Ω is 0, it means that the hydrogen fuel cell of the hydrogen energy storage unit is running.

3. The low-carbon scheduling method for an integrated energy system containing hydrogen storage according to claim 1, characterized in that: The operating constraints of the hydrogen energy storage unit include: the mass of stored hydrogen follows the following equation: in, represents the mass of hydrogen stored at time t, represents the mass of hydrogen stored at time t+Δt, 0≤P t EP ≤P EP，max , the electrolytic power of the electrolytic cell is limited by the maximum electrolytic power P of the electrolytic cell EP，max , η2 represents the efficiency of hydrogen fuel cell consuming hydrogen to generate electricity, P t Hout is the output power of the hydrogen fuel cell; 0≤P t Hout ≤P Hout,max , the output power of the hydrogen fuel cell is limited by the maximum output power of the fuel cell; for any hydrogen energy storage unit, its electrolyzer and hydrogen fuel cell cannot work at the same time, satisfying P t EP P t Hout =0; Indicates the amount of hydrogen leakage per unit time of the hydrogen energy storage unit, where: is the leakage related to the diffusion of hydrogen molecules, α1 represents the first adjustment coefficient, D is the diffusion coefficient, represents the concentration gradient; α2kPa is the leakage related to hydrogen molecule penetration, α2 represents the second adjustment coefficient, k is the permeability coefficient, and Pa is the hydrogen storage pressure; is the pore leakage, α3 is the third adjustment coefficient, S is a constant representing the leakage area, V H is the volume of hydrogen, ΔPa is the pressure difference between the inside and outside of the leak; Due to the gas pressure and volume restrictions of the hydrogen storage tank, the hydrogen storage capacity of the hydrogen storage tank is limited. The mass of hydrogen stored at any time t satisfies the following constraints: in, is the maximum hydrogen storage mass of the hydrogen storage tank.

4. The low-carbon scheduling method for an integrated energy system containing hydrogen storage according to claim 1, characterized in that: The unit time cost model of a battery energy storage unit is as follows: in, represents the SOC of the battery energy storage unit at time t, L c Battery energy storage power cycle life, C B is the hardware cost of the battery energy storage unit, D t is the depth of charge and discharge, E is the battery capacity, η3 represents the charging efficiency of the battery energy storage unit, and η4 represents the discharging efficiency of the battery energy storage unit.

5. The low-carbon scheduling method for an integrated energy system containing hydrogen storage according to claim 1, characterized in that: The operating constraints of the battery energy storage unit include: The SOC of the battery energy storage unit follows the following equation: in, represents the SOC of the battery energy storage unit at time t; represents the SOC of the battery energy storage unit at time t+Δt; η3 represents the charging efficiency of the battery energy storage unit; P t ch is the charging power of the battery energy storage unit, 0≤P t ch≤P ch，max , the charging power of the battery energy storage unit is limited by the maximum charging power P of the battery energy storage unit ch，max ; η4 represents the discharge efficiency of the battery energy storage unit; P t disch is the output power of the battery energy storage unit, 0≤P t disch ≤P disch,max , the output power of the battery energy storage unit is limited by the maximum output power P of the battery energy storage unit disch,max ; For any battery energy storage unit, it cannot be charged and discharged at the same time, satisfying P t ch P t disch =0; During the operation of the battery energy storage unit, the SOC meets the following requirements: Among them, SOC B,min is the minimum SOC of the battery energy storage unit, SOC B，max is the maximum SOC of the battery energy storage unit.

6. The low-carbon scheduling method for an integrated energy system containing hydrogen storage according to claim 1, characterized in that: In the process of meeting load demand, carbon consumption mainly comes from power supply of the power grid. If carbon cost is introduced into the power supply cost of the power grid, the unit time cost model of the power grid is as follows: in, is the electricity purchase price at time t, At time t, the electricity price is, β is the carbon emission coefficient, βP t grid It represents the carbon emission per unit time. Γ is a binary parameter. When Γ is 1, it means obtaining electricity from the grid. When Γ is 0, it means selling electricity to the grid.

7. The low-carbon scheduling method for an integrated energy system containing hydrogen storage according to claim 1, characterized in that: The integrated energy system needs to meet the supply and demand power balance: P t need +∑P t EP +∑P t ch -∑P t Hout -∑P t disch -P t grid -∑P t g =0; Among them, P t need is the load power demand at time t, ∑P t ch is the sum of the charging powers of all battery energy storage units in the integrated energy system at time t; ∑P t EP is the sum of the electrolysis power of all hydrogen energy storage units in the integrated energy system at time t; ∑P t Hout is the output power sum of all hydrogen energy storage units in the integrated energy system at time t; ∑P t disch is the output power sum of all battery energy storage units in the integrated energy system at time t; P t grid is the power supply of the power grid at time t, when P t grid When the value is negative, it indicates that power is supplied to the grid; ∑P t g is the sum of the power supply powers of all clean energy power supply units in the integrated energy system at time t.

8. The low-carbon scheduling method for an integrated energy system containing hydrogen storage according to claim 1, characterized in that: The unit time cost model of the clean energy power supply unit is as follows: Among them, C gx is the hardware cost of the clean energy power supply unit, L gx For the life of the clean energy power supply unit, The maintenance cost per unit time of the clean energy power supply unit. When x is 1, it represents a photovoltaic power generation unit, and when x is 2, it represents a wind power generation unit.

9. The low-carbon scheduling method for an integrated energy system containing hydrogen storage according to claim 1, characterized in that: Building an intelligent agent: The decision network as an actor and the evaluation network as a critic in reinforcement learning are constructed through three layers of fully connected layers and mapping fully connected layers with a set number of hidden units. The non-mapped fully connected layer uses the tanh activation function, and the mapped fully connected layer uses the relu activation function; Constructing the state of the integrated energy system: The real-time output power P of each clean energy in the integrated energy system t g , Load demand power P t need , the amount of hydrogen stored in the hydrogen energy storage unit SOC of battery storage unit Electricity purchase price and electricity sales price As can be observed: Constructing actions to control the operation of the integrated energy system: In the integrated energy system, the objective function is achieved by adjusting the energy storage unit, and the electrolysis power P of the electrolyzer of the hydrogen energy storage unit is t EP , the output power P of the hydrogen fuel cell t Hout , the charging power P of the battery energy storage unit t ch And the output power P of the battery energy storage unit t disch As an action: Action = (P t EP ,P t Hout ,P t ch ,P t disch ); Build Rewards: Based on the unit time cost of each time step of the objective function, a decision penalty term is introduced to construct the reward for each step: Among them, κ is the proportionality coefficient, U is a constant, D(P t EP ,P t Hout ,P t ch ,P t disch ) is a function to determine whether the action exceeds the constraint limit. When there is an action that exceeds the constraint limit, D(P t EP ,P t Hout ,P t ch ,P t disch ) takes the value of 1, otherwise, D(P t EP ,P t Hout ,P t ch ,P t disch ) takes the value of 0; The reward is accumulated at each step: R=∑ρr t , where ρ is the weighting coefficient; Build experience pool: Get the status of the experience pool based on the number of operations of the integrated energy system.

10. The low-carbon scheduling method for an integrated energy system containing hydrogen storage according to claim 9, characterized in that: Initialize the parameters of the decision network to obtain the decision network Π, initialize the parameters of the evaluation network to obtain the evaluation network Q, and define the target decision network Π' and the target evaluation network Q' obtained through training; Take the initialized Π and Q as the target: Q'←Q, Π'←Π; Randomly select a batch of state experience data from the experience pool; input the current state into the network to obtain the current moment action, obtain the current step reward corresponding to the current moment action and the state combination at the next moment and add them to the experience pool; (Action t ,State t ,r t ,State t+1 ); Random batch sampling of experience data from the experience pool is provided to the target decision network Π', the decision network Π, the target evaluation network Q', and the evaluation network Q; the target decision network Π' and the decision network Π take the state as input and the action as output; The target evaluation network Q', the evaluation network Q takes the state action as input and the evaluation as output; The evaluation mean square error of the evaluation network Q and the target evaluation network Q' for each batch of empirical data is used as the evaluation network loss function, and the parameters of the evaluation network Q are updated using the gradient descent algorithm according to the evaluation network loss function; Among them, the evaluation network loss is: L Q =RMS(Y,Q(State,Action)); Where, Y=(y1,y2,......,y Z ) is the output of the target evaluation network, Z is the batch size; Q (State, Action) is the output of the evaluation network, State, Action are the corresponding actions and states contained in each batch of experience data, and RMS () represents the mean square error; Among them, r t k is the single-step reward corresponding to the output of the k-th target evaluation network, ζ is the discount factor, The action output by the target decision network Π' according to the state at the next moment; The mean of the evaluation of the evaluation network Q on the decision of the decision network Π is used as the loss function of the decision network, and the evaluation is maximized to make the decision optimal; among which, the loss function of the decision network is: L Π =E(Q(State,Π(State))), where E() represents averaging, and the parameters of the decision network Π are updated using the gradient descent algorithm according to the evaluation network loss function; The target evaluation network Q' is updated periodically with the parameters of the evaluation network Q according to the learning rate lr: Q'←lrQ+(1-lr)Q'; According to the learning rate lr, the parameters of the decision network Π are used to update the target decision network Π' regularly: Π'←lrΠ+(1-lr)Π'.

Images