Scheduling optimization method and system of integrated energy system

Abstract

The invention discloses a scheduling optimization method and system for an integrated energy system, and the method comprises the steps: building a non-isothermal model of an HCNG network and a thermodynamic model of a DHN network according to the dynamic characteristics of the flowing of an internal medium of an energy network in the integrated energy system, so as to construct a dynamic model of the integrated energy system, therefore, the interaction flexibility margin between the energy networks in the integrated energy system is determined; using the interaction flexibility margin to construct a day-ahead and real-time two-stage scheduling model of the integrated energy system; and optimizing the decision process of the scheduling model by adopting a depth deterministic strategy gradient algorithm to obtain an optimized scheduling result. According to the method, the day-ahead and real-time two-stage scheduling model of the integrated energy system can be constructed based on the non-isothermal model of the HCNG network, so that the reliability of the scheduling model is improved, and the scheduling operation problem under the condition of no uncertain priori knowledge is solved.

Description

Technical Field

[0001] The present invention relates to the field of integrated energy technology, and in particular to a scheduling optimization method and system for an integrated energy system. Background Art

[0002] Electricity networks, district heating networks (DHNs) and natural gas networks are usually integrated into integrated energy systems. These integrated energy systems are interconnected through energy conversion equipment, which can improve the overall efficiency of the energy system and promote the optimal allocation and efficient use of resources. For example, one is to achieve a one-way interconnection mode between the power grid and the gas grid through combined heat and power (CHP) units, which can improve system performance and efficiency; the other is to achieve a two-way interconnection between the power grid and the gas grid through electrolysis hydrogen production and fuel cell technology, which can enhance the flexibility and synergy of the integrated energy system. The gas grid generally uses a hydrogen-enriched compressed natural gas (HCNG) network, which is a hydrogen-mixed integrated energy network formed by injecting hydrogen into the existing natural gas infrastructure. Therefore, modeling the complex interactions between heterogeneous energy networks in an integrated energy system is crucial to improving system performance.

[0003] However, the current research on the dynamic model of HCNG network is mainly based on the isothermal assumption, ignoring the significant heat exchange process between the pipe wall and the surrounding environment. When this dynamic model is applied to the scheduling of integrated energy systems, it may lead to unreliable scheduling strategies. At the same time, the current research on the optimization of integrated energy system scheduling mainly focuses on three aspects: robust optimization, stochastic programming methods, and distributed robust optimization. However, these three aspects of research require accurate probability distribution of uncertainty to establish the worst case or random distribution, which is difficult to solve the day-ahead-real-time scheduling operation problem without prior knowledge of uncertainty, and cannot fully utilize the scheduling potential of flexible resources in the electricity-heat-gas integrated energy system. Summary of the invention

[0004] The present invention provides a scheduling optimization method and system for an integrated energy system. Based on a non-isothermal model of an HCNG network, the flexibility margin of the integrated energy system is determined, a day-ahead and real-time two-stage scheduling model of the integrated energy system is constructed, the reliability of the scheduling model is improved, and a DDPG optimization scheduling model is adopted to solve scheduling operation problems without prior knowledge of uncertainty.

[0005] In order to achieve the above object, an embodiment of the present invention provides a scheduling optimization method for an integrated energy system, comprising:

[0006] According to the dynamic characteristics of the flow of the medium inside the energy network in the integrated energy system, a non-isothermal model of the HCNG network and a thermodynamic model of the DHN network are established; wherein the integrated energy system is an energy system integrating the power network, the district heating network and the natural gas network;

[0007] According to the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network, a dynamic model of the integrated energy system is constructed;

[0008] Determining the interactive flexibility margin between energy networks in the integrated energy system according to the integrated energy system dynamic model;

[0009] Using the interactive flexibility margin, constructing a day-ahead and real-time two-stage dispatch model of the integrated energy system;

[0010] A deep deterministic policy gradient algorithm is used to optimize the decision-making process of the day-ahead and real-time two-stage scheduling model of the integrated energy system to obtain the optimized scheduling result.

[0011] As an improvement of the above scheme, the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network are established according to the dynamic characteristics of the medium flow inside the energy network in the integrated energy system, including:

[0012] Obtain the dynamic characteristics of mixed gas flow in HCNG network pipelines in integrated energy systems and establish a non-isothermal model of HCNG network;

[0013] Obtaining the dynamic characteristics of water flow in the DHN network pipeline in the integrated energy system and establishing a thermodynamic model of the DHN network;

[0014] Among them, the expression of the non-isothermal model of the HCNG network is:

[0015] m g c p dT g +μ g (T s -T g )dx=0,

[0016] The expression of the thermodynamic model of the DHN network is:

[0017]

[0018] In the formula, m g is the mass flow rate of the HCNG network; c p is the gas heat capacity of the HCNG network; T g is the gas temperature of the HCNG network; μ g is the thermal conductivity of the HCNG network; Ts is the external ambient temperature; c h is the heat capacity of water in the DHN network; h A is the density of water in the DHN network; h is the cross-sectional area of ​​the pipe in the DHN network; h is the temperature of water in the DHN network; m h is the mass flow rate of the DHN network; h is the heat dissipation coefficient of the DHN network.

[0019] As an improvement of the above scheme, the dynamic model of the integrated energy system is constructed according to the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network, including:

[0020] determining a heat exchange process between the HCNG network and an external environment according to a non-isothermal model of the HCNG network;

[0021] determining heat conduction of the DHN network according to a thermodynamic model of the DHN network;

[0022] According to the heat exchange process and the heat conduction, obtaining the temperature of the hydrogen storage system of the integrated energy system;

[0023] Determining the hydrogen doping ratio of the HCNG network according to the Wobbe index;

[0024] Determining the capacity of the hydrogen storage system according to the temperature of the hydrogen storage system and the hydrogen blending ratio;

[0025] A dynamic model of the integrated energy system is constructed according to the temperature of the hydrogen storage system and the capacity of the hydrogen storage system.

[0026] As an improvement of the above solution, determining the interactive flexibility margin between energy networks in the integrated energy system according to the integrated energy system dynamic model includes:

[0027] According to the integrated energy system dynamic model, obtaining the real-time output gas volume of each natural gas source node of the HCNG network and the real-time injection power of each external grid access node of the power network;

[0028] According to the real-time output gas volume and the real-time injected power, the interactive flexibility margin between energy networks in the integrated energy system is obtained.

[0029] As an improvement of the above scheme, the day-ahead and real-time two-stage scheduling model of the integrated energy system includes: a day-ahead scheduling stage model of the integrated energy system and a real-time scheduling stage model of the integrated energy system;

[0030] Among them, the objective function of the integrated energy system day-ahead dispatching stage model is:

[0031] f da (x da )=λ da,E F da,E (x da )-λ da,R F da,R (x da ),

[0032] The constraints of the integrated energy system day-ahead dispatch model are:

[0033] sA da x da ≤ b da ,C da x da =d da ,

[0034] The objective function of the real-time dispatching model of the integrated energy system is:

[0035]

[0036] The constraints of the real-time dispatching model of the integrated energy system are:

[0037]

[0038] In the formula, F da is the target of the day-ahead dispatch model of the integrated energy system; da is the decision variable of the day-ahead dispatching model of the integrated energy system; da,E is the economic benefit weight coefficient; F da,E is the economic benefit target of the day-ahead dispatching model of the comprehensive energy system; da,R is the operational flexibility weight coefficient; F da,R is the operational flexibility target of the day-ahead dispatch model of the integrated energy system; da , b da are the coefficient matrix and constant vector of the inequality constraints of the electricity-heat-gas network; C da d da are the coefficient matrix and constant vector of the equation constraints of the electricity-heat-gas network; F rt is the target of the real-time stage scheduling model of the integrated energy system; rt is the decision variable of the real-time stage scheduling model of the integrated energy system; N is the total number of natural gas source nodes; is the natural gas purchase cost coefficient at time t; is the output gas volume of the nth natural gas source node at time t; M is the total number of external power grid access nodes; is the electricity purchase cost coefficient at the time t; A is the injected power of the mth external grid access node at time t; rt , b rt are the coefficient matrix and constant vector of the inequality constraints related only to the decision variables in the real-time scheduling phase; C rt d rt are the coefficient matrix and constant vector of the equality constraints only related to the decision variables in the real-time scheduling stage; H rt , B rt are the coefficient matrices and constant vectors of other inequality constraints including the electricity-heat-gas network node constraints; D rt 、f rt are the coefficient matrix and constant vector of the inequality constraint containing the uncertainty constraint of renewable energy output; u rt is the uncertainty variable in the real-time dispatching stage of the integrated energy system; da0 is the day-ahead dispatching strategy of the integrated energy system; rt 、E rt are the coefficient matrices and constant vectors of other equation constraints including the electricity-heat-gas network branch constraints; K rt 、p rt are the coefficient matrix and constant vector of the equality constraints containing the flow / temperature mixing constraints, respectively.

[0039] As an improvement of the above scheme, the deep deterministic policy gradient algorithm is used to optimize the decision-making process of the day-ahead and real-time two-stage scheduling model of the integrated energy system to obtain an optimized scheduling result, including:

[0040] Reconstructing the decision process of the day-ahead and real-time two-stage scheduling model of the integrated energy system into a Markov decision process;

[0041] A deep deterministic policy gradient algorithm is used to iteratively solve the state space and decision space of the Markov decision process to obtain the optimized scheduling result.

[0042] As an improvement of the above scheme, the Markov decision process includes: a state space, a decision space, a reward function, a state transition probability distribution function and a discount factor;

[0043] The state space is a set of state variables of the integrated energy system and the external energy network;

[0044] The decision space is a set of decision actions performed by the integrated energy system;

[0045] The reward function is used to characterize the economic benefits of electricity and natural gas transactions and deviation penalties obtained after the integrated energy system executes the decision;

[0046] The state transition probability distribution function represents the probability that the system executes the current decision in the current state and transfers to the next state;

[0047] The discount factor is used to weigh the impact of the immediate reward obtained at each time step on the cumulative reward.

[0048] As an improvement of the above scheme, the deep deterministic policy gradient algorithm includes an actor network and a critic network;

[0049] The expression of the update gradient of the actor network is:

[0050]

[0051] The loss function of the critic network is:

[0052]

[0053] In the formula, θ μ is the actor network parameter; L is the update gradient of the actor network; N l is the total number of time steps; θ Q is the critic network parameter; Q is the critic network; s t is the state variable of the integrated energy system at time t; μ is the actor network; F L is the loss function of the critic network; r t is the state variable s of the integrated energy system at time t t Execution decision t The immediate reward value after γ∈[0,1] is the discount factor; Q′ is the target critic network; s t+1 is the state variable of the integrated energy system at time t+1; μ′ is the target actor network; θ μ′ is the target actor network parameter; θ Q′ are the target critic network parameters.

[0054] In order to achieve the above objectives, an embodiment of the present invention provides a scheduling optimization system for an integrated energy system, including:

[0055] A network model building module is used to build a non-isothermal model of the HCNG network and a thermodynamic model of the DHN network according to the dynamic characteristics of the medium flow inside the energy network in the integrated energy system; wherein the integrated energy system is an energy system integrating the power network, the district heating network and the natural gas network;

[0056] A dynamic model building module, used to build a dynamic model of the integrated energy system according to the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network;

[0057] A flexibility margin determination module, used to determine the interactive flexibility margins between energy networks in the integrated energy system according to the integrated energy system dynamic model;

[0058] A scheduling model building module, used to build a day-ahead and real-time two-stage scheduling model of the integrated energy system by using the interactive flexibility margin;

[0059] The scheduling model optimization module is used to optimize the decision-making process of the day-ahead and real-time two-stage scheduling model of the integrated energy system by adopting a deep deterministic policy gradient algorithm to obtain an optimized scheduling result.

[0060] As an improvement of the above solution, the network model building module is used to:

[0061] Obtain the dynamic characteristics of mixed gas flow in HCNG network pipelines in integrated energy systems and establish a non-isothermal model of HCNG network;

[0062] Obtaining the dynamic characteristics of water flow in the DHN network pipeline in the integrated energy system and establishing a thermodynamic model of the DHN network;

[0063] Among them, the expression of the non-isothermal model of the HCNG network is:

[0064] m g c p dT g +μ g (T s -T g )dx=0,

[0065] The expression of the thermodynamic model of the DHN network is:

[0066]

[0067] In the formula, m g is the mass flow rate of the HCNG network; c p is the gas heat capacity of the HCNG network; T g is the gas temperature of the HCNG network; μ g is the thermal conductivity of the HCNG network; T s is the external ambient temperature; c h is the heat capacity of water in the DHN network; h A is the density of water in the DHN network; h is the cross-sectional area of ​​the pipe in the DHN network;h is the temperature of water in the DHN network; m h is the mass flow rate of the DHN network; h is the heat dissipation coefficient of the DHN network.

[0068] Compared with the prior art, the scheduling optimization method and system of an integrated energy system disclosed in an embodiment of the present invention establish a non-isothermal model of the HCNG network and a thermodynamic model of the DHN network according to the dynamic characteristics of the flow of the medium inside the energy network in the integrated energy system; wherein the integrated energy system is an energy system integrating the power network, the regional heating network and the natural gas network; a dynamic model of the integrated energy system is constructed according to the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network; the interactive flexibility margin between the energy networks in the integrated energy system is determined according to the dynamic model of the integrated energy system; a day-ahead and real-time two-stage scheduling model of the integrated energy system is constructed by using the interactive flexibility margin; a deep deterministic policy gradient algorithm is used to optimize the decision-making process of the day-ahead and real-time two-stage scheduling model of the integrated energy system to obtain an optimized scheduling result. Based on the non-isothermal model of the HCNG network, a dynamic model of the integrated energy system is constructed, which can describe the impact of real-time fluctuations and transient behaviors of heterogeneous energy networks, and provide technical support for fully exploring and utilizing the flexible adjustment potential of heat and gas networks and improving operational efficiency; fully utilize the interactive flexibility margin between energy networks to construct a day-ahead and real-time two-stage scheduling model for the integrated energy system, improve the reliability of the scheduling model, achieve day-ahead scheduling decisions that take into account both economic benefits and operational flexibility, and provide a real-time scheduling decision correction model to deal with system source and load uncertainties; use the DDPG optimization scheduling model to solve scheduling and operation problems without prior knowledge of uncertainty, achieve coordination and cooperation among controllable resources, and ensure the safe and stable operation of the integrated energy system under the access of renewable energy. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a flow chart of a scheduling optimization method for an integrated energy system provided by an embodiment of the present invention;

[0070] Figure 2 It is a structural schematic diagram of energy exchange between integrated energy systems provided by an embodiment of the present invention;

[0071] Figure 3 It is an iterative convergence process of a DDPG algorithm provided by an embodiment of the present invention;

[0072] Figure 4 It is a timing diagram of a day-ahead and real-time two-stage scheduling strategy for a comprehensive energy system provided by an embodiment of the present invention;

[0073] Figure 5 It is a structural diagram of a scheduling optimization system for an integrated energy system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] It should be noted that the terms "comprises" and "specifically" and any variations of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0076] See also Figure 1 , Figure 1 1 is a flow chart of a scheduling optimization method for an integrated energy system provided by an embodiment of the present invention. The scheduling optimization method for an integrated energy system includes:

[0077] S1, establishing a non-isothermal model of the HCNG network and a thermodynamic model of the DHN network according to the dynamic characteristics of the medium flow inside the energy network in the integrated energy system; wherein the integrated energy system is an energy system integrating the power network, the district heating network and the natural gas network;

[0078] S2, constructing a dynamic model of the integrated energy system according to the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network;

[0079] S3, determining the interactive flexibility margin between energy networks in the integrated energy system according to the dynamic model of the integrated energy system;

[0080] S4, constructing a day-ahead and real-time two-stage dispatch model of the integrated energy system using the interactive flexibility margin;

[0081] S5, using a deep deterministic policy gradient algorithm, optimizes the decision-making process of the day-ahead and real-time two-stage scheduling model of the integrated energy system to obtain an optimized scheduling result.

[0082] Exemplarily, the dynamic characteristics of medium flow in various heterogeneous energy networks of the integrated energy system are analyzed, and the dynamic characteristics of the medium flow include the dynamic characteristics of gas flow in the HCNG network pipeline and the dynamic characteristics of water flow in the DHN pipeline. According to the dynamic characteristics of the medium flow in the energy network in the integrated energy system, a non-isothermal model of the HCNG network and a thermodynamic model of the DHN network are established; wherein the integrated energy system is an energy system integrating the power network, the district heating network and the natural gas network, and according to the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network, energy is exchanged with the power network, for example, see Figure 2 , Figure 2 The schematic diagram of the structure of energy exchange between a comprehensive energy system provided by an embodiment of the present invention mainly includes a power network, a HCNG network and a DHN network, and adopts a power-to-hydrogen and heat (P2HH) system to realize the energy conversion of electricity, heat and hydrogen; wherein the power network and the external power grid conduct electric power x Grid Trading, wind turbines and CHP units generate electricity to provide power (P WT and P CHP ) to the power grid, which then transmits the electric energy (x EL and P P2HH ) supplies electricity load and P2HH unit operation; CHP unit generates heat energy Q CHP To DHN network; CHP units in DHN network provide heat energy Q HL , used to supply heat load, waste heat Q in the electrolysis process of P2HH unit P2HH is recovered to provide additional heat source for supplying heat load, hydrogen (m HT and m H ), which is used to store in the hydrogen storage system and supply to the P2HH unit in the HCNG network for operation; the hydrogen produced by the P2HH unit in the HCNG network can be mixed with natural gas x according to a certain hydrogen blending ratio GW Mix, the resulting mixed gas (x GL and m CHP) The gas load and CHP units in the supply network, the hydrogen storage system is used to alleviate the imbalance between natural gas demand and output. A polyhedral uncertainty set is used to represent the uncertainty on the source side and the load side. The non-isothermal model of the HCNG network is used to exchange heat with the external environment to obtain relevant parameters of the integrated energy system (such as temperature, pressure, hydrogen blending ratio, etc.), and build a dynamic model of the integrated energy system; obtain the real-time output gas volume of each natural gas source node of the HCNG network and the real-time injection power of each external grid access node of the power network; obtain the interactive flexibility margin between energy networks in the integrated energy system; use the interactive flexibility margin to determine various constraints and scheduling objectives, and obtain a two-stage day-ahead and real-time scheduling model for the integrated energy system. A deep deterministic policy gradient algorithm is used to optimize the Markov decision process of the two-stage day-ahead and real-time scheduling model of the integrated energy system to obtain an optimized scheduling result.

[0083] Specifically, the step S1 includes:

[0084] S11, obtain the dynamic characteristics of the mixed gas flow in the HCNG network pipeline in the integrated energy system and establish a non-isothermal model of the HCNG network;

[0085] S12, obtaining the dynamic characteristics of water flow in the DHN network pipeline in the integrated energy system, and establishing a thermodynamic model of the DHN network;

[0086] Among them, the expression of the non-isothermal model of the HCNG network is:

[0087] m g c p dT g +μ g (T s -T g )dx=0,

[0088] The expression of the thermodynamic model of the DHN network is:

[0089]

[0090] In the formula, m g is the mass flow rate of the HCNG network; c p is the gas heat capacity of the HCNG network; T g is the gas temperature of the HCNG network; μ g is the thermal conductivity of the HCNG network; T s is the external ambient temperature; c h is the heat capacity of water in the DHN network; h A is the density of water in the DHN network; his the cross-sectional area of ​​the pipe in the DHN network; h is the temperature of water in the DHN network; m h is the mass flow rate of the DHN network; h is the heat dissipation coefficient of the DHN network.

[0091] For example, the dynamic characteristics of the mixed gas flow in the HCNG network pipeline in the integrated energy system can be expressed by the mass conservation equation and the momentum conservation equation, namely:

[0092]

[0093]

[0094] In the formula, p g is the pressure of the mixed gas in the HCNG network pipeline; ρ g is the density of the mixed gas in the HCNG network pipeline; g is the acceleration of gravity; λ g is the friction coefficient; D g is the inner diameter of the HCNG network pipeline; v is the gas molar volume; t is the time; l is the pipeline length.

[0095] The gas velocity is expressed using mass flow rate and is defined as:

[0096] m g =ρ g v g

[0097] In the formula, m g A is the mass flow rate of the HCNG network; g is the pipe cross-sectional area of ​​the HCNG network.

[0098] Ignoring the convection term and pipe inclination of the HCNG network and introducing the mass flow rate, the above mass conservation equation and momentum conservation equation can be transformed into:

[0099]

[0100] Where R is the ideal gas constant; T g is the gas temperature of the HCNG network.

[0101] It is worth noting that the gas in the HCNG network must be pressurized by a compressor to maintain sufficient gas pressure before being supplied to users. According to the first law of thermodynamics, under the condition of constant volume, pressurization will significantly increase the temperature of the gas, and the gas temperature at the gas source node is much higher than the ambient temperature. Due to the high thermal conductivity of the pipeline wall, the heat conduction between the external environment and the gas in the HCNG network is intense, and the gas temperature in the HCNG network will gradually decrease. The large change in gas temperature in the HCNG network greatly affects the assessment of pressure and mass flow rate, resulting in changes in the operating domain of the integrated energy system.

[0102] In the HCNG network non-isothermal model, the enthalpy change is incorporated into the heat exchange process between the gas in the HCNG pipeline and the external environment. The expression (energy conservation equation) of the HCNG network non-isothermal model is:

[0103] m g c p dT g +μ g (T s -T g )dx=0,

[0104] In the formula, c p is the gas heat capacity of the HCNG network; μ g is the thermal conductivity of the HCNG network; T s is the external ambient temperature.

[0105] In the DHN network of the integrated energy system, water is considered to be an incompressible heat carrier, and the dynamic characteristic model of water flow in its pipeline can be derived as follows:

[0106]

[0107] In the formula, m h is the mass flow rate of the DHN network; m h0 is the mass flow reference value of the DHN network; h is the density of water in the DHN network; p h A is the pressure of the DHN network; h is the cross-sectional area of ​​the pipe in the DHN network; h is the heat dissipation coefficient of the DHN network; D h is the inner diameter of the pipe in the DHN network.

[0108] Therefore, the expression of the thermodynamic model of DHN is:

[0109]

[0110] In the formula, c h is the heat capacity of water in the DHN network; Th is the temperature of water in the DHN network; h h =c h m h T h ,h h is the heat flow of the DHN network.

[0111] Specifically, the step S2 includes:

[0112] S21, determining a heat exchange process between the HCNG network and an external environment according to a non-isothermal model of the HCNG network;

[0113] S22, determining the heat conduction of the DHN network according to a thermodynamic model of the DHN network;

[0114] S23, acquiring a temperature of a hydrogen storage system of the integrated energy system according to the heat exchange process and the heat conduction;

[0115] S24, determining the hydrogen doping ratio of the HCNG network according to the Wobbe index;

[0116] S25, determining the capacity of the hydrogen storage system according to the temperature of the hydrogen storage system and the hydrogen blending ratio;

[0117] S26, constructing a dynamic model of the integrated energy system according to the temperature of the hydrogen storage system and the capacity of the hydrogen storage system.

[0118] For example, in the HCNG network non-isothermal model, the heat exchange process between the gas in the HCNG pipeline and the external environment can be expressed as follows:

[0119]

[0120] In the formula, μ g is the thermal conductivity of the HCNG network; T s is the external ambient temperature.

[0121] The heat conduction equation of the DHN network is expressed as follows:

[0122]

[0123] The two-port equivalent frequency domain network characteristics of the DHN network and the HCNG network can be represented by a π-type equivalent circuit. In the π-type equivalent circuit, two grounding branches form a path between the actual node of the pipeline and the grounding node. The energy flow injected into the grounding node through the grounding branch indicates that the inlet flow exceeds the outlet flow, reflecting the "charging" state of the pipeline where the temperature of the compressed natural gas or thermal medium rises; negative injection at the grounding node means that the outflow is greater than the inflow, and the pipeline is in a "releasing" state; the equivalent capacitor is used to describe the energy storage capacity of the pipeline.

[0124] The hydrogen storage system is an important flexibility regulation resource in the integrated energy system. The constraints related to its gas injection / release process are as follows:

[0125]

[0126] In the formula, is the pressure of the hydrogen storage system at time t; ε is the molar mass of hydrogen; T HS is the temperature of the hydrogen storage system; V HS is the capacity of the hydrogen storage system; is the hydrogen injection rate at time t; The upper limit of hydrogen injection / output rate of the hydrogen storage system; is the upper limit of the pressure of the hydrogen storage system; p HS It is the lower limit of the pressure of the hydrogen storage system.

[0127] Specifically, the step S3 includes:

[0128] S31, according to the integrated energy system dynamic model, obtaining the real-time output gas volume of each natural gas source node of the HCNG network and the real-time injection power of each external grid access node of the power network;

[0129] S32, obtaining the interactive flexibility margin between energy networks in the integrated energy system according to the real-time output gas volume and the real-time injected power.

[0130] Specifically, the day-ahead and real-time two-stage scheduling model of the integrated energy system includes: a day-ahead scheduling stage model of the integrated energy system and a real-time scheduling stage model of the integrated energy system;

[0131] Among them, the objective function of the integrated energy system day-ahead dispatching stage model is:

[0132] f da (x da )=λ da,E F da,E (x da )-λ da,R F da,R (x da ),

[0133] The constraints of the integrated energy system day-ahead dispatch model are:

[0134] sA da x da ≤ b da ,C da x da =d da ,

[0135] The objective function of the real-time dispatching model of the integrated energy system is:

[0136]

[0137] The constraints of the real-time dispatching model of the integrated energy system are:

[0138]

[0139] In the formula, F da is the target of the day-ahead dispatch model of the integrated energy system; da is the decision variable of the day-ahead dispatching model of the integrated energy system; da,E is the economic benefit weight coefficient; F da,E is the economic benefit target of the day-ahead dispatch model of the integrated energy system, N is the total number of natural gas source nodes; is the natural gas purchase cost coefficient at time t; is the output gas volume of the nth natural gas source node at time t; M is the total number of external power grid access nodes; is the electricity purchase cost coefficient at the time t; is the injected power of the mth external grid access node at time t; da,R is the operational flexibility weight coefficient; F da,R is the operational flexibility target of the day-ahead dispatch model of the integrated energy system, K is the total number of load nodes; is the allowable power fluctuation of the kth load node at time t; Z is the total number of renewable energy nodes; A is the allowable power fluctuation of the zth renewable energy node at time t; da , b da are the coefficient matrix and constant vector of the inequality constraints of the electricity-heat-gas network; C da d da are the coefficient matrix and constant vector of the equation constraints of the electricity-heat-gas network; F rt is the target of the real-time stage scheduling model of the integrated energy system; rt A is the decision variable of the real-time stage scheduling model of the integrated energy system; rt , b rt are the coefficient matrix and constant vector of the inequality constraints related only to the decision variables in the real-time scheduling phase; C rt d rt are the coefficient matrix and constant vector of the equality constraints only related to the decision variables in the real-time scheduling stage; Hrt , B rt are the coefficient matrices and constant vectors of other inequality constraints including the electricity-heat-gas network node constraints; D rt 、f rt are the coefficient matrix and constant vector of the inequality constraint containing the uncertainty constraint of renewable energy output; u rt is the uncertainty variable in the real-time dispatching stage of the integrated energy system; da0 is the day-ahead dispatching strategy of the integrated energy system; rt 、E rt are the coefficient matrices and constant vectors of other equation constraints including the electricity-heat-gas network branch constraints; K rt 、p rt are the coefficient matrix and constant vector of the equality constraints containing the flow / temperature mixing constraints, respectively.

[0140] It is worth noting that other inequality constraints containing electricity-heat-gas network node constraints represent inequality constraints containing electricity-heat-gas network node constraints other than the inequality constraints containing electricity-heat-gas network node constraints that are only related to the decision variables of the real-time scheduling stage, and other equality constraints containing electricity-heat-gas network branch constraints represent equality constraints containing electricity-heat-gas network branch constraints other than the decision variables of the real-time scheduling stage.

[0141] The embodiment of the present invention makes full use of the interactive flexibility margin between energy networks, constructs the day-ahead and real-time two-stage scheduling model of the integrated energy system, realizes the day-ahead scheduling decision that takes into account both economic benefits and operational flexibility, and provides a reference solution for correcting the real-time scheduling decision to deal with the uncertainty of system sources and loads. The day-ahead scheduling stage model of the integrated energy system is used to solve the hourly operation strategy and provide guidance for the operation plan of the inelastic units, including determining the switching state of the generator and managing the transactions between the integrated energy system and the external energy network, and providing adjustment margin for subsequent real-time scheduling. The real-time scheduling stage model of the integrated energy system is used to further adjust the operation strategy obtained from the day-ahead scheduling strategy on a 15-minute time scale to deal with the uncertainty caused by random renewable energy generation and uncontrolled loads.

[0142] The step S5 comprises:

[0143] S51, reconstructing the decision process of the day-ahead and real-time two-stage scheduling model of the integrated energy system into a Markov decision process;

[0144] S52, using the deep deterministic policy gradient algorithm, iteratively solves in the state space and decision space of the Markov decision process to obtain the optimized scheduling result.

[0145] In order to circumvent the limitation of dimensionality and improve the optimization performance in the absence of prior knowledge of system uncertainty, the embodiment of the present invention adopts a deep deterministic policy gradient algorithm (DDPG) to optimize the scheduling decision process of the integrated energy system, and adjusts various flexible resources in the integrated energy system to adapt to the random renewable energy output.

[0146] Specifically, the Markov decision process includes: a state space, a decision space, a reward function, a state transition probability distribution function and a discount factor;

[0147] The state space is a set of state variables of the integrated energy system and the external energy network;

[0148] The decision space is a set of decision actions performed by the integrated energy system;

[0149] The reward function is used to characterize the economic benefits of electricity and natural gas transactions and deviation penalties obtained after the integrated energy system executes the decision;

[0150] The state transition probability distribution function represents the probability that the system executes the current decision in the current state and transfers to the next state;

[0151] The discount factor is used to weigh the impact of the immediate reward obtained at each time step on the cumulative reward.

[0152] Exemplarily, the state space is a set of state variables of the integrated energy system and the external energy network, namely:

[0153]

[0154] In the formula, s t is the state variable at time t; S is the state space set; is the pressure of the ith node of the HCNG network at time t; is the temperature of DHN at the jth node at time t; is the pressure of the hydrogen storage system at time t.

[0155] The decision space is the set of decision actions executed by the integrated energy system, namely:

[0156]

[0157] In the formula, a t is the decision action executed at time t; A is the decision space set; is the injected gas mass flow rate of P2HH at time t during the day-ahead scheduling phase; is the injection gas mass flow correction value at time t in the P2HH real-time scheduling phase; is the injected gas mass flow rate at time t in the CHP day-ahead scheduling phase; is the injection gas mass flow correction value at time t in the CHP real-time scheduling stage; is the hydrogen injection rate at time t in the day-ahead scheduling phase; It is the correction value of hydrogen injection rate at time t in the real-time scheduling stage; is the injected power of the mth external grid access node at time t during the day-ahead scheduling phase; is the injection power correction value of the mth external grid access node at time t in the real-time scheduling stage; is the output gas volume of the nth natural gas source node at time t during the day-ahead scheduling phase; It is the corrected value of the output gas volume of the nth natural gas source node at time t in the real-time scheduling stage.

[0158] The reward function is used to characterize the execution decision of the integrated energy system. t The economic benefits of electricity and natural gas trading and the deviation penalty are obtained, namely

[0159]

[0160] In the formula, r t For the integrated energy system in state s t Execution decision t The immediate reward value after λ S is the deviation penalty factor; D S (s t ,a t ) represents the state variable s of the integrated energy system t Execution decision t The degree of deviation from the feasible region.

[0161] The state transition probability distribution function represents the system in state s t Execution decision t Transfer to state s t+1 The probability of

[0162] f=S×A×S→[0,1],

[0163] The discount factor is used to weigh the impact of the immediate reward obtained at each time step on the cumulative reward. The cumulative reward from time t to the end of time T is defined as:

[0164]

[0165] Where γ∈[0,1] is the discount factor. The lower the value, the more the decision maker values ​​short-term returns. The higher the value, the more likely it is that the decision maker will adopt a more forward-looking strategy. t For cumulative rewards.

[0166] Specifically, the deep deterministic policy gradient algorithm includes an actor network and a critic network;

[0167] The expression of the update gradient of the actor network is:

[0168]

[0169] The loss function of the critic network is:

[0170]

[0171] In the formula, θ μ is the actor network parameter; L is the update gradient of the actor network; N l is the total number of time steps; θ Q is the critic network parameter; Q is the critic network; s t is the state variable of the integrated energy system at time t; μ is the actor network; F L is the loss function of the critic network; r t is the state variable s of the integrated energy system at time t t Execution decision t The immediate reward value after γ∈[0,1] is the discount factor; Q′ is the target critic network; s t+1 is the state variable of the integrated energy system at time t+1; μ′ is the target actor network; θ μ′ is the target actor network parameter; θ Q′ are the target critic network parameters.

[0172] For example, for the reconstructed Markov decision process of the integrated energy system scheduling problem, the DDPG reinforcement learning algorithm with an actor-critic framework is used to iteratively solve the optimal integrated energy system scheduling strategy in the continuous state space and decision space. The iterative convergence process of the DDPG algorithm is as follows: Figure 3 As shown, Figure 3 It is an iterative convergence process of a DDPG algorithm provided by an embodiment of the present invention; the y-axis represents the expected reward corresponding to the scheduling strategy. Through strategy iteration, the expected reward moving average of the scheduling scheme generated by the action network shows an overall oscillating upward trend. The strategy oscillation is more obvious at the beginning of training, but gradually converges to a stable value. The integrated energy system scheduling scheme generated by the trained action network has a lower operating cost and meets the requirements of optimized scheduling. The actor-critic framework includes a policy function and a state-action value function. The policy function acts as an actor and in the system state s t Generate decision a tand interact with it. The state-action value function acts as a critic, evaluating the effectiveness of the actor's decision and providing guidance for subsequent actions. The Bellman optimal equation is used to recursively evaluate the optimal strategy, namely:

[0173] Q(a t ,s t )=r(a t ,s t )+γE[maxQ(a t+1 ,s t+1 )],

[0174] In the formula, Q(a t ,s t ) is state s t Execute decision a t The state action value function of r(a t ,s t ) is state s t Execute decision a t The immediate reward value obtained; E[·] is the expectation.

[0175] The DDPG algorithm uses a neural network to establish two approximate functions of the actor-critic framework, including the parameter θ μ The actor network μ and the parameter θ Q The critic network Q. The actor network receives the system state s t , and in a t =π(s t |θ μ ) generates a deterministic policy, while the critic network evaluates the associated cost Q(s t ,a t |θ μ ) to assist in the correction of the actor network. Therefore, the Q value can be expressed as follows:

[0176] Q(a t ,s t )=r(a t ,s t )+γE[Q(s t+1 ,μ(s t+1 |θ μ )|θ Q )],

[0177] In the formula, θ μ are the actor network parameters; θ Q is the critic network parameter. The goal of critic network training is to minimize the loss, that is:

[0178]

[0179] In the formula, F Lis the critic network loss function; N l is the total number of time steps; Q′ is the target critic network; μ′ is the target actor network; θ μ′ is the target actor network parameter; θ Q′ are the target critic network parameters.

[0180] Based on the gradient descent algorithm, the following updated gradient of the actor network can be obtained to optimize the actor network parameter θ μ ,Right now:

[0181]

[0182] Where L is the update gradient of the actor network.

[0183] In order to ensure the stability of the iterative process, DDPG combines the target actor and target critic networks and implements updates to solve the optimal scheduling strategy of the integrated energy system. According to the optimal scheduling strategy of the integrated energy system, the optimized scheduling result is obtained.

[0184] like Figure 4 As shown, Figure 4 It is a timing diagram of a day-ahead and real-time two-stage scheduling strategy for an integrated energy system provided by an embodiment of the present invention. Figure 4 a shows the day-ahead dispatch and real-time dispatch scheme of CHP output power. During the period from 0:00 to 6:00, the output of the CHP unit is reduced, and more electricity is converted into hydrogen through P2HH. 24.5MWh of uncertain wind energy is stored in the form of hydrogen in hydrogen tanks. Figure 4 As shown in b, during the period from 6:00 to 12:00, the power consumption of P2HH was reduced by 33.9MWh to meet the random load demand, saving $4407 in operating costs. The gas supply required during this period was provided by the hydrogen storage system. During the period from 12:00 to 18:00, the gas in the HCNG network pipeline was injected into the CHP unit to increase the output power by 15.0MWh, and the power consumption of P2HH was also reduced by 7.4MW to cope with the uncertainty of renewable energy output. Due to the avoidance of additional power purchases from the grid during the peak period of time-of-use electricity prices, operating costs were further reduced by $2912. Figure 4 As shown in c, the real-time dispatch correction of the power grid is moderate compared with CHP and P2HH. Figure 4 As shown in Figure d, flexible resources including hydrogen storage system, DHN and HCNG network provide sufficient support for real-time power balance and can avoid load loss penalty of 4202. Therefore, compared with the traditional scheduling scheme, the proposed two-stage optimization strategy of day-ahead-real-time scheduling can provide an economic benefit of $11521.

[0185] The embodiment of the present invention discloses a scheduling optimization method for an integrated energy system. The method establishes a non-isothermal model of a HCNG network and a thermodynamic model of a DHN network according to the dynamic characteristics of the flow of a medium inside an energy network in the integrated energy system. The integrated energy system is an energy system integrating an electric power network, a regional heating network and a natural gas network. The dynamic model of the integrated energy system is constructed according to the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network. The interactive flexibility margin between energy networks in the integrated energy system is determined according to the dynamic model of the integrated energy system. The day-ahead and real-time two-stage scheduling model of the integrated energy system is constructed by using the interactive flexibility margin. The decision-making process of the day-ahead and real-time two-stage scheduling model of the integrated energy system is optimized by using a deep deterministic policy gradient algorithm to obtain an optimized scheduling result. Based on the non-isothermal model of the HCNG network, a dynamic model of the integrated energy system is constructed, which can describe the impact of real-time fluctuations and transient behaviors of heterogeneous energy networks, and provide technical support for fully exploring and utilizing the flexible adjustment potential of heat and gas networks and improving operational efficiency; fully utilize the interactive flexibility margin between energy networks to construct a day-ahead and real-time two-stage scheduling model for the integrated energy system, improve the reliability of the scheduling model, achieve day-ahead scheduling decisions that take into account both economic benefits and operational flexibility, and provide a real-time scheduling decision correction model to deal with system source and load uncertainties; use the DDPG optimization scheduling model to solve scheduling and operation problems without prior knowledge of uncertainty, achieve coordination and cooperation among controllable resources, and ensure the safe and stable operation of the integrated energy system under the access of renewable energy.

[0186] See also Figure 5 , Figure 5 1 is a schematic diagram of a scheduling optimization system 10 for an integrated energy system provided by an embodiment of the present invention. The scheduling optimization system 10 for an integrated energy system includes:

[0187] The network model building module 11 is used to build a non-isothermal model of the HCNG network and a thermodynamic model of the DHN network according to the dynamic characteristics of the medium flow inside the energy network in the integrated energy system; wherein the integrated energy system is an energy system integrating the power network, the district heating network and the natural gas network;

[0188] A dynamic model building module 12, used to build a dynamic model of the integrated energy system according to the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network;

[0189] A flexibility margin determination module 13, used to determine the interactive flexibility margin between energy networks in the integrated energy system according to the integrated energy system dynamic model;

[0190] A scheduling model building module 14, used to build a day-ahead and real-time two-stage scheduling model of the integrated energy system using the interactive flexibility margin;

[0191] The scheduling model optimization module 15 is used to optimize the decision-making process of the day-ahead and real-time two-stage scheduling model of the integrated energy system by using a deep deterministic policy gradient algorithm to obtain an optimized scheduling result.

[0192] Specifically, the network model building module 11 is used to:

[0193] Obtain the dynamic characteristics of mixed gas flow in HCNG network pipelines in integrated energy systems and establish a non-isothermal model of HCNG network;

[0194] Obtaining the dynamic characteristics of water flow in the DHN network pipeline in the integrated energy system and establishing a thermodynamic model of the DHN network;

[0195] Among them, the expression of the non-isothermal model of the HCNG network is:

[0196] m g c p dT g +μ g (T s -T g )dx=0,

[0197] The expression of the thermodynamic model of the DHN network is:

[0198]

[0199] In the formula, m g is the mass flow rate of the HCNG network; c p is the gas heat capacity of the HCNG network; T g is the gas temperature of the HCNG network; μ g is the thermal conductivity of the HCNG network; T s is the external ambient temperature; c h is the heat capacity of water in the DHN network; h A is the density of water in the DHN network; h is the cross-sectional area of ​​the pipe in the DHN network; h is the temperature of water in the DHN network; m h is the mass flow rate of the DHN network; h is the heat dissipation coefficient of the DHN network.

[0200] A scheduling optimization system 10 for an integrated energy system provided in an embodiment of the present invention can implement all processes of the scheduling optimization method for the integrated energy system in the above-mentioned embodiment. The functions of each module in the system and the technical effects achieved are respectively the same as the functions and technical effects achieved by the scheduling optimization method for the integrated energy system in the above-mentioned embodiment, and will not be repeated here.

[0201] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A scheduling optimization method for an integrated energy system, characterized in that: include: According to the dynamic characteristics of the flow of the medium inside the energy network in the integrated energy system, a non-isothermal model of the HCNG network and a thermodynamic model of the DHN network are established; wherein the integrated energy system is an energy system integrating the power network, the district heating network and the natural gas network; According to the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network, a dynamic model of the integrated energy system is constructed; Determining the interactive flexibility margin between energy networks in the integrated energy system according to the integrated energy system dynamic model; Using the interactive flexibility margin, constructing a day-ahead and real-time two-stage dispatch model of the integrated energy system; A deep deterministic policy gradient algorithm is used to optimize the decision-making process of the day-ahead and real-time two-stage scheduling model of the integrated energy system to obtain the optimized scheduling result.

2. The scheduling optimization method of the integrated energy system according to claim 1, characterized in that: According to the dynamic characteristics of the flow of the medium inside the energy network in the integrated energy system, a non-isothermal model of the HCNG network and a thermodynamic model of the DHN network are established, including: Obtain the dynamic characteristics of mixed gas flow in HCNG network pipelines in integrated energy systems and establish a non-isothermal model of HCNG network; Obtaining the dynamic characteristics of water flow in the DHN network pipeline in the integrated energy system and establishing a thermodynamic model of the DHN network; Among them, the expression of the non-isothermal model of the HCNG network is: m g c p dT g +μ g (T s -T g )dx=0, The expression of the thermodynamic model of the DHN network is: In the formula, m g is the mass flow rate of the HCNG network; c p is the gas heat capacity of the HCNG network; T g is the gas temperature of the HCNG network; μ g is the thermal conductivity of the HCNG network; T s is the external ambient temperature; c h is the heat capacity of water in the DHN network; h A is the density of water in the DHN network; h is the cross-sectional area of ​​the pipe in the DHN network; h is the temperature of water in the DHN network; m h is the mass flow rate of the DHN network; h is the heat dissipation coefficient of the DHN network.

3. The scheduling optimization method of the integrated energy system according to claim 1, characterized in that: The integrated energy system dynamic model is constructed based on the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network, including: determining a heat exchange process between the HCNG network and an external environment according to a non-isothermal model of the HCNG network; determining heat conduction of the DHN network according to a thermodynamic model of the DHN network; According to the heat exchange process and the heat conduction, obtaining the temperature of the hydrogen storage system of the integrated energy system; Determining the hydrogen doping ratio of the HCNG network according to the Wobbe index; Determining the capacity of the hydrogen storage system according to the temperature of the hydrogen storage system and the hydrogen blending ratio; A dynamic model of the integrated energy system is constructed according to the temperature of the hydrogen storage system and the capacity of the hydrogen storage system.

4. The scheduling optimization method of the integrated energy system according to claim 1, characterized in that: Determining the interactive flexibility margin between energy networks in the integrated energy system according to the integrated energy system dynamic model includes: According to the integrated energy system dynamic model, obtaining the real-time output gas volume of each natural gas source node of the HCNG network and the real-time injection power of each external grid access node of the power network; According to the real-time output gas volume and the real-time injected power, the interactive flexibility margin between energy networks in the integrated energy system is obtained.

5. The scheduling optimization method of the integrated energy system according to claim 4, characterized in that: The day-ahead and real-time two-stage scheduling model of the integrated energy system includes: a day-ahead scheduling stage model of the integrated energy system and a real-time scheduling stage model of the integrated energy system; Among them, the objective function of the integrated energy system day-ahead dispatching stage model is: minF da (x da )=λ da,E F da,E (x da )-λ da,R F da,R (x da ), The constraints of the integrated energy system day-ahead dispatch model are: s.t.A da x da ≤b da ,C da x da =d da , The objective function of the real-time dispatching model of the integrated energy system is: The constraints of the real-time dispatching model of the integrated energy system are: In the formula, F da is the target of the day-ahead dispatch model of the integrated energy system; da is the decision variable of the day-ahead dispatching model of the integrated energy system; da,E is the economic benefit weight coefficient; F da,E is the economic benefit target of the day-ahead dispatching model of the comprehensive energy system; da,R is the operational flexibility weight coefficient; F da,R is the operational flexibility target of the day-ahead dispatch model of the integrated energy system; da 、b da are the coefficient matrix and constant vector of the inequality constraints of the electricity-heat-gas network; C da ,d da are the coefficient matrix and constant vector of the equation constraints of the electricity-heat-gas network; F rt is the target of the real-time stage scheduling model of the integrated energy system; rt is the decision variable of the real-time stage scheduling model of the integrated energy system; N is the total number of natural gas source nodes; is the natural gas purchase cost coefficient at time t; is the output gas volume of the nth natural gas source node at time t; M is the total number of external power grid access nodes; is the electricity purchase cost coefficient at the time t; A is the injected power of the mth external grid access node at time t; rt 、b rt are the coefficient matrix and constant vector of the inequality constraints related only to the decision variables in the real-time scheduling phase; C rt ,d rt are the coefficient matrix and constant vector of the equality constraints only related to the decision variables in the real-time scheduling stage; H rt , B rt are the coefficient matrices and constant vectors of other inequality constraints including the electricity-heat-gas network node constraints; D rt 、f rt are the coefficient matrix and constant vector of the inequality constraint containing the uncertainty constraint of renewable energy output; u rt is the uncertainty variable in the real-time dispatching stage of the integrated energy system; da0 is the day-ahead dispatching strategy of the integrated energy system; rt 、E rt are the coefficient matrices and constant vectors of other equation constraints including the electricity-heat-gas network branch constraints; K rt 、p rt are the coefficient matrix and constant vector of the equality constraints containing the flow / temperature mixing constraints, respectively.

6. The scheduling optimization method of the integrated energy system according to claim 1, characterized in that: The deep deterministic policy gradient algorithm is used to optimize the decision-making process of the day-ahead and real-time two-stage scheduling model of the integrated energy system to obtain an optimized scheduling result, including: Reconstructing the decision process of the day-ahead and real-time two-stage scheduling model of the integrated energy system into a Markov decision process; A deep deterministic policy gradient algorithm is used to iteratively solve the state space and decision space of the Markov decision process to obtain the optimized scheduling result.

7. The scheduling optimization method of the integrated energy system according to claim 6, characterized in that: The Markov decision process includes: a state space, a decision space, a reward function, a state transition probability distribution function and a discount factor; The state space is a set of state variables of the integrated energy system and the external energy network; The decision space is a set of decision actions performed by the integrated energy system; The reward function is used to characterize the economic benefits of electricity and natural gas transactions and deviation penalties obtained after the integrated energy system executes the decision; The state transition probability distribution function represents the probability that the system executes the current decision in the current state and transfers to the next state; The discount factor is used to weigh the impact of the immediate reward obtained at each time step on the cumulative reward.

8. The scheduling optimization method of the integrated energy system according to claim 6, characterized in that: The deep deterministic policy gradient algorithm includes an actor network and a critic network; The expression of the update gradient of the actor network is: The loss function of the critic network is: In the formula, θ μ is the actor network parameter; L is the update gradient of the actor network; N l is the total number of time steps; θ Q is the critic network parameter; Q is the critic network; s t is the state variable of the integrated energy system at time t; μ is the actor network; F L is the loss function of the critic network; r t is the state variable s of the integrated energy system at time t t Execution decision t The immediate reward value after γ∈[0,1] is the discount factor; Q′ is the target critic network; s t+1 is the state variable of the integrated energy system at time t+1; μ′ is the target actor network; θ μ′ is the target actor network parameter; θ Q′ are the target critic network parameters.

9. A scheduling optimization system for an integrated energy system, characterized in that: include: A network model building module is used to build a non-isothermal model of the HCNG network and a thermodynamic model of the DHN network according to the dynamic characteristics of the medium flow inside the energy network in the integrated energy system; wherein the integrated energy system is an energy system integrating the power network, the district heating network and the natural gas network; A dynamic model building module, used to build a dynamic model of the integrated energy system according to the non-isothermal model of the HCNG network and the thermodynamic model of the DHN network; A flexibility margin determination module, used to determine the interactive flexibility margins between energy networks in the integrated energy system according to the integrated energy system dynamic model; A scheduling model building module, used to build a day-ahead and real-time two-stage scheduling model of the integrated energy system by using the interactive flexibility margin; The scheduling model optimization module is used to optimize the decision-making process of the day-ahead and real-time two-stage scheduling model of the integrated energy system by adopting a deep deterministic policy gradient algorithm to obtain an optimized scheduling result.

10. The dispatch optimization system of the integrated energy system according to claim 9, characterized in that: The network model building module is used to: Obtain the dynamic characteristics of mixed gas flow in HCNG network pipelines in integrated energy systems and establish a non-isothermal model of HCNG network; Obtaining the dynamic characteristics of water flow in the DHN network pipeline in the integrated energy system and establishing a thermodynamic model of the DHN network; Among them, the expression of the non-isothermal model of the HCNG network is: m g c p dT g +μ g (T s -T g )dx=0, The expression of the thermodynamic model of the DHN network is: In the formula, m g is the mass flow rate of the HCNG network; c p is the gas heat capacity of the HCNG network; T g is the gas temperature of the HCNG network; μ g is the thermal conductivity of the HCNG network; T s is the external ambient temperature; c h is the heat capacity of water in the DHN network; h A is the density of water in the DHN network; h is the cross-sectional area of ​​the pipe in the DHN network; h is the temperature of water in the DHN network; m h is the mass flow rate of the DHN network; h is the heat dissipation coefficient of the DHN network.

Images