Multi-objective optimization scheduling method for integrated energy microgrid based on exergy efficiency

Abstract

The present invention provides a multi-objective optimization scheduling method for an integrated energy microgrid based on exergy efficiency. The method involves proportionally allocating the natural gas power generated by the methane reactor to calculate the natural gas power input to the hydrogen-doped cogeneration and hydrogen-doped gas boilers, thereby obtaining the natural gas exergy input to the integrated energy microgrid. A hydrogen production allocation coefficient is set based on the ratio of hydrogen power output from photocatalytic hydrogen production to the total hydrogen power output from alkaline electrolyzer and photocatalytic hydrogen production. A hydrogen storage tank output allocation coefficient is set based on the ratio of hydrogen energy stored in the hydrogen storage tank from photocatalytic hydrogen production and alkaline electrolyzer production to obtain the hydrogen energy exergy input to the integrated energy microgrid. The method then calculates the overall exergy efficiency of the integrated energy microgrid. Real-time scheduling is performed with the optimization objectives of improving the overall exergy efficiency, economy, and low carbon performance of the integrated energy microgrid. This method ensures efficient energy use within the system while maintaining economy and low carbon performance.

Description

Technical Field

[0001] The present invention relates to the technical field of optimized operation of an integrated energy microgrid with multiple hydrogen production sources, and in particular to a multi-objective optimization scheduling method for an integrated energy microgrid based on exergy efficiency. Background Art

[0002] Integrated energy microgrids integrate multiple distributed energy sources (solar, wind, and energy storage), combined heat and power generation, and other equipment, enabling efficient and coordinated operation of these devices. This is crucial to the development of new power systems. Therefore, research on optimized scheduling methods for integrated energy microgrids has important practical value and practical significance.

[0003] At present, there have been a lot of studies on the optimization scheduling methods of integrated energy microgrids. Invention patent CN118630735A takes into account the diversified utilization of hydrogen energy, and constructs a low-carbon economic scheduling model for hydrogen-containing energy power systems with the optimization goals of minimizing carbon trading costs and system operating costs. Invention patent CN118863430A constructs and solves a master-slave game model for a hydrogen-containing integrated energy system, achieving dual optimization of the economy and low carbon of different stakeholders. However, with the rapid development of new power systems, in addition to improving the economy and low carbon of system operation, higher requirements are also placed on the energy efficiency of integrated energy microgrid operation. However, the above invention patent only focuses on the "quantity" of energy, but ignores the "quality" differences between different energies. Its optimization scheduling results do not take into account the energy efficiency of system operation, and have certain limitations. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the technical problem to be solved by the present invention is to provide a multi-objective optimization scheduling method for an integrated energy microgrid based on exergy efficiency, which is used to solve the problems existing in the existing technology.

[0005] The present invention solves the technical problem by adopting the following technical solutions:

[0006] A multi-objective optimization scheduling method for integrated energy microgrid based on exergy efficiency includes the following contents:

[0007] The natural gas power generated by the methane reactor is proportionally allocated to calculate the natural gas power input to the hydrogen-blended cogeneration and hydrogen-blended gas boilers, and then the natural gas exergy input to the integrated energy microgrid is calculated;

[0008] The hydrogen production apportionment coefficient is set according to the ratio of the hydrogen power output by photocatalytic hydrogen production to the total hydrogen power output by the alkaline electrolyzer hydrogen production and photocatalytic hydrogen production;

[0009] The hydrogen storage tank output allocation ratio coefficient is set according to the ratio of hydrogen energy stored in the hydrogen storage tank from photocatalytic hydrogen production and alkaline electrolysis cell hydrogen production;

[0010] The hydrogen production apportionment ratio coefficient and the hydrogen storage tank output apportionment ratio coefficient are then used to calculate the hydrogen power allocated to each hydrogen-consuming device by photocatalytic hydrogen production, and then the hydrogen energy exergy input into the integrated energy microgrid by photocatalytic hydrogen production is calculated;

[0011] The overall exergy efficiency of the integrated energy microgrid is calculated by combining the exergy of the electrical load and the thermal load, the exergy of the integrated energy microgrid selling electricity to the main grid, the exergy of the integrated energy microgrid purchasing electricity, the exergy of the photovoltaic generator set, and the exergy of the wind turbine set.

[0012] With the optimization goal of improving the overall exergy efficiency, economy and low carbon of the integrated energy microgrid, deep reinforcement learning is used to achieve real-time scheduling of the integrated energy microgrid.

[0013] The equipment used in the integrated energy microgrid includes photocatalytic hydrogen production, alkaline electrolyzers, new energy generator sets, hydrogen-blended gas boilers, hydrogen-blended cogeneration, hydrogen fuel cells, carbon capture, energy storage equipment and methane reactors; among them, the energy storage equipment includes electrical energy storage, thermal energy storage and hydrogen storage tanks, and the new energy generator sets include photovoltaic generator sets and wind turbine generator sets.

[0014] The calculation formula of the overall exergy efficiency of the integrated energy microgrid is:

[0015] ;

[0016] in, is the exergy efficiency at time t; 、 are the exergy of electrical load and thermal load at time t respectively; Exergy that sells electricity to the main grid for integrated energy microgrids; Purchase electricity for integrated energy microgrids; 、 They are the power generation exergy of photovoltaic generator sets and the power generation exergy of wind turbine generator sets respectively; The hydrogen energy exergy that is input into the integrated energy microgrid for photocatalytic hydrogen production, This is the natural gas exergy input into the integrated energy microgrid.

[0017] The optimal solution distance method is used to integrate the three indicators of exergy efficiency, economy and low carbon, and construct the objective function of multi-objective optimization; the economy indicator is the operating cost of the integrated energy microgrid, and the low carbon indicator is the actual carbon emissions E of the integrated energy microgrid. q ; Set the objective function Target(I t )for:

[0018]

[0019] Among them, I tis the high-dimensional point in the target value space combining exergy efficiency, economy and low carbon performance at time t; best is the theoretical optimal solution of the scheduling scheme; “|| ||2” represents the two-norm.

[0020] Based on the constructed objective function, the integrated energy microgrid scheduling process is expressed as a Markov decision process, including state space, action space and reward function. The deep reinforcement learning adopts a double-delay deep deterministic policy gradient algorithm; the action space is the action space under the over-limit truncation guarantee. , defined as:

[0021]

[0022] in, is the lower limit of hydrogen doping ratio, which is 0; It is the regulating variable of the charging / discharging power of the electric energy storage; is the power change rate of device i; P i,ramping is the upper limit of the climbing of device i; P i,t-1 is the operating power of device i at time t-1; and are the regulating variables of the thermoelectric ratio and hydrogen doping ratio respectively; is the adjustable upper limit of the thermoelectric ratio; It is the adjustable lower limit of the thermoelectric ratio; is the mapping coefficient of hydrogen doping ratio; P i,min is the minimum operating power of device i; P i,max is the maximum operating power of device i.

[0023] By constructing a Markov decision process in a random environment and setting an action space under over-limit truncation protection, the system is guaranteed to operate within the safety constraints and achieve safe and stable operation of the system; based on the knowledge of the physical model, a reward function with penalty terms containing objective functions and operation constraints is set to guide the intelligent agent to explore safe, efficient, economical and low-carbon scheduling solutions in an uncertain environment.

[0024] Furthermore, the scheduling method also includes a knowledge rule module, which includes a conditional judgment process and a knowledge rule replacement process. The conditional judgment process refers to identifying the current state of the integrated energy microgrid and judging whether the state is within the application scope of the knowledge rule; the knowledge rule replacement process is to generate the action of the corresponding equipment if the current state is within the application scope of the knowledge rule, and randomly replace the corresponding action in the action space according to the set action output probability.

[0025] Furthermore, the action output probability for:

[0026]

[0027] in, is the action selection control probability reduction factor; z is the number of rounds of interaction between the double-delay deep deterministic policy gradient algorithm and the integrated energy microgrid.

[0028] The knowledge rules are the knowledge rule requirements for the operation of corresponding equipment set based on the system optimization objectives and the operating status of the integrated energy microgrid.

[0029] The knowledge rule module is introduced into the double-delay deep deterministic policy gradient algorithm, and the double-delay deep deterministic policy gradient algorithm is executed with the actions processed by the knowledge rule module.

[0030] Considering that electric energy storage should provide electric energy to the integrated energy microgrid during high electricity price periods to alleviate energy supply pressure and improve system economy, the knowledge rule of electric energy storage is to set the electric energy storage discharge action according to the upper and lower limit constraints of the electric energy storage capacity and the upper limit of the electric energy storage discharge power. The knowledge rule is expressed as:

[0031]

[0032] in, is the regulating variable of the energy storage charging / discharging power; Clip is the truncation function, and the truncation range is [0,1]; The mean is μ and the variance is Gaussian distribution function; is the storage capacity of the electric energy storage at t-1; The upper limit of electric energy storage capacity; The upper limit of charge / discharge power; 、 The start and end times of the high electricity price period.

[0033] Considering that electric boilers should minimize electricity consumption during periods of high electricity prices to avoid increasing electricity consumption and causing an increase in the operating costs of the integrated energy microgrid, the knowledge rule for electric boilers is to set the shutdown action of the electric boiler according to the ramp constraint of the electric boiler. The knowledge rule is expressed as:

[0034]

[0035] in, is the rate of change of electric power input to the electric boiler, is the electric power input to the electric boiler at the last moment, P EB,ramping It is the upper limit of the electric power ramp of the electric boiler.

[0036] Considering that the alkaline electrolyzer should also minimize electricity consumption during high electricity price periods to avoid increasing electricity consumption and causing an increase in the operating cost of the integrated energy microgrid, the knowledge rule of the alkaline electrolyzer is to set the shutdown action of the alkaline electrolyzer according to the alkaline electrolyzer's ramp constraint; the knowledge rule is expressed as:

[0037]

[0038] in, is the rate of change of electric power input to the alkaline electrolytic cell, P P2G,t-1 P is the electric power input to the alkaline electrolytic cell at the last moment; P2G,ramping It is the upper limit of the electric power ramp of the alkaline electrolyzer.

[0039] Considering that the power output of hydrogen-blended cogeneration should be increased during periods of high electricity prices, in order to meet the power load demand of the integrated energy microgrid while supplying the excess power generated to the grid for higher economic efficiency, the knowledge rule for hydrogen-blended cogeneration is to set the full power output action of hydrogen-blended cogeneration according to the power output upper limit and ramp constraint of hydrogen-blended cogeneration; the knowledge rule is expressed as:

[0040]

[0041] in, is the rate of change of the output power of hydrogen-doped cogeneration, is the output power of hydrogen-doped cogeneration at the last moment, The upper limit of the output power of hydrogen-doped cogeneration, P CHP,ramping It is the upper limit of the ramp rate of the combined heat and power electric power.

[0042] Furthermore, the process of obtaining natural gas exergy input into the integrated energy microgrid is:

[0043] The gas power generated by the methane reactor is distributed to the corresponding units as fuel according to the ratio of gas power required by hydrogen-blended cogeneration and hydrogen-blended gas boilers;

[0044] The gas power required for the hydrogen-doped cogeneration and hydrogen-doped gas boiler scheduling process is subtracted from the gas power provided by the methane reactor to obtain the natural gas power provided by the natural gas grid to the hydrogen-doped cogeneration and hydrogen-doped gas boiler.

[0045] Based on the temperature of hydrogen-doped cogeneration and hydrogen-doped gas boiler combustion process, the natural gas exergy input to the integrated energy microgrid is calculated. Specifically, the natural gas exergy input to the integrated energy microgrid is Calculate according to the following formula:

[0046]

[0047]

[0048] in, 、 They are the natural gas exergy input to hydrogen-blended cogeneration and hydrogen-blended gas boilers, 、 are the gas power input to hydrogen-blended cogeneration and hydrogen-blended gas boilers, 、 is the temperature of hydrogen-blended cogeneration and hydrogen-blended gas boiler combustion process, is the natural gas power generated by the methane reactor, and T1 is the ambient temperature.

[0049] Furthermore, the hydrogen energy exergy of the photocatalytic hydrogen production input into the integrated energy microgrid is the sum of the exergy of the photocatalytic hydrogen production that flows into the hydrogen-doped cogeneration and hydrogen-doped gas boiler, and the exergy of the photocatalytic hydrogen production that flows into the hydrogen fuel cell and the methane reactor. Specifically, the hydrogen energy exergy of the photocatalytic hydrogen production input into the integrated energy microgrid is for:

[0050]

[0051] in, 、 The hydrogen produced by photocatalysis can flow into the exergy of hydrogen-blended cogeneration and hydrogen-blended gas boilers. 、 Exergy that allows hydrogen produced by photocatalysis to flow into hydrogen fuel cells and methane reactors;

[0052]

[0053] in, 、 The hydrogen produced by photocatalysis can be input into the hydrogen-doped cogeneration and hydrogen-doped gas boiler; T1 represents the ambient temperature, in K; 、 It is the temperature of the combustion process of hydrogen-blended cogeneration and hydrogen-blended gas boiler.

[0054] Furthermore, the hydrogen produced by photocatalysis can be input into the hydrogen power of hydrogen-doped cogeneration and hydrogen-doped gas boilers. 、 , obtained according to the following formula;

[0055]

[0056] in, is the hydrogen production allocation ratio coefficient at time t; is the hydrogen storage tank output allocation ratio coefficient at time t; is the hydrogen storage tank output allocation ratio coefficient at t-1; is the hydrogen production allocation ratio coefficient at t-1; It is a 0-1 variable, which is 1 when the hydrogen storage tank is storing hydrogen and 0 when it is releasing hydrogen; 、 are the hydrogen charging / discharging efficiency, respectively; is the storage capacity of the hydrogen storage tank during the scheduling period T, where T is 24; is the initial output allocation ratio coefficient of the hydrogen storage tank; is the initial capacity of the hydrogen storage tank; is the hydrogen power input to device j for photocatalytic hydrogen production; is the hydrogen power of input device j; j takes values of [1, 2, 3, 4], representing hydrogen-blended cogeneration, hydrogen-blended gas boiler, methane reactor, and hydrogen fuel cell, respectively; P EL,t P is the hydrogen power output by the alkaline electrolyzer at time t; PHP,t is the hydrogen power output by photocatalytic hydrogen production at time t; is the hydrogen storage / discharge power at time t; It is the hydrogen storage / desorption power at time t-1. When it is greater than 0, it is hydrogen desorption, otherwise it is hydrogen storage.

[0057] Furthermore, the index values of exergy efficiency, economy, and low carbon are transformed using the Sigmoid function so that their values are mapped to the same dimension:

[0058]

[0059] Among them, x k is the kth index that needs to be transformed, is the value of the kth index after transformation; k takes the values of 1, 2, and 3, representing the exergy efficiency index, economic index, and low-carbon index, respectively; θ k is the weight parameter of the kth indicator;

[0060] After the above transformation, I t ={ , , }, 、 and are the exergy efficiency, system operating cost and actual carbon emissions of the system after Sigmoid transformation.

[0061] A priority experience replay mechanism can also be introduced in deep reinforcement learning.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] The present invention fills the gap in the prior art in the exergy efficiency calculation of hydrogen-containing integrated energy systems, constructs a "black box" exergy efficiency analysis model for hydrogen-containing integrated energy microgrids, and expands the exergy efficiency calculation scenarios for hydrogen-containing integrated energy microgrids. For systems with multiple hydrogen production sources, by setting different energy flow allocation coefficients, such as the hydrogen production allocation coefficient and the hydrogen storage tank output allocation coefficient, photocatalytic hydrogen production and alkaline electrolyzer hydrogen production are calculated separately, thereby realizing real-time "tracking" of the hydrogen energy flow direction of different hydrogen production units and realizing the calculation of the exergy of multiple hydrogen energy flows.

[0064] 2. This invention comprehensively considers multiple indicators such as exergy efficiency, economy, and low carbon, integrates these three optimization indicators, and constructs a multi-objective optimization objective function. This effectively balances the contradictions between energy efficiency, economy, and carbon emission reduction in system operation, ensuring efficient energy use of the system while maintaining economy and low carbon.

[0065] 3. This paper also proposes a deep reinforcement learning framework that integrates knowledge rules with an improved double-delayed deep deterministic policy gradient algorithm (TD3 algorithm + prioritized experience replay pool). This framework optimizes the knowledge rules to generate high-quality training samples at the beginning of training, guiding the agent to explore optimal solutions. Furthermore, it implements over-constraint truncation to ensure that the agent's output actions remain within the constraints, enabling the safe operation of integrated energy microgrids and ensuring the safety of deep reinforcement learning in practical application scenarios. It also addresses the overestimation problem in the update process of the double-delayed policy gradient algorithm (DDPG algorithm) and the deep quenching neural network algorithm (DQN algorithm). BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a schematic diagram of the structure of an electricity-gas-heat-hydrogen integrated energy microgrid according to an embodiment;

[0067] Figure 2 Schematic diagram of the "black box" exergy efficiency analysis model of the integrated energy microgrid in the present invention;

[0068] Figure 3 A schematic diagram of a network architecture for deep reinforcement learning according to an embodiment of the present invention;

[0069] Figure 4 This is a reward curve diagram of the training process of Example 3 of the present invention;

[0070] Figure 5 This is a graph of electric power scheduling results output by the scheduling method in Example 3 of the present invention;

[0071] Figure 6 This is a diagram of the thermal power scheduling results output by the scheduling method in Example 3 of the present invention;

[0072] Figure 7 This is a hydrogen power scheduling result diagram output by the scheduling method in Example 3 of the present invention;

[0073] Figure 8 This is a graph showing the carbon emissions and exergy efficiency scheduling results output by the scheduling method in Example 3 of the present invention. DETAILED DESCRIPTION

[0074] The present invention will be further explained below with reference to the accompanying drawings and embodiments, but they are not intended to limit the scope of protection of the present application.

[0075] Example 1:

[0076] This embodiment uses a multi-objective optimization scheduling method for an integrated energy microgrid based on exergy efficiency. The integrated energy microgrid contains multiple hydrogen production sources, including photocatalytic hydrogen production, alkaline electrolyzers, new energy generators, hydrogen-blended gas boilers, hydrogen-blended cogeneration, hydrogen fuel cells, carbon capture, energy storage equipment, and methane reactors. The energy storage equipment includes electrical energy storage, thermal energy storage, and hydrogen storage tanks (hydrogen energy storage), and the new energy generators include photovoltaic generators and wind turbines. Specifically, the following steps are involved:

[0077] The natural gas power generated by the methane reactor is proportionally allocated to calculate the natural gas power input to the hydrogen-blended cogeneration and hydrogen-blended gas boilers, and then the natural gas exergy input to the integrated energy microgrid is calculated;

[0078] The hydrogen production apportionment coefficient is set according to the ratio of the hydrogen power output by photocatalytic hydrogen production to the total hydrogen power output by the alkaline electrolyzer hydrogen production and photocatalytic hydrogen production;

[0079] The hydrogen storage tank output allocation ratio coefficient is set according to the ratio of hydrogen energy stored in the hydrogen storage tank from photocatalytic hydrogen production and alkaline electrolysis cell hydrogen production;

[0080] Then, the hydrogen production apportionment ratio coefficient and the hydrogen storage tank output apportionment ratio coefficient are used to calculate the hydrogen power allocated to each hydrogen-consuming device by photocatalytic hydrogen production, and then the hydrogen energy exergy input into the integrated energy microgrid by photocatalytic hydrogen production is calculated;

[0081] The overall exergy efficiency of the integrated energy microgrid is calculated by combining the exergy of the electrical load and the thermal load, the exergy of the integrated energy microgrid selling electricity to the main grid, the exergy of the integrated energy microgrid purchasing electricity, the exergy of the photovoltaic generator set, and the exergy of the wind turbine set.

[0082] With the optimization goal of improving the overall exergy efficiency, economy and low carbon of the integrated energy microgrid, deep reinforcement learning is used to achieve real-time scheduling of the integrated energy microgrid.

[0083] The overall exergy efficiency of the integrated energy microgrid at time t is:

[0084]

[0085] in, Exergy efficiency; 、 are the exergy of electrical load and thermal load at time t respectively; Exergy that sells electricity to the main grid for integrated energy microgrids; Purchase electricity for integrated energy microgrids; 、 They are the power generation exergy of photovoltaic generator sets and the power generation exergy of wind turbine generator sets respectively; The hydrogen energy exergy that is input into the integrated energy microgrid for photocatalytic hydrogen production, This is the natural gas exergy input into the integrated energy microgrid.

[0086] Example 2:

[0087] The scheduling method of this embodiment further includes a knowledge rule module, which includes a condition determination process and a knowledge rule replacement process. The condition determination process refers to identifying the current state of the integrated energy microgrid and determining whether the current state is within the application scope of the knowledge rule.

[0088] The knowledge rule replacement process means that if the current state is within the application range of the knowledge rule, the action of the corresponding device is generated, and the corresponding action in the action space is randomly replaced according to the set action output probability.

[0089] The action output probability for:

[0090]

[0091] in, is the action selection control probability reduction factor; z is the number of rounds of interaction between the double-delay deep deterministic policy gradient algorithm and the integrated energy microgrid.

[0092] The knowledge rules are the knowledge rule requirements for the operation of corresponding equipment set based on the system optimization objectives and the operating status of the integrated energy microgrid; including knowledge rules for electric energy storage, electric boilers, alkaline electrolyzers and hydrogen-doped cogeneration.

[0093] Based on the independent exploration of the double-delay deep deterministic policy gradient algorithm, the knowledge rule module is introduced into the double-delay deep deterministic policy gradient algorithm, and the double-delay deep deterministic policy gradient algorithm is executed with the actions processed by the knowledge rule module.

[0094] Based on the optimal solution distance method, the three indicators of exergy efficiency, economy and low carbon are integrated to construct the objective function of multi-objective optimization; the economy indicator is the operating cost of the integrated energy microgrid, and the low carbon indicator is the actual carbon emissions E of the integrated energy microgrid. q ; Set the objective function Target(I t )for:

[0095]

[0096] Among them, I t is the high-dimensional point in the target value space combining exergy efficiency, economy and low carbon at time t, I best is the theoretical optimal solution of the scheduling scheme, I best ={1, 0, 0}; “|| ||2” represents the two-norm.

[0097] The historical data described in this invention includes electricity and heat load demand, power generation from renewable energy generators, time-of-use electricity prices, and ambient temperature. This historical data is used to train a deep reinforcement learning network architecture (i.e., an intelligent agent). The real-time state of the integrated energy microgrid is then input into the trained deep reinforcement learning network architecture, enabling real-time scheduling of the integrated energy microgrid under uncertain conditions.

[0098] Example 3:

[0099] In this embodiment, the electricity-gas-heat-hydrogen integrated energy microgrid, such as Figure 1 As shown, the integrated energy microgrid includes photocatalytic hydrogen production, an alkaline electrolyzer, a dynamic hydrogen doping unit (hydrogen doping gas boiler, hydrogen doping cogeneration), a methane reactor, a hydrogen fuel cell, carbon capture, an electric boiler, energy storage equipment (electrical energy storage, thermal energy storage, hydrogen storage tanks), and a new energy generator set. The hydrogen storage tank provides hydrogen energy storage, while the photocatalytic hydrogen production, alkaline electrolyzer, methane reactor, dynamic hydrogen doping unit (hydrogen doping gas boiler, hydrogen doping cogeneration), hydrogen fuel cell, and hydrogen storage tanks form a diversified hydrogen energy utilization structure. The hydrogen demand of the integrated energy microgrid is met by both water electrolysis in the alkaline electrolyzer and photocatalytic hydrogen production. Electricity demand is provided by the new energy generator set, hydrogen doping cogeneration, hydrogen fuel cell, electric energy storage, and the upstream power grid. Heat demand is met by the dynamic hydrogen doping unit, electric boiler, and thermal energy storage.

[0100] The process of the multi-objective optimization scheduling method for integrated energy microgrid based on exergy efficiency is:

[0101] S1, calculation of exergy efficiency;

[0102] The exergy efficiency analysis model of the integrated energy microgrid coupled with electricity, gas, heat and hydrogen is constructed as follows:

[0103] The exergy efficiency analysis of the integrated energy microgrid first treats the integrated energy microgrid equivalently from the perspective of external characteristics, and constructs an exergy flow "black box" model that can transform the complex energy flow and coupling relationship within the system into an input and output relationship. The integrated energy microgrid "black box" exergy efficiency analysis model is as follows: Figure 2 As shown, the supply side inputs energy E to the integrated energy microgridx,in , using this as payment exergy, it outputs energy E to the demand side through a series of processes such as transmission, conversion, storage and distribution within the integrated energy microgrid. x,out , and use this as the revenue exergy to meet the needs of all forms of energy in the integrated energy microgrid. According to the exergy reduction principle of the second law of thermodynamics, during the energy conversion process within the integrated energy microgrid, it is inevitable that a part of the energy will disappear in the natural environment, which is the exergy loss E. x,loss .

[0104] Exergy loss will lead to a decrease in exergy efficiency. The mathematical model is:

[0105] (1)

[0106] In an integrated energy microgrid, the exergy input to the "black box" model consists of the exergy of electricity purchased from the upstream power grid, the exergy generated by renewable energy generators (wind power, photovoltaic power), the exergy of hydrogen energy fed into the system through photocatalytic hydrogen production, and the exergy purchased from natural gas. The revenue exergy output from the "black box" model consists of the exergy of user-side electrical loads, thermal loads, and the exergy of electricity sold by the integrated energy microgrid. By introducing the allocation coefficients for different energy flows and applying the second law of thermodynamics, the exergy calculation for energy flows involving multiple hydrogen production sources is achieved.

[0107] The overall exergy efficiency of the integrated energy microgrid at time t can be expressed as:

[0108] (2)

[0109] in, is the exergy efficiency at time t; 、 are the exergy of electrical load and thermal load at time t respectively; Exergy that sells electricity to the main grid for integrated energy microgrids; Purchase electricity for integrated energy microgrids; 、 They are the power generation exergy of photovoltaic generator sets and the power generation exergy of wind turbine generator sets respectively; The hydrogen energy exergy that is input into the integrated energy microgrid for photocatalytic hydrogen production, This is the natural gas exergy input into the integrated energy microgrid.

[0110] According to the first and second laws of thermodynamics, electrical energy can be completely converted into other forms of energy due to its high conversion efficiency. Therefore, it is regarded as pure exergy in exergy analysis, and its energy quality coefficient is defined as 1. Therefore, the exergy of the integrated energy microgrid selling electricity to the main grid is and the exergy of the electrical load at time t The calculation formula is as follows:

[0111] (3)

[0112] in, is the energy quality coefficient of electric energy; It is a 0-1 variable, which is 1 when the integrated energy microgrid sells electricity to the grid and 0 when it purchases electricity; P grid,t The amount of electricity exchanged between the integrated energy microgrid and the power grid. A value greater than 0 indicates electricity purchase, and a value less than 0 indicates electricity sale. is the electric load in the integrated energy microgrid.

[0113] Exergy of heat load It can be expressed as:

[0114] (4)

[0115] in, is the energy quality coefficient of thermal energy; T2 is the heat source temperature, unit is K; T1 is the ambient temperature, unit is K; is the heat load of the integrated energy microgrid.

[0116] Natural gas exergy input into the integrated energy microgrid Theoretically, it can be directly solved from the chemical reaction formula, but the practical application is limited by the heat resistance level of existing equipment, and the theoretical combustion temperature of the fuel is often unable to be reached. Therefore, the natural gas exergy is equivalent to the heat exergy generated at its combustion temperature during the combustion process. The natural gas purchased from the natural gas grid and the natural gas produced by the methane reactor are used as raw materials to flow into the hydrogen-blended cogeneration and hydrogen-blended gas boiler for production capacity. According to the exergy efficiency "black box" model, the natural gas produced by the methane reactor is converted from the rest of the energy flow in the system, rather than input exergy. Therefore, the natural gas exergy generated by the methane reaction should be subtracted from the gas power input to the gas equipment to calculate the input natural gas exergy. According to the "proportional allocation" principle, the gas power generated by the methane reactor is proportionally distributed to the corresponding units as fuel according to the gas power input to the hydrogen-blended cogeneration and the gas power input to the hydrogen-blended gas boiler. Its specific input to the integrated energy microgrid is the natural gas exergy. It can be expressed as:

[0117] (5)

[0118] (6)

[0119] in, 、 The natural gas exergy input to hydrogen-blended cogeneration and hydrogen-blended gas boilers, 、 are the gas power input to hydrogen-blended cogeneration and hydrogen-blended gas boilers, 、 is the temperature of hydrogen-blended cogeneration and hydrogen-blended gas boiler combustion process, The natural gas power generated by the methane reactor.

[0120] 、 They represent the natural gas power provided by the natural gas grid to hydrogen-blended cogeneration and hydrogen-blended gas boilers respectively.

[0121] The hydrogen power generated by photocatalytic hydrogen production flows in multiple directions, including to hydrogen storage tanks, hydrogen fuel cells, and dynamic hydrogen blending units (hydrogen blending gas boilers, hydrogen blending cogeneration). To accurately calculate the paid exergy of the input hydrogen energy and avoid confusion with the hydrogen produced by the alkaline electrolyzer inside the "black box", the following measures are adopted:

[0122] The sources of hydrogen energy flowing into the target production capacity equipment include alkaline electrolysis cell hydrogen production, photocatalytic hydrogen production and hydrogen storage tanks, as shown in the following formula:

[0123] (7)

[0124] Among them, P EL,t is the hydrogen power output of the alkaline electrolyzer; P PHP,t The hydrogen power output for photocatalytic hydrogen production; is the hydrogen storage / release power at time t, when it is greater than 0, it is hydrogen release, otherwise it is hydrogen storage; is the hydrogen power input to the methane reactor at time t; The hydrogen power input to the hydrogen fuel cell at time t; To input hydrogen power for hydrogen-blended cogeneration; It is the hydrogen power input to the hydrogen-blended gas boiler.

[0125] In order to achieve real-time "tracking" of the flow of hydrogen produced by photocatalysis, the hydrogen production allocation ratio coefficient is set according to the "proportional allocation" principle. and hydrogen storage tank output sharing ratio coefficient :

[0126] (8)

[0127] in, is the hydrogen production allocation ratio coefficient at time t; is the hydrogen storage tank output allocation ratio coefficient at time t; is the hydrogen storage tank output allocation ratio coefficient at t-1; is the hydrogen production allocation ratio coefficient at t-1; It is a 0-1 variable, which is 1 when the hydrogen storage tank is storing hydrogen and 0 when it is releasing hydrogen; 、 are the hydrogen charging / discharging efficiency, respectively; is the storage capacity of the hydrogen storage tank during the scheduling period T, where T is 24; is the initial output allocation ratio coefficient of the hydrogen storage tank; is the initial capacity of the hydrogen storage tank; is the hydrogen power input to device j for photocatalytic hydrogen production; is the hydrogen power of input device j; j is [1, 2, 3, 4], representing hydrogen-blended cogeneration, hydrogen-blended gas boiler, methane reactor, and hydrogen fuel cell, respectively; t-1 is the previous time period at time t; P EL,t P is the hydrogen power output by the alkaline electrolyzer at time t; PHP,t is the hydrogen power output by photocatalytic hydrogen production at time t; is the hydrogen storage / discharge power at time t, It is the hydrogen storage / desorption power at time t-1. When it is greater than 0, it is hydrogen desorption, otherwise it is hydrogen storage.

[0128] According to the hydrogen storage / release action, the payable exergy of hydrogen produced by photocatalysis is:

[0129] (9)

[0130] in, 、 The hydrogen produced by photocatalytic hydrogen production can flow into the exergy of hydrogen-blended cogeneration and hydrogen-blended gas boilers. 、 The hydrogen produced by photocatalytic hydrogen production can be input into the hydrogen power of hydrogen-doped cogeneration and hydrogen-doped gas boilers.

[0131] Since the hydrogen energy input to the hydrogen fuel cell and the methane reactor is different from the energy produced through the combustion process, the hydrogen fuel cell directly converts chemical energy into electrical energy through an electrochemical reaction without the intermediate changes of thermal energy and mechanical energy. The methane reactor converts hydrogen energy into methane. The exergy calculation model is:

[0132] (10)

[0133] (11)

[0134] in, 、 The exergy produced by photocatalytic hydrogen production can flow into hydrogen fuel cells and methane reactors. 、 The hydrogen produced by photocatalytic hydrogen production can be input into the hydrogen power of hydrogen fuel cells and methane reactors. are the energy quality coefficients of hydrogen energy input into hydrogen fuel cells and methane reactors, E HFC The maximum electrical power that can be generated by the chemical reaction process of a hydrogen fuel cell; G is the change in Gibbs free energy, is the Gibbs free energy of hydrogen, is the Gibbs free energy of oxygen, is the Gibbs free energy of water; H is the enthalpy change of the substance, which represents the maximum energy that hydrogen can release; is the enthalpy of hydrogen, is the enthalpy of oxygen, is the enthalpy of water. The derivation form of the quality factor of the methane reactor is similar to (11), so it will not be repeated here.

[0135] The hydrogen energy exergy input into the integrated energy microgrid by photocatalytic hydrogen production is equal to the sum of the above-mentioned exergy, which can be specifically expressed as:

[0136] (12)

[0137] S2, based on the optimal solution distance method, integrates the three indicators of exergy efficiency, economy, and low carbon, and constructs the objective function of multi-objective optimization;

[0138] The economic index is the operating cost of the integrated energy microgrid, and the low-carbon index is the actual carbon emissions of the integrated energy microgrid E q ;

[0139] The exergy efficiency index represents the energy efficiency of the integrated energy microgrid operation and is determined by formula (2) in step S1.

[0140] The mathematical model of economic indicators is expressed as follows:

[0141] (13)

[0142] Among them, F total,t is the operating cost of the integrated energy microgrid, C YS,t 、C MG,t 、C NG,t They are CO2 storage and transportation costs, electricity trading fees, and natural gas trading fees.

[0143] The expression of CO2 storage and transportation cost is:

[0144] (14)

[0145] in, is the unit cost of CO2 storage and transportation, E CC,t Carbon emissions from the operation of gas-fired equipment (hydrogen-blended cogeneration and hydrogen-blended gas boilers), is the amount of CO2 required for the methane reactor reaction at time t, is the capture efficiency of carbon capture.

[0146] Electricity transaction fee C MG,t The expression is:

[0147] (15)

[0148] in, 、 are the unit prices of electricity purchased and sold from the power grid; P grid,t It is the amount of electricity that interacts between the integrated energy microgrid and the power grid.

[0149] Natural gas cost C NG,t The expression is:

[0150] (16)

[0151] in, is the natural gas price at time t, V CHP,t 、V GB,t The volume of natural gas required for the operation of hydrogen-doped cogeneration and hydrogen-doped gas boilers, V CH 4 ,t is the volume of methane produced by the methane reactor.

[0152] The mathematical model of low-carbon index is expressed as follows:

[0153]

[0154] Among them, E q,t is the actual carbon emissions of the integrated energy microgrid at time t, is the carbon emission coefficient per unit of electricity purchased; It is a 0-1 variable, which is 1 when purchasing electricity and 0 when selling electricity; M r Indicates the ratio of the relative molecular mass of CO2 to carbon, E CHP,t 、E GB,t is the carbon emissions of hydrogen-blended cogeneration and hydrogen-blended gas boilers at time t, 、 are the carbon content and carbon oxidation rate of natural gas per unit calorific value, is the thermoelectric conversion coefficient, is the input gas power of hydrogen-blended cogeneration, The gas power input to the hydrogen-blended gas boiler.

[0155] If the traditional TOPSIS method is used, it is necessary to calculate the optimal and worst evaluation values of the scheduling results, and to sort the performance of the solutions according to the distance between the optimal and worst solutions. However, the setting of the worst solution distance differentiates solutions with the same performance, further limiting the exploration scope of the agent and causing difficulties in training the agent. Therefore, this embodiment improves the traditional TOPSIS method and uses the optimal solution distance method to construct the objective function of multi-objective optimization:

[0156] (19)

[0157] Among them, I tis the high-dimensional point in the target value space combining exergy efficiency, economy and low carbon at time t, I best is the theoretical optimal solution of the scheduling scheme, || ||2 represents the two-norm;

[0158] At the same time, considering that the inconsistent indicator dimensions of the above three indicators will affect the calculation of spatial distance, the indicator values of exergy efficiency, economy, and low carbon are transformed using the Sigmoid function to map their values to the same dimension:

[0159] (20)

[0160] Among them, x k is the kth index that needs to be transformed, is the value of the kth index after transformation; k takes the values of 1, 2, and 3, representing the exergy efficiency index, economic index, and low-carbon index, respectively; θ k is the weight parameter of the kth indicator; after the above transformation, I t ={ , , }, 、 and are the exergy efficiency after Sigmoid transformation, the system operating cost and the actual carbon emissions of the system. During the system operation, the higher the exergy efficiency, the closer it is to a high-efficiency system, while the lower the operating cost and system carbon emissions, the more economical and low-carbon the system is. Therefore, the theoretical optimal solution of the scheduling scheme is set to I best ={1, 0, 0}.

[0161] S3: Based on the system optimization goals and system operation status, set the knowledge rules and requirements for the operation of each device in the integrated energy microgrid:

[0162] This embodiment mainly explores the optimal solution in the intelligent environment, supplemented by expert knowledge guidance, and establishes knowledge rules for the operation of various devices in the integrated energy microgrid during periods of high electricity prices. The main equipment involved in the knowledge rules are electric energy storage, electric boilers, alkaline electrolyzers, and hydrogen-doped cogeneration.

[0163] During periods of high electricity prices, energy storage is constrained by the upper and lower limits of energy storage capacity, and its knowledge rules are expressed as follows:

[0164] (twenty one)

[0165] in, is the regulating variable of the energy storage charging / discharging power; Clip is the truncation function, and the truncation range is [0,1]; The mean is μ and the variance is Gaussian distribution function; is the storage capacity of the electric energy storage at t-1; The upper limit of electric energy storage capacity; The upper limit of charge / discharge power; 、 The start and end times of the high electricity price period.

[0166] Using electric boilers during periods of high electricity prices not only reduces the system's operating economy, but also degrades the system's energy quality by converting high-quality electricity into heat. The knowledge rules for this are as follows:

[0167] (twenty two)

[0168] in, is the rate of change of electric power input to the electric boiler, is the electric power input to the electric boiler at the last moment, P EB,ramping It is the upper limit of the electric power ramp of the electric boiler.

[0169] The generation of electricity by alkaline electrolyzers during periods of high electricity prices will lead to an increase in the system's electricity consumption during peak periods, resulting in a decrease in the electricity sales of the integrated energy microgrid, thus affecting the economic efficiency of the system operation. The knowledge rules are expressed as follows:

[0170] (twenty three)

[0171] in, is the rate of change of electric power input to the alkaline electrolytic cell, P P2G,t-1 is the input power of the alkaline electrolytic cell at the previous moment; P P2G,ramping It is the upper limit of the electric power ramp of the alkaline electrolyzer.

[0172] The hydrogen-doped cogeneration system increases power generation during periods of high electricity prices. This not only provides electricity for peak load periods, but also allows excess electricity to be sold to the grid during periods of high electricity prices, improving the economic efficiency of the system. The knowledge rules are as follows:

[0173] (twenty four)

[0174] in, is the rate of change of the output power of hydrogen-doped cogeneration, is the output power of hydrogen-doped cogeneration at time t-1, The upper limit of the output power of hydrogen-doped cogeneration, P CHP,ramping It is the upper limit of the ramp rate of the combined heat and power electric power.

[0175] S4, constructs a Markov decision process in a random environment and determines the state space of the data-driven reinforcement learning agent, the action space under the over-limit truncation guarantee, and the reward function considering the optimization objectives and constraints:

[0176] The state of the integrated energy microgrid includes the storage capacity of the energy storage equipment at time t, the demand for electricity and heat loads, the output of the new energy generator set, the unit price of electricity purchased / sold by the grid, the power of photocatalytic hydrogen production, the ambient temperature, the action at the previous moment, and the scheduling moment. Since the carbon trading base price and the natural gas price are fixed values, they are not used as state quantities. The state space Expressed as:

[0177] (25)

[0178] Among them, S m,t is the storage capacity of the energy storage equipment at time t, including hydrogen storage tanks, electrical energy storage, and thermal energy storage; P new,t Power generation for new energy generators, including wind power and photovoltaic power generation; T 1,t is the ambient temperature at time t; a t-1 is the action at the previous moment; P PHP,t The hydrogen power output by the photocatalytic hydrogen production equipment; 、 are the unit prices of electricity purchased / sold from the power grid respectively; is the electrical load of the integrated energy microgrid; is the heat load of the integrated energy microgrid.

[0179] The action space is determined by the input power P of the alkaline electrolyzer P2G,t , output power of hydrogen-doped cogeneration , output thermal power of gas boiler , heat-to-electricity ratio of thermal hydrogen cogeneration , hydrogen blending ratio of dynamic hydrogen blending unit , hydrogen power input to the methane reactor , hydrogen power input to hydrogen fuel cells , charging / discharging power of electric energy storage , electric boiler output thermal power When the above variables are determined, the output of other parts can be obtained according to the corresponding mathematical model and equality constraints. The action space is defined as follows:

[0180] (26)

[0181] in, is the traditional state space; P P2G,t is the input electrical power of the alkaline electrolyzer; The output electric power of hydrogen-doped cogeneration; is the output thermal power of the gas boiler; is the heat-to-electricity ratio of hydrogen-doped cogeneration; The hydrogen blending ratio of the dynamic hydrogen blending unit, including the hydrogen blending ratio of hydrogen blended cogeneration and hydrogen blended gas boiler; is the charging / discharging power of the electric energy storage; Output thermal power for electric boiler.

[0182] In order to meet the climbing constraint of the device and realize the normalization of the action space, the above action space is decomposed as follows:

[0183] (27)

[0184] (28)

[0185] (29)

[0186] in, is the power change rate of device i; P i,ramping is the upper limit of the climbing of device i; P i,t is the operating power of device i at time t; P i,t-1 is the operating power of device i at time t-1; The upper limit of the charging / discharging power of the energy storage; 、 、 are the regulating variables of the electric energy storage charge / discharge power, thermoelectric ratio and hydrogen doping ratio respectively; The upper limit of the adjustable thermoelectric ratio; is the mapping coefficient of hydrogen doping ratio, which is set to 0.2; The decomposed action space is passed through the Tanh layer and then output, so that the action variable In the range [-1,1].

[0187] The above steps satisfy the equipment ramp constraints, but the upper and lower limits of operating power, thermal power ratio, and hydrogen blending ratio are not yet fully satisfied. By truncating the exceeding limit actions in the action space, the output action can strictly meet the upper and lower limit constraints:

[0188] (30)

[0189] in, is the action space under the protection of over-limit truncation, is the lower limit of hydrogen doping ratio, which is 0; It is the regulating variable of the charging / discharging power of the electric energy storage; is the power change rate of device i; P i,ramping is the upper limit of the climbing of device i; P i,t-1 is the operating power of device i at time t-1; and are the regulating variables of the thermoelectric ratio and hydrogen doping ratio respectively; is the adjustable upper limit of the thermoelectric ratio; It is the adjustable lower limit of the thermoelectric ratio; is the mapping coefficient of hydrogen doping ratio; P i,min is the minimum operating power of device i; P i,max is the maximum operating power of device i.

[0190] In this embodiment, the objective function and constraints of the integrated energy microgrid operation are transformed into a reward function with a penalty term. The reward r obtained by the agent at time t is t Expressed as:

[0191] (31)

[0192] Among them, F c (s t , a t ) is the constraint penalty function, F u (s t , a t ) is the power imbalance penalty function; are the cost scaling factor, constraint penalty scaling factor, and power imbalance penalty scaling factor, respectively, and , F total (s t , a t ) is the operating cost of the integrated energy microgrid; s t is the state space; a t It is the action space under the protection of over-limit truncation.

[0193] Constraint penalty function F c (s t , a t ) is calculated as:

[0194] (32)

[0195] (33)

[0196] Among them, M i,t M is the over-limit value of the operating power of device i; MG,t is the over-limit value of the power interaction with the upper-level power grid; The storage capacity exceeds the limit, including electricity, heat and hydrogen storage tanks; The hydrogen blending ratio exceeds the limit, including hydrogen blending cogeneration and hydrogen blending gas boilers; K CHP,t The heat-to-electricity ratio of cogeneration exceeds the limit; is the specific gravity coefficient, which is used to increase the penalty ratio of hydrogen blending ratio exceeding the limit; P i,t is the operating power of device i at time t; M MG,t 、 、 , K CHP,t The calculation process is similar to the calculation of the equipment operating power exceeding the limit value, so it will not be repeated here.

[0197] Power imbalance penalty function F u (s t , a t ) is expressed as:

[0198] (34)

[0199] Among them, P CCS,t Electricity consumed for carbon capture, The electricity consumed by the electric boiler, is the output power of the hydrogen fuel cell at time t, is the output thermal power of hydrogen-doped cogeneration at time t, Output thermal power for hydrogen-blended gas boiler, Output heat power for electric boiler, is the thermal energy storage charging / discharging power at time t.

[0200] S5, combines knowledge rules with the double-delayed deep deterministic policy gradient algorithm to obtain the network architecture of deep reinforcement learning for the integrated energy microgrid:

[0201] This embodiment sets knowledge rules for electric energy storage, electric boilers, alkaline electrolyzers and hydrogen-doped cogeneration, mainly based on the autonomous exploration of the double-delay deep deterministic policy gradient algorithm, and introduces a knowledge rule module. The knowledge rule module includes a condition judgment process and a knowledge rule replacement process. The condition judgment process identifies the current state of the intelligent agent and determines whether the state is within the application scope of the knowledge rule; the knowledge rule replacement process means that if the current state is within the application scope of the knowledge rule, the action of the corresponding device is generated, and the corresponding action in the action space is randomly replaced according to the set action probability, such as Figure 3 As shown. The action output probability of the knowledge rule for:

[0202] (35)

[0203] in, is the factor that reduces the probability of action selection control; z is the number of rounds of interaction between the algorithm and the integrated energy microgrid; the probability of action output of the knowledge rule decreases with the increase of the number of learning times, that is, in the early stage of training, the probability of action output of the knowledge rule is high, allowing the intelligent agent to quickly master the knowledge rule; as the number of learning times continues to increase, the probability of action output of the knowledge rule is gradually reduced, while the opportunity for autonomous exploration of the intelligent agent is increased, so that the intelligent agent can quickly and autonomously explore the optimization strategy beyond the knowledge rule.

[0204] The network architecture of deep reinforcement learning in this embodiment is as follows Figure 3 As shown in the figure, it includes a policy network, a value network 1, a value network 2, a knowledge rule module, and a priority experience replay pool. The policy network includes a main policy network and a target policy network. The value network 1 includes a main value network 1 and a target value network 1. The value network 2 includes a main value network 2 and a target value network 2. It is denoted as the improved TD3 algorithm. The specific construction and update process is as follows:

[0205] In order to reduce the problem of overestimation of Q value in the value network, the smaller Q value in the two target value networks is selected to construct the time difference target value y:

[0206] (36)

[0207] Among them, r is the reward value of training sample t, γ is the discount factor, 、 are the target Q values of target value network 1 and target value network 2 respectively, is the target strategy of the target strategy network, s t+1 is the state space at time t+1.

[0208] Adding a random Gaussian noise to the target policy network can alleviate the overfitting problem of the valuation function in the policy network:

[0209] (37)

[0210] Among them, ε represents Gaussian noise, c represents the cutoff boundary value of the strategy smoothing noise, and the clip function represents the truncation function. The mean is 0 and the variance is Gaussian distribution function.

[0211] Finally, the gradient descent algorithm is used to minimize the error between the estimated value and the target value. , thereby updating the parameters in the two value networks:

[0212] (38)

[0213] Where E( ) is the expected function, is the network parameter of the main value network u, is the learning rate of the principal value network u, Represents the gradient calculation function of the principal value network u parameter, is the action-value function of the principal value network u.

[0214] By using sampled policy gradient Main strategy network parameters Update according to formula (39):

[0215] (39)

[0216] (40)

[0217] in, is the main strategy network learning rate, is the gradient information of the value network, is the gradient information of the main policy network, E represents the expected function, The strategy of the main strategy network, Main policy network parameters.

[0218] The priority experience replay mechanism is introduced into the double-delay deep deterministic policy gradient algorithm to enable the agent to use important experience tuples more efficiently and improve training efficiency. In the priority experience replay mechanism, the temporal difference error δ n The size of is used to measure the priority of the sampled experience tuple n.

[0219] Priority is determined using a ranking-based priority approach: n =1 / rank n In the example, the rank of the nth sample is n The absolute timing difference error |δ n |δ n | means the following:

[0220] (41)

[0221] Among them, S n+1 is the state space of the n+1th sample, S n is the state space of the nth sample, a n is the action space of the nth sample.

[0222] During the training process, the agent is trained based on the priority p of the experience tuples in the buffer. n Calculate the sampling probability of each experience tuple, and then perform importance sampling on the experience tuple according to the sampling probability. Sampling probability P n It is expressed as follows:

[0223] (42)

[0224] in, Determines the number of priorities used. =0 indicates random sampling, and m is the total sample size.

[0225] However, since experiences with high temporal difference errors are replayed more frequently, this may cause the neural network to oscillate or diverge during training. To solve this problem, importance sampling weights can be used when calculating weight changes. :

[0226]

[0227] Among them, N b is the size of the priority experience buffer, and the parameter β determines the amount of correction used; For samples l Importance sampling weights; l The value range is 1 ~m An integer.

[0228] After introducing the priority experience replay mechanism, the loss function and sampling policy gradient of the policy network are reformulated as follows:

[0229] (44)

[0230] (45)

[0231] Using the following soft update method to update the target value network and target strategy network parameters can improve the stability of the learning process:

[0232] (46)

[0233] in, Indicates the soft update rate, is the main strategy network parameter, are the target strategy network parameters, is the main value network parameter, is the target value network parameter, u is 1 or 2.

[0234] S6 uses historical data such as electric and thermal load demand, power generation of new energy generators, time-of-use electricity prices, and ambient temperature to train the intelligent agent of step S5, and inputs the real-time status of the integrated energy microgrid into the trained intelligent agent to achieve real-time scheduling of the integrated energy microgrid under uncertain environments.

[0235] The system scheduling period in this embodiment is 24 hours, with a 1-hour interval. The electricity price for participating in power market transactions is time-of-use pricing, with peak electricity prices from 12:00 PM to 6:00 PM. The natural gas price is 4.0 yuan / m³, and the carbon trading base price is 140 yuan / ton. Training is performed on the AIStation artificial intelligence platform using the Pytorch deep learning framework for 5,000 training rounds. The initial learning rate for both the policy and value networks is set to 0.0001, the reward discount factor is 0.99, and the soft update rate for both the target value network and the target policy network is 0.001. The exploration noise is set to 0.25, and the policy noise is set to 0.15. Both the policy and value networks have four hidden layers, each with 64 neurons. The maximum capacity of the prioritized experience replay pool is 50,000, and the training batch size is 1,000.

[0236] The training process includes experience replay, sampling, calculating target values, calculating losses, updating value network parameters, calculating policy gradients, updating policy network parameters, and updating target value network and target policy network. Through these steps, the agent can continuously optimize the value network and policy network and learn the optimal policy. The training results are as follows: Figure 4 As shown in the figure (the reward value converges to close to 0 and the training reaches the optimal state). After the training is completed, the real-time state of the integrated energy microgrid is input and the coordinated control action is output to achieve real-time scheduling of the integrated energy microgrid.

[0237] The winter day environment state is input into the trained deep reinforcement learning network architecture, and the system operation results are as follows Figure 5 As shown. Figure 5 It can be seen that during peak electricity price periods, electricity demand is shared by new energy generators, hydrogen-blended cogeneration and hydrogen fuel cells. Taking into account the exergy efficiency and economic indicators, the system sells excess electricity to the power grid, maintaining energy "quality" while achieving higher economic benefits. Figure 6 It shows that during peak electricity price period, hydrogen-doped cogeneration and hydrogen-doped gas boilers are the main heating equipment, while during valley electricity price period, the heat load demand is mainly met by the coordinated work of hydrogen-doped cogeneration, hydrogen-doped gas boilers and electric boilers. Figure 7 It can be seen that the power of photocatalytic hydrogen production is the largest during the peak period of photovoltaic power generation, and the electrolyzer does not produce hydrogen during the peak period of electricity price. Figure 8 The results show that exergy efficiency remains lower during peak heat usage (00:00-6:00) compared to other periods, due to the partial conversion of high-quality electrical energy into heat. However, the exergy efficiency remains above 0.5, with a 24-hour average exergy efficiency of 0.701, a relatively high level. The total daily operating cost is 53,800 yuan, and the daily carbon emissions are 20.38 tons. This analysis shows that the method of the present invention can achieve the coordinated operation of the integrated energy microgrid with economic benefits, low carbon emissions, and high energy efficiency.

[0238] To verify the superiority of the improved TD3 algorithm in the scheduling of integrated energy microgrids, we compared its scheduling results with those of integrated energy microgrids based on the Deep Q Network (DQN) algorithm and the Deep Deterministic Policy Gradient (DDPG) algorithm. Fifteen days of data were randomly selected from March as a test sample to evaluate the scheduling results. The scheduling results are the average total cost, carbon emissions, and exergy efficiency of the 15-day test data, as shown in Table 1. As shown in Table 1, the integrated energy microgrid scheduling strategy output by the improved TD3 algorithm is significantly superior to that of the DQN and DDPG algorithms.

[0239]

[0240] Existing technologies focus solely on the "quantity" of energy, while ignoring the differences in "quality" between different energies. Their optimized scheduling results do not consider the energy efficiency of system operation, and therefore have certain limitations. This invention models exergy efficiency and uses it in multi-objective optimization, using exergy efficiency as an energy efficiency indicator, fully accounting for the differences in energy quality between different energies. This invention uses the optimal solution distance method to construct a multi-objective optimization objective function that incorporates exergy efficiency, economy, and low carbon performance. This balances the contradictions between carbon emission reduction, economy, and energy-efficient operation, allowing the optimized scheduling solution to take into account economy, low carbon performance, and high-quality energy use.

[0241] The existing technology does not provide clear theoretical guidance for calculating system exergy efficiency under multiple hydrogen production sources. The present invention expands the exergy efficiency calculation scenario of hydrogen-containing integrated energy microgrids. By constructing a sharing ratio coefficient, it achieves real-time "tracking" of the hydrogen energy flow direction of different hydrogen production units and realizes the calculation of exergy under multiple hydrogen energy flows. In addition, the present invention uses data-driven deep reinforcement learning to achieve multi-objective optimization and scheduling of integrated energy microgrids. It can realize real-time calculation of exergy under different environmental changes and dynamic formulation of scheduling plans according to changes in the external environment, effectively dealing with multiple uncertainties in the system.

[0242] The present invention introduces knowledge rules into the TD3 algorithm, dynamically optimizes the knowledge rules in the early stage of training to generate excellent training samples, accelerates the convergence speed of the intelligent agent's training process, guides the intelligent agent to "jump out" of the local optimal solution, and improves the convergence performance of the algorithm.

[0243] Finally, it should be noted that the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art will recognize that the technical solutions of the present invention may be modified or replaced with equivalents without departing from the basic concepts and essential features of the present invention. Such modifications or equivalents are deemed to be within the scope of protection defined by the claims of the present invention.

[0244] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A method based on The multi-objective optimization scheduling method of integrated energy microgrid with high efficiency is characterized by: The integrated energy microgrid contains multiple hydrogen production sources, including photocatalytic hydrogen production and alkaline electrolysis cells. The scheduling method Includes the following: The natural gas power generated by the methane reactor is proportionally allocated to calculate the natural gas power input to the hydrogen-blended cogeneration and hydrogen-blended gas boilers, and then the natural gas input to the integrated energy microgrid is calculated. ; The hydrogen production apportionment coefficient is set according to the ratio of the hydrogen power output by photocatalytic hydrogen production to the total hydrogen power output by the alkaline electrolyzer hydrogen production and photocatalytic hydrogen production; The hydrogen storage tank output allocation ratio coefficient is set according to the ratio of hydrogen energy stored in the hydrogen storage tank from photocatalytic hydrogen production and alkaline electrolysis cell hydrogen production; Then, the hydrogen production apportionment ratio coefficient and the hydrogen storage tank output apportionment ratio coefficient are used to calculate the hydrogen power allocated to each hydrogen-using device by photocatalytic hydrogen production, and then the hydrogen energy input of photocatalytic hydrogen production into the integrated energy microgrid is calculated. ; Combined with the electrical load and heat load , integrated energy microgrid sells electricity to the main grid , integrated energy microgrid electricity purchase , power generation from photovoltaic generators and wind turbines , calculate the overall energy microgrid efficiency; Heat load Expressed as: Among them, λ h is the energy quality coefficient of thermal energy; T2 is the heat source temperature, unit is K; T1 is the ambient temperature, unit is K; is the heat load of the integrated energy microgrid; Natural gas fed into the integrated energy microgrid Expressed as: in, The natural gas input to hydrogen-blended cogeneration and hydrogen-blended gas boilers , are the gas power input to hydrogen-blended cogeneration and hydrogen-blended gas boilers, is the temperature of hydrogen-blended cogeneration and hydrogen-blended gas boiler combustion process, natural gas power generated for the methane reactor; Hydrogen production allocation ratio coefficient and hydrogen storage tank output sharing ratio coefficient for: in, is the hydrogen production allocation ratio coefficient at time t; is the hydrogen storage tank output allocation ratio coefficient at time t; is the hydrogen storage tank output allocation ratio coefficient at t-1; is the hydrogen production allocation ratio coefficient at t-1; μ HyS It is a 0-1 variable, which is 1 when the hydrogen storage tank is storing hydrogen and 0 when it is releasing hydrogen; are the hydrogen charging / discharging efficiency; S HyS,t＝T is the storage capacity of the hydrogen storage tank during the scheduling period T, where T is 24; is the initial output allocation ratio coefficient of the hydrogen storage tank; S HyS,ini is the initial capacity of the hydrogen storage tank; is the hydrogen power input to device j for photocatalytic hydrogen production; is the hydrogen power of input device j; j is [1, 2, 3, 4], representing hydrogen-blended cogeneration, hydrogen-blended gas boiler, methane reactor, and hydrogen fuel cell, respectively; t-1 is the previous time period at time t; P EL,t P is the hydrogen power output by the alkaline electrolyzer at time t; PHP,t P is the hydrogen power output by photocatalytic hydrogen production at time t; t HyS is the hydrogen storage / discharge power at time t, is the hydrogen storage / release power at time t-1, when it is greater than 0, it is hydrogen release, otherwise it is hydrogen storage; Photocatalytic hydrogen production and hydrogen energy input into integrated energy microgrid Expressed as: in, The hydrogen produced by photocatalytic hydrogen production can flow into hydrogen fuel cells and methane reactors ; The hydrogen produced by photocatalytic hydrogen production can flow into hydrogen-blended cogeneration and hydrogen-blended gas boilers. , expressed as formula (9): In formula (9), The hydrogen produced by photocatalytic hydrogen production can be input into hydrogen-doped cogeneration and hydrogen-doped gas boilers; To improve the overall efficiency of integrated energy microgrid Using deep reinforcement learning to achieve real-time scheduling of integrated energy microgrids with efficiency, economy, and low carbon as optimization goals; The deep reinforcement learning adopts a double-delay deep deterministic policy gradient algorithm, and introduces a knowledge rule module into the double-delay deep deterministic policy gradient algorithm, and executes the double-delay deep deterministic policy gradient algorithm with actions processed by the knowledge rule module; The knowledge rule module includes a conditional judgment process and a knowledge rule replacement process. The conditional judgment process refers to identifying the current state of the integrated energy microgrid and judging whether the state is within the application scope of the knowledge rule. The knowledge rule replacement process is to generate the action of the corresponding device if the current state is within the application scope of the knowledge rule, and randomly replace the corresponding action in the action space according to the set action output probability; The action output probability ε1 is: Where β1 is the probability reduction factor of action selection control; z is the number of rounds of interaction between the double-delayed deep deterministic policy gradient algorithm and the integrated energy microgrid; The integrated energy microgrid scheduling process is expressed as a Markov decision process, including state space, action space and reward function; the action space is the action space a under the over-limit truncation guarantee t , defined as: in, is the lower limit of hydrogen doping ratio, which is 0; It is the regulating variable of the charging / discharging power of the electric energy storage; is the power change rate of device i; P i,ramping is the upper limit of the climbing of device i; P i,t-1 is the operating power of device i at time t-1; and are the regulating variables of the thermoelectric ratio and hydrogen doping ratio respectively; is the adjustable upper limit of the thermoelectric ratio; is the adjustable lower limit of the thermoelectric ratio; β re is the mapping coefficient of hydrogen doping ratio; P i,min is the minimum operating power of device i; P i,max is the maximum operating power of device i; By truncating the over-limit actions in the action space, the output actions can strictly meet the upper and lower limit constraints.

2. The scheduling method according to claim 1, characterized in that: The equipment used in the integrated energy microgrid includes photocatalytic hydrogen production, alkaline electrolyzers, new energy generator sets, hydrogen-blended gas boilers, hydrogen-blended cogeneration, hydrogen fuel cells, carbon capture, energy storage equipment and methane reactors; among them, the energy storage equipment includes electrical energy storage, thermal energy storage and hydrogen storage tanks, and the new energy generator sets include photovoltaic generator sets and wind turbine generator sets.

3. The scheduling method according to claim 1, characterized in that: Natural gas fed into the integrated energy microgrid The process of obtaining is: The gas power generated by the methane reactor is distributed to the corresponding units as fuel according to the ratio of gas power required by hydrogen-blended cogeneration and hydrogen-blended gas boilers; The gas power required for the hydrogen-doped cogeneration and hydrogen-doped gas boiler scheduling process is subtracted from the gas power provided by the methane reactor to obtain the natural gas power provided by the natural gas grid to the hydrogen-doped cogeneration and hydrogen-doped gas boiler. Based on the temperature of hydrogen-doped cogeneration and hydrogen-doped gas boiler combustion process, the natural gas input to the integrated energy microgrid is calculated. .

4. The scheduling method according to claim 1, characterized in that: During periods of high electricity prices, the knowledge rules for electric energy storage are to set the electric energy storage discharge action based on the upper and lower limit constraints of the electric energy storage capacity and the upper limit of the electric energy storage discharge power; the knowledge rules for electric boilers are to set the electric boiler shutdown action based on the electric boiler's climbing constraints; the knowledge rules for alkaline electrolyzers are to set the alkaline electrolyzer shutdown action based on the alkaline electrolyzer's climbing constraints; the knowledge rules for hydrogen-blended cogeneration are to set the full power output action of hydrogen-blended cogeneration based on the electric power output upper limit and climbing constraints of hydrogen-blended cogeneration.

Images