A multi-time scale scheduling control method and system for an electricity-hydrogen-heat coupling system

Abstract

This invention discloses a multi-timescale scheduling and control method and system for an electric-hydrogen-thermal coupled system. Existing scheduling and control methods for electric-hydrogen-thermal coupled systems do not consider the output characteristics of various types of energy units and the multivariate uncertainties and response characteristics of each device, which can introduce energy coupling risks into the system. The scheduling and control method of this invention includes the framework construction of the electric-hydrogen-thermal coupled system, the construction of a day-ahead stage operation optimization model, an intraday stage deviation optimization model, a real-time stage deviation optimization model, and deep reinforcement learning for solution. This invention achieves multi-timescale scheduling and control optimization of the electric-hydrogen-thermal coupled system considering the equipment adjustment sequence through the day-ahead stage operation optimization model, the intraday stage deviation optimization model, and the real-time stage deviation optimization model; this invention employs deep reinforcement learning to achieve multi-timescale solution.

Description

Technical Field

[0001] This invention relates to, specifically, a multi-timescale scheduling and control method and system for an electro-hydrogen-thermal coupled system. Background Technology

[0002] Under the "dual carbon" goal, building a new power system based on new energy sources and promoting the consumption of new energy and improving energy utilization efficiency have become key priorities. Producing hydrogen from curtailed wind and solar power not only promotes the consumption of new energy sources, but the produced hydrogen energy also has environmentally friendly characteristics. Therefore, coupling electricity and hydrogen energy to form an electro-hydrogen coupling system is an important way to achieve the "dual carbon" goal.

[0003] Existing methods have conducted fundamental research on the electric-hydrogen-thermal coupled system. However, due to the distributed power supply, the coupling of the hydrogen network and the thermal energy network, and the different output characteristics of various types of energy units, the scheduling response time of each part is different. This will bring the risk of energy coupling to the system. It is necessary to study the response characteristics and uncertainty characteristics of various subsystems in the electric-hydrogen-thermal coupled system, as well as the scheduling and control methods at different time scales. Summary of the Invention

[0004] To address the shortcomings of existing scheduling and control methods for electric-hydrogen-thermal coupled systems, which fail to consider the output characteristics of various energy units and the multivariate uncertainties and response characteristics of each device, this invention provides a multi-timescale scheduling and control method and system for electric-hydrogen-thermal coupled systems. This method utilizes a day-ahead stage operation optimization model, an intraday stage deviation optimization model, and a real-time stage deviation optimization model to achieve multi-timescale scheduling and control optimization of the electric-hydrogen-thermal coupled system, taking into account the order of device adjustments. Furthermore, it employs deep reinforcement learning to achieve multi-timescale solutions.

[0005] Therefore, the present invention adopts the following technical solution: a multi-timescale scheduling and control method for an electric-hydrogen-thermal coupled system, which includes the framework construction of the electric-hydrogen-thermal coupled system, the construction of a day-ahead stage operation optimization model, the construction of an intraday stage deviation optimization model, the construction of a real-time stage deviation optimization model, and deep reinforcement learning solution.

[0006] Furthermore, the aforementioned electric-hydrogen-thermal coupling system comprises a source end, a conversion and storage end, and a load end from energy production to consumption; the source end includes wind power generation and photovoltaic power generation; the conversion and storage end includes batteries, electric-to-thermal equipment, electrolyzers, hydrogen storage tanks, and hydrogen fuel cells; the load end includes three types of loads: electrical load, thermal load, and hydrogen load; within the coupling system, there are also an electrical energy system, a thermal energy system, and a hydrogen energy system; when the power supply of the electric-hydrogen-thermal coupling system is less than the demand, it interacts with the external power grid to purchase electricity; when the power supply exceeds the demand, it interacts with the external power grid to sell electricity.

[0007] Furthermore, the aforementioned day-ahead phase operation optimization model includes:

[0008] 1) Objective function

[0009] The objective function is to maximize the clean energy consumption rate and minimize the operation optimization cost.

[0010] 2) Constraints

[0011] The constraints of an electro-hydrogen-thermal coupled system include electrical balance constraints, hydrogen balance constraints, and thermal balance constraints.

[0012] In the day-ahead phase, existing technologies typically use the operating cost of the electro-hydrogen-thermal coupling system as the objective function. However, in addition to economic efficiency, other performance aspects of the electro-hydrogen-thermal coupling system should also be considered. This invention, based on existing technologies, constructs a day-ahead phase operation optimization model that minimizes operating costs and maximizes clean energy utilization.

[0013] Furthermore, the objective function for minimizing the operational optimization cost is shown in equation (1):

[0014]

[0015] In the formula: This represents the total cost of the coupled system during the day-ahead phase. These are the startup cost, operating cost, maintenance cost, first-type demand response compensation cost, and curtailment penalty cost of the coupled system, respectively; I is the set of equipment, I = {pv, wind, EC, ba, HST, HFC, E2H}, where pv, wind, EC, ba, HST, HFC, and E2H are photovoltaic power generation, wind power generation, electrolyzer, battery, hydrogen storage tank, hydrogen fuel cell, and electric-to-heat conversion equipment, respectively. This represents the startup state of unit i at time t; The startup cost of unit i; a 1i a 2i a 3i These are the coefficients of the primary, secondary, and constant terms of the output of unit i, respectively. The output of unit i at time t in the day-ahead phase; Let i be the installed capacity of unit i; Let be the maintenance factor for unit i; The unit call cost for the first type of demand response. Let t be the proportion of the first type of electrical load demand response. The reference electrical load demand for the first type of electrical load demand response at time t; These represent the amount of wind and solar power curtailed at time t during the daytime period; u pun Penalty costs per unit of wind and solar curtailment;

[0016] The objective function that maximizes the clean energy absorption rate is shown in equation (2):

[0017]

[0018] In the formula: These are the actual outputs of photovoltaic power generation and wind power generation, respectively. The average clean energy utilization rate of the distributed coupled system in the current stage;

[0019] The constraint conditions of the electro-hydrogen-thermal coupling system are shown in equation (3):

[0020]

[0021] In the formula: The photovoltaic power output at time t during the day-ahead phase; This refers to the wind power output at time t during the current period. The electricity generated by the hydrogen fuel cell at time t in the current phase; This represents the amount of discharge from the battery at time t during the current period. The number of calls made for the first type of demand response at time t before the current date; The amount of electricity consumed by the electrolytic cell at time t before the date; The amount of charge on the battery at time t represents the current period. The electricity consumption for heat conversion from electricity at time t before the date; The electrical load demand at time t before the date; The amount of hydrogen produced by the electrolyzer at time t before the date; This represents the amount of hydrogen consumed by the hydrogen fuel cell at time t before the current date. This represents the hydrogen load demand at time t before the current date; The installed capacity of the electrolytic cell at time t is the value of the capacity of the cell. The installed capacity of hydrogen fuel cells at time t is the capacity of the fuel cells at that time. The installed capacity of the electric-to-heat conversion equipment at time t before the date; This represents the heat load demand at time t before the current date; These represent the hydrogen release and filling volume of the hydrogen storage tank during the daytime phase, respectively.

[0022] Furthermore, the intraday stage deviation optimization model includes:

[0023] 1) Objective function

[0024] During the intraday phase, considering the uncertainties of wind power, photovoltaic power, and electrical load in the power system, the uncertainty of hydrogen load in the hydrogen energy system, and the uncertainty of heat load in the thermal energy system, the objective function is to minimize equipment adjustment costs and deviation penalty costs.

[0025] 2) Constraints

[0026] Power balance constraints during the intraday phase.

[0027] Furthermore, in the intraday stage deviation optimization model, based on the response characteristics of each device, the devices that can be adjusted during the intraday stage include batteries, electrolyzers, hydrogen fuel cells, and electrothermal conversion equipment; the objective function for the device adjustment cost is shown in equation (4):

[0028]

[0029] In the formula: The adjustment costs for the intraday phase of the coupled system; These are the adjustment costs for the battery, electrolyzer, hydrogen fuel cell, and electric-to-heat conversion equipment during the intraday phase of the coupling system, respectively. The cost of invoking a response to a second type of demand; These represent the battery's adjusted discharge and charge amounts at time t during the intraday period. These represent the output adjustments of the electrolyzer, hydrogen fuel cell, and electrothermal conversion equipment at time t during the intraday phase. These are the unit adjustment costs for batteries, electrolyzers, hydrogen fuel cells, and electric-to-heat conversion equipment during the intraday period, respectively. The unit call cost for responding to the second type of demand; The demand response ratio of the second type of electrical load at time t. The second type of electrical load demand response is the baseline electrical load demand at time t;

[0030] The objective function for minimizing the deviation penalty cost is shown in equation (5):

[0031]

[0032] In the formula: The cost of penalizing deviations during the intraday phase; These are the deviations for the electrical energy system, hydrogen energy system, and thermal energy system, respectively. These are the unit deviation penalty costs for electric energy systems, hydrogen energy systems, and thermal energy systems, respectively.

[0033] The power balance constraint condition for the intraday phase is shown in equation (6):

[0034]

[0035] In the formula: The photovoltaic power output at time t during the day; The wind power output at time t during the day; This represents the adjustment amount for the hydrogen fuel cell at time t during the intraday phase. This represents the battery discharge adjustment amount at time t during the intraday phase. This represents the second type of demand response call volume at time t within the day. This represents the battery charging adjustment amount at time t during the intraday phase. This represents the adjustment amount of the hydrogen storage tank at time t during the intraday phase. This represents the hydrogen load demand at time t during the intraday period. This refers to the adjustment amount of energy released from the hydrogen storage tank at time t during the intraday phase. This refers to the adjustment amount of the electric-to-heat conversion equipment at time t during the day. This represents the heat load demand at time t during the day. This represents the adjustment amount of hydrogen consumption of the hydrogen fuel cell at time t during the intraday phase. These represent the power consumption adjustment amounts for the electrolytic cell and the electric-to-heat conversion equipment at time t during the intraday phase. These represent the adjustments to the heat generation of the electrolyzer and hydrogen fuel cell at time t during the intraday period.

[0036] Furthermore, the real-time stage deviation optimization model includes:

[0037] 1) Objective function

[0038] The real-time phase also aims to minimize adjustment costs and deviation penalty costs. Considering the response characteristics of each device, deviations in the real-time phase are mainly satisfied by the third type of demand response, the battery, and interaction with the external network.

[0039] 2) Constraints

[0040] Power balance constraints during the real-time phase.

[0041] In both intraday and real-time phases, existing technologies for optimizing electro-hydrogen-thermal coupling systems do not consider the response characteristics and uncertainties of individual devices. However, due to the varying scheduling response times of different components, this introduces energy coupling risks to the system, necessitating deviation adjustments. Existing technologies, considering the overall electro-hydrogen-thermal coupling system, fail to account for the device adjustment sequence, resulting in suboptimal adjustment plans. This invention, building upon existing technologies, considers the device adjustment sequence to optimize the multi-timescale scheduling control of the electro-hydrogen-thermal coupling system.

[0042] Furthermore, in the aforementioned real-time stage deviation optimization model, the objective function that minimizes adjustment costs is shown in equation (7):

[0043]

[0044] In the formula: The adjustment cost for the real-time phase of the coupled system; For real-time stage storage batteries; The cost of responding to the third type of demand; These represent the battery's adjusted discharge and charge amounts at time t during the real-time phase. The unit call cost for responding to the third type of demand; The third type of electrical load demand response ratio at time t. The third type of electrical load demand response is the baseline electrical load demand at time t;

[0045] The objective function for minimizing the deviation penalty cost in the real-time stage deviation optimization model is shown in equation (8):

[0046]

[0047] In the formula: For real-time stage deviation costs; These are the deviations for the electrical energy system, hydrogen energy system, and thermal energy system, respectively.

[0048] The power balance constraint condition for the real-time stage is shown in equation (9):

[0049]

[0050] In the formula, The photovoltaic power output at time t in the real-time phase; The wind power output at time t is the real-time stage. The amount of electricity purchased from the power grid at time t in the real-time phase; The real-time electrical load demand at time t; This represents the amount of electricity sold to the grid at time t in the real-time phase. This represents the hydrogen load demand at time t in the real-time phase. This represents the heat load demand at time t in the real-time phase.

[0051] Furthermore, the deep reinforcement learning solution model is configured with a maximum number of training iterations N. max Construct the action value function D(l,s,w) of the electro-hydrogen-thermal coupled system, which represents the value of taking action s in state l; store the current state, action, and action value function in the experience pool w;

[0052] The loss function K(w) optimized by deep reinforcement learning is constructed using the Q-learning algorithm, as shown in Equation (10):

[0053]

[0054] In the formula: TarQ is the target output value, and PreQ is the predicted output value; t s t Represent the state and action at time t, respectively;

[0055] Perform actionst Afterwards, add rewards. The target output value is updated as shown in equation (11):

[0056]

[0057] In the formula: To predict the decay rate of the output value, a particle swarm optimization algorithm is used for optimization.

[0058] After updating the target output value, the state and the updated value for the next moment are also stored in the experience pool. At the same time, it is determined whether the maximum number of training iterations has been reached. If the maximum number of training iterations has been reached, the target output value at this time is the final output value; if the maximum number of training iterations has not been reached, training continues.

[0059] For solving problems at multiple time scales, existing technologies commonly use genetic algorithms. However, the electro-hydrogen-thermal coupled system involves multiple variables and dynamically changing environmental information at the day-ahead, intraday, and real-time stages. This invention, building upon existing technologies, employs deep reinforcement learning to achieve multi-time scale solutions.

[0060] Another technical solution adopted in this invention is: a multi-timescale scheduling and control system for an electric-hydrogen-thermal coupled system, which includes a framework building unit for the electric-hydrogen-thermal coupled system, a day-ahead stage operation optimization model building unit, an intraday stage deviation optimization model building unit, a real-time stage deviation optimization model building unit, and a deep reinforcement learning solution model.

[0061] The beneficial effects of this invention are as follows: This invention can track the output curve of distributed energy, improve the clean energy consumption rate of the electric-hydrogen-thermal coupling system, and reduce operating costs; This invention considers the response characteristics and uncertainties of each device, and can achieve optimal adjustment plan, thereby improving energy utilization efficiency; This invention uses deep reinforcement learning to solve the problem, which can improve the accuracy and speed of the solution. Attached Figure Description

[0062] Figure 1 This is a framework diagram of the electro-hydrogen-thermal coupling system of the present invention;

[0063] Figure 2 This is a diagram showing the day-ahead scheduling results of the electric-hydrogen-thermal coupling system in Embodiment 1 of the present invention.

[0064] Figure 3 This is a diagram showing the scheduling results of the electric-hydrogen-thermal coupling system during the daytime phase in Embodiment 1 of the present invention;

[0065] Figure 4 This is a diagram showing the real-time scheduling results of the electric-hydrogen-thermal coupling system in Embodiment 1 of the present invention.

[0066] Figure 5This is a flowchart of the multi-timescale scheduling and control method for the electric-hydrogen-thermal coupled system in Embodiment 1 of the present invention;

[0067] Figure 6 This is a flowchart of deep reinforcement learning in Embodiment 1 of the present invention. Detailed Implementation

[0068] The embodiments of the present invention will be given below with reference to the accompanying drawings, and the technical solutions of the present invention will be further elaborated and described in detail through the embodiments. The following embodiments are only some embodiments of the present invention.

[0069] Example 1

[0070] This embodiment presents a multi-timescale scheduling and control method for an electric-hydrogen-thermal coupled system, which includes the framework construction of the electric-hydrogen-thermal coupled system, the construction of the day-ahead stage operation optimization model, the construction of the intraday stage deviation optimization model, the construction of the real-time stage deviation optimization model, and deep reinforcement learning solution.

[0071] I. Framework Construction of Electro-Hydrogen-Thermocoupled System

[0072] The electro-hydrogen-thermal coupling system encompasses the energy production and consumption process, including the source, conversion and storage, and load ends. The source end includes wind and solar power generation; the conversion and storage end includes batteries (ba), electricity-to-heat (E2H) equipment, electrolytic cells (EC), hydrogen storage tanks (HST), and hydrogen fuel cells (HFC); and the load end includes three load types: electrical load, thermal load, and hydrogen load. Within the coupling system, there are electrical, thermal, and hydrogen energy systems. When the electricity supply of the electro-hydrogen-thermal coupling system is less than demand, it exchanges electricity with the external power grid to purchase electricity; when the electricity supply exceeds demand, it exchanges electricity with the external power grid to sell electricity. The framework of the electro-hydrogen-thermal coupling system is attached. Figure 1 As shown.

[0073] II. Construction of the Current Stage Operation Optimization Model

[0074] 21) Objective function

[0075] The objective function is to maximize the clean energy consumption rate and minimize the operation optimization cost.

[0076] The objective function for minimizing the operational optimization cost is shown in equation (1):

[0077]

[0078] In the formula: This represents the total cost of the coupled system during the day-ahead phase. These are the startup cost, operating cost, maintenance cost, first-type demand response compensation cost, and curtailment penalty cost of the coupled system, respectively; I is the set of equipment, I = {pv, wind, EC, ba, HST, HFC, E2H}, where pv, wind, EC, ba, HST, HFC, and E2H are photovoltaic power generation, wind power generation, electrolyzer, battery, hydrogen storage tank, hydrogen fuel cell, and electric-to-heat conversion equipment, respectively. This represents the startup state of unit i at time t; The startup cost of unit i; a 1i a 2i a 3i These are the coefficients of the primary, secondary, and constant terms of the output of unit i, respectively. The output of unit i at time t in the day-ahead phase; Let i be the installed capacity of unit i; Let be the maintenance factor for unit i; The unit call cost for the first type of demand response. Let t be the proportion of the first type of electrical load demand response. The reference electrical load demand for the first type of electrical load demand response at time t; These represent the amount of wind and solar power curtailed at time t during the daytime period; u pun The penalty cost is calculated per unit of wind and solar power curtailment.

[0079] The objective function that maximizes the clean energy absorption rate is shown in equation (2):

[0080]

[0081] In the formula: These are the actual outputs of photovoltaic power generation and wind power generation, respectively. The average clean energy utilization rate of the distributed coupled system in the current stage;

[0082] 22) Constraints

[0083] The constraints of an electro-hydrogen-thermal coupled system include electrical balance constraints, hydrogen balance constraints, and thermal balance constraints.

[0084] The constraint conditions of the electro-hydrogen-thermal coupling system are shown in equation (3):

[0085]

[0086] In the formula: The photovoltaic power output at time t during the day-ahead phase; This refers to the wind power output at time t during the current period. The electricity generated by the hydrogen fuel cell at time t in the current phase; This represents the amount of discharge from the battery at time t during the current period. The number of calls made for the first type of demand response at time t before the current date; The amount of electricity consumed by the electrolytic cell at time t before the date; The amount of charge on the battery at time t represents the current period. The electricity consumption for heat conversion from electricity at time t before the date; The electrical load demand at time t before the date; The amount of hydrogen produced by the electrolyzer at time t before the date; This represents the amount of hydrogen consumed by the hydrogen fuel cell at time t before the current date. This represents the hydrogen load demand at time t before the current date; The installed capacity of the electrolytic cell at time t is the value of the capacity of the cell. The installed capacity of hydrogen fuel cells at time t is the capacity of the fuel cells at that time. The installed capacity of the electric-to-heat conversion equipment at time t before the date; This represents the heat load demand at time t before the current date; These represent the hydrogen release and filling volume of the hydrogen storage tank during the daytime phase, respectively.

[0087] From the appendix Figure 5 It is known that during the day-ahead phase, scheduling is carried out 24 hours in advance, with a scheduling duration of 1 hour. Simultaneously, within the internal coupling system, based on the day-ahead wind and solar power forecasts and various load forecasts, the objective functions are to minimize operational optimization costs (as shown in Equation (1)) and maximize clean energy consumption (as shown in Equation (2)). This process calls upon the first type of demand response and determines the start-up, shutdown, and output plans for wind turbines, photovoltaic generators, batteries, electrothermal equipment, hydrogen fuel cells, and electrolyzers, thus forming the day-ahead reporting plan. Based on the reporting plan, clearing, verification, and day-ahead fee settlement are conducted in the external electricity market.

[0088] III. Construction of Intraday Stage Deviation Optimization Model

[0089] 31) Objective function

[0090] During the intraday phase, considering the uncertainties of wind power, photovoltaic power, and electrical load in the power system, the uncertainty of hydrogen load in the hydrogen energy system, and the uncertainty of heat load in the thermal energy system, the objective function is to minimize equipment adjustment costs and deviation penalty costs.

[0091] In the intraday stage deviation optimization model, based on the response characteristics of each device, the devices that can be adjusted during the intraday stage include batteries, electrolyzers, hydrogen fuel cells, and electrothermal conversion equipment; the objective function for device adjustment cost is shown in equation (4):

[0092]

[0093] In the formula: The adjustment costs for the intraday phase of the coupled system; These are the adjustment costs for the battery, electrolyzer, hydrogen fuel cell, and electric-to-heat conversion equipment during the intraday phase of the coupling system, respectively. The cost of invoking a response to a second type of demand; These represent the battery's adjusted discharge and charge amounts at time t during the intraday period. These represent the output adjustments of the electrolyzer, hydrogen fuel cell, and electrothermal conversion equipment at time t during the intraday phase. These are the unit adjustment costs for batteries, electrolyzers, hydrogen fuel cells, and electric-to-heat conversion equipment during the intraday period, respectively. The unit call cost for responding to the second type of demand; The demand response ratio of the second type of electrical load at time t. The second type of electrical load demand response baseline electrical load demand at time t.

[0094] The objective function for minimizing the deviation penalty cost is shown in equation (5):

[0095]

[0096] In the formula: The cost of penalizing deviations during the intraday phase; These are the deviations for the electrical energy system, hydrogen energy system, and thermal energy system, respectively. These are the unit deviation penalty costs for the electric energy system, hydrogen energy system, and thermal energy system, respectively.

[0097] 32) Constraints

[0098] Power balance constraints during the intraday phase.

[0099] The power balance constraint condition for the intraday phase is shown in equation (6):

[0100]

[0101] In the formula: The photovoltaic power output at time t during the day; The wind power output at time t during the day; This represents the adjustment amount for the hydrogen fuel cell at time t during the intraday phase. This represents the battery discharge adjustment amount at time t during the intraday phase. This represents the second type of demand response call volume at time t within the day. This represents the battery charging adjustment amount at time t during the intraday phase. This represents the adjustment amount of the hydrogen storage tank at time t during the intraday phase. This represents the hydrogen load demand at time t during the intraday period. This refers to the adjustment amount of energy released from the hydrogen storage tank at time t during the intraday phase. This refers to the adjustment amount of the electric-to-heat conversion equipment at time t during the day. This represents the heat load demand at time t during the day. This represents the adjustment amount of hydrogen consumption of the hydrogen fuel cell at time t during the intraday phase. These represent the power consumption adjustment amounts for the electrolytic cell and the electric-to-heat conversion equipment at time t during the intraday phase. These represent the adjustments to the heat generation of the electrolyzer and hydrogen fuel cell at time t during the intraday period.

[0102] Unlike existing technologies that do not consider the adjustment sequence of each device, this invention combines the characteristic analysis of each system to optimize deviations. (See attached...) Figure 5 It is known that during the intraday phase, scheduling is carried out 4 hours in advance, with a scheduling duration of 15 minutes. At the same time, the intraday wind power generation, photovoltaic power generation and various load forecasts are carried out in the internal coupling system, and the deviation between the forecast results of the day-ahead phase and the intraday phase is calculated, and the second type of demand response is called. At the same time, it is determined whether the second type of demand response can meet the deviation size. If it can meet the deviation size, an intraday deviation declaration plan is formed; if it cannot meet the deviation size, the output of the equipment that can be adjusted during the intraday phase is adjusted according to the response characteristics of various equipment, with the objective function of minimizing equipment adjustment cost (as shown in Equation (4)) and minimizing deviation penalty cost (as shown in Equation (5)). The equipment that can be adjusted during the intraday phase includes batteries, electrolyzers, hydrogen fuel cells and electric heat conversion equipment. Based on this, an intraday deviation declaration plan is formed. In the external electric energy market, intraday clearing, result determination and intraday fee settlement are carried out.

[0103] IV. Construction of Real-Time Stage Deviation Optimization Model

[0104] 41) Objective function

[0105] The real-time phase also aims to minimize adjustment costs and deviation penalty costs. Considering the response characteristics of each device, deviations in the real-time phase are mainly satisfied by the third type of demand response, the battery, and interaction with the external network.

[0106] In the aforementioned real-time stage deviation optimization model, the objective function that minimizes adjustment costs is shown in equation (7):

[0107]

[0108] In the formula: The adjustment cost for the real-time phase of the coupled system; For real-time stage storage batteries; The cost of responding to the third type of demand; These represent the battery's adjusted discharge and charge amounts at time t during the real-time phase. The unit call cost for responding to the third type of demand; The third type of electrical load demand response ratio at time t. The third type of electrical load demand response baseline electrical load demand at time t.

[0109] The objective function for minimizing the deviation penalty cost in the real-time stage deviation optimization model is shown in equation (8):

[0110]

[0111] In the formula: For real-time stage deviation costs; These are the deviations for the electrical energy system, hydrogen energy system, and thermal energy system, respectively.

[0112] 42) Constraints

[0113] Power balance constraints during the real-time phase.

[0114] The power balance constraint condition for the real-time stage is shown in equation (9):

[0115]

[0116] In the formula, The photovoltaic power output at time t in the real-time phase; The wind power output at time t is the real-time stage. The amount of electricity purchased from the power grid at time t in the real-time phase; The real-time electrical load demand at time t; This represents the amount of electricity sold to the grid at time t in the real-time phase. This represents the hydrogen load demand at time t in the real-time phase. This represents the heat load demand at time t in the real-time phase.

[0117] Unlike existing technologies that do not consider the adjustment sequence of each device, this invention performs deviation optimization by combining the characteristic analysis of each system during the real-time stage. (See attached...) Figure 5 It is known that scheduling is carried out 15 minutes in advance, with a scheduling duration of 5 minutes, and the objective function is to minimize the adjustment cost (as shown in Equation (7)) and the deviation cost (as shown in Equation (8)). At the same time, in the internal coupling system, real-time wind power generation, photovoltaic power generation and various load forecasts are performed, the deviation between the intraday stage and the real-time stage forecast results is calculated, and the third type of demand response is called. At the same time, it is determined whether the third type of demand response can meet the deviation size. If it can meet the deviation size, a real-time deviation reporting plan is formed. If it cannot meet the deviation size, the output of the storage battery is adjusted, and it is checked whether the storage battery can meet the remaining deviation amount. Real-time clearing, result determination and real-time cost settlement are carried out in the external electric energy market.

[0118] V. Solving using deep reinforcement learning

[0119] Unlike other existing technologies that mostly employ genetic algorithms for solving the problem, this invention takes into account two aspects: firstly, the existence of multiple variables in the electro-hydrogen-thermal coupling system at the day-ahead, intraday, and real-time stages; and secondly, the dynamic changes in environmental information such as energy storage, wind power generation, and photovoltaic power generation within the electro-hydrogen-thermal coupling system at each moment. Therefore, the multi-timescale scheduling and control solution of the electro-hydrogen-thermal coupling system is a dynamic problem, which is solved using deep reinforcement learning.

[0120] The specific solution process is as follows: First, based on the constraints of the coupled system, identify the environmental parameters of the coupled system; initialize the experience pool, setting the maximum number of training iterations and the number of training iterations per round; execute actions in the action space based on the state space such as wind and solar power output and load demand, and calculate the reward value through the reward function; then, optimize the loss function using PSO, calculate the loss value, and update the reward function; further determine whether the maximum number of training iterations has been reached. If the maximum number of training iterations has been reached, the target output value at this point is the final output value. If the maximum number of training iterations has not been reached, continue training. The solution process is attached. Figure 6 .

[0121] The deep reinforcement learning solution is described, with a maximum training iteration count N. max Construct the action value function D(l,s,w) of the electro-hydrogen-thermal coupled system, which represents the value of taking action s in state l; store the current state, action, and action value function in the experience pool w;

[0122] The loss function K(w) optimized by deep reinforcement learning is constructed using the Q-learning algorithm, as shown in Equation (10):

[0123]

[0124] In the formula: TarQ is the target output value, and PreQ is the predicted output value; t s t Represent the state and action at time t, respectively; and perform the action s. t Afterwards, add rewards. The target output value is updated as shown in equation (11):

[0125]

[0126] In the formula: To predict the decay rate of the output value, a particle swarm optimization algorithm is used for optimization.

[0127] After updating the target output value, the state and the updated value for the next moment are also stored in the experience pool. At the same time, it is determined whether the maximum number of training iterations has been reached. If the maximum number of training iterations has been reached, the target output value at this time is the final output value; if the maximum number of training iterations has not been reached, training continues.

[0128] Using the method of this invention, a case study analysis is conducted on an electric-hydrogen coupling system in a specific region. The unit dispatch costs for the first, second, and third types of load demand response are set to RMB 0.6 / kWh, RMB 0.85 / kWh, and RMB 1.00 / kWh, respectively. Based on the characteristics of each device, the day-ahead, intraday, and real-time dispatch response times are determined to be 30 min, 15 min, and 5 min, respectively. The electric-hydrogen-thermal coupling system is set to connect to the external grid at 10kV, and the interaction with the external grid adopts the peak-valley electricity price for general industrial and commercial use in a certain province (1-10kV). It is also assumed that the purchase and sale prices of electricity are consistent when interacting with the external grid. The unit deviation penalty cost for the electric, thermal, and hydrogen energy systems is set to 20% of the sales price of the electric, thermal, and hydrogen energy systems.

[0129] The scheduling results of the electro-hydrogen-thermal coupled system during the day-ahead, intraday, and real-time phases are attached. Figure 2-4 As shown, a positive value for the hydrogen storage tank indicates hydrogen storage, while a negative value indicates hydrogen release; a negative value for the interaction with the power grid indicates electricity sales, while a positive value indicates electricity purchases.

[0130] From the appendix Figure 2 -Appendix Figure 4 It is known that the first, second, and third types of demand responses are all invoked between 12:00 and 20:00. During the day-ahead period, from 0:00 to 7:00, the off-peak electricity demand, wind and solar power output far exceeds the electricity load demand. At this time, on the one hand, the electrolyzer consumes electricity to convert it into hydrogen energy, which is stored in the hydrogen storage tank to supply the hydrogen load demand; therefore, the hydrogen storage tank is positive from 0:00 to 8:00. On the other hand, batteries are charging, and the heat load demand during the off-peak period is mainly met by the output of the electro-thermal conversion equipment and the waste heat from the electrolyzer. During peak electricity demand, the output of wind and solar power cannot meet the electricity load demand and the electro-thermal conversion demand. At this time, the electricity demand shortfall is generated by battery discharge and hydrogen fuel cell consumption of hydrogen energy. The hydrogen load demand is mainly met by releasing the hydrogen energy stored in the hydrogen storage tank during the off-peak period, reducing the consumption of high-priced electricity. During the peak electricity hours from 18:00 to 21:00, in addition to meeting the demand for hydrogen load, the hydrogen storage tank also needs to supply electricity to the hydrogen fuel cells. At the same time, in order to meet the high demand for electricity, it is also necessary to purchase electricity from the grid.

[0131] During the intraday phase, adjustments to batteries, Type II demand response equipment, electro-thermal conversion equipment, and hydrogen storage tanks are primarily used to address intraday and day-ahead deviations. During the real-time phase, adjustments to batteries and Type III demand response equipment are mainly used to address intraday and real-time deviations.

[0132] Example 2

[0133] This embodiment is a multi-timescale scheduling and control system for an electric-hydrogen-thermal coupled system, which includes a framework building unit for the electric-hydrogen-thermal coupled system, a day-ahead stage operation optimization model building unit, an intraday stage deviation optimization model building unit, a real-time stage deviation optimization model building unit, and a deep reinforcement learning solution model.

[0134] The aforementioned electro-hydrogen-thermal coupling system comprises a source end, a conversion and storage end, and a load end, from energy production to consumption. The source end includes wind power generation and photovoltaic power generation; the conversion and storage end includes batteries, electro-thermal conversion equipment, electrolyzers, hydrogen storage tanks, and hydrogen fuel cells; the load end includes three types of loads: electrical load, thermal load, and hydrogen load. Within the coupling system, there are electrical energy systems, thermal energy systems, and hydrogen energy systems. When the power supply of the electro-hydrogen-thermal coupling system is less than the demand, it exchanges electricity with the external power grid to purchase electricity; when the power supply exceeds the demand, it exchanges electricity with the external power grid to sell electricity.

[0135] The day-ahead phase operation optimization model construction unit includes the day-ahead phase operation optimization model, which comprises:

[0136] 1) Objective function

[0137] The objective function is to maximize the clean energy consumption rate and minimize the operation optimization cost.

[0138] The objective function for minimizing the operational optimization cost is shown in equation (1):

[0139]

[0140] In the formula: This represents the total cost of the coupled system during the day-ahead phase. These are the startup cost, operating cost, maintenance cost, first-type demand response compensation cost, and curtailment penalty cost of the coupled system, respectively; I is the set of equipment, I = {pv, wind, EC, ba, HST, HFC, E2H}, where pv, wind, EC, ba, HST, HFC, and E2H are photovoltaic power generation, wind power generation, electrolyzer, battery, hydrogen storage tank, hydrogen fuel cell, and electric-to-heat conversion equipment, respectively. This represents the startup state of unit i at time t; The startup cost of unit i; a 1i a 2i a 3iThese are the coefficients of the primary, secondary, and constant terms of the output of unit i, respectively. The output of unit i at time t in the day-ahead phase; Let i be the installed capacity of unit i; Let be the maintenance factor for unit i; The unit call cost for the first type of demand response. Let t be the proportion of the first type of electrical load demand response. The reference electrical load demand for the first type of electrical load demand response at time t; These represent the amount of wind and solar power curtailed at time t during the daytime period; u pun The penalty cost is calculated per unit of wind and solar power curtailment.

[0141] The objective function that maximizes the clean energy absorption rate is shown in equation (2):

[0142]

[0143] In the formula: These are the actual outputs of photovoltaic power generation and wind power generation, respectively. This represents the average clean energy utilization rate of the distributed coupled system in the current phase.

[0144] 2) Constraints

[0145] The constraints of an electro-hydrogen-thermal coupled system include electrical balance constraints, hydrogen balance constraints, and thermal balance constraints.

[0146] The constraint conditions of the electro-hydrogen-thermal coupling system are shown in equation (3):

[0147]

[0148] In the formula: The photovoltaic power output at time t during the day-ahead phase; This refers to the wind power output at time t during the current period. The electricity generated by the hydrogen fuel cell at time t in the current phase; This represents the amount of discharge from the battery at time t during the current period. The number of calls made for the first type of demand response at time t before the current date; The amount of electricity consumed by the electrolytic cell at time t before the date; The amount of charge on the battery at time t represents the current period. The electricity consumption for heat conversion from electricity at time t before the date; The electrical load demand at time t before the date; The amount of hydrogen produced by the electrolyzer at time t before the date; H represents the amount of hydrogen consumed by the hydrogen fuel cell at time t before the current date; s h1y,t represents the hydrogen load demand at time t before day t; The installed capacity of the electrolytic cell at time t is the value of the capacity of the cell. The installed capacity of hydrogen fuel cells at time t is the capacity of the fuel cells at that time. The installed capacity of the electric-to-heat conversion equipment at time t before the date; This represents the heat load demand at time t before the current date; These represent the hydrogen release and filling volume of the hydrogen storage tank during the daytime phase, respectively.

[0149] The intraday stage deviation optimization model construction unit includes the following:

[0150] 1) Objective function

[0151] During the intraday phase, considering the uncertainties of wind power, photovoltaic power, and electrical load in the power system, the uncertainty of hydrogen load in the hydrogen energy system, and the uncertainty of heat load in the thermal energy system, the objective function is to minimize equipment adjustment costs and deviation penalty costs.

[0152] In the intraday stage deviation optimization model, based on the response characteristics of each device, the devices that can be adjusted during the intraday stage include batteries, electrolyzers, hydrogen fuel cells, and electrothermal conversion equipment; the objective function for device adjustment cost is shown in equation (4):

[0153]

[0154] In the formula: The adjustment costs for the intraday phase of the coupled system; These are the adjustment costs for the battery, electrolyzer, hydrogen fuel cell, and electric-to-heat conversion equipment during the intraday phase of the coupling system, respectively. The cost of invoking a response to a second type of demand; These represent the battery's adjusted discharge and charge amounts at time t during the intraday period. These represent the output adjustments of the electrolyzer, hydrogen fuel cell, and electrothermal conversion equipment at time t during the intraday phase. These are the unit adjustment costs for batteries, electrolyzers, hydrogen fuel cells, and electric-to-heat conversion equipment during the intraday period, respectively. The unit call cost for responding to the second type of demand; The demand response ratio of the second type of electrical load at time t. The second type of electrical load demand response baseline electrical load demand at time t.

[0155] The objective function for minimizing the deviation penalty cost is shown in equation (5):

[0156]

[0157] In the formula: The cost of penalizing deviations during the intraday phase; These are the deviations for the electrical energy system, hydrogen energy system, and thermal energy system, respectively. These are the unit deviation penalty costs for the electric energy system, hydrogen energy system, and thermal energy system, respectively.

[0158] 2) Constraints

[0159] Power balance constraints during the intraday phase.

[0160] The power balance constraint condition for the intraday phase is shown in equation (6):

[0161]

[0162] In the formula: The photovoltaic power output at time t during the day; The wind power output at time t during the day; This represents the adjustment amount for the hydrogen fuel cell at time t during the intraday phase. This represents the battery discharge adjustment amount at time t during the intraday phase. This represents the second type of demand response call volume at time t within the day. This represents the battery charging adjustment amount at time t during the intraday phase. This represents the adjustment amount of the hydrogen storage tank at time t during the intraday phase. This represents the hydrogen load demand at time t during the intraday period. This refers to the adjustment amount of energy released from the hydrogen storage tank at time t during the intraday phase. This refers to the adjustment amount of the electric-to-heat conversion equipment at time t during the day. This represents the heat load demand at time t during the day. This represents the adjustment amount of hydrogen consumption of the hydrogen fuel cell at time t during the intraday phase. These represent the power consumption adjustment amounts for the electrolytic cell and the electric-to-heat conversion equipment at time t during the intraday phase. These represent the adjustments to the heat generation of the electrolyzer and hydrogen fuel cell at time t during the intraday period.

[0163] The real-time stage deviation optimization model construction unit includes the following:

[0164] 1) Objective function

[0165] The real-time phase also aims to minimize adjustment costs and deviation penalty costs. Considering the response characteristics of each device, deviations in the real-time phase are mainly satisfied by the third type of demand response, the battery, and interaction with the external network.

[0166] In the aforementioned real-time stage deviation optimization model, the objective function that minimizes adjustment costs is shown in equation (7):

[0167]

[0168] In the formula: The adjustment cost for the real-time phase of the coupled system; For real-time stage storage batteries; The cost of responding to the third type of demand; These represent the battery's adjusted discharge and charge amounts at time t during the real-time phase. The unit call cost for responding to the third type of demand; The third type of electrical load demand response ratio at time t. The third type of electrical load demand response baseline electrical load demand at time t.

[0169] The objective function for minimizing the deviation penalty cost in the real-time stage deviation optimization model is shown in equation (8):

[0170]

[0171] In the formula: For real-time stage deviation costs; These are the deviations for the electrical energy system, hydrogen energy system, and thermal energy system, respectively.

[0172] 2) Constraints

[0173] Power balance constraints during the real-time phase.

[0174] The power balance constraint condition for the real-time stage is shown in equation (9):

[0175]

[0176] In the formula, The photovoltaic power output at time t in the real-time phase; The wind power output at time t is the real-time stage. The amount of electricity purchased from the power grid at time t in the real-time phase; The real-time electrical load demand at time t; This represents the amount of electricity sold to the grid at time t in the real-time phase. This represents the hydrogen load demand at time t in the real-time phase. This represents the heat load demand at time t in the real-time phase.

[0177] In the deep reinforcement learning solution model, the deep reinforcement learning solution sets a maximum number of training iterations N. max Construct the action value function D(l,s,w) of the electro-hydrogen-thermal coupled system, representing the value of taking action s in state l; store the current state, action, and action value function in the experience pool w;

[0178] The loss function K(w) optimized by deep reinforcement learning is constructed using the Q-learning algorithm, as shown in Equation (10):

[0179]

[0180] In the formula: TarQ is the target output value, and PreQ is the predicted output value; t s t Represent the state and action at time t, respectively; and perform the action s. t Afterwards, add rewards. The target output value is updated as shown in equation (11):

[0181]

[0182] In the formula: To predict the decay rate of the output value, a particle swarm optimization algorithm is used for optimization.

[0183] After updating the target output value, the state and the updated value for the next moment are also stored in the experience pool. At the same time, it is determined whether the maximum number of training iterations has been reached. If the maximum number of training iterations has been reached, the target output value at this time is the final output value; if the maximum number of training iterations has not been reached, training continues.

[0184] Based on the content of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

Claims

1. A method for multi-time scale scheduling control of an electro-hydro-thermal coupling system, characterized in that, This includes the framework construction of the electric-hydrogen-thermal coupling system, the construction of the day-ahead phase operation optimization model, the construction of the intraday phase deviation optimization model, the construction of the real-time phase deviation optimization model, and the solution using deep reinforcement learning. The aforementioned day-to-day operational optimization model includes: 1) Objective function The objective function is to maximize the clean energy consumption rate and minimize the operation optimization cost. 2) Constraints The constraints of an electro-hydrogen-thermal coupled system include electrical balance constraints, hydrogen balance constraints, and thermal balance constraints. The objective function for minimizing the operational optimization cost is shown in equation (1): （1） In the formula: This represents the total cost of the coupled system during the day-ahead phase. , , , , These are the startup cost, operating cost, maintenance cost, first-type demand response compensation cost, and curtailment penalty cost of the coupled system, respectively. For equipment collection, , pv, wind, EC, ba, HST, HFC, E2H These include photovoltaic power generation, wind power generation, electrolyzers, storage batteries, hydrogen storage tanks, hydrogen fuel cells, and electric-to-heat conversion equipment. For the unit i exist t The startup status at any given moment; For the unit i Startup costs; , , The units i The coefficients of the first, second, and constant terms of the output; For the current stage t Time crew i contribution; For the unit i The installed capacity; For the unit i Maintenance factor; For the first type of demand response, the unit call cost is... for t The demand response ratio of Category I electrical loads at any given time. for t The first type of electrical load demand response is based on the baseline electrical load demand. , Each day is a separate phase. t Constantly forfeiting air and solar energy; Penalty costs per unit of wind and solar curtailment; The objective function that maximizes the clean energy absorption rate is shown in equation (2): （2） In the formula: , These are the actual outputs of photovoltaic power generation and wind power generation, respectively. The average clean energy utilization rate of the distributed coupled system in the current stage; The constraint conditions of the electro-hydrogen-thermal coupling system are shown in equation (3): （3） In the formula: For the current stage t Photovoltaic power generation output at all times; For the current stage t Wind power output at all times; For the current stage t Electricity generation from hydrogen fuel cells at all times; For the current stage t The amount of discharge from the battery at any given time; For the day before t The number of calls to the first type of demand response at any given moment; For the day before t The electrolytic cell consumes electricity at all times; For the current stage t The battery charge level at all times; For the day before t The amount of electricity consumed in the process of converting electricity into heat; For the day before t Real-time electrical load demand; For the day before t The hydrogen production rate of the electrolyzer at any given time; For the day before t The amount of hydrogen consumed by a hydrogen fuel cell at any given time; For the day before t Hydrogen load demand at any given time; For the day before t The installed capacity of the electrolytic cell at any given time; For the day before t The installed capacity of hydrogen fuel cells at any given time; For the day before t The installed capacity of the electro-thermal conversion equipment at all times; For the day before t Constant heat load demand; , These represent the hydrogen release and filling volume of the hydrogen storage tank during the daytime phase, respectively.

2. The multi-timescale scheduling and control method for an electro-hydrogen-thermal coupled system according to claim 1, characterized in that, The aforementioned electro-hydrogen-thermal coupling system comprises a source end, a conversion and storage end, and a load end, from energy production to consumption. The source end includes wind power generation and photovoltaic power generation; the conversion and storage end includes batteries, electro-thermal conversion equipment, electrolyzers, hydrogen storage tanks, and hydrogen fuel cells; the load end includes three types of loads: electrical load, thermal load, and hydrogen load. Within the coupling system, there are electrical energy systems, thermal energy systems, and hydrogen energy systems. When the power supply of the electro-hydrogen-thermal coupling system is less than the demand, it exchanges electricity with the external power grid to purchase electricity; when the power supply is greater than the demand, it exchanges electricity with the external power grid to sell electricity.

3. The multi-timescale scheduling and control method for an electro-hydrogen-thermal coupled system according to claim 1, characterized in that, The intraday stage deviation optimization model includes: 1) Objective function During the intraday phase, considering the uncertainties of wind power, photovoltaic power, and electrical load in the power system, the uncertainty of hydrogen load in the hydrogen energy system, and the uncertainty of heat load in the thermal energy system, the objective function is to minimize equipment adjustment costs and deviation penalty costs. 2) Constraints Power balance constraints during the intraday phase.

4. The multi-timescale scheduling and control method for an electro-hydrogen-thermal coupled system according to claim 3, characterized in that, In the intraday stage deviation optimization model, based on the response characteristics of each device, the devices that can be adjusted during the intraday stage include batteries, electrolyzers, hydrogen fuel cells, and electrothermal conversion equipment; the objective function for device adjustment cost is shown in equation (4): （4） In the formula: The adjustment costs for the intraday phase of the coupled system; , , , These are the adjustment costs for the battery, electrolyzer, hydrogen fuel cell, and electric-to-heat conversion equipment during the intraday phase of the coupling system, respectively. The cost of invoking a response to a second type of demand; , Intraday phase t The battery constantly adjusts its discharge and charge levels. , , Intraday phase t Adjustments to the output of the electrolyzer, hydrogen fuel cell, and electrothermal conversion equipment at all times; , , , These are the unit adjustment costs for batteries, electrolyzers, hydrogen fuel cells, and electric-to-heat conversion equipment during the intraday period, respectively. The unit call cost for responding to the second type of demand; for t The demand response ratio of Category II electrical loads at any given time. for t The second category of electrical load demand response is based on the baseline electrical load demand. The objective function for minimizing the deviation penalty cost is shown in equation (5): （5） In the formula: The cost of penalizing deviations during the intraday phase; , , These are the deviations for the electrical energy system, hydrogen energy system, and thermal energy system, respectively. , , These are the unit deviation penalty costs for electric energy systems, hydrogen energy systems, and thermal energy systems, respectively. The power balance constraint for the intraday phase is shown in equation (6): （6） In the formula: Intraday phase t Photovoltaic power generation output at all times; Intraday phase t Wind power output at all times; Intraday phase t Adjustments to the hydrogen fuel cell at any given time; Intraday phase t Adjust the battery discharge level at all times; Intraday phase t The number of calls to the second type of demand at any given time; Intraday phase t Adjust the battery charging level at all times; Intraday phase t The adjustment amount of the hydrogen storage tank at any given time; Intraday phase t Hydrogen load demand at any time; Intraday phase t Adjust the amount of energy released from the hydrogen storage tank at all times; Intraday phase t Adjustments to the electric-to-heat conversion equipment at all times; Intraday phase t Constant heat load demand; Intraday phase t Adjustment of hydrogen consumption in a hydrogen fuel cell at any time; , Intraday phase t Adjustment of power consumption of electrolytic cells and electric-to-heat conversion equipment at all times; , Intraday phase t Adjustment of heat generation in the electrolyzer and hydrogen fuel cell at all times.

5. The multi-timescale scheduling and control method for an electro-hydrogen-thermal coupled system according to claim 4, characterized in that, The real-time stage deviation optimization model includes: 1) Objective function The real-time phase also aims to minimize adjustment costs and deviation penalty costs. Considering the response characteristics of each device, deviations in the real-time phase are mainly satisfied by the third type of demand response, the battery, and interaction with the external network. 2) Constraints Power balance constraints in the real-time phase.

6. The multi-timescale scheduling and control method for an electro-hydrogen-thermal coupled system according to claim 5, characterized in that, In the aforementioned real-time stage deviation optimization model, the objective function that minimizes adjustment costs is shown in equation (7): （7） In the formula: The adjustment cost for the real-time phase of the coupled system; For real-time stage storage batteries; The cost of responding to the third type of demand; , Real-time stage t The battery constantly adjusts its discharge and charge levels. The unit call cost for responding to the third type of demand; for t The demand response ratio of Category III electrical loads at any given time. for t The third category of electrical load demand response is based on the baseline electrical load demand. The objective function for minimizing the deviation penalty cost in the real-time stage deviation optimization model is shown in equation (8): （8） In the formula: For real-time stage deviation costs; , , These are the deviations for the electrical energy system, hydrogen energy system, and thermal energy system, respectively. The power balance constraint condition for the real-time stage is shown in equation (9): （9） In the formula, For real-time stage t Photovoltaic power generation output at all times; For real-time stage t Wind power output at all times; For real-time stage t Electricity purchased from the power grid at all times; For real-time stage t Real-time electrical load demand; For real-time stage t The amount of electricity sold to the power grid at all times; For real-time stage t Hydrogen load demand at any time; For real-time stage t Constant heat load demand.

7. The multi-timescale scheduling and control method for an electro-hydrogen-thermal coupled system according to claim 1, characterized in that, The deep reinforcement learning solution is described, with a maximum number of training iterations set. Construct the action value function of the electro-hydrogen-thermal coupled system , indicating state Take action below The value lies in storing the current state, actions, and action value functions in the experience pool. middle; A loss function optimized for deep reinforcement learning is constructed using the Q-learning algorithm. As shown in equation (10): （10） In the formula: Output the target value. To predict the output value; , They represent t The state and actions at any given moment; Execute action Afterwards, add rewards. The target output value is updated as shown in equation (11): （11） In the formula: To predict the decay rate of the output value, a particle swarm optimization algorithm is used for optimization. After updating the target output value, the state and the updated value for the next moment are also stored in the experience pool. At the same time, it is determined whether the maximum number of training iterations has been reached. If the maximum number of training iterations has been reached, the target output value at this time is the final output value; if the maximum number of training iterations has not been reached, training continues.

8. A multi-timescale scheduling and control system for an electric-hydrogen-thermal coupled system, used to implement the multi-timescale scheduling and control method for the electric-hydrogen-thermal coupled system according to any one of claims 1-7, characterized in that, It includes a framework building unit for an electric-hydrogen-thermal coupled system, a day-ahead stage operation optimization model building unit, an intraday stage deviation optimization model building unit, a real-time stage deviation optimization model building unit, and a deep reinforcement learning solution model.

Images