A method and system for coordinated optimization scheduling of electric hydrogen heat and cold storage in a park

Abstract

The present invention discloses a coordinated optimization scheduling method and system for electric hydrogen heat cold storage in a park, and the method includes: converting the physical model of the multi-energy scheduling of the park into an improved DDQN intelligent agent; the improved DDQN intelligent agent determines the first step action, that is, the action of all 0-1 variables according to the behavior strategy based on the operating state given by the environment, and judges whether the 0-1 variable constraint is satisfied; if satisfied, the second step action and are solved by the interior point method, and judge whether the energy storage operation constraint is satisfied; if satisfied, the upper limit value is taken when the upper limit is exceeded, and the lower limit value is taken when the lower limit is exceeded; judge whether other constraints are satisfied; if not satisfied, the reward value is set to a minimum value and fed back to the intelligent agent; the action of the intelligent agent at the current moment, the system state given by the environment, and the uncertainty are input into the state conversion function to obtain the system state at the new moment. It can effectively process high-dimensional continuous action space and can better fit the optimization scheduling problem of coordinated electric hydrogen heat cold storage in the park.

Description

Technical Field

[0001] The present invention belongs to the technical field of integrated energy dispatching, and in particular relates to a method and system for coordinated optimization dispatching of electric hydrogen heat and cold storage in a park. Background Art

[0002] As a flexible and adjustable resource, energy storage can help the park achieve energy autonomy and benign access to the grid through low charging and high discharging. Introducing a hydrogen energy system on the demand side, the electrolysis of hydrogen technology can be used to convert electrical energy into hydrogen energy for storage when there is surplus electricity, and the fuel cell power generation technology can be used to convert hydrogen energy into electrical energy for transmission when there is a shortage of electricity. This can suppress the fluctuation of renewable energy in a renewable and zero-emission manner, while saving the operating costs of the park. In addition to hydrogen storage devices, the park's commonly used flexible adjustment resources include electrochemical energy storage and thermal energy storage; considering that the energy consumption of the refrigeration system in the park accounts for a prominent proportion of the overall energy consumption in high temperature weather in summer, an ice storage air conditioning system is introduced in the park to meet the cooling load demand and save cooling costs. The ice storage system is mainly composed of a base load unit, a dual-mode unit, and an ice storage tank. Among them, the base load unit provides cooling demand; at low peak electricity prices, the dual-mode unit stores the cold in the ice storage tank in the form of ice by making ice, and at peak electricity prices, the ice storage tank melts ice to release cold, and the dual-mode unit turns on the cooling mode as appropriate. By utilizing the electricity price difference to rationally dispatch the units in the ice storage system, the effect of optimizing costs and stabilizing cooling can be achieved.

[0003] The coordinated optimization scheduling model of electric hydrogen heat and cold storage in the park is a dynamic complex mixed integer nonlinear programming (MINLP) problem. At present, a hybrid solution combining traditional optimization methods and heuristic algorithms is usually used to solve such problems. However, the optimization results of the hybrid algorithm are heavily dependent on the accuracy of the prediction; when the prediction error is large, the accuracy of the results decreases. In addition, hybrid optimization algorithms often use assumptions; generally speaking, the premise assumptions are difficult to hold in real problems. In the solution process, hybrid algorithms often transform or eliminate the constraints in the model, thereby converting complex models into a form that is mathematically easy to solve. This transformation reduces the accuracy of the model and also causes deviations in the solution. If the impact of uncertainty needs to be further considered, it will bring greater challenges to the solution. Summary of the invention

[0004] The present invention provides a park electric hydrogen heat and cold storage coordinated optimization scheduling method and system, which are used to solve the technical problems of being unable to adapt to the solution of high-dimensional continuous complex large-scale problems and the low efficiency and accuracy of the algorithm.

[0005] In a first aspect, the present invention provides a method for coordinated optimization scheduling of electric hydrogen heat and cold storage in a park, comprising:

[0006] Under the preset constraints, a multi-energy dispatch physical model of the park is constructed with the minimum total operating cost as the objective function. The constraints include system power balance constraints, unit operation constraints, energy storage operation constraints, interconnection line power constraints, and power flow constraints.

[0007] Converting the park multi-energy scheduling physical model into an improved DDQN agent, wherein the improved DDQN agent includes an online network and a target network;

[0008] Improve the running state of DDQN agent based on the environment , according to the behavioral strategy Determine the first step, that is, the action of all 0-1 variables , and judge Whether the 0-1 variable constraint is satisfied;

[0009] like If the 0-1 variable constraint is not met, the reward value is set to the minimum value Feedback to the improved DDQN agent, ending this round, if If the 0-1 variable constraint is satisfied, the interior point method is used to solve the second step action and , and judge Whether the energy storage operation constraints are met;

[0010] like If the energy storage operation constraints are met, then When the upper limit is exceeded, the upper limit value is taken. When the lower limit is exceeded, the lower limit value is taken;

[0011] like If the energy storage operation constraints are not met, then Whether the system power balance constraints, unit operation constraints, interconnection line power constraints and power flow constraints are met;

[0012] like If the system power balance constraints, unit operation constraints, tie line power constraints and power flow constraints are met, the reward function is calculated And feedback to improve the DDQN agent;

[0013] like If the system power balance constraint, unit operation constraint, tie line power constraint and power flow constraint are not met, the reward value is set to the minimum value. And feedback to the agent to terminate this round;

[0014] The current moment The actions of the agent and the system state given by the environment and uncertainty Input state transition function , get the system state at the new time .

[0015] In a second aspect, the present invention provides a park electric hydrogen heat and cold storage coordinated optimization scheduling system, comprising:

[0016] A construction module is configured to construct a park multi-energy scheduling physical model under preset constraints with the minimum total operating cost as the objective function, wherein the constraints include system power balance constraints, unit operation constraints, energy storage operation constraints, interconnection line power constraints, and power flow constraints;

[0017] A conversion module, configured to convert the park multi-energy scheduling physical model into an improved DDQN agent, wherein the improved DDQN agent includes an online network and a target network;

[0018] The first judgment module is configured to improve the running state of the DDQN agent based on the environment , according to the behavioral strategy Determine the first step, that is, the action of all 0-1 variables , and judge Whether the 0-1 variable constraint is satisfied;

[0019] The second judgment module is configured as follows: If the 0-1 variable constraint is not met, the reward value is set to the minimum value Feedback to the improved DDQN agent, ending this round, if If the 0-1 variable constraint is satisfied, the interior point method is used to solve the second step action and , and judge Whether the energy storage operation constraints are met;

[0020] The value module is configured as follows: If the energy storage operation constraints are met, then When the upper limit is exceeded, the upper limit value is taken. When the lower limit is exceeded, the lower limit value is taken;

[0021] The third judgment module is configured as follows: If the energy storage operation constraints are not met, then Whether the system power balance constraints, unit operation constraints, interconnection line power constraints and power flow constraints are met;

[0022] The first feedback module is configured as follows: If the system power balance constraints, unit operation constraints, tie line power constraints and power flow constraints are met, the reward function is calculated And feedback to improve the DDQN agent;

[0023] The second feedback module is configured as follows: If the system power balance constraint, unit operation constraint, tie line power constraint and power flow constraint are not met, the reward value is set to the minimum value. And feedback to the agent to terminate this round;

[0024] Output module, configured to output the current time The actions of the agent and the system state given by the environment and uncertainty Input state transition function , get the system state at the new time .

[0025] In a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps of the coordinated optimization scheduling method for park electricity, hydrogen, heat, cooling and storage according to any embodiment of the present invention.

[0026] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program instructions are executed by a processor, the processor executes the steps of the park electric hydrogen heat and cold storage coordinated optimization scheduling method of any embodiment of the present invention.

[0027] The park electric hydrogen heat and cold storage coordinated optimization scheduling method and system of the present application has the following beneficial effects:

[0028] Taking into full consideration the various load requirements of electricity, hydrogen, heat, cold storage, the first large-scale park system including wind turbines, photovoltaics, electrolysis devices, fuel cells, electric boilers, ice storage systems, electrochemical energy storage, thermal energy storage and hydrogen storage devices was designed. The system fits the actual situation of large parks and realizes energy autonomy, multi-source complementarity and economic operation within the park. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 A flow chart of a method for coordinated optimization scheduling of electric hydrogen heat and cold storage in a park provided by an embodiment of the present invention;

[0031] Figure 2 A structural block diagram of a park electric hydrogen heat and cold storage coordinated optimization scheduling system provided by an embodiment of the present invention;

[0032] Figure 3 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] See also Figure 1 , which shows a flow chart of a method for coordinated optimization scheduling of electric hydrogen heat and cold storage in a park of the present application.

[0035] like Figure 1 As shown in the figure, the coordinated optimization dispatching method of electric hydrogen heat and cold storage in the park specifically includes the following steps:

[0036] Step S101, under preset constraints, a park multi-energy scheduling physical model is constructed with the minimum total operating cost as the objective function, wherein the constraints include system power balance constraints, unit operation constraints, energy storage operation constraints, tie line power constraints, and power flow constraints;

[0037] Step S102, converting the park multi-energy scheduling physical model into an improved DDQN agent, wherein the improved DDQN agent includes an online network and a target network;

[0038] Step S103: Improve the running state of the DDQN agent based on the environment , according to the behavioral strategy Determine the first step, that is, the action of all 0-1 variables , and judge Whether the 0-1 variable constraint is satisfied;

[0039] Step S104: If If the 0-1 variable constraint is not met, the reward value is set to the minimum value Feedback to the improved DDQN agent, ending this round, if If the 0-1 variable constraint is satisfied, the interior point method is used to solve the second step action and , and judge Whether the energy storage operation constraints are met.

[0040] It should be noted that It is the collection of charging and discharging work of all energy storage in the park. It is the output collection of all the park units except wind power, photovoltaic and dual-mode units.

[0041] Step S105: If If the energy storage operation constraints are met, then When the upper limit is exceeded, the upper limit value is taken. When the lower limit is exceeded, the lower limit value is taken;

[0042] Step S106: If If the energy storage operation constraints are not met, then Whether the system power balance constraints, unit operation constraints, interconnection line power constraints and power flow constraints are met;

[0043] Step S107, if If the system power balance constraints, unit operation constraints, tie line power constraints and power flow constraints are met, the reward function is calculated And feedback to improve the DDQN agent;

[0044] Step S108, if If the system power balance constraint, unit operation constraint, tie line power constraint and power flow constraint are not met, the reward value is set to the minimum value. And feedback to the agent to terminate this round;

[0045] Step S109: the current time The actions of the agent and the system state given by the environment and uncertainty Input state transition function , get the system state at the new time .

[0046] In summary, the method of the present application, the double Deep Q-Network (DDQN) is an algorithm in deep reinforcement learning, which is optimized by the DQN algorithm. DDQN combines deep neural networks with Q learning, and uses deep neural networks to approximate the value function, thereby effectively processing high-dimensional and complex state spaces; at the same time, the target network and experience replay training network are used to increase the efficiency and stability of training. Compared with DQN, the DDQN algorithm uses online networks and target networks to complete action selection and action value estimation respectively, which can effectively alleviate the problem of overestimation of Q values, making the algorithm more robust and stable. In addition, the algorithm continuously updates the weights of the neural network during the interaction with the changing environment, which can better meet dynamic and uncertain scenarios. In the early stage of the algorithm, the agent explores and tries new actions in order to discover potential better strategies. The above advantages make the algorithm very suitable for solving the coordinated optimization scheduling of electric hydrogen heat and cold storage in the park established by the present invention. In terms of strategy selection, the DDQN agent selects the optimal action by balancing exploration and utilization. In the new state space, collect more data to avoid falling into the local optimum; in the middle and late stages of the algorithm, the agent makes more use of the currently learned strategies to select actions and maximize rewards, thereby making optimal decisions based on existing knowledge and improving performance.

[0047] In a specific embodiment, the park electric hydrogen heat and cold storage coordinated optimization scheduling model established by the present invention is a dynamic complex mixed integer nonlinear programming (MINLP) problem. At present, a hybrid solution combining traditional optimization methods and heuristic algorithms is usually used to solve such problems. However, the optimization results of the hybrid algorithm are heavily dependent on the accuracy of the prediction; when the prediction error is large, the accuracy of the result decreases. In addition, hybrid optimization algorithms often use assumptions; generally speaking, the premise assumptions are difficult to establish in real problems. In the solution process, hybrid algorithms often transform or eliminate the constraints in the model, thereby converting complex models into a form that is easy to solve mathematically. This transformation reduces the accuracy of the model and also causes deviations in the solution. If the influence of uncertainty needs to be further considered, it will bring greater challenges to the solution. In view of the shortcomings of using mathematical methods to solve dynamic complex MINLP problems, the present invention introduces a deep reinforcement learning method (Deep Reinforcement Learning, DRL). DRL can automatically learn complex strategies from raw data without the need to manually design specific features or rules. At the same time, with the help of deep neural networks, DRL can not only effectively process high-dimensional state and action spaces, but also achieve a high degree of fitting of nonlinear problems. In addition, traditional mathematical methods usually assume that the environment is static and deterministic, while DRL's continuous learning allows it to continuously adapt and improve strategies in dynamic and uncertain environments, making it more robust than traditional methods.

[0048] Deterministic policy gradient (Deep Q-Network, DQN) is a classic algorithm for deep reinforcement learning. DQN combines deep neural networks with Q learning, and uses the characteristics of layer-by-layer processing and multi-feature changes to adapt to complex models; at the same time, it uses the target network and experience replay for learning and training, breaking the correlation of the park power grid operation data and reaching convergence faster. However, since DQN uses a single network for action selection and action value evaluation, it often overestimates the Q value, which affects the convergence of the algorithm. In order to solve the above shortcomings, the present invention uses the DDQN algorithm improved on the basis of the DQN algorithm. DDQN uses an online network to select actions and a target network to evaluate Q values, achieving better strategy selection and more accurate Q value estimation.

[0049] However, the DDQN agent takes a long time to determine the optimal action through the balance of exploration and utilization, and the enumeration method used is more suitable for discrete action space. Considering that the output of distributed power sources and units in the park is mostly continuous variables, the present invention improves the DDQN algorithm and breaks down the action of the agent into two steps for calculation: through the exploration and utilization of DDQN to calculate the 0-1 variables in the model, on the basis of determining the discrete variables, the interior point method is used to calculate the continuous variables in the model. The redesign of DDQN makes it more suitable for solving high-dimensional continuous complex large-scale problems, greatly improving the efficiency and accuracy of the algorithm. Specifically including:

[0050] Step 1: Establish a physical model for multi-energy scheduling in large-scale parks.

[0051] 1.1. Objective function

[0052] In the face of large-scale parks with coupled electricity, hydrogen, heat, cold and storage, a multi-energy coordinated optimization scheduling model is established. The model aims to minimize the total operating cost. The total operating cost includes the cost of purchasing and selling energy between the park and the power grid, the cost of equipment operation and maintenance, the penalty cost of load loss, and the loss cost of energy storage, which can be specifically expressed as:

[0053] , (1)

[0054] In the formula, is the cost of energy purchase and sale between the park and the large power grid at time t, is the equipment operation and maintenance cost at time t, is the load loss penalty cost at time t, is the energy storage loss cost at time t;

[0055] , (2)

[0056] In the formula, is the electricity purchase price of the park at time t, is the power purchased by the park at time t, is the heat purchase price of the park at time t, is the amount of heat purchased by the park at time t, is the price of hydrogen purchased by the park at time t, is the hydrogen purchase power of the park at time t, is the electricity price of the park at time t, is the electricity sales power of the park at time t, is the time period;

[0057] , (3)

[0058] In the formula, is the operation and maintenance cost of the wind turbine at time t, is the output of the fan at time t, is the operation and maintenance cost of photovoltaic at time t, is the output of photovoltaic at time t, is the operation and maintenance cost of the electrolysis device at time t, is the power consumption of the electrolysis device at time t, is the operation and maintenance cost of the fuel cell at time t, is the electrical power output of the fuel cell at time t, is the operation and maintenance cost of the electric boiler at time t, is the power consumption of the electric boiler at time t, is the operation and maintenance cost of the baseload unit at time t, is the power consumption of the base load unit at time t, is the operation and maintenance cost of the dual-operating unit at time t, is the power consumption of the dual-operating unit at time t;

[0059] , (4)

[0060] In the formula, is the cost of the power-off load, For power failure load, is the cost of the heat loss load, is the heat loss load, is the cost of cooling load, is the cooling load, is the cost of hydrogen loss load, is the hydrogen loss load;

[0061] , (5)

[0062] In the formula, is the battery energy stored at time t, is the charging power of the battery energy storage at time t, is the life attenuation cost of the heat storage tank at time t, is the heat storage capacity of the heat storage tank, is the life decay cost of the ice storage tank at time t, is the cold storage capacity of the ice storage tank, is the lifetime decay cost of hydrogen energy storage at time t, The charging power of hydrogen energy storage, is the discharge power of the battery energy storage at time t, is the heat release of the heat storage tank, is the cooling capacity of the ice storage tank, It is the hydrogen release power of hydrogen energy storage.

[0063] 1.2. Constraints.

[0064] The constraints of the park's electricity, hydrogen, heat, cooling and storage coordinated optimization scheduling model include system power balance constraints, unit operation constraints, energy storage operation constraints, and power constraints of the interconnection line between the park and the large power grid, which are specifically expressed as:

[0065] System power balance constraint: The coordinated optimization scheduling of the park must ensure the real-time balance of system power, hydrogen energy, heat and cooling, as shown in equations (6) to (9):

[0066] , (6)

[0067] In the formula, is the output of the fan at time t, is the output of photovoltaic at time t, is the discharge power of the battery energy storage at time t, is the charging power of the battery energy storage at time t, is the hydrogen production power of the electrolysis device at time t, is the electrical power output of the fuel cell at time t, is the power consumption of the electric boiler at time t, is the hydrogen consumption power of the fuel cell at time t, is the power consumption of the dual-operating unit at time t, is the charging power of the battery energy storage at time t, is the electricity sales power at time t, is the electrical load at time t, is the power failure load of the park at time t;

[0068] , (7)

[0069] In the formula, is the heat release of the heat storage tank at time t, is the heat storage capacity of the heat storage tank at time t, is the heating capacity of the electric boiler at time t, is the power purchased and sold at time t, is the heat load at time t, is the heat loss load at time t;

[0070] , (8)

[0071] In the formula, is the cooling capacity of the ice storage tank at time t, is the cooling capacity of the ice storage tank at time t, is the cooling capacity of the base load unit, is the cooling capacity of the dual-mode unit, is the cooling load at time t, is the cooling load at time t;

[0072] , (9)

[0073] In the formula, is the hydrogen discharge power of hydrogen energy storage at time t, is the charging power of hydrogen energy storage at time t, is the hydrogen production power of the electrolysis device at time t, is the hydrogen consumption power of the fuel cell at time t, is the hydrogen purchase power at time t, is the hydrogen load at time t, is the hydrogen loss load at time t;

[0074] The unit operation constraints include the electrolyzer constraints of equations (10) and (11), the fuel cell constraints of equations (12) and (13), the electric boiler constraints of equations (14) and (15), the base load unit constraints of equations (16) to (19), and the dual-operating unit constraints of equations (20) to (27):

[0075] , (10)

[0076] , (11)

[0077] In the formula, is the hydrogen production power of the electrolysis device at time t, is the power consumption of the electrolytic cell at time t, is the hydrogen production efficiency of the electrolyzer, is the upper limit of the power consumption of the electrolytic cell;

[0078] , (12)

[0079] , (13)

[0080] In the formula, is the power consumption of the fuel cell at time t, is the hydrogen consumption power of the fuel cell at time t, is the power generation efficiency of the fuel cell, is the upper limit of fuel cell power consumption;

[0081] , (14)

[0082] , (15)

[0083] In the formula, is the power consumption of the electric boiler at time t, is the heating capacity of the electric boiler at time t, For the moment The working temperature of the electric boiler, is the thermal resistance of the electric boiler, The thermal resistance of the electric boiler. is the time interval between two moments, For the moment +1 The working temperature of the electric boiler, For the moment The ambient temperature, The upper limit of the heat generated by the electric boiler;

[0084] , (16)

[0085] , (17)

[0086] , (18)

[0087] , (19)

[0088] In the formula, is the cooling capacity of the base load unit, is the power consumption of the base load unit, For the base crew at time The load factor, , , These are the unit parameters of the base load unit. is the rated cooling capacity of the base load unit, The upper limit of the cooling capacity generated by the base load unit, is the lower limit of the load factor of the base load unit;

[0089] , (20)

[0090] ,(twenty one)

[0091] ,(twenty two)

[0092] ,(twenty three)

[0093] ,(twenty four)

[0094] , (25)

[0095] , (26)

[0096] , (27)

[0097] In the formula, Indicates that the dual-mode unit is in cooling mode. The cooling capacity generated by the dual-mode unit in refrigeration mode, Indicates that the dual-mode unit is in ice-making mode. The cooling capacity generated by the dual-mode unit in ice-making mode. is the load rate of the dual-operating unit, , , These are the unit parameters of dual-operating units. is the power consumption of the dual-operating unit, is the rated cooling capacity of the dual-mode unit, is the load rate of the dual-mode unit in cooling condition, is the load rate of the dual-mode unit in ice-making condition, Indicates that the dual-mode unit is in ice-making mode. The upper limit of the cooling capacity of the dual-mode unit. The upper limit of ice production in dual-mode units. is the lower limit of the refrigeration load rate of the dual-mode unit. It is the lower limit of the load rate of the dual-mode unit in ice-making condition;

[0098] The energy storage constraints consist of charging and discharging power constraints, energy storage capacity constraints, and charging and discharging state constraints. The constraints of the energy storage battery are shown in equations (28) to (32), and the constraints of the heat storage tank, ice storage tank operation constraints, and hydrogen energy storage constraints are shown in equations (33) to (37), (38) to (44), and (45) to (47), respectively:

[0099] , (28)

[0100] , (29)

[0101] , (30)

[0102] , (31)

[0103] , (32)

[0104] In the formula, Indicates that the energy storage battery is at time Charge, Indicates that the energy storage battery is at time Not charging, is the maximum charging power of the energy storage converter, is the maximum discharge power of the energy storage converter, Indicates that the energy storage battery is at time Discharge, Indicates that the energy storage battery is not Discharge, is the maximum discharge power of the energy storage converter, For energy storage batteries at all times The capacity, For energy storage batteries at all times -1 capacity, For the charging efficiency of the energy storage battery, The energy storage converter is at time The charging power, The energy storage converter is at time The discharge power, is the energy discharge efficiency of the energy storage battery, is the maximum capacity of the energy storage battery, is the minimum state of charge of the energy storage battery, is the maximum state of charge of the energy storage battery;

[0105] , (33)

[0106] , (34)

[0107] , (35)

[0108] , (36)

[0109] , (37)

[0110] In the formula, is the charging power of the heat storage tank at time t, is the binary variable of the heat storage tank being charged at time t, is the maximum heating power of the heat storage tank, is the heat release power of the heat storage tank at time t, is a binary variable that indicates the energy released by the heat storage tank at the time. is the maximum heat release power of the heat storage tank, For the heat storage tank at time The capacity, For the heat storage tank at time -1 capacity, is the self-loss coefficient of the heat storage tank, is the charging efficiency of the heat storage tank, is the energy release efficiency of the heat storage tank, is the time interval between two moments, is the upper limit of the capacity of the heat storage tank;

[0111] , (38)

[0112] , (39)

[0113] , (40)

[0114] , (41)

[0115] , (42)

[0116] , (43)

[0117] , (44)

[0118] In the formula, is the cooling power of the ice storage tank at time t, is the binary variable of ice storage tank charging at time t, is the maximum cooling power of the ice storage tank, is the cooling power of the ice storage tank at time t, is the binary variable of the energy released by the ice storage tank at time t, is the maximum cooling power of the ice storage tank, For the heat storage tank at time The capacity, For the heat storage tank at time The capacity, is the self-loss coefficient of the ice storage tank, is the charging efficiency of the ice storage tank, is the energy release efficiency of the ice storage tank, is the upper limit of the capacity of the heat storage tank, The charging power of hydrogen energy storage, is a binary variable for charging the hydrogen storage at time t, is the maximum charging power of hydrogen energy storage, is the hydrogen release power of hydrogen energy storage, is a binary variable for hydrogen energy storage to release energy at time t, is the maximum hydrogen release power of hydrogen energy storage;

[0119] , (45)

[0120] , (46)

[0121] , (47)

[0122] In the formula, For hydrogen storage at all times The capacity, For hydrogen storage at all times The capacity, The charging efficiency of hydrogen energy storage, is the energy release efficiency of hydrogen energy storage, is the upper limit of the capacity of the hydrogen storage tank;

[0123] The expression of tie line power constraint is:

[0124] , (48)

[0125] , (49)

[0126] In the formula, is the maximum electric power allowed to be exchanged between the microgrid and the main grid tie line, is the maximum hydrogen power allowed to be exchanged between the microgrid and the main grid tie line, is the hydrogen purchase power at time t, is the charging power of the battery energy storage at time t, is the electricity sales power at time t;

[0127] The expression of power flow constraint is:

[0128] , (50)

[0129] , (51)

[0130] , (53)

[0131] , (54)

[0132] , (55)

[0133] In the formula, For the moment node The voltage, For the moment node The voltage, For the node All connected nodes, For line The conductivity, For the moment node With Node The phase angle, For line The electrical susceptance, For the moment With Node The active power delivered by the connected energy source, For the moment node The active load, For the moment With Node Reactive power from connected energy sources, For the moment node The reactive load, For Node The voltage lower limit, For Node The upper voltage limit, For Node The lower limit of the phase angle, For Node The upper limit of the phase angle, For line The upper limit of active power allowed to flow.

[0134] Step 2: Description of the park multi-energy coordination optimization problem based on improved DDQN.

[0135] 2.1. Transformation of Markov decision process.

[0136] The coordinated optimization of the park's electric hydrogen, heat and cold storage is achieved by adjusting the charging and discharging power of each energy storage. Therefore, this step describes the optimization scheduling problem as a Markov decision (MDP) process, discretizes the problem of continuous time steps, and expresses the state, action, state transition, reward, and action value function as follows:

[0137] Status: Park time System status for:

[0138] , (56)

[0139] , (57)

[0140] , (58)

[0141] In the formula, is the combination of price, operation and maintenance, and decay costs, is the output of the fan in period t, is the output of photovoltaic power in period t, is the collection of all loads and energy demands in the park. For the moment node The voltage, For the moment node The phase angle of M is the set of all nodes in the park. is the electricity purchase price of the park in time period t, is the electricity price of the park in time period t, is the heat purchase price of the park in time period t, is the price of hydrogen purchased by the park in time period t, is the operation and maintenance cost of the wind turbine in period t, is the operation and maintenance cost of photovoltaic power generation in period t, is the operation and maintenance cost of the electrolysis device in period t, is the operation and maintenance cost of the fuel cell in period t, is the operation and maintenance cost of the electric boiler in period t, is the operation and maintenance cost of the baseload unit in period t, is the operation and maintenance cost of the dual-operating unit in period t, is the cost of the power-off load, is the cost of the heat loss load, is the cost of cooling load, is the cost of hydrogen loss load, The battery energy storage for time period t, is the life attenuation cost of the heat storage tank in period t, is the life decay cost of the ice storage tank in period t, is the lifetime decay cost of hydrogen energy storage in period t, is the electrical load in time period t, is the heat load in period t, is the cooling load in period t, is the hydrogen load at time t;

[0142] Action: Park Moments Action for:

[0143] , (59)

[0144] , (60)

[0145] , (61)

[0146] , (62)

[0147] In the formula, It is the output collection of the park units except wind power, photovoltaic and dual-operating units. It is the collection of charging and discharging work of all energy storage in the park. is the set of all 0-1 variables, is the power consumption of the electrolysis device in time period t, is the operation and maintenance cost of the fuel cell in period t, is the power consumption of the electric boiler in time period t, is the power consumption of the base load unit in time period t, is the power consumption of the dual-operating unit in time period t, is the charging power of the battery energy storage during time period t, is the heat storage capacity of the heat storage tank in period t, is the cooling capacity of the ice storage tank in time period t, is the charging power of hydrogen energy storage at time t, is the discharge power of the battery energy storage during time period t, is the heat release of the heat storage tank, is the cooling capacity of the ice storage tank, is the hydrogen release power of hydrogen energy storage, Indicates that the energy storage battery is at time Charge, Indicates that the energy storage battery is at time Not charging, Indicates that the energy storage battery is at time Discharge, Indicates that the energy storage battery is at time No discharge, is a binary variable that indicates the energy released by the heat storage tank at the time. is a binary variable for the heat storage tank. and Both are working states of dual-mode hosts. The host is in cooling condition. The host is in ice-making condition. is a binary variable for ice storage in the ice storage tank, is a binary variable for ice melting in ice storage tanks, is a binary variable for the energy stored in the hydrogen energy storage device, A binary variable for discharging hydrogen from a hydrogen energy storage device;

[0148] State transfer: The park starts from time arrive State transition , that is, the current state of the system ,action and uncertainty Jointly determine state transition:

[0149] , (63)

[0150] In the formula, is the state transfer function;

[0151] Reward: The goal of the park's electricity, hydrogen, heat, cold storage and coordinated optimization scheduling is to minimize the total operating cost, so The reward function for:

[0152] , (64)

[0153] In the formula, is the cost of energy purchase and sale between the park and the large power grid in time period t, is the equipment operation and maintenance cost in period t, is the load loss penalty cost in time period t, is the energy storage loss cost in period t;

[0154] Optimal action value function: After observing the system state, use the optimal action value function to evaluate the system state and the quality of the action, expressed as:

[0155] , (65)

[0156] In the formula, For operational strategies, is the expression for the discounted return, is the optimal action-value function, To find the mathematical symbol for expectation, is the number of time periods, is the discount rate used to calculate the current value of future rewards, is the set of all states, is the collection of all actions.

[0157] 2.2. Park electricity, hydrogen, heat, cooling and storage coordinated optimization scheduling framework based on improved DDQN.

[0158] After establishing the MDP, the original problem is transformed into a deep reinforcement learning task. The present invention designs an improved DDQN method to solve the task.

[0159] First, the DDQN agent is based on the running state given by the environment , according to the behavioral strategy Determine the first step, that is, the action of all 0-1 variables Secondly, judge Whether the 0-1 variable constraint is satisfied, if not, the reward value is set to the minimum value Feedback to the agent to end this round; if satisfied, use the interior point method to solve the second step action and .in, The energy storage operation constraints of equations (28) to (47) need to be satisfied. If not satisfied, the variable value is taken as the maximum or minimum value to further judge Situation: When When the system power balance constraints, unit operation constraints, tie line power constraints and power flow constraints are satisfied, the reward function is calculated And feedback to the agent; if the above constraints are not met, the reward value is set to the minimum value And feedback to the agent, terminating this round. Finally, the environment will The actions of the agent and the system state given by the environment and uncertainty Input state transition function , get the system state at the new time .

[0160] Step 3: Improve the training process of the DDQN algorithm.

[0161] 3.1. Improve the offline training process of the DDQN algorithm.

[0162] In the process of coordinated optimization of electric hydrogen, heat and cold storage in the park based on deep reinforcement learning, the improved DDQN agent needs to be trained offline first. The DDQN agent uses a deep neural network to approximate function, and trains the network by combining experience replay and target network. On the one hand, experience replay stores the interaction experience between the agent and the environment in a buffer, and randomly extracts a sample from it for training in each round. This method breaks the correlation between data, improves sample utilization efficiency and training stability. On the other hand, the algorithm introduces independent online networks and target network , the target network uses different parameters from the online network and is updated at regular intervals. By separating action selection and Value evaluation, DDQN can reduce the Overestimation of value, cut off Learning bias due to bootstrapping.

[0163] In the data collection phase, the parameters of the main network and the target network and the experience replay buffer are initialized first, and then the training loop is entered. At the beginning of each round, the current state is obtained from the environment , then use -greedy behavior strategy determines the moment The first step :

[0164] , (66)

[0165] In the formula, Main Network Parameters, Indicates when an action is selected hour, As training progresses, it gradually decreases value, realizing the transformation of the DDQN algorithm from exploration to utilization.

[0166] When determining discrete variables Then, use the interior point method to solve the second step and , thereby determining the action The agent performs actions and observes rewards and the state at the next moment After that, experience playback will be the current data Stored in a buffer.

[0167] In the parameter update phase, we first randomly extract samples from the experience replay buffer. , calculate the target value:

[0168] , (67)

[0169] In the formula, is the target Q value, is the reward value, Indicates the target network Parameters before updating. Represents the discount rate used to calculate the current value of future rewards.

[0170] Secondly, the loss function is minimized by gradient descent to update the parameters of the main network. :

[0171] , (68)

[0172] , (69)

[0173] In the formula, is the loss function defined. By minimizing The Q value can be approached to the true observation value . and Represent the parameters before and after the main network is updated. Represents the learning rate, which needs to be adjusted manually.

[0174] Finally, the target network Updated parameters To update:

[0175] , (70)

[0176] 3.2. Improve the online training process of the DDQN algorithm.

[0177] After the offline training of the improved DDQN agent is completed, the obtained parameters will be directly used for the coordinated optimization scheduling of the park's electricity-hydrogen-heat-cold-storage. The input of the model is the park's photovoltaic and wind power output, the amplitude and phase angle of the node voltage, the node load and energy demand, the various parameters of the unit and energy storage, and the upper and lower limit constraints; the output of the model is the action strategy of the energy storage and unit in 24 hours a day, including the status and charging and discharging power of each energy storage, and the start and stop and output status of each unit.

[0178] The improved DDQN agent with adjusted parameters is used to train the coordinated optimization scheduling task of the electric hydrogen heat and cold storage in the park online. At this moment, the agent first observes the system state Determine the first step On this basis, the second step action is calculated using the interior point method and , thus getting the action The agent performs an action And interact with the environment, observe the reward of environmental feedback and the new state Afterwards, based on the above information This cycle repeats to obtain the optimal dispatch results of each energy storage and unit within 24 hours a day.

[0179] In summary, at this stage, a hybrid solution combining traditional optimization methods and heuristic algorithms is usually used to solve the multi-energy coordinated optimization scheduling problem of the park. However, the optimization results of the hybrid algorithm are heavily dependent on the accuracy of the prediction, and the algorithm's premise assumptions and model transformation also reduce the accuracy of the solution. If the influence of uncertainty needs to be further considered, it will bring greater challenges to the solution. In view of the above shortcomings, the present invention converts the original problem into a deep reinforcement learning task and uses the improved DDQN method to solve it. DDQN can automatically learn complex strategies from raw data, and continuously adapt and improve in a dynamic and uncertain environment. It has strong robustness, but it is mostly used in discrete action spaces and is not suitable for the scenario of the present invention. The present invention makes innovative improvements on the basis of the original DDQN, and decomposes the action of the intelligent agent into two steps for calculation: through the exploration and utilization of 0-1 variables in the calculation model by DDQN, on the basis of determining discrete variables, the continuous variables in the calculation model are calculated using the interior point method. The redesign of DDQN enables it to effectively handle high-dimensional continuous action spaces and better fit the problems of the present invention.

[0180] The method of this embodiment has the following beneficial effects: 1. Taking into full consideration the multiple load requirements of electricity-hydrogen-heat-cold-storage, a large-scale park system including fans, photovoltaics, electrolysis devices, fuel cells, electric boilers, ice storage systems, electrochemical energy storage, thermal energy storage and hydrogen storage devices is designed for the first time. The system fits the actual situation of large parks and realizes energy autonomy, multi-source complementarity and economic operation within the park.

[0181] 2. Propose a comprehensive energy optimization scheduling method for the park. This method takes the minimum energy cost of the park as the goal, fully considers the comprehensive optimization scheduling of all energy forms in the park, such as electricity, hydrogen, heat, cold, and storage, and realizes the economic, stable and safe operation of the multi-energy coupling park system.

[0182] 3. A DRL-based coordinated optimization dispatching method for electric hydrogen, heat, cold storage in the park is proposed, which automatically learns optimization decisions from historical data and realizes real-time online adjustment of the park's electric multi-energy dispatching. This method can cope well with uncertain and dynamic complex scenarios and has the advantages of high robustness and high fitting.

[0183] 4. Improve and innovate on the original DDQN algorithm, and determine the decision-making actions of the DDQN agent in steps. First, determine the actions of 0-1 variables through the exploration and utilization of DDQN, and then use the interior point method to calculate the actions of continuous variables. The improved DDQN method proposed in this invention can better handle the continuous and high-dimensional state and action space of the park.

[0184] 5. An optimization scheduling framework based on the improved DDQN algorithm was designed, and the optimal scheduling instructions were generated through offline and online training.

[0185] See also Figure 2 , which shows a structural block diagram of a park electric hydrogen heat and cold storage coordinated optimization scheduling system of the present application.

[0186] like Figure 2 As shown, the park electric hydrogen heat and cold storage coordinated optimization scheduling system 200 includes a construction module 210, a conversion module 220, a first judgment module 230, a second judgment module 240, a value taking module 250, a third judgment module 260, a first feedback module 270, a second feedback module 280 and an output module 290.

[0187] The construction module 210 is configured to construct a park multi-energy scheduling physical model with the minimum total operating cost as the objective function under preset constraints, wherein the constraints include system power balance constraints, unit operation constraints, energy storage operation constraints, tie line power constraints, and power flow constraints;

[0188] A conversion module 220, configured to convert the park multi-energy scheduling physical model into an improved DDQN agent, wherein the improved DDQN agent includes an online network and a target network;

[0189] The first judgment module 230 is configured to improve the running state of the DDQN agent based on the environment , according to the behavioral strategy Determine the first step, that is, the action of all 0-1 variables , and judge Whether the 0-1 variable constraint is satisfied;

[0190] The second judgment module 240 is configured to If the 0-1 variable constraint is not met, the reward value is set to the minimum value Feedback to the improved DDQN agent, ending this round, if If the 0-1 variable constraint is satisfied, the interior point method is used to solve the second step action and , and judge Whether the energy storage operation constraints are met;

[0191] The value acquisition module 250 is configured as follows: If the energy storage operation constraints are met, then When the upper limit is exceeded, the upper limit value is taken. When the lower limit is exceeded, the lower limit value is taken;

[0192] The third judgment module 260 is configured to If the energy storage operation constraints are not met, then Whether the system power balance constraints, unit operation constraints, interconnection line power constraints and power flow constraints are met;

[0193] The first feedback module 270 is configured to If the system power balance constraints, unit operation constraints, tie line power constraints and power flow constraints are met, the reward function is calculated And feedback to improve the DDQN agent;

[0194] The second feedback module 280 is configured as follows: If the system power balance constraint, unit operation constraint, tie line power constraint and power flow constraint are not met, the reward value is set to the minimum value. And feedback to the agent to terminate this round;

[0195] Output module 290, configured to output the current time The actions of the agent and the system state given by the environment and uncertainty Input state transition function , get the system state at the new time .

[0196] It should be understood that Figure 2 Modules and references documented in Figure 1 Therefore, the operations and features described above for the method and the corresponding technical effects are also applicable to Figure 2 The modules in it will not be described in detail here.

[0197] A computer program, wherein when the program instructions are executed by a processor, the processor is caused to execute the coordinated optimization scheduling method for electric hydrogen, heat and cold storage in a park in any of the above method embodiments;

[0198] As an implementation mode, the computer-readable storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are configured as follows:

[0199] Under the preset constraints, a multi-energy dispatch physical model of the park is constructed with the minimum total operating cost as the objective function. The constraints include system power balance constraints, unit operation constraints, energy storage operation constraints, interconnection line power constraints, and power flow constraints.

[0200] Converting the park multi-energy scheduling physical model into an improved DDQN agent, wherein the improved DDQN agent includes an online network and a target network;

[0201] Improve the running state of DDQN agent based on the environment , according to the behavioral strategy Determine the first step, that is, the action of all 0-1 variables , and judge Whether the 0-1 variable constraint is satisfied;

[0202] like If the 0-1 variable constraint is not met, the reward value is set to the minimum value Feedback to the improved DDQN agent, ending this round, if If the 0-1 variable constraint is satisfied, the interior point method is used to solve the second step action and , and judge Whether the energy storage operation constraints are met;

[0203] like If the energy storage operation constraints are met, then When the upper limit is exceeded, the upper limit value is taken. When the lower limit is exceeded, the lower limit value is taken;

[0204] like If the energy storage operation constraints are not met, then Whether the system power balance constraints, unit operation constraints, interconnection line power constraints and power flow constraints are met;

[0205] like If the system power balance constraints, unit operation constraints, tie line power constraints and power flow constraints are met, the reward function is calculated And feedback to improve the DDQN agent;

[0206] like If the system power balance constraint, unit operation constraint, tie line power constraint and power flow constraint are not met, the reward value is set to the minimum value. And feedback to the agent to terminate this round;

[0207] The current moment The actions of the agent and the system state given by the environment and uncertainty Input state transition function , get the system state at the new time .

[0208] The computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the park's electric hydrogen heat cold storage coordinated optimization scheduling system, etc. In addition, the computer-readable storage medium may include a high-speed random access memory, and may also include a memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the computer-readable storage medium may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the park's electric hydrogen heat cold storage coordinated optimization scheduling system via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0209] Figure 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 3 As shown, the device includes: a processor 310 and a memory 320. The electronic device may also include: an input device 330 and an output device 340. The processor 310, the memory 320, the input device 330 and the output device 340 may be connected via a bus or other means. Figure 3 The example of connection via bus is taken. The memory 320 is the above-mentioned computer-readable storage medium. The processor 310 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 320, that is, implements the above-mentioned method embodiment of the park electric hydrogen heat cold storage coordinated optimization scheduling method. The input device 330 can receive input digital or character information, and generate key signal input related to user settings and function control of the park electric hydrogen heat cold storage coordinated optimization scheduling system. The output device 340 may include display devices such as display screens.

[0210] The electronic device can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0211] As an implementation mode, the electronic device is applied to a park electric hydrogen heat cold storage coordinated optimization scheduling system, and is used for a client, and includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can:

[0212] Under the preset constraints, a multi-energy dispatch physical model of the park is constructed with the minimum total operating cost as the objective function. The constraints include system power balance constraints, unit operation constraints, energy storage operation constraints, interconnection line power constraints, and power flow constraints.

[0213] Converting the park multi-energy scheduling physical model into an improved DDQN agent, wherein the improved DDQN agent includes an online network and a target network;

[0214] Improve the running state of DDQN agent based on the environment , according to the behavioral strategy Determine the first step, that is, the action of all 0-1 variables , and judge Whether the 0-1 variable constraint is satisfied;

[0215] like If the 0-1 variable constraint is not met, the reward value is set to the minimum value Feedback to the improved DDQN agent, ending this round, if If the 0-1 variable constraint is satisfied, the interior point method is used to solve the second step action and , and judge Whether the energy storage operation constraints are met;

[0216] like If the energy storage operation constraints are met, then When the upper limit is exceeded, the upper limit value is taken. When the lower limit is exceeded, the lower limit value is taken;

[0217] like If the energy storage operation constraints are not met, then Whether the system power balance constraints, unit operation constraints, interconnection line power constraints and power flow constraints are met;

[0218] like If the system power balance constraints, unit operation constraints, tie line power constraints and power flow constraints are met, the reward function is calculated And feedback to improve the DDQN agent;

[0219] like If the system power balance constraint, unit operation constraint, tie line power constraint and power flow constraint are not met, the reward value is set to the minimum value. And feedback to the agent to terminate this round;

[0220] The current moment The actions of the agent and the system state given by the environment and uncertainty Input state transition function , get the system state at the new time .

[0221] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0222] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A coordinated optimization scheduling method for electric hydrogen heat and cold storage in a park, characterized in that: include: Under the preset constraints, a multi-energy dispatch physical model of the park is constructed with the minimum total operating cost as the objective function. The constraints include system power balance constraints, unit operation constraints, energy storage operation constraints, interconnection line power constraints, and power flow constraints. The total operating cost includes the energy purchase and sale costs between the park and the power grid, equipment operation and maintenance costs, load loss penalty costs, and energy storage loss costs. The expression of the objective function is: ， In the formula, is the cost of energy purchase and sale between the park and the large power grid at time t, is the equipment operation and maintenance cost at time t, is the load loss penalty cost at time t, is the energy storage loss cost at time t; ， In the formula, is the electricity purchase price of the park at time t, is the power purchased by the park at time t, is the heat purchase price of the park at time t, is the amount of heat purchased by the park at time t, is the price of hydrogen purchased by the park at time t, is the hydrogen purchase power of the park at time t, is the electricity price of the park at time t, is the electricity sales power of the park at time t, is the time period; ， In the formula, is the operation and maintenance cost of the wind turbine at time t, is the output of the fan at time t, is the operation and maintenance cost of photovoltaic at time t, is the output of photovoltaic at time t, is the operation and maintenance cost of the electrolysis device at time t, is the power consumption of the electrolysis device at time t, is the operation and maintenance cost of the fuel cell at time t, is the electrical power output of the fuel cell at time t, is the operation and maintenance cost of the electric boiler at time t, is the power consumption of the electric boiler at time t, is the operation and maintenance cost of the baseload unit at time t, is the power consumption of the base load unit at time t, is the operation and maintenance cost of the dual-operating unit at time t, is the power consumption of the dual-operating unit at time t; ， In the formula, is the cost of the power-off load, For power failure load, is the cost of the heat loss load, is the heat loss load, is the cost of cooling load, is the cooling load, is the cost of the lost hydrogen load, is the hydrogen loss load; ， In the formula, is the battery energy stored at time t, is the charging power of the battery energy storage at time t, is the life attenuation cost of the heat storage tank at time t, is the heat storage capacity of the heat storage tank, is the life decay cost of the ice storage tank at time t, is the cold storage capacity of the ice storage tank, is the lifetime decay cost of hydrogen energy storage at time t, The charging power of hydrogen energy storage, is the discharge power of the battery energy storage at time t, is the heat release of the heat storage tank, is the cooling capacity of the ice storage tank, The hydrogen release power for hydrogen energy storage; The multi-energy scheduling physical model of the park is converted into an improved DDQN agent, wherein the improved DDQN agent includes an online network and a target network, wherein the multi-energy scheduling physical model of the park is converted into an improved DDQN agent including: Describe the optimization scheduling problem as a Markov decision process, discretize the problem of continuous time steps, and express the state, action, state transition, reward, and action value function as: Status: Park time System status for: ， ， ， In the formula, is the combination of price, operation and maintenance, and decay costs, is the output of the fan in period t, is the output of photovoltaic power in period t, is the collection of all loads and energy demands in the park. For the moment node The voltage, For the moment node The phase angle of M is the set of all nodes in the park. is the electricity purchase price of the park in time period t, is the electricity price of the park in time period t, is the heat purchase price of the park in time period t, is the price of hydrogen purchased by the park in period t, is the operation and maintenance cost of the wind turbine in period t, is the operation and maintenance cost of photovoltaic power generation in period t, is the operation and maintenance cost of the electrolysis device in period t, is the operation and maintenance cost of the fuel cell in period t, is the operation and maintenance cost of the electric boiler in period t, is the operation and maintenance cost of the baseload unit in period t, is the operation and maintenance cost of the dual-operating unit in period t, is the cost of the power-off load, is the cost of the heat loss load, is the cost of cooling load, is the cost of the lost hydrogen load, The battery energy storage for time period t, is the life attenuation cost of the heat storage tank in period t, is the life decay cost of the ice storage tank in period t, is the lifetime decay cost of hydrogen energy storage in period t, is the electrical load in time period t, is the heat load in period t, is the cooling load in period t, is the hydrogen load at time t; Action: Park Moments Action for: ， ， ， ， In the formula, It is the output collection of the park units except wind power, photovoltaic and dual-operating units. It is the collection of charging and discharging work of all energy storage in the park. is the set of all 0-1 variables, is the power consumption of the electrolysis device in time period t, is the operation and maintenance cost of the fuel cell in period t, is the power consumption of the electric boiler in time period t, is the power consumption of the base load unit in time period t, is the power consumption of the dual-operating unit in time period t, is the charging power of the battery energy storage during time period t, is the heat storage capacity of the heat storage tank in period t, is the cooling capacity of the ice storage tank in time period t, is the charging power of hydrogen energy storage at time t, is the discharge power of the battery energy storage during time period t, is the heat release of the heat storage tank, is the cooling capacity of the ice storage tank, is the hydrogen release power of hydrogen energy storage, Indicates that the energy storage battery is at time Charge, Indicates that the energy storage battery is at time Not charging, Indicates that the energy storage battery is at time Discharge, Indicates that the energy storage battery is at time No discharge, is a binary variable that indicates the energy released by the heat storage tank at the time. is a binary variable for the heat storage tank. and Both are working states of dual-mode hosts. The host is in cooling condition. The host is in ice-making condition. is a binary variable for ice storage in the ice storage tank, is a binary variable for ice melting in ice storage tanks, is a binary variable for the energy stored in the hydrogen energy storage device, A binary variable for discharging hydrogen from a hydrogen energy storage device; State transfer: The park starts from time arrive State transition , that is, the current state of the system ,action and uncertainty Jointly determine state transition: ， In the formula, is the state transfer function; Reward: The goal of a park's electricity, hydrogen, heat, and cooling storage coordinated optimization scheduling is to minimize the total operating cost, so The reward function for: ， In the formula, is the cost of energy purchase and sale between the park and the large power grid in time period t, is the equipment operation and maintenance cost in period t, is the load loss penalty cost in time period t, is the energy storage loss cost in period t; Optimal action value function: After observing the system state, the optimal action value function is used to evaluate the state of the system and the quality of the action, which is expressed as: ， In the formula, For operational strategies, is the expression for the discounted return, is the optimal action-value function, To find the mathematical symbol for expectation, is the number of time periods, is the discount rate used to calculate the current value of future rewards, is the set of all states, is the set of all actions; Improve the running state of DDQN agent based on the environment , according to the behavioral strategy Determine the first step, that is, the action of all 0-1 variables , and judge Whether the 0-1 variable constraint is satisfied; like If the 0-1 variable constraint is not met, the reward value is set to the minimum value Feedback to the improved DDQN agent, ending this round, if If the 0-1 variable constraint is satisfied, the interior point method is used to solve the second step action and , and judge Whether the energy storage operation constraints are met, where: It is the collection of charging and discharging work of all energy storage in the park. It is the output aggregation of the park units except wind power, photovoltaic and dual-operating units; like If the energy storage operation constraints are met, then When the upper limit is exceeded, the upper limit value is taken. When the lower limit is exceeded, the lower limit value is taken; like If the energy storage operation constraints are not met, then Whether the system power balance constraints, unit operation constraints, interconnection line power constraints and power flow constraints are met; like If the system power balance constraints, unit operation constraints, tie line power constraints and power flow constraints are met, the reward function is calculated And feedback to improve the DDQN agent; like If the system power balance constraint, unit operation constraint, tie line power constraint and power flow constraint are not met, the reward value is set to the minimum value. And feedback to the agent to terminate this round; The current moment The actions of the agent and the system state given by the environment and uncertainty Input state transition function , get the system state at the new time .

2. A method for coordinated optimization scheduling of electric hydrogen heat and cold storage in a park according to claim 1, characterized in that: The constraints include system power balance constraints, unit operation constraints, energy storage operation constraints, tie line power constraints and power flow constraints; The expression of the system power balance constraint is: ， In the formula, is the output of the fan at time t, is the output of photovoltaic at time t, is the discharge power of the battery energy storage at time t, is the charging power of the battery energy storage at time t, is the hydrogen production power of the electrolysis device at time t, is the electrical power output of the fuel cell at time t, is the power consumption of the electric boiler at time t, is the hydrogen consumption power of the fuel cell at time t, is the power consumption of the dual-operating unit at time t, is the charging power of the battery energy storage at time t, is the electricity sales power at time t, is the electrical load at time t, is the power failure load of the park at time t; ， In the formula, is the heat release of the heat storage tank at time t, is the heat storage capacity of the heat storage tank at time t, is the heating capacity of the electric boiler at time t, is the power purchased and sold at time t, is the heat load at time t, is the heat loss load at time t; ， In the formula, is the cooling capacity of the ice storage tank at time t, is the cooling capacity of the ice storage tank at time t, is the cooling capacity of the base load unit, is the cooling capacity of the dual-mode unit, is the cooling load at time t, is the cooling load at time t; ， In the formula, is the hydrogen discharge power of hydrogen energy storage at time t, is the charging power of hydrogen energy storage at time t, is the hydrogen production power of the electrolysis device at time t, is the hydrogen consumption power of the fuel cell at time t, is the hydrogen purchase power at time t, is the hydrogen load at time t, is the hydrogen loss load at time t; The expression of the unit operation constraint is: ， ， In the formula, is the hydrogen production power of the electrolysis device at time t, is the power consumption of the electrolytic cell at time t, is the hydrogen production efficiency of the electrolyzer, is the upper limit of the power consumption of the electrolytic cell; ， ， In the formula, is the power consumption of the fuel cell at time t, is the hydrogen consumption power of the fuel cell at time t, is the power generation efficiency of the fuel cell, is the upper limit of fuel cell power consumption; ， ， In the formula, is the power consumption of the electric boiler at time t, is the heating capacity of the electric boiler at time t, For the moment The working temperature of the electric boiler, is the thermal resistance of the electric boiler, The thermal resistance of the electric boiler. is the time interval between two moments, For the moment +1 The working temperature of the electric boiler, For the moment The ambient temperature, The upper limit of the heat generated by the electric boiler; ， ， ， ， In the formula, is the cooling capacity of the base load unit, is the power consumption of the base load unit, For the base crew at time The load factor, , , These are the unit parameters of the base load unit. is the rated cooling capacity of the base load unit, The upper limit of the cooling capacity generated by the base load unit, is the lower limit of the load factor of the base load unit; ， ， ， ， ， ， ， ， In the formula, Indicates that the dual-mode unit is in cooling mode. The cooling capacity generated by the dual-mode unit in refrigeration mode, Indicates that the dual-mode unit is in ice-making mode. The cooling capacity generated by the dual-mode unit in ice-making mode. is the load rate of the dual-operating unit, , , These are the unit parameters of dual-operating units. is the power consumption of the dual-operating unit, is the rated cooling capacity of the dual-mode unit, is the load rate of the dual-mode unit in cooling condition, is the load rate of the dual-mode unit in ice-making condition, Indicates that the dual-mode unit is in ice-making mode. The upper limit of the cooling capacity of the dual-mode unit. The upper limit of ice production in dual-mode units. is the lower limit of the refrigeration load rate of the dual-mode unit. It is the lower limit of the load rate of the dual-mode unit in ice-making condition; The expression of the energy storage operation constraint is: ， ， ， ， ， In the formula, Indicates that the energy storage battery is at time Charge, Indicates that the energy storage battery is at time Not charging, is the maximum charging power of the energy storage converter, is the maximum discharge power of the energy storage converter, Indicates that the energy storage battery is at time Discharge, Indicates that the energy storage battery is not Discharge, is the maximum discharge power of the energy storage converter, For energy storage batteries at all times The capacity, For energy storage batteries at all times -1 capacity, For the charging efficiency of the energy storage battery, The energy storage converter is at time The charging power, The energy storage converter is at time The discharge power, is the energy discharge efficiency of the energy storage battery, is the maximum capacity of the energy storage battery, is the minimum state of charge of the energy storage battery, is the maximum state of charge of the energy storage battery; ， ， ， ， ， In the formula, is the charging power of the heat storage tank at time t, is the binary variable of the heat storage tank being charged at time t, is the maximum heating power of the heat storage tank, is the heat release power of the heat storage tank at time t, is a binary variable that indicates the energy released by the heat storage tank at the time. is the maximum heat release power of the heat storage tank, For the heat storage tank at time The capacity, For the heat storage tank at time -1 capacity, is the self-loss coefficient of the heat storage tank, is the charging efficiency of the heat storage tank, is the energy release efficiency of the heat storage tank, is the time interval between two moments, is the upper limit of the capacity of the heat storage tank; ， ， ， ， ， ， ， In the formula, is the cooling power of the ice storage tank at time t, is the binary variable of ice storage tank charging at time t, is the maximum cooling power of the ice storage tank, is the cooling power of the ice storage tank at time t, is the binary variable of the energy released by the ice storage tank at time t, is the maximum cooling power of the ice storage tank, For the heat storage tank at time The capacity, For the heat storage tank at time The capacity, is the self-loss coefficient of the ice storage tank, is the charging efficiency of the ice storage tank, is the energy release efficiency of the ice storage tank, is the upper limit of the capacity of the heat storage tank, The charging power of hydrogen energy storage, is a binary variable for charging the hydrogen storage at time t, is the maximum charging power of hydrogen energy storage, is the hydrogen release power of hydrogen energy storage, is a binary variable for hydrogen energy storage to release energy at time t, is the maximum hydrogen release power of hydrogen energy storage; ， ， ， In the formula, For hydrogen storage at all times The capacity, For hydrogen storage at all times The capacity, The charging efficiency of hydrogen energy storage, is the energy release efficiency of hydrogen energy storage, is the upper limit of the capacity of the hydrogen storage tank; The expression of the tie line power constraint is: ， ， In the formula, is the maximum electric power allowed to be exchanged between the microgrid and the main grid tie line, is the maximum hydrogen power allowed to be exchanged between the microgrid and the main grid tie line, is the hydrogen purchase power at time t, is the charging power of the battery energy storage at time t, is the electricity sales power at time t; The expression of the power flow constraint is: ， ， ， ， ， In the formula, For the moment node The voltage, For the moment node The voltage, For the node All connected nodes, For line The conductivity, For the moment node With Node The phase angle, For line The electrical susceptance, For the moment With Node The active power delivered by the connected energy source, For the moment node The active load, For the moment With Node Reactive power from connected energy sources, For the moment node The reactive load, For Node The voltage lower limit, For Node The upper voltage limit, For Node The lower limit of the phase angle, For Node The upper limit of the phase angle, For line The upper limit of active power allowed to flow.

3. A park electricity, hydrogen, heat and cold storage coordinated optimization scheduling system, characterized in that: include: A construction module is configured to construct a park multi-energy dispatch physical model under preset constraints with the minimum total operating cost as the objective function, wherein the constraints include system power balance constraints, unit operation constraints, energy storage operation constraints, interconnection line power constraints, and power flow constraints, wherein the total operating cost includes the cost of purchasing and selling energy between the park and the power grid, equipment operation and maintenance costs, load loss penalty costs, and energy storage loss costs; The expression of the objective function is: ， In the formula, is the cost of energy purchase and sale between the park and the large power grid at time t, is the equipment operation and maintenance cost at time t, is the load loss penalty cost at time t, is the energy storage loss cost at time t; ， In the formula, is the electricity purchase price of the park at time t, is the power purchased by the park at time t, is the heat purchase price of the park at time t, is the amount of heat purchased by the park at time t, is the price of hydrogen purchased by the park at time t, is the hydrogen purchase power of the park at time t, is the electricity price of the park at time t, is the electricity sales power of the park at time t, is the time period; ， In the formula, is the operation and maintenance cost of the wind turbine at time t, is the output of the fan at time t, is the operation and maintenance cost of photovoltaic at time t, is the output of photovoltaic at time t, is the operation and maintenance cost of the electrolysis device at time t, is the power consumption of the electrolysis device at time t, is the operation and maintenance cost of the fuel cell at time t, is the electrical power output of the fuel cell at time t, is the operation and maintenance cost of the electric boiler at time t, is the power consumption of the electric boiler at time t, is the operation and maintenance cost of the baseload unit at time t, is the power consumption of the base load unit at time t, is the operation and maintenance cost of the dual-operating unit at time t, is the power consumption of the dual-operating unit at time t; ， In the formula, is the cost of the power-off load, For power failure load, is the cost of the heat loss load, is the heat loss load, is the cost of cooling load, is the cooling load, is the cost of the lost hydrogen load, is the hydrogen loss load; ， In the formula, is the battery energy stored at time t, is the charging power of the battery energy storage at time t, is the life attenuation cost of the heat storage tank at time t, is the heat storage capacity of the heat storage tank, is the life decay cost of the ice storage tank at time t, is the cold storage capacity of the ice storage tank, is the lifetime decay cost of hydrogen energy storage at time t, The charging power of hydrogen energy storage, is the discharge power of the battery energy storage at time t, is the heat release of the heat storage tank, is the cooling capacity of the ice storage tank, The hydrogen release power for hydrogen energy storage; A conversion module is configured to convert the park multi-energy scheduling physical model into an improved DDQN agent, wherein the improved DDQN agent includes an online network and a target network, wherein the conversion of the park multi-energy scheduling physical model into the improved DDQN agent includes: Describe the optimization scheduling problem as a Markov decision process, discretize the problem of continuous time steps, and express the state, action, state transition, reward, and action value function as: Status: Park time System status for: ， ， ， In the formula, is the combination of price, operation and maintenance, and decay costs, is the output of the fan in period t, is the output of photovoltaic power in period t, is the collection of all loads and energy demands in the park. For the moment The voltage at the node, For the moment node The phase angle of M is the set of all nodes in the park. is the electricity purchase price of the park in time period t, is the electricity price of the park in time period t, is the heat purchase price of the park in time period t, is the price of hydrogen purchased by the park in period t, is the operation and maintenance cost of the wind turbine in period t, is the operation and maintenance cost of photovoltaic power generation in period t, is the operation and maintenance cost of the electrolysis device in period t, is the operation and maintenance cost of the fuel cell in period t, is the operation and maintenance cost of the electric boiler in period t, is the operation and maintenance cost of the baseload unit in period t, is the operation and maintenance cost of the dual-operating unit in period t, is the cost of the power-off load, is the cost of the heat loss load, is the cost of cooling load, is the cost of the lost hydrogen load, The battery energy storage for time period t, is the life attenuation cost of the heat storage tank in period t, is the life decay cost of the ice storage tank in period t, is the lifetime decay cost of hydrogen energy storage in period t, is the electrical load in time period t, is the heat load in period t, is the cooling load in period t, is the hydrogen load at time t; Action: Park Moments Action for: ， ， ， ， In the formula, It is the output collection of the park units except wind power, photovoltaic and dual-operating units. It is the collection of charging and discharging work of all energy storage in the park. is the set of all 0-1 variables, is the power consumption of the electrolysis device in time period t, is the operation and maintenance cost of the fuel cell in period t, is the power consumption of the electric boiler in time period t, is the power consumption of the base load unit in time period t, is the power consumption of the dual-operating unit in time period t, is the charging power of the battery energy storage during time period t, is the heat storage capacity of the heat storage tank in period t, is the cooling capacity of the ice storage tank in time period t, is the charging power of hydrogen energy storage at time t, is the discharge power of the battery energy storage during time period t, is the heat release of the heat storage tank, is the cooling capacity of the ice storage tank, is the hydrogen release power of hydrogen energy storage, Indicates that the energy storage battery is at time Charge, Indicates that the energy storage battery is at time Not charging, Indicates that the energy storage battery is at time Discharge, Indicates that the energy storage battery is at time No discharge, is a binary variable that indicates the energy released by the heat storage tank at the time. is a binary variable for the heat storage tank. and Both are working states of dual-mode hosts. The host is in cooling condition. The host is in ice-making condition. is a binary variable for ice storage in the ice storage tank, is a binary variable for ice melting in ice storage tanks, is a binary variable for the energy stored in the hydrogen energy storage device, A binary variable for discharging hydrogen from a hydrogen energy storage device; State transfer: The park starts from time arrive State transition , that is, the current state of the system ,action and uncertainty Jointly determine state transition: ， In the formula, is the state transfer function; Reward: The goal of a park's electricity, hydrogen, heat, and cooling storage coordinated optimization scheduling is to minimize the total operating cost, so The reward function for: ， In the formula, is the cost of energy purchase and sale between the park and the large power grid in time period t, is the equipment operation and maintenance cost in period t, is the load loss penalty cost in time period t, is the energy storage loss cost in period t; Optimal action value function: After observing the system state, the optimal action value function is used to evaluate the state of the system and the quality of the action, which is expressed as: ， In the formula, For operational strategies, is the expression for the discounted return, is the optimal action-value function, To find the mathematical symbol for expectation, is the number of time periods, is the discount rate used to calculate the current value of future rewards, is the set of all states, is the set of all actions; The first judgment module is configured to improve the running state of the DDQN agent based on the environment , according to the behavioral strategy Determine the first step, that is, the action of all 0-1 variables , and judge Whether the 0-1 variable constraint is satisfied; The second judgment module is configured as follows: If the 0-1 variable constraint is not met, the reward value is set to the minimum value Feedback to the improved DDQN agent, ending this round, if If the 0-1 variable constraint is satisfied, the interior point method is used to solve the second step action and , and judge Whether the energy storage operation constraints are met; The value module is configured as follows: If the energy storage operation constraints are met, then When the upper limit is exceeded, the upper limit value is taken. When the lower limit is exceeded, the lower limit value is taken; The third judgment module is configured as follows: If the energy storage operation constraints are not met, then Whether the system power balance constraints, unit operation constraints, interconnection line power constraints and power flow constraints are met; The first feedback module is configured as follows: If the system power balance constraints, unit operation constraints, tie line power constraints and power flow constraints are met, the reward function is calculated And feedback to improve the DDQN agent; The second feedback module is configured as follows: If the system power balance constraint, unit operation constraint, tie line power constraint and power flow constraint are not met, the reward value is set to the minimum value. And feedback to the agent to terminate this round; Output module, configured to output the current time The actions of the agent and the system state given by the environment and uncertainty Input state transition function , get the system state at the new time .

4. An electronic device, characterized in that: include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in any one of claims 1 to 2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 2 is implemented.

Images