Electricity-hydrogen complementary multi-building energy supply system in a park and its coordinated scheduling method
Through electro-hydrogen complementarity and deep reinforcement learning, low-carbon energy-saving operation in smart parks under the carbon neutrality goal is achieved, solving the problems of traditional electric energy storage safety and centralized scheduling and computing burden, optimizing energy utilization and cost, and enhancing Resilience and resilience of the park.
Patent Information
- Application Number
- CN202111533241.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-15
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-12-15
AI Technical Summary
Traditional electric energy storage methods have the risk of spontaneous combustion in places with dense crowds or high safety requirements. The centralized scheduling method has heavy calculation burden and high privacy risks, making it difficult to effectively allocate renewable energy, and requires a safe, flexible and efficient energy. Management plan.
The multi-building energy supply system in the park is adopted with electricity and hydrogen complementary. Through the MADDPG algorithm and deep reinforcement learning method, electricity is shared between buildings, hydrogen-containing energy storage systems are configured, and the agent is dispatched based on real-time data to realize distributed coordinated scheduling and optimize energy. Utilization and cost.
The low-carbon energy-saving operation of smart parks under the carbon neutrality goal was achieved, the user's energy consumption cost was optimized, the park's resilience and resilience were enhanced, and the dimensional disasters and suboptimal actions were avoided, which verified the effectiveness and advantages of the method.
Smart Images

Figure CN114331059B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an energy management technology, and in particular to a multi-building energy supply system in a park considering the complementarity of electricity and hydrogen under the dual carbon goals and a coordinated scheduling method thereof. Background Art
[0002] With the continuous innovation and development of renewable energy power generation technology and the substantial reduction in costs, its penetration rate is gradually increasing, which may bring intermittent and reliability problems to the power grid, so new solutions are needed to flexibly allocate resources in the system. Improving energy utilization through the conversion and storage of multiple energy sources is the current mainstream research direction, among which the role of energy storage is particularly important. However, traditional electrical energy storage methods, such as batteries and supercapacitors, have the risk of spontaneous combustion and are not suitable for installation in places with dense traffic or high safety requirements. Hydrogen energy storage technology is not only safe and stable, but also has almost no carbon emissions in the energy conversion process, which can effectively solve the bottleneck problem of "carbon neutrality" development.
[0003] As a terminal application form of the power system, using smart buildings as modular units to build a smart park energy supply system from the bottom up can not only realize the flexible scheduling of the power system, but also enhance the "resilience" of the smart park, so that the park has stronger defense and recovery capabilities when facing extreme scenarios. However, buildings are located at the bottom of the system, with small capacity and different energy consumption behaviors. It is necessary to install hydrogen energy storage equipment in the buildings to improve the controllability, coordination and local consumption of renewable energy. In real scenarios, buildings are often distributed in clusters. Coordinating the interaction of power between buildings can not only realize the synergy of internal resources and the local consumption of renewable energy, but also play a transitional role in the interaction of power between the distribution system and users.
[0004] Most of the existing research focuses on centralized scheduling methods, which require frequent information interaction, have a heavy computational burden, and have the potential risk of exposing user privacy. Distributed optimization scheduling can decompose the problem into multiple sub-problems that can be solved independently, and only a small amount of information interaction is needed to achieve the global optimum. In recent years, deep reinforcement learning methods have the ability to make decisions with high speed and accuracy, and have performed well in aspects such as building energy consumption assessment, power equipment scheduling, demand side management, and automatic voltage control. They are very suitable for smart parks with high real-time and autonomy requirements and are insensitive to voltage fluctuations. Applying deep reinforcement learning methods to distributed coordinated scheduling of smart parks, the intelligent agent can quickly determine the scheduling plan in actual operation after offline training using only a small amount of local state information. Summary of the invention
[0005] In response to the problem of establishing smart parks, a multi-building energy supply system with electric and hydrogen complementarity and its coordinated scheduling method were proposed. By utilizing the complementary characteristics of electric and hydrogen and sharing of electricity among buildings, the operating structure of a carbon-neutral smart park was constructed. With the support of the MADDPG (multi-agent reinforcement learning) algorithm, each building can achieve overall optimization in the self-optimization process, avoiding the curse of dimensionality and the selection of suboptimal actions.
[0006] The technical solution of the present invention is: an electric-hydrogen complementary multi-building energy supply system in a park, in which each building in the park belongs to a different entity, and a hydrogen energy storage system is configured for buildings that cannot be powered off; all buildings together constitute a whole, and the buildings are connected by a bus, and there is power sharing between the buildings; each building is assigned an intelligent agent, and there is a communication line between the intelligent agent and the controller of the building equipment, which is used to obtain real-time status data measured by sensors; at the same time, the real-time status data is stored by the intelligent agent to which it belongs, and transmitted to the intelligent agents of other buildings via the communication line at regular intervals for offline learning of the intelligent agent; the intelligent agent determines the optimal reference value of the controllable components inside the building online based on the real-time data and transmits it to the controller of the corresponding component, and the controller controls the device to track the optimal reference value by controlling the inverter and the intelligent switch.
[0007] Furthermore, the park consisting of the buildings is connected to the distribution network through a public connection point controlled by a park manager, and each building is equipped with photovoltaic panels, wind turbines, diesel generators and hydrogen energy storage systems of different capacities according to its own load.
[0008] Furthermore, the coordinated scheduling method of the electric-hydrogen complementary park multi-building energy supply system specifically includes the following steps:
[0009] 1) According to the actual park project, analyze the operation mechanism of energy flow, information flow and control flow within the park, classify them according to the reliability and economic requirements of different buildings, and establish a multi-building energy supply system in the park with hydrogen energy storage and electric hydrogen complementation;
[0010] 2) Model the energy and load in the power supply system separately. For the four links of hydrogen energy storage system, namely electrolysis, hydrogenation, dehydrogenation and combustion, the operation model and the load state function of the hydrogen storage tank are established respectively, and they are applied as constraints in the scheduling model of the multi-building energy supply system in the park with complementary electricity and hydrogen;
[0011] 3) Establish an interactive mechanism for interconnecting multiple buildings in the multi-building energy supply system of the park. With the support of multi-agent reinforcement learning algorithm, based on the historical data of the buildings, the real-time energy supply scheduling of the park is realized in the way of "offline centralized learning and online distributed execution".
[0012] Further, the step 2) specifically includes the following steps:
[0013] 2.1) Model the energy and loads in the multi-building operating environment of the established carbon-neutral campus;
[0014] 2.2) Form a scheduling strategy based on the supply and demand of the park:
[0015] 2.2.1) After each building generates electricity to meet its own load demand, there is surplus power: If there is surplus power from renewable energy output at this time, the hydrogen energy storage system will be used to store the surplus energy. If there is still surplus power, it will be sold to the distribution network to obtain revenue;
[0016] 2.2.2) The power generation of each building cannot meet its own load demand: At this time, the energy stored in the building’s own hydrogen energy storage system is used first until the capacity limit of the hydrogen storage tank or the maximum discharge power of the fuel cell is reached;
[0017] 2.2.3) After some buildings have met their own load demand, there is surplus power, while some buildings cannot meet their own load demand: Adhering to the principle of local consumption of renewable energy power generation, buildings with power shortages shall purchase electricity from buildings with surplus power first. When the surplus power of the former is less than the power shortage of the latter, the surplus power shall be supplied by their own energy storage batteries until the lower limit of the energy storage battery capacity. If it still cannot meet the power demand, the surplus power shall be obtained from the distribution network; when the surplus power of the former is greater than the load demand of the latter, the surplus power during peak hours shall be sold to the distribution network, or the surplus power during normal and valley hours shall be charged to the energy storage batteries before being sold to the distribution network;
[0018] 2.3) Establish a park optimization scheduling model based on the optimization goal of minimizing the total operating cost of the entire park.
[0019] Further, the specific steps of step 3) are:
[0020] 3.1) Markov decision process;
[0021] The multi-agent deep reinforcement learning algorithm is a random game architecture based on the Markov decision process.<S,A,R,P,γ> Represents, where the state vector group S = {s1, s2, ..., s i ,…,s I} represents the state set of the environment in which each building scheduling agent is located. I is the number of agents. Each building is fully aware of its own equipment status, but cannot perceive the equipment status of other buildings, that is, the observation space o of agent i i =s i ; Joint action A = {a1, a2, ..., a i ,…,a I} represents the action set of each agent; the reward R = {r1,r2,…,r i ,…,r I} represents the reward S×A×S→R obtained by each agent when performing an action in the current state; the state transition matrix P represents the probability distribution of the environment transferring from the current state to the next state after the agent performs the current action; the attenuation coefficient γ of the cumulative discounted return represents the influence of the long-term return on the current action;
[0022] The observation space of the building includes load demand, the energy status of the hydrogen storage tank in the previous period, the output power of wind power and photovoltaic power, and the current scheduling period;
[0023] 3.2) Offline centralized training;
[0024] Each agent is an actor-critic architecture. The policy network and target network of the actor and critic use deep Q networks to approximate the actual value of the optimal strategy and value, solving the problem that actions and values are discrete and cannot cover all possible values.
[0025] 3.3) Online distributed execution: Park executes action A t And get the reward R of the current period t and the environmental state S in the next period t+1 , then S t+1 It serves as the agent input for the next period and makes decisions for that period until the scheduling actions for all periods are obtained.
[0026] The beneficial effects of the present invention are as follows: the electric-hydrogen complementary multi-building energy supply system of the present invention and its coordinated scheduling method, the intelligent building containing the hydrogen energy storage system uses the electricity price to guide the hydrogen energy storage system to flexibly adjust the net load curve of the building under the premise of considering the power source of hydrogen electrolysis, optimizing the user's energy cost while ensuring the low-carbon and energy-saving operation of the system. The MADDPG method is combined with the operation structure of the smart park, and a distributed real-time economic scheduling method is designed, so that each building can achieve the overall optimality in the process of self-optimization. The effectiveness of the proposed method is verified through example analysis and comparison, and it has the advantages of clear logical structure and practical rationality. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 Establish a flow chart for the coordinated dispatching method of the electric-hydrogen complementary park multi-building energy supply system of the present invention;
[0028] Figure 2 This is a schematic diagram of the structure of a multi-building energy supply system in a park with electricity and hydrogen complementation according to an embodiment of the present invention;
[0029] Figure 3 A specific scheduling strategy flow chart of an embodiment of the present invention;
[0030] Figure 4a A distribution diagram of a training data set for a smart building according to the first embodiment of the present invention;
[0031] Figure 4b This is a distribution diagram of the training data set of the smart building according to the second embodiment of the present invention;
[0032] Figure 4c This is a distribution diagram of the training data set of the smart building in the third embodiment of the present invention;
[0033] Figure 5 The return value curve diagrams of the buildings in the three embodiments of the present invention are as follows;
[0034] Figure 6a This is a diagram of optimization results of the smart building according to the first embodiment of the present invention without considering the green hydrogen production condition;
[0035] Figure 6b This is a diagram of optimization results of the smart building in Example 2 of the present invention without considering the green hydrogen production condition;
[0036] Figure 6c This is a diagram of optimization results of the smart building in Example 3 of the present invention without considering the green hydrogen production condition;
[0037] Figure 6d This is a diagram of optimization results of the smart building in Embodiment 1 of the present invention under the condition of green hydrogen production;
[0038] Figure 6e This is a diagram of optimization results of the smart building in Example 2 of the present invention under the condition of green hydrogen production;
[0039] Figure 6f This is a diagram of optimization results of the smart building in Example 3 of the present invention under the condition of green hydrogen production;
[0040] Figure 7a A price-driven interactive electric power diagram of a building according to an embodiment of the present invention;
[0041] Figure 7b Building interactive electrical power diagram considering green hydrogen production for embodiments of the present invention. DETAILED DESCRIPTION
[0042] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0043] like Figure 1 The method for coordinating and dispatching the energy supply system of multiple buildings in a park with complementary electricity and hydrogen is shown as a flow chart, and the method includes the following steps:
[0044] S1, based on actual park projects, analyzed in detail the operating mechanisms of energy flow, information flow and control flow within the park, classified them according to the reliability and economic requirements of different buildings, and established a multi-building energy supply system for the park with hydrogen energy storage and electric-hydrogen complementation;
[0045] S2 established detailed operation models and load state functions of hydrogen storage tanks for the four links of hydrogen energy storage electrolysis, hydrogenation, dehydrogenation and combustion, and applied them as constraints in the scheduling model of the multi-building energy supply system of the park with complementary electricity and hydrogen;
[0046] In order to realize the autonomous operation capability between buildings, S3 established an interactive mechanism for interconnecting multiple buildings in the multi-building energy supply system of the park, and realized the real-time energy supply scheduling of the smart park in the way of "offline centralized learning and online distributed execution" based on the historical data of the buildings;
[0047] S4 To verify the effectiveness of the method proposed in the present invention, the scheduling strategy based on price signals was compared with the scheduling method considering hydrogen production from renewable energy, and the optimization performance of the method proposed in the present invention was compared with other deep reinforcement learning methods.
[0048] In step S1, the operation mechanism of energy flow, information flow and control flow within the park is analyzed in detail for the actual smart park project, and different buildings are classified according to the reliability and economic requirements, and an electric-hydrogen complementary park multi-building energy supply system is established. The structural schematic diagram of the park multi-building energy supply system of the embodiment is shown in FIG. Figure 2 shown.
[0049] In the park, each smart building belongs to a different entity, and each building is equipped with a hydrogen energy storage system, and its scheduling is to minimize its own operating costs. But they together form a whole, the buildings are connected by a bus, and there is power sharing between the buildings. Each building is assigned an intelligent agent i, and there is a communication line between the intelligent agent and the controller of the building equipment to obtain real-time status data measured by sensors. At the same time, this data will be stored and transmitted to the intelligent agents of other buildings through the communication line at regular intervals. The intelligent agents directly communicate with each other for offline learning of the intelligent agents. Based on the real-time data, the intelligent agent determines the optimal reference value of the controllable components inside the building online and transmits it to the controller of the corresponding component. The controller controls the equipment to track these reference values by controlling the inverter and intelligent switch. In this way, the overall optimization of the park is achieved in a fully distributed manner from the process of self-optimization of each building.
[0050] Assume that the set of scheduling periods for the whole day is Ω T ={1,…,t,…,T}, the set of buildings in the park Ω B={1,…,n,…,N}, the smart park composed of smart buildings is connected to the distribution network through a public connection point (PCC) controlled by the park manager. Each building is equipped with photovoltaic panels, wind turbines and diesel generators of different capacities. Buildings with high power supply reliability requirements (such as hospitals, police stations, etc.) need to be equipped with hydrogen energy storage systems. The corresponding set of hydrogen energy storage systems is At the same time, the peak load regulation effect of flexible load is also considered. The load requirements of some buildings are not high, and the power supply to some flexible loads can be reduced within a certain range. The corresponding set is The above components are equipped with corresponding sensors, controllers and intelligent switches.
[0051] In step S2, detailed operation models and load state functions of hydrogen storage tanks are established for the four links of hydrogen energy storage system electrolysis, hydrogenation, dehydrogenation and combustion, and they are applied as constraints in the scheduling model of smart park. The specific steps are as follows:
[0052] Step S21: Modeling key equipment in the established carbon-neutral smart park multi-building operating environment;
[0053] Electrolysis process:
[0054]
[0055] In the formula, is the hydrogen output obtained by electrolysis; η EL is the electrolysis efficiency; EL is the conversion efficiency of the electrolyzer; The electrical power consumed by the electrolyzer; The maximum and minimum values of the electric power consumed by the electrolytic cell.
[0056] Hydrogenation process:
[0057]
[0058] In the formula, It is the liquid hydrogen stored in the hydrogen storage tank; LH is the volumetric storage density of the hydrogen storage unit (LOHC); is the hydrogenation reaction efficiency; The electrical power consumed for the hydrogenation reaction; is the power consumption rate of the hydrogenation reaction.
[0059] Dehydrogenation process:
[0060]
[0061] In the formula, Hydrogen produced by dehydrogenation reactions for use in fuel cells to generate electricity; is the dehydrogenation reaction efficiency; The electrical power consumed for the dehydrogenation reaction; is the power consumption rate of the dehydrogenation reaction.
[0062] The energy conversion process of hydrogen fuel cells and the output constraints of fuel cells:
[0063]
[0064] In the formula, is the electrical power generated by the fuel cell; η FC is the hydrogen-electricity conversion factor; ξ FC The power generation efficiency of the fuel cell; They are the upper and lower limits of hydrogen fuel cell power generation respectively.
[0065] Load status of hydrogen energy storage system
[0066]
[0067] In the formula, μ loss ≈0 represents the loss coefficient of the hydrogen storage tank, Δt=t-t0, t0 represents the initial time of the optimization cycle, They are the upper and lower limits of the load of the hydrogen energy storage system respectively.
[0068] The diesel generator is used as the controllable unit, and its mathematical model is:
[0069]
[0070] In the formula, Indicates the output of the diesel generator in the current period; Indicates the upper and lower limits of the diesel generator output.
[0071] The mathematical model of flexible load is:
[0072]
[0073] In the formula, represents the net load of the nth building in time period t; represents the fixed load of the nth building in time period t; represents the flexible load of the nth building in time period t; It represents the reduction of flexible load of the nth building in time period t.
[0074] Step S22: forming a scheduling strategy according to the different supply and demand of the park.
[0075] Figure 3 The scheduling strategy flow chart of the embodiment is as follows:
[0076] (1) After each building generates electricity to meet its own load demand, there is surplus power. If there is surplus power from renewable energy output at this time, the hydrogen energy storage system will be used to store this energy first. If there is still surplus power, it will be sold to the distribution network to generate revenue;
[0077] (2) The power generation of each building cannot meet its own load demand. At this time, the energy stored in the building’s own hydrogen energy storage system is used first until the capacity limit of the hydrogen storage tank or the maximum discharge power of the fuel cell is reached;
[0078] (3) After some buildings have generated electricity to meet their own load demand, there is surplus power, while some buildings cannot meet their own load demand. Adhering to the principle of local consumption of renewable energy power generation, buildings with power shortages purchase electricity from buildings with surplus power first. When the surplus power of the former is less than the power shortage of the latter, the surplus power is supplied by their own energy storage batteries until the lower limit of the energy storage battery capacity is reached. If it still cannot meet the power demand, the surplus power is obtained from the distribution network; when the surplus power of the former is greater than the load demand of the latter, the surplus power is sold to the distribution network (peak period) or first charged to the energy storage battery before selling to the distribution network (normal period, valley period).
[0079] Step S23: Establishing a park optimization scheduling model according to the optimization goal of minimizing the total operating cost of the entire park.
[0080] Taking into account the power purchase cost, operation cost, maintenance cost, carbon emission cost, compensation cost for reducing flexible loads and power sales revenue, the specific objective function is as follows:
[0081]
[0082] The electricity purchase cost of the nth building includes the cost of purchasing electricity from other buildings and the upper power grid:
[0083]
[0084] In the formula, m represents the mth building and m≠n, are the electricity purchased by the nth building from building m and the upper power grid, and Indicates the electricity purchase price between buildings and the electricity purchase price of the upper-level power grid.
[0085] The operating cost is mainly the fuel cost of diesel generator:
[0086]
[0087] In the formula, a i , b i and c i is the fuel cost coefficient of the diesel generator.
[0088] Maintenance costs mainly include distributed power supply maintenance costs and energy storage battery depreciation costs:
[0089]
[0090] In the formula, σ PV , σ WT , and σ DE Represent the operation and maintenance costs of photovoltaic, wind turbine, hydrogen energy storage system and diesel generator respectively; and They represent the output power of photovoltaic, wind turbine and diesel generator of the nth building in time period t respectively.
[0091] Carbon emission costs from diesel generators and distribution networks:
[0092]
[0093] In the formula, and represent the carbon emission cost coefficients of diesel generators and the upper-level power grid respectively.
[0094] Compensation cost of flexible loads:
[0095]
[0096] The electricity sales revenue is the electricity sold by each building to the microgrid level and distribution network level:
[0097]
[0098] At time period t, the power balance constraints of each building are:
[0099]
[0100] There are upper and lower limits on power exchange between the main grid and buildings, and between buildings:
[0101]
[0102] In step S3, for the smart buildings with autonomous operation capabilities, an interactive mechanism for interconnecting multiple buildings is established, and based on the historical data of the buildings, the real-time scheduling of the smart park is realized in the way of "offline centralized learning and online distributed execution". The specific steps are as follows:
[0103] Step S31: Markov decision process;
[0104] The multi-agent deep reinforcement learning algorithm is a random game architecture based on the Markov decision process, which can be used to<S,A,R,P,γ> In which, the state vector group S = {s1, s2, ..., si ,…,s I} represents the state set of the environment in which each building scheduling agent is located. I is the number of agents. Each building is fully aware of its own equipment status, but cannot perceive the equipment status of other buildings, that is, the observation space o of agent i i =s i ; Joint action A = {a1, a2, ..., a i ,…,a I} represents the action set of each agent; the reward R = {r1,r2,…,r i ,…,r I} represents the reward S×A×S→R obtained by each agent when performing an action in the current state; the state transfer matrix P represents the probability distribution of the environment transferring from the current state to the next state after the agent performs the current action; the attenuation coefficient γ of the cumulative discounted return represents the influence of the future return on the current action.
[0105] The observation space of the building includes load demand (including fixed load and flexible load), the energy state of the hydrogen storage tank in the previous period, the output power of wind power and photovoltaic power, and the current scheduling period, specifically:
[0106]
[0107] The action output by the intelligent agent can be represented by the response power of the flexible load and the power generated by the diesel generator, as follows:
[0108]
[0109] After determining the response power of the flexible load and the power generation power of the diesel generator, the power purchase and sale of the electrolytic cell, fuel cell and building can be determined by the formulated scheduling strategy.
[0110] According to the scheduling model of the case, the goal of minimizing the operating costs of each building is converted into the form of maximizing the reward of the agent, and the reward expression obtained by the nth agent in the scheduling period t is obtained as follows:
[0111] r i=n,t =-F n,t
[0112] Step S32: Offline centralized training;
[0113] Each agent is an actor-critic (A2C) architecture. The policy network and target network of both the actor and the critic use a deep Q network to approximate the actual value of the optimal policy and value, solving the problem that actions and values are discrete and cannot cover all possible values. Assume that the parameter set of the deep Q network corresponding to I agents is θ = (θ1, θ2, ..., θi ,…,θ I ), then the joint strategy is π=(π(θ1),π(θ2),…,π(θ i ),…,π(θ I )) Actor's main network θ π and the target network θ π The outputs of ' are the diesel generator output and the flexible load response of the current period and the next period, respectively. The action of the next period is determined by the state entered by the agent based on the environment when executing the action of the current period. The main network θ of Critic Q and the target network θ Q 'Estimate the values of the state value function Q and the action value function Q' according to the current period strategy and the next period strategy respectively.
[0114] The state value function represents the environment in state S t The discounted return obtained after executing the strategy π, the action value function represents the environment in state S t Next, perform action A t After that, if we continue to execute strategy π, we can get the expected value of the return. The Bellman equation of the state value function and action value function can be expressed as follows:
[0115] V i (s i )=E i [r i,t+1 +γV i (S t+1 )|S t =s]
[0116] Q i (s,a)=E i [r i,t+1 +γQ i (S t+1 ,A t+1 )|S t =s,A t =a]
[0117] The training steps of the MADDPG algorithm are as follows:
[0118] (1) Set the number of training times N;
[0119] (2) Initialize a random action sampling process;
[0120] (3) Setting initial values of LOHC initial state and other states;
[0121] (4) Set the number of training rounds T = 24;
[0122] (5) Each Actor main network is based on o i,tuse Determine action a i,t Constitute the action set A t ;
[0123] (6) Building group operating environment execution action A t , and returns a reward R to each agent t , current state S t and the next state S t+1 ;
[0124] (7) Actor target network based on A t ,o i,t+1 Determine the next round action a i,t+1 Constitute the action set A t+1 ;
[0125] (8) S t ,o i,t ,A t ,r i,t ,S t+1 ,o i,t+1 ,A t+1 Store in experience pool D i ;
[0126] (9) Update the agent’s network parameters in each round;
[0127] (10) From D i Randomly sample small batches of data;
[0128] (11) Use Calculate the loss function of Actor and Critic;
[0129] (12) According to the strategy gradient, Update the main network parameters of Actor and Critic;
[0130] (13) The agent’s turn ends;
[0131] (14) Use Soft update of the target network of Critic and Actor;
[0132] (15) End of round;
[0133] (16) Training is completed.
[0134] Step S33: Online distributed execution.
[0135] Table 1 shows the neural network structure and hyperparameter settings of the embodiment. t And get the reward R of the current period t and the environmental state S in the next periodt+1 , then S t+1 It serves as the agent input for the next period and makes decisions for that period until the scheduling actions for all periods are obtained.
[0136] Table 1
[0137]
[0138]
[0139] In step S4, in order to verify the effectiveness of the method proposed in the present invention, the scheduling strategy based on price signals is compared with the scheduling method considering hydrogen production from renewable energy, and the optimization performance of the method proposed in the present invention is compared with other deep reinforcement learning methods. The specific steps are:
[0140] Step S41: Based on the training data and the environmental parameters, each building agent is trained to obtain the reward value curves of the three building agents during the training process;
[0141] Figure 4a-4c The distribution diagram of the training data set of the smart building in the three embodiments of the present invention is shown below. Figure 5 The return value curves of each building for the three examples are as follows: Figure 5 It can be seen that the agent gets a small reward value at the beginning of training, and the distribution of the reward value is relatively loose, indicating that the agent is not stable at this time and is in the initial exploration stage; after about 8,000 trainings, the reward value obtained by the agent gradually increases during the training process and finally stabilizes in a small range, indicating that the agent has gained experience through continuous trial and error, and finally learned the optimal scheduling strategy that minimizes the system operation cost. Since the training data in each training round changes, the reward value will oscillate during the training process.
[0142] Step S42: To illustrate the real-time scheduling performance of the system, the scheduling data of the smart park on May 1, 2018 is taken as an example, and the scheduling conditions of two scenarios are simulated based on the distributed coordinated scheduling method proposed in the present invention;
[0143] Scenario 1: The building uses traditional electric energy storage. The operation of the energy storage device is not restricted by the power source, and the dispatch strategy adopts a general market price-driven strategy.
[0144] Scenario 2: The building adopts a hydrogen energy storage system and considers renewable energy hydrogen production equipment. The electricity source for hydrogen production is all renewable energy, and the scheduling strategy adopts the strategy proposed in step S22.
[0145] Figure 6a~6f are the scheduling results under the two scheduling strategies of the embodiment, where Figure 6a , 6c6e are the scheduling results of buildings 1, 2, and 3 in scenario 1, and 6b, 6d, and 6f are the scheduling results of buildings 1, 2, and 3 in scenario 2. It can be seen that in scenario 1, the hydrogen energy storage system is charged and discharged under the guidance of electricity prices. It is charged during valley electricity prices and when the electricity load is small to prepare for the discharge demand during peak load periods. For example, during the period of 23:00-04:00, buildings 1, 2, and 3 all choose to convert electricity into liquid hydrogen for storage during this period; during the period of 06:00-09:00, it is a period of flat electricity prices. At this time, building 1 is short of electricity while buildings 2 and 3 have excess electricity, so the power shortage of building 1 is supplemented by buildings 2 and 3; in contrast, during the period of 15:00-17:00, the electricity load of buildings 1, 2, and 3 is high and the wind and solar output are insufficient, and the entire microgrid is short of electricity. Therefore, buildings 1 and 3 choose to release the electricity stored in the hydrogen energy storage system, and the remaining power shortage is supplemented by purchasing electricity from the distribution network. Building 2 does not have a hydrogen energy storage system and can only choose to purchase electricity from the distribution network. In the period of 11:00-14:00, the sum of the wind and solar output of buildings 1 and 3 is greater than the load demand, and the hydrogen energy storage system has stored enough electricity in the early stage of scheduling, so buildings 1 and 3 choose to transmit the excess electricity to obtain benefits; while building 2 is in a state of power shortage at this time, and is supplemented by diesel generators. In the period of 18:00-22:00, it is the peak electricity price period, and all buildings are in a state of power shortage. The hydrogen energy storage systems of buildings 1 and 3 cannot fully meet their electricity demand due to output power limitations, so diesel generators need to be called to supplement the remaining power shortage. In scenario 2, the electricity source for hydrogen production is the renewable energy power generation device in the smart park, so the electrolyzer operates in the period when renewable energy is more abundant, such as 4:00-8:00. The action of the hydrogen fuel cell is similar to scenario 1, concentrated in the period of 15:00-21:00, when the electricity price is high, and the hydrogen fuel cell output is to reduce the operating cost of the smart park. During the period of 11:00-14:00, although renewable energy is also abundant, the electricity price is high at this time, and the building still chooses to sell electricity to the distribution network to obtain revenue. The total operating costs of the smart park under the two scenarios are 2416.341 yuan and 2231.559 yuan respectively. The carbon emission cost of each building is shown in Table 2. The total carbon emission costs of the park are 15.254 yuan and 12.278 yuan respectively. The smart park structure designed by the present invention considering the complementarity of electricity and hydrogen reduces the operating cost by 7.643% and the carbon emission level by 19.510% compared with the smart park using traditional electric energy storage.
[0146] Table 2
[0147]
[0148]
[0149] Figure 7a , 7bThis is a building interactive power diagram based on price drive and considering green hydrogen production in an embodiment of the present invention, combined with Figure 7a , 7b By comparing the two scheduling strategies, it can be seen that the buildings in Scenario 1 choose to purchase electricity and store energy from the distribution network during valley electricity price periods, while during flat and peak electricity price periods, since the electricity sales price is higher than the internal transaction price, the buildings are more inclined to sell their surplus power to the distribution network to increase their own profits; while in Scenario 2, the buildings produce green hydrogen when there is surplus output from renewable energy, which is equivalent to increasing the priority of internal power interaction and improving the park's ability to absorb renewable energy locally.
[0150] Step S43: In order to verify the effectiveness of the multi-building coordinated scheduling method for a carbon-neutral smart park based on the MADDPG algorithm, the scheduling method based on the MADDPG algorithm is compared with the scheduling method based on the DQN and the scheduling method based on the DDPG.
[0151] Table 3 compares the optimization results of different algorithms in the embodiment, and gives the statistical data of the average daily operating costs after optimization of the three methods. Among them, the daily operating cost based on the MADQN algorithm is 2437.124 yuan, an increase of 5.19% compared with the MADDPG algorithm; the average daily operating cost based on the DDPG algorithm is 2355.129 yuan, an increase of 2.08% compared with the MADDPG algorithm. This is because in the MADQN method, due to the flexible load response amount and the output of hydrogen energy storage need to take the set discrete value, the action selection cannot cover the entire action space, and the screened action is likely to be a suboptimal action. The DDPG algorithm uses a single agent to schedule all buildings at the same time. The state set and action set are very large, which causes the agent to select a suboptimal action, and a large amount of communication is required in actual operation. It can be seen that the scheduling method based on the MADDPG algorithm proposed in the present invention can more easily explore the optimal action in the action space than the other two algorithms, and is more suitable for solving the coordinated scheduling problem of multi-intelligent buildings containing hydrogen energy storage systems.
[0152] Table 3
[0153]
[0154] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. A coordinated dispatching method for a multi-building energy supply system in a park with electricity and hydrogen complementation, characterized in that: The specific steps include: 1) According to the actual park project, analyze the operation mechanism of energy flow, information flow and control flow within the park, classify them according to the reliability and economic requirements of different buildings, and establish a multi-building energy supply system in the park with hydrogen energy storage and electric hydrogen complementation; In the park, each building belongs to a different entity, and a hydrogen energy storage system is configured for buildings that cannot be powered off. All buildings together form a whole, connected by a bus, and there is power sharing between buildings. Each building is assigned an intelligent agent, and there is a communication line between the intelligent agent and the controller of the building equipment to obtain real-time status data measured by sensors. At the same time, the real-time status data is stored by the intelligent agent to which it belongs, and transmitted to the intelligent agents of other buildings through the communication line at regular intervals for offline learning of the intelligent agent. Based on real-time data, the intelligent agent determines the optimal reference value of the controllable components inside the building online and transmits it to the controller of the corresponding component. The controller controls the inverter and intelligent switch to make the equipment track the optimal reference value. 2) Model the energy and load in the power supply system separately. For the four links of hydrogen energy storage system, namely electrolysis, hydrogenation, dehydrogenation and combustion, the operation model and the load state function of the hydrogen storage tank are established respectively, and they are applied as constraints in the scheduling model of the multi-building energy supply system in the park with complementary electricity and hydrogen; The specific steps include: 2.1) Model the energy and loads in the multi-building operating environment of the established carbon-neutral campus; 2.2) Form a scheduling strategy based on the supply and demand of the park: 2.2.1) After each building generates electricity to meet its own load demand, there is surplus power: If there is surplus power from renewable energy output at this time, the hydrogen energy storage system will be used to store the surplus energy. If there is still surplus power, it will be sold to the distribution network to obtain revenue; 2.2.2) The power generation of each building cannot meet its own load demand: At this time, the energy stored in the building’s own hydrogen energy storage system is used first until the capacity limit of the hydrogen storage tank or the maximum discharge power of the fuel cell is reached; 2.2.3) After some buildings have met their own load demand, there is surplus power, while some buildings cannot meet their own load demand: Adhering to the principle of local consumption of renewable energy power generation, buildings with power shortages shall purchase electricity from buildings with surplus power first. When the surplus power of the former is less than the power shortage of the latter, the surplus power shall be supplied by their own energy storage batteries until the lower limit of the energy storage battery capacity. If it still cannot meet the power demand, the surplus power shall be obtained from the distribution network; when the surplus power of the former is greater than the load demand of the latter, the surplus power during peak hours shall be sold to the distribution network, or the surplus power during normal and valley hours shall be charged to the energy storage batteries before being sold to the distribution network; 2.3) Establish an optimal scheduling model for the park based on the optimization goal of minimizing the total operating cost of the entire park; 3) Establish an interactive mechanism for interconnecting multiple buildings in the park's multi-building energy supply system. With the support of a multi-agent reinforcement learning algorithm, based on the historical data of the buildings, the park's real-time energy supply scheduling is achieved in the form of "offline centralized learning and online distributed execution".
2. The coordinated scheduling method for the electric-hydrogen complementary multi-building energy supply system in a park according to claim 1 is characterized in that: The park composed of buildings in step 1) is connected to the distribution network through a public connection point controlled by a park manager, and each building is equipped with photovoltaic panels, wind turbines, diesel generators and hydrogen energy storage systems of different capacities according to its own load.
3. The coordinated scheduling method for the electric-hydrogen complementary multi-building energy supply system in a park according to claim 1 is characterized in that: The specific steps of step 3) are: 3.1) Markov decision process; The multi-agent deep reinforcement learning algorithm is a random game architecture based on the Markov decision process.<S,A,R,P,γ> Represents, where the state vector group S = {s1, s2, ..., s i ,…,s I } represents the state set of the environment in which each building scheduling agent is located. I is the number of agents. Each building is fully aware of its own equipment status, but cannot perceive the equipment status of other buildings, that is, the observation space o of agent i i =s i ; Joint action A = {a1, a2, ..., a i ,…,a I } represents the action set of each agent; the reward R = {r1,r2,…,r i ,…,r I } represents the reward S×A×S→R obtained by each agent when performing an action in the current state; the state transition matrix P represents the probability distribution of the environment transferring from the current state to the next state after the agent performs the current action; the attenuation coefficient γ of the cumulative discounted return represents the influence of the long-term return on the current action; The observation space of the building includes load demand, the energy status of the hydrogen storage tank in the previous period, the output power of wind power and photovoltaic power, and the current scheduling period; 3.2) Offline centralized training; Each agent is an actor-critic architecture. The policy network and target network of the actor and critic use deep Q networks to approximate the actual value of the optimal strategy and value, solving the problem that actions and values are discrete and cannot cover all possible values. 3.3) Online distributed execution: Park executes action A t And get the reward R of the current period t and the environmental state S in the next period t+1 , then S t+1 It serves as the agent input for the next period and makes decisions for that period until the scheduling actions for all periods are obtained.
