Hydrogen energy storage auxiliary power system control method and system based on dynamic reward DDPG

Abstract

The invention relates to a dynamic reward DDPG-based hydrogen energy storage auxiliary power system control method and system, and the method comprises the following steps: constructing a frequency adjustment model of a power system, and introducing a hydrogen energy storage system as an auxiliary power adjustment resource; on the basis of a deep reinforcement learning algorithm, a DDPG intelligent agent is constructed to interact with a power system, features are extracted from a system state, an optimal control strategy is generated, the DDPG intelligent agent adopts dynamic rewards changing along with a training period, the sparse reward problem is effectively avoided, and the adaptability of the intelligent agent to complex working conditions is improved; according to the optimal control strategy, power output of the hydrogen energy storage system is adjusted in real time, and it is ensured that the system frequency is stable. Compared with the prior art, the method can quickly respond and maintain the stability of the system frequency in a scene that the frequency deviation is increased in an emergency of the power system, and greatly improves the operation stability and intelligence of the power system.

Description

Technical Field

[0001] The present invention relates to the technical field of power system dispatching and control, and in particular to a hydrogen energy storage-assisted power system control method and system based on dynamic reward DDPG. Background Art

[0002] The power industry has flourished over the past few decades, with technological advancements and industrial transformation progressing in parallel. As a key factor in balancing supply and demand, power system frequency regulation is attracting increasing attention from researchers worldwide. Furthermore, renewable energy generation is rapidly developing, and the scale and capacity of its integration into the grid are increasing. However, its inherent intermittent nature introduces uncertainties into grid frequency regulation. Against this backdrop, energy storage technologies that can dispatch power at any time offer an effective solution. Common energy storage technologies include flywheels, supercapacitors, superconducting coils, batteries, and hydrogen. Each of these technologies has its own strengths and weaknesses, each playing a key role in specific scenarios. Hydrogen storage, with its advantages of long-term, large-capacity storage, cross-regional energy dispatch, high energy density, and clean, low-carbon nature, has become a core direction for the long-term, deep decarbonization of future energy systems.

[0003] There has been a lot of research at home and abroad on the optimization control of power systems assisted by hydrogen energy storage and other related technologies. In response to the challenges brought by the volatility and randomness of the source load side to the frequency regulation of the power system, an existing technology introduces a hydrogen energy storage system and proposes a power allocation principle to enable the hydrogen energy storage system to track the mismatched power, thereby ensuring the effectiveness and rapidity of the power system frequency control. There is also an existing technology that proposes a dynamic droop coefficient control strategy for secondary frequency regulation of thermal power units assisted by energy storage batteries, which improves the frequency deviation and is better than the fixed droop coefficient control. CN119051070A discloses a deep reinforcement learning load frequency control method for energy storage-assisted thermal power units. The load frequency control problem of the power system is modeled as a Markov decision model. Combining the Markov decision model with the output control law of the energy storage system, a deep reinforcement learning load frequency control framework based on the Actor-Critic architecture is constructed. Combining the deep deterministic policy gradient (DDPG) algorithm and the random network distillation technology, a dual-output load frequency controller (agent) is designed, which can achieve complementary and coordinated control of the energy storage system and the thermal power unit, effectively improving the frequency regulation effect of the power system.

[0004] Although the above existing technologies all consider the frequency regulation of power systems with the participation of hydrogen energy storage, as the capacity of new energy grid connection continues to expand, the instantaneous integration and removal of hydrogen energy storage will inevitably bring instability to the power system, increasing the challenge of frequency regulation. Traditional frequency regulation methods can only regulate the frequency of the power system under simple operating conditions, and their ability to regulate frequency under the complex operating conditions mentioned above is limited. In addition, with the participation of hydrogen energy storage systems, the frequency regulation process of the power system should consider more factors, further increasing the difficulty of frequency regulation. Therefore, there is an urgent need for more intelligent control methods to cope with the potential complex operating conditions of the power system. Summary of the Invention

[0005] The purpose of the present invention is to provide a hydrogen energy storage assisted power system control method and system based on dynamic reward DDPG, which can cope with the situation of increased frequency deviation in sudden power system situations, quickly respond and maintain the stability of system frequency, and improve the stability and intelligence of power system operation.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A method for controlling a hydrogen energy storage-assisted power system based on a dynamic reward DDPG includes the following steps:

[0008] Build a frequency regulation model for the power system and introduce a hydrogen energy storage system as an auxiliary power regulation resource;

[0009] Based on a deep reinforcement learning algorithm, a DDPG agent is constructed to interact with the power system, extract features from the system state, and generate an optimal control strategy. The DDPG agent uses dynamic rewards that change over the training cycle.

[0010] According to the optimal control strategy, the power output of the hydrogen energy storage system is adjusted in real time to ensure the stability of the system frequency.

[0011] The frequency regulation model of the power system is:

[0012]

[0013] Where, X gi 、P gi 、P li , Δf i with u i are the governor position increment, turbine output, load change, system frequency deviation and controller input respectively; T gi 、T ti 、T pi are the time constants of the governor, steam turbine, and generator respectively; K pi is the gain coefficient; R i is the speed regulator coefficient.

[0014] The hydrogen energy storage system includes a hydrogen storage tank, an electrolyzer and a hydrogen fuel cell. The corresponding model is expressed as follows:

[0015]

[0016] 0≤P el,t ≤P el,max

[0017] E ce,t =E el,t ·η ce

[0018]

[0019] 0≤E ce,t ≤E ce,max

[0020] 0≤E out,t ≤E out,max

[0021] Where, E el,t is the hydrogen production of the electrolyzer, P el,t is the input power of the electrolyzer at time t, η el is the electrolysis efficiency of the electrolytic cell, is the calorific value of hydrogen, Δt is the time coefficient, P el,max is the maximum input power that the electrolytic cell can withstand, E ce,t is the amount of hydrogen compressed at time t, η ce is the compression efficiency of hydrogen, is the hydrogen content of the hydrogen energy storage model at time t, E out,t is the hydrogen output of the hydrogen storage tank at time t, is the minimum hydrogen storage capacity of the hydrogen storage tank, is the maximum hydrogen storage capacity of the hydrogen storage tank, E ce,max E is the maximum value of hydrogen input that the hydrogen storage tank can accept. out,max It is the maximum value of hydrogen output that the hydrogen storage tank can accept.

[0022] The regulation process of the hydrogen energy storage system as an auxiliary power regulation resource is as follows: when the unit output reaches its peak, the excess output is electrolyzed in the electrolyzer to achieve the conversion of electricity into hydrogen. The converted hydrogen energy is compressed and stored in the hydrogen storage tank and provided to hydrogen-consuming equipment on demand. In the power system, the power system's peak-shaving and valley-filling are achieved through the electricity-hydrogen-electricity cycle:

[0023]

[0024] Where, P HFC (t) and ηHFC They are the hydrogen input power, output power and conversion efficiency of the hydrogen fuel cell respectively.

[0025] When the hydrogen energy storage system participates in auxiliary power regulation, the frequency regulation of the power system is:

[0026]

[0027] Where, P gi 、P li 、P hi , Δf i are respectively turbine output, load change, power output of hydrogen energy storage system, and system frequency deviation; T pi is the generator time constant; K pi is the gain coefficient.

[0028] The state space of the DDPG agent is the system frequency deviation.

[0029] The action space of the DDPG agent is the correction value of the system frequency, that is, the speed regulator control value.

[0030] The dynamic rewards are:

[0031]

[0032] In the formula, a determines the specific size of each reward, which is determined by multiple experiments, e and o are the frequency deviation and the current training target respectively, and y i with y i ′ are the actual output value and the reference value, ε is a parameter used to prevent the reward value from being too large, and n is the power of the training target.

[0033] The DDPG agent training process is as follows:

[0034] For the current state of the environment, the action main network outputs the action under the current strategy;

[0035] Evaluate the main network to obtain the evaluation result of the current action and calculate the current loss based on the evaluation result;

[0036] Optimize the parameters of the action main network with the goal of minimizing the current loss;

[0037] The exponential moving average method is used to update the parameters of the action target network;

[0038] Based on the environmental state at the next moment, the action target network outputs the next action;

[0039] The evaluation target network obtains the evaluation result of the next moment according to the next action and the environmental state at the next moment, and calculates the loss of the next moment based on the evaluation result;

[0040] Optimize and evaluate the parameters of the main network with the goal of minimizing the loss at the next moment;

[0041] The exponential moving average method is used to update the parameters of the evaluation target network.

[0042] A hydrogen energy storage auxiliary power system control system based on dynamic reward DDPG, comprising:

[0043] Frequency regulation model construction module: Builds a frequency regulation model for the power system and introduces a hydrogen energy storage system as an auxiliary power regulation resource;

[0044] Control strategy generation module: Based on the deep reinforcement learning algorithm, a DDPG agent is constructed to interact with the power system, extract features from the system state, and generate the optimal control strategy. The DDPG agent is trained using dynamic rewards that change over the training cycle.

[0045] Control module: Based on the optimal control strategy, it adjusts the power output of the hydrogen energy storage system in real time to ensure the stability of the system frequency.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] This invention builds on the existing power system frequency regulation model by introducing a hydrogen energy storage subsystem. The power system's dispatch center monitors power on both the supply and demand sides in real time. A deep reinforcement learning controller is designed and improved, enabling timely response to changes in system frequency deviations through the use of dynamic rewards. This controller then sends power control commands to the generator units or hydrogen energy storage system based on the frequency deviation to restore frequency. If power output demand increases, the hydrogen storage battery releases pre-stored energy. When excess power is present, hydrogen can be produced by water electrolysis, storing the excess energy for subsequent replenishment or sale, significantly improving the stability and economic efficiency of the power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of the method of the present invention;

[0049] Figure 2 This is a structural diagram of the hydrogen energy storage system of the present invention;

[0050] Figure 3 This is a frequency regulation model diagram of the hydrogen energy storage auxiliary power system of the present invention;

[0051] Figure 4 This is the DDPG agent structure diagram of the present invention;

[0052] Figure 5 is a frequency control strategy diagram of the present invention;

[0053] Figure 6A frequency deviation diagram of a non-hydrogen energy storage power system under stable operating conditions provided by an embodiment of the present invention;

[0054] Figure 7 A frequency deviation diagram of a hydrogen energy storage power system under stable operating conditions provided by an embodiment of the present invention;

[0055] Figure 8 A load drop diagram of a power system provided by an embodiment of the present invention;

[0056] Figure 9 A frequency deviation diagram of a non-hydrogen energy storage power system during a sudden load drop provided by an embodiment of the present invention;

[0057] Figure 10 A frequency deviation diagram of a hydrogen energy storage power system during a sudden load drop provided by an embodiment of the present invention;

[0058] Figure 11 A load surge diagram of a power system provided by an embodiment of the present invention;

[0059] Figure 12 A frequency deviation diagram of a non-hydrogen energy storage power system during a sudden load increase provided by an embodiment of the present invention;

[0060] Figure 13 This is a frequency deviation diagram of a hydrogen energy storage power system when the load suddenly increases according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0062] In order to achieve rapid response, maintain the stability of system frequency and improve the stability and intelligence of power system operation in the event of increased frequency deviation in power system emergencies, this embodiment provides a hydrogen energy storage assisted power system control method based on dynamic reward deep deterministic policy gradient (DR-DDPG), such as Figure 1 As shown, the following steps are included:

[0063] S1, build a frequency regulation model for the power system and introduce a hydrogen energy storage system as an auxiliary power regulation resource.

[0064] In this embodiment, the frequency regulation model of the power system is:

[0065]

[0066]

[0067] Where, X gi、P gi 、P li , Δf i with u i are the governor position increment, turbine output, load change, system frequency deviation and controller input respectively; T gi 、T ti 、T pi are the time constants of the governor, steam turbine, and generator respectively; K pi is the gain coefficient; R i is the speed regulator coefficient.

[0068] like Figure 2 As shown in Figure 1, the hydrogen energy storage (HES) system includes a hydrogen storage tank, an electrolyzer, and a hydrogen fuel cell. The specific models are shown in (4)-(10):

[0069]

[0070] 0≤P el,t ≤P el,max (5)

[0071] E ce,t =E el,t ·η ce (6)

[0072]

[0073] 0≤E ce,t ≤E ce,max (9)

[0074] 0≤E out,t ≤E out,max (10)

[0075] Where, E el,t is the hydrogen production of the electrolyzer, P el,t is the input power of the electrolyzer at time t, η el is the electrolysis efficiency of the electrolytic cell, is the calorific value of hydrogen, Δt is the time coefficient, P el,max is the maximum input power that the electrolytic cell can withstand, E ce,t is the amount of hydrogen compressed at time t, η ce is the compression efficiency of hydrogen, is the hydrogen content of the hydrogen energy storage model at time t, E out,t is the hydrogen output of the hydrogen storage tank at time t, is the minimum hydrogen storage capacity of the hydrogen storage tank, is the maximum hydrogen storage capacity of the hydrogen storage tank, E ce,max E is the maximum value of hydrogen input that the hydrogen storage tank can accept. out,maxIt is the maximum value of hydrogen output that the hydrogen storage tank can accept.

[0076] The hydrogen energy storage system, as an auxiliary power regulation resource, operates as follows: When the unit output reaches its peak, the excess output is electrolyzed in the electrolyzer to convert electricity into hydrogen. The converted hydrogen energy is compressed and stored in the hydrogen storage tank and provided to hydrogen-consuming equipment on demand. In the power system, peak-shaving and valley-filling are achieved through the electricity-hydrogen-electricity cycle:

[0077]

[0078] Where, P HFC (t) and η HFC They are the hydrogen input power, output power and conversion efficiency of the hydrogen fuel cell respectively.

[0079] When hydrogen energy storage system participates in auxiliary power regulation, such as Figure 3 As shown, the frequency regulation of the power system is:

[0080]

[0081] Where, P gi 、P li 、P hi , Δf i are respectively turbine output, load change, power output of hydrogen energy storage system, and system frequency deviation; T pi is the generator time constant; K pi is the gain coefficient.

[0082] S2, based on the deep reinforcement learning algorithm, builds a DDPG intelligent agent to interact with the power system, extract features from the system state, and generate the optimal control strategy. Among them, the DDPG intelligent agent uses dynamic rewards that change with the training cycle, effectively avoiding the sparse reward problem and improving the intelligent agent's adaptability to complex working conditions.

[0083] Based on the constructed power system model, the state space of the DDPG agent is the system frequency deviation, and the action space is the correction value of the system frequency, that is, the speed regulator control variable.

[0084] Under certain specific working conditions, the power system will inevitably encounter various interferences from the internal and external environment, thereby affecting the control of the load side, which brings great challenges to the frequency regulation of the power system. In addition, when the unit output is insufficient or surplus, hydrogen energy storage needs to be incorporated or removed in a timely manner to balance the output of the unit. In order to better learn the model characteristics of power system frequency regulation with the participation of hydrogen energy storage, this embodiment designs a dynamic reward (DR) driven DDPG. Unlike traditional fixed reward training, a dynamic reward that changes with the training process is designed, thereby decomposing the training task and setting the dynamic reward mechanism that changes with the training cycle as follows:

[0085]

[0086] In the formula, a determines the specific size of each reward, which is determined by multiple experiments, e and o are the frequency deviation and the current training target respectively, and y i with y i ′ are the actual output value and the reference value, ε is a parameter used to prevent the reward value from being too large, and n is the power of the training target.

[0087] like Figure 4 As shown in the figure, the DDPG agent training process is as follows:

[0088] A1, for the environment state s at time t, the action main network θ outputs the action a under the current strategy μ;

[0089] a t =μ θ (s t )(14)

[0090] A2, evaluate the main network to get the evaluation result Q of the current action w (s,a), and calculate the current loss L a ;

[0091] Q w (s,a)=E[u t |S=s t ,A=a t ](15)

[0092] L a =-Q w (s,a)(16)

[0093] A3, to minimize L a Optimize the parameters of the action main network θ for the goal;

[0094] A4, uses the exponential moving average method to update the parameters of the action target network θ′;

[0095] θ′ t+1=τθ′+(1-τθ′ t ) (17)

[0096] A5, for the next moment’s environment state s′, the action target network outputs action a′;

[0097] a′ t =μ′ θ (s′) (18)

[0098] A6, the evaluation target network obtains the evaluation result Q at the next moment based on a′ and s′ w′ (s′, a′), and calculate the loss L at the next moment a ';

[0099]

[0100] A7, to minimize L a ′ is the parameter of the target optimization evaluation main network λ;

[0101] A8: Update the parameters of the evaluation target network λ′ using an exponential moving average method. This process refers to step A4 and will not be described in detail in this embodiment.

[0102] S3, based on the optimal control strategy, adjusts the power output of the hydrogen energy storage system in real time to ensure the stability of the system frequency.

[0103] According to the training results, the power output of the hydrogen energy storage system is adjusted in real time, such as Figure 5 As shown in the figure, the whole frequency modulation process is as follows:

[0104] B1, DR-DDPG agent observes the system state at time t (frequency deviation s t );

[0105] B2, the agent calculates and outputs the current optimal action a;

[0106] B3, the system accepts a and feeds back the current reward r t to the intelligent agent;

[0107] B4, the system state is expressed as the state transition probability P 12 becomes s t+1 ;

[0108] P 12 =P(s t+1 |s t ,a t )(20)

[0109] B5, repeat B1-B4 until the cumulative reward reaches the expected value or the training cycle reaches the set value.

[0110] This embodiment also provides a hydrogen energy storage auxiliary power system control system based on dynamic reward DDPG, including:

[0111] Frequency regulation model construction module: Builds a frequency regulation model for the power system and introduces a hydrogen energy storage system as an auxiliary power regulation resource;

[0112] Control strategy generation module: Based on the deep reinforcement learning algorithm, a DDPG agent is constructed to interact with the power system, extract features from the system state, and generate the optimal control strategy. The DDPG agent is trained using dynamic rewards that change over the training cycle.

[0113] Control module: Based on the optimal control strategy, it adjusts the power output of the hydrogen energy storage system in real time to ensure the stability of the system frequency.

[0114] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0115] In order to verify the feasibility of the proposed hydrogen energy storage-assisted power system intelligent control method based on dynamic reward deep deterministic policy gradient (DR-DDPG), the integral absolute error (IAE) is used to verify the control performance of the proposed method, and its formula is shown in (21). In addition, Case 1 is designed to study the control performance of the proposed method during the stable operation of the power system; Case 2 is used to verify the effectiveness of the proposed method when the power system encounters a sudden disturbance that causes the unit output demand to change; In addition, the superiority of the proposed method compared with the baseline DDPG is compared.

[0116]

[0117] Table 1 Detailed parameter settings of power system

[0118] parameter Value <![CDATA[T g ]]> 0.08 <![CDATA[T t ]]> 0.3 <![CDATA[T p ]]> 21 <![CDATA[K p ]]> 120 R 2.5

[0119] (1) Stable working conditions

[0120] The power system operates under stable conditions most of the time, continuously and stably transmitting electricity. Therefore, it is very necessary to study its operation under stable conditions.

[0121] like Figure 6 As shown in Figure 2, thanks to the high efficiency of the proposed DR-DDPG controller, under stable operating conditions, the power system can quickly stabilize the frequency deviation and output power to the rated value after a small overshoot, even without the assistance of hydrogen energy storage. A detailed IAE comparison is summarized in Table 2.

[0122] Figure 7 This study demonstrates the results of hydrogen energy storage-assisted power system frequency regulation under stable operating conditions. Compared to the case without hydrogen energy storage, the control of system frequency deviation and output power is more efficient, converges more quickly, and overshoot is minimized, fully demonstrating the beneficial role of hydrogen energy storage in assisting power system frequency regulation. Detailed IAE comparison results are summarized in Table 2.

[0123] Table 2 Comparison of frequency adjustment results

[0124]

[0125] (2) Interference conditions

[0126] Under certain sudden or extreme working conditions, various forms and intensities of interference caused by internal and external environmental factors, line faults and large load incorporation greatly restrict the stable operation of the power system, which manifests as deviations in the load of the power system, such as Figure 8 and Figure 11 Therefore, it is of great significance to study the frequency regulation of the power system when interference occurs.

[0127] like Figure 9 As shown in Figure 2, when a disturbance causes a sudden drop in the power system load, the proposed method can quickly stabilize the system's frequency deviation and output power to the rated value with minimal overshoot, validating the effectiveness of the present invention. A detailed IAE comparison is summarized in Table 2.

[0128] Figure 10 This demonstrates the control effectiveness of the proposed method with the assistance of hydrogen energy storage when interference causes a sudden drop in the power system load. At this point, after a minimal overshoot, the system quickly stabilizes the frequency deviation and output power, effectively coping with the interference condition. This is superior to the situation without hydrogen energy storage assistance, fully demonstrating the effectiveness of hydrogen energy storage assistance in regulating the power system frequency. A detailed IAE comparison is summarized in Table 2. Furthermore, to further verify the superiority of the proposed method, the control results of the baseline DDPG under all operating conditions are summarized in Table 2.

[0129] In addition, the situation when the power system load suddenly increases due to interference is considered, such as Figure 12-13 shown. Figure 12-13 The good control curve shows that the method proposed in the present invention can quickly stabilize the power system frequency deviation to near 0 under the condition of sudden load increase, with a small overshoot and a short adjustment time, which fully verifies the effectiveness of the present invention.

[0130] By adopting the method proposed in the present invention, the power system can cope with conventional stable operating conditions and sudden interference conditions. The participation of hydrogen energy storage can specifically adjust the active output of the power system, thereby alleviating the output and frequency regulation pressure of the unit and further improving the stability of the system.

[0131] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A hydrogen energy storage auxiliary power system control method based on dynamic reward DDPG, characterized in that: The following steps are involved: Build a frequency regulation model for the power system and introduce a hydrogen energy storage system as an auxiliary power regulation resource; Based on a deep reinforcement learning algorithm, a DDPG agent is constructed to interact with the power system, extract features from the system state, and generate an optimal control strategy. The DDPG agent uses dynamic rewards that change over the training cycle. According to the optimal control strategy, the power output of the hydrogen energy storage system is adjusted in real time to ensure the stability of the system frequency.

2. A hydrogen energy storage auxiliary power system control method based on dynamic reward DDPG according to claim 1, characterized in that: The frequency regulation model of the power system is: Where, X gi 、P gi 、P li , Δf i with u i They are governor position increment, turbine output, load change, system frequency deviation and controller input respectively; T gi 、T ti 、T pi are the time constants of the governor, steam turbine, and generator respectively; K pi is the gain coefficient; R i is the speed regulator coefficient.

3. A hydrogen energy storage auxiliary power system control method based on dynamic reward DDPG according to claim 1, characterized in that: The hydrogen energy storage system includes a hydrogen storage tank, an electrolyzer and a hydrogen fuel cell. The corresponding model is expressed as follows: 0≤P el,t ≤P el,max AND ce,t =And el,t ·η ce 0≤E ce,t ≤E ce,max 0≤E out,t ≤E out,max Where, E el,t is the hydrogen production of the electrolyzer, P el,t is the input power of the electrolyzer at time t, η el is the electrolysis efficiency of the electrolytic cell, is the calorific value of hydrogen, Δt is the time coefficient, P el,max is the maximum input power that the electrolytic cell can withstand, E ce,t is the amount of hydrogen compressed at time t, η ce is the compression efficiency of hydrogen, is the hydrogen content of the hydrogen energy storage model at time t, E out,t is the hydrogen output of the hydrogen storage tank at time t, is the minimum hydrogen storage capacity of the hydrogen storage tank, is the maximum hydrogen storage capacity of the hydrogen storage tank, E ce,max E is the maximum value of hydrogen input that the hydrogen storage tank can accept. out,max It is the maximum value of hydrogen output that the hydrogen storage tank can accept.

4. A hydrogen energy storage auxiliary power system control method based on dynamic reward DDPG according to claim 1, characterized in that: The regulation process of the hydrogen energy storage system as an auxiliary power regulation resource is as follows: when the unit output reaches its peak, the excess output is electrolyzed in the electrolyzer to achieve the conversion of electricity into hydrogen. The converted hydrogen energy is compressed and stored in the hydrogen storage tank and provided to hydrogen-consuming equipment on demand. In the power system, the power system's peak-shaving and valley-filling are achieved through the electricity-hydrogen-electricity cycle: Where, P HFC (t) and η HFC They are the hydrogen input power, output power and conversion efficiency of the hydrogen fuel cell respectively.

5. The method for controlling a hydrogen energy storage auxiliary power system based on dynamic reward DDPG according to claim 1, characterized in that: When the hydrogen energy storage system participates in auxiliary power regulation, the frequency regulation of the power system is: Where, P gi 、P li 、P hi , Δf i are respectively turbine output, load change, power output of hydrogen energy storage system, and system frequency deviation; T pi is the generator time constant; K pi is the gain coefficient.

6. A hydrogen energy storage auxiliary power system control method based on dynamic reward DDPG according to claim 1, characterized in that: The state space of the DDPG agent is the system frequency deviation.

7. A hydrogen energy storage auxiliary power system control method based on dynamic reward DDPG according to claim 1, characterized in that: The action space of the DDPG agent is the correction value of the system frequency, that is, the speed regulator control value.

8. The method for controlling a hydrogen energy storage auxiliary power system based on dynamic reward DDPG according to claim 1, characterized in that: The dynamic rewards are: In the formula, a determines the specific size of each reward, which is determined by multiple experiments, e and o are the frequency deviation and the current training target respectively, and y i with y i ′ are the actual output value and the reference value, ε is a parameter used to prevent the reward value from being too large, and n is the power of the training target.

9. The method for controlling a hydrogen energy storage auxiliary power system based on dynamic reward DDPG according to claim 1, characterized in that: The DDPG agent training process is as follows: For the current state of the environment, the action main network outputs the action under the current strategy; Evaluate the main network to obtain the evaluation result of the current action and calculate the current loss based on the evaluation result; Optimize the parameters of the action main network with the goal of minimizing the current loss; The exponential moving average method is used to update the parameters of the action target network; Based on the environmental state at the next moment, the action target network outputs the next action; The evaluation target network obtains the evaluation result of the next moment according to the next action and the environmental state at the next moment, and calculates the loss of the next moment based on the evaluation result; Optimize and evaluate the parameters of the main network with the goal of minimizing the loss at the next moment; The exponential moving average method is used to update the parameters of the evaluation target network.

10. A hydrogen energy storage auxiliary power system control system based on dynamic reward DDPG, characterized in that: include: Frequency regulation model construction module: Builds a frequency regulation model for the power system and introduces a hydrogen energy storage system as an auxiliary power regulation resource; Control strategy generation module: Based on the deep reinforcement learning algorithm, a DDPG agent is constructed to interact with the power system, extract features from the system state, and generate the optimal control strategy. The DDPG agent is trained using dynamic rewards that change over the training cycle. Control module: Based on the optimal control strategy, it adjusts the power output of the hydrogen energy storage system in real time to ensure the stability of the system frequency.

Images