Hybrid electrolytic cell cluster control method, device and equipment and storage medium

Abstract

The invention discloses a hybrid electrolytic cell cluster control method, device and equipment and a storage medium. The method comprises the following steps: constructing a hybrid electrolytic cell system model corresponding to a hybrid electrolytic cell cluster; based on historical data of a hybrid electrolytic cell cluster, adopting a depth deterministic strategy gradient reinforcement learning algorithm to perform offline training on the hybrid electrolytic cell system model by taking maximization of an operation index as a target to obtain a trained hybrid electrolytic cell system model; the historical data comprises wind and light data at historical moments; inputting current data of a mixed electrolytic cell cluster into the trained mixed electrolytic cell system model to obtain control data of the mixed electrolytic cell cluster; the current data comprises the wind and light data at the current moment. The intelligent level and the self-adaptive capacity of the control method can be improved, the accuracy of the control power of the hybrid electrolytic cell cluster is improved, and then the production efficiency of the hybrid electrolytic cell cluster is improved.

Description

Technical Field

[0001] The present application relates to the technical field of hybrid electrolyzers, and in particular to a hybrid electrolyzer cluster control method, device, equipment and storage medium. Background Art

[0002] In recent years, wind and solar energy have become an important part of the global energy transformation. However, the volatility of wind and solar energy has brought challenges to the stable operation of the power system. In order to solve this problem, integrated energy systems containing hybrid electrolyzers have gradually attracted attention. This system can effectively balance the supply and demand of electricity and improve the production efficiency of hydrogen by integrating multiple electrolyzer technologies with energy storage equipment, thus providing a new solution for achieving sustainable energy supply. Among them, the various electrolyzer technologies include alkaline electrolyzers and proton exchange membrane (PEM) electrolyzers.

[0003] The control technology for hybrid electrolyzer clusters plays an important role in the integrated energy system of hybrid electrolyzers. However, existing hybrid electrolyzer cluster control methods mostly rely on simple start-stop strategies and average distribution strategies. Such strategies lack intelligence and adaptive capabilities, resulting in the difficulty of the integrated energy system of hybrid electrolyzers to maintain optimal operating conditions when energy supply fluctuates and load demand changes. In addition, the existing hybrid electrolyzer cluster control methods (such as simple start-stop strategies and average distribution strategies) ignore the physical characteristics of the hybrid electrolyzer cluster itself during the scheduling strategy solution process, which will result in the control power of the hybrid electrolyzer cluster obtained by the solution being inaccurate, which in turn leads to low production efficiency of the hybrid electrolyzer cluster. Summary of the invention

[0004] Based on the above problems, the present application provides a hybrid electrolytic cell cluster control method, the purpose of which is to improve the intelligence level and adaptability of the control method, improve the accuracy of the control power of the hybrid electrolytic cell cluster, and thereby improve the production efficiency of the hybrid electrolytic cell cluster.

[0005] The embodiments of the present application disclose the following technical solutions:

[0006] The first aspect of the present application provides a hybrid electrolyzer cluster control method, the method comprising:

[0007] Constructing a hybrid electrolyzer system model corresponding to a hybrid electrolyzer cluster; the hybrid electrolyzer cluster includes a photovoltaic device, a wind turbine, an energy storage battery, an electrolyzer set and a hydrogen storage tank; the electrolyzer set includes an alkaline electrolyzer and a PEM electrolyzer; the construction of the hybrid electrolyzer system model depends on constraints and physical characteristics of the electrolyzer set;

[0008] Based on the historical data of the hybrid electrolyzer cluster, a deep deterministic policy gradient reinforcement learning algorithm is used to perform offline training on the hybrid electrolyzer system model with the goal of maximizing the operating indicators to obtain a trained hybrid electrolyzer system model; the historical data includes wind and solar data at historical moments; the wind and solar data includes photovoltaic power, wind turbine power, energy storage battery capacity, hydrogen load demand, hydrogen storage tank pressure and electricity price data;

[0009] The current data of the hybrid electrolyzer cluster is input into the trained hybrid electrolyzer system model to obtain the control data of the hybrid electrolyzer cluster; the current data includes the wind and solar data at the current moment; the control data includes the power allocation ratio of the energy storage battery, the power allocation ratio of the electrolyzer set and the power of the grid.

[0010] Optionally, the hybrid electrolyzer system model includes:

[0011] P PV,t =η PV A PV IR

[0012]

[0013] P b,t =U b I b,t

[0014]

[0015] Where P PV,t , η PV , A PV and IR are the power, efficiency, effective area and radiation intensity of the photovoltaic device at time t, respectively; P WT,t 、v in 、v r 、v out 、v t and P r are the power, cut-in wind speed, rated wind speed, cut-out wind speed, real-time wind speed and rated power of the wind turbine at time t; C t The capacity of the energy storage battery at time t; ρ, S and CP are the air density, the area covered by the wind turbine and the performance coefficient respectively; P b,t ,I b,t , U b , η c and η d are the charge and discharge power, charge and discharge current, charge and discharge voltage, and charge efficiency and discharge efficiency of the energy storage battery at time t; m alk,i,t , P alk,i,t , U alk,i,t and η alkare the hydrogen production, power, voltage and Faraday efficiency of the i-th alkaline electrolyzer at time t; m pem,i,t , P pem,i,t , U pem,i,t and η pem are the hydrogen production, power, voltage and Faraday efficiency of the i-th PEM electrolyzer at time t; n, F and R are the molar mass of hydrogen, the number of transferred electrons per mole, the Faraday constant and the universal gas constant, respectively; P pre t 、z t , T H and V H are the pressure, compression coefficient, temperature and volume of the hydrogen storage tank at time t; m load,t is the hydrogen load at time t; Δt is the time interval; M is the molar mass of hydrogen.

[0016] Optionally, the constraint condition is:

[0017] 0≤P PV,t ≤P PV,max

[0018] 0≤C t ≤C max

[0019] -P b,max ≤P b,t ≤P b,max

[0020] 0≤P alk,i,t ≤P alk,max

[0021] 0≤P pem,i,t ≤P pem,max

[0022] P pre min ≤P pre t ≤P pre max

[0023] -P E,max ≤P E,t ≤P E,max

[0024]

[0025] Where P PV,t is the power of the wind turbine at time t; P PV,max is the maximum output power of the photovoltaic device; C tis the capacity of the energy storage battery at time t; C max and P b,max are the maximum capacity and maximum charge and discharge power of the energy storage battery respectively; P b,t is the charging and discharging power of the energy storage battery at time t; P alk,max and P pem,max are the maximum power of alkaline electrolyzer and PEM electrolyzer respectively; P alk,i,t is the power of the i-th alkaline electrolytic cell at time t; P pem,i,t is the power of the i-th PEM electrolyzer at time t; P pre t is the pressure of the hydrogen storage tank at time t; P pre min and P pre max are the minimum pressure and maximum pressure of the hydrogen storage tank respectively; P E,t P is the power purchased or sold at time t; E,max P is the maximum power of electricity purchased or sold; WT,t is the power of the photovoltaic device at time t; P PV,t is the power of the wind turbine at time t.

[0026] Optionally, the physical characteristics of the electrolytic cell set are:

[0027] U alk,i,t+1 =U alk,i,t +η alk,ht

[0028] U pem,i,t+1 =U pem,i,t +η pem,ht

[0029] U alk,i,t+1 =U alk,i,t +η alk,de Δt

[0030] U pem,i,t+1 =U pem,i,t +η pem,de Δt

[0031]

[0032] Where U alk,i,t is the voltage of the ith alkaline electrolytic cell at time t; U pem,i,t is the voltage of the i-th PEM electrolyzer at time t; η alk ,h t and η pem, ht They are the hot start voltage increase for alkaline electrolyzer and PEM electrolyzer respectively; Δt is the time interval; t resis the time resolution; η alk,de ,t alk,cst 、CST alk ,t alk,hst and HST alk are the voltage decay rate, cold start time, number of cold start time steps, hot start time and number of hot start time steps of the alkaline electrolyzer respectively; η pem,de ,t pem,cst 、CST pem ,t pem,hst and HST pem They are the voltage decay rate, cold start time, cold start time steps, hot start time and hot start time steps of the PEM electrolyzer.

[0033] Optionally, the hybrid electrolyzer system model is offline trained using a deep deterministic policy gradient reinforcement learning algorithm based on the historical data of the hybrid electrolyzer cluster with the goal of maximizing the operating index to obtain a trained hybrid electrolyzer system model, specifically including:

[0034] Establishing environmental parameters for reinforcement learning operation based on the hybrid electrolyzer system model;

[0035] Let the value of t be 1;

[0036] Let the value of k be 1;

[0037] Initialize the parameters of the experience pool, actor network, and critic network as the experience pool, actor network, and critic network of the kth iteration;

[0038] Inputting historical data of the hybrid electrolyzer cluster into the critic network of the kth iteration, and obtaining the state space at time t based on the environment parameters of the reinforcement learning operation;

[0039] According to the state space at time t and based on the actor network of the kth iteration, the action space to be processed at time t is obtained;

[0040] Add the adaptive noise to the action space to be processed at time t to obtain the action space at time t;

[0041] Input the action space at time t into the environment parameters of the reinforcement learning operation to obtain the state space at time t+1 and the reward at time t;

[0042] Construct a quadruple based on the state space at time t, the pending action space at time t, the state space at time t+1, and the reward at time t, and store the quadruple in the experience storage pool of the kth iteration;

[0043] Determine whether the number of quadruplets in the experience storage pool of the kth iteration is greater than a preset capacity, and obtain a first determination result;

[0044] If the first judgment result is no, then the value of t is increased by 1, and the process returns to the step of inputting the historical data of the hybrid electrolyzer cluster into the critic network of the kth iteration, and obtaining the state space at time t based on the environmental parameters of the reinforcement learning operation;

[0045] If the first judgment result is yes, then the parameters of the actor network and the critic network of the kth iteration are updated according to the adaptive learning rate using a soft update strategy to obtain an updated actor network and a critic network, and it is determined whether the maximum number of iterations has been reached to obtain a second judgment result;

[0046] If the second judgment result is yes, the training ends, and the parameters of the hybrid electrolyzer system model are updated based on the updated actor network and the updated critic network to obtain a trained hybrid electrolyzer system model;

[0047] If the second judgment result is no, then judging whether the maximum time step is reached, and obtaining a third judgment result;

[0048] If the third judgment result is no, then the value of t is increased by 1, and the step of obtaining the action space to be processed at time t based on the state space at time t and the actor network of the kth iteration is returned;

[0049] If the third judgment result is yes, the updated actor network and critic network are used as the actor network and critic network of the k+1th iteration, the value of k is increased by 1, and the step of inputting the historical data of the hybrid electrolyzer cluster into the critic network of the kth iteration is returned, and based on the environmental parameters of the reinforcement learning operation, the state space at time t is obtained.

[0050] Optionally, the environmental parameters are:

[0051] s t =(P PV,t ,P WT,t ,C t ,EP t ,m load,t ,P pre t )

[0052] a' t =(P b,t ,P alk,1,t ,P alk,2,t ggg,P pem,1,t,P pem,2,t ggg,P E,t )

[0053]

[0054] In the formula, s t 、a' t and r t are the state space, pending action space and reward at time t respectively; P PV,t is the power of the wind turbine at time t; P WT,t is the power of the photovoltaic device at time t; C t The capacity of the energy storage battery at time t; EP t is the electricity price at time t; m load,t is the hydrogen load at time t; P pre t is the pressure of the hydrogen storage tank at time t; P b,t is the charging and discharging power of the energy storage battery at time t; P alk,1,t is the power of the first alkaline electrolytic cell at time t; P pem,1,t is the power of the first PEM electrolyzer at time t; P E,t is the power purchased or sold at time t; λ1 is the incentive coefficient of the power purchase cost or power sales income; Δt is the time interval; λ2 is the reward coefficient for the sum of hydrogen production of all alkaline electrolyzers; m alk ,i ,t is the hydrogen production of the i-th alkaline electrolyzer at time t; λ3 is the bonus coefficient of the sum of hydrogen production of all PEM electrolyzers; m pem,i,t is the hydrogen production of the i-th PEM electrolyzer at time t; λ4 is the penalty coefficient for over-capacity of the energy storage battery; C max is the maximum capacity of the energy storage battery; λ5 is the penalty coefficient for over-limit pressure of the hydrogen storage tank; P pre min and P pre max are the minimum and maximum pressures of the hydrogen storage tank respectively; λ6 is the power imbalance penalty coefficient; P alk,i,t is the power of the i-th alkaline electrolyzer at time t.

[0055] Optionally, the hybrid electrolyzer cluster control method further includes:

[0056] Acquire real-time data of the hybrid electrolyzer cluster; the real-time data includes wind and solar data collected during the operation of the hybrid electrolyzer cluster;

[0057] Based on the real-time data of the hybrid electrolyzer cluster and the environmental parameters, a deep deterministic policy gradient reinforcement learning algorithm is used to update the parameters of the trained hybrid electrolyzer system model with the purpose of maximizing operating indicators.

[0058] A second aspect of the present application provides a hybrid electrolyzer cluster control device, the device comprising:

[0059] A construction unit is used to construct a hybrid electrolyzer system model corresponding to a hybrid electrolyzer cluster; the hybrid electrolyzer cluster includes a photovoltaic device, a wind turbine, an energy storage battery, an electrolyzer set and a hydrogen storage tank; the electrolyzer set includes an alkaline electrolyzer and a PEM electrolyzer; the construction of the hybrid electrolyzer system model depends on constraint conditions and physical characteristics of the electrolyzer set;

[0060] A training unit is used to perform offline training on the hybrid electrolyzer system model based on historical data of the hybrid electrolyzer cluster using a deep deterministic policy gradient reinforcement learning algorithm with the goal of maximizing the operating index, so as to obtain a trained hybrid electrolyzer system model; the historical data includes wind and solar data at historical moments; the wind and solar data includes photovoltaic power, wind turbine power, energy storage battery capacity, hydrogen load demand, hydrogen storage tank pressure and electricity price data;

[0061] A control unit is used to input the current data of the hybrid electrolyzer cluster into the trained hybrid electrolyzer system model to obtain the control data of the hybrid electrolyzer cluster; the current data includes the wind and solar data at the current moment; the control data includes the power allocation ratio of the energy storage battery, the power allocation ratio of the electrolyzer set and the power of the grid.

[0062] The third aspect of the present application provides a computer device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the hybrid electrolyzer cluster control method provided in the first aspect of the present application.

[0063] The fourth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the hybrid electrolytic cell cluster control method provided in the first aspect of the present application is implemented.

[0064] Compared with the prior art, this application has the following beneficial effects:

[0065] The present application comprehensively considers the constraints and physical characteristics to construct a hybrid electrolyzer system model corresponding to the hybrid electrolyzer cluster; it overcomes the problem of low production efficiency of the hybrid electrolyzer cluster caused by ignoring the physical characteristics of the hybrid electrolyzer cluster itself in the prior art. Furthermore, the hybrid electrolyzer system model is trained offline by using a deep deterministic policy gradient reinforcement learning algorithm through historical data and established environmental parameters for the purpose of maximizing operating indicators, which can overcome the problem of lack of intelligence and adaptability in the prior art, because the hybrid electrolyzer system model is trained using a deep deterministic policy gradient reinforcement learning algorithm, which can make the trained hybrid electrolyzer system model have certain intelligence and adaptability, and further improve the production efficiency of the hybrid electrolyzer cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0067] Figure 1 A flow chart of a hybrid electrolyzer cluster control method provided in an embodiment of the present application;

[0068] Figure 2 A flowchart of a deep deterministic policy gradient reinforcement learning algorithm provided in an embodiment of the present application;

[0069] Figure 3 A structural diagram of the actor network provided in the embodiment of the present application;

[0070] Figure 4 A structural diagram of a critic network provided in an embodiment of the present application;

[0071] Figure 5 A schematic diagram of the application of the DDPG algorithm provided in the embodiment of the present application;

[0072] Figure 6 A schematic diagram of the working of the hybrid electrolyzer system model provided in the embodiment of the present application;

[0073] Figure 7 A structural diagram of a hybrid electrolytic cell cluster control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] As described above, current hybrid electrolyzer cluster control methods mostly rely on simple start-stop strategies and average distribution strategies. Such strategies lack intelligence and adaptability, and ignore the physical characteristics of the hybrid electrolyzer cluster itself, resulting in inaccurate control power of the hybrid electrolyzer cluster, which in turn leads to low production efficiency of the hybrid electrolyzer cluster.

[0075] After research, the inventors found that building a hybrid electrolyzer system model based on the physical characteristics of the electrolyzer set can improve the intelligence level and adaptability of the control method, and that offline training of the hybrid electrolyzer system model using a deep deterministic policy gradient reinforcement learning algorithm can improve the accuracy of the control power of the hybrid electrolyzer cluster, thereby improving the production efficiency of the hybrid electrolyzer cluster.

[0076] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0077] Figure 1 is a flow chart of the hybrid electrolyzer cluster control method, now see Figure 1 The hybrid electrolyzer cluster control method is described as follows:

[0078] S101. Construct a hybrid electrolyzer system model corresponding to the hybrid electrolyzer cluster.

[0079] The hybrid electrolyzer cluster includes a photovoltaic device, a wind turbine, an energy storage battery, an electrolyzer set and a hydrogen storage tank; the electrolyzer set includes an alkaline electrolyzer and a PEM electrolyzer; and the construction of the hybrid electrolyzer system model depends on constraints and the physical characteristics of the electrolyzer set.

[0080] The hybrid electrolyser system model includes:

[0081] P PV,t =η PV A PV IR

[0082]

[0083] P b,t =U b I b,t

[0084]

[0085] Where PPV,t , η PV , A PV and IR are the power, efficiency, effective area and radiation intensity of the photovoltaic device at time t, respectively; P WT,t 、v in 、v r 、v out 、v t and P r are the power, cut-in wind speed, rated wind speed, cut-out wind speed, real-time wind speed and rated power of the wind turbine at time t; C t The capacity of the energy storage battery at time t; ρ, S and CP are the air density, the area covered by the wind turbine and the performance coefficient respectively; P b,t ,I b,t , U b , η c and η d are the charge and discharge power, charge and discharge current, charge and discharge voltage, and charge efficiency and discharge efficiency of the energy storage battery at time t; m alk,i,t , P alk,i,t , U alk,i,t and η alk are the hydrogen production, power, voltage and Faraday efficiency of the i-th alkaline electrolyzer at time t; m pem,i,t , P pem,i,t , U pem,i,t and η pem are the hydrogen production, power, voltage and Faraday efficiency of the i-th PEM electrolyzer at time t; n, F and R are the molar mass of hydrogen, the number of transferred electrons per mole, the Faraday constant and the universal gas constant, respectively; P pre t 、z t , T H and V H are the pressure, compression coefficient, temperature and volume of the hydrogen storage tank at time t; m load,t is the hydrogen load at time t; Δt is the time interval; M is the molar mass of hydrogen.

[0086] There is no limit on the rated power and quantity of alkaline electrolyzers and the rated power and quantity of PEM electrolyzers, which can be determined according to the local wind and solar resources and the capacity range of the power grid; in addition, the presence of energy storage batteries and hydrogen storage tanks makes the hybrid electrolyzer system model have flexible adjustment capabilities, enhances the stability and safety of the hybrid electrolyzer system model, and cooperates with the power grid to improve the economic benefits of electric energy.

[0087] The constraints are:

[0088] 0≤P PV,t ≤PPV,max

[0089] 0≤C t ≤C max

[0090] -P b,max ≤P b,t ≤P b,max

[0091] 0≤P alk,i,t ≤P alk,max

[0092] 0≤P pem,i,t ≤P pem,max

[0093] P pre min ≤P pre t ≤P pre max

[0094] -P E,max ≤P E,t ≤P E,max

[0095]

[0096] Where P PV,max is the maximum output power of the photovoltaic device; C max and P b,max are the maximum capacity and maximum charge and discharge power of the energy storage battery respectively; P alk,max and P pem,max are the maximum power of alkaline electrolyzer and PEM electrolyzer respectively; P pre min and P pre max are the minimum pressure and maximum pressure of the hydrogen storage tank respectively; P E,t P is the power purchased or sold at time t; E,max The maximum power for purchasing or selling electricity.

[0097] At present, the electrolyzer is generally considered to be an ideal hydrogen producer when modeling. In fact, there are many dynamic characteristics of the electrolyzer that are very important for the stable operation and scheduling judgment of the system. For example, the voltage of the electrolyzer usually increases with the operating time, which is closely related to the quality of its membrane electrode material; in addition, the electrolyzer has a certain response time. It takes time to warm up from the standby state to the hot standby state, and then it is powered on to produce hydrogen, and gradually reaches a steady state after the hot start process; but during the hot start process, because the electrolyzer and its accessories have not reached a stable working state, additional voltage increases are generated. At this time, the power consumption of producing the same hydrogen will increase relative to the normal working state. Therefore, the physical characteristics of the electrolyzer set considering the physical properties and fluctuation adaptability of the alkaline electrolyzer and PEM electrolyzer are as follows:

[0098] U alk,i,t+1 =U alk,i,t +η alk,ht

[0099] U pem,i,t+1 =U pem,i,t +η pem,ht

[0100] U alk,i,t+1 =U alk,i,t +η alk,de Δt

[0101] U pem,i,t+1 =U pem,i,t +η pem,de Δt

[0102]

[0103] Where η alk,ht and η pem,ht They are the hot start voltage increase for alkaline electrolyzer and PEM electrolyzer respectively; Δt is the time interval; t res is the time resolution, that is, every t res time, the output is updated once according to the current state; η alk,de ,t alk,cst 、CST alk ,t alk,hst and HST alk are the voltage decay rate, cold start time, number of cold start time steps, hot start time and number of hot start time steps of the alkaline electrolyzer respectively; η pem,de ,t pem,cst 、CST pem ,t pem,hst and HST pem They are the voltage decay rate, cold start time, cold start time steps, hot start time and hot start time steps of the PEM electrolyzer.

[0104] η alk,ht and η pem,ht It is only considered when the electrolyzer is in hot start state. Because the electrolyzer does not produce hydrogen during cold start, enters hot standby state after cold start, and enters stable working state after hot start. Hydrogen is produced during hot start.

[0105] S102. Based on the historical data of the hybrid electrolyzer cluster, a deep deterministic policy gradient reinforcement learning algorithm (DDPG) is used to perform offline training on the hybrid electrolyzer system model with the goal of maximizing the operating indicators to obtain a trained hybrid electrolyzer system model.

[0106] The historical data includes wind and solar data at historical moments; the wind and solar data includes photovoltaic power, wind turbine power, energy storage battery capacity, hydrogen load demand, hydrogen storage tank pressure and electricity price data.

[0107] The DDPG algorithm has become one of the most popular reinforcement learning algorithms for complex energy systems because of its ability to effectively handle continuous action spaces and its ability to learn online. However, the algorithm still has problems such as slow convergence, overly sensitive to initial conditions, and insufficient exploration.

[0108] In order to solve the defects of the DDPG algorithm, this application designs adaptive noise, adaptive learning rate and custom network architecture based on the DDPG algorithm, so that the intelligent agent can dynamically optimize the training process and enhance its adaptability to the system model. Figure 2 The flowchart of the deep deterministic policy gradient reinforcement learning algorithm shown in the figure explains the deep deterministic policy gradient reinforcement learning algorithm:

[0109] S201. Establishing environmental parameters for reinforcement learning operation based on the hybrid electrolyzer system model.

[0110] S202. Set the value of t to 1, and set the value of k to 1.

[0111] S203, initializing the parameters of the experience storage pool, the actor network, and the critic network as the experience storage pool, the actor network, and the critic network of the kth iteration.

[0112] Figure 3 is the structure diagram of the actor network, such as Figure 3 The actor network shown adopts a custom network architecture, and the network structure of the actor network can be four fully connected layers, three batch normalization layers, three ELU activation functions, and a tanh activation function.

[0113] The state space is input into the actor network, and the data is processed by the fully connected layer to integrate the features. Then it is normalized by the batch normalization layer, and then processed by the ELU activation function. Then it enters the fully connected layer, batch normalization layer and ELU activation function again to get the processed data. Then it enters the fully connected layer, batch normalization layer and ELU activation function for the third time to get the data after three processing. The data after three processing is input into the fully connected layer for feature integration, and finally the tanh activation function is used to get the final action space.

[0114] However, in specific applications, the actor network can also include layer normalization layers, group normalization layers, other activation layers, layer-by-layer adaptive learning rate scaling layers, etc.

[0115] Figure 4 is the structure diagram of the critic network, such as Figure 4 The critic network shown uses a custom network architecture, and the network structure of the actor network can be four fully connected layers, three batch normalization layers, and three ELU activation functions.

[0116] The state-action space is input into the critic network, and the data is processed by the fully connected layer to integrate the features. Then it is normalized by the batch normalization layer, and then processed by the ELU activation function. Then it enters the fully connected layer, batch normalization layer and ELU activation function again to obtain the processed data. Then it enters the fully connected layer, batch normalization layer and ELU activation function for the third time to obtain the data after three processing. Finally, the data after three processing is input into the fully connected layer for feature integration to obtain the predicted total reward under the action value output by the current state acquisition actor network.

[0117] However, in specific applications, the critic network can also include layer normalization layers, group normalization layers, other activation layers, layer-by-layer adaptive learning rate scaling layers, etc.

[0118] Figure 3 and Figure 4 An example of the architecture of a custom actor network and a critic network is given respectively. Compared with the commonly used Relu activation function, the use of the ELU activation function will reduce the impact of bias offset, making the normal gradient closer to the unit natural gradient, thereby accelerating the learning of the mean toward zero, and there will be no problem of dead neurons. In addition, the introduction of the batch normalization layer can accelerate training, stabilize the convergence of the model, and reduce the sensitivity to the initial weights, especially helping to avoid the problem of gradient vanishing or explosion during gradient updates. Through the custom network architecture design, the DDPG algorithm can better adapt to the optimization scheduling problem of complex energy systems, while enhancing the ability of transfer learning and greatly improving the generalization and application capabilities of the model.

[0119] S204, inputting the historical data of the hybrid electrolytic cell cluster into the critic network of the kth iteration, and obtaining the state space at time t based on the environmental parameters of the reinforcement learning operation.

[0120] S205 , obtaining the action space to be processed at time t based on the state space at time t and the actor network of the kth iteration.

[0121] S206 , adding the adaptive noise to the action space to be processed at time t to obtain the action space at time t.

[0122] Adaptive Noise N t It is subject to expectation of 0 and variance of σ 2 The normal distribution of 2 The linear decay is based on the number of training times, as follows:

[0123]

[0124] S207, input the action space at time t into the environment parameters of the reinforcement learning operation to obtain the state space at time t+1 and the reward at time t.

[0125] S208. Construct a quadruple based on the state space at time t, the pending action space at time t, the state space at time t+1, and the reward at time t, and store the quadruple into the experience storage pool of the kth iteration.

[0126] S209: Determine whether the number of quadruplets in the experience storage pool of the kth iteration is greater than a preset capacity, and obtain a first determination result.

[0127] S210. If the first judgment result is no, add 1 to the value of t and return to S204.

[0128] S211. If the first judgment result is yes, a soft update strategy is used to update the parameters of the actor network and the critic network of the kth iteration according to an adaptive learning rate to obtain an updated actor network and a critic network.

[0129] The actor network and critic network parameters are updated as follows:

[0130] y t =r t +γQ'(s t+1 ,μ'(s t+1 |θ')|ω')

[0131] L Q (ω)=E[(Q(s t ,μ(s t |θ)|ω)-yt ) 2 ]

[0132] ω t+1 =ω t +α▽ ω L Q (ω)

[0133]

[0134] θ t+1 =θ t +α▽ θ J(θ).

[0135] In the formula, μ(s t |θ) is the actor network; θ is the network parameter of the actor network; μ'(s t+1 |θ') is the target actor network; θ' is the network parameter of the target actor network; Q(s,a|ω) is the critic network; ω is the network parameter of the critic network; Q(s,a|ω') is the target critic network; ω' is the network parameter of the target critic network; γ is the discount factor, which represents the present value of future returns.

[0136] The soft update strategy is as follows, where τ is the soft update factor.

[0137] ω' t+1 =(1-τ)ω' t +τω t+1

[0138] θ' t+1 =(1-τ)θ' t +τθ t+1 .

[0139] S212, determine whether the maximum number of iterations k has been reached max , and obtain the second judgment result.

[0140] S213. If the second judgment result is yes, the training ends, and the parameters of the hybrid electrolyzer system model are updated based on the updated actor network and the updated critic network to obtain a trained hybrid electrolyzer system model.

[0141] S214: If the second judgment result is no, then judge whether the maximum time step is reached to obtain a third judgment result. If the third judgment result is no, then return to S210.

[0142] S215. If the third judgment result is yes, the updated actor network and critic network are used as the actor network and critic network of the k+1th iteration, the value of k is increased by 1, and the process returns to S204.

[0143] The environmental parameters are:

[0144] s t =(P PV,t ,P WT,t ,C t ,EP t ,m load,t ,P pre t )

[0145] a' t =(P b,t ,P alk,1,t ,P alk,2,t ggg,P pem,1,t ,P pem,2,t ggg,P E,t )

[0146]

[0147] In the formula, s t 、a' t and r t are the state space, pending action space and reward at time t; EP t is the electricity price at time t; P pem,1,t is the power of the first PEM electrolyzer at time t; λ1 is the incentive coefficient for the cost of electricity purchase or the income from electricity sales; λ2 is the incentive coefficient for the sum of the hydrogen production of all alkaline electrolyzers; λ3 is the incentive coefficient for the sum of the hydrogen production of all PEM electrolyzers; λ4 is the penalty coefficient for over-capacity of the energy storage battery; λ5 is the penalty coefficient for over-pressure of the hydrogen storage tank; λ6 is the penalty coefficient for power imbalance.

[0148] The design of the reward function reflects that the hybrid electrolyzer system model aims to maximize economic benefits. At the same time, the reward function coefficient can be flexibly adjusted, so that the hybrid electrolyzer system model can achieve multiple goals. If λ1=0, the hybrid electrolyzer system model will aim to maximize hydrogen production.

[0149] Figure 5 This is a schematic diagram of the application of the DDPG algorithm. Figure 5 Explain the application of the DDPG algorithm:

[0150] Initialize θ, θ', ω, ω', and set k = 0. Initialize the parameters of the actor network and the critic network, initialize the parameters of the target actor network and the critic network, initialize the experience replay pool, and set the total number of training times k = 0.

[0151] Initialize state s0 and set t = 0; for each round, initialize the state space in the reinforcement learning running environment to get s t , let t=0.

[0152] Actor Network a' t ; For each training step, let t = t + 1, and get the execution action a' from the actor network according to st t .

[0153] Add custom noise N t ; and add adaptive noise N t Get a t =a' t +N t .

[0154] Hybrid electrolyzer system model reinforcement learning operating environment; enter the reinforcement learning operating environment to get the next state s t+1 With the current reward r t .

[0155] s t , a t ,s t+1 , r t ; The quaternion (s t , a t ,s t+1 , r t ) is stored in the experience replay pool.

[0156] Determine whether the experience pool data is greater than or equal to the set value. If not, update the state s. t , let t = t + 1, and return "Actor network a' t If so, the experience pool randomly samples B groups of quadruplets, and the loss function is calculated through the four network output values.

[0157] Update θ, ω according to the learning rate α.

[0158] Soft update θ' and ω'.

[0159] Determine whether the maximum number of training times k has been reached max If yes, the training is completed; if no, determine whether the maximum time has been reached. If yes, return to the initialization state s0 and set t = 0. If no, update the state s t , let t = t + 1, and return "Actor network a' t” steps.

[0160] S103, inputting the current data of the hybrid electrolytic cell cluster into the trained hybrid electrolytic cell system model to obtain the control data of the hybrid electrolytic cell cluster.

[0161] The current data includes the wind and solar data at the current moment; the control data includes the power allocation ratio of the energy storage battery, the power allocation ratio of the electrolyzer set and the power of the power grid.

[0162] The hybrid electrolyzer cluster control method further includes:

[0163] Acquire real-time data of the hybrid electrolyzer cluster, including wind and solar data collected during the operation of the hybrid electrolyzer cluster.

[0164] Based on the real-time data of the hybrid electrolyzer cluster and the environmental parameters, a deep deterministic policy gradient reinforcement learning algorithm is used to update the parameters of the trained hybrid electrolyzer system model with the purpose of maximizing operating indicators.

[0165] Figure 6 This is the working diagram of the hybrid electrolyzer system model. Figure 6 The working process of the hybrid electrolyzer system model is explained:

[0166] According to the wind turbine power, photovoltaic power, hydrogen load demand, energy storage battery capacity, hydrogen storage tank pressure and electricity price data at the current time t, the optimal power allocation method is determined to maximize the system benefit during the entire time period. That is, the optimal scheduling and operation of the hybrid electrolyzer system is achieved while ensuring power balance, the input or output power of each component does not exceed the maximum value, and the hydrogen storage tank pressure and energy storage battery capacity are within the limit range.

[0167] The intelligent agent can continuously learn the optimal action in any state space. The trained hybrid electrolyzer system model obtains real-time data of the state space through the data interface, obtains the optimal scheduling plan and outputs it to the control system, so as to optimize the economic benefits of the hybrid electrolyzer system model and maximize the hydrogen production. In addition, the intelligent agent can also continuously learn in the current environment and continuously update its own parameters, so that it can adapt to the latest changes of the hybrid electrolyzer system model and continuously optimize the operating parameters of the hybrid electrolyzer system model.

[0168] This application fully considers the differences between alkaline electrolyzers and PEM electrolyzers in their physical properties and fluctuation adaptability, so that the hybrid electrolyzer system model containing the hybrid electrolyzer can take into account both the economy and the ability to adapt to wind and solar power fluctuations, thereby enhancing the safety of the system's stable operation.

[0169] This application further improves the stability of the DDPG algorithm through the design of adaptive noise, adaptive learning rate and customized network architecture, thereby dynamically optimizing the training process and enhancing the adaptability of the hybrid electrolyzer system model.

[0170] Through real-time monitoring and data collection, the reinforcement learning agent pre-trained based on historical data can perform real-time optimization and scheduling of the hybrid electrolyzer system model, and learn online and continuously optimize the operating parameters of the electrolyzer during operation, thereby significantly improving the economic benefits and hydrogen production of the hybrid electrolyzer system model.

[0171] Based on the hybrid electrolytic cell cluster control method provided in the aforementioned embodiment, accordingly, a hybrid electrolytic cell cluster control device is also provided in the present application. Figure 7 This is the structural diagram of the hybrid electrolyzer cluster control device. Figure 7 The functions of each unit in the hybrid electrolyzer cluster control device are described as follows:

[0172] A construction unit is used to construct a hybrid electrolyzer system model corresponding to a hybrid electrolyzer cluster. The hybrid electrolyzer cluster includes a photovoltaic device, a wind turbine, an energy storage battery, an electrolyzer set and a hydrogen storage tank; the electrolyzer set includes an alkaline electrolyzer and a PEM electrolyzer; and the construction of the hybrid electrolyzer system model depends on the constraints and the physical characteristics of the electrolyzer set.

[0173] The hybrid electrolyser system model includes:

[0174] P PV,t =η PV A PV IR

[0175]

[0176] P b,t =U b I b,t

[0177]

[0178] The constraints are:

[0179] 0≤P PV,t ≤P PV,max

[0180] 0≤C t ≤C max

[0181] -P b,max ≤P b,t ≤P b,max

[0182] 0≤Palk,i,t ≤P alk,max

[0183] 0≤P pem,i,t ≤P pem,max P pre min ≤P pre t ≤P pre max

[0184] -P E,max ≤P E,t ≤P E,max

[0185]

[0186] The physical characteristics of the electrolytic cell set are: U alk,i,t+1 =U alk,i,t +η alk,ht U pem,i,t+1 =U pem,i,t +η pem,ht U alk,i,t+1 =U alk,i,t +η alk,de ΔtU pem,i,t+1 =U pem,i,t +η pem,de Δt

[0187]

[0188] A training unit is used to perform offline training on the hybrid electrolyzer system model based on historical data of the hybrid electrolyzer cluster using a deep deterministic policy gradient reinforcement learning algorithm with the goal of maximizing operating indicators to obtain a trained hybrid electrolyzer system model; the historical data includes wind and solar data at historical moments; the wind and solar data includes photovoltaic power, wind turbine power, energy storage battery capacity, hydrogen load demand, hydrogen storage tank pressure and electricity price data.

[0189] The training unit includes a construction module, a setting module, an initialization module, a first operation module, a second operation module, a processing module, a third operation module, a storage module, a first judgment module and a second judgment module. The function of each module in the training unit is now described:

[0190] A building module is used to establish environmental parameters for reinforcement learning operation based on the hybrid electrolyzer system model.

[0191] The setting module is used to set the value of t to 1 and the value of k to 1.

[0192] The initialization module is used to initialize the parameters of the experience storage pool, actor network and critic network as the experience storage pool, actor network and critic network of the kth iteration.

[0193] The first operation module is used to input the historical data of the hybrid electrolyzer cluster into the critic network of the kth iteration, and obtain the state space at time t based on the environmental parameters of the reinforcement learning operation.

[0194] The second operation module is used to obtain the action space to be processed at time t based on the state space at time t and the actor network of the kth iteration.

[0195] The processing module is used to add the adaptive noise to the action space to be processed at time t to obtain the action space at time t.

[0196] The third operation module is used to input the action space at time t into the environment parameters of the reinforcement learning operation to obtain the state space at time t+1 and the reward at time t.

[0197] The storage module is used to construct a four-tuple based on the state space at time t, the pending action space at time t, the state space at time t+1, and the reward at time t, and store the four-tuple into the experience storage pool of the kth iteration.

[0198] The first judgment module is used to judge whether the number of quadruplets in the experience storage pool of the kth iteration is greater than the preset capacity, and obtain a first judgment result. If the first judgment result is no, the value of t is increased by 1, and the first operation module is returned.

[0199] The second judgment module is used to, if the first judgment result is yes, use a soft update strategy to update the parameters of the actor network and the critic network of the kth iteration according to the adaptive learning rate to obtain an updated actor network and a critic network, and judge whether the maximum number of iterations has been reached to obtain a second judgment result. If the second judgment result is yes, the training ends, and the parameters of the hybrid electrolyzer system model are updated based on the updated actor network and the updated critic network to obtain a trained hybrid electrolyzer system model.

[0200] The third judgment module is used to judge whether the maximum time step is reached if the second judgment result is no, and obtain a third judgment result. If the third judgment result is no, the value of t is increased by 1, and the second operation module is returned. If the third judgment result is yes, if the second judgment result is no, the updated actor network and critic network are used as the actor network and critic network of the k+1th iteration, the value of k is increased by 1, and the first operation module is returned.

[0201] The environmental parameters are:

[0202] s t =(P PV,t ,P WT,t ,C t ,EP t ,m load,t ,P pre t )

[0203] a' t =(P b,t ,P alk,1,t ,P alk,2,t ggg,P pem,1,t ,P pem,2,t ggg,P E,t )

[0204]

[0205] A control unit is used to input the current data of the hybrid electrolyzer cluster into the trained hybrid electrolyzer system model to obtain the control data of the hybrid electrolyzer cluster; the current data includes the wind and solar data at the current moment; the control data includes the power allocation ratio of the energy storage battery, the power allocation ratio of the electrolyzer set and the power of the grid.

[0206] The hybrid electrolyzer cluster control device also includes:

[0207] The acquisition unit is used to acquire real-time data of the hybrid electrolyzer cluster. The real-time data includes wind and solar data collected during the operation of the hybrid electrolyzer cluster.

[0208] An updating unit is used to update the parameters of the trained hybrid electrolyzer system model based on the real-time data of the hybrid electrolyzer cluster and the environmental parameters by adopting a deep deterministic policy gradient reinforcement learning algorithm with the purpose of maximizing operating indicators.

[0209] A computer device is also provided in an embodiment of the present application. The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a hybrid electrolyzer cluster control method.

[0210] A computer-readable storage medium is also provided in an embodiment of the present application, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, a hybrid electrolytic cell cluster control method is implemented.

[0211] It should be noted that each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device, equipment and storage medium embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0212] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A hybrid electrolyzer cluster control method, characterized in that: The hybrid electrolyzer cluster control method comprises: Constructing a hybrid electrolyzer system model corresponding to a hybrid electrolyzer cluster; the hybrid electrolyzer cluster includes a photovoltaic device, a wind turbine, an energy storage battery, an electrolyzer set and a hydrogen storage tank; the electrolyzer set includes an alkaline electrolyzer and a PEM electrolyzer; the construction of the hybrid electrolyzer system model depends on constraints and physical characteristics of the electrolyzer set; Based on the historical data of the hybrid electrolyzer cluster, a deep deterministic policy gradient reinforcement learning algorithm is used to perform offline training on the hybrid electrolyzer system model with the goal of maximizing the operating indicators to obtain a trained hybrid electrolyzer system model; the historical data includes wind and solar data at historical moments; the wind and solar data includes photovoltaic power, wind turbine power, energy storage battery capacity, hydrogen load demand, hydrogen storage tank pressure and electricity price data; The current data of the hybrid electrolyzer cluster is input into the trained hybrid electrolyzer system model to obtain the control data of the hybrid electrolyzer cluster; the current data includes the wind and solar data at the current moment; the control data includes the power allocation ratio of the energy storage battery, the power allocation ratio of the electrolyzer set and the power of the grid.

2. The hybrid electrolyzer cluster control method according to claim 1, characterized in that: The hybrid electrolyzer system model includes: P PV,t =η PV TO PV GO P b,t =U b AND b,t Where P PV,t , η PV , A PV and IR are the power, efficiency, effective area and radiation intensity of the photovoltaic device at time t, respectively; P WT,t 、v in 、v r 、v out 、v t and P r are the power, cut-in wind speed, rated wind speed, cut-out wind speed, real-time wind speed and rated power of the wind turbine at time t; C t The capacity of the energy storage battery at time t; ρ, S and CP are the air density, the area covered by the wind turbine and the performance coefficient respectively; P b,t ,I b,t , U b , η c and η d are the charge and discharge power, charge and discharge current, charge and discharge voltage, and charge efficiency and discharge efficiency of the energy storage battery at time t; m alk,i,t , P alk,i,t , U alk,i,t and η alk are the hydrogen production, power, voltage and Faraday efficiency of the i-th alkaline electrolyzer at time t; m pem,i,t , P pem,i,t , U pem,i,t and η pem are the hydrogen production, power, voltage and Faraday efficiency of the i-th PEM electrolyzer at time t; n, F and R are the molar mass of hydrogen, the number of transferred electrons per mole, the Faraday constant and the universal gas constant, respectively; P pre t 、z t , T H and V H are the pressure, compression coefficient, temperature and volume of the hydrogen storage tank at time t; m load,t is the hydrogen load at time t; Δt is the time interval; M is the molar mass of hydrogen.

3. The hybrid electrolyzer cluster control method according to claim 1, characterized in that: The constraints are: 0≤P PV,t ≤P PV,max 0≤C t ≤C max -P b,max ≤P b,t ≤P b,max 0≤P alk,i,t ≤P alk,max 0≤P pem,i,t ≤P pem,max P pre min ≤P pre t ≤P pre max -P E,max ≤P E,t ≤P E,max Where P PV,t is the power of the wind turbine at time t; P PV,max is the maximum output power of the photovoltaic device; C t is the capacity of the energy storage battery at time t; C max and P b,max are the maximum capacity and maximum charge and discharge power of the energy storage battery respectively; P b,t is the charging and discharging power of the energy storage battery at time t; P alk,max and P pem,max are the maximum power of alkaline electrolyzer and PEM electrolyzer respectively; P alk,i,t is the power of the i-th alkaline electrolytic cell at time t; P pem,i,t is the power of the i-th PEM electrolyzer at time t; P pre t is the pressure of the hydrogen storage tank at time t; P pre min and P pre max are the minimum pressure and maximum pressure of the hydrogen storage tank respectively; P E,t P is the power purchased or sold at time t; E,max P is the maximum power of electricity purchased or sold; WT,t is the power of the photovoltaic device at time t; P PV,t is the power of the wind turbine at time t.

4. The hybrid electrolyzer cluster control method according to claim 1, characterized in that: The physical characteristics of the electrolyzer set are: IN alk,i,t+1 =U alk,i,t +η alk,ht IN pem,i,t+1 =U pem,i,t +η pem,ht IN alk,i,t+1 =U alk,i,t +η alk,de Δt IN pem,i,t+1 =U pem,i,t +η pem,de Δt Where U alk,i,t is the voltage of the ith alkaline electrolytic cell at time t; U pem,i,t is the voltage of the i-th PEM electrolyzer at time t; η alk,ht and η pem,ht They are the hot start voltage increase for alkaline electrolyzer and PEM electrolyzer respectively; Δt is the time interval; t res is the time resolution; η alk,de ,t alk,cst 、CST alk ,t alk,hst and HST alk They are the voltage decay rate, cold start time, number of cold start time steps, hot start time and number of hot start time steps of the alkaline electrolyzer; η pem,de ,t pem,cst 、CST pem ,t pem,hst and HST pem They are the voltage decay rate, cold start time, cold start time steps, hot start time and hot start time steps of the PEM electrolyzer.

5. The hybrid electrolyzer cluster control method according to any one of claims 1 to 4, characterized in that: The hybrid electrolyzer system model is trained offline using a deep deterministic policy gradient reinforcement learning algorithm based on the historical data of the hybrid electrolyzer cluster with the goal of maximizing the operating index to obtain a trained hybrid electrolyzer system model, specifically including: Establishing environmental parameters for reinforcement learning operation based on the hybrid electrolyzer system model; Let the value of t be 1; Let the value of k be 1; Initialize the parameters of the experience pool, actor network, and critic network as the experience pool, actor network, and critic network of the kth iteration; Inputting historical data of the hybrid electrolyzer cluster into the critic network of the kth iteration, and obtaining the state space at time t based on the environment parameters of the reinforcement learning operation; According to the state space at time t and based on the actor network of the kth iteration, the action space to be processed at time t is obtained; Add the adaptive noise to the action space to be processed at time t to obtain the action space at time t; Input the action space at time t into the environment parameters of the reinforcement learning operation to obtain the state space at time t+1 and the reward at time t; Construct a quadruple based on the state space at time t, the pending action space at time t, the state space at time t+1, and the reward at time t, and store the quadruple in the experience storage pool of the kth iteration; Determine whether the number of quadruplets in the experience storage pool of the kth iteration is greater than a preset capacity, and obtain a first determination result; If the first judgment result is no, then the value of t is increased by 1, and the process returns to the step of inputting the historical data of the hybrid electrolyzer cluster into the critic network of the kth iteration, and obtaining the state space at time t based on the environmental parameters of the reinforcement learning operation; If the first judgment result is yes, then the parameters of the actor network and the critic network of the kth iteration are updated according to the adaptive learning rate using a soft update strategy to obtain an updated actor network and a critic network, and it is determined whether the maximum number of iterations has been reached to obtain a second judgment result; If the second judgment result is yes, the training ends, and the parameters of the hybrid electrolyzer system model are updated based on the updated actor network and the updated critic network to obtain a trained hybrid electrolyzer system model; If the second judgment result is no, then judging whether the maximum time step is reached, and obtaining a third judgment result; If the third judgment result is no, then the value of t is increased by 1, and the step of obtaining the action space to be processed at time t based on the state space at time t and the actor network of the kth iteration is returned; If the third judgment result is yes, the updated actor network and critic network are used as the actor network and critic network of the k+1th iteration, the value of k is increased by 1, and the step of inputting the historical data of the hybrid electrolyzer cluster into the critic network of the kth iteration is returned, and based on the environmental parameters of the reinforcement learning operation, the state space at time t is obtained.

6. The hybrid electrolyzer cluster control method according to claim 5, characterized in that: The environmental parameters are: s t =(P PV,t ,P WT,t ,C t ,EP t ,m load,t ,P pre t ) a' t =(P b,t ,P alk,1,t ,P alk,2,t ggg,P pem,1,t ,P pem,2,t ggg,P E,t ) In the formula, s t 、a' t and r t are the state space, pending action space and reward at time t respectively; P PV,t is the power of the wind turbine at time t; P WT,t is the power of the photovoltaic device at time t; C t The capacity of the energy storage battery at time t; EP t is the electricity price at time t; m load,t is the hydrogen load at time t; P pre t is the pressure of the hydrogen storage tank at time t; P b,t is the charging and discharging power of the energy storage battery at time t; P alk,1,t is the power of the first alkaline electrolytic cell at time t; P pem,1,t is the power of the first PEM electrolyzer at time t; P E,t is the power purchased or sold at time t; λ1 is the incentive coefficient of the power purchase cost or power sales income; Δt is the time interval; λ2 is the reward coefficient for the sum of hydrogen production of all alkaline electrolyzers; m alk ,i ,t is the hydrogen production of the i-th alkaline electrolyzer at time t; λ3 is the bonus coefficient of the sum of hydrogen production of all PEM electrolyzers; m pem, i ,t is the hydrogen production of the i-th PEM electrolyzer at time t; λ4 is the penalty coefficient for over-capacity of energy storage battery; C max is the maximum capacity of the energy storage battery; λ5 is the penalty coefficient for over-limit pressure of hydrogen storage tank; P pre min and P pre max are the minimum and maximum pressures of the hydrogen storage tank respectively; λ6 is the power imbalance penalty coefficient; P alk,i,t is the power of the i-th alkaline electrolyzer at time t.

7. The hybrid electrolyzer cluster control method according to claim 1, characterized in that: The hybrid electrolyzer cluster control method further includes: Acquire real-time data of the hybrid electrolyzer cluster; the real-time data includes wind and solar data collected during the operation of the hybrid electrolyzer cluster; Based on the real-time data of the hybrid electrolyzer cluster and the environmental parameters, a deep deterministic policy gradient reinforcement learning algorithm is used to update the parameters of the trained hybrid electrolyzer system model with the purpose of maximizing operating indicators.

8. A hybrid electrolyzer cluster control device, characterized in that: The hybrid electrolyzer cluster control device comprises: A construction unit is used to construct a hybrid electrolyzer system model corresponding to a hybrid electrolyzer cluster; the hybrid electrolyzer cluster includes a photovoltaic device, a wind turbine, an energy storage battery, an electrolyzer set and a hydrogen storage tank; the electrolyzer set includes an alkaline electrolyzer and a PEM electrolyzer; the construction of the hybrid electrolyzer system model depends on constraint conditions and physical characteristics of the electrolyzer set; A training unit is used to perform offline training on the hybrid electrolyzer system model based on historical data of the hybrid electrolyzer cluster using a deep deterministic policy gradient reinforcement learning algorithm with the goal of maximizing the operating index, so as to obtain a trained hybrid electrolyzer system model; the historical data includes wind and solar data at historical moments; the wind and solar data includes photovoltaic power, wind turbine power, energy storage battery capacity, hydrogen load demand, hydrogen storage tank pressure and electricity price data; A control unit is used to input the current data of the hybrid electrolyzer cluster into the trained hybrid electrolyzer system model to obtain the control data of the hybrid electrolyzer cluster; the current data includes the wind and solar data at the current moment; the control data includes the power allocation ratio of the energy storage battery, the power allocation ratio of the electrolyzer set and the power of the grid.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the hybrid electrolyzer cluster control method described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the hybrid electrolyzer cluster control method described in any one of claims 1 to 7 is implemented.

Images