A fuel cell anode hydrogen exhaust valve control method, device, medium and product

Abstract

The present application discloses a fuel cell anode hydrogen exhaust valve control method, equipment, medium and product, which relates to the field of fuel cell control technology. The method includes obtaining the system state and corresponding reward value of the fuel cell system, inputting the system state and corresponding reward value of the fuel cell system into a trained prediction model to obtain a control action; the prediction model is a neural network model based on a reinforcement learning algorithm; according to the control action, the anode exhaust valve of the fuel cell system is controlled. The present application introduces reinforcement learning technology into the control of the fuel cell anode hydrogen exhaust valve, and utilizes its powerful adaptive and learning capabilities, through continuous interactive learning and optimization strategies with the fuel cell system, so that the trained prediction model can flexibly respond to complex and changing working conditions, effectively improving the stability and reliability of the system.

Description

Technical Field

[0001] The present application relates to the field of fuel cell control technology, and in particular to a fuel cell anode hydrogen exhaust valve control method, equipment, medium and product. Background Art

[0002] At present, fuel cell vehicles rely on the electrochemical reaction of hydrogen and oxygen to drive the vehicle system. This technology can provide clean power support for vehicles and avoid the emission of harmful pollutants. Automotive proton exchange membrane fuel cell systems usually use dead-end anodes. During operation, nitrogen on the cathode side will penetrate into the anode through the concentration gradient, causing nitrogen accumulation on the anode side and reducing the hydrogen concentration at the anode.

[0003] For the control of the anode exhaust valve of a high-power fuel cell system, most systems use a fixed exhaust valve working interval and working time to perform periodic exhaust. Although this method can meet the working needs of the system under steady-state conditions, it cannot be flexibly adjusted under variable working conditions. At the same time, due to the lack of real-time observation of gas components, in order to avoid fuel starvation, the exhaust interval is generally set short, which causes a large amount of hydrogen to be discharged directly before reacting, reducing the utilization rate of hydrogen and reducing economic benefits. Summary of the invention

[0004] The purpose of this application is to provide a fuel cell anode hydrogen exhaust valve control method, equipment, medium and product, which can flexibly control the exhaust valve under various working conditions to ensure stable and reliable operation of the system.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for controlling a fuel cell anode hydrogen exhaust valve, comprising:

[0007] Obtain the system status of the fuel cell system and the corresponding reward value.

[0008] The system state of the fuel cell system and the corresponding reward value are input into a trained prediction model to obtain a control action; the prediction model is a neural network model based on a reinforcement learning algorithm.

[0009] According to the control action, the anode exhaust valve of the fuel cell system is controlled.

[0010] In a second aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned fuel cell anode hydrogen exhaust valve control method.

[0011] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements the above-mentioned fuel cell anode hydrogen exhaust valve control method.

[0012] In a fourth aspect, the present application provides a computer program product, including a computer program, which implements the above-mentioned fuel cell anode hydrogen exhaust valve control method when executed by a processor.

[0013] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0014] The present application provides a fuel cell anode hydrogen exhaust valve control method, equipment, medium and product. First, the system state of the fuel cell system and the corresponding reward value are obtained; then, a control action is obtained according to the system state of the fuel cell system and the corresponding reward value using a trained neural network model based on a reinforcement learning algorithm; finally, according to the control action, the anode exhaust valve of the fuel cell system is controlled. The present application introduces reinforcement learning technology into the control of the fuel cell anode hydrogen exhaust valve, utilizes its powerful adaptive ability, and through continuous interactive learning and optimization strategies with the fuel cell system, enables the trained prediction model to flexibly respond to complex and changing working conditions, effectively improving the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flow chart of a method for controlling a fuel cell anode hydrogen exhaust valve provided in one embodiment of the present application;

[0017] Figure 2 A schematic block diagram of a fuel cell system model provided in one embodiment of the present application;

[0018] Figure 3 A flow chart of the reinforcement learning algorithm training provided in one embodiment of the present application;

[0019] Figure 4 A schematic diagram of a fuel cell anode hydrogen exhaust valve control method provided in one embodiment of the present application;

[0020] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application.

[0021] Reference numerals:

[0022] 1-Hydrogen storage tank, 2-Hydrogen pressure reducing valve, 3-Stack entry pressure control proportional valve, 4-Ejector, 5-Stack entry temperature sensor, 6-Stack entry pressure sensor, 7-Stack exit temperature sensor, 8-Stack exit pressure sensor, 9-Water separator, 10-Exhaust valve, 11-Stack, 12-DCDC converter. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] With the continuous development of new energy technologies, fuel cells, as a clean and efficient energy conversion device, have received widespread attention. In fuel cell systems, the control of the anode hydrogen exhaust valve is crucial to hydrogen utilization and system stability. Automotive proton exchange membrane fuel cell systems usually use dead-end anodes. During operation, nitrogen on the cathode side will penetrate into the anode through the drive of the concentration gradient, causing nitrogen accumulation on the anode side and reducing the hydrogen concentration at the anode. If the exhaust valve cannot exhaust in time, it may cause fuel starvation in the fuel cell, and in severe cases, it may also cause catalyst dissolution and carbon corrosion, thereby affecting the operating life of the fuel cell system. On the other hand, if continuous exhaust is performed to reduce the nitrogen concentration, a large amount of hydrogen will not participate in the reaction and will be directly discharged, which will reduce the hydrogen utilization of the system and is not conducive to the economy of the fuel cell system. Therefore, determining the appropriate exhaust valve working time to balance the hydrogen utilization and the degree of anode nitrogen accumulation is an important part of fuel cell system control.

[0025] At present, reinforcement learning has achieved remarkable results in the fields of autonomous driving, robot control, financial transactions, game agents, and medical diagnosis due to its strong adaptive ability and efficient decision-making ability. Reinforcement learning can achieve efficient decision-making in complex and dynamic environments by continuously optimizing strategies through interactive learning with the environment. In the field of fuel cell anode exhaust valve control, traditional control methods are often difficult to cope with the nonlinearity and complex dynamic changes of the system, while reinforcement learning algorithms can effectively cope with these challenges and improve the stability and efficiency of the system by continuously learning and adjusting strategies. Therefore, the application prospects of reinforcement learning in fuel cell anode exhaust valve control are broad and worthy of in-depth research and exploration.

[0026] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0027] In an exemplary embodiment, Figure 1 As shown, a method for controlling a fuel cell anode hydrogen exhaust valve is provided, the method comprising the following steps 101 to 103, wherein:

[0028] Step 101: Obtain the system state of the fuel cell system and the corresponding reward value.

[0029] Step 102: Input the system state of the fuel cell system and the corresponding reward value into a trained prediction model to obtain a control action; the prediction model is a neural network model based on a reinforcement learning algorithm.

[0030] Step 103: According to the control action, the anode exhaust valve of the fuel cell system is controlled.

[0031] Furthermore, the reward function r used to calculate the reward value is:

[0032]

[0033] Among them, S purge The exhaust valve status, 1 is open, 0 is closed; is the anode nitrogen concentration, is the anode nitrogen concentration threshold; is the hydrogen utilization rate, r + and r - are positive rewards and negative rewards respectively, k 1 and k 2 is the reward weight coefficient.

[0034] The reward value of each action pair is calculated based on the state and performance indicators of the fuel cell system. The reward value reflects the impact of the current action on the system performance and is an important basis for optimizing the strategy during reinforcement learning. The reward function clarifies the learning goal of the agent by defining what is a "good" result. The agent will achieve this goal by maximizing the cumulative reward (i.e., return). In this application, it is necessary to enable the agent to maximize the hydrogen utilization rate of the system while maintaining the anode nitrogen concentration below a certain threshold.

[0035] By adjusting k in the above reward function 1 and k 2 The size of these two coefficients can change the emphasis of the reward function to formulate a more conservative (tending to maintain low nitrogen concentration) or aggressive (tending to maintain high hydrogen utilization) exhaust strategy.

[0036] In the above reward function, hydrogen utilization rate is a continuous reward and needs to be given all the time. The three states corresponding to the formula are: 1. The exhaust valve is closed and the nitrogen concentration is lower than the threshold, in which case an additional reward is given; 2. The exhaust valve is closed and the nitrogen concentration is higher than the threshold, in which case an additional penalty is given; 3. The exhaust valve is open, in which case an additional penalty is given.

[0037] Furthermore, the system state includes cathode gas temperature, anode gas temperature, cathode gas pressure, anode gas pressure, cathode and cathode gas pressures, anode nitrogen concentration, hydrogen utilization rate and load current; and the control action includes opening the anode exhaust valve of the fuel cell system and closing the anode exhaust valve of the fuel cell system.

[0038] Furthermore, the reinforcement learning algorithm is a twin-delayed deep deterministic policy gradient algorithm, namely the TD3 (TwinDelayed Deep Deterministic Policy Gradient) algorithm.

[0039] TD3 is an advanced reinforcement learning algorithm for training agents to solve problems in continuous action spaces. Compared with the traditional DDPG (Deep Deterministic Policy Gradient), TD3 has improved stability and performance. It adopts a dual Q network and a delayed update strategy to reduce over-estimation by minimizing the difference between the two Q values, thereby improving the stability and convergence speed of training. In addition, TD3 also introduces a target policy network (Actor network) and a dual-objective Q network (Critic network), which reduces over-optimization by delaying the update of the target network, further improving the performance and generalization ability of the agent in complex environments. TD3 has been widely used in various continuous control problems, such as robotics and autonomous driving, and has demonstrated its superior performance in handling practical complex tasks.

[0040] Furthermore, the process of determining the trained prediction model includes:

[0041] Build a fuel cell system model.

[0042] Initialize the fuel cell system model.

[0043] Based on the initialized fuel cell system model, the prediction model is trained by reinforcement learning to obtain a trained prediction model.

[0044] As an optional implementation, Figure 2The fuel cell system model includes a hydrogen storage tank 1, a hydrogen pressure reducing valve 2, a stack pressure control proportional valve 3, an ejector 4, a stack temperature sensor 5, a stack pressure sensor 6, a stack temperature sensor 7, a stack pressure sensor 8, a water separator 9, an exhaust valve 10, a fuel cell stack 11, and a DCDC converter 12.

[0045] Among them, there are two control component actuators of the hydrogen supply system, namely the stack inlet pressure control proportional valve 3 and the stack outlet exhaust valve 10. The stack inlet pressure feedback control is completed by adjusting the opening of the stack inlet pressure control proportional valve 3, and the exhaust operation is performed by adjusting the switch of the exhaust valve 10.

[0046] Functions of the main components:

[0047] The fuel cell stack 11 (i.e., fuel cell) is used to generate electricity. Generally, a fuel cell generates heat while generating electricity.

[0048] The in-stack pressure sensor 6 and the out-stack pressure sensor 8 are used to obtain the in-stack and out-stack gas pressure information.

[0049] Into-combustion pressure control proportional valve 3: By adjusting its opening, the pressure of hydrogen entering the reactor is controlled. This article also calls it the hydrogen inlet proportional valve or proportional valve.

[0050] Water separator 9: used to separate the liquid water component in the anode outflow gas.

[0051] Exhaust valve 10: used to exhaust anode gas. The present invention mainly focuses on this actuator.

[0052] Ejector 4: used to ensure hydrogen flow and hydrogen recirculation.

[0053] The general working process of the fuel cell system is as follows: first, hydrogen enters the anode of the fuel cell from the high-pressure hydrogen storage tank 1 through the hydrogen pressure reducing valve 2 and then through the proportional valve 3. The hydrogen after the reaction and a small amount of nitrogen and water vapor are discharged through the exhaust valve 10. The water separator 9 is used to separate the liquid water produced by the reaction, and the ejector 4 is used for hydrogen recycling.

[0054] Furthermore, based on the initialized fuel cell system model, the prediction model is trained by reinforcement learning to obtain a trained prediction model, which specifically includes:

[0055] Initialize the network parameters of the prediction model.

[0056] A certain number of state-action pairs are randomly sampled in the experience pool; the state-action pairs in the experience pool come from the interaction process between the prediction model and the fuel cell system model; each of the state-action pairs includes a first system state, a control action, a reward value, and a second system state; the reward value is calculated based on the first system state; the second system state is the response state of the fuel cell system model after executing the control action under the first system state.

[0057] The network parameters of the prediction model are updated according to the state-action pairs, and the process returns to the step of "randomly sampling a certain number of state-action pairs in the experience pool", and the process is repeated until the cumulative reward value converges to obtain a trained prediction model.

[0058] In this application, the first system state is also referred to as the current state, and the control action is also referred to as an action.

[0059] Experience replay is a step in the TD3 algorithm. Experience replay: During system operation, we record the system status information (system temperature, load current, cathode and anode gas pressure, anode nitrogen concentration estimate, hydrogen utilization estimate) and the corresponding action pair (i.e., exhaust valve action command). These data are stored in the experience replay module (i.e., experience pool), forming an experience replay memory. The role of the experience replay module is to use these historical data in subsequent reinforcement learning training, making the learning process more stable and efficient.

[0060] The system temperature can be obtained through the stack entry temperature sensor 5 and the stack exit temperature sensor 7 ; the anode and cathode gas pressures can be obtained through the stack entry pressure sensor 6 and the stack exit pressure sensor 8 .

[0061] As an optional implementation, see Figure 3 , the specific training process based on TD3 is as follows:

[0062] Initialize two Critic network parameters w 1 ,w 2 And an Actor network parameter θ.

[0063] Initialize two target Critic network parameters and one target Actor network parameters as follows:

[0064]

[0065] in, are the initial values ​​of the corresponding network parameters.

[0066] Initialize the experience pool buffer quantity.

[0067] Set the action exploration noise a~π(s)+∈, The reward value r, the next state s', is stored in the experience pool as a state-action pair (s, a, r, s'). Among them, π(s) is the output strategy when the state is s, ∈ is the noise, It represents a Gaussian distribution (normal distribution) with a mathematical expectation of 0 and a standard deviation of σ, that is, the noise conforms to the Gaussian distribution, also known as Gaussian noise.

[0068] For the above state-action pair: action a is the exhaust valve state, the current state s includes the load current, system temperature, cathode and anode gas pressure, anode nitrogen concentration estimate and hydrogen utilization rate estimate, and r refers to the reward value calculated based on the current state s (the form of the reward function has been given above). When the current state is s and the reward value is r, action a is executed, and the response state of the system at the next moment is s'.

[0069] With the preset mini-batch size, randomly sample state-action pairs (s, a, r, s′) in the experience pool. These sampled state-action pairs will participate in the parameter update of the TD3 neural network.

[0070] The target actor network outputs actions to the critic network for network update:

[0071]

[0072] in, The network parameters are The target behavior strategy when ∈′ is the Gaussian noise of the target behavior strategy. The clip(,,) function is used to limit the value of the Gaussian noise to a given minimum value -c and maximum value c. That is, if the value of the Gaussian noise is greater than the maximum value c, it is set equal to the maximum value c. If the value of the Gaussian noise is less than the minimum value -c, it is set equal to the minimum value -c.

[0073] Using estimated actions The reward value estimated by the critic network and the smaller value calculated by the two critic networks are used for calculation

[0074]

[0075] Among them, r is the action based on the estimate of the critic network The estimated reward value, γ is the decay coefficient, The network parameters are The critic network is based on the state s′ and action Calculate the Q value.

[0076] Calculate the critic loss function J(wi ), and update the critic network parameters w by gradient descent 1 、w 2 ,

[0077]

[0078] Where N is the number of samples in a mini-batch. The target Q value is an estimate of the future return calculated using the current strategy or the target strategy. is the current Q value function when the network parameter is w i The estimated value below.

[0079] Update the actor network parameters θ through a deterministic gradient strategy:

[0080]

[0081] in, is the gradient of the loss function with respect to the policy parameters θ. is the gradient of the action value function with respect to the action, a = π θ (s) indicates that action a is achieved through the current strategy π θ Generated. represents the gradient of the policy function with respect to the policy parameter θ.

[0082] Update the target network parameters:

[0083]

[0084] Among them, ρ w and ρ θ are the learning rates of the critic and actor networks, respectively.

[0085] This training process will continue to cycle, and simulation training will be performed under various current conditions based on the set random current. When the cumulative value of the reward for each episode (round, also called round) of the simulation result converges, it can be considered that the reinforcement learning algorithm has completed its own training and can be used in practice.

[0086] The final result is a reinforcement learning agent. Specifically, it is the parameters of the neural network model contained in the TD3 agent. With these parameters, it is equivalent to having a complete reinforcement learning agent based on TD3. Figure 4 In the figure, the "TD3-based reinforcement learning agent" at the bottom is the training process, and the "TD3-based reinforcement learning agent" at the top is the trained model on the left, which can be used in practice.

[0087] Based on the trained reinforcement learning algorithm, the exhaust valve control of the real fuel cell system is performed:

[0088] Under the operating conditions of the actual system, the reinforcement learning algorithm is given the corresponding state quantity s and reward value feedback r, and the algorithm will output the appropriate exhaust valve action state a value to achieve the maximum control effect of the cumulative reward function.

[0089] Further, the fuel cell system model is initialized, specifically including:

[0090] Setting the variation range of the load current of the fuel cell system model to a preset range; the preset range is the variation range of the load current of the real fuel cell system under corresponding working conditions;

[0091] Set the change type of the fuel cell system model load current to random change.

[0092] As an optional implementation, the initialization process is: randomly loading a current operating condition in the fuel cell system model.

[0093] Initialization here refers to the design of simulation conditions. The main task of initializing the fuel cell system model is to design random current conditions, because the training of neural networks is generally cyclical. For simulation, the load current size changes randomly within a certain range each time, so that the trained reinforcement learning algorithm has the applicable performance under various working conditions.

[0094] In an exemplary embodiment, the fuel cell anode hydrogen exhaust valve control method may also be as follows:

[0095] Step 201: Initialize the fuel cell system model, wherein the initialization process is: randomly load a current operating condition, and obtain data required for subsequent steps through the fuel cell system model.

[0096] Step 202: Determine the number of observations and outputs of the reinforcement learning algorithm and their respective numerical ranges, wherein the observation is the system state s and the output is the control action a.

[0097] Step 203: Determine the form of the reward function.

[0098] Based on the state and performance indicators of the fuel cell system, we calculate the reward value for each action pair. The reward value reflects the impact of the current action on the system performance and is an important basis for optimizing the strategy during reinforcement learning. The reward function clarifies the learning goal of the agent by defining what is a "good" result. The agent will achieve this goal by maximizing the cumulative reward (i.e., return). The specific form of the reward function is shown above.

[0099] Step 204: Based on the random working condition injection of step 201, the state-action pair determined in step 202, and the reward function determined in step 3, reinforcement learning training is performed;

[0100] Step 204 is the stage of reinforcement learning training. After all conditions are prepared, the reinforcement learning algorithm based on TD3 will start training and update its own neural network parameters.

[0101] In general, the present invention realizes intelligent control of the hydrogen exhaust valve through two stages: offline learning and training and online deployment.

[0102] 1. Offline learning and training

[0103] 1.1 Initialization and data collection: First, we initialize the fuel cell system model. In order to ensure that the model can obtain sufficient training data under different working conditions, we inject random current conditions. This model contains key parameters of the fuel cell system, such as estimated nitrogen concentration, hydrogen utilization, and exhaust valve status. These data will be used in the subsequent training process.

[0104] 1.2 Reward Calculation.

[0105] 1.3 Reinforcement learning training.

[0106] After sufficient training, the reinforcement learning agent outputs optimized exhaust policies. These policies will be used to control the actual system in the online deployment phase.

[0107] 2. Online deployment stage

[0108] 2.1 System operation and monitoring: Under actual working conditions, the fuel cell system operates in real time. We obtain the real-time status and performance indicators of the system through sensors and monitoring equipment. The actual working conditions and observation information as well as the reward value will be transmitted to the prediction model in real time for online optimization.

[0109] 2.2 Reinforcement learning online optimization: The TD3-based prediction model receives the observation data of the fuel cell system in real time during the online deployment phase. The prediction model uses the strategy obtained from previous offline training and combines it with the current real-time data to make online adjustments and optimizations. This online optimization mechanism ensures that the system can still maintain efficient operation under dynamic conditions.

[0110] 2.3 Exhaust strategy execution: The prediction model outputs the optimal exhaust strategy instructions based on real-time data and optimization strategies. The system controls the action of the hydrogen exhaust valve according to these instructions to achieve precise control of hydrogen emissions.

[0111] In terms of optimizing the control effect of the exhaust valve on the anode side of the fuel cell, the present invention can make the system have the following advantages:

[0112] 1) The prediction model trained by reinforcement learning can cope with various complex working conditions and formulate the optimal control strategy of the exhaust valve.

[0113] 2) By modifying the weights of each value in the reward function, the emphasis on hydrogen utilization and nitrogen concentration can be changed, so that a conservative or aggressive control strategy can be trained.

[0114] 3) By combining offline learning and online optimization, the fuel cell anode hydrogen exhaust valve control method based on reinforcement learning proposed in the present invention significantly improves the hydrogen utilization rate and overall performance of the fuel cell system. In the offline learning stage, the system uses historical data to optimize the strategy through the TD3 algorithm; in the online deployment stage, the reinforcement learning agent adjusts the strategy in real time to ensure the efficient operation of the system under dynamic conditions. This method not only improves the stability and reliability of the system, but also provides a new solution for the intelligent control of fuel cells. Through the control method of the present invention, the system can not only cope with complex working conditions, but also adjust the focus of the control strategy according to demand, so as to better meet the needs of different application scenarios.

[0115] The present application also provides an application scenario, which applies the above-mentioned fuel cell anode hydrogen exhaust valve control method. Specifically: the fuel cell anode hydrogen exhaust valve control method provided in this embodiment can be applied in the fuel cell system performance control scenario of new energy vehicles.

[0116] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data of the fuel cell system. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a fuel cell anode hydrogen exhaust valve control method is implemented.

[0117] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0118] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0119] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0120] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0121] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0122] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0123] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for controlling a fuel cell anode hydrogen exhaust valve, characterized in that: The fuel cell anode hydrogen exhaust valve control method comprises: Obtaining a system state of the fuel cell system and a corresponding reward value; the system state includes cathode gas temperature, anode gas temperature, cathode gas pressure, anode gas pressure, anode nitrogen concentration, hydrogen utilization rate and load current; the reward function r used to calculate the reward value is: ； in, The exhaust valve status, 1 is open, 0 is closed; is the anode nitrogen concentration, is the anode nitrogen concentration threshold; is the hydrogen utilization rate, Positive rewards Negative rewards, is the reward weight coefficient; The system state of the fuel cell system and the corresponding reward value are input into a trained prediction model to obtain a control action; the prediction model is a neural network model based on a reinforcement learning algorithm; the reinforcement learning algorithm is a double-delayed deep deterministic policy gradient algorithm; the control action includes opening the anode exhaust valve of the fuel cell system and closing the anode exhaust valve of the fuel cell system; According to the control action, the anode exhaust valve of the fuel cell system is controlled.

2. The fuel cell anode hydrogen exhaust valve control method according to claim 1, characterized in that: The process of determining the trained prediction model includes: Build a fuel cell system model; Initialize the fuel cell system model; Based on the initialized fuel cell system model, the prediction model is trained by reinforcement learning to obtain a trained prediction model.

3. The fuel cell anode hydrogen exhaust valve control method according to claim 2, characterized in that: Based on the initialized fuel cell system model, the prediction model is trained through reinforcement learning to obtain a trained prediction model, which specifically includes: Initialize the network parameters of the prediction model; A certain number of state-action pairs are randomly sampled in the experience pool; the state-action pairs in the experience pool come from the interaction process between the prediction model and the fuel cell system model; each state-action pair includes a first system state, a control action, a reward value, and a second system state; the reward value is calculated based on the first system state; the second system state is the response state of the fuel cell system model after executing the control action under the first system state; The network parameters of the prediction model are updated according to the state-action pair, and the process returns to step "randomly sampling a certain number of state-action pairs in the experience pool", and the process is repeated until the reward accumulation value converges to obtain a trained prediction model.

4. The fuel cell anode hydrogen exhaust valve control method according to claim 2, characterized in that: Initialize the fuel cell system model, including: Setting the variation range of the load current of the fuel cell system model to a preset range; the preset range is the variation range of the load current of the real fuel cell system under corresponding working conditions; Set the change type of the fuel cell system model load current to random change.

5. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the fuel cell anode hydrogen exhaust valve control method according to any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the fuel cell anode hydrogen exhaust valve control method according to any one of claims 1 to 4 is implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the fuel cell anode hydrogen exhaust valve control method according to any one of claims 1 to 4 is implemented.

Images