Fuel cell stack reliability evaluation method based on strategy gradient and game

Abstract

The invention relates to the technical field of fuel cell stack monitoring, and discloses a fuel cell stack reliability evaluation method based on strategy gradient and game, which comprises the following steps: training a first agent, a second agent and a third agent by adopting a multi-agent depth deterministic strategy gradient method, the Stackelberg game model is adopted to carry out leader and follower division on the first agent, the second agent and the third agent in training, the optimal strategy of the leader is utilized to adjust the optimal strategy of the follower, and the optimal first agent, the optimal second agent and the optimal third agent are obtained; real-time monitoring of the health state of the stack is realized, the electricity price is dynamically adjusted according to market demands, and meanwhile, a reliability evaluation index system is established for evaluating the operation stability and power supply reliability of the fuel cell stack; according to the method, the economic benefit is improved, and meanwhile, the stability and reliability of power supply are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of fuel cell stack monitoring, and in particular to a fuel cell stack reliability assessment method based on strategy gradient and game. Background Art

[0002] As a key component of power supply, the reliability of fuel cell stacks plays a vital role in the stability and safety of power systems. Most existing evaluation methods are based on static models, which are insufficient in adapting to changes in market and operating conditions, especially in responding to emergencies and system failures, and fail to provide sufficient dynamic evaluation and timely response. The reliability of the stack not only directly affects its performance and life, but is also the key to the continuity and reliability of power supply. Current monitoring and optimization technologies for the health status of the stack mainly focus on local parameter adjustments, and fail to fully cover the impact of the entire energy system. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides a fuel cell stack reliability assessment method based on strategy gradient and game theory, which is used to solve the problem that the prior art cannot provide sufficient dynamic assessment and timely response, resulting in low efficiency of stack health status monitoring and power optimization.

[0004] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0005] A fuel cell stack reliability assessment method based on strategy gradient and game theory comprises the following steps:

[0006] S1. Construct a first agent, evaluate the health status of the fuel cell stack by monitoring the key operating parameters of the fuel cell stack, and establish the reward function, state space and action space of the first agent;

[0007] S2. Construct a second agent to dynamically adjust the electricity price according to the health status of the fuel cell stack and the electricity demand on the user side to maximize the profit of the electricity price, and establish the reward function, state space and action space of the second agent;

[0008] S3. Construct a third agent to make purchase decisions based on electricity prices and user-side electricity demand, obtain electricity purchase costs, minimize electricity costs, and establish the reward function, state space, and action space of the third agent;

[0009] S4. Use the multi-agent deep deterministic policy gradient method to train the first agent, the second agent and the third agent to maximize the cumulative reward of each agent, and use the Stackelberg game model to divide the first agent, the second agent and the third agent in training into leaders and followers, and use the leader's optimal strategy to adjust the follower's optimal strategy to obtain the optimal first agent, the second agent and the third agent;

[0010] S5. Using the optimal first intelligent agent, adjust key operating parameters of the fuel cell stack to obtain an optimal health state of the fuel cell stack;

[0011] S6. Use the best second agent to adjust the electricity price and obtain the best electricity price profit;

[0012] S7, using the optimal third agent to adjust the amount of electricity purchased by the user to obtain the optimal electricity cost;

[0013] S8. Based on the parameter information after the adjustment of the optimal first intelligent agent, the second intelligent agent and the third intelligent agent, a reliability evaluation index system is established to evaluate the operation stability and power supply reliability of the fuel cell stack and the power system.

[0014] Furthermore, step S1 specifically includes:

[0015] S11. Construct a first intelligent agent to evaluate the health status of the fuel cell stack by monitoring the key operating parameters of the fuel cell stack, namely:

[0016]

[0017] Where H represents the health status coefficient of the fuel cell stack, P t represents the actual output power of the fuel cell stack at time t, η t represents the rated electrical efficiency of the fuel cell stack at the tth moment, c represents the attenuation ratio limit that the fuel cell stack can withstand, η 0 represents the rated electrical efficiency of the fuel cell stack at the initial moment, e represents the exponential function, N start Indicates the number of startups of the fuel cell stack, T′ run represents the operating time of the fuel cell stack, α, β, γ, and λ all represent weight coefficients of key operating parameters of the fuel cell stack;

[0018] S12. The health status of the fuel cell stack is used as the reward function of the first agent, that is:

[0019] R1=H

[0020] Where R1 represents the reward function of the first agent;

[0021] S13. Establish the state space of the first agent, namely:

[0022] S 1 = {P,η,N start ,T′ run ,T avg ,P′ max}

[0023] Among them, S 1 represents the state space of the first agent, P represents the actual output power of the fuel cell stack, η represents the electrical efficiency of the fuel cell stack, T avg Represents the average temperature of the fuel cell stack during operation, P′ max Indicates the maximum pressure during the operation of the fuel cell stack;

[0024] S14. Establish the action space of the first agent, namely:

[0025]

[0026] Among them, A 1 represents the action space of the first agent, T represents the temperature of the fuel cell stack, is the hydrogen partial pressure, represents the oxygen partial pressure, R′ hum Relative humidity, Indicates the excess oxygen ratio.

[0027] Furthermore, step S2 specifically includes:

[0028] S21. Construct a second intelligent agent to dynamically adjust the electricity price according to the health status of the fuel cell stack and the electricity demand on the user side to obtain the goal of maximizing the electricity price profit, that is:

[0029]

[0030] Among them, O represents the profit maximization target of electricity price, P″ represents the electricity price, Q represents the electricity demand on the user side, C represents the total cost of the power station, and P″ base represents the basic electricity price, r represents the price adjustment rate, H threshold represents the preset health indicator threshold, D represents the basic power demand, σ represents the price elasticity coefficient, and τ represents the influence coefficient of the health status of the fuel cell stack on the power demand on the user side;

[0031] S22. The profit maximization objective of the second agent is used as the reward function of the second agent, that is:

[0032] R 2 =O

[0033] Among them, R 2 represents the reward function of the second agent;

[0034] S23, establish the state space of the second agent, that is:

[0035] S 2 ={P″ base ,r,H,H threshold ,D,σ,τ}

[0036] Among them, S 2 represents the state space of the second agent;

[0037] S24. Establish the action space of the second agent, namely:

[0038] A 2 ={P″}

[0039] Among them, A 2 Represents the action space of the second agent.

[0040] Furthermore, step S3 specifically includes:

[0041] S31. Construct a third intelligent agent to make a purchase decision based on the electricity price and the electricity demand on the user side, and obtain the electricity purchase cost, that is:

[0042] C electricity =P″×Q

[0043] Among them, C electricity represents the electricity purchase cost;

[0044] S32. The negative value of the electricity purchase cost is used as the reward function of the third agent, that is:

[0045] R 3 =-C electricity

[0046] Among them, R 3 represents the reward function of the third agent;

[0047] S33, establish the state space of the third agent, namely:

[0048] S 3 ={P″,D,σ}

[0049] Among them, S 3 represents the state space of the third agent;

[0050] S34. Establish the action space of the third agent, namely:

[0051] A 3 ={K}

[0052] Among them, A 3 represents the action space of the third agent, and K represents the amount of electricity purchased on the user side.

[0053] Furthermore, step S4 specifically includes:

[0054] S41, respectively establishing a strategy network for the first agent, the second agent, and the third agent; wherein the strategy network includes an action network and a value network;

[0055] S42, respectively inputting the reward functions, state spaces, and action spaces corresponding to the first agent, the second agent, and the third agent into their respective strategy networks for training, and maximizing the cumulative reward of each agent by obtaining the strategy gradient of each agent;

[0056] S43, using the Stackelberg game model to divide the first intelligent agent, the second intelligent agent and the third intelligent agent in training into leaders and followers, and updating the policy gradient of each intelligent agent;

[0057] S44. Repeat steps S42-S43 until the set maximum number of iterations is reached, and the optimal first agent, second agent, and third agent are obtained.

[0058] Furthermore, the policy gradient of each agent in step S42 is:

[0059]

[0060]

[0061] in, represents the gradient, θ i represents the policy network parameters of agent i, J represents the cumulative reward of agent i, θ 1 ,θ 2 ,θ 3 Represent the policy network parameters of the first agent, the second agent, and the third agent respectively, represents the mean calculation, Q′ represents the value evaluation value of the policy network, s represents the state space of agent i, and a 1 、a 2 、a 3 Respectively represent the action space of the first agent, the second agent, and the third agent, R represents the reward of each agent, represents the weight coefficient of the policy network, Q(S - ,a - ) represents the state space S of each agent at the previous moment - With action space a - .

[0062] Furthermore, the cumulative reward of each agent maximized in step S42 is the total return obtained by each agent from the multi-agent environment when performing strategy optimization to maximize the action decision of each agent.

[0063] Furthermore, in step S43, the specific process of using the Stackelberg game model to divide the first agent, the second agent, and the third agent in training into leaders and followers is as follows:

[0064] The first intelligent agent is taken as the leader, and the second and third intelligent agents are taken as the first and second followers respectively; first, the leader makes a decision, and then the first and second followers optimize their own strategies according to the decision made by the leader.

[0065] Furthermore, the update formula of the policy gradient of each agent in step S43 is:

[0066]

[0067] Among them, θ′ i represents the updated policy network parameters of agent i, and α represents the learning rate of the policy network.

[0068] Further, step S8 specifically includes:

[0069] S51, obtaining the optimal key operating parameters of the fuel cell stack after the optimal first intelligent agent is adjusted, and establishing an expected energy non-supply index, namely:

[0070]

[0071] Among them, EENS stands for expected energy non-supply indicator;

[0072] S52, obtaining the optimal key operating parameters of the fuel cell stack after the optimal first intelligent agent is adjusted, and establishing a system average interruption duration index, namely:

[0073]

[0074] Among them, SAIDI represents the system average interruption duration index;

[0075] S53. Establish the system average interruption frequency index indicator, namely:

[0076]

[0077] Among them, SAIFI represents the system average interruption frequency index;

[0078] S54. Evaluate the operation stability and power supply reliability of the fuel cell stack and the power system based on the expected energy non-supply index, the system average interruption duration index, and the system average interruption frequency index index;

[0079] S55, evaluating the operation stability of the fuel cell stack based on the optimal health state of the fuel cell stack;

[0080] S56. Based on the optimal second intelligent agent, the optimal electricity price after adjustment by the third intelligent agent, and the predicted electricity purchase amount, the effectiveness of user demand side management is evaluated by evaluating the impact of electricity price adjustment on electricity purchase amount.

[0081] The present invention has the following beneficial effects:

[0082] 1. The present invention proposes a fuel cell stack reliability assessment method based on policy gradient and game, which uses a multi-agent deep deterministic policy gradient method combined with a Stackelberg game model to train the first agent, the second agent and the third agent, and obtains the optimal first agent, the second agent and the third agent to make the optimal strategy adjustment, aiming to enhance the operation stability of a high-power fuel cell stack and the reliability of an electric-hydrogen integrated energy system;

[0083] 2. Through detailed modeling of the fuel cell stack and its interaction with the electricity market, in-depth analysis of the impact of possible failure modes on performance, and comprehensive consideration of the operation of the fuel cell stack under different conditions, key indicators such as expected energy shortage, system average interruption duration and system average interruption frequency index are calculated to provide the electricity market with an accurate measure of the reliability of the fuel cell stack, thereby optimizing the allocation of electricity resources and ensuring the continuity of supply. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 A schematic flow chart of a fuel cell stack reliability assessment method based on strategy gradient and game theory proposed in the present invention;

[0085] Figure 2 Schematic diagram of the interaction principle of the three intelligent agents. DETAILED DESCRIPTION

[0086] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0087] like Figure 1As shown, a fuel cell stack reliability assessment method based on strategy gradient and game theory includes the following steps S1-S8:

[0088] S1. Construct a first intelligent agent, evaluate the health status of the fuel cell stack by monitoring the key operating parameters of the fuel cell stack, and establish the reward function, state space, and action space of the first intelligent agent.

[0089] In this embodiment, the first intelligent agent is constructed mainly to monitor the health status of the fuel cell stack, that is, through subsequent training, it will learn how to adjust the operating parameters according to the status of the fuel cell stack to maximize the health status of the fuel cell stack.

[0090] Specifically, step S1 includes S11-S14:

[0091] S11. Construct a first intelligent agent to evaluate the health status of the fuel cell stack by monitoring the key operating parameters of the fuel cell stack, namely:

[0092]

[0093] Where H represents the health status coefficient of the fuel cell stack, P t represents the actual output power of the fuel cell stack at time t, η t represents the rated electrical efficiency of the fuel cell stack at the tth moment, c represents the attenuation ratio limit that the fuel cell stack can withstand, η 0 represents the rated electrical efficiency of the fuel cell stack at the initial moment, e represents the exponential function, N start Indicates the number of startups of the fuel cell stack, T′ run represents the operating time of the fuel cell stack, and α, β, γ, and λ all represent weight coefficients of key operating parameters of the fuel cell stack.

[0094] S12. The health status of the fuel cell stack is used as the reward function of the first agent, that is:

[0095] R1=H

[0096] Where R1 represents the reward function of the first agent.

[0097] In this embodiment, the first intelligent agent is mainly responsible for monitoring and optimizing the health status of the high-power fuel cell stack; its core task is to evaluate and improve the health status coefficient of the fuel cell stack through real-time monitoring of the key operating parameters of the fuel cell stack, and directly set the health coefficient as the reward function of the first intelligent agent; this design makes every decision of the first intelligent agent aimed at maximizing the health status of the fuel cell stack, thereby indirectly improving the operating efficiency and life of the fuel cell stack.

[0098] S13. Establish the state space of the first agent, namely:

[0099] S 1 = {P,η,N start ,T′ run ,T avg ,P′ max}

[0100] Among them, S 1 represents the state space of the first agent, P represents the actual output power of the fuel cell stack, η represents the electrical efficiency of the fuel cell stack, R avg Represents the average temperature of the fuel cell stack during operation, P′ max Indicates the maximum pressure during fuel cell stack operation.

[0101] S14. Establish the action space of the first agent, namely:

[0102]

[0103] Among them, A 1 represents the action space of the first agent, R represents the temperature of the fuel cell stack, is the hydrogen partial pressure, represents the oxygen partial pressure, R′ hum Relative humidity, Indicates the excess oxygen ratio.

[0104] In this embodiment, the action space includes all possible actions that the first agent can take, which are aimed at adjusting the key operating parameters of the fuel cell stack to optimize its health state. For example, the first agent may affect the performance and life of the fuel cell stack by adjusting the hydrogen partial pressure, oxygen partial pressure, operating temperature, etc. In this way, the first agent can not only improve the performance and reliability of the fuel cell stack, but also play a coordinating role in the entire system and work with other agents to achieve the overall optimization goal of the system.

[0105] S2. Construct a second intelligent agent to dynamically adjust the electricity price according to the health status of the fuel cell stack and the electricity demand on the user side to maximize the electricity price profit, and establish the reward function, state space and action space of the second intelligent agent.

[0106] In this embodiment, the second agent dynamically adjusts the electricity price according to the health status of the fuel cell stack and the market supply and demand conditions to maximize profits; its core task is to formulate and adjust the electricity price strategy to respond to market changes and the health status of the fuel cell stack to maximize the profits of the power station. Its goal is to balance the supply and demand relationship by optimizing the electricity price, improve the economic benefits of the power station, and ensure the stability of the electricity price and user acceptance.

[0107] Specifically, step S2 specifically includes S21-S24:

[0108] S21. Construct a second intelligent agent to dynamically adjust the electricity price according to the health status of the fuel cell stack and the electricity demand on the user side to obtain the goal of maximizing the electricity price profit, that is:

[0109]

[0110] Among them, O represents the profit maximization target of electricity price, P″ represents the electricity price, Q represents the electricity demand on the user side, C represents the total cost of the power station, and P″ base represents the basic electricity price, r represents the price adjustment rate, H threshold represents the preset health indicator threshold, D represents the basic electricity demand, σ represents the price elasticity coefficient, and τ represents the influence coefficient of the health status of the fuel cell stack on the electricity demand on the user side.

[0111] S22. The profit maximization objective of the second agent is used as the reward function of the second agent, that is:

[0112] R 2 =O

[0113] Among them, R 2 represents the reward function of the second agent.

[0114] S23, establish the state space of the second agent, that is:

[0115] S 2 ={P″ base ,r,H,H threshold ,D,σ,τ}

[0116] Among them, S 2 Represents the state space of the second agent.

[0117] S24. Establish the action space of the second agent, namely:

[0118] A 2 ={P″}

[0119] Among them, A 2 Represents the action space of the second agent.

[0120] In this embodiment, the action space is mainly to adjust the electricity price in response to market changes and the health of the fuel cell stack. In this way, the second agent can dynamically adjust the electricity price to maximize the profit of the power station, taking into account the health of the fuel cell stack and changes in market demand, which not only improves the economic benefits of the power station, but also enhances its adaptability to market fluctuations.

[0121] S3. Construct a third intelligent agent to make purchase decisions based on electricity prices and user-side electricity demand, obtain electricity purchase costs, minimize electricity costs, and establish the third intelligent agent's reward function, state space, and action space.

[0122] In this embodiment, the third agent is mainly used to simulate the market response on the user side and make purchasing decisions based on electricity prices and electricity demand; its core task is to simulate the market response on the user side and make purchasing decisions based on the electricity price set by the second agent and its own electricity demand; its goal is to reflect the market sensitivity to changes in electricity prices and the electricity demand on the user side, thereby providing market feedback to the second agent and helping it adjust its electricity price strategy.

[0123] Specifically, step S3 specifically includes S31-S34:

[0124] S31. Construct a third intelligent agent to make a purchase decision based on the electricity price and the electricity demand on the user side, and obtain the electricity purchase cost, that is:

[0125] C electricity =P″″×Q

[0126] Among them, C electricity Represents the cost of purchasing electricity.

[0127] S32. The negative value of the electricity purchase cost is used as the reward function of the third agent, that is:

[0128] R 3 =-C electricity

[0129] Among them, R 3 represents the reward function of the third agent.

[0130] S33, establish the state space of the third agent, namely:

[0131] S 3 ={P″,D,σ}

[0132] Among them, S 3 Represents the state space of the third agent.

[0133] In this embodiment, the action space is mainly to adjust the amount of electricity purchased in response to changes in electricity prices and its own needs; in this way, the third agent can quantify its decision-making process and minimize its electricity purchase costs by adjusting the amount of electricity purchased, while taking into account the market's sensitivity to changes in electricity prices.

[0134] S34. Establish the action space of the third agent, namely:

[0135] A 3 ={K}

[0136] Among them, A 3 represents the action space of the third agent, and K represents the amount of electricity purchased on the user side.

[0137] S4. Use the multi-agent deep deterministic policy gradient method to train the first agent, the second agent and the third agent to maximize the cumulative reward of each agent, and use the Stackelberg game model to divide the first agent, the second agent and the third agent in training into leaders and followers, and use the optimal strategy of the leader to adjust the optimal strategy of the follower to obtain the optimal first agent, the second agent and the third agent.

[0138] In this embodiment, a multi-agent deep deterministic policy gradient method is adopted, and the three agents can learn and adapt to environmental changes and optimize their own strategies; the Stackelberg game model is used to coordinate the decisions among the agents and realize the strategic interaction between the leader (fuel cell operator) and the follower (electricity market user); this method can not only improve the operating efficiency and market competitiveness of the fuel cell, but also realize the real-time monitoring and optimization of the health status of the fuel cell in the complex background of multi-energy coupling, as well as the dynamic pricing strategy of the electricity market, and promote the energy system to develop in a more efficient and reliable direction.

[0139] Specifically, step S4 includes S41-S44:

[0140] S41. Establishing strategy networks for the first agent, the second agent, and the third agent respectively; wherein the strategy network includes an action network and a value network.

[0141] In this embodiment, the core is that each agent has its own action network and value network. The action network is responsible for outputting actions, and the value network is responsible for evaluating the value of actions. In the training process, the value network can access the information of all agents, while in the execution phase, the action network of each agent can only make decisions based on its own local observations. Therefore, the goal of the multi-agent deep deterministic policy gradient method is to maximize the cumulative reward of each agent.

[0142] In addition, the value network of the first agent can access the information of the second and third agents to evaluate the impact of their actions on the entire system; the value networks of the second and third agents can also access the information of other agents; in this way, each agent can learn and adopt the best strategy in a multi-agent environment to achieve its own goals while taking into account the behavior of other agents, such as Figure 2As shown, the first agent optimizes the performance of the battery stack by adjusting the operating parameters. The changes in these parameters directly affect the health status of the battery stack, which in turn affects the electricity price decision of the second agent; the second agent adjusts the electricity price according to the health status of the battery stack and market demand, which in turn affects the electricity purchasing decision of the third agent; the market response of the third agent is fed back to the second agent to help it further optimize the electricity price strategy; this interaction and feedback mechanism enables the entire system to continuously learn and adapt in a dynamic environment to achieve the optimization of overall performance; that is, through the multi-agent deep deterministic policy gradient method, the agents can handle complex multi-agent interaction problems, thereby achieving collaborative optimization.

[0143] S42, respectively input the reward functions, state spaces and action spaces corresponding to the first agent, the second agent and the third agent into their respective strategy networks for training, and maximize the cumulative reward of each agent by obtaining the strategy gradient of each agent.

[0144] Specifically, the policy gradient of each agent in step S42 is:

[0145]

[0146]

[0147] in, represents the gradient, θ i represents the policy network parameters of agent i, J represents the cumulative reward of agent i, θ 1 ,θ 2 ,θ 3 Represent the policy network parameters of the first agent, the second agent, and the third agent respectively, represents the mean calculation, Q′ represents the value evaluation value of the policy network, s represents the state space of agent i, and a 1 、a 2 、a 3 Respectively represent the action space of the first agent, the second agent, and the third agent, R represents the reward of each agent, represents the weight coefficient of the policy network, Q(S - ,a - ) represents the state space S of each agent at the previous moment - With action space a - .

[0148] Specifically, the cumulative reward of each agent maximized in step S42 is the total return obtained by each agent from the multi-agent environment when performing strategy optimization to maximize the action decision of each agent.

[0149] S43. Use the Stackelberg game model to divide the first agent, the second agent, and the third agent in training into leaders and followers, and update the policy gradient of each agent.

[0150] Specifically, in step S43, the specific process of using the Stackelberg game model to divide the first agent, the second agent, and the third agent in training into leaders and followers is as follows:

[0151] The first intelligent agent is taken as the leader, and the second and third intelligent agents are taken as the first and second followers respectively; first, the leader makes a decision, and then the first and second followers optimize their own strategies according to the decision made by the leader.

[0152] Specifically, the update formula of the policy gradient of each agent in step S43 is:

[0153]

[0154] Among them, θ′ i represents the updated policy network parameters of agent i, and α represents the learning rate of the policy network.

[0155] S44. Repeat steps S42-S43 until the set maximum number of iterations is reached, and the optimal first agent, second agent, and third agent are obtained.

[0156] In this embodiment, the above steps are iterated through learning, and the first agent, the second agent and the third agent gradually find the optimal strategy, thereby achieving Stackelberg equilibrium. Among them, the Stackelberg game is a hierarchical decision-making model, which includes a leader and one or more followers; the leader makes a decision first, and then the followers optimize their strategies according to the leader's decision; this game model is often used to describe the strategic interaction between participants with different decision-making powers and information levels; the leader's goal is to maximize his own utility function, and the followers choose their own optimal strategies given the leader's decision; at the same time, when making decisions, the leader will take into account the reactions of the followers, and the specific process is: the objective function of the first agent is to maximize the health status coefficient of the fuel cell stack, and its action space includes the adjustment of the stack operating parameters; the objective function of the second agent is to maximize the profit, and its action space includes the setting of the electricity price; the objective function of the third agent is to minimize the electricity cost, and its action space includes the adjustment of the electricity purchase amount; in the Stackelberg game established in this embodiment, the first agent makes a decision first, and then the second agent and the third agent optimize their strategies according to the decision of the first agent; the second agent dynamically adjusts the electricity price according to the health status of the fuel cell stack and market demand, and the third agent determines the electricity purchase amount according to the electricity price and its own needs. At the same time, the multi-agent deep deterministic policy gradient method plays a key role in this process. It allows each agent to learn how to optimize its own strategy in a multi-agent environment, that is, the first agent predicts the responses of the second and third agents through learning, and adjusts its own operating parameters accordingly. The second and third agents also optimize their own strategies through learning to respond to the operations of the first agent and the strategies of other agents; through this interaction and learning process, the first, second and third agents can gradually reach a Stackelberg equilibrium, at which the strategy of each agent is the optimal response to the strategies of other agents. This equilibrium state helps the entire system to achieve fuel cell stack performance optimization and market efficiency improvement.

[0157] In addition, during the entire training process described above, the update of the value network (leader and follower) is: Q new =r1+γ1V(s′,θ L ,θ F )、loss=(Q current -Q new ) 2 , where Q new represents the updated value network, r1 represents the reward, γ1 represents the discount factor, V represents the value function, s ′ represents the updated state, θ Lrepresents the leader’s strategy parameter, θ F represents the strategy parameter of the follower, loss represents the loss function of the value network, Q current represents the policy network before update; the update of the action network (leader and follower) is: Among them, Q″ represents the value assessment, a L 、a F Represent the actions of the leader and the follower respectively, and s represents the state; finally, the follower updates the strategy by maximizing the expected return when the leader's strategy is given.

[0158] S5. Using the optimal first intelligent agent, adjust the key operating parameters of the fuel cell stack to obtain the optimal health state of the fuel cell stack.

[0159] S6. Use the optimal second intelligent agent to adjust the electricity price and obtain the optimal electricity price profit.

[0160] S7. Utilize the best third agent to adjust the amount of electricity purchased by the user to obtain the best electricity cost.

[0161] S8. Based on the parameter information after the adjustment of the optimal first intelligent agent, the second intelligent agent and the third intelligent agent, a reliability evaluation index system is established to evaluate the operation stability and power supply reliability of the fuel cell stack and the power system.

[0162] Specifically, step S8 specifically includes S81-S86:

[0163] S81, obtaining the optimal key operating parameters of the fuel cell stack after the optimal first intelligent agent is adjusted, and establishing an expected energy shortage indicator, namely:

[0164]

[0165] Here, EENS stands for expected energy non-supply indicator.

[0166] In this embodiment, based on the key operating parameters provided by the optimal first intelligent agent, the ratio of energy that cannot be supplied within a specific time to the total required energy is calculated, that is, the expected energy non-supply index is established. Among them, the planned supply energy is the total energy that the fuel cell stack should provide to the load according to the scheduling plan within a specific time period; the actual supply energy is the energy actually provided to the load by the fuel cell stack within a specific time period; therefore, by calculating the expected energy non-supply index, it can be judged whether the fuel cell stack is running stably, specifically: if the EENS value is maintained at a low and stable level, it indicates that the fuel cell stack can provide energy as planned and run stably; if the EENS value increases abnormally or fluctuates greatly, it indicates that the fuel cell stack has performance problems or insufficient maintenance, resulting in unstable energy supply.

[0167] S82, obtaining the optimal key operating parameters of the fuel cell stack after the optimal first intelligent agent is adjusted, and establishing a system average interruption duration indicator, namely:

[0168]

[0169] Among them, SAIDI represents the system average interruption duration index.

[0170] In this embodiment, the optimal health status assessment result of the first intelligent agent is used to record the duration of each power outage and calculate the average value, that is, to establish the system average outage duration index. Among them, the power outage duration is the length of time from the beginning of each power outage to the restoration of power supply by the battery stack; the number of affected users is the number of independent loads or users affected by each power outage. Therefore, by calculating the system average outage duration index, it is possible to determine whether the power supply of the power system is reliable, specifically: if the SAIDI value is low, that is, the interruption duration is short and infrequent, it means that the power outage has little impact on users and the power supply reliability is high; if the SAIDI value is high, that is, the interruption duration is long or frequent, it means that the power outage has a great impact on users and the power supply reliability is low.

[0171] S83. Establish the system average interruption frequency index indicator, namely:

[0172]

[0173] Among them, SAIFI represents the system average interruption frequency index.

[0174] In this embodiment, based on the data of the optimal first intelligent agent, the number of power outages within a specific time period is counted, and the average number of outages experienced by each user is calculated, that is, the system average outage frequency index index is established; wherein the total number of power outages is the total number of all power stack power outage events that occurred within a specific time period; the total number of users is the total number of all independent loads or users that may be affected by the outage within a specific time period. Therefore, by calculating the system average outage frequency index index, it is possible to determine whether the power supply system is stable, specifically: if the SAIFI value is low, that is, the number of outages is small, indicating that the fuel cell stack is running smoothly and there are few outages; if the SAIFI value is high, that is, the number of outages is large, it means that the fuel cell stack may frequently fail or performance degrades, and the operation is unstable.

[0175] S84. Evaluate the operational stability and power supply reliability of the fuel cell stack and the power system based on the expected energy non-supply index, the system average interruption duration index, and the system average interruption frequency index index.

[0176] In summary, when EENS, SAIDI and SAIFI are low, it indicates that the fuel cell stack operates stably and has high power supply reliability, that is, it has high stability and high reliability; when EENS, SAIDI and SAIFI are high, it indicates that the fuel cell stack operates unstable and has low power supply reliability, that is, it has low stability and low reliability.

[0177] S85. Based on the optimal health status of the fuel cell stack, evaluate the operating stability of the fuel cell stack.

[0178] S86. Based on the optimal second agent, the optimal electricity price after adjustment by the third agent, and the predicted electricity purchase amount, the effectiveness of user demand side management is evaluated by evaluating the impact of electricity price adjustment on electricity purchase amount.

[0179] In summary, the fuel cell stack reliability assessment method based on strategy gradient and game proposed in the present invention aims to enhance the stability of the operation of high-power fuel cell stacks and the reliability of the electric-hydrogen integrated energy system. The system deeply analyzes the impact of possible failure modes on performance through fine modeling of the stack and its interaction with the power market, and comprehensively considers the operation of the stack under different conditions. By calculating key indicators such as expected energy unavailable (EENS), system average interruption duration (SAIDI) and system average interruption frequency index (SAIFI), the present invention can provide the power market with an accurate measure of the reliability of the stack, thereby optimizing the allocation of power resources and ensuring the continuity of supply.

[0180] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0181] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A fuel cell stack reliability assessment method based on strategy gradient and game, characterized in that: The following steps are involved: S1. Construct a first agent, evaluate the health status of the fuel cell stack by monitoring the key operating parameters of the fuel cell stack, and establish the reward function, state space and action space of the first agent; S2. Construct a second agent to dynamically adjust the electricity price according to the health status of the fuel cell stack and the electricity demand on the user side to maximize the profit of the electricity price, and establish the reward function, state space and action space of the second agent; S3. Construct a third agent to make purchase decisions based on electricity prices and user-side electricity demand, obtain electricity purchase costs, minimize electricity costs, and establish the reward function, state space, and action space of the third agent; S4. Use the multi-agent deep deterministic policy gradient method to train the first agent, the second agent and the third agent to maximize the cumulative reward of each agent, and use the Stackelberg game model to divide the first agent, the second agent and the third agent in training into leaders and followers, and use the leader's optimal strategy to adjust the follower's optimal strategy to obtain the optimal first agent, the second agent and the third agent; S5. Using the optimal first intelligent agent, adjust key operating parameters of the fuel cell stack to obtain an optimal health state of the fuel cell stack; S6. Use the best second agent to adjust the electricity price and obtain the best electricity price profit; S7, using the optimal third agent to adjust the amount of electricity purchased by the user to obtain the optimal electricity cost; S8. Based on the parameter information after the adjustment of the optimal first intelligent agent, the second intelligent agent and the third intelligent agent, a reliability evaluation index system is established to evaluate the operation stability and power supply reliability of the fuel cell stack and the power system.

2. The fuel cell stack reliability assessment method based on strategy gradient and game according to claim 1 is characterized in that: Step S1 specifically includes: S11. Construct a first intelligent agent to evaluate the health status of the fuel cell stack by monitoring the key operating parameters of the fuel cell stack, namely: Where H represents the health status coefficient of the fuel cell stack, P t represents the actual output power of the fuel cell stack at time t, η t represents the rated electrical efficiency of the fuel cell stack at the tth moment, c represents the attenuation ratio limit that the fuel cell stack can withstand, η0 represents the rated electrical efficiency of the fuel cell stack at the initial moment, e represents the exponential function, N start Indicates the number of startups of the fuel cell stack, T r ′ un represents the operating time of the fuel cell stack, α, β, γ, and λ all represent weight coefficients of key operating parameters of the fuel cell stack; S12. The health status of the fuel cell stack is used as the reward function of the first agent, that is: R1=H Where R1 represents the reward function of the first agent; S13. Establish the state space of the first agent, namely: S1={P,η,N start ,T r r un ,T avg ,P m ′ ax } Where s1 represents the state space of the first agent, p represents the actual output power of the fuel cell stack, η represents the electrical efficiency of the fuel cell stack, T avg Represents the average temperature of the fuel cell stack during operation, P m ′ ax Indicates the maximum pressure during the operation of the fuel cell stack; S14. Establish the action space of the first agent, namely: Where A1 represents the action space of the first agent, T represents the temperature of the fuel cell stack, is the hydrogen partial pressure, represents the oxygen partial pressure, R′ hum Relative humidity, Indicates the excess oxygen ratio.

3. The fuel cell stack reliability assessment method based on strategy gradient and game according to claim 2 is characterized in that: Step S2 specifically includes: S21. Construct a second intelligent agent to dynamically adjust the electricity price according to the health status of the fuel cell stack and the electricity demand on the user side to obtain the goal of maximizing the electricity price profit, that is: Among them, O represents the profit maximization target of electricity price, P″ represents the electricity price, Q represents the electricity demand on the user side, C represents the total cost of the power station, and P″ base represents the basic electricity price, r represents the price adjustment rate, H threshold represents the preset health indicator threshold, D represents the basic power demand, σ represents the price elasticity coefficient, and τ represents the influence coefficient of the health status of the fuel cell stack on the power demand on the user side; S22. The profit maximization objective of the second agent is used as the reward function of the second agent, that is: R2=O Where R2 represents the reward function of the second agent; S23, establish the state space of the second agent, that is: S2={P″ base ,r,H,H threshold ,D,σ,τ} Where S2 represents the state space of the second agent; S24. Establish the action space of the second agent, namely: A2={P″} Among them, A2 represents the action space of the second agent.

4. The fuel cell stack reliability assessment method based on strategy gradient and game according to claim 3 is characterized in that: Step S3 specifically includes: S31. Construct a third intelligent agent to make a purchase decision based on the electricity price and the electricity demand on the user side, and obtain the electricity purchase cost, that is: C electricity =P″×Q Among them, C electricity represents the electricity purchase cost; S32. The negative value of the electricity purchase cost is used as the reward function of the third agent, that is: R3=-C electricity Where R3 represents the reward function of the third agent; S33, establish the state space of the third agent, namely: S3={P″,D,σ} Among them, S3 represents the state space of the third agent; S34. Establish the action space of the third agent, namely: A3={K} Among them, A3 represents the action space of the third agent, and K represents the electricity purchase amount on the user side.

5. The fuel cell stack reliability assessment method based on strategy gradient and game according to claim 4 is characterized in that: Step S4 specifically includes: S41, respectively establishing a strategy network for the first agent, the second agent, and the third agent; wherein the strategy network includes an action network and a value network; S42, respectively inputting the reward functions, state spaces, and action spaces corresponding to the first agent, the second agent, and the third agent into their respective strategy networks for training, and maximizing the cumulative reward of each agent by obtaining the strategy gradient of each agent; S43, using the Stackelberg game model to divide the first intelligent agent, the second intelligent agent and the third intelligent agent in training into leaders and followers, and updating the policy gradient of each intelligent agent; S44. Repeat steps S42-S43 until the set maximum number of iterations is reached, and the optimal first agent, second agent, and third agent are obtained.

6. The fuel cell stack reliability assessment method based on strategy gradient and game according to claim 5 is characterized in that: The policy gradient of each agent in step S42 is: in, represents the gradient, θ i represents the policy network parameters of agent i, J represents the cumulative reward of agent i, θ1, θ2, θ3 represent the policy network parameters of the first agent, the second agent, and the third agent, respectively. represents the mean calculation, Q ′ represents the value evaluation value of the strategy network, s represents the state space of agent i, a1, a2, a3 represent the action space of the first agent, the second agent, and the third agent respectively, R represents the reward of each agent, represents the weight coefficient of the policy network, Q(S - ,a - ) represents the state space S of each agent at the previous moment - With action space a _ .

7. The fuel cell stack reliability assessment method based on strategy gradient and game according to claim 6, characterized in that: In step S42, the cumulative reward of each agent is maximized, which is the total return obtained by each agent from the multi-agent environment when performing strategy optimization to maximize the action decision of each agent.

8. The fuel cell stack reliability assessment method based on strategy gradient and game according to claim 7, characterized in that: The specific process of using the Stackelberg game model in step S43 to divide the first agent, the second agent, and the third agent in training into leaders and followers is as follows: The first intelligent agent is taken as the leader, and the second and third intelligent agents are taken as the first and second followers respectively; first, the leader makes a decision, and then the first and second followers optimize their own strategies according to the decision made by the leader.

9. The fuel cell stack reliability assessment method based on strategy gradient and game according to claim 8, characterized in that: The update formula of the policy gradient of each agent in step S43 is: Among them, θ i ′ represents the updated policy network parameters of agent i, and α represents the learning rate of the policy network.

10. The fuel cell stack reliability assessment method based on strategy gradient and game according to claim 9, characterized in that: Step S8 specifically includes: S51, obtaining the optimal key operating parameters of the fuel cell stack after the optimal first intelligent agent is adjusted, and establishing an expected energy non-supply index, namely: Among them, EENS stands for expected energy non-supply indicator; S52, obtaining the optimal key operating parameters of the fuel cell stack after the optimal first intelligent agent is adjusted, and establishing a system average interruption duration index, namely: Among them, SAIDI represents the system average interruption duration index; S53. Establish the system average interruption frequency index indicator, namely: Among them, SAIFI represents the system average interruption frequency index; S54. Evaluate the operation stability and power supply reliability of the fuel cell stack and the power system based on the expected energy non-supply index, the system average interruption duration index, and the system average interruption frequency index index; S55, evaluating the operation stability of the fuel cell stack based on the optimal health state of the fuel cell stack; S56. Based on the optimal second intelligent agent, the optimal electricity price after adjustment by the third intelligent agent, and the predicted electricity purchase amount, the effectiveness of user demand side management is evaluated by evaluating the impact of electricity price adjustment on electricity purchase amount.

Images