Grid-connected control method for hydrogen fuel cell hybrid power system

Abstract

The invention discloses a hydrogen fuel cell hybrid power system grid-connected control method, which comprises the following steps of: analyzing and establishing a plurality of models of a system, comprehensively considering a plurality of cost weights on the basis of a hydrogen consumption model of the system, and constructing a comprehensive cost function; establishing a system operation state model; the speed, the acceleration and the system operation state serve as input of a self-adaptive Elman neural network together, a state transition matrix real-time correction factor of the system in the state is obtained, and a basic state transition matrix is adjusted through the real-time correction factor to obtain a real-time state transition matrix after the influence of the current driving condition on system state transition is considered; and according to the real-time state transition matrix, predicting the system demand power within a period of time, solving the comprehensive cost to enable the cost to be minimum, and completing the power distribution between the multiple stacks of fuel cells and storage batteries. According to the method, the fuel economy of the system is ensured, the internal balance of the multi-stack fuel cell system is improved, the service life of the system is prolonged, and the real-time prediction capability of the system is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of hydrogen fuel cell hybrid power systems, and in particular relates to a grid-connected control method for a hydrogen fuel cell hybrid power system. Background Art

[0002] As countries around the world pay more and more attention to environmental protection, the research and utilization of clean energy has become a trend and focus in the energy field. In this context, proton exchange membrane fuel cells, as a cutting-edge clean energy technology, are attracting the deep attention and wide praise of countries around the world. Its unique advantages, including but not limited to high-efficiency energy conversion, low emission pollution, rapid start-up response and long-term operation stability, provide strong technical support and solutions for the green transformation of the energy structure. In order to further optimize energy utilization efficiency and effectively recover energy during braking, fuel cells are usually combined with energy storage systems to build a fuel cell hybrid system. In addition, people also connect multiple fuel cells in parallel to form a multi-stack fuel cell system to break through the single-cell power bottleneck and achieve higher power output. At present, the research on the grid-connected control method of hydrogen fuel cell hybrid systems mainly has the following problems:

[0003] Insufficient consideration of system durability: Traditional grid-connected control methods consider system fuel economy and pay less attention to system durability. As a result, although the system fuel economy is good, the system service life is not long and the system maintenance cost is high.

[0004] Multi-stack fuel cell systems have poor internal balance: Traditional grid-connected control methods fine-tune the output power of individual fuel cells, but do not consider the impact of the individual on the overall output performance and stability.

[0005] State prediction is not targeted: Most of the existing state prediction models construct a unified state transfer matrix for the entire system operation process. This means that no matter what state the system is in, such as acceleration or idling, for the same speed or power, the predicted probability of the system's required power at the next moment is the same, which reduces the accuracy of the prediction.

[0006] Poor real-time prediction: Most grid-connected control methods based on model prediction build a state transfer matrix based on historical data, system characteristics and expert knowledge. It describes how the system state changes when there is no real-time information input. The system is constantly changing during operation, and the state transfer matrix also changes differently at different times. Traditional methods do not consider the real-time performance of each system operation. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes a hydrogen fuel cell hybrid power system grid-connected control method, which improves the internal balance of the multi-stack fuel cell system while ensuring the system fuel economy, prolongs the system service life, and improves the system's real-time prediction capability.

[0008] To achieve the above object, the technical solution adopted by the present invention is: a grid-connected control method for a hydrogen fuel cell hybrid power system, comprising the steps of:

[0009] S100 analyzes and establishes a system hydrogen consumption model, a fuel cell and battery performance degradation model, a multi-stack fuel cell system internal balance evaluation model, and a battery state of charge constraint model. On the basis of the system hydrogen consumption model, the additional fuel cell cost weight caused by the internal imbalance of the multi-stack fuel cell system, the additional battery cost weight caused by battery performance degradation, and the additional battery cost weight caused by the battery state of charge constraint are comprehensively considered to construct a comprehensive cost function of the system;

[0010] S200 establishes a system operation status model, determines the system operation status according to the required power, speed and acceleration of the system operation, and constructs a state transfer matrix under different system states through historical operation data;

[0011] S300 uses the speed and acceleration of the system together with the system operating state as the input of the adaptive Elman neural network for the system at time t, obtains the real-time correction factor of the state transfer matrix of the system in this state, and uses the real-time correction factor to adjust the basic state transfer matrix to obtain the real-time state transfer matrix after considering the influence of the current driving conditions on the system state transfer;

[0012] S400 predicts the system power demand within a period of time according to the real-time state transfer matrix in step S300, and uses a deep reinforcement learning method based on twin delayed deep deterministic policy gradient to solve the comprehensive cost in the form of increments during this period to minimize it, and completes the power allocation between multiple fuel cells and batteries at this moment;

[0013] S500 determines whether the operation of the hybrid power system is finished. If so, the whole process of the hydrogen fuel cell hybrid power system is connected to the grid and controlled. Otherwise, the process goes to the next moment and returns to step S300.

[0014] Furthermore, the hydrogen consumption of the hydrogen fuel cell hybrid system is composed of the fuel cell hydrogen consumption and the battery equivalent hydrogen consumption. The system hydrogen consumption model is:

[0015] H sys =f1(P fc )+kf2(P bat );

[0016] Among them, H sysis the hydrogen consumption of the multi-stack fuel cell system; f1 function is the fuel cell hydrogen consumption function; k is the charge state correction coefficient of the battery; f2 function is the equivalent hydrogen consumption function of the lithium battery; P fc and P bat Represent the output power of fuel cell and battery respectively;

[0017] The performance degradation of the fuel cell is related to the output power. The fuel cell performance degradation model is:

[0018] D fc_loss =g(P fc,t ,P fc,t-1 );

[0019] Among them, D fc_loss is the performance decay of the fuel cell; g represents the performance decay function of the fuel cell; P fc,t represents the output power of the fuel cell at time t, P fc,t-1 Represents the output power of the fuel cell at time t-1.

[0020] Furthermore, the battery performance degradation is included in the energy management target, and the additional battery cost weight caused by the battery performance degradation is:

[0021]

[0022] Among them, γ1 represents the additional battery cost weight caused by battery performance degradation; P bat Indicates the output power of the battery; N cycle is the number of cycles of the battery; Q bat is the rated capacity of the battery; t0 represents the initial time of system operation; ΔT represents the total operation time of the system; t represents time;

[0023] If the battery state of charge is too high or too low for a long time, its life will be reduced. The additional battery cost weight caused by the battery state of charge constraint is:

[0024]

[0025] Where γ2 represents the additional battery cost weight caused by the battery state of charge constraint; SOC represents the battery state of charge; SOC high and SOC low Respectively represent the upper and lower limits of the set battery state of charge;

[0026] The consistency of the service life of multiple stack fuel cells is conducive to extending the service life of the multi-stack system. The weight of the additional fuel cell cost caused by the internal imbalance of the multi-stack fuel cell system is:

[0027]

[0028] Among them, w i represents the additional fuel cell cost weight caused by the internal imbalance of the multi-stack fuel cell system; D fci,t Indicates the real-time attenuation of the fuel cell; D fci,init is the initial attenuation degree of the fuel cell before it is put into use; i represents the i-th fuel cell, and the serial numbers of the fuel cells are sorted from small to large according to their performance attenuation degree, that is, the weight of the fuel cell with the smallest attenuation degree is 0.

[0029] Furthermore, considering the overall optimization of the system, the comprehensive cost of the system includes the system's operating hydrogen consumption, power source performance degradation, battery state of charge constraints, and the internal balance of the multi-stack fuel cell system. The comprehensive cost function of the system is:

[0030]

[0031] Where J represents the comprehensive cost function of the system; α i , β1 and β2 are the additional cost coefficients of coordination of multiple fuel cell stacks, additional cost coefficients of battery performance degradation and additional cost coefficients of battery state of charge constraint respectively; m fc represents the fuel cell cost; m bat Represents the battery cost; C fci represents the hydrogen consumption of the i-th fuel cell; C bat represents the equivalent hydrogen consumption of the battery; n represents the number of fuel cells contained in the fuel cell system.

[0032] Furthermore, the hydrogen consumption functions of the fuel cell and the battery are fitted with quadratic functions, and the system comprehensive cost function is converted into an incremental form with a similar form. The incremental form comprehensive cost function is established as:

[0033]

[0034] in,

[0035]

[0036] Where ΔJ represents the comprehensive cost increment of the system; Δm fc and Δm bat represents the cost increment of fuel cell and battery respectively; v∈{fcibat}; ΔCv represents the hydrogen consumption increment of fuel cell or battery; t represents time t; P v,t and ΔP v,t Respectively represent the output power and increment of the fuel cell or battery at time t; a v and b vare the quadratic coefficient and the linear coefficient of the hydrogen consumption of the fuel cell or battery with respect to its output power; n represents the number of fuel cells contained in the fuel cell system.

[0037] Furthermore, a system operation state model is established to judge the system operation state according to the required power, speed and acceleration of the system operation, and a state transfer matrix under different system states is constructed through historical operation data; constructing separate state transfer matrices for different states is conducive to refining the changes of the system under different states;

[0038] The operating state identification model is:

[0039] OS(t)=sign(P(t))+sign(v(t))+sign(a(t));

[0040] OS is the system's operating state identification function; sign is the sign function; P(t) is the system's required power, v(t) and a(t) represent the system's speed and acceleration at this time, respectively;

[0041] When OS(t) is -1, 0, 1, 2, 3 respectively, it means that the system is in the state of regenerative braking, idling traction, deceleration braking, steady speed driving and acceleration start.

[0042] Furthermore, the speed and acceleration of the system together with the basic state transfer matrix are used as inputs of the adaptive Elman neural network, and the output is the real-time correction factor of the state transfer matrix of the system in this state. The real-time correction factor is used to adjust the basic state transfer matrix to obtain the real-time state transfer matrix after considering the influence of the current driving conditions on the system state transfer;

[0043] The basic state transfer matrix is ​​corrected according to the real-time information of the system. The correction formula is:

[0044] P′=a v,a ·P;

[0045] Among them, α v,a It is the correction factor corresponding to the current running state, speed and acceleration; P is the basic state transfer matrix; P' is the real-time state transfer matrix.

[0046] Furthermore, the training process of the adaptive Elman neural network is as follows: obtain the five-tuple (P t ,v,a,P,P t+1 ), respectively representing the current power, speed, acceleration, basic state transfer matrix and power at the next moment; the probability correction factor α is obtained from the speed and acceleration v,a , the basic probability transfer matrix P is modified by the correction formula and then applied to the current power P t, get the predicted power, and compare it with the actual power P t+1 Compare and adjust the network weights to minimize the loss function, and repeat the above process until the network converges. In the process of adjusting the network weights, the learning rate of the algorithm is adaptively adjusted according to the training situation, considering increasing the search ability in the early stage of training and focusing on the convergence ability in the later stage.

[0047] Furthermore, the real-time state transfer matrix of the current system is determined, and the system power demand within a period of time is predicted based on the real-time state transfer matrix. The deep reinforcement learning method based on the twin delayed deep deterministic policy gradient is used to solve the comprehensive cost of the incremental form within this period to minimize it.

[0048] The twin delayed deep deterministic policy gradient algorithm includes six networks, namely Actor network μ(·|θ μ ), Critic1 network Q1(·|θ Q1 ), Critic2 network Q2(·|θ Q2 ), TargetActor network μ'(·|θ μ '), TargetCritic1 network Q'1(·|θ Q ' 1 ) and TargetCritic2 network Q'2(·|θ Q ' 2 ); the Actor network is updated by maximizing the cumulative expected return, the Critic1 and Critic2 networks are updated by minimizing the error between the evaluation value and the target value, and all target networks are updated by soft update; ultimately, the corresponding system actions are adopted in different system states to maximize the reward for interacting with the system;

[0049] The grid-connected control of a hydrogen fuel cell hybrid system using a twin-delayed deep deterministic policy gradient-based deep reinforcement learning method includes the following steps:

[0050] (1) Data preparation and network initialization: Establish the state space and action space of the hydrogen fuel cell hybrid system, build the target network structure and initialize it. The twin delayed deep deterministic policy gradient network consists of an Actor network and two Critic networks and their corresponding Target networks.

[0051] (2) Network training: Training the initialized twin delayed deep deterministic policy gradient network to obtain the mapping relationship between the system actions that interact with the system and obtain the highest reward under different system states; including the following steps:

[0052] ① Critic1 and Critic2 network update process: Use the TargetActor network to calculate the action under state s', then add noise to the target action a' based on the target strategy smoothing regularization method to improve the accuracy of target value estimation and ensure the stability of the network training process; estimate the target value through two sets of Critic networks with the same network architecture. Here, the two sets of Critic networks with the same network architecture will use the smaller value between the two to estimate the state action value of the next state action pair (s', a') when calculating the target value, thereby effectively avoiding the overestimation caused by the maximization problem; use the gradient descent algorithm to minimize the error between the evaluation value and the target value, and update the parameters in the Critic1 and Critic2 networks;

[0053] ②Actor network update process: Considering the stability of the training process, the Actor network is updated with a delay, that is, the Critic network is updated multiple times before the Actor network is updated;

[0054] ③ Update process of the target network: The target network is updated by soft update, which is usually achieved by introducing a learning rate to weighted average the old target network parameters and the new corresponding network parameters, and then assigning them to the target network;

[0055] ④Training end condition: When all networks converge, the twin delayed deep deterministic policy gradient network training is completed;

[0056] (3) Real-time model evaluation: The trained twin delayed deep deterministic policy gradient network is used for implementation evaluation during the operation of the hydrogen fuel cell hybrid system to complete the grid-connected control process;

[0057] (4) Network optimization: Optimize network parameters.

[0058] Furthermore, the fuel cell hybrid system needs to be subject to physical constraints during actual operation, including maximum and minimum limits on the output power of the fuel cell and the battery, maximum and minimum limits on the battery state of charge, the fuel cell output power change rate not exceeding 12.5% ​​of its rated power, and the matching of the power source output power and the load power. The relationship is expressed as:

[0059]

[0060] P fci is the output power of the i-th fuel cell; P fcimax and P fcimin are the maximum and minimum output power of the i-th fuel cell respectively; SOC is the state of charge of the battery; SOC max and SOC minare the maximum and minimum charge states of the battery respectively; P bat is the output power of the battery; P batmax and P batmin are the maximum and minimum output power of the battery respectively; ΔP fci is the output power increment of the i-th fuel cell; P fci,rated is the rated power of the i-th fuel cell; P load is the power required when the system is running; n is the number of fuel cells included in the fuel cell system.

[0061] The beneficial effects of adopting this technical solution are:

[0062] The present invention discloses a method for grid-connected control of a hydrogen fuel cell hybrid power system that takes into account both economy and durability. The invention determines the operating state of the system, predicts the system power demand in the future based on the real-time state transfer matrix, and then takes the minimum comprehensive cost of the system as the goal, and uses a deep reinforcement learning method based on a twin delayed deep deterministic policy gradient to optimize the output power of the multi-stack fuel cell system and the battery system, thereby achieving an effective allocation of the power demand of the hybrid power system. In a hybrid power system, the fuel cell serves as the main power source of the system, providing most of the power required when the system is running, and the battery system serves as an auxiliary power source to supplement the power shortage. The present invention improves the internal balance of the multi-stack fuel cell system while ensuring the fuel economy of the system, prolongs the service life of the system, and improves the system's real-time prediction capabilities.

[0063] The grid-connected control method of the hydrogen fuel cell hybrid power system proposed in the present invention establishes an optimization target for minimizing the overall system cost, and realizes comprehensive optimization of the overall system hydrogen consumption, power source performance degradation, internal balance of the multi-stack fuel cell system, and battery charge state constraints.

[0064] The present invention focuses on improving the durability of the system, especially the durability of the multi-stack fuel cell system. The present invention takes the fuel cell with the most serious performance degradation as the benchmark, assigns a smaller evaluation weight to the fuel cell with less performance degradation, and assigns a larger evaluation weight to the fuel cell with greater performance degradation, thereby guiding the performance of the multi-stack fuel cells to gradually balance and improving the service life of the multi-stack fuel cell system.

[0065] The present invention obtains the correction factor of the basic state transfer matrix by collecting the real-time speed and real-time acceleration of the system as the input of the adaptive Elman neural network, thereby improving the disadvantage that the state transfer matrix of the traditional method is only obtained based on historical data during the prediction process and lacks real-time performance, making the prediction process more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1The present invention is a schematic flow chart of a method for controlling a grid-connected hydrogen fuel cell hybrid power system. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described below with reference to the accompanying drawings.

[0068] The present invention discloses a grid-connected control method for a hydrogen fuel cell hybrid power system that takes into account both economy and durability. The method establishes a system hydrogen consumption model, a fuel cell and battery performance attenuation model, a multi-stack fuel cell system internal balance evaluation model, and a battery charge state constraint model; comprehensively considers the above four factors, and merges multiple objectives into a system comprehensive cost function; judges the state of system operation according to the required power, speed, and acceleration of the system operation, and obtains a basic state transfer matrix through historical operation data. The basic state transfer matrix is ​​constructed based on historical data, system characteristics, and expert knowledge. It describes how the system state is transferred when there is no real-time information input. In order to improve real-time performance, the probability correction factor under the real-time speed and acceleration of the system is obtained through an adaptive Elman neural network, and it is corrected in real time to obtain the real-time state transfer matrix under different system states, thereby correcting different The system can make targeted real-time predictions in the state, so that the system has good adaptability in real-time prediction; during the operation of the system, the output voltage, output current and other information of the fuel cell and battery are collected, the hydrogen consumption of the power system is calculated by establishing a good model, the performance attenuation of the fuel cell and battery is evaluated, and the system comprehensive cost function at that moment is obtained. The state of the system operation is judged according to the required power, speed and acceleration of the system operation, and the system required power is predicted within a period of time according to the real-time state transfer matrix. Then, the deep reinforcement learning method based on the twin delayed deep deterministic policy gradient is used to obtain the optimal power allocation result of the system within this period of time with the goal of minimizing the system comprehensive cost, thereby realizing the grid-connected control of the hydrogen fuel cell hybrid system. This method improves the balance inside the multi-stack fuel cell system while ensuring the fuel economy of the system, prolongs the service life of the system, and improves the real-time prediction capability of the system. In addition, during the implementation of the proposed method, the operation of the hydrogen fuel cell hybrid system is restricted by the output characteristics of the power source and physical conditions.

[0069] In this embodiment, see Figure 1 As shown, the present invention proposes a grid-connected control method for a hydrogen fuel cell hybrid power system, comprising the steps of:

[0070] S100 analyzes and establishes a system hydrogen consumption model, a fuel cell and battery performance degradation model, a multi-stack fuel cell system internal balance evaluation model, and a battery state of charge constraint model. On the basis of the system hydrogen consumption model, it comprehensively considers the additional fuel cell cost weight caused by the internal imbalance of the multi-stack fuel cell system, the additional battery cost weight caused by battery performance degradation, and the additional battery cost weight caused by battery state of charge constraints to construct a comprehensive cost function of the system.

[0071] (1) System hydrogen consumption: including the hydrogen consumption of the fuel cell and the equivalent hydrogen consumption of the battery. The hydrogen consumption model established during the system operation is:

[0072] H sys =f1(P fc )+kf2(P bat );

[0073] Among them, H sys is the hydrogen consumption of the multi-stack fuel cell system; f1 function is the fuel cell hydrogen consumption function, usually a quadratic polynomial about its output power; k is the charge state correction coefficient of the battery; f2 function is the equivalent hydrogen consumption function of the lithium battery, usually a quadratic polynomial about the output power of the lithium battery; P fc and P bat Represent the output power of the fuel cell and battery respectively.

[0074] (2) Fuel cell performance degradation: The performance degradation of a fuel cell is related to its output power. The fuel cell degradation model is:

[0075] D fc_loss =g(P fc,t ,P fc,t-1 );

[0076] Where D fc_loss is the performance decay of the fuel cell; g represents the performance decay function of the fuel cell; P fc,t represents the output power of the fuel cell at time t.

[0077] Specifically, the performance degradation of a fuel cell is related to its start-stop, high-power operation, low-power operation, and load variation, which can be expressed as follows:

[0078] D fc_loss =D on / off +D low +D high +D chg ;

[0079] in,

[0080]

[0081] Where Plowlim , P highlim They represent the upper limit of the low power operation range and the lower limit of the high power operation range of the fuel cell respectively; n on / off is the number of starts and stops during the operation of the fuel cell; t low and t high They represent the time when the fuel cell operates at low power and high power respectively; e1, e2, e3 and e4 represent the attenuation coefficients of the fuel cell in four states respectively; D on / off , D low , D high , D chg They respectively represent the performance degradation of the fuel cell caused by start-stop, low-power operation, high-power operation and variable load operation.

[0082] (3) Constructing the additional hydrogen consumption weight due to battery performance degradation: The battery performance degradation model is related to its discharge depth. The additional battery cost weight due to battery performance degradation is:

[0083]

[0084] Among them, γ1 represents the additional battery cost weight caused by battery performance degradation; P bat Indicates the output power of the battery; N cycle is the number of cycles of the battery; Q bat is the rated capacity of the battery; t0 represents the initial time of system operation; ΔT represents the total operation time of the system; t represents time.

[0085] (4) Constructing the additional hydrogen consumption weight of battery state of charge loss: If the battery state of charge is too high or too low for a long time, its life will be reduced. The additional battery cost weight caused by the battery state of charge constraint is:

[0086]

[0087] Where γ2 represents the additional battery cost weight caused by the battery state of charge constraint; SOC represents the battery state of charge; SOC high and SOC low They respectively represent the set upper and lower limits of the battery state of charge.

[0088] (5) Constructing the additional hydrogen consumption weight for the balance of multi-stack fuel cells: The consistent service life of the multi-stack fuel cells is conducive to extending the service life of the multi-stack system. The additional fuel cell cost weight caused by the internal imbalance of the multi-stack fuel cell system is:

[0089]

[0090] Among them, w irepresents the additional fuel cell cost weight caused by the internal imbalance of the multi-stack fuel cell system; D fci,t Indicates the real-time attenuation of the fuel cell; D fci,init is the initial attenuation degree of the fuel cell before it is put into use; i represents the i-th fuel cell, and the serial numbers of the fuel cells are sorted from small to large according to their performance attenuation degree, that is, the weight of the fuel cell with the smallest attenuation degree is 0.

[0091] (6) Constructing the comprehensive cost index of the system: including the system's operating hydrogen consumption, power source performance degradation, battery charge state constraints, and the internal balance of the multi-stack fuel cell system. The comprehensive cost function of the system is:

[0092]

[0093] Where J represents the comprehensive cost function of the system; α i , β1 and β2 are the additional cost coefficients of coordination of multiple fuel cell stacks, additional cost coefficients of battery performance degradation and additional cost coefficients of battery state of charge constraint respectively; m fc represents the fuel cell cost; m bat Represents the battery cost; C fci represents the hydrogen consumption of the i-th fuel cell; C bat represents the equivalent hydrogen consumption of the battery; n represents the number of fuel cells contained in the fuel cell system.

[0094] Preferably, a comprehensive cost function in the form of system increments is constructed: the hydrogen consumption functions of the fuel cell and the battery can be fitted with quadratic functions. After the comprehensive cost function is converted into an incremental form, the hydrogen consumption increment is positively correlated with the power increment of each power source. In the process of reducing the comprehensive cost, the power increment of each power source is also constrained, which is conducive to improving the durability of the system. The incremental form comprehensive cost function is:

[0095]

[0096] in,

[0097]

[0098] Where ΔJ represents the comprehensive cost increment of the system; Δm fc and Δm bat represents the cost increment of fuel cell and battery respectively; v∈{fci bat}; ΔCv represents the hydrogen consumption increment of fuel cell or battery; t represents time t; P v,t and ΔP v,t Respectively represent the output power and increment of the fuel cell or battery at time t; a v and b vare the quadratic coefficient and the linear coefficient of the hydrogen consumption of the fuel cell or battery with respect to its output power; n represents the number of fuel cells contained in the fuel cell system.

[0099] S200 establishes a system operation status model, determines the system operation status according to the required power, speed and acceleration of the system operation, and constructs a state transfer matrix under different system states through historical operation data.

[0100] Constructing separate state transfer matrices for different states is helpful to refine the changes of the system in different states and make the prediction more accurate; the operating state identification model is as follows:

[0101] OS(t)=sign(P(t))+sign(v(t))+sign(a(t));

[0102] OS is the system's operating status identification function; sign is the sign function; P(t) is the system's required power, v(t) and a(t) represent the system's speed and acceleration at this time, respectively.

[0103] When OS(t) is -1, 0, 1, 2, 3 respectively, it means that the system is in the state of regenerative braking, idling traction, deceleration braking, steady speed driving and acceleration start.

[0104] For the system at time t, S300 uses the system speed and acceleration together with the system operating state as the input of the adaptive Elman neural network to obtain the real-time correction factor of the state transfer matrix of the system in this state. The real-time correction factor is used to adjust the basic state transfer matrix to obtain the real-time state transfer matrix after considering the impact of the current driving conditions on the system state transfer.

[0105] (1) Correction process: The speed and acceleration of the system together with the basic state transfer matrix are used as the input of the adaptive Elman neural network, and the output is the real-time correction factor of the state transfer matrix of the system in this state. The real-time correction factor is used to adjust the basic state transfer matrix to obtain the real-time state transfer matrix after considering the impact of the current driving conditions on the system state transfer. Since the basic state transfer matrix is ​​constructed based on historical data, system characteristics and expert knowledge, it describes how the system state is transferred when there is no real-time information input. However, the real-time information is constantly changing during the operation of the system, which makes the prediction of future working conditions based on the basic state transfer matrix lack of real-time performance. Therefore, it is necessary to correct the basic state transfer matrix according to the real-time information of the system. The correction process is as follows:

[0106] P′=α v,a ·P;

[0107] α v,aIt is the correction factor corresponding to the current running state, speed and acceleration; P is the basic state transfer matrix; P' is the real-time state transfer matrix.

[0108] (2) Training process: Obtaining quintuples (P) from historical data t ,v,a,P,P t+1 ), respectively representing the current power, speed, acceleration, basic state transfer matrix and power at the next moment; the probability correction factor α is obtained from the speed and acceleration v,a , the basic probability transfer matrix P is modified by the correction formula in claim 10 and then applied to the current power P t , get the predicted power, and compare it with the actual power P t+1 Compare and adjust the network weights to minimize the loss function, and repeat the above process until the network converges. In the process of adjusting the network weights, the learning rate of the algorithm is adaptively adjusted according to the training situation, considering increasing the search ability in the early stage of training and focusing on the convergence ability in the later stage.

[0109] S400 predicts the system power demand within a period of time based on the real-time state transfer matrix in step S300, and uses a deep reinforcement learning method based on twin delayed deep deterministic policy gradient to solve the comprehensive cost in incremental form during this period to minimize it, and completes the power allocation between multiple fuel cells and batteries at this moment.

[0110] (1) Analysis and solution scope: The physical constraints of the fuel cell hybrid system can be expressed as:

[0111]

[0112] P fci is the output power of the i-th fuel cell; P fcimax and P fcimin are the maximum and minimum output power of the i-th fuel cell respectively; SOC is the state of charge of the battery; SOC max and SOC min are the maximum and minimum charge states of the battery respectively; P bat is the output power of the battery; P batmax and P batmin are the maximum and minimum output power of the battery respectively; ΔP fci is the output power increment of the i-th fuel cell; P fci,rated is the rated power of the i-th fuel cell; P load is the power required when the system is running; n is the number of fuel cells included in the fuel cell system.

[0113] (2) A deep reinforcement learning method based on twin-delayed deep deterministic policy gradient is used to solve the problem: the system power demand within a period of time is predicted according to the real-time state transfer matrix under the current state, and the deep reinforcement learning method based on twin-delayed deep deterministic policy gradient is used to solve the comprehensive cost of the incremental form within this period to minimize it; the twin-delayed deep deterministic policy gradient algorithm is a deterministic deep reinforcement learning algorithm under the Actor-Critic (AC) framework, which includes six networks, namely, Actor network μ(·|θ μ ), Critic1 network Q1(·|θ Q1 ), Critic2 network Q2(·|θ Q2 ), Target Actor network μ'(·|θ μ '), Target Critic1 network Q'1(·|θ Q ' 1 ) and TargetCritic2 network Q'2(·|θ Q ' 2 ), which makes the twin delayed deep deterministic policy gradient algorithm have good stability and robustness; the Actor network is updated by maximizing the cumulative expected return, and the Critic1 and Critic2 networks are updated by minimizing the error between the evaluation value and the target value. All target networks are updated by soft update; finally, the corresponding system actions are adopted in different system states to maximize the reward obtained by interacting with the system; specifically, the use of the deep reinforcement learning method based on the twin delayed deep deterministic policy gradient to control the grid connection of the hydrogen fuel cell hybrid system includes the following steps:

[0114] (1) Data preparation and network initialization: Establish the state space and action space of the hydrogen fuel cell hybrid system, build the target network structure and initialize it. The twin delayed deep deterministic policy gradient network consists of an Actor network and two Critic networks and their corresponding Target networks.

[0115] (2) Network training: Train the initialized twin delayed deep deterministic policy gradient network to obtain the mapping relationship between the system actions that interact with the system under different system states and obtain the highest reward. Specifically, the following steps are included:

[0116] ① Critic1 and Critic2 network update process: Use the Target Actor network to calculate the action under state s', then add noise to the target action a' based on the target strategy smoothing regularization method to improve the accuracy of target value estimation and ensure the stability of the network training process; estimate the target value through two sets of Critic networks with the same network architecture. Here, the two sets of Critic networks with the same network architecture will use the smaller value between the two to estimate the state action value of the next state action pair (s', a') when calculating the target value, thereby effectively avoiding the overestimation caused by the maximization problem; use the gradient descent algorithm to minimize the error between the evaluation value and the target value, and update the parameters in the Critic1 and Critic2 networks;

[0117] ②Actor network update process: Considering the stability of the training process, the Actor network is updated with a delay, that is, the Critic network is updated multiple times before the Actor network is updated;

[0118] ③ Update process of the target network: The target network is updated by soft update, which is usually achieved by introducing a learning rate to weighted average the old target network parameters and the new corresponding network parameters, and then assigning them to the target network;

[0119] ④Training end condition: When all networks converge, the twin delayed deep deterministic policy gradient network training is completed;

[0120] (3) Real-time model evaluation: The trained twin delayed deep deterministic policy gradient network is used for implementation evaluation during the operation of the hydrogen fuel cell hybrid system to complete the grid-connected control process;

[0121] (4) Network optimization: Optimize network parameters (such as learning rate, number of network layers, etc.) to achieve better grid-connected control effect.

[0122] S500 determines whether the operation of the hybrid power system is finished. If so, the whole process of the hydrogen fuel cell hybrid power system is connected to the grid and controlled. Otherwise, the process goes to the next moment and returns to step S300.

[0123] The grid-connected control method of the hydrogen fuel cell hybrid system proposed in the present invention can optimize the operation of the system from multiple angles, greatly improve the durability of the system, especially the durability of the multi-stack fuel cell system, and realize the distribution of the load power of the hydrogen fuel cell hybrid system between the multi-stack fuel cells and the battery with the goal of minimizing the overall cost of the system, while ensuring the fuel economy of the system and improving the balance within the multi-stack fuel cell system. In addition, the present invention improves the prediction method, corrects the basic state transfer matrix with real-time information of speed and acceleration, and improves the real-time prediction capability.

[0124] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for controlling a hydrogen fuel cell hybrid power system in grid connection, characterized in that: Includes steps: S100 analyzes and establishes a system hydrogen consumption model, a fuel cell and battery performance degradation model, a multi-stack fuel cell system internal balance evaluation model, and a battery state of charge constraint model. On the basis of the system hydrogen consumption model, the additional fuel cell cost weight caused by the internal imbalance of the multi-stack fuel cell system, the additional battery cost weight caused by battery performance degradation, and the additional battery cost weight caused by the battery state of charge constraint are comprehensively considered to construct a comprehensive cost function of the system; S200 establishes a system operation status model, determines the system operation status according to the required power, speed and acceleration of the system operation, and constructs a state transfer matrix under different system states through historical operation data; S300 uses the speed and acceleration of the system together with the system operating state as the input of the adaptive Elman neural network for the system at time t, obtains the real-time correction factor of the state transfer matrix of the system in this state, and uses the real-time correction factor to adjust the basic state transfer matrix to obtain the real-time state transfer matrix after considering the influence of the current driving conditions on the system state transfer; S400 predicts the system power demand within a period of time according to the real-time state transfer matrix in step S300, and uses a deep reinforcement learning method based on twin delayed deep deterministic policy gradient to solve the comprehensive cost in the form of increments during this period to minimize it, and completes the power allocation between multiple fuel cells and batteries at this moment; S500 determines whether the operation of the hybrid power system is completed. If it is completed, the whole process of the hydrogen fuel cell hybrid power system is connected to the grid and controlled. Otherwise, it enters the next moment and returns to step S300.

2. A hydrogen fuel cell hybrid power system grid connection control method according to claim 1, characterized in that: The hydrogen consumption of the hydrogen fuel cell hybrid system is composed of the fuel cell hydrogen consumption and the battery equivalent hydrogen consumption. The system hydrogen consumption model is: H sys =f1(P fc )+kf2(P bat ); Among them, H sys is the hydrogen consumption of the multi-stack fuel cell system; f1 function is the fuel cell hydrogen consumption function; k is the charge state correction coefficient of the battery; f2 function is the equivalent hydrogen consumption function of the lithium battery; P fc and P bat Represent the output power of fuel cell and battery respectively; The performance degradation of the fuel cell is related to the output power. The fuel cell performance degradation model is: D fc_loss =g(P fc,t ,P fc,t-1 ); Among them, D fc_loss is the performance decay of the fuel cell; g represents the performance decay function of the fuel cell; P fc,t represents the output power of the fuel cell at time t, P fc,t-1 Represents the output power of the fuel cell at time t-1.

3. A hydrogen fuel cell hybrid power system grid connection control method according to claim 2, characterized in that: The battery performance degradation is included in the energy management target, and the additional battery cost weight caused by the battery performance degradation is: Among them, γ1 represents the additional battery cost weight caused by battery performance degradation; P bat Indicates the output power of the battery; N cycle is the number of cycles of the battery; Q bat is the rated capacity of the battery; t0 represents the initial time of system operation; ΔT represents the total operation time of the system; t represents time; If the battery state of charge is too high or too low for a long time, its life will be reduced. The additional battery cost weight caused by the battery state of charge constraint is: Where γ2 represents the additional battery cost weight caused by the battery state of charge constraint; SOC represents the battery state of charge; SOC high and SOC low Respectively represent the upper and lower limits of the set battery state of charge; The consistency of the service life of multiple stack fuel cells is conducive to extending the service life of the multi-stack system. The weight of the additional fuel cell cost caused by the internal imbalance of the multi-stack fuel cell system is: Among them, w i represents the additional fuel cell cost weight caused by the internal imbalance of the multi-stack fuel cell system; D fci,t Indicates the real-time attenuation of the fuel cell; D fci,init is the initial attenuation degree of the fuel cell before it is put into use; i represents the i-th fuel cell, and the serial numbers of the fuel cells are sorted from small to large according to their performance attenuation degree, that is, the weight of the fuel cell with the smallest attenuation degree is 0.

4. A method for controlling a hydrogen fuel cell hybrid power system in grid connection according to any one of claim 3, characterized in that: Considering the overall optimization of the system, the comprehensive cost of the system includes the system's operating hydrogen consumption, power source performance degradation, battery charge state constraints, and the internal balance of the multi-stack fuel cell system. The comprehensive cost function of the system is: Where J represents the comprehensive cost function of the system; α i , β1 and β2 are the additional cost coefficients of coordination of multiple fuel cell stacks, additional cost coefficients of battery performance degradation and additional cost coefficients of battery state of charge constraint respectively; m fc represents the fuel cell cost; m bat Represents the battery cost; C fci represents the hydrogen consumption of the i-th fuel cell; C bat represents the equivalent hydrogen consumption of the battery; n represents the number of fuel cells contained in the fuel cell system.

5. A hydrogen fuel cell hybrid power system grid connection control method according to claim 4, characterized in that: The hydrogen consumption functions of fuel cells and batteries are fitted with quadratic functions, and the system comprehensive cost function is converted into an incremental form with similar form. The incremental form comprehensive cost function is established as follows: in, Where ΔJ represents the comprehensive cost increment of the system; Δm fc and Δm bat represents the cost increment of fuel cell and battery respectively; v∈{fcibat}; ΔCv represents the hydrogen consumption increment of fuel cell or battery; t represents time t; P v,t and ΔP v,t Respectively represent the output power and increment of the fuel cell or battery at time t; a v and b v are the quadratic coefficient and the linear coefficient of the hydrogen consumption of the fuel cell or battery with respect to its output power; n represents the number of fuel cells contained in the fuel cell system.

6. A hydrogen fuel cell hybrid power system grid connection control method according to claim 1, characterized in that: Establish a system operation status model, judge the system operation status according to the required power, speed and acceleration of the system operation, and build a state transfer matrix under different system states through historical operation data; building a separate state transfer matrix for different states is conducive to refining the changes of the system under different states; The operating state identification model is: OS(t)=sign(P(t))+sign(v(t))+sign(a(t)); OS is the system's operating state identification function; sign is the sign function; P(t) is the system's required power, v(t) and a(t) represent the system's speed and acceleration at this time, respectively; When OS(t) is -1, 0, 1, 2, 3 respectively, it means that the system is in the state of regenerative braking, idling traction, deceleration braking, steady speed driving and acceleration start.

7. A hydrogen fuel cell hybrid power system grid connection control method according to claim 1, characterized in that: The speed and acceleration of the system together with the basic state transfer matrix are used as the input of the adaptive Elman neural network, and the output is the real-time correction factor of the state transfer matrix of the system in this state. The real-time correction factor is used to adjust the basic state transfer matrix to obtain the real-time state transfer matrix after considering the influence of the current driving conditions on the system state transfer; The basic state transfer matrix is ​​corrected according to the real-time information of the system. The correction formula is: P'=α v,a ·P; Among them, α v,a It is the correction factor corresponding to the current running state, speed and acceleration; P is the basic state transfer matrix; P' is the real-time state transfer matrix.

8. A hydrogen fuel cell hybrid power system grid connection control method according to claim 7, characterized in that: The training process of the adaptive Elman neural network is as follows: obtain the five-tuple (P t ,v,a,P,P t+1 ), respectively representing the current power, speed, acceleration, basic state transfer matrix and power at the next moment; the probability correction factor α is obtained from the speed and acceleration v,a , the basic probability transfer matrix P is modified by the correction formula and then applied to the current power P t , get the predicted power, and compare it with the actual power P t+1 Compare and adjust the network weights to minimize the loss function, and repeat the above process until the network converges. In the process of adjusting the network weights, the learning rate of the algorithm is adaptively adjusted according to the training situation, considering increasing the search ability in the early stage of training and focusing on the convergence ability in the later stage.

9. The method for controlling a hydrogen fuel cell hybrid power system in grid connection according to claim 1, characterized in that: Determine the real-time state transfer matrix of the current system, predict the system power demand within a period of time based on the real-time state transfer matrix, and use the deep reinforcement learning method based on the twin delayed deep deterministic policy gradient to solve the comprehensive cost of the incremental form during this period to minimize it; The twin delayed deep deterministic policy gradient algorithm includes six networks, namely Actor network μ(·|θ μ ), Critic1 network Q1(·|θ Q1 ), Critic2 network Q2(·|θ Q2 ), Target Actor network μ'(·|θ μ '), TargetCritic1 network Q'1(·|θ Q ' 1 ) and Target Critic2 network Q'2(·|θ Q ' 2 ); the Actor network is updated by maximizing the cumulative expected return, the Critic1 and Critic2 networks are updated by minimizing the error between the evaluation value and the target value, and all target networks are updated by soft update; ultimately, the corresponding system actions are adopted in different system states to maximize the reward for interacting with the system; The grid-connected control of a hydrogen fuel cell hybrid system using a twin-delayed deep deterministic policy gradient-based deep reinforcement learning method includes the following steps: (1) Data preparation and network initialization: Establish the state space and action space of the hydrogen fuel cell hybrid system, build the target network structure and initialize it. The twin delayed deep deterministic policy gradient network consists of an Actor network and two Critic networks and their corresponding Target networks. (2) Network training: Train the initialized twin delayed deep deterministic policy gradient network to obtain the mapping relationship between the system actions that interact with the system to obtain the highest reward under different system states; Includes steps: ① Critic1 and Critic2 network update process: Use the Target Actor network to calculate the action under state s', then add noise to the target action a' based on the target strategy smoothing regularization method to improve the accuracy of target value estimation and ensure the stability of the network training process; estimate the target value through two sets of Critic networks with the same network architecture. Here, the two sets of Critic networks with the same network architecture will use the smaller value between the two to estimate the state action value of the next state action pair (s', a') when calculating the target value, thereby effectively avoiding the overestimation caused by the maximization problem; use the gradient descent algorithm to minimize the error between the evaluation value and the target value, and update the parameters in the Critic1 and Critic2 networks; ②Actor network update process: Considering the stability of the training process, the Actor network is updated with a delay, that is, the Critic network is updated multiple times before the Actor network is updated; ③ Update process of the target network: The target network is updated by soft update, which is usually achieved by introducing a learning rate to weighted average the old target network parameters and the new corresponding network parameters, and then assigning them to the target network; ④Training end condition: When all networks converge, the twin delayed deep deterministic policy gradient network training is completed; (3) Real-time model evaluation: The trained twin delayed deep deterministic policy gradient network is used for implementation evaluation during the operation of the hydrogen fuel cell hybrid system to complete the grid-connected control process; (4) Network optimization: Optimize network parameters.

10. The method for controlling a hydrogen fuel cell hybrid power system in grid connection according to claim 1, characterized in that: The fuel cell hybrid system is subject to physical constraints during actual operation, including maximum and minimum limits on the output power of the fuel cell and battery, maximum and minimum limits on the battery state of charge, the fuel cell output power change rate not exceeding 12.5% ​​of its rated power, and the matching of the power source output power and the load power. The relationship is expressed as: P fci is the output power of the i-th fuel cell; P fcimax and P fcimin are the maximum and minimum output power of the i-th fuel cell respectively; SOC is the state of charge of the battery; SOC max and SOC min are the maximum and minimum charge states of the battery respectively; P bat is the output power of the battery; P batmax and P batmin are the maximum and minimum output power of the battery respectively; ΔP fci is the output power increment of the i-th fuel cell; P fci,rated is the rated power of the i-th fuel cell; P load is the power required when the system is running; n is the number of fuel cells included in the fuel cell system.

Images