DQN energy management strategy optimization method considering fuel cell output power reduction

Abstract

The invention relates to a DQN energy management strategy optimization method considering output power reduction of a fuel cell. The method is used for solving the problem that an existing fuel cell cannot meet power requirements due to aging performance degradation. The method comprises the steps that the life cycle of a fuel cell of a hybrid electric vehicle end is divided into a plurality of decline stages, and when the update operation duration of different decline stages is exceeded, the hybrid electric vehicle end updates an output power-current curve and sends the curve to a cloud end, and meanwhile the cloud end requests to update a DQN energy management strategy model; when receiving the updated output power-current curve, the cloud end updates training data by using the updated output power-current curve so as to train and optimize a DQN energy management strategy model, and sends the optimized DQN energy management strategy model to the hybrid electric vehicle end; and the hybrid electric vehicle end receives the optimized DQN energy management strategy model and updates the optimized DQN energy management strategy model to a controller to determine a power distribution scheme. According to the scheme, the energy utilization rate during fuel cell performance degradation can be improved, and the durability of the fuel cell is improved.

Description

Technical Field

[0001] The present invention relates to a fuel cell vehicle power system energy management technology, and in particular to a DQN energy management strategy optimization method taking into account the decrease in fuel cell output power. Background Art

[0002] As human society continues to develop, addressing the critical challenge of sustainably meeting the growing energy demand has become a top priority for countries around the world. Proton exchange membrane fuel cells are valued for their pollution-free operation, high energy density, and energy conversion efficiency, and many vehicles have been developed that use fuel cells as the main power source. However, the development of fuel cell hybrid vehicles is susceptible to many obstacles, including the impact of fuel cell degradation. Although fuel cell degradation is inevitable, energy management strategies that cannot adapt to fuel cell degradation will reduce the fuel economy of fuel cell hybrid vehicles. Therefore, it is crucial to develop new energy management control strategies based on the characteristics of fuel cell degradation to optimize the fuel economy of degraded fuel cell hybrid vehicles. Summary of the invention

[0003] The purpose of the present invention is to propose a DQN energy management strategy optimization method that takes into account the decrease in fuel cell output power, thereby improving the energy utilization rate of the fuel cell and improving the durability of the fuel cell. The specific technical solution is as follows.

[0004] A DQN energy management strategy optimization method considering the decrease of fuel cell output power, the method comprising the following steps: dividing the life cycle of a fuel cell at a hybrid vehicle end into a number of decay stages, when exceeding the update operation time set for different decay stages, the hybrid vehicle end updates the output power-current curve and sends the updated output power-current curve to the cloud, and requests the cloud to update the DQN energy management strategy model; when the cloud receives the updated output power-current curve and the request, the cloud updates the training data using the updated output power-current curve to train and optimize the DQN energy management strategy model, and sends the optimized DQN energy management strategy model to the hybrid vehicle end; the hybrid vehicle end receives the optimized DQN energy management strategy model and updates it to a controller to determine a power allocation plan; wherein the DQN energy management strategy model is established based on deep reinforcement learning, and a Q-learning multi-objective particle swarm optimization algorithm is used for training and optimization.

[0005] In one implementation of the above technical solution, the decay stages are divided according to the estimated value of the fuel cell state of health SOH, including an initial period of no obvious decay, a slight decay stage, a moderate decay stage, a significant decay stage, and a severe decay stage.

[0006] In one implementation of the above technical solution, the hybrid vehicle end updates the output power-current curve, and the steps include: obtaining the similarity of the polarization curves before and after the fuel cell decay by polynomial fitting the decay state of the fuel cell from 0% to 10% and the current drop value corresponding to the decay state under different current intensities, and obtaining the estimated value of the health state SOH of the fuel cell at time t; calculating the polarization curve value based on the similarity of the polarization curves before and after the decay and the estimated value of the health state SOH of the fuel cell at time t, so as to correct the output power-current curve:

[0007]

[0008] U deg =U initial -ΔU

[0009] P fc =NU deg I fc η elec

[0010] Where: ΔU ultimate is the maximum voltage drop value when the fuel cell decays by 10%, ΔU0 is the change of open circuit voltage, I rated is the rated current, ΔU is the voltage drop value under a certain current state, U deg is the output voltage of the fuel cell monomer under a certain current state, U deg is the corrected fuel cell system output power, N is the number of fuel cell units, η elec is the electricity conversion efficiency of the fuel cell system, I fc is the fuel cell output current, P fc Corrected actual fuel cell output power.

[0011] In one embodiment of the above technical solution, the fuel cell state of health SOH estimate is estimated using a preset LSTM model, which uses the output voltage, output current, inlet air pressure, inlet air temperature, inlet hydrogen pressure and inlet hydrogen temperature of the current fuel cell system to predict the estimated value of the current state of health SOH.

[0012] In one implementation of the above technical solution, the DQN energy management strategy model modifies the actual output power P of the fuel cell fc 、Vehicle power requirement P load and the power battery SOC as the state quantity S = {SOC, P load , P fc}, taking the fuel cell power change as the action variable, and setting the variable load discount factor of the fuel cell decay to make the action variable satisfy the decay law, and setting the reward function as the function of the instantaneous hydrogen consumption cost, fuel cell decay cost and power battery equivalent hydrogen consumption cost: ω1, ω2, ω3 are the weighted coefficients of the three objectives, J fc is the fuel cell hydrogen consumption, J bat is the equivalent hydrogen consumption of the power battery, Df c Fuel cell degradation costs, is the price of hydrogen per kilogram, ξ fc is the price per kilowatt (kw) of fuel cell.

[0013] In one implementation of the above technical solution, in the DQN energy management strategy model, the fitness value of the optimization algorithm is set to hydrogen consumption, fuel cell durability and capacity decay of the power battery.

[0014] The present invention also provides a hybrid electric vehicle which adopts any of the above methods to distribute power.

[0015] The present invention also provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute any one of the above methods.

[0016] Finally, the present invention proposes a DQN energy management strategy optimization system that takes into account the decline in fuel cell output power. The system includes a request module, a training optimization module, and a model update module; the request module is configured at the hybrid vehicle end, which divides the life cycle of the fuel cell at the hybrid vehicle end into several decay stages, and sets the updated operating time according to the decay stage. When the cumulative operating time of the fuel cell after the update exceeds the updated operating time, the hybrid vehicle end updates the output power-current curve and sends the updated output power-current curve to the cloud, and requests the cloud to update the DQN energy management strategy model; the training optimization module is configured at the cloud, and when the cloud receives the updated output power-current curve and the request, it updates the training data using the updated output power-current curve to train and optimize the DQN energy management strategy model, and sends the optimized DQN energy management strategy model to the hybrid vehicle end; the model update module is configured at the hybrid vehicle end, and receives the optimized DQN energy management strategy model and updates it to the controller to determine the power allocation plan.

[0017] The beneficial technical effect of the present invention is that when the performance of the fuel cell declines and the output power decreases, the power distribution among multiple power systems is efficiently realized. It can also be adaptively adjusted according to the health status of the fuel cell to improve energy utilization and improve the durability of the fuel cell. When the fuel cell using this energy management strategy optimization method is applied to a fuel cell hybrid vehicle, the economy of the whole vehicle can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 , one A schematic diagram of a method flow chart in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] After the fuel cell ages, the performance degradation will cause the output power of the aged fuel cell system to decrease at the same current, which cannot meet the power demand. If the power-current curve is not updated, the controller always refers to the power-current curve at the initial moment to distribute power, which will not meet the power demand of the whole vehicle, and cannot effectively slow down the decline trend of the fuel cell and aggravate the capacity loss of the power battery, thus reducing the performance of the whole vehicle.

[0021] In this case, the fuel cell decline stage is set, and the cumulative running time of the fuel cell after the update is compared. i and update runtime T degradation , judge and update and optimize the online fuel cell output power-current curve, control the output power of the fuel cell according to the optimized current, so that it can meet the target reference power, improve the durability of the fuel cell, and thus improve the economy of using fuel cell vehicles.

[0022] The following is a clear and complete description of how to implement the technical solution of this case. Obviously, the described implementation method is only a part of the implementation method of this case, not all of the implementation methods. Based on the implementation method in this case, all other implementation methods obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] (I) Health status SOH value assessment

[0024] Build an LSTM model as a health status SOH value evaluation model.

[0025] The LSTM model includes a forget gate, an input gate, and an output gate, in which the control of information flow is achieved through the structure of the gate. t =σ(W fC C t +W fh h t-1 +b f ), W and b are weights and biases respectively, and their superscripts indicate the corresponding relationship, such as W fC is the forget gate corresponding to C t The weight, W fh is the hidden layer h t-1 The weight of b f is the forget gate bias, σ(·) is the sigmoid activation function. Input gate i t =σ(W iC C t +w ih h t-1 +b i ), input node S t =g t ×i t +S t-1 ×f t , b i , b g are the input gate and input node bias respectively. Output gate o t , o t =σ(W oC C t +W oh h t-1 +b o ), b o is the output gate bias, and the new hidden state t represents the current time.

[0026] In this case, the output voltage, output current, intake air pressure, intake air temperature, intake hydrogen pressure and intake hydrogen temperature of the fuel cell system are used as inputs of an LSTM model, and the health status SOH value is used as its corresponding output.

[0027] The working data of the fuel cell is continuously collected, and the data is grouped by time to construct an input-output group. For the output group, the health status SOH value is evaluated according to its output voltage in the following way: the fuel cell is in a healthy state at the initial moment, and the SOH is 0. When the voltage drops by 10%, the working life of the fuel cell is considered to have ended, and the SOH value is 1 at this time. By establishing a relationship between the voltage drop ratio and the SOH value, the estimated value of SOH is output. SOH=1-D, U rated，init is the initial output voltage, U rated,degratedis the output voltage at the current moment.

[0028] The input data and its corresponding SOH value are taken as sample pairs, and the sample pairs are taken as training data.

[0029] The trained LTSM model is used to predict the current state of health SOH value based on the output voltage, output current, inlet air pressure, inlet air temperature, inlet hydrogen pressure and inlet hydrogen temperature of the current fuel cell system.

[0030] (II) Output power-current curve update

[0031] The attenuation of output performance when a fuel cell deteriorates will be reflected in the change of the output power-current curve. The change of the power-current curve will affect the fuel cell reference current calculated when distributing the power of the fuel cell vehicle power system.

[0032] After the fuel cell ages, the performance decay will cause the output power of the aged fuel cell to decrease under the same current, which cannot meet the power demand. At the same time, when the same power request arrives, the calculated fuel cell reference current is exactly the same as the reference current at the initial moment. The controller controls the output power of the fuel cell according to the initial reference current, which will make it lower than the target reference power.

[0033] When distributing energy, computing resources and update frequency need to be considered. Frequent updates of energy management strategies occupy computing resources and affect the normal driving of the vehicle. At the same time, fuel cell degradation is a slow process, and the effectiveness of power allocation scheme updates and adaptability to fuel cell degradation need to be comprehensively considered.

[0034] Therefore, by setting different decay stages and updating the running time T degradation By comparing the cumulative running time T of the updated fuel cell at different decay stages i The update duration T is set degradation , thereby determining whether to update the output power-current curve and achieving a balance between computing resources and the effectiveness of updating the output power-current curve.

[0035] In one embodiment, the degradation stage is divided into the following according to the SOH value of the fuel cell: an initial stage with no obvious degradation, a slight degradation stage, a moderate degradation stage, a significant degradation stage, and a severe degradation stage. degradation The setting of T is set according to the fuel cell decline stage and actual situation. As the decline stage intensifies, T is gradually shortened. degradation If it is determined that the update running time T set for the decay stage has not been exceeded, degradation, continue to use the output power-current curve of the current decay stage, obtain the reference current, and control the output power of the fuel cell to meet the target reference power. If it is judged that the update operation time T set in the decay stage is exceeded degradation , the current output power-current curve is updated and corrected to obtain a new reference current to meet the target reference power, and the updated cumulative running time T is calculated after each correction. i Set to zero to provide a reference for the next update judgment.

[0036] The polarization curve value can be calculated based on the similarity of the polarization curves before and after the decay and the estimated value of SOH at time t, and the output power-current curve can be corrected. The formula is as follows:

[0037]

[0038] U deg =U initial -ΔU

[0039] P fc =NU deg I fc η elec

[0040] Where: ΔU ultimate is the maximum voltage drop value when the fuel cell decays by 10%, ΔU0 is the change of open circuit voltage, I rated is the rated current, ΔU is the voltage drop value under a certain current state, U deg is the output voltage of the fuel cell monomer under a certain current state, U deg is the corrected fuel cell system output power, N is the number of fuel cell units, η elec is the electricity conversion efficiency of the fuel cell system, I fc is the fuel cell output current, P fc Corrected actual fuel cell output power.

[0041] Since the polarization curve shows a linear relationship before and after degradation, that is, the voltage value corresponding to different current intensities, the voltage drop value of the polarization curve with different decay degrees under the same current intensity corresponds to the SOH value of the fuel cell. Based on this linear relationship, the voltage value at each current density is obtained through the previously estimated SOH value, and then the polarization curve after decay can be updated. In one embodiment, the similarity of the polarization curves before and after decay is obtained by polynomial fitting of the decay state of the fuel cell from 0% to 10% under different current intensities and the current drop value corresponding to the decay state.

[0042] 3. Energy distribution

[0043] Since the output of the energy management strategy is a power change value, it will be converted into a current demand instruction in the control to obtain the desired fuel cell output power. After decay, the output power obtained by the same demand current instruction will decrease, and the energy consumption of the power system, the actual output power of the fuel cell, and the actual output power of the power battery will all change.

[0044] Therefore, combined with the vehicle operating conditions and the corrected output power-current curve, the vehicle side needs to use the optimized DQN energy management strategy model to determine the power allocation plan. However, due to the limited computing resources on the vehicle side, the optimized DQN energy management strategy model deployed on the vehicle side is a migration model, and the model is trained on the cloud before migration.

[0045] Specifically, the hybrid vehicle updates the output power-current curve, sends the updated output power-current curve to the cloud, and requests to update the DQN energy management strategy model. When the cloud receives the updated output power-current curve, it uses it to update the training data to train and optimize the DQN energy management strategy model, and sends the optimized DQN energy management strategy model to the hybrid vehicle. The operating conditions and vehicle power system model are pre-stored in the cloud. Exemplarily, the operating condition is the CLTC condition. The cloud updates the vehicle power system model using the received output power-current curve.

[0046] The DQN energy management strategy model is pre-built in the cloud based on deep reinforcement learning. The QMOPSO algorithm (Q-learning multi-objective particle swarm optimization) is used for training optimization in combination with the vehicle operating conditions. The optimization parameters include learning rate, discount factor, exploration rate, experience replay buffer size, target network update frequency, and number of neural network layers. The fitness value of the optimization algorithm is set to hydrogen consumption, fuel cell durability, and power battery capacity attenuation. The actual output power P of the fuel cell is corrected in this algorithm. fc 、Vehicle power requirement P load and the power battery SOC as the state quantity S: S = {SOC, P load , P fc}. Since the dynamic response of the fuel cell is slow, excessive load variation will accelerate the aging rate of the fuel cell. For example, the reference is a fuel cell system with a rated power of 36kw for a passenger car, and the load variation does not exceed 5kw. Multiple actions are set to determine the output power of the fuel cell. The action variable A is set for the fuel cell power variation: A = ξ{-5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5}, ξ is the load variation discount factor for fuel cell degradation. When determining the action, select the power variation value that maximizes the reward function. The reward function is a function that takes into account the instantaneous hydrogen consumption cost, the fuel cell degradation cost, and the equivalent hydrogen consumption cost of the power battery. An example is: ω1, ω2, ω3 are the weighted coefficients of the three objectives, J fc is the fuel cell hydrogen consumption, J bat is the equivalent hydrogen consumption of the power battery, D fc For fuel cell degradation costs, is the price of hydrogen per kilogram, ξ fc is the price per kilowatt (kw) of fuel cell.

[0047] After training, the vehicle receives the optimized DQN energy management strategy model and updates it to the controller to determine the power allocation plan.

[0048] From the above, it can be seen that the optimization of the DQN energy management strategy model uses the QMOPSO algorithm to adjust the parameters of the DQN algorithm to improve the economy of this energy management strategy, including multiple objectives such as hydrogen consumption, fuel cell durability and capacity attenuation of power batteries. Although the optimization of the DQN energy management strategy model can also adopt swarm intelligence optimization algorithms (such as PSO algorithm, GA algorithm, etc.), most of the multi-objective problems are optimized by weighted sum methods, and multiple optimization objectives cannot be effectively balanced. Therefore, this case uses the QMOPSO algorithm to optimize the DQN energy management strategy, and through multi-objective optimization, more precise control of the power distribution of the power system is achieved and the economy of fuel cell vehicles is improved.

[0049] For the implementation process of the above method, please see Figure 1A method flow chart, the steps include: dividing the life cycle of a fuel cell at a hybrid vehicle end into a number of decay stages, when exceeding the update operation time set for different decay stages, the hybrid vehicle end updates the output power-current curve and sends the updated output power-current curve to the cloud, and requests the cloud to update the DQN energy management strategy model; when the cloud receives the updated output power-current curve and the request, the cloud updates the training data using the updated output power-current curve to train and optimize the DQN energy management strategy model, and sends the optimized DQN energy management strategy model to the hybrid vehicle end; the hybrid vehicle end receives the optimized DQN energy management strategy model and updates it to the controller to determine the power allocation plan.

[0050] Through the description of the above embodiments, the technicians in the relevant field can clearly understand that according to the disclosed method, the corresponding system can be realized. Exemplarily, an energy management strategy optimization system considering the decline of fuel cell output power includes a request module, a training optimization module, and a model update module. The request module is configured at the hybrid vehicle end, which divides the life cycle of the fuel cell at the hybrid vehicle end into several decay stages, and sets the updated operation time according to the decay stage. When the cumulative operation time of the fuel cell after the update exceeds the updated operation time, the hybrid vehicle end updates the output power-current curve and sends the updated output power-current curve to the cloud, and requests the cloud to update the DQN energy management strategy model. The training optimization module is configured at the cloud. When the cloud receives the updated output power-current curve and the request, it updates the training data using the updated output power-current curve to train and optimize the DQN energy management strategy model, and sends the optimized DQN energy management strategy model to the hybrid vehicle end. The model update module is configured at the hybrid vehicle end, which receives the optimized DQN energy management strategy model and updates it to the controller to determine the power allocation plan.

[0051] Through the description of the above embodiments, those skilled in the art can clearly understand that the method disclosed herein can be applied to hybrid electric vehicles.

[0052] Through the description of the above implementation modes, those skilled in the art can clearly understand that the method and system of the present disclosure can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits, etc. However, for the present disclosure, software program implementation is a better implementation mode in most cases.

[0053] Although the embodiments of the present invention are described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments and application fields, and the above specific embodiments are only illustrative and instructive, rather than restrictive. A person of ordinary skill in the art can also make many forms under the guidance of this specification and without departing from the scope of protection of the claims of the present invention, all of which belong to the protection of the present invention.

Claims

1. A DQN energy management strategy optimization method considering the decrease in fuel cell output power, characterized in that: The method comprises the following steps: The life cycle of the fuel cell of the hybrid vehicle is divided into several decay stages. When the set update operation time of different decay stages is exceeded, the hybrid vehicle updates the output power-current curve and sends the updated output power-current curve to the cloud, and requests the cloud to update the DQN energy management strategy model. When the cloud receives the updated output power-current curve and the request, it updates the training data using the updated output power-current curve to train and optimize the DQN energy management strategy model, and sends the optimized DQN energy management strategy model to the hybrid vehicle end; The hybrid vehicle receives the optimized DQN energy management strategy model and updates it to the controller to determine the power allocation plan; Among them, the DQN energy management strategy model is established based on deep reinforcement learning, and the Q-learning multi-objective particle swarm optimization algorithm is used for training optimization.

2. The method according to claim 1, characterized in that The degradation stages are divided according to the estimated value of the fuel cell state of health SOH, including an initial period without obvious degradation, a slight degradation stage, a moderate degradation stage, a significant degradation stage, and a severe degradation stage.

3. The method according to claim 1, characterized in that The hybrid vehicle end updates the output power-current curve, and the steps include: By fitting the fuel cell decay state from 0% to 10% under different current intensities and the current drop value corresponding to the decay state with a polynomial, the similarity of the polarization curves before and after the decay of the fuel cell is obtained, and at the same time, the estimated value of the fuel cell health state SOH at time t is obtained; Based on the similarity of the polarization curves before and after the decay and the estimated value of the fuel cell health state SOH at time t, the polarization curve value is calculated to correct the output power-current curve: U deg =U initial -ΔU P fc =NO deg and fc η elec Where: ΔU ultimate is the maximum voltage drop value when the fuel cell decays by 10%, ΔU0 is the change of open circuit voltage, I rated is the rated current, ΔU is the voltage drop value under a certain current state, U deg For The output voltage of the fuel cell monomer under a certain current state, U deg is the corrected fuel cell system output power, N is the number of fuel cell units, η elec is the electricity conversion efficiency of the fuel cell system, I fc is the fuel cell output current, P fc Corrected actual fuel cell output power.

4. The method according to claim 3, characterized in that The estimated value of the fuel cell health state SOH is estimated using a preset LSTM model, which predicts the estimated value of the current health state SOH based on the output voltage, output current, intake air pressure, intake air temperature, intake hydrogen pressure and intake hydrogen temperature of the current fuel cell system.

5. The method according to claim 1, characterized in that: The DQN energy management strategy model modifies the actual output power of the fuel cell Pfc and the required power of the vehicle P load and the power battery SOC as the state quantity S = {SOC, P load , P fc }, taking the fuel cell power change as the action variable, and setting the variable load discount factor of the fuel cell decay to make the action variable satisfy the decay law, and setting the reward function as the function of the instantaneous hydrogen consumption cost, fuel cell decay cost and power battery equivalent hydrogen consumption cost: ω1, ω2, ω3 are the weighted coefficients of the three objectives, J fc is the fuel cell hydrogen consumption, J bat is the equivalent hydrogen consumption of the power battery, D fc Fuel cell degradation costs, is the price of hydrogen per kilogram, ξ fc is the price per kW of fuel cell.

6. The method according to claim 1, characterized in that: In the DQN energy management strategy model, the fitness values ​​of the optimization algorithm are set as hydrogen consumption, fuel cell durability, and capacity decay of the power battery.

7. A hybrid electric vehicle, characterized in that: Power distribution is performed using any one of the methods of claims 1 to 7.

8. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 7.

9. A DQN energy management strategy optimization system considering the decrease in fuel cell output power, characterized in that: The system includes a request module, a training optimization module, and a model update module; The request module is configured at the hybrid vehicle end, and divides the life cycle of the fuel cell at the hybrid vehicle end into several decay stages. When the set update operation time of different decay stages is exceeded, the hybrid vehicle end updates the output power-current curve and sends the updated output power-current curve to the cloud, and requests the cloud to update the DQN energy management strategy model at the same time; The training optimization module is configured in the cloud. When the cloud receives the updated output power-current curve and the request, the training data is updated using the updated output power-current curve to train and optimize the DQN energy management strategy model, and the optimized DQN energy management strategy model is sent to the hybrid vehicle end. The model updating module is configured at the hybrid vehicle end, and receives the optimized DQN energy management strategy model and updates it to the controller to determine the power allocation scheme; Among them, the DQN energy management strategy model is established based on deep reinforcement learning, and the Q-learning multi-objective particle swarm optimization algorithm is used for training optimization.

Images