Fuel cell vehicle energy management method fusing TRANSFORMER and TD3 algorithms

Abstract

The invention provides a fuel cell automobile energy management method fusing a TRANSFORMER algorithm and a TD3 algorithm, and the method comprises the steps: comprehensively considering the energy flow and consumption characteristics in a vehicle system in modeling, and building a multi-objective optimization control frame including battery aging, fuel cell durability constraint, hydrogen consumption and cabin temperature comfort; in addition, future multi-step power demands are predicted by using a TRANSFORMER model and a TD3 algorithm, the consideration of future driving conditions is realized, an optimal action is generated according to the real-time state of the vehicle on the basis, prediction errors can be effectively reduced, the adaptability of an energy management strategy can be improved, and the energy management efficiency is improved. And the method has important practical significance on optimal balance of driving performance, system durability and hydrogen consumption of a real vehicle.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of fuel cell hybrid power system energy management, and particularly relates to a fuel cell vehicle energy management method combining a deep learning model and a TD3 algorithm. BACKGROUND

[0002] At present, the energy management technology for fuel cell hybrid systems mostly uses vehicle speed prediction as a substitute target for power demand. Since this method can only partially reflect the real-time state of the vehicle, it is not enough to capture more complex actual energy demand. In addition, the future power demand of the vehicle is not only affected by external factors such as vehicle speed and slope, but also closely related to internal components such as the transmission system and the motor. However, existing energy management strategies generally lack comprehensive consideration of these aspects. Therefore, how to provide a vehicle power prediction function with higher precision and an energy management method that optimizes the energy distribution performance of the vehicle according to the energy and structural characteristics of fuel cell hybrid electric vehicles is a technical problem that needs to be solved in the field. SUMMARY

[0003] Therefore, in view of the technical problems existing in the field, the application provides a fuel cell vehicle energy management method combining TRANSFORMER and TD3 algorithm, which specifically comprises the following steps:

[0004] Step one, obtaining vehicle state information, power battery state information and fuel cell state information of the fuel cell hybrid vehicle; wherein the vehicle state information includes vehicle speed v, acceleration acc, drive motor speed ω motor , drive motor torque T motor , drive motor efficiency η motor , cabin temperature T in , transmission system demand power P d,t and air conditioning system power consumption P ac ; the power battery state information includes power battery voltage and current, internal resistance and SOC; the fuel cell state information includes fuel cell output power P FC , efficiency η FC and power change rate ΔP FC ;

[0005] Step two, establishing an automobile longitudinal dynamics model according to the vehicle dynamics; sequentially establishing a fuel cell hydrogen consumption model based on voltage attenuation, a power battery energy consumption model, a power battery capacity attenuation model, a drive motor model and an air conditioning system power model according to the topology structure of the fuel cell hybrid power system;

[0006] Step 3: Build a final prediction model based on the TRANSFORMER network architecture to predict and output future acceleration and slope information based on the input vehicle speed, acceleration, and road slope. Combined with the models built in Step 2, this model calculates the future multi-step power demand.

[0007] Step 4: Utilize the future power demand and vehicle speed v for the TD3 algorithm t , acceleration acc t , slope θ t , power battery SOC, cabin temperature T in,t and fuel cell power P fc,t Define state variables and state space; use the fuel cell power change rate ΔP FC,t and air conditioning system capacity Q cool,t Define action variables and action space; comprehensively consider vehicle hydrogen consumption, battery life, battery SOC maintenance, fuel cell voltage maximum attenuation threshold, and cabin temperature comfort, and establish a corresponding reward function;

[0008] Step 5: Initialize environmental parameters and construct a training set using vehicle status information extracted from historical data or standard operating condition data. Use the training set to train the established final prediction model and TD3 algorithm. Apply the trained final prediction model online and use its output of power demand prediction results based on real-time vehicle status information to generate the corresponding optimal action variables.

[0009] Furthermore, the specific form of the vehicle longitudinal dynamics model established in step 2 is as follows:

[0010]

[0011] Among them, F t 、v t and acc t They represent the driving force, speed and acceleration of the vehicle respectively, m is the vehicle weight, g is the acceleration of gravity, C f and C d are the road resistance coefficient and the air resistance coefficient, θ t is the road slope, ρ air is the air density, A f is the frontal area of ​​the vehicle, δ t is the vehicle rotation mass conversion factor;

[0012] The specific form of the fuel cell hydrogen consumption model based on voltage decay and its constraints are as follows:

[0013]

[0014] ι3=0.2·P fc,max≤P fc ≤0.8·P fc,max

[0015] Among them, V d is the fuel cell voltage, t is time, κ1, κ2, κ3 correspond to the decay rates of different power intervals, κ4 is the decay rate when the load power changes, P fc is the fuel cell power, P fc,max Indicates its rated value;

[0016] The specific form of the power battery energy consumption model is as follows:

[0017]

[0018] Among them, SOC is the battery state of charge, I bat is the battery current, Q bat is the battery capacity, P bat is the battery power, V ocv and R bat are open circuit voltage and internal resistance respectively;

[0019] The specific form of the power battery capacity attenuation model is as follows:

[0020]

[0021] Where c is the power battery discharge rate, N EOL (c) and Ah EOL (c) the number of cycles and ampere-hour throughput at the end of life, respectively;

[0022] The specific form of the drive motor model is as follows:

[0023] η motor =f(ω motor ,T motor )

[0024]

[0025] Among them, r wh is the wheel radius, i g is the final drive ratio; the corresponding transmission system power requirement is calculated using the following formula:

[0026]

[0027] Among them, η DC / AC is the efficiency of the DC / AC converter;

[0028] The air conditioning system power model consists of the relationship between air conditioning power consumption and cooling capacity and the heat balance equation. Its specific form is as follows:

[0029]

[0030] Among them, P ac is the air conditioning power consumption, PLR is the partial load rate, Q g is the rated cooling capacity, T in and T out are the internal and external temperatures, respectively, COP stands for coefficient of performance, V air is the volume of air in the cabin, is the specific heat capacity of air, Q heat is the cabin heat load, Q cool For the cooling capacity.

[0031] Furthermore, the TRANSFORMER network architecture in step 3 is composed of a multi-head self-attention mechanism, a fully connected network, layer normalization, and residual connections, while the input embedding position code is used to extract temporal features; the final prediction model It consists of a TRANSFORMER layer and a fully connected layer, and its model parameters are calculated through the following loss function To update:

[0032]

[0033] in, represents the mathematical expectation, and They represent the driving state sequence of the historical state and the predicted state, L and H represent the step size of the historical state and the predicted state, respectively. are the model network parameters;

[0034] After obtaining the future acceleration and slope prediction results, the future power demand is calculated by combining the established models using the following formula:

[0035]

[0036] in, are fixed parameters of the physical model.

[0037] Furthermore, the TD3 algorithm in step 4 specifically consists of an actor network μ and two independent critic networks Q. The algorithm agent generates an action variable a based on the current state s and enters the next state after receiving a reward r. The critic network is used to output a corresponding comprehensive score Q(s,a) for the action-reward relationship. The actor network is used to maximize the Q(s,a) output by the critic network through training.

[0038] The actor network μ is updated by performing the following form of gradient descent on the objective function corresponding to the optimization goal:

[0039]

[0040] where J (θ μ ) is the objective function, Q1 represents the critic network 1, θ μ is the actor network parameter, s t is the state variable at time t;

[0041] The critic network is specifically based on the following loss function The update is realized as follows:

[0042]

[0043] where θ is the critic network parameter, δ is the time error, and γ is the discount factor; is the target action at time t+1 after adding noise, which is specifically represented as follows:

[0044]

[0045] where ξ represents Gaussian noise, which is added to improve the smoothness of the target action value estimation;

[0046] To improve the training stability, a target network is introduced in both the actor network and the critic network, and a delayed update method that lags behind the corresponding network parameter is adopted, which is specifically represented as follows:

[0047]

[0048] where τ is a soft update factor.

[0049] The fuel cell vehicle energy management method provided by the above-mentioned application of the present application fuses the TRANSFORMER and the TD3 algorithm, which comprehensively considers the energy flow and consumption characteristics of the vehicle system in modeling, establishes a multi-objective optimization control framework containing battery aging, fuel cell durability constraints, hydrogen consumption, and vehicle cabin temperature comfort, and uses the TRANSFORMER model and the TD3 algorithm to predict future multi-step power demand, realizes the consideration of future driving conditions, and generates the optimal action according to the real-time state of the vehicle, which can effectively reduce the prediction error and improve the adaptability of the energy management strategy, and has important practical significance for achieving the best balance between the driving performance, system durability and hydrogen consumption of the real vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is the flowchart of the method provided by the present application;

[0051] Figure 2 is the principle framework diagram of the TD3 algorithm in the method provided by the present application. DETAILED DESCRIPTION

[0052] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0053] The present invention provides a fuel cell vehicle energy management method that integrates TRANSFORMER and TD3 algorithms, such as Figure 1 As shown, the specific steps include:

[0054] Step 1: Obtain vehicle status information, power battery status information, and fuel cell status information of the fuel cell hybrid vehicle; wherein the vehicle status information includes: vehicle speed v, acceleration acc, drive motor speed ω motor , drive motor torque T motor , drive motor efficiency η motor , cabin temperature T in , transmission system required power P d,t And the air conditioning system power consumption P ac The power battery status information includes: power battery voltage and current, internal resistance and SOC; the fuel cell status information includes: fuel cell output power P FC , efficiency η FC and power change rate ΔP FC ;

[0055] Step 2: Establish a longitudinal dynamics model of the vehicle based on vehicle dynamics; establish a fuel cell hydrogen consumption model based on voltage decay, a power battery energy consumption model, a power battery capacity decay model, a drive motor model, and an air conditioning system power model in sequence for the fuel cell hybrid system topology;

[0056] Step 3: Build a final prediction model based on the TRANSFORMER network architecture to predict and output future acceleration and slope information based on the input vehicle speed, acceleration, and road slope. Combined with the models built in Step 2, this model calculates the future multi-step power demand.

[0057] Step 4: Utilize the future power demand and vehicle speed v for the TD3 algorithm t , acceleration acc t , slope θ t , power battery SOC, cabin temperature T in,t and fuel cell power P fc,t Define state variables and state space; use the fuel cell power change rate ΔP FC,t and air conditioning system capacity Q cool,tDefine action variables and action space; comprehensively consider vehicle hydrogen consumption, battery life, battery SOC maintenance, fuel cell voltage maximum attenuation threshold, and cabin temperature comfort, and establish a corresponding reward function;

[0058] Step 5: Initialize environmental parameters and construct a training set using vehicle status information extracted from historical data or standard operating condition data. Use the training set to train the established final prediction model and TD3 algorithm. Apply the trained final prediction model online and use its output of power demand prediction results based on real-time vehicle status information to generate the corresponding optimal action variables.

[0059] In a preferred embodiment of the present invention, the specific form of the vehicle longitudinal dynamics model established in step 2 is as follows:

[0060]

[0061] Among them, F t 、v t and acc t They represent the driving force, speed and acceleration of the vehicle respectively, m is the vehicle weight, g is the acceleration of gravity, C f and C d are the road resistance coefficient and the air resistance coefficient, θ t is the road slope, ρ air is the air density, A f is the frontal area of ​​the vehicle, δ t is the vehicle rotation mass conversion factor;

[0062] The specific form of the fuel cell hydrogen consumption model based on voltage decay and its constraints are as follows:

[0063]

[0064] ι1=P fc >0.8·P fc,max

[0065] ι2=P fc <0.2·P fc,max

[0066] ι3=0.2·P fc,max ≤P fc ≤0.8·P fc,max

[0067] Among them, V d is the fuel cell voltage, t is time, κ1, κ2, κ3 correspond to the decay rates of different power intervals, κ4 is the decay rate when the load power changes, P fc is the fuel cell power, P fc,maxIndicates its rated value;

[0068] The specific form of the power battery energy consumption model is as follows:

[0069]

[0070] Among them, SOC is the battery state of charge, I bat is the battery current, Q bat is the battery capacity, P bat is the battery power, V ocv and R bat are open circuit voltage and internal resistance respectively;

[0071] The specific form of the power battery capacity attenuation model is as follows:

[0072]

[0073] Where c is the power battery discharge rate, N EOL (c) and Ah EOL (c) the number of cycles and ampere-hour throughput at the end of life, respectively;

[0074] The specific form of the drive motor model is as follows:

[0075] η motor =f(ω motor ,T motor )

[0076]

[0077] Among them, r wh is the wheel radius, i g is the final drive ratio; the corresponding transmission system power requirement is calculated using the following formula:

[0078]

[0079] Among them, η DC / AC is the efficiency of the DC / AC converter;

[0080] The air conditioning system power model consists of the relationship between air conditioning power consumption and cooling capacity and the heat balance equation. Its specific form is as follows:

[0081]

[0082] Among them, P ac is the air conditioning power consumption, PLR is the partial load rate, Q g is the rated cooling capacity, T in and T out are the internal and external temperatures, respectively, COP stands for coefficient of performance, V air is the volume of air in the cabin, is the specific heat capacity of air, Q heat is the cabin heat load, Q cool For the cooling capacity.

[0083] In a preferred embodiment of the present invention, the TRANSFORMER network architecture in step 3 is composed of a multi-head self-attention mechanism, a fully connected network, layer normalization, and a residual connection, and the embedded position code is input to facilitate the extraction of temporal features; the final prediction model It consists of a TRANSFORMER layer and a fully connected layer, and its model parameters are calculated through the following loss function To update:

[0084]

[0085] in, represents the mathematical expectation, and They represent the driving state sequence of the historical state and the predicted state, L and H represent the step size of the historical state and the predicted state, respectively. are the model network parameters;

[0086] After obtaining the future acceleration and slope prediction results, the future power demand is calculated by combining the established models using the following formula:

[0087]

[0088] in, are fixed parameters of the physical model.

[0089] In a preferred embodiment of the present invention, Figure 2 As shown in Figure 2, the TD3 algorithm in step 4 consists of an actor network μ and two independent critic networks Q. The algorithm agent generates an action variable a based on the current state s and obtains the next state after the reward r. The critic network is used to output the corresponding comprehensive score Q(s, a) for the action-reward. The actor network is used to maximize the Q(s, a) output by the critic network through training.

[0090] The actor network μ is updated by performing the following form of gradient descent on the objective function corresponding to the optimization goal:

[0091]

[0092] Among them, J(θ μ ) is the objective function, Q1 represents the critic network 1, θ μ is the actor network parameter, s t is the state variable at time t;

[0093] The critic network is specifically based on the following loss function Implementation update:

[0094]

[0095] in, are the critic network parameters, δ is the time error, and γ is the discount factor; is the target action at time t+1 after adding noise, which is specifically expressed as follows:

[0096]

[0097] Where ξ represents Gaussian noise, which is added to improve the smoothness of the target action value estimation;

[0098] To improve training stability, the target network is introduced into both the actor network and the critic network, and a delayed update method is adopted that lags behind the corresponding network parameters, as shown below:

[0099]

[0100] Where τ is the soft update factor.

[0101] It should be understood that the size of the serial numbers of the steps in the embodiment of the present invention does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0102] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A fuel cell vehicle energy management method integrating TRANSFORMER and TD3 algorithms, characterized by: The specific steps include: Step 1: Obtain vehicle status information, power battery status information, and fuel cell status information of the fuel cell hybrid vehicle; wherein the vehicle status information includes: vehicle speed v, acceleration acc, drive motor speed ω motor , drive motor torque T motor , drive motor efficiency η motor , cabin temperature T in , transmission system required power P d,t And the air conditioning system power consumption P ac The power battery status information includes: power battery voltage and current, internal resistance and SOC; the fuel cell status information includes: fuel cell output power P FC , efficiency η FC and power change rate ΔP FC ; Step 2: Establish a longitudinal dynamics model of the vehicle based on vehicle dynamics; establish a fuel cell hydrogen consumption model based on voltage decay, a power battery energy consumption model, a power battery capacity decay model, a drive motor model, and an air conditioning system power model in sequence for the fuel cell hybrid system topology; Step 3: Build a final prediction model based on the TRANSFORMER network architecture to predict and output future acceleration and slope information based on the input vehicle speed, acceleration, and road slope. Combined with the models built in Step 2, this model calculates the future multi-step power demand. Step 4: Utilize the future power demand and vehicle speed v for the TD3 algorithm t , acceleration acc t , slope θ t , power battery SOC, cabin temperature T in,t and fuel cell power P fc,t Define state variables and state space; use the fuel cell power change rate ΔP FC,t and air conditioning system capacity Q cool,t Define action variables and action space; comprehensively consider vehicle hydrogen consumption, battery life, battery SOC maintenance, fuel cell voltage maximum attenuation threshold, and cabin temperature comfort, and establish a corresponding reward function; Step 5: Initialize environmental parameters and construct a training set using vehicle status information extracted from historical data or standard operating condition data. Use the training set to train the established final prediction model and TD3 algorithm. Apply the trained final prediction model online and use its output of power demand prediction results based on real-time vehicle status information to generate the corresponding optimal action variables.

2. The method according to claim 1, wherein: The specific form of the vehicle longitudinal dynamics model established in step 2 is as follows: Among them, F t 、v t and acc t They represent the driving force, speed and acceleration of the vehicle respectively, m is the vehicle weight, g is the acceleration due to gravity, C f and C d are the road resistance coefficient and the air resistance coefficient, θ t is the road slope, ρ air is the air density, A f is the frontal area of ​​the vehicle, δ t is the vehicle rotation mass conversion factor; The specific form of the fuel cell hydrogen consumption model based on voltage decay and its constraints are as follows: ι1=P fc >0.8·P fc,max ι2=P fc <0.2·P fc,max ι3=0.2·P fc,max ≤P fc ≤0.8·P fc,max Among them, V d is the fuel cell voltage, t is time, κ1, κ2, κ3 correspond to the decay rates of different power intervals, κ4 is the decay rate when the load power changes, P fc is the fuel cell power, P fc,max Indicates its rated value; The specific form of the power battery energy consumption model is as follows: Among them, SOC is the battery state of charge, I bat is the battery current, Q bat is the battery capacity, P bat is the battery power, V ocv and R bat are open circuit voltage and internal resistance respectively; The specific form of the power battery capacity attenuation model is as follows: Wherein, c is the power battery discharge rate, N EOL (c) and Ah EOL (c) the number of cycles and ampere-hour throughput at the end of life, respectively; The specific form of the drive motor model is as follows: or motor =f(ω motor ,T motor ) Among them, r wh is the wheel radius, i g is the final drive ratio; the corresponding transmission system power requirement is calculated using the following formula: Among them, η DC / AC is the efficiency of the DC / AC converter; The air conditioning system power model consists of the relationship between air conditioning power consumption and cooling capacity and the heat balance equation. Its specific form is as follows: Among them, P ac is the air conditioning power consumption, PLR is the partial load rate, Q g is the rated cooling capacity, T in and T out are the internal and external temperatures, respectively, COP stands for coefficient of performance, V air is the volume of air in the cabin, is the specific heat capacity of air, Q heat is the cabin heat load, Q cool For the cooling capacity.

3. The method according to claim 2, wherein: In step 3, the TRANSFORMER network architecture is composed of a multi-head self-attention mechanism, a fully connected network, layer normalization, and residual connections. At the same time, the input embedding position code is used to extract temporal features. Finally, Prediction Model It consists of a TRANSFORMER layer and a fully connected layer, and its model parameters are calculated through the following loss function To update: in, represents the mathematical expectation, and They represent the driving state sequence of the historical state and the predicted state, L and H represent the step size of the historical state and the predicted state, respectively. are the model network parameters; After obtaining the future acceleration and slope prediction results, the future power demand is calculated by combining the established models using the following formula: in, are fixed parameters of the physical model.

4. The method according to claim 3, wherein: The TD3 algorithm in step 4 consists of an actor network μ and two independent critic networks Q. The algorithm agent generates an action variable a based on the current state s and obtains a reward r before entering the next state. The critic network is used to output a corresponding comprehensive score Q(s, a) for the action-reward relationship. The actor network is used to maximize the Q(s, a) output by the critic network through training. The actor network μ is updated by performing the following form of gradient descent on the objective function corresponding to the optimization goal: Among them, J(θ μ ) is the objective function, Q1 represents the critic network 1, θ μ is the actor network parameter, s t is the state variable at time t; The critic network is specifically based on the following loss function Implementation update: in, are the critic network parameters, δ is the time error, and γ is the discount factor; is the target action at time t+1 after adding noise, which is specifically expressed as follows: Where ξ represents Gaussian noise, which is added to improve the smoothness of the target action value estimation; To improve training stability, the target network is introduced into both the actor network and the critic network, and a delayed update method is adopted that lags behind the corresponding network parameters, as shown below: i μ′ ←tth μ +(1-τ)θ μ′ Where τ is the soft update factor.

Images