Hydrogen fuel cell bus energy-saving driving control method and system

Abstract

The application provides a hydrogen fuel cell bus energy-saving driving control method and system, and the method comprises the following steps: a hierarchical double-layer deep reinforcement learning architecture is designed; the upper-layer deep reinforcement learning controls the longitudinal speed of the hydrogen fuel cell bus by acquiring the speed, acceleration and ramp road condition and other traffic environment information of the hydrogen fuel cell bus and the front vehicle, so as to ensure the safe and stable driving of the hydrogen fuel cell bus; the lower-layer deep reinforcement learning optimizes the energy management of the hydrogen fuel cell by the speed output by the upper layer; meanwhile, a rapid road comprising three ramps with different traffic flows is designed to perform offline training on the deep reinforcement learning algorithm, so as to obtain the optimized energy-saving driving control. The application realizes the longitudinal speed control and energy management optimization of the hydrogen fuel cell bus by using the double-layer deep reinforcement learning method, so that the hydrogen fuel cell bus can realize energy-saving driving.

Description

Technical Field

[0001] This invention relates to the field of energy management and driving control technology for hydrogen fuel cell buses, and in particular to an energy-saving driving control method for hydrogen fuel cell buses based on two-layer deep reinforcement learning. Background Technology

[0002] The transportation industry is developing rapidly, and its energy consumption accounts for 20% of global energy consumption. With continuous technological and economic development, environmental problems caused by carbon emissions are increasingly attracting attention. As a crucial component of industry, the automotive sector plays a vital role in developing and applying new energy vehicles to reduce carbon emissions. Hydrogen fuel cell vehicles are recognized and considered one of the most promising new energy vehicle options due to their advantages such as zero pollution, zero emissions, and long range. As a key technology for fuel cell buses, energy management strategies (EMS) aim to improve vehicle efficiency and energy economy by rationally allocating power between the fuel cell and the power battery. Currently, most common energy management strategies are rule-based or optimization-based, which suffer from drawbacks such as high computational costs and reliance on empirical samples.

[0003] Deep reinforcement learning (DRL) is a method that combines deep learning with reinforcement learning, integrating the perceptual capabilities of deep learning with the decision-making abilities of reinforcement learning. With the development of DRL algorithms, DRL has begun to be applied to energy management strategies (EMS) for hydrogen fuel cell buses. However, most current EMS research for hydrogen fuel cell buses focuses on energy management systems with fixed operating conditions and simple structures, failing to comprehensively and widely apply energy management methods to actual vehicle operation. With the development of vehicle-to-everything (V2X) technology, further applying deep reinforcement learning algorithms to real-time energy management strategies for hydrogen fuel cell buses in complex traffic environments has significant practical application value. Summary of the Invention

[0004] The technical problem to be solved by this invention is to propose an energy-saving driving control method and system for hydrogen fuel cell buses based on two-layer deep reinforcement learning.

[0005] To solve the above-mentioned technical problems, the present invention proposes the following technical solution:

[0006] A method for energy-saving driving control of a hydrogen fuel cell bus includes the following steps:

[0007] S1. Construct a power system model for a hydrogen fuel cell bus and propose the maximum and minimum following distances;

[0008] S2. Utilize deep reinforcement learning methods to construct an energy-saving driving optimization method for hydrogen fuel cell buses based on a two-layer deep reinforcement learning algorithm;

[0009] S3. Establish a two-layer DDPG network, including constructing a first DDPG network to coordinate with the maximum and minimum following distance for longitudinal speed control of the hydrogen fuel cell bus, and constructing a second DDPG network for energy management optimization control based on the power system model of the hydrogen fuel cell bus.

[0010] S4. Two-layer network state sharing: Construct a two-layer deep reinforcement learning network architecture. Input the hydrogen fuel cell bus speed output by the first DDPG network into the second DDPG network. The second DDPG network performs energy management of the hydrogen fuel cell bus and optimizes the energy-saving driving of the hydrogen fuel cell bus.

[0011] Furthermore, in step S1, a power system model for a hydrogen fuel cell bus is constructed, including a power battery, a fuel cell, a hydrogen storage tank, a DC / DC converter, a drive motor, and drive wheels. The fuel cell and the hydrogen storage tank are connected using hydrogen gas, the fuel cell and the power battery are connected to the DC / DC converter, the DC / DC converter is connected to the drive motor using electricity, and the drive motor is mechanically connected to the drive wheels.

[0012] Furthermore, the maximum and minimum following distances for the hydrogen fuel cell bus in step S1 are as follows:

[0013] L min = 2 + 0.5v + 0.0625v 2

[0014] L max =10 + v + 0.0825 v 2

[0015] Among them, L min L max These represent the maximum and minimum following distance limits for hydrogen fuel cell buses, respectively, and v represents the speed of the hydrogen fuel cell bus.

[0016] Furthermore, step S2 includes optimization of longitudinal speed control and energy management for the hydrogen fuel cell bus, specifically including the following sub-steps:

[0017] S201: Define the state, actions, reward function, optimal action-value function, and optimal control strategy of a hydrogen fuel cell bus in a deep reinforcement learning algorithm;

[0018] S202: The deep reinforcement learning agent receives the current state of the hydrogen fuel cell bus and executes the actions defined in S201 to change the speed of the hydrogen fuel cell bus and the output power of the fuel cell.

[0019] S203: The environment changes due to the action, the agent obtains the new state of the hydrogen fuel cell bus, and returns the new state and the reward brought by the action to the deep reinforcement learning agent;

[0020] S204: In the new state of the hydrogen fuel cell bus, the agent will continue to execute the actions defined in S201. In this way, the deep reinforcement learning agent will continuously interact with the environment until the optimal action-value function, i.e., the multi-objective Q-value vector, and the optimal control strategy are obtained.

[0021] Furthermore, step S201 specifically includes:

[0022] (1) The state of the hydrogen fuel cell bus in the deep reinforcement learning algorithm includes: the current speed and acceleration of the hydrogen fuel cell bus, the current speed and acceleration of the vehicle in front, the current distance between the hydrogen fuel cell bus and the vehicle in front, the current equivalent hydrogen consumption of the hydrogen fuel cell bus, the current output power of the hydrogen fuel cell, and the current remaining charge of the power battery.

[0023] (2) Actions in deep reinforcement learning algorithms include: action A1 and action A2, defined as follows:

[0024] A1={P fc |P fc ∈(0,60)}

[0025] A2 = {a | a ∈ [-1.5, 0.7]}

[0026] Among them, action A1 is the speed change of the hydrogen fuel cell bus, i.e., the acceleration per second; action A2 is the output power of the hydrogen fuel cell; P fc denoted as the output power value of the hydrogen fuel cell, and 'a' as the speed change value of the hydrogen fuel cell bus.

[0027] (3) The reward function, which includes the equivalent hydrogen consumption cost, the range of power fluctuation difference of hydrogen fuel cell, the safe following cost of hydrogen fuel cell buses, and the comfortable following cost, is as follows:

[0028] R(s,a)=R1(s,a)+R2(s,a)+R3(s,a)+R4(s,a)

[0029] R1(s, a) = C h

[0030] R2(s, a) = C p

[0031] R3(s, a) = C s

[0032] R4(s, a) = C c

[0033] Where R(s, a) is the reward function vector for choosing action a in state s, R1(s, a) is the reward function reward1; R2(s, a) is the reward function reward2; R3(s, a) is the reward function reward3; R4(s, a) is the reward function reward4; C h For equivalent hydrogen consumption cost; C p The range of power fluctuation differences in hydrogen fuel cells; C s Cost of following the vehicle for safety of hydrogen fuel cell buses; C c To reduce the cost of comfort-following in hydrogen fuel cell buses;

[0034] (4) The specific formula for calculating the optimal action-value function is as follows:

[0035] Q * (s, a) = Q π (s, a) = maxE[R] t+1 +λQ * (s t+1 a t+1 )|s t a t ]

[0036] Among them, Q π (s, a) is the action-value function for choosing action a in policy π state s; t a t Let s be the state and action at time t; t+1 a t+1 ,R t+1 Let be the state, action, and reward function at time t+1; λ∈[0,1] is the discount factor.

[0037] Furthermore, step S3 mainly includes the following sub-steps:

[0038] S301: Establish a DDPG network, including a first DDPG network and a second DDPG network;

[0039] S302: Using the first DDPG network, optimize the longitudinal speed of the hydrogen fuel cell bus and execute the control action A1 to change the speed of the hydrogen fuel cell bus at the next moment; using the second DDPG network, optimize the energy management of the hydrogen fuel cell and execute the control action A2 to change the output power of the hydrogen fuel cell bus fuel cell at the next moment.

[0040] Further, in step S4, a two-layer deep reinforcement learning network architecture is constructed, including: sharing the energy-saving driving optimization state of the hydrogen fuel cell bus based on the two-layer deep reinforcement learning algorithm constructed in step S2 in the two-layer network constructed in step S3 to accelerate the training speed of the agent; after sharing, the states of the first DDPG network and the second DDPG network both include: the current speed of the hydrogen fuel cell bus, the current acceleration of the hydrogen fuel cell, the current speed of the vehicle in front, the current acceleration of the vehicle in front, the current distance between the hydrogen fuel cell bus and the vehicle in front, the current equivalent hydrogen consumption of the hydrogen fuel cell bus, the current output power of the hydrogen fuel cell, and the current remaining charge of the power battery.

[0041] Furthermore, step S4 includes the following sub-steps:

[0042] S401: Offline training; The model is trained using a two-layer deep reinforcement learning algorithm to learn the control policy, i.e., the mapping relationship between the input state and the action parameters.

[0043] S402: Read out the parameters of each trained conditional network and download the control strategy to the vehicle controller (VCU);

[0044] S403: Online learning; obtain information about the current state of the hydrogen fuel cell bus, and apply it to the trained conditional network to optimize the energy-saving driving of the hydrogen fuel cell bus through online real-time adjustment.

[0045] Furthermore, in step S401, offline training includes establishing a simulation environment for an urban expressway containing three sets of ramps with different characteristics for offline training; and training the model using a two-layer deep reinforcement learning algorithm to learn the control strategy, i.e., the mapping relationship between the input state and the action parameters.

[0046] Furthermore, the present invention also includes an energy-saving driving control system for hydrogen fuel cell buses, which uses the above-mentioned energy-saving driving control method to achieve energy-saving driving optimization control in complex traffic scenarios.

[0047] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0048] 1. This invention proposes a two-layer deep reinforcement learning network architecture for energy-saving driving control of hydrogen fuel cell buses, which handles longitudinal speed control and energy management control respectively. The control strategy does not need to establish an inherent model environment. It can complete energy-saving driving optimization through the self-learning ability of deep reinforcement learning, thereby realizing efficient driving and energy saving of hydrogen fuel cell buses.

[0049] 2. This invention broadens the application of deep reinforcement learning theory and provides a new research approach for deep reinforcement learning methods in technologies such as fuel cell vehicle energy management and autonomous driving, which has strong practical application significance. Attached Figure Description

[0050] Figure 1 This is a flowchart of the energy-saving driving control method for hydrogen fuel cell buses of the present invention;

[0051] Figure 2 This is a diagram showing the relationship between the core components of the fuel cell modeling in this invention;

[0052] Figure 3 This is a schematic diagram of the energy-saving driving control method for hydrogen fuel cell buses based on two-layer deep reinforcement learning, as proposed in this invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the application will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in this invention. All non-innovative embodiments based on this invention by other researchers in the art are within the protection scope of this invention.

[0054] This invention provides an energy-saving driving control method for hydrogen fuel cell buses, such as... Figure 1 As shown, it includes the following steps:

[0055] S1. Construct a power system model for a hydrogen fuel cell bus and propose the maximum and minimum following distances;

[0056] S2. Utilize deep reinforcement learning methods to construct an energy-saving driving optimization method for hydrogen fuel cell buses based on a two-layer deep reinforcement learning algorithm;

[0057] S3. Establish a two-layer DDPG network, including constructing a first DDPG network to coordinate with the maximum and minimum following distance for longitudinal speed control of the hydrogen fuel cell bus, and constructing a second DDPG network for energy management optimization control based on the power system model of the hydrogen fuel cell bus.

[0058] S4. Two-layer network state sharing: Construct a two-layer deep reinforcement learning network architecture. Input the hydrogen fuel cell bus speed output by the first DDPG network into the second DDPG network. The second DDPG network performs energy management of the hydrogen fuel cell bus and optimizes the energy-saving driving of the hydrogen fuel cell bus.

[0059] The energy-saving driving control method for hydrogen fuel cell buses described in this invention constructs a power system model for the hydrogen fuel cell bus in step S1, as follows: Figure 2As shown, the model includes a power battery, a hydrogen fuel cell, a hydrogen storage tank, a DC / DC converter, a drive motor, and drive wheels. The fuel cell and the hydrogen storage tank are connected using hydrogen gas. The fuel cell and the power battery are connected to the DC / DC converter. The DC / DC converter is electrically connected to the drive motor. The drive motor is mechanically connected to the drive wheels.

[0060] Based on the above-mentioned hydrogen fuel cell bus power system model, this invention proposes the following formulas for the maximum and minimum following distances of hydrogen fuel cell buses:

[0061] L min = 2 + 0.5v + 0.0625v 2

[0062] L max =10 + v + 0.0825 v 2

[0063] Among them, L min L max These represent the maximum and minimum following distance limits for hydrogen fuel cell buses, respectively, and v represents the speed of the hydrogen fuel cell bus.

[0064] Furthermore, the energy-saving driving control method for hydrogen fuel cell buses described in this invention constructs an energy-saving driving optimization method for hydrogen fuel cell buses based on a two-layer deep reinforcement learning algorithm in step S2, including longitudinal speed control and energy management optimization control of the hydrogen fuel cell bus.

[0065] like Figure 3 As shown, the specific steps of this control method are as follows:

[0066] S201: Define the state, actions, reward function, optimal action-value function, and optimal control strategy of a hydrogen fuel cell bus in a deep reinforcement learning algorithm;

[0067] S202: The deep reinforcement learning agent receives the current state of the hydrogen fuel cell bus and executes the action to change the speed of the hydrogen fuel cell bus and the output power of the fuel cell.

[0068] S203: The environment changes due to the action. The agent obtains the new state of the hydrogen fuel cell bus and returns the new state and the reward brought by the action to the deep reinforcement learning agent.

[0069] S204: In the new state of the hydrogen fuel cell bus, the agent will continue to perform the action, and so on. The deep reinforcement learning agent will continuously interact with the environment until the optimal action-value function, i.e., the multi-objective Q-value vector, and the optimal control strategy are obtained.

[0070] The hydrogen fuel cell bus state, actions, reward function, optimal action-value function, and optimal control strategy defined in step S201 of the deep reinforcement learning algorithm are as follows:

[0071] First, the state S of the hydrogen fuel cell bus in the deep reinforcement learning algorithm is defined, which includes current traffic information and fuel cell information, specifically: the speed and acceleration of the hydrogen fuel cell bus, the speed and acceleration of the vehicle in front at the current moment, the distance between the hydrogen fuel cell bus and the vehicle in front at the current moment, the equivalent hydrogen consumption of the hydrogen fuel cell bus at the current moment, the output power of the hydrogen fuel cell at the current moment, and the remaining charge of the power battery at the current moment.

[0072] Then, the actions in the deep reinforcement learning algorithm are defined as the speed change (i.e., acceleration per second) of the hydrogen fuel cell bus and the output power of the hydrogen fuel cell, namely: action A1 and action A2, as detailed in the following formula:

[0073] A1={P fc |P fc ∈(0,60)}

[0074] A2 = {a | a ∈ [-1.5, 0.7]}

[0075] Among them, action A1 is the speed change of the hydrogen fuel cell bus, i.e., the acceleration per second; action A2 is the output power of the hydrogen fuel cell; P fc denoted as the output power value of the hydrogen fuel cell, and 'a' as the speed change value of the hydrogen fuel cell bus.

[0076] Next, the reward function R in the deep reinforcement learning algorithm is defined, which includes the equivalent hydrogen consumption cost, the range of power fluctuation difference of the hydrogen fuel cell, the safe following cost of the hydrogen fuel cell bus, and the comfortable following cost, specifically as follows:

[0077] R(s,a)=R1(s,a)+R2(s,a)+R3(s,a)+R4(s,a)

[0078] R1(s, a) = C h

[0079] R2(s, a) = C p

[0080] R3(s, a) = C s

[0081] R4(s, a) = C c

[0082] Where R(s, a) is the reward function vector for choosing action a in state s, R1(s, a) is the reward function reward1; R2(s, a) is the reward function reward2; R3(s, a) is the reward function reward3; R4(s, a) is the reward function reward4; C h For equivalent hydrogen consumption cost; C p The range of power fluctuation differences in hydrogen fuel cells; C s Cost of following the vehicle for safety of hydrogen fuel cell buses; C c Cost of comfort for hydrogen fuel cell buses.

[0083] Specifically, the equivalent hydrogen consumption cost C in deep reinforcement learning algorithms is defined. h The specific calculation formula is as follows:

[0084] C h =-h

[0085] Where h is the equivalent hydrogen consumption.

[0086] Hydrogen fuel cell power fluctuation range C p The specific calculation formula is as follows:

[0087]

[0088] in, These represent the fuel cell output power at the current time t and the next time t+1, respectively.

[0089] Safety Cost of Hydrogen Fuel Cell Bus (C) s The specific calculation formula is as follows:

[0090]

[0091] Where L represents the current distance between the hydrogen fuel cell bus and the vehicle in front.

[0092] The cost of comfort following the vehicle in a hydrogen fuel cell bus (C) c The specific calculation formula is as follows:

[0093]

[0094] Among them, a _ This represents the change in vehicle acceleration.

[0095] Finally, the optimal action-value function in deep reinforcement learning algorithms is defined, and its specific calculation formula is as follows:

[0096] Q * (s, a) = Q π (s, a) = maxE[R] t+1 +λQ * (st+1 a t+1 )|s t a t ]

[0097] In the above formula, Q π (s, a) is the action-value function for choosing action a in policy π state s; t a t Let s be the state and action at time t; t+1 a t+1 R t+1 Let be the state, action, and reward function at time t+1; λ∈[0,1] is the discount factor;

[0098] Furthermore, the energy-saving driving control method for hydrogen fuel cell buses described in this invention, in step S3, constructs a two-layer DDPG network, including constructing a first DDPG network to control the longitudinal speed of the hydrogen fuel cell bus in conjunction with the maximum and minimum following distances, and constructing a second DDPG network to perform energy management optimization control based on the hydrogen fuel cell bus power system model; it is divided into the following two steps:

[0099] S301: Establish a DDPG network, including a first DDPG network and a second DDPG network;

[0100] S302: Using the first DDPG network, optimize the longitudinal speed of the hydrogen fuel cell bus and execute the control action A1 to change the speed of the hydrogen fuel cell bus at the next moment; using the second DDPG network, optimize the energy management of the hydrogen fuel cell and execute the control action A2 to change the output power of the hydrogen fuel cell bus fuel cell at the next moment.

[0101] In step S301, the DDPG network is established, and the specific method is as follows:

[0102] The Actor-Critic method takes the current Actor network, target Actor network, current Critic network, and target Critic network as inputs, with parameters θ, θ′, ω, ω′, a decay factor γ, a soft update coefficient r, the number of samples for batch gradient descent m, the target Q-network parameter update frequency C, the maximum number of iterations T, and a random noise function N. The output is the optimal current Actor network parameters θ and the optimal current Critic network parameters ω.

[0103] First, randomly initialize θ, ω, ω′=ω, θ′=θ, and clear the experience replay set M.

[0104] In each episode:

[0105] Initialize S as the first state of the current state sequence, obtain its feature vector Φ(S), and in the current network of the Actor, obtain the action A = π based on the state S. θ (Φ(S))+N, execute action A, obtain new state S′, reward R and whether to terminate state is end, then store {Φ(S), A, R, Φ(S′), is end} in experience replay total M, update state S=S′.

[0106] Sample m samples {Φ(S) from the experience replay set. j A j R j ,Φ(S′ j ), isend j}, j = 1, 2, ..., m, calculate the current target Q value yj:

[0107]

[0108] Using the mean squared error loss function Update all parameters ω of the current Critic network using backpropagation of the neural network's gradient. The Actor updates all parameters θ of the current network through backpropagation of gradients from the neural network.

[0109] Continue by updating the parameters of the Critict target network and the Actor target network:

[0110] ω′←rω+(1-r)ω′

[0111] θ′←rθ+(1-r)θ′

[0112] If S′ is a terminating state, the current iteration is complete; otherwise, repeat the above steps.

[0113] Meanwhile, in step S302, two DDPG networks are used to optimize the longitudinal speed of the hydrogen fuel cell bus and the energy management of the fuel cell, respectively. This mainly includes: establishing a first DDPG network to optimize the longitudinal speed of the hydrogen fuel cell bus, with its control action being A1; and establishing a DDPG_2 network to optimize the energy management of the hydrogen fuel cell, with its action being A2.

[0114] Specifically, the energy-saving driving optimization state of the hydrogen fuel cell bus based on the two-layer deep reinforcement learning algorithm proposed in step S2 is shared in the two-layer network to accelerate the training speed of the intelligent agent; that is, the states of the first DDPG network and the DDPG network are the current speed of the hydrogen fuel cell bus, the current acceleration of the hydrogen fuel cell, the current speed of the vehicle in front, the current acceleration of the vehicle in front, the current distance between the hydrogen fuel cell bus and the vehicle in front, the current equivalent hydrogen consumption of the hydrogen fuel cell bus, the current output power of the hydrogen fuel cell, and the current remaining charge of the power battery.

[0115] Furthermore, the energy-saving driving control method for hydrogen fuel cell buses described in this invention, in step S4, constructs a two-layer deep reinforcement learning network architecture to optimize the energy-saving driving of hydrogen fuel cell buses, mainly including the following steps:

[0116] Step S401: Offline training; The model is trained using a two-layer deep reinforcement learning algorithm to learn the control policy, i.e., the mapping relationship between the input state and the action parameters;

[0117] Step S402: Read out the parameters of each trained conditional network and download the control strategy to the vehicle controller (VCU);

[0118] Step S403: Online learning; obtain information about the current status of the hydrogen fuel cell bus, and apply it to the trained conditional network to optimize the energy-saving driving of the hydrogen fuel cell bus through online real-time adjustment.

[0119] Specifically, the offline training in step S401 includes establishing a simulation environment for an urban expressway containing three sets of ramps with different characteristics for offline training; training the model using a two-layer deep reinforcement learning algorithm to learn the control strategy, i.e., the mapping relationship between the input state and the action parameters, mainly includes the following steps:

[0120] 1) Initialize the diverse experience replay buffer D; deduplicate the weighted vector experience pool W;

[0121] 2) Initialize the conditional network value function; obtain the weight vector w at the current time step. t And add it to W;

[0122] 3) The agent randomly selects an action a with probability ε. t The order was given to the environment to execute the a t Otherwise, execute action a. t =agmax a∈A Q CN (a j s j ;w)·w;

[0123] 4) The environment executes this a t Return reward r t and new state t+1 ;

[0124] 5) The agent will perform this state transition process: (s t a t r t ,s t+1 Saved to the diverse experience playback buffer D;

[0125] 6) Randomly select a portion of samples from the diverse experience replay buffer D, using (s j a j r j s j+1 ) indicates that w is randomly selected from the experience pool W of the weight vector. j Then, the target neural network is trained and updated. The learning process is as follows:

[0126]

[0127]

[0128] Among them, y j and y′ j For tags; r j λ represents the reward during the learning process; λ is the discount factor.

[0129] 7) Define TD error for network updates. TD error is:

[0130]

[0131] 8) Every N - In each round, the parameters of the online network are copied to the target network;

[0132] 9) Once the training steps are completed, the conditional network training is finished.

[0133] In particular, the present invention also includes an energy-saving driving control system for hydrogen fuel cell buses, which employs any of the energy-saving driving control methods described in the above specific embodiments to achieve energy-saving driving optimization control of hydrogen fuel cell buses in complex traffic scenarios.

[0134] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present invention, and these should also be considered within the scope of protection of the present invention.

Claims

1. A method for energy-saving driving control of a hydrogen fuel cell bus, characterized in that, Includes the following steps: S1. Construct a power system model for a hydrogen fuel cell bus and propose the maximum and minimum following distances; S2. Utilize deep reinforcement learning methods to construct an energy-saving driving optimization method for hydrogen fuel cell buses based on a two-layer deep reinforcement learning algorithm; S3. Establish a two-layer DDPG network, including constructing a first DDPG network to coordinate with the maximum and minimum following distance for longitudinal speed control of the hydrogen fuel cell bus, and constructing a second DDPG network for energy management optimization control based on the hydrogen fuel cell bus power system model, including the following sub-steps: S301: Establish a DDPG network, including a first DDPG network and a second DDPG network; S302: Optimize the longitudinal speed of the hydrogen fuel cell bus using the first DDPG network and execute actions. This is used to change the speed of the hydrogen fuel cell bus at the next moment; the second DDPG network is used to optimize the energy management of the hydrogen fuel cell and execute actions. This is used to change the output power of the hydrogen fuel cell bus fuel cell at the next moment; among which, the action... For the speed change of the hydrogen fuel cell bus, the action This refers to the output power of the hydrogen fuel cell; S4. Two-layer network state sharing: Constructing a two-layer deep reinforcement learning network architecture, including: sharing the energy-saving driving optimization state of the hydrogen fuel cell bus based on the two-layer deep reinforcement learning algorithm constructed in step S2 in the two-layer network constructed in step S3 to accelerate the algorithm training speed; after sharing, the states of the first DDPG network and the second DDPG network both include: the current speed of the hydrogen fuel cell bus, the current acceleration of the hydrogen fuel cell, the current speed of the vehicle in front, the current acceleration of the vehicle in front, the current distance between the hydrogen fuel cell bus and the vehicle in front, the current equivalent hydrogen consumption of the hydrogen fuel cell bus, the current output power of the hydrogen fuel cell, and the current remaining charge of the power battery; The speed of the hydrogen fuel cell bus output from the first DDPG network is input to the second DDPG network, which then manages the energy of the hydrogen fuel cell bus and optimizes its energy-saving driving performance. This includes the following sub-steps: S401: Offline training; The model is trained using a two-layer deep reinforcement learning algorithm to learn the control policy, i.e., the mapping relationship between the input state and the action parameters. S402: Read out the parameters of each trained conditional network and download the control strategy to the vehicle controller (VCU); S403: Online learning, obtaining information about the current state of the hydrogen fuel cell bus, and working together with the trained conditional network to optimize the energy-saving driving of the hydrogen fuel cell bus through online real-time adjustments.

2. The energy-saving driving control method for a hydrogen fuel cell bus according to claim 1, characterized in that: In step S1, a power system model for a hydrogen fuel cell bus is constructed, including a power battery, a fuel cell, a hydrogen storage tank, a DC / DC converter, a drive motor, and drive wheels. The fuel cell and the hydrogen storage tank are connected using hydrogen gas, the fuel cell and the power battery are connected to the DC / DC converter, the DC / DC converter is electrically connected to the drive motor, and the drive motor is mechanically connected to the drive wheels.

3. The energy-saving driving control method for a hydrogen fuel cell bus according to claim 2, characterized in that, The maximum and minimum following distances for the hydrogen fuel cell bus in step S1 are as follows: ； ； in, , These are the maximum and minimum following distance limits for hydrogen fuel cell buses. The speed of the hydrogen fuel cell bus.

4. The energy-saving driving control method for a hydrogen fuel cell bus according to claim 1, characterized in that: Step S2 includes longitudinal speed control and energy management optimization for the hydrogen fuel cell bus, specifically including the following sub-steps: S201: Define the state, actions, reward function, optimal action-value function, and optimal control strategy of a hydrogen fuel cell bus in a deep reinforcement learning algorithm; S202: The deep reinforcement learning agent receives the current state of the hydrogen fuel cell bus and executes the action to change the speed of the hydrogen fuel cell bus and the output power of the fuel cell. S203: The environment changes due to the action. The agent obtains the new state of the hydrogen fuel cell bus and returns the new state and the reward brought by the action to the deep reinforcement learning agent. S204: In the new state of the hydrogen fuel cell bus, the agent will continue to perform the action, and so on. The deep reinforcement learning agent will continuously interact with the environment until the optimal action-value function, i.e., the multi-objective Q-value vector, and the optimal control strategy are obtained.

5. The energy-saving driving control method for a hydrogen fuel cell bus according to claim 4, characterized in that: Step S201 is as follows: (1) The state of the hydrogen fuel cell bus in the deep reinforcement learning algorithm includes: the current speed and acceleration of the hydrogen fuel cell bus, the current speed of the vehicle in front, the current acceleration of the vehicle in front, the current distance between the hydrogen fuel cell bus and the vehicle in front, the current equivalent hydrogen consumption of the hydrogen fuel cell bus, the current output power of the hydrogen fuel cell, and the current remaining charge of the power battery. (2) Actions in deep reinforcement learning algorithms include: and actions Defined as follows: ； ； Among them, actions For the speed change of the hydrogen fuel cell bus, i.e., acceleration per second; action This refers to the output power of the hydrogen fuel cell; This represents the output power of the hydrogen fuel cell. This represents the speed change value of the hydrogen fuel cell bus. (3) The reward function, which includes the equivalent hydrogen consumption cost, the range of power fluctuation difference of hydrogen fuel cell, the safety following cost of hydrogen fuel cell bus, and the comfort following cost, is as follows: ； ； ； ； ； Where R(s,a) is the reward function vector for choosing action a in state s, R1(s,a) is reward function reward 1; R2(s,a) is reward function reward 2; R3(s,a) is reward function reward 3; R4(s,a) is reward function reward 4; This is the equivalent hydrogen consumption cost; This refers to the range of power fluctuation differences in hydrogen fuel cells. Costs associated with the safe following of hydrogen fuel cell buses; To reduce the cost of comfort-following in hydrogen fuel cell buses; (4) The specific formula for calculating the optimal action-value function is as follows: (s,a)= (s,a)=max E[ +λ ( , )| , ]; in, (s,a) is in the strategy Action-value function for choosing action a in state s; , Let t be the state and action at time t; , , Let be the state, action, and reward function at time t+1; This is the discount factor.

6. The energy-saving driving control method for a hydrogen fuel cell bus according to claim 1, characterized in that, In step S401, offline training includes establishing a simulation environment for an urban expressway containing three sets of ramps with different characteristics for offline training; and training the model using a two-layer deep reinforcement learning algorithm to learn the control strategy, i.e., the mapping relationship between the input state and the action parameters.

7. An energy-saving driving control system for a hydrogen fuel cell bus, characterized in that, Energy-saving driving optimization control in complex traffic scenarios is achieved by adopting any one of the energy-saving driving control methods in claims 1 to 6.

Images