Electric vehicle longitudinal control method based on deep reinforcement learning and sliding mode control

Abstract

The invention provides an electric vehicle longitudinal control method based on deep reinforcement learning and sliding mode control, and aims to solve the problem that the vehicle following speed and the vehicle distance are not accurately controlled during longitudinal driving of an automatic driving vehicle. Firstly, a relation model of the distance, the speed and the acceleration between a controlled vehicle and a front vehicle is established; then, longitudinal control is conducted through a sliding-mode control algorithm in combination with a vehicle longitudinal dynamics model, and sliding-mode control gain is introduced to improve the anti-interference capacity of the system. Secondly, a depth deterministic strategy algorithm DDPG method is used to carry out adaptive setting on sliding mode control gain, and an expected acceleration is output to a lower layer controller. And finally, designing a switching control strategy of an accelerator and a brake based on the inverse longitudinal dynamics model to realize stable switching of vehicle driving and braking and complete vehicle longitudinal control. Deep reinforcement learning and sliding mode control are fused into longitudinal control of the electric vehicle, the control precision is high, the adaptability is high, and safe and efficient driving of the vehicle is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of longitudinal control of electric vehicles, and specifically relates to a longitudinal control method of an electric vehicle based on deep reinforcement learning and sliding mode control. Background Art

[0002] The longitudinal control technology of electric vehicles relies on advanced control methods to accurately track decision modules, precisely control the vehicle's driving speed and driving distance, and achieve safe and efficient driving of vehicles. It is a hot topic and a difficult problem in the field of autonomous vehicles. By optimizing the control algorithm, the response speed and adaptability of the autonomous driving system can be further improved, laying the foundation for the popularization and application of intelligent transportation systems in the future.

[0003] Some existing patents, such as the invention patent with patent number CN118906841A, have designed a method for longitudinal control of the wire-controlled chassis of electric vehicles. The unmanned driving controller sends the expected speed to the vehicle controller, and the vehicle controller identifies and compares the actual speed and the expected speed, controls the acceleration, deceleration and braking of the vehicle, which can improve the energy utilization rate of the vehicle during deceleration and make parking on slopes safer, but the patent lacks exploration of the robustness of longitudinal control. The invention patent with patent number CN114872705A has designed a method for longitudinal control of a wire-controlled vehicle based on acceleration closed-loop control. Through the design of the outer speed loop and the inner torque loop, the control torque is calculated and output to the actuator, but the control parameters are fixed parameters and lack adaptive capabilities.

[0004] However, in a changing traffic environment, the longitudinal control of electric vehicles faces many challenges. Existing control methods, such as PID control and fuzzy control, can meet the control requirements to a certain extent, but their performance is often limited when dealing with the uncertainty and interference of complex traffic environments, making it difficult to achieve optimal driving stability. At the same time, although deep reinforcement learning has shown great potential in the field of intelligent control, existing research has not fully explored the use of this technology to optimize the longitudinal control strategy of electric vehicles, and has failed to give full play to its advantages in dealing with complex system and environmental interactions. Therefore, how to develop an electric vehicle longitudinal control algorithm with self-learning ability and strong robustness to improve driving stability and control accuracy is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention proposes a longitudinal control method for an electric vehicle based on deep reinforcement learning and sliding mode control to solve the problem of inaccurate control of following speed and vehicle spacing during longitudinal driving of existing autonomous driving vehicles. By combining deep reinforcement learning and sliding mode control theory, an innovative longitudinal control method for an electric vehicle is provided, which effectively optimizes the longitudinal control performance of the vehicle and improves driving safety and comfort.

[0006] A longitudinal control method for an electric vehicle based on deep reinforcement learning and sliding mode control, wherein the transmission system is composed of a power battery, a battery energy management system, a motor, a motor controller, a reducer, and wheels. The motor controller controls the motor to drive the wheels via the reducer, and the motor is powered by a power battery driven by the battery energy management system. The control method includes:

[0007] S1: Establish a relationship model between the distance, speed and acceleration between the controlled vehicle and the vehicle in front

[0008] S2: Combined with the vehicle longitudinal dynamics model, the sliding mode control algorithm is used for vehicle longitudinal control. The control law is designed by combining equivalent control with arrival control. In order to overcome the influence of interference, the sliding mode control gain η is introduced to maintain stability.

[0009] S3: By receiving vehicle status information, the DDPG network collects the error between the expected command and the actual output command and the control signal of the sliding mode controller, and calculates the cumulative reward value according to the designed reward and punishment function (safety factor, speed error, comfort and sensitivity factor). Then, the surrounding environment status information, expected acceleration value and cumulative reward value obtained during the training process are stored in the cache pool, and sample data is randomly collected for neural network training, so as to adaptively adjust the sliding mode control gain η and output the expected acceleration a. ref (t) To the lower level controller.

[0010] S4: Design the throttle-brake switching control strategy based on the inverse longitudinal dynamics model to achieve smooth switching between vehicle driving and braking, and finally output the motor torque and brake master cylinder pressure to the actuator to complete the longitudinal control of the vehicle.

[0011] Furthermore, step S1 is specifically as follows:

[0012] Assuming that the vehicle (the controlled vehicle) is the nth vehicle, the vehicle in front is the n-1th vehicle. The relationship between the distance, speed, acceleration of the vehicle and the relative distance between the two vehicles is:

[0013]

[0014] In the formula, x n (t) is the position of the nth vehicle, v n (t) is the speed of the nth vehicle, a n (t) is the acceleration of the nth vehicle, Δx n (t) is the relative distance between the two vehicles.

[0015] The distance error e(t) between the two vehicles is further deduced as:

[0016] e(t)=Δx n (t)+L (2)

[0017] Where L is the braking distance.

[0018] Furthermore, the step S2 is specifically as follows:

[0019] S21: Considering the vehicle dynamics relationship, the motor torque T can be obtained m With vehicle acceleration a x The relationship is:

[0020]

[0021] In the formula, r w is the wheel radius, i tran is the transmission system speed ratio, J e is the equivalent moment of inertia of the transmission system, γ eff is the transmission system efficiency, ρ d is the air density, v x is the vehicle speed, F f is the rolling resistance, T b is the braking torque.

[0022] Speed Acceleration Convert equation (3) to the standard form of equation (4):

[0023]

[0024] In the formula, M, f, c are system parameters, u is the control parameter,

[0025] Definition i1 =e(t), The state equation of formula (4) is:

[0026]

[0027] S22: Design sliding surface function:

[0028] S i =q 1 e i1 +q 2 e i2 q 1 >0,q 2 >0 (6)

[0029] In the formula, q 1 ,q 2 Is a normal number.

[0030] S23: The sliding mode control law design adopts the combination of equivalent control and arrival control, namely:

[0031]

[0032] In the formula, u eq is an equivalent control term, u sw is the arrival control term, η is the sliding mode switching gain, which can be adjusted in real time through the DDPG algorithm.

[0033] Based on the control quantity calculation formula (7) and the vehicle dynamics relationship (3), the expected acceleration a can be calculated: ref .

[0034] The step S4 is specifically as follows:

[0035] S41: Determine the vehicle state according to the expected acceleration. When the expected acceleration value a ref When it is greater than 0.1, it enters the drive control mode; when the expected acceleration value a ref When it is less than -0.1, it enters the braking control mode; when the expected acceleration value a ref When the absolute value is less than 0.1, the vehicle state does not belong to the driving control and braking control modes, and it decelerates under the effects of rolling resistance, air resistance, and slope resistance;

[0036] S42: When the vehicle enters the driving control mode, the expected motor torque T is obtained in combination with the vehicle dynamics relationship. m,req The calculation formula is:

[0037]

[0038] In the formula, F w is the air resistance, F i is the slope resistance, F j is the acceleration resistance, m v is the total mass of the vehicle, g is the acceleration due to gravity, f r is the rolling resistance coefficient, θ road is the road slope angle, m v is the total mass of the vehicle, g is the acceleration due to gravity, f r is the rolling resistance coefficient, θ road is the road slope angle, C D is the air resistance coefficient, A f is the frontal area, and δ is the rotating mass conversion factor.

[0039] S43: When the vehicle enters the brake control mode, the brake master cylinder pressure P is obtained by combining the vehicle force and the brake system structure. b The calculation formula is:

[0040]

[0041] In the formula, K bis the brake efficiency factor, and R is the brake action radius.

[0042] Compared with the prior art, the advantages of the present invention are:

[0043] 1. The present invention proposes a method for longitudinal control of electric vehicles based on deep reinforcement learning and sliding mode control, which solves the problem of inaccurate control of following vehicle speed and vehicle spacing during longitudinal driving of autonomous vehicles, and provides a new solution for longitudinal control of electric vehicles.

[0044] 2. The electric vehicle longitudinal control method based on deep reinforcement learning and sliding mode control described in the present invention combines the sliding mode control algorithm with the deep deterministic policy gradient (DDPG) method, and realizes precise control of the vehicle's desired acceleration through adaptive setting of the sliding mode control gain, thereby significantly improving the system's anti-interference ability and control accuracy.

[0045] 3. The electric vehicle longitudinal control method based on deep reinforcement learning and sliding mode control described in the present invention designs a throttle and brake switching control strategy based on the inverse longitudinal dynamics model, realizes the smooth switching of vehicle drive and braking, effectively improves the adaptability and driving safety of vehicle longitudinal control, and provides important guiding significance for the longitudinal control of autonomous driving vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the electric vehicle system structure used in the embodiment;

[0047] Figure 2 A flow chart of a longitudinal control method for an electric vehicle based on deep reinforcement learning and sliding mode control used in an embodiment;

[0048] Figure 3 A flow chart of adjusting the sliding mode control gain using the DDPG reinforcement learning method used in the embodiment;

[0049] Figure 4 A schematic diagram of the DDPG reinforcement learning method used in the embodiment;

[0050] Figure 5 Flow chart of the throttle and brake switching control method based on the inverse longitudinal dynamics model used in the embodiment.

[0051] Labels in the figure: 1. Power battery, 2. Battery energy management system, 3. Motor, 4. Motor controller, 5. Reducer, 6. Wheel. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described examples are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Example

[0054] A longitudinal control method for an electric vehicle based on deep reinforcement learning and sliding mode control, wherein the transmission system is as follows Figure 1 As shown, it is composed of a power battery 1, a battery energy management system 2, a motor 3, a motor controller 4, a reducer 5 and a wheel 6. The motor controller 4 controls the motor 3 to drive the wheel 6 via the reducer 5, and the motor 3 is powered by a power battery driven by the battery energy management system 2.

[0055] The flowchart of the longitudinal control method based on deep reinforcement learning and sliding mode control for electric vehicles is as follows Figure 2 As shown. First, a relationship model of the distance, speed and acceleration between the controlled vehicle and the vehicle in front is established; then, combined with the vehicle longitudinal dynamics model, a sliding mode control algorithm is used for longitudinal control, and a sliding mode control gain is introduced to improve the system's anti-interference ability. Secondly, the deep deterministic strategy algorithm DDPG (Deep Deterministic Policy Gradient) method is used to adaptively adjust the sliding mode control gain, and output the desired acceleration to the lower-level controller. Finally, based on the inverse longitudinal dynamics model, a switching control strategy for the throttle and brake is designed to achieve smooth switching between vehicle drive and braking and complete vehicle longitudinal control. The longitudinal control method specifically includes:

[0056] S1: Establish a relationship model between the distance, speed and acceleration between the controlled vehicle and the vehicle in front

[0057] Assuming that the vehicle (the controlled vehicle) is the nth vehicle, the vehicle in front is the n-1th vehicle. The relationship between the distance, speed, acceleration of the vehicle and the relative distance between the two vehicles is:

[0058]

[0059] In the formula, x n (t) is the position of the nth vehicle, v n (t) is the speed of the nth vehicle, a n (t) is the acceleration of the nth vehicle, Δx n (t) is the relative distance between the two vehicles.

[0060] The distance error e(t) between the two vehicles is further deduced as:

[0061] e(t)=Δx n (t)+L (2)

[0062] Where L is the braking distance.

[0063] S2: Combined with the vehicle longitudinal dynamics model, the sliding mode control algorithm is used for vehicle longitudinal control. The control law is designed by combining equivalent control with arrival control. In order to overcome the influence of interference, the sliding mode control gain η is introduced to maintain stability, such as Figure 3 As shown, the specific process is:

[0064] S21: Considering the vehicle dynamics relationship, the motor torque T can be obtained m With vehicle acceleration a x The relationship is:

[0065]

[0066] In the formula, r w is the wheel radius, i tran is the transmission system speed ratio, J e is the equivalent moment of inertia of the transmission system, γ eff is the transmission system efficiency, ρ d is the air density, v x is the vehicle speed, F f is the rolling resistance, T b is the braking torque.

[0067] Speed Acceleration Convert equation (3) to the standard form of equation (4):

[0068]

[0069] In the formula, M, f, c are system parameters, u is the control parameter,

[0070] Definition i1 =e(t), The state equation of formula (4) is:

[0071]

[0072] S22: Design sliding surface function:

[0073] S i =q 1 e i1 +q 2 e i2 q 1 >0,q 2 >0 (6)

[0074] In the formula, q 1 ,q 2 Is a normal number.

[0075] S23: The sliding mode control law design adopts the combination of equivalent control and arrival control, namely:

[0076]

[0077] In the formula, u eq is an equivalent control term, u sw is the arrival control term, η is the sliding mode switching gain, which can be adjusted in real time through the DDPG algorithm.

[0078] Based on the control quantity calculation formula (7) and the vehicle dynamics relationship (3), the expected acceleration a can be calculated: ref .

[0079] S3: DDPG reinforcement learning method is used to realize the parameter tuning of sliding mode control module, such as Figure 4 By receiving vehicle status information, the DDPG network collects the error between the expected command and the actual output command and the control signal of the sliding mode controller, and calculates the cumulative reward value according to the designed reward and punishment function (safety factor, speed error, comfort and sensitivity factor). Then, the surrounding environment status information, expected acceleration value and cumulative reward value obtained during the training process are stored in the cache pool, and sample data is randomly collected for neural network training, so as to adaptively adjust the sliding mode control gain η and output the expected acceleration a ref (t) To the lower level controller.

[0080] S4: Design the throttle-brake switching control strategy based on the inverse longitudinal dynamics model to achieve smooth switching between vehicle driving and braking, and finally output the motor torque and brake master cylinder pressure to the actuator to complete the longitudinal control of the vehicle, such as Figure 5 shown.

[0081] The specific process is:

[0082] S41: Determine the vehicle state according to the expected acceleration. When the expected acceleration value a ref When it is greater than 0.1, it enters the drive control mode; when the expected acceleration value a ref When it is less than -0.1, it enters the braking control mode; when the expected acceleration value a ref When the absolute value is less than 0.1, the vehicle state does not belong to the driving control and braking control modes, and it decelerates under the effects of rolling resistance, air resistance, and slope resistance;

[0083] S42: When the vehicle enters the driving control mode, the expected motor torque T is obtained in combination with the vehicle dynamics relationship. m,req The calculation formula is:

[0084]

[0085] In the formula, F w is the air resistance, F i is the slope resistance, F j is the acceleration resistance, m v is the total mass of the vehicle, g is the acceleration due to gravity, f r is the rolling resistance coefficient, θ road is the road slope angle, m v is the total mass of the vehicle, g is the acceleration due to gravity, f r is the rolling resistance coefficient, θ road is the road slope angle, C D is the air resistance coefficient, A f is the frontal area, and δ is the rotating mass conversion factor.

[0086] S43: When the vehicle enters the brake control mode, the brake master cylinder pressure P is obtained by combining the vehicle force and the brake system structure. b The calculation formula is:

[0087]

[0088] In the formula, K b is the brake efficiency factor, and R is the brake action radius.

[0089] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A longitudinal control method for electric vehicles based on deep reinforcement learning and sliding mode control, characterized in that: The steps include: S1: Establish a relationship model between the distance, speed and acceleration between the controlled vehicle and the preceding vehicle; S2: Combined with the vehicle longitudinal dynamics model, the sliding mode control algorithm is used for vehicle longitudinal control. The control law is designed by combining equivalent control with arrival control. In order to overcome the influence of interference, the sliding mode control gain η is introduced to maintain stability. S3: By receiving vehicle status information, the DDPG network collects the error between the expected command and the actual output command and the control signal of the sliding mode controller, calculates the cumulative reward value according to the designed reward and punishment function, and then stores the surrounding environment status information, expected acceleration value and cumulative reward value obtained during the training process into the cache pool, randomly collects sample data for neural network training, and adaptively adjusts the sliding mode control gain η to output the expected acceleration a ref (t) to the lower level controller; S4: Based on the inverse longitudinal dynamics model, the throttle-brake switching control strategy is designed to achieve smooth switching between vehicle driving and braking, and finally output the motor torque and brake master cylinder pressure to the actuator to complete the longitudinal control of the vehicle.

Images