Brake control method based on comfort of automatic driving vehicle

By using the TD3 deep reinforcement learning algorithm to train the Q matrix in stages in the braking control of autonomous vehicles, dynamically adjusting the weight coefficient, the vehicle's comfort problem under different working conditions is solved, and more efficient braking control is achieved.

CN119568090AActive Publication Date: 2025-03-07HAINAN UNIV

Patent Information

Application Number
CN202411636033.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-15
Publication Date
2025-03-07
Estimated Expiration
2044-11-15

AI Technical Summary

Technical Problem

In the braking control of autonomous driving vehicles, it is difficult to maintain the comfort of the vehicle under different operating conditions. Especially in emergency operating conditions, the pitch angle of the vehicle varies greatly, and the Q weight coefficient of the model prediction controller is set to a fixed value, resulting in poor braking effect.

Method used

The TD3 deep reinforcement learning algorithm is used to train the Q matrix in the model prediction control in stages, and the Q weight coefficient is dynamically adjusted to adapt to the needs of different braking stages to generate the most comfortable control sequence.

Benefits of technology

By dynamically adjusting the Q weight coefficient, the jitter in the vehicle pitch angle changes can be reduced, the comfort of the braking process can be improved, and the real-time adaptability of the autonomous driving system can be improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119568090A_ABST
    Figure CN119568090A_ABST
Patent Text Reader

Abstract

The invention relates to a braking control method based on the comfort of an automatic driving vehicle, which comprises the following steps of: 1, collecting the speed and distance information of a front obstacle, and analyzing the collected road information and vehicle information; comprehensively judging whether the condition is an emergency condition or not by combining the current speed of the vehicle, the speed of the front vehicle, the distance information with the obstacle and the like; 2, learning a Q matrix in model prediction control through a TD3 deep reinforcement learning algorithm; 3, the model prediction control model estimates the current braking stage according to the information input data, and calls a Q matrix which is suitable for the current stage and has the best comfort from a strategy library to carry out optimization solution; 4, calling the trained Q matrix to carry out optimization solution so as to generate a most comfortable control sequence; and 5, the control sequence is transmitted to a vehicle control module for signal conversion, braking torque is generated, and the pitch angle of the whole vehicle is made to be within a comfortable range.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of autonomous driving and discloses a braking control method based on the comfort of an autonomous driving vehicle. Background Art

[0002] In recent years, with the development of autonomous driving technology, comfort has become one of the key factors in improving passenger experience. Among them, braking control plays a vital role in affecting the riding experience. Braking control can be divided into emergency braking and normal braking. At present, comfortable braking for normal conditions is mainly divided into three categories. One is to control the vehicle suspension, and distribute different total braking forces and front and rear axle distribution ratios at different vehicle speeds to ensure that the vehicle pitch angle during braking is always maintained within a comfortable range. The second category is to optimize the braking deceleration with the help of empirical formulas under normal conditions, thereby improving the comfort of the vehicle. The third category is to jointly control the vehicle suspension and braking system, but the cost is relatively high. Although scholars have conducted many studies, there are still shortcomings:

[0003] 1. Many scholars use braking force distribution to perform comfortable braking and derive the relationship between deceleration and braking force. For example, they observe the change of the minimum pitch angle and use feedforward control to distribute the braking force, but this method can only guarantee the control effect under quasi-static conditions.

[0004] 2. Many scholars also focus on suspension control and use two model predictive controllers to control the distribution of total braking force and front and rear axle braking force. However, they cannot take advantage of the decoupling of the control system's braking force, and the two model predictive controllers are relatively complex. Therefore, most scholars set the weight coefficient of the model predictive controller to a fixed value, which greatly reduces the braking effect and affects the passenger experience.

[0005] In summary, at present, many comfortable braking methods use model predictive controllers to control vehicles. However, during the braking process, the pitch angles in the early, middle and late stages of the braking period are quite different. The Q weight coefficient of the model predictive controller is set to a fixed value, and the surrounding environment and vehicle speed change at any time, so it is difficult to achieve a comfortable braking effect. Summary of the invention

[0006] The present invention discloses a braking control method based on the comfort of an autonomous driving vehicle, and the braking control method based on the comfort of an autonomous driving vehicle comprises the following steps:

[0007] Step 1: Collect the speed and distance information of the obstacle ahead, analyze the collected road information and vehicle information, and then comprehensively judge whether it is an emergency condition based on the current speed of the vehicle, the speed of the vehicle ahead, and the distance information to the obstacle;

[0008] Step 2: Learn the Q matrix in model predictive control through the TD3 deep reinforcement learning algorithm;

[0009] Step 3: The model predictive control model estimates the current braking stage according to the information input data, and calls the Q matrix that best suits the current stage of comfort from the strategy library for optimization and solution;

[0010] Step 4: Call the trained Q matrix for optimization and solution to generate the most comfortable control sequence;

[0011] Step 5: The control sequence is transmitted to the vehicle control module for signal conversion to generate a braking torque so that the vehicle pitch angle is within a comfortable range throughout the entire process.

[0012] Furthermore, in step 2, the following steps are also included:

[0013] Step 21, constructing a TD3 training module;

[0014] Step 22, continuously adjust the hyperparameters during the training process, and observe the control effects of acceleration, pitch angle, and pitch angular velocity to achieve the most matching reward weight coefficient;

[0015] Step 23, use the deep reinforcement learning algorithm TD3 to train the Q weight coefficients of acceleration, pitch angle, and pitch angular velocity in the early, middle, and late stages of braking, evaluate the reward evaluation effects of different weight coefficients, and put the trained Q weight coefficients back into the strategy library.

[0016] Furthermore, in step 21, in order to ensure that the vehicle can brake smoothly, the reward function of the TD3 training module is set to a function related to vehicle comfort:

[0017] r a =-γ1·|aa T |

[0018] r θ =-γ2·|θ-θ T |

[0019]

[0020] They are acceleration a, pitch angle θ, and pitch angular velocity. The reward is γ1, γ2, γ3, and γ4, respectively. T represents the target acceleration, θ T represents the target pitch angle, Indicates the target pitch angular velocity;

[0021] r总 The expression of is the sum of the three:

[0022]

[0023] Among them, w1, w2, and w3 are the weight coefficients of the acceleration reward stage, the pitch angle reward stage, and the pitch angular velocity stage respectively;

[0024] The training is carried out in stages. The braking stage is mainly divided into the early, middle and late stages. In the early and middle stages of vehicle braking, MPC a It has a greater weight in this stage, followed by pitch angle, and finally pitch angle angular velocity. The reward expression in the early and middle stages is:

[0025]

[0026] w 11 Indicates the acceleration weight coefficient in the early and middle stages of braking, w 21 represents the pitch angle weight coefficient in the early and middle stages of braking, w 31 Indicates the weight coefficient of pitch velocity in the middle stage before braking.

[0027] At the end of vehicle braking, MPC In this stage, the weight should be large, so the pitch velocity is the main target in the late braking stage, followed by the pitch angle, and finally the acceleration; the reward expression in the late stage is:

[0028]

[0029] w 21 represents the acceleration weight coefficient in the late braking stage, w 22 represents the pitch angle weight coefficient in the late braking stage, w 32 Indicates the pitch velocity weight coefficient in the late braking stage.

[0030] Furthermore, in step 21, the target network update mode is soft update, and the soft update expression is:

[0031] θ target ←τ·θ+(1-·τ)·θ target

[0032] τ is the soft update coefficient, θ represents the current network parameters, and θ target represents the target network parameters;

[0033] The expression of the objective function is as follows:

[0034]

[0035] is the objective function J with respect to the policy network parameter θμ The gradient of is the gradient of the Q function with respect to the action, is the action gradient of the policy network, i is the sample number, and N is the total number of samples;

[0036] The parameter θ of the Q-value function is optimized by minimizing the mean square error between the predicted value and the target value. Q , so the expression of the loss function is:

[0037]

[0038] Q(s i ,a i |θ Q ) represents the predicted value, y i Indicates the target value.

[0039] Furthermore, in step 3, the model predictive controller continuously obtains Q matrix parameters from the TD3 training module, matches different weight coefficients in different braking stages, generates corresponding actions and states, and puts them back into the TD3 training module experience pool, repeating this cycle and continuously updating the Q matrix library.

[0040] Furthermore, in step 4, according to the model predictive control principle, the comfort optimization function can be obtained as:

[0041]

[0042] Where J represents the comfort optimization function, m represents the sampling step, m = 1:1:N P , N P represents the prediction step length, y(k+m|t) represents the control output prediction value, and y ref (k+m|t) represents the control output reference value, N C represents the control step length, Δu(k+m|t) represents the control input increment, Q is the system output weight system matrix, R is the control weight coefficient matrix, ρ is the weight coefficient, and ε is the relaxation factor; q a is the acceleration weight coefficient, q θ is the vehicle pitch angle weight coefficient, is the vehicle pitch angular velocity weight coefficient;

[0043] The minimum value min J corresponding to the comfort objective function solver and the prevention of sudden changes in the control quantity are added, so the following constraints are added:

[0044] u min (k+m)≤u(k+m|k)≤u max (k+m)

[0045] Δu min(k+m)≤Δu(k+m|k)≤Δu max (k+m)

[0046] u min Indicates the minimum value of the control input, u max Indicates the maximum value of the control input, Δu max Indicates the maximum value of the control increment, Δu mim Indicates the minimum value of the control increment;

[0047] With reference to the expected vehicle speed, according to the output parameter Y of the acceleration prediction model, a series of optimal acceleration increments ΔU(t) under system constraints can be solved, and the first acceleration increment Δu(k|t) of the series is taken, and the acceleration control amount at the previous moment is added to obtain the current acceleration control amount u. The beneficial effects achieved by the present invention are:

[0048] Improve braking comfort: Use the TD3 deep reinforcement learning algorithm to train the Q matrix of the model predictive control in stages. The adaptive MPC dynamically adjusts the Q weight coefficient to adapt to the solution in different braking stages, reducing the jitter caused by changes in pitch angle, thereby improving comfort.

[0049] Real-time adaptability: Dynamically adjusted MPC can better adapt to the ever-changing driving environment, thereby improving the flexibility and responsiveness of the autonomous driving system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flow chart of a braking control method based on the comfort of an autonomous driving vehicle proposed by the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described below in conjunction with specific embodiments, and the advantages and features of the present invention will become clearer as the description proceeds. However, these embodiments are exemplary only and do not constitute any limitation to the scope of the present invention. It should be understood by those skilled in the art that the details and forms of the technical solution of the present invention may be modified or replaced without departing from the spirit and scope of the present invention, but these modifications and replacements all fall within the scope of protection of the present invention.

[0052] like Figure 1 As shown, the present invention provides a braking control method based on the comfort of an autonomous driving vehicle, and the braking control method based on the comfort of an autonomous driving vehicle comprises the following steps:

[0053] Step 1: First, use sensors such as on-board laser radar and binocular cameras to collect speed and distance information of obstacles ahead, analyze the collected road information and vehicle information, and then combine the current speed of the vehicle, the speed of the vehicle ahead, and the distance information to the obstacle to comprehensively determine whether it is in an emergency or non-emergency condition.

[0054] Specifically, the following steps are also included:

[0055] Step 11, scene recognition;

[0056] The relevant measurement data is obtained through the on-board laser radar, binocular camera and other sensors, and the data is input into the scene recognition module. The scene recognition module can make estimates based on these measurement data to obtain the speed of the autonomous driving vehicle, the speed of the vehicle in front, the distance from the vehicle in front to the autonomous driving vehicle, the pitch angle of the vehicle, etc.

[0057] Step 12, making a judgment based on the road information and vehicle information obtained by the data acquisition module, so as to switch the working mode;

[0058] The main working process is as follows: If the scene recognition module obtains that the speed of the vehicle in front decreases sharply or other sudden scenes, combined with the brake plate displacement distance is too large, the vehicle switches to the emergency braking condition. At this time, safety is more important than comfort, and emergency braking should be performed. On the contrary, if the data acquisition module obtains that the vehicle in front stops, or is at a red light at the intersection, combined with the brake plate displacement distance is not large, the vehicle is in normal braking condition, and then subsequent comfort braking is performed.

[0059] Step 2: Learn the Q matrix in model predictive control through the TD3 deep reinforcement learning algorithm.

[0060] The TD3 training module uses the TD3 deep reinforcement learning algorithm to learn the Q matrix parameters in the model predictive control, and generates corresponding prediction model outputs based on road information and speed information to facilitate subsequent optimal control calculations.

[0061] The TD3 training module sets up a suitable reward function with the goal of minimizing acceleration, pitch angle, and pitch angular velocity to evaluate the performance of different weight coefficients of the Q matrix of the model predictive control system. During the training process, the TD3 algorithm interacts with the environment to collect experience and optimize. First, the action is selected through the policy network, and the current state is used as input. The currently selected action, that is, the corresponding Q matrix, is applied to the model predictive controller to obtain the next state and reward. At the same time, the experience is stored in the experience playback buffer, and then the policy network and value function network are updated by minimizing the loss function defined by the TD3 algorithm.

[0062] Specifically, the following steps are also included:

[0063] Step 21: Build TD3 training module

[0064] The reinforcement learning model observation states used to output the prediction model are longitudinal velocity, longitudinal acceleration, and vehicle pitch angle, and the prediction model output can be represented by the sequence:

[0065] [v(t),a(t)]......[v(t+P-1),a(t+P-1)]

[0066] v(t) represents velocity, a(t) represents acceleration, and P represents the prediction time step.

[0067] TD3 is an actor-critic algorithm that contains a policy network (Actor) and two value networks (Critic). The policy network (Actor) inputs the state observed by the agent and outputs the action u(s|θ μ )

[0068] where θ μ is the parameter of the policy network, which aims to maximize the Q value output by the Critic network. The value network (Critic) evaluates the Q value Q(s,a|θ) of the state-action pair (s,a) Q ), where θ Q is the parameter of the Critic network. Two Critic networks are used to calculate the Q value of the action, and their minimum value is taken as the target value to reduce over-estimation. The target Q value satisfies the following expression:

[0069]

[0070] Where r is the immediate reward, μtarget is the target policy network, and γ is the discount factor. The value of the discount factor is between 0 and 1. The larger the value, the more important the future reward is. In the actual braking control process, in order to ensure that the vehicle can brake smoothly, the reward function of the deep reinforcement learning model is set to a function related to vehicle comfort:

[0071] r a =-γ1·|aa T |

[0072] r θ =-γ2·|θ-θ T |

[0073]

[0074] They are acceleration a, pitch angle θ, and pitch angular velocity. The reward is γ1, γ2, and γ3 are the respective reward weight coefficients. The smaller the difference between the acceleration, pitch angle, and pitch angular velocity and the target value, the higher the reward value. 总 The expression of is the sum of the three:

[0075]

[0076] Among them, w1, w2, and w3 are the weight coefficients of the acceleration reward stage, the pitch angle reward stage, and the pitch angular velocity stage. The training is carried out in stages. The braking stage is mainly divided into the early, middle, and late stages. In the early and middle stages of vehicle braking, the focus is mainly on the safe distance of the vehicle, so acceleration is the main goal. a In this stage, it should have a greater weight, followed by the pitch angle, and finally the pitch angle angular velocity. The reward expression in the early and middle stages is:

[0077]

[0078] At the end of vehicle braking, the vehicle speed is relatively slow. When the vehicle speed drops to 0, the vehicle will "shake". The weight should be larger in this stage, so the pitch velocity is the main target in the late stage of braking, followed by the pitch angle, and finally the acceleration. The reward expression in the late stage is

[0079]

[0080] This phased training can better fit the current required weight coefficient.

[0081] The target network update method is soft update, and the soft update expression is:

[0082] θ target ←τ·θ+(1-τ)·θ target

[0083] τ is the soft update coefficient. The expression of the objective function is as follows:

[0084]

[0085] is the objective function J with respect to the policy network parameter θ μ The gradient of the Q function for each sample i is calculated by and the policy network for the action gradient The sum of the predicted values ​​and the target values ​​is then averaged to obtain the final gradient estimate. The parameter θ of the Q value function is then optimized by minimizing the mean square error between the predicted value and the target value. Q , so the expression of the loss function is:

[0086]

[0087] Step 22, during the training process, the hyperparameters are continuously adjusted to observe the control effects of acceleration, pitch angle, and pitch angular velocity to achieve the best matching γ1, γ2, and γ3;

[0088] Step 23, in addition, the acceleration, pitch angle, and pitch angular velocity Q weight coefficients in the early, middle, and late stages of braking are different, so the three weight coefficients are trained using the deep reinforcement learning algorithm TD3 to evaluate the reward evaluation effects of different weight coefficients, and the trained Q weight coefficients are put back into the strategy library.

[0089] Step 3: The model predictive control model estimates the current braking stage (early, middle, and late stages) based on the information input data, and calls the Q matrix that best suits the current stage for comfort from the strategy library for optimization and solution.

[0090] The MPC actual control module is based on the vehicle dynamics model, and uses model predictive control to calculate the optimal control strategy by inputting vehicle status and environmental information in real time. In addition, the MPC actual control module can call the most suitable Q weight coefficient for MPC from the TD3 strategy library to facilitate the MPC optimal value solution and ultimately generate a control signal with strong comfort.

[0091] The real-time adjustment of weight coefficients is that the TD3 training module dynamically changes the Q weight parameters in the comfort objective function during the MPC control process, so that the optimal control signal sequence output by MPC is further optimized. The comfort of the vehicle is effectively guaranteed. The following are several key aspects of the real-time adjustment of weight coefficients:

[0092] For a complete braking control, the braking process is divided into the early, middle, and late stages. The vehicle vibration in the early and middle stages of braking is mainly caused by the change in acceleration, while in the late stage of braking, the dizziness of passengers mainly comes from the vehicle vibration caused by the sudden change of the vehicle pitch angle, which causes the passenger riding experience to deteriorate.

[0093] Therefore, the emphasis of model predictive control should be different in different braking stages. This braking method combines TD3 algorithm with model predictive control for joint training. In the early and middle stages of braking, a larger acceleration weight coefficient q is set. a and the pitch angle weight coefficient q θ Because in the early and middle stages of braking, the braking distance accounts for a large proportion of the entire braking process and needs to be strictly followed, acceleration and pitch angle are used as the main optimization targets and pitch angle velocity is used as the secondary optimization target. In the late stage of braking, the vehicle speed is slow, and the pitch angle will suddenly change when the vehicle slides to stop, and the vehicle body will shake. Therefore, in the late stage of braking, pitch angle velocity is used as the main optimization target, the secondary optimization target is the pitch angle, and the third optimization target is acceleration. The TD3 training module should adjust and set a larger pitch angle velocity weight coefficient

[0094] The model predictive controller continuously obtains Q matrix parameters from the TD3 training module, matches different weight coefficients at different braking stages, generates corresponding actions and states, and puts them back into the TD3 training module experience pool, repeating this cycle and continuously updating the Q matrix library. Finally, the model predictive controller can obtain the trained Q matrix parameters at different braking stages and generate a soothing braking signal.

[0095] Step 4: Call the trained Q matrix for optimization and solution to generate the most comfortable control sequence;

[0096] The optimal value solution is to optimize the vehicle state and change to ensure that the optimal control sequence can be output in different braking stages, thereby improving passenger comfort. The following is the specific optimal value solution principle:

[0097] Comfort objective function construction:

[0098] The model predictive control model mainly predicts the vehicle acceleration and its own pitch angle. The state quantities include acceleration, pitch angle, and pitch angle velocity.

[0099] The optimization solver can solve the optimal acceleration under the constraints according to the expected vehicle speed and comfort optimization function, and transmit the control signal to the controller.

[0100] According to the model predictive control principle, the comfort optimization function can be obtained as:

[0101]

[0102] Where Q is the system output weight system matrix, R is the control weight coefficient matrix, ρ is the weight coefficient, and ε is the relaxation factor. a ,q θ , They correspond to the acceleration weight coefficient, the vehicle pitch angle weight coefficient, and the vehicle pitch angular velocity weight coefficient respectively.

[0103] In order to obtain the optimal control sequence of the MPC controller, the minimum value min J corresponding to the comfort objective function solver, and prevent the control quantity from changing suddenly, the following constraints are added:

[0104] u min (k+m)≤u(k+m|k)≤u max (k+m)

[0105] Δu min (k+m)≤Δu(k+m|k)≤Δu max (k+m)

[0106] With reference to the expected vehicle speed, according to the output parameter Y of the acceleration prediction model, a series of optimal acceleration increments ΔU(t) under the system constraints can be solved. The first acceleration increment Δu(k|t) of the series is taken and added to the acceleration control amount at the previous moment to obtain the current acceleration control amount u.

[0107] Step 5: The control sequence is transmitted to the vehicle control module for signal conversion, generating braking torque and executing the operation, so that the vehicle pitch angle is within a comfortable range throughout the whole process.

[0108] After receiving the control signal, the vehicle control module performs the corresponding braking action. If the data acquisition module determines that the vehicle is in an emergency, the vehicle control module will perform emergency braking. If the vehicle control module receives the braking instruction from the MPC optimization solution module, it will generate the corresponding braking control amount and perform comfortable braking. Then, the vehicle and surrounding data are obtained in real time through the on-board sensor kit, and the data is passed to the scene recognition module. This cycle repeats to continuously achieve comfortable braking.

[0109] The present invention is not limited to the above-mentioned specific implementation modes. A person skilled in the art may implement the present invention in various other specific implementation modes according to the embodiments and the disclosure of the drawings. Therefore, any design that adopts the design structure and concept of the present invention and makes some simple transformations or changes shall fall within the scope of protection of the present invention.

Claims

1. A braking control method based on the comfort of an autonomous driving vehicle, characterized in that: The braking control method based on the comfort of the autonomous driving vehicle comprises the following steps: Step 1: Collect the speed and distance information of the obstacle ahead, analyze the collected road information and vehicle information, and then comprehensively judge whether it is an emergency condition based on the current speed of the vehicle, the speed of the vehicle ahead, and the distance information to the obstacle; Step 2: Learn the Q matrix in model predictive control through the TD3 deep reinforcement learning algorithm; Step 3: The model predictive control model estimates the current braking stage according to the information input data, and calls the Q matrix that best suits the current stage of comfort from the strategy library for optimization and solution; Step 4: Call the trained Q matrix for optimization and solution to generate the most comfortable control sequence; Step 5: The control sequence is transmitted to the vehicle control module for signal conversion to generate a braking torque so that the vehicle pitch angle is within a comfortable range throughout the entire process.

2. The braking control method based on the comfort of the autonomous driving vehicle according to claim 1 is characterized in that: In step 2, the following steps are also included: Step 21, constructing a TD3 training module; Step 22, continuously adjust the hyperparameters during the training process, and observe the control effects of acceleration, pitch angle, and pitch angular velocity to achieve the most matching reward weight coefficient; Step 23, use the deep reinforcement learning algorithm TD3 to train the Q weight coefficients of acceleration, pitch angle, and pitch angular velocity in the early, middle, and late stages of braking, evaluate the reward evaluation effects of different weight coefficients, and put the trained Q weight coefficients back into the strategy library.

3. The braking control method based on the comfort of the autonomous driving vehicle according to claim 2 is characterized in that: In step 21, in order to ensure that the vehicle can brake smoothly, the reward function of the TD3 training module is set to a function related to vehicle comfort: r a =-γ1·|a-a T | r θ =-γ2·|θ-θ T | r a , r θ , They are acceleration a, pitch angle θ, and pitch angular velocity. The reward is γ1, γ2, γ3, and γ4, respectively. T represents the target acceleration, θ T represents the target pitch angle, Indicates the target pitch angular velocity; r 总 The expression of is the sum of the three: Among them, w1, w2, and w3 are the weight coefficients of the acceleration reward stage, the pitch angle reward stage, and the pitch angular velocity stage respectively; The training is carried out in stages. The braking stage is mainly divided into the early, middle and late stages. In the early and middle stages of vehicle braking, MPC a It has a greater weight in this stage, followed by pitch angle, and finally pitch angle angular velocity. The reward expression in the early and middle stages is: w 11 Indicates the acceleration weight coefficient in the early and middle stages of braking, w 21 represents the pitch angle weight coefficient in the early and middle stages of braking, w 31 Indicates the weight coefficient of pitch velocity in the middle stage before braking. At the end of vehicle braking, MPC In this stage, the weight should be large, so the pitch velocity is the main target in the late braking stage, followed by the pitch angle, and finally the acceleration; the reward expression in the late stage is: w 21 represents the acceleration weight coefficient in the late braking stage, w 22 represents the pitch angle weight coefficient in the late braking stage, w 32 Indicates the pitch velocity weight coefficient in the late braking stage.

4. The braking control method based on the comfort of the autonomous driving vehicle according to claim 3 is characterized in that: In step 21, the target network update mode is soft update, and the soft update expression is: i target ←τ·θ+(1-τ)·θ target τ is the soft update coefficient, θ represents the current network parameters, and θ target represents the target network parameters; The expression of the objective function is as follows: is the objective function J with respect to the policy network parameter θ μ The gradient of is the gradient of the Q function with respect to the action, is the action gradient of the policy network, i is the sample number, and N is the total number of samples; The parameter θ of the Q-value function is optimized by minimizing the mean square error between the predicted value and the target value. Q , so the expression of the loss function is: Q(s i , a i |θ Q ) represents the predicted value, y i Indicates the target value.

5. The braking control method based on the comfort of the autonomous driving vehicle according to claim 1, characterized in that: In step 3, the model predictive controller continuously obtains Q matrix parameters from the TD3 training module, matches different weight coefficients in different braking stages, generates corresponding actions and states, and puts them back into the TD3 training module experience pool, repeating this cycle and continuously updating the Q matrix library.

6. The braking control method based on the comfort of the autonomous driving vehicle according to claim 1, characterized in that: In step 4, according to the model predictive control principle, the comfort optimization function can be obtained as: Where J represents the comfort optimization function, m represents the sampling step, m = 1:1:N P , N P represents the prediction step length, y(k+m|t) represents the control output prediction value, and y ref (k+m|t) represents the control output reference value, N C represents the control step length, Δu(k+m|t) represents the control input increment, Q is the system output weight system matrix, R is the control weight coefficient matrix, ρ is the weight coefficient, and ε is the relaxation factor; q a is the acceleration weight coefficient, q θ is the vehicle pitch angle weight coefficient, is the vehicle pitch angular velocity weight coefficient; The minimum value min J corresponding to the comfort objective function solver and the prevention of sudden changes in the control quantity are added, so the following constraints are added: u min (k+m)≤u(k+m|k)≤u max (k+m) Δu min (k+m)≤Δu(k+m|k)≤Δu max (f+m) u min Indicates the minimum value of the control input, u max Indicates the maximum value of the control input, Δu max Indicates the maximum value of the control increment, Δu min Indicates the minimum value of the control increment; With reference to the expected vehicle speed, according to the output parameter Y of the acceleration prediction model, a series of optimal acceleration increments ΔU(t) under the system constraints can be solved. The first acceleration increment Δu(k|t) of the series is taken and added to the acceleration control amount at the previous moment to obtain the current acceleration control amount u.

Citation Information

Patent Citations

  • Electric vehicle economical self-adaptive cruise control method and system based on reinforcement learning

    CN114771520A

  • Train emergency parking method based on small sample data enhanced ensemble learning

    CN114852129A

  • Intelligent network connection new energy automobile integrated energy efficiency optimization control method and system

    CN116203839A

  • Training system and method of action strategy model

    CN118133006A

  • TD3 unmanned driving decision-making method based on multi-experience playback pool

    CN118545093A

Cited By

  • A cooperative braking method and device for automatic driving vehicle lane changing

    CN122808667A