An intelligent driving optimization control method integrating global optimization and safety protection
By combining deep reinforcement learning and model prediction control, intelligent driving vehicles achieve safe and efficient driving in complex traffic scenarios, solving the problem of insufficient local optimization and robustness, and improving the generalization and robustness of control strategies.
Patent Information
- Application Number
- CN202410914038.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-09
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2044-07-09
AI Technical Summary
It is difficult for intelligent driving vehicles to achieve safe and efficient driving in complex traffic scenarios. The existing methods have local optimal problems, insufficient control accuracy and low robustness, especially in the face of emergencies.
Using a hierarchical structure to combine deep reinforcement learning algorithms with model prediction and control methods, using the global optimization ability of reinforcement learning and the security of model prediction and control, we can make decisions on the upper reinforcement learning module and the lower model prediction and control module to realize intelligent driving. The vehicle is safe and efficient.
On the premise of ensuring safety protection, the exploration of intelligent driving control strategies has been increased, generalization and robustness have been improved, and local optimization has been avoided. It is suitable for intelligent vehicle collision avoidance and adaptive cruise control in complex scenarios. .
Smart Images
Figure CN118618404B_ABST
Abstract
Description
Technical field:
[0001] The present invention belongs to the field of intelligent driving, and specifically is an intelligent driving optimization control method integrating global optimization and safety protection. Technical background:
[0002] Traffic flows on urban roads and highways are important traffic scenarios that today's intelligent driving vehicles have to face. The large number and variety of traffic participants in these scenarios increase the complexity of traffic scenarios, resulting in huge challenges for intelligent driving. Intelligent driving mainly includes cruise control and collision avoidance functions, especially collision avoidance, which is the last line of defense to ensure vehicle driving safety. In order to improve traffic efficiency, vehicles need to cross traffic flows at the highest possible speed, but this process must ensure that no collisions occur.
[0003] The realization of intelligent driving functions requires the use of multiple sensors and cameras to obtain information about the road conditions in front of the vehicle. Based on the perceived information, the intelligent driving vehicle completes the cruising and collision avoidance tasks. The common implementation method is path planning and tracking control, but this method is less effective in dealing with emergencies on the road. In recent years, deep reinforcement learning has been increasingly widely used in intelligent driving. Reinforcement learning methods with global optimization characteristics can often learn a wealth of event samples, and when applied online, they can respond to complex emergencies.
[0004] Model predictive control is a common intelligent driving control method, which realizes intelligent driving functions by controlling intelligent vehicles to track path trajectories. Although the model predictive control method can handle constrained optimization problems and play a role in safety protection, its real-time performance is difficult to guarantee, and it is easy to fall into local optimality, and the control action is sometimes not guaranteed to be optimal. In addition, when facing complex working conditions and traffic scenarios, it is necessary not only to cope with the challenges of model establishment, but also to deal with many other problems, including frequent manual parameter adjustments, insufficient control accuracy, and low robustness of the model in practical applications.
[0005] Intelligent driving control strategies using reinforcement learning can directly map state inputs end-to-end to control actions such as acceleration, deceleration, and steering. The learning process of reinforcement learning methods is a global optimization process, so the global optimal strategy can be learned. However, the generalization ability is weak, and the performance is poor in untrained scenarios, making it difficult to apply in situations where safety is critical. Summary of the invention:
[0006] In view of the shortcomings of the prior art and in order to solve the problems existing in the above technical background, the present invention provides an intelligent driving optimization control method integrating global optimization and safety protection. The method adopts a hierarchical structure to combine the deep reinforcement learning algorithm with the model predictive control method, fully utilizes the global optimization ability of reinforcement learning, and uses model predictive control as a safety shield to achieve safe and efficient driving of intelligent driving vehicles in complex scenarios.
[0007] The technical solution adopted by the present invention to solve the technical problem is as follows:
[0008] The present invention is an intelligent driving optimization control method integrating global optimization and safety protection, which includes an upper layer reinforcement learning optimization reference sequence and a lower layer vehicle model predictive control. The upper layer uses deep learning and reinforcement learning methods to train an environment prediction module and a reinforcement learning module respectively. Deep learning uses the long short-term memory neural network method, and reinforcement learning uses the SAC method. The lower layer uses the MPC method and the long short-term memory neural network method to solve and convert the optimal control action, so that the vehicle can complete the intelligent driving task.
[0009] The method comprises the following steps:
[0010] Step 1: Build the driving environment:
[0011] The driving environment that intelligent driving vehicles have to deal with is complex and changeable, which is the main challenge facing today's intelligent driving. The factors that affect intelligent driving mainly include road conditions, weather conditions and traffic conditions. According to the above factors, the present invention divides the driving environment into four scene types, namely, cruise control scene, car-following cruise scene, emergency collision avoidance scene and lane change collision avoidance scene. The real driving environment is a combination and switching of the above four scenes. Therefore, intelligent driving vehicles decide on horizontal and vertical control actions by sensing the motion information of surrounding traffic participants on the road. When the control actions are executed by the vehicle, the intelligent driving function can be realized.
[0012] Step 2: Design reinforcement learning module:
[0013] Step 2.1, state space design:
[0014] The state is an important part of the Markov decision process, which represents the environmental feature information related to the training task. For intelligent driving tasks in the driving environment, the speed and relative distance between the ego vehicle and the surrounding vehicles in the driving environment can intuitively reflect the relative motion relationship between the ego vehicle and the surrounding vehicles. Using relative distance and speed as the state input of the reinforcement learning module is conducive to the agent's understanding of the driving environment. In addition, the vehicle state information of the ego vehicle is also used as the state input of the reinforcement learning module. Define the state of the reinforcement learning module as s t ={v1,d1,v2,d2,…,vi ,d i ,θ,x,y,v ego},v i is the speed of the vehicle, d i is the distance between the ego vehicle and the surrounding vehicles, i is the number of surrounding vehicles, i=1,…,n, θ is the heading angle of the ego vehicle, x and y are the horizontal and vertical position coordinates of the ego vehicle, respectively, v ego is the vehicle speed.
[0015] Step 2.2, action space design:
[0016] When completing the intelligent driving task in the four scenes of the driving environment in step 1, it is necessary to control the lateral and longitudinal motion of the intelligent driving vehicle. Compared with the traditional control method, the deep reinforcement learning method relies on the neural network to output multi-dimensional control actions end-to-end, which can achieve the coupling of lateral and longitudinal control. In order to complete the intelligent driving task of the vehicle, the present invention defines the action of the reinforcement learning module as a t =[throttle,brake,steer], throttle is the vehicle acceleration action, the interval is [0,1], brake is the vehicle braking action, the interval is [0,1], steer is the vehicle steering action, the interval is [-1,1].
[0017] Step 2.3, reward function design:
[0018] In the training process of reinforcement learning, the reward value plays an evaluation role in the control action selected by the agent, thereby guiding the agent to learn a reasonable control strategy. According to the division of the driving environment in step 1, for the emergency collision avoidance scenario, the present invention defines the reward function of the intelligent driving task in this scenario as formula (1), for the lane change collision avoidance scenario, the present invention defines the reward function of the intelligent driving task in this scenario as formula (2), for the cruise control scenario, the present invention defines the reward function of the intelligent driving task in this scenario as formula (3), for the following cruise scenario, the present invention defines the reward function of the intelligent driving task in this scenario as formula (4),
[0019]
[0020]
[0021]
[0022]
[0023] d front is the longitudinal distance between the ego vehicle and the vehicle in front, △v is the relative speed between the ego vehicle and the vehicle in front, l is the lateral distance between the ego vehicle and the target lane, v ideal is the ideal vehicle speed during cruise control, dideal is the ideal following distance during cruising, g k is a constant, k=1,…,18.
[0024] Step 2.4, experience playback:
[0025] In the deep reinforcement learning method, the experience replay mechanism is an important guarantee for accelerating the training process and achieving algorithm convergence. Experience replay allows the training of the intelligent agent to use historical experience multiple times, thereby improving the utilization rate of experience samples. The present invention defines the experience pool as D, saves the training data at each moment into the experience pool, and when the number of experiences in the experience pool reaches 4b, randomly extracts b experiences from the experience pool to form a batch for training.
[0026] Step 2.5. Select reinforcement learning algorithm:
[0027] Different reinforcement learning methods have their own advantages and disadvantages. The present invention selects the SAC algorithm as the reinforcement learning algorithm for intelligent vehicles to complete the intelligent driving task in the driving environment in step 1. The SAC algorithm can select actions in a continuous action space. Due to the introduction of maximum entropy, the strategy learned by the intelligent agent is more random, which helps to improve the generalization of the control strategy.
[0028] Step 2.6, reinforcement learning module training:
[0029] Initialize the network parameters of each neural network in the SAC algorithm ω1,ω2, is the Actor network parameter, ω1 is the Critic1 network parameter, ω2 is the Critic2 network parameter, is the target Critic1 network parameter, is the target Critic2 network parameter. In the first round of training, the environmental information obtained from the driving environment in step 1 constitutes the state s at time t in step 2.1 t Status t Input to the Actor network, and output the action a in the action space range of step 2.2 through global optimization t Action a t Executed into the driving environment, the reward function in step 2.3 calculates the reward value r at time t t , and get the state s at time t+1 t+1 . Will (s t ,a t ,r t ,s t+1) is stored in the experience pool D in step 2.4. When the number of experiences in the experience pool reaches 4b, b experiences are randomly taken out of the experience pool. The experiences are input into the Critic1 network, the Critic2 network, the target Critic1 network and the target Critic2 network to obtain the Q value and target Q value Use formula (5) to calculate the time difference target,
[0030]
[0031]
[0032]
[0033] γ is the discount factor. By minimizing the loss functions in equations (6) and (7), the Critic1 network and the Critic2 network are updated.
[0034] Sampling actions using reparameterization
[0035]
[0036] The loss function in equation (8) is used to update the Actor network.
[0037] α is the entropy regularization coefficient. The target Critic1 network and the target Critic2 network are updated using equations (9) and (10).
[0038]
[0039]
[0040] τ is a soft update coefficient. The above training process is iterated continuously until the algorithm converges. After the algorithm converges, the optimal set of network parameters is selected and loaded into the Actor network to complete the training of the reinforcement learning module of the present invention.
[0041] Step 3: Design the environment prediction module:
[0042] This step is used to design an environment prediction module, which is used for the equivalent driving environment so as to interact with the reinforcement learning module to obtain the reference sequence. In order to maximize the equivalent driving environment, this module uses a large number of experience samples obtained during the SAC algorithm training in step 2.6 for deep learning training. Deep learning uses the long short-term memory neural network method. After training and learning, the environment prediction module can be equivalent to the driving environment and predict the state at the next moment based on the current state and action. Specifically, the reinforcement learning module is based on the state s at time t. t Decision to take action a tThen the environment prediction module predicts the environment state at the next moment and obtains the predicted state for the next step. The reinforcement learning module then predicts the state Decision action, this process repeats N times P Step, N P It is the prediction time domain of the MPC control module. P The predicted state of the step is used as the reference sequence of the MPC control module Y t RL = {v t ,θ t} is the reference state at time t, v t and θ t are the reference speed and reference heading angle of the vehicle at time t.
[0043] Step 4: Design the MPC control module:
[0044] Step 4.1: Establish a prediction model:
[0045] The two-degree-of-freedom vehicle model is used as the vehicle model, as shown in equations (11) and (12),
[0046]
[0047]
[0048] Among them, the state quantity is x t =[x,y,θ] Τ , the control quantity is u t =[v,δ] Τ δ is the front wheel turning angle of the vehicle, β is the side slip angle of the center of mass of the vehicle, l r is the distance between the vehicle's center of mass and the rear axle.
[0049] Step 4.2, determine the optimization goal:
[0050] After establishing the ego-vehicle prediction model, the driving comfort and control action stability are taken into consideration in the optimization objective, and hard constraints are imposed on the control action. The optimization objective of the model predictive control is designed as shown in formula (13):
[0051]
[0052] in, To predict the output sequence, U(t) = {u t ,u t+1 ,…,u t+m-1} is the control action sequence, Γ y With Γ u is the weight parameter, m is the control time domain, u maxwith u min is the upper and lower limits of the control action, △u max and △u min To control the upper and lower limits of the action change. Substitute the above optimization objectives to solve and obtain the optimal control instruction u at the current moment opt =[v opt ,δ opt ] Τ .
[0053] Step 5: Design the action conversion module:
[0054] In this step, the long short-term memory neural network is used to perform deep learning training through a large number of calibration samples to obtain the action conversion module. The action conversion module converts the optimal control instruction u solved by the MPC control module in step 4.2 into opt =[v opt ,δ opt ] Τ It is converted into the vehicle's accelerator pedal opening, brake pedal opening and steering wheel angle to control the vehicle for intelligent driving.
[0055] The beneficial effect of the present invention is as follows: the present invention relates to an intelligent driving optimization control method integrating global optimization and safety protection, the method comprises two layers, and the upper and lower layers. By combining reinforcement learning with a model predictive control method, the exploration degree of the intelligent driving control strategy is maximized while ensuring safety protection, and the dilemma of local optimality is avoided. At the same time, the method improves the generalization and robustness of the intelligent driving control strategy, and has good application prospects in the fields of intelligent vehicle collision avoidance control and adaptive cruise control. Description of the drawings:
[0056] Figure 1 It is a schematic diagram of the hierarchical combination of reinforcement learning and model predictive control of the present invention.
[0057] Figure 2 It is a schematic diagram of an application scenario of the present invention. Specific implementation method:
[0058] The present invention is described in detail below with reference to the accompanying drawings.
[0059] The present invention proposes an intelligent driving optimization control method that integrates global optimization and safety protection. The upper layer uses deep learning and reinforcement learning methods to train the environment prediction module and reinforcement learning module respectively. The lower layer uses the MPC control module and deep learning methods to solve and convert the optimal control action so that the vehicle can complete the intelligent driving task. Deep learning uses the long short-term memory neural network method, and reinforcement learning uses the SAC method. Figure 1 The schematic diagram specifically includes the following steps:
[0060] Step 1: Build the driving environment:
[0061] The driving environment that smart driving vehicles have to deal with is complex and changeable, which is the main challenge facing smart driving today. The factors that affect smart driving mainly include road conditions, weather conditions, and traffic conditions. The present invention divides the driving environment into four scene types based on the above factors, namely, cruise control scene, car-following cruise scene, emergency collision avoidance scene, and lane change collision avoidance scene. The actual driving environment is a combination and switching of the above four scenes. Therefore, smart driving vehicles decide on lateral and longitudinal control actions by sensing the motion information of surrounding traffic participants on the road. The control actions are executed by the vehicle to realize the smart driving function. The driving environment targeted by the present invention is such as Figure 2 shown.
[0062] Step 2: Design reinforcement learning module:
[0063] Step 2.1, state space design:
[0064] The state is an important part of the Markov decision process, which represents the environmental feature information related to the training task. For intelligent driving tasks in the driving environment, the speed and relative distance between the ego vehicle and the surrounding vehicles in the driving environment can intuitively reflect the relative motion relationship between the ego vehicle and the surrounding vehicles. Using relative distance and relative speed as the state input of the reinforcement learning module is conducive to the agent's understanding of the driving environment. In addition, the vehicle state information of the ego vehicle is also used as the state input of the reinforcement learning module. Therefore, the state of the reinforcement learning module is defined as s t ={v1,d1,v2,d2,…,v i ,d i ,θ,x,y,v ego}. i is the speed of the vehicle, d i is the distance between the ego vehicle and the surrounding vehicles, i is the number of surrounding vehicles, i=1,…,n, θ is the heading angle of the ego vehicle, x and y are the horizontal and vertical position coordinates of the ego vehicle, respectively, v ego is the vehicle speed.
[0065] Step 2.2, action space design:
[0066] When completing the intelligent driving task in the four scenes in the driving environment in step 1, it is necessary to control the lateral and longitudinal motion of the intelligent driving vehicle. Compared with the traditional control method, the deep reinforcement learning method relies on the neural network to output multi-dimensional control actions end-to-end, which can achieve the coupling of lateral and longitudinal control. In order to complete the intelligent driving task of the vehicle, the present invention defines the action of the reinforcement learning module as a t=[throttle,brake,steer], throttle is the vehicle acceleration action, the interval is [0,1], brake is the vehicle braking action, the interval is [0,1], steer is the vehicle steering action, the interval is [-1,1].
[0067] Step 2.3, reward function design:
[0068] In the training process of reinforcement learning, the reward value plays an evaluation role in the control action selected by the agent, thereby guiding the agent to learn a reasonable control strategy. According to the division of the driving environment in step 1, for the emergency collision avoidance scenario, the present invention defines the reward function of the intelligent driving task in this scenario as formula (14), for the lane change collision avoidance scenario, the present invention defines the reward function of the intelligent driving task in this scenario as formula (15), for the cruise control scenario, the present invention defines the reward function of the intelligent driving task in this scenario as formula (16), for the following cruise scenario, the present invention defines the reward function of the intelligent driving task in this scenario as formula (17),
[0069]
[0070]
[0071]
[0072]
[0073] d front is the longitudinal distance between the ego vehicle and the vehicle in front, △v is the relative speed between the ego vehicle and the vehicle in front, l is the lateral distance between the ego vehicle and the target lane, v ideal is the ideal vehicle speed during cruise control, d ideal The ideal following distance when cruising.
[0074] Step 2.4, experience playback:
[0075] In the deep reinforcement learning method, the experience replay mechanism is an important guarantee for accelerating the training process and achieving algorithm convergence. Experience replay allows the training of the intelligent agent to use historical experience multiple times, thereby improving the utilization rate of experience samples. The present invention defines the experience pool as D, and saves the training data at each moment into the experience pool. When the number of experiences in the experience pool reaches 4b, b experiences are randomly selected from the experience pool to form a batch for training. Specifically, at the Tth moment of the Ith round of training, b experiences are randomly selected from the experience pool to form an experience sample batch for training. In the present invention, b is 64.
[0076] Step 2.5. Select reinforcement learning algorithm:
[0077] Different reinforcement learning methods have their own advantages and disadvantages. The present invention selects the SAC algorithm as the reinforcement learning algorithm for intelligent vehicles to complete the intelligent driving task in the driving environment in step 1. The SAC algorithm can select actions in a continuous action space. Due to the introduction of maximum entropy, the strategy learned by the intelligent agent is more random, which helps to improve the generalization of the control strategy.
[0078] Step 2.6, reinforcement learning module training:
[0079] α is set to -2, τ is 0.005, and γ is 0.99 to initialize the network parameters of each neural network in the SAC algorithm. ω1,ω2, is the Actor network parameter, ω1 is the Critic1 network parameter, ω2 is the Critic2 network parameter, is the target Critic1 network parameter, is the target Critic2 network parameter. In the first round of training, the environmental information obtained from the driving environment in step 1 constitutes the state s at time t in step 2.1 t Status t Input to the Actor network, and output the action a in the action space range of step 2.2 through global optimization t Action a t Executed into the driving environment, the reward function in step 2.3 calculates the reward value r at time t t , and get the state s at time t+1 t+1 . Will (s t ,a t ,r t ,s t+1 ) is stored in the experience pool D in step 2.4. When the number of experiences in the experience pool reaches 256, 64 experiences are randomly taken out of the experience pool. The experience is input into the Critic1 network, the Critic2 network, the target Critic1 network and the target Critic2 network to obtain the Q value and target Q value Use formula (18) to calculate the time series difference target,
[0080]
[0081]
[0082]
[0083] By minimizing the loss functions in equations (19) and (20), the Critic1 network and the Critic2 network are updated.
[0084] Sampling actions using reparameterization
[0085]
[0086] The Actor network is updated using the loss function in equation (21).
[0087] The target Critic1 network and the target Critic2 network are updated using equations (22) and (23).
[0088]
[0089]
[0090] The above training process is continuously iterated until the algorithm converges, and the optimal set of network parameters after the algorithm converges is selected and loaded into the Actor network to complete the training of the reinforcement learning module of the present invention.
[0091] Step 3: Design the environment prediction module:
[0092] This step is used to design an environment prediction module, which is used for the equivalent driving environment so as to interact with the reinforcement learning module to obtain the reference sequence. In order to maximize the equivalent driving environment, this module uses a large number of experience samples obtained during the SAC algorithm training in step 2.6 for deep learning training. Deep learning uses the long short-term memory neural network method. After training and learning, the environment prediction module can be equivalent to the driving environment and predict the state at the next moment based on the current state and action. Specifically, the reinforcement learning module is based on the state s at time t. t Decision to take action a t Then the environment prediction module predicts the environment state at the next moment and obtains the predicted state for the next step. The reinforcement learning module then predicts the state Decision action, the process repeats 15 steps, 15 is the prediction time domain of the MPC control module. The predicted state of the vehicle in the next 15 steps is obtained as the reference sequence of the MPC control module Y t RL = {v t ,θ t} is the reference state at time t, v t and θ t are the reference speed and reference heading angle of the vehicle at time t.
[0093] Step 4: Design the MPC control module:
[0094] Step 4.1: Establish a prediction model:
[0095] The two-degree-of-freedom vehicle model is used as the prediction model, as shown in equations (24) and (25),
[0096]
[0097]
[0098] Among them, the state quantity is x t =[x,y,θ] Τ , the control quantity is u t =[v,δ] Τ δ is the front wheel turning angle of the vehicle, β is the side slip angle of the center of mass of the vehicle, l r is the distance between the vehicle's center of mass and the rear axle.
[0099] Step 4.2, determine the optimization goal:
[0100] After establishing the ego-vehicle prediction model, the driving comfort and control action stability are taken into consideration in the optimization objective, and hard constraints are imposed on the control action. The optimization objective of the MPC control is designed as shown in Equation (26):
[0101]
[0102] Where Y = {Y t ,Y t+1 ,…,Y t+14} is the predicted output sequence, U(t)={u t ,u t+1 ,…,u t+9} is the control action sequence, Γ y With Γ u is the weight parameter, the control time domain is 10, u max with u min is the upper and lower limits of the control action, △u max and △u min To control the upper and lower limits of the action change. Substitute the above optimization objectives to solve and obtain the optimal control instruction u at the current moment opt =[v opt ,δ opt ] Τ .
[0103] Step 5: Design the action conversion module:
[0104] In this step, the long short-term memory neural network is used to perform deep learning training through a large number of calibration samples to obtain the action conversion module. The action conversion module converts the optimal control instruction u solved by the MPC control module in step 4.2 into opt =[v opt ,δ opt ]Τ It is converted into the vehicle's accelerator pedal opening, brake pedal opening and steering wheel angle to control the vehicle for intelligent driving.
[0105] To sum up: the present invention proposes an intelligent driving optimization control method that integrates global optimization and safety protection. Under the premise of ensuring safe driving, it can significantly increase the exploration degree of the intelligent driving control strategy, avoid falling into the dilemma of local optimality, improve the generalization and robustness of the intelligent driving control strategy, and realize safe and efficient driving of intelligent driving vehicles in complex scenarios. It has good application prospects in the fields of intelligent vehicle collision avoidance control and adaptive cruise control.
Claims
1. An intelligent driving optimization control method integrating global optimization and safety protection, Features: The method includes two layers, the upper layer determines the heading angle and speed reference sequence through reinforcement learning global optimization and inputs it to the lower layer, and the lower layer determines the control action through optimization solution to control the vehicle to realize intelligent driving; wherein, the upper layer includes the driving environment, reinforcement learning module, and environment prediction module, and the lower layer includes the MPC control module and the action conversion module; The state space of the reinforcement learning module in the upper layer is defined as s t = {v 1 ,d 1 ,v 2 ,d 2 ,…,v i ,d i ,θ,x,y,v ego };v i is the speed of the vehicle, d i is the distance between the ego vehicle and the surrounding vehicles, i is the number of surrounding vehicles, i=1,…,n, θ is the heading angle of the ego vehicle, x and y are the horizontal and vertical position coordinates of the ego vehicle, respectively, v ego is the speed of the vehicle; the action space of the reinforcement learning module is defined as a t =[throttle,brake,steer], throttle is the vehicle acceleration action, brake is the vehicle braking action, and steer is the vehicle steering action; the reinforcement learning module is trained by interacting with the driving environment. For different driving environments, the present invention designs different reward functions; for emergency collision avoidance scenarios, the reward function of the reinforcement learning module is defined as formula (1); for lane change collision avoidance scenarios, the reward function of the reinforcement learning module is defined as formula (2); for constant speed cruise scenarios, the reward function of the reinforcement learning module is defined as formula (3); for vehicle following cruise scenarios, the reward function of the reinforcement learning module is defined as formula (4); Where, d front is the longitudinal distance between the ego vehicle and the vehicle in front, △v is the relative speed between the ego vehicle and the vehicle in front, l is the lateral distance between the ego vehicle and the target lane, v ideal is the ideal vehicle speed during cruise control, d ideal is the ideal following distance during cruising, g k is a constant, k = 1,…,18; The environment prediction module is used for the equivalent driving environment. After training and learning, this module can predict the state at the next moment based on the state and action at the current moment. Specifically, the reinforcement learning module predicts the state at the next moment based on the state s at the moment t. t Decision to take action a t Then the environment prediction module predicts the environment state at the next moment and obtains the predicted state for the next step. The reinforcement learning module then predicts the state Decision action, this process repeats N times P Step, N P The prediction time domain of the MPC control module; obtain the vehicle's N future P The predicted state of the step is used as the reference sequence of the MPC control module Y t RL = {v t ,θ t } is the reference state at time t, v t and θ t are the reference speed and reference heading angle of the vehicle at time t; The MPC control module in the lower layer optimizes the optimal control instruction using the heading angle and speed reference sequence provided by the upper layer; specifically, the MPC control module includes a prediction model and an optimization target. The prediction model is established using a two-degree-of-freedom vehicle model, as shown in equations (5) and (6). Among them, the state quantity is x t =[x,y,θ] Τ , the control quantity is u t =[v,δ] Τ ; δ is the front wheel turning angle of the vehicle, β is the side slip angle of the vehicle's center of mass, l r is the distance between the vehicle's center of mass and the rear axle; The optimization objective takes into account the ride comfort and control action stability, imposes hard constraints on the control action, and defines the optimization objective of the model predictive control as shown in formula (7): in, To predict the output sequence, U(t) = {u t ,u t+1 ,…,u t+m-1 } is the control action sequence, Γ y With Γ u is the weight parameter, m is the control time domain, u max with u min is the upper and lower limits of the control action, △u max and △u min To control the upper and lower limits of the action change; the reference sequence obtained from the upper layer Substitute the above optimization objectives to solve and obtain the optimal control instruction u at the current moment opt =[v opt ,δ opt ] Τ ; Finally, the action conversion module converts the control instructions optimized and solved by the MPC control module into control actions of the intelligent driving vehicle in the driving environment, and controls the vehicle to achieve intelligent driving.
2. According to claim 1, an intelligent driving optimization control method integrating global optimization and safety protection, Features: The driving environment is divided into four scene types, namely, a cruise control scene, a vehicle-following cruise scene, an emergency collision avoidance scene, and a lane change collision avoidance scene.
3. According to claim 1, an intelligent driving optimization control method integrating global optimization and safety protection, Features: The action conversion module converts the optimal control instruction u solved by the MPC control module opt =[v opt ,δ opt ] Τ It is converted into the vehicle's accelerator pedal opening, brake pedal opening and steering wheel angle to control the vehicle to achieve intelligent driving.