Intelligent driving path planning method and system based on deep reinforcement learning

Abstract

The application relates to an intelligent driving path planning method and system based on deep reinforcement learning, which comprises the following steps: constructing a multi-dimensional state space containing acceleration guide quantity under a Frenet coordinate system; inputting the state space into an improved TD3 deep reinforcement learning network, outputting path decision actions by an actor policy network; calculating a safety half shaft by an elliptical vehicle body contour model and establishing a minimum safety distance model, and performing safety correction on the actions in combination with an AM mechanism; taking the discrete path points after correction as references, constructing a segmented quintic polynomial quadratic programming problem, and solving the optimal trajectory meeting continuity and kinematic constraints; obtaining the states of cooperative vehicles through V2V communication, and performing multi-vehicle cooperative path adjustment based on a constant time interval strategy; dynamically adjusting the front wheel steering angle output by an adaptive MPC lateral controller, and online optimizing the parameter output of the longitudinal acceleration by a fuzzy PID longitudinal controller, so that the vehicle can track the target trajectory.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to an intelligent driving path planning method and system based on deep reinforcement learning. Background Technology

[0002] With the rapid development of intelligent driving technology, path planning directly determines the driving safety, traffic efficiency, and passenger comfort of autonomous driving systems in complex and dynamic traffic scenarios. In typical complex scenarios such as urban signalized intersections, highway overtaking, and multi-vehicle cooperative U-turns, intelligent driving vehicles not only need to cope with strict road geometry constraints and dynamically changing obstacle environments, but also need to achieve a balance between multiple objectives such as safety, driving efficiency, energy economy, and passenger comfort.

[0003] Currently, path planning methods for autonomous driving typically employ deep deterministic policy gradient algorithms, such as deep double-Q networks and deep deterministic policy gradient algorithms. These methods use a fixed cruising speed as the training objective for the agent. The state space generally only contains basic environmental information such as the vehicle's speed, position, distance and speed difference with the vehicle in front, and traffic light status. The reward function is usually a simple weighted sum of two parts: a following safety reward and a cruising speed following reward. During training, the agent is guided to converge towards a preset fixed cruising speed, outputting longitudinal acceleration control commands, which, in conjunction with an independent lateral controller, achieve longitudinal speed adjustment and lateral trajectory following. This approach achieves a certain level of control effectiveness in relatively simple single-lane following scenarios, and the network structure is relatively simple with fast training and convergence speeds.

[0004] The aforementioned technology uses a fixed cruising speed as the training target, forcing the agent to converge towards the preset speed. It is unable to autonomously discover a better driving strategy based on the actual traffic environment, resulting in insufficient global optimality and poor performance in scenarios that require flexible speed adjustment.

[0005] Based on this, this application provides an intelligent driving path planning method and system based on deep reinforcement learning. Summary of the Invention

[0006] To address the issue that when a fixed cruising speed is used as the training target, the agent is forced to converge to a preset speed, making it unable to autonomously discover better driving strategies based on the actual traffic environment, resulting in insufficient global optimality and poor performance in scenarios requiring flexible speed adjustment, this application provides an intelligent driving path planning method and system based on deep reinforcement learning.

[0007] Firstly, this application provides an intelligent driving path planning method based on deep reinforcement learning, which adopts the following technical solution: including: The system receives environmental perception information, decouples vehicle motion in the Frenet coordinate system, calculates acceleration guidance, and constructs a multi-dimensional state space containing the acceleration guidance; the acceleration guidance is the ratio of human-like driving reference acceleration to the maximum limit value of output acceleration. The multidimensional state space is input into the improved TD3 deep reinforcement learning network. Its Actor policy network outputs path decision actions, and the dual Critic network takes the minimum value to suppress value overestimation. The network adopts a free speed training mode, and the reward function is composed of a weighted sum of safety, driving efficiency, energy economy, comfort, and artificial potential field obstacle avoidance penalty. An elliptical vehicle body profile model is used to dynamically calculate the longitudinal and lateral safety half-axis, and a minimum safety distance model is established. Through the Action Masking mechanism, when the acceleration of the path decision action exceeds the safety reference acceleration, the safety reference acceleration is used instead to obtain the corrected path decision action. Using the discrete path points in the modified path decision action as a reference, a piecewise fifth-order polynomial quadratic programming problem is constructed, and the optimal trajectory that satisfies the constraints of continuity, road boundary and kinematic curvature is obtained by solving it. The vehicle obtains the status of the cooperating vehicles through V2V communication, and adjusts the aiming distance or desired speed based on the constant time-distance strategy and spacing error; the lead vehicle calculates the desired speed correction based on the number of vehicles in the queue behind and the remaining time of the green light, and obtains the target trajectory after the cooperative adjustment. The adaptive MPC lateral controller dynamically adjusts the predicted time-domain output front wheel steering angle, and the fuzzy PID longitudinal controller optimizes the parameters online to output longitudinal acceleration, driving the vehicle to follow the target trajectory.

[0008] Preferably, the step of inputting the multidimensional state space into the improved TD3 deep reinforcement learning network, where the Actor policy network outputs path decision actions and the dual Critic network takes the minimum value to suppress value overestimation, includes: The improved TD3 deep reinforcement learning network's Actor policy network takes the multi-dimensional state space as input and outputs the path decision action, which includes a lateral offset sequence of multiple path points uniformly distributed along the reference path and a longitudinal desired velocity. The dual-Critic network comprises a first Critic network and a second Critic network with identical network structures but different initial parameters. They estimate the action value of the path decision action in the current state, and the minimum of the two values ​​is used to calculate the temporal difference error target value to suppress the overestimation of action value by the single Critic network. The temporal difference error target value is jointly determined by the immediate reward at the current step size, the reward discount factor, the next action output by the target Actor network, and the value estimate of the next action by the target Critic network. The free speed training mode refers to a mode where driving efficiency reward components are used as positive drivers and energy consumption economy reward components are used as negative constraints. During the training process, the Actor policy network autonomously optimizes the speed curve instead of following a preset fixed cruising speed. The reward function includes: a safety reward component composed of a penalty for the difference between the current safety reference acceleration and the actual acceleration, and a penalty for the difference between the actual arc length difference between the vehicle and the adjacent vehicle and the preset minimum stationary safety distance. The safety reference acceleration is calculated by the intelligent driving following model. The driving efficiency reward component is the arc length distance traveled by the vehicle along the reference path within the current control step. The energy economy reward component is the negative value of the instantaneous fuel consumption logarithm calculated by the VT-Micro fuel consumption model multiplied by the fuel consumption penalty scaling factor. The comfort reward component is the negative value of the acceleration penalty coefficient multiplied by the absolute value of the longitudinal acceleration. The artificial potential field obstacle avoidance penalty component includes road boundary penalty and obstacle penalty. The road boundary penalty is determined by multiplying the road boundary penalty coefficient by the road boundary potential field function value. A penalty is generated when the vehicle deviates laterally and approaches the road boundary. The obstacle penalty is determined by multiplying the obstacle penalty coefficient by the sum of the two-dimensional Gaussian potential field function values ​​of all surrounding obstacles. The longitudinal and lateral standard deviations of the two-dimensional Gaussian potential field function are dynamically determined according to the length and width dimensions of each obstacle. The weight coefficients of each component in the reward function are determined using the analytic hierarchy process (AHP). A judgment matrix is ​​constructed with safety as the highest priority. The initial weight vector is calculated using the eigenvector method. After the consistency ratio is confirmed to be less than a preset threshold by a consistency check, simulation calibration is performed to obtain the final weight ratio.

[0009] Preferably, the longitudinal and lateral safety half-axles are dynamically calculated using an elliptical vehicle body contour model to establish a minimum safety distance model; through an Action Masking mechanism, when the acceleration of the path decision action exceeds the safety reference acceleration, it is replaced by the safety reference acceleration to obtain the corrected path decision action, including: The elliptical vehicle body profile model is centered on the vehicle's center of gravity. The longitudinal safety half-axis is composed of the sum of the vehicle's half-length, the distance traveled during the driver's braking reaction time, and the braking distance corresponding to the relative speed. The lateral safety half-axis is obtained by multiplying the vehicle's half-width by a preset expansion coefficient. The longitudinal safety half-axis dynamically increases with the increase of the vehicle's current speed and the relative speed between the vehicle and the obstacle, so that the elliptical vehicle body profile model adaptively represents the vehicle's safety range at different vehicle speeds. Based on the relative positional relationship between the vehicle and the obstacle, a first minimum safe distance model is established between the vehicle and the vehicle in front in the same lane, a second minimum safe distance model is established between the vehicle and the vehicle in front in the left lane, and a third minimum safe distance model is established between the vehicle and the vehicle behind in the left lane. Each of the minimum safe distance models uses the speed difference between the vehicle and the corresponding obstacle, the time of the corner collision, the longitudinal safety half-axis and the lateral safety half-axis as parameters to calculate the minimum safe distance required to avoid a corner collision; Within each control step, based on the Intelligent Driving Follow Model (IDM), the current safe reference acceleration is calculated according to the vehicle's current speed, the arc length difference between the vehicle and the adjacent vehicle, and the speed difference. The expected acceleration in the path decision action is compared with the safe reference acceleration. If the expected acceleration is greater than the safe reference acceleration, the expected acceleration is replaced by the safe reference acceleration as the expected acceleration in the corrected path decision action. At the same time, the state and action pair that triggers the safety limit is recorded in the experience pool of the improved TD3 deep reinforcement learning network, so that the Actor policy network learns to avoid generating high-risk actions that exceed the safety boundary in subsequent training.

[0010] Preferably, the step of constructing a piecewise fifth-order polynomial quadratic programming problem using discrete path points in the modified path decision action as references, and solving it to obtain the optimal trajectory that satisfies the constraints of continuity, road boundaries, and kinematic curvature, includes: In the corrected path decision action, the discrete path points are used as reference trajectory points. The entire trajectory is divided into M trajectory segments along the longitudinal arc length coordinate. Each trajectory segment is represented by a fifth-order polynomial with the longitudinal arc length coordinate as the independent variable, which represents the lateral offset within the segment. The fifth-order polynomial expression of the i-th trajectory segment consists of six polynomial coefficients. The optimization variable is a coefficient vector formed by concatenating the polynomial coefficients of all trajectory segments in sequence. The objective function is composed of a weighted sum of a reference trajectory fit term and a smoothness term. The reference trajectory fit term is the sum of squares of the differences between the lateral offset calculated by the polynomial at each sampling point within the trajectory segment and the lateral offset of the corresponding reference trajectory point. The smoothness term is the sum of squares of the second derivatives of the polynomial at each sampling point with respect to the longitudinal arc length coordinate. The priority between the reference trajectory fit term and the smoothness term is adjusted by a preset following accuracy weight matrix and a smoothness weight matrix. The constraints of the fifth-order polynomial expression include: positional continuity constraints where adjacent trajectory segments have equal polynomial values ​​at the connection point, tangent continuity constraints where the first derivative is equal, and curvature continuity constraints where the second derivative is equal; the road boundary constraints require that the sum of the lateral offset calculated by the polynomial at each sampling point and the lateral safety half-axis does not exceed half the road width. The kinematic curvature constraint requires that the absolute value of the trajectory curvature calculated by the polynomial at each sampling point does not exceed the maximum curvature threshold corresponding to the minimum turning radius of the vehicle; the starting state constraint requires that the polynomial value, first derivative, and second derivative at the starting point of the first trajectory segment are equal to the lateral offset of the vehicle's current position, the first derivative and second derivative of the lateral offset with respect to the arc length; the ending state constraint requires that the polynomial value and first derivative at the ending point of the last trajectory segment are equal to the planned terminal lateral offset and the first derivative of the lateral offset corresponding to the terminal direction angle. The quadratic programming problem is a convex optimization problem. It is solved by the interior point method or the effective set method to obtain the optimal solution of the coefficient vector. The polynomials of each trajectory segment are spliced ​​together according to the arc length to obtain the optimal trajectory from the current position of the vehicle to the planned endpoint, which satisfies the constraints of continuity, road boundary and kinematic curvature.

[0011] Preferably, the step of obtaining the status of the cooperating vehicles through V2V communication, adjusting the aiming distance or desired speed based on a constant time-distance strategy and spacing error; and the lead vehicle calculating the desired speed correction based on the number of vehicles in the queue behind and the remaining time of the green light to obtain the coordinated adjusted target trajectory includes: The system acquires real-time information on the arc length position, speed, and acceleration of surrounding cooperative vehicles in the Frenet coordinate system via V2V communication; adopts a constant time-distance cooperative control strategy, defining the desired following distance as the sum of the preset minimum stationary safety distance and the time distance to the front of the vehicle multiplied by the current speed of the vehicle itself; and defines the distance error as the difference between the arc length position of the adjacent vehicle and the arc length position of the vehicle itself, minus the desired following distance. When the vehicle is following another vehicle, the distance error is fed back to the pre-aiming distance. If the actual distance is greater than the expected following distance, the pre-aiming distance is reduced to enhance the following sensitivity, so that the vehicle accelerates to approach the vehicle in front. If the actual distance is less than the expected following distance, the pre-aiming distance is increased to smooth out the oscillation, so that the vehicle decelerates smoothly to increase the distance. When the vehicle is the lead vehicle, the initial expected speed is calculated, the remaining green light time at the current intersection and the number of vehicles in the queue behind the vehicle when it passes the target intersection are obtained, and the expected speed correction is calculated; the expected speed correction is calculated based on the road free-flow speed, the initial expected speed and the remaining green light time. During the current green light period at the intersection, the initial expected speed is added to the expected speed correction amount to obtain the corrected expected speed; when the green light at the target intersection ends, the corrected expected speed is reduced back to the initial expected speed according to a preset decay function; The target trajectory after collaborative adjustment is a complete spatiotemporal trajectory formed by using the corrected desired velocity as the longitudinal control target and the path geometry of the optimal trajectory.

[0012] Preferably, when the V2V communication has a fixed time delay, after acquiring the motion state information of surrounding cooperative vehicles, a first-order prediction compensation mechanism is used to correct the received lag state: if the following vehicle receives the lag arc length position, lag speed, and lag acceleration of the preceding vehicle at time t minus τ at time t, then the speed of the preceding vehicle at the current time is approximated by a first-order Taylor expansion, and the predicted value is the sum of the products of the lag speed and the lag acceleration multiplied by the communication delay τ; the predicted value is used to replace the original received lag speed and is substituted into the constant time-distance cooperative control strategy to calculate the expected following distance and the distance error; The headway is greater than twice the communication delay τ to satisfy the queue stability condition that the platoon spacing error does not amplify upstream along the vehicle queue in the frequency domain, thus ensuring that the target trajectory after coordinated adjustment remains stable under communication delay conditions.

[0013] Preferably, the adaptive MPC lateral controller dynamically adjusts the predicted time-domain output front wheel steering angle, and the fuzzy PID longitudinal controller optimizes the parameter output longitudinal acceleration online to drive the vehicle to follow the target trajectory, including: The adaptive MPC lateral controller uses the path geometry in the target trajectory after collaborative adjustment as the reference path and the lateral offset and heading angle of the vehicle in the Frenet coordinate system as the following state. The predictive model of the adaptive MPC lateral controller is established based on a three-degree-of-freedom vehicle dynamics model and a magic formula tire model. The lateral velocity, longitudinal velocity, yaw angle, yaw rate, global lateral position, and global longitudinal position of the vehicle are used as state variables, and the front wheel steering angle is used as the control variable. The mapping relationship between vehicle speed and optimal prediction time domain is fitted by a fifth-order polynomial. When the vehicle speed is lower than the preset low-speed threshold, the prediction time domain takes the first preset value. When the vehicle speed is higher than the preset high-speed threshold, the prediction time domain takes the second preset value, and the second preset value is greater than the first preset value. When the vehicle speed is between the two, the prediction time domain value is calculated by the fifth-order polynomial, so as to realize the adaptive adjustment of the prediction time domain in the full vehicle speed range. The objective function of the adaptive MPC lateral controller includes a lateral position error penalty term, a heading angle error penalty term, and a control increment penalty term. The constraints include front wheel steering angle magnitude constraint, front wheel steering angle change rate constraint, tire slip angle constraint, center of gravity slip angle constraint, and road adhesion condition constraint. In each control cycle, the objective constrained rolling optimization problem is solved, and the first element of the optimal control sequence is taken as the front wheel steering angle control command at the current moment. The fuzzy PID longitudinal controller takes the corrected expected speed in the target trajectory after collaborative adjustment as the following target, and takes the speed error and speed error change rate between the corrected expected speed and the actual speed of the vehicle as input. It performs fuzzy inference through a preset fuzzy rule base and outputs the proportional parameter correction amount, integral parameter correction amount and derivative parameter correction amount. These are superimposed with the preset initial proportional parameter, initial integral parameter and initial derivative parameter to obtain the optimized proportional parameter, integral parameter and derivative parameter. The initial control quantity is calculated based on the speed error using the optimized proportional, integral, and derivative parameters. When the initial control quantity is positive, it is mapped to a throttle opening command and the brake pressure is set to zero. When the initial control quantity is negative, it is mapped to a brake pressure command and the throttle opening is set to zero, thus obtaining the longitudinal acceleration control command. The vehicle is driven to follow the target trajectory after coordinated adjustment by the front wheel steering angle control command and the longitudinal acceleration control command.

[0014] Secondly, this application discloses an intelligent driving path planning device based on deep reinforcement learning, which adopts the following technical solution, including: The environmental perception module is used to receive environmental perception information, decouple vehicle motion in the Frenet coordinate system and calculate acceleration guidance amount, and construct a multi-dimensional state space containing the acceleration guidance amount; the acceleration guidance amount is the ratio of human-like driving reference acceleration to the maximum limit value of output acceleration. The path decision module is used to input the multi-dimensional state space into the improved TD3 deep reinforcement learning network. Its Actor policy network outputs path decision actions, and the dual Critic network takes the minimum value to suppress value overestimation. The network adopts a free speed training mode, and the reward function is composed of a weighted sum of safety, driving efficiency, energy economy, comfort, and artificial potential field obstacle avoidance penalty. The path correction module is used to dynamically calculate the longitudinal and lateral safety half-axis using an elliptical vehicle body contour model and establish a minimum safety distance model. Through the Action Masking mechanism, when the acceleration of the path decision action exceeds the safety reference acceleration, it is replaced by the safety reference acceleration to obtain the corrected path decision action. The trajectory generation module is used to construct a piecewise fifth-order polynomial quadratic programming problem with reference to the discrete path points in the modified path decision action, and solve it to obtain the optimal trajectory that satisfies the constraints of continuity, road boundary and kinematic curvature. The trajectory adjustment module is used to obtain the status of the cooperating vehicles through V2V communication and adjust the aiming distance or desired speed based on the constant time-distance strategy and spacing error; the lead vehicle calculates the desired speed correction based on the number of vehicles in the queue behind and the remaining time of the green light to obtain the target trajectory after the cooperative adjustment. The trajectory driving module is used to dynamically adjust the predicted time-domain output front wheel steering angle of the adaptive MPC lateral controller and optimize the output longitudinal acceleration of the fuzzy PID longitudinal controller online, driving the vehicle to follow the target trajectory.

[0015] Thirdly, this application also provides a control device, the device comprising: It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed, such as the deep reinforcement learning-based intelligent driving path planning method described above.

[0016] Fourthly, this application also provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executed as described above regarding the intelligent driving path planning method based on deep reinforcement learning.

[0017] In summary, this application constructs a multi-dimensional state space in the Frenet coordinate system, incorporating acceleration guidance, to integrate human-like driving prior knowledge into the agent's environmental perception, thereby improving the generalization ability to random traffic scenarios. The state space is input into an improved TD3 deep reinforcement learning network using a free-speed training mode. The Actor policy network outputs the path geometry and longitudinal velocity profile in parallel, while a dual Critic network minimizes value overestimation. A multi-objective weighted reward function, considering safety, driving efficiency, energy economy, comfort, and artificial potential field obstacle avoidance penalties, guides the agent to autonomously discover the globally optimal driving strategy in complex scenarios. A geometric safety boundary is constructed using an elliptical vehicle body contour model and a minimum safety distance model, combined with Action... The masking hard constraint mechanism performs safety verification and correction on the network output actions, ensuring path safety from both soft penalty and hard constraint levels. Using the corrected discrete path points as references, a piecewise fifth-order polynomial quadratic programming problem is constructed to solve for a smooth and kinematically feasible optimal trajectory under constraints of continuity, road boundaries, and kinematic curvature. Furthermore, the cooperative vehicle states are obtained through V2V communication, and multi-vehicle cooperative path adjustment is achieved based on a constant time-distance strategy and the lead vehicle's expected speed correction. Finally, an adaptive MPC lateral controller dynamically adjusts the predicted time-domain output front wheel angle, and a fuzzy PID longitudinal controller optimizes the output longitudinal acceleration online, driving the vehicle to accurately follow the target trajectory. This integrates the global policy exploration capability of deep reinforcement learning with the real-time adjustment capability of multi-vehicle cooperation, overcoming the limitations of existing methods in policy search space and lack of cooperative capabilities. It can generate optimal driving trajectories in complex dynamic scenarios such as urban intersections, highway overtaking, and multi-vehicle coordinated U-turns. Moreover, the network adopts a free-speed training mode, effectively solving the problem in existing methods where using a fixed cruising speed as the training target severely limits the agent's policy exploration space in complex scenarios. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an intelligent driving path planning method based on deep reinforcement learning.

[0019] Figure 2 This is a structural block diagram of an intelligent driving path planning device based on deep reinforcement learning. Detailed Implementation

[0020] The following combination Figures 1-2 This application will be described in further detail.

[0021] Reference Figure 1 The embodiments of this application include at least steps S10 to S60.

[0022] S10 receives environmental perception information, decouples vehicle motion in the Frenet coordinate system and calculates acceleration guidance quantity, and constructs a multi-dimensional state space containing acceleration guidance quantity; the acceleration guidance quantity is the ratio of human-like driving reference acceleration to the maximum limit value of output acceleration. S20 inputs a multi-dimensional state space into the improved TD3 deep reinforcement learning network. Its Actor policy network outputs path decision actions, and the dual Critic network takes the minimum value to suppress value overestimation. The network adopts a free speed training mode, and the reward function is composed of a weighted sum of safety, driving efficiency, energy economy, comfort, and artificial potential field obstacle avoidance penalties. S30 uses an elliptical body contour model to dynamically calculate the longitudinal and lateral safety half-axis and establish a minimum safety distance model; through the Action Masking mechanism, when the acceleration of the path decision action exceeds the safety reference acceleration, it is replaced by the safety reference acceleration to obtain the corrected path decision action; S40, taking the discrete path points in the corrected path decision action as a reference, construct a piecewise fifth-order polynomial quadratic programming problem, and solve it to obtain the optimal trajectory that satisfies the constraints of continuity, road boundary and kinematic curvature. The S50 obtains the status of the cooperating vehicles through V2V communication and adjusts the aiming distance or desired speed based on a constant time-distance strategy and spacing error; the lead vehicle calculates the desired speed correction based on the number of vehicles in the queue behind and the remaining time of the green light, thus obtaining the target trajectory after the cooperative adjustment. The S60 features an adaptive MPC lateral controller that dynamically adjusts the predicted time-domain output of the front wheel steering angle, and a fuzzy PID longitudinal controller that optimizes parameters online to output longitudinal acceleration, driving the vehicle to follow the target trajectory.

[0023] Specifically, this application provides an intelligent driving path planning method based on deep reinforcement learning, applicable to complex and dynamic traffic scenarios such as urban signalized intersections, highway overtaking, and multi-vehicle coordinated U-turns. The method consists of a complete processing flow, including state space construction, path decision generation, safety constraint verification, trajectory optimization, multi-vehicle cooperative path adjustment, and adaptive trajectory following. The specific implementation methods for each step are described in detail below: S10: State Space Construction Intelligent driving vehicles acquire surrounding environmental perception information through onboard sensors, V2I communication, and V2V communication, which includes three parts: the vehicle's motion status, such as position, speed, acceleration, and heading angle; the status of surrounding obstacles, such as the position, speed, acceleration, and geometric dimensions of other vehicles; and road traffic environment information, such as road geometric parameters, lane line positions, current traffic light phase, and signal timing scheme.

[0024] After obtaining the above information, the vehicle motion is decoupled in the Frenet coordinate system. Taking the road centerline or the current lane centerline as the reference path, the vehicle's motion in the two-dimensional plane is decomposed into a longitudinal arc length coordinate component s along the tangent of the reference path and a lateral offset component l perpendicular to the reference path. This separates the path geometry from the vehicle's longitudinal motion, reducing the coupling complexity of the subsequent optimization problem.

[0025] The acceleration guidance value is then calculated. The acceleration guidance value is defined as the ratio of the human-like driving reference acceleration calculated by the IDM car-following model at the current moment to the preset maximum limit of output acceleration. The calculation formula is as follows: ; In the formula, φ is the acceleration guide quantity, which is dimensionless and ranges from 0 to 1; The human-like driving reference acceleration is calculated by the IDM car-following model based on the current speed of the vehicle, the arc length difference between the vehicle and the adjacent vehicle in front, and the speed difference, and is expressed in meters per second squared. The preset maximum output acceleration is expressed in meters per second squared (m / s²), typically between three and five m / s². This acceleration guide acts as a reference dimension input in the state space, providing the deep reinforcement learning agent with a reference signal for human driving behavior in the current traffic environment. However, it does not directly constrain the agent's output actions, thus endowing the agent with an understanding of traffic rules and safe driving principles without compressing the policy optimization space.

[0026] Based on the above work, a multi-dimensional state space is constructed. This state space includes the following state variables: the longitudinal arc length coordinate s of the vehicle on the reference path, the vehicle speed v, the vehicle acceleration a, the arc length difference Δs between the vehicle and the adjacent preceding vehicle, the speed difference Δv between the vehicle and the adjacent preceding vehicle, the acceleration difference Δa between the vehicle and the adjacent preceding vehicle, the current traffic light phase state TL (a value of zero indicates a green light, and a value of one indicates a red light), the remaining time LT from the current time until the vehicle passes the expected passage time at the downstream intersection, and the aforementioned acceleration guidance quantity φ.

[0027] The remaining time LT from the expected passage time at the downstream intersection is calculated using a periodic dynamic prediction method. At preset fixed time intervals (usually five seconds), the vehicle obtains the motion status information and signal timing information of all preceding vehicles between itself and the downstream intersection via V2I communication. The simulation of the preceding vehicle's movement is performed using the current traffic environment as the initial condition, with the human-driven vehicle employing an IDM car-following model. When the vehicle passes the intersection immediately adjacent to the preceding vehicle, the simulation end time is recorded. This simulation end time is added to the vehicle's preset car-following distance to obtain the expected passage time DT. The remaining time LT is then obtained by subtracting the current time from this expected passage time. This mechanism updates the expected passage time based on real-time traffic conditions within each prediction cycle, ensuring that the remaining time in the state space always reflects the latest traffic flow prediction results.

[0028] S20: Path Decision Generation The multidimensional state space is input into a pre-trained improved dual-delay deep deterministic policy gradient deep reinforcement learning network (improved TD3 network). This network comprises three core neural network modules: an Actor policy network, a first Critic value network, and a second Critic value network. The two Critic networks have the same structure but different initial parameters to avoid generating the same estimation bias during training.

[0029] The output layer of the Actor policy network consists of a parallel path geometry branch and a longitudinal control branch. The path geometry branch outputs a sequence of lateral offsets for N path points uniformly distributed along the reference path. The spatial geometry of the planned path is defined. The longitudinal control branch outputs the desired acceleration, defining the velocity profile along the path. Both branches share traffic scene features extracted from the intermediate hidden layer, and their endpoints output through independent fully connected layers. During the feature learning phase, path geometry and velocity control are coupled; during the output phase, they are decoupled. The desired longitudinal velocity is obtained by integrating the desired acceleration over time. The path decision action is composed of the aforementioned lateral offset sequence and the desired longitudinal velocity.

[0030] During the training phase, after the Actor policy network outputs the original values, motion exploration noise following a preset normal distribution is applied to the desired acceleration, and random perturbations following a preset uniform distribution are applied to the lateral offset sequence. After amplitude limiting, training samples are collected through interaction with the environment. During the inference and deployment phase, only the original values ​​are output without adding noise or perturbations.

[0031] The first and second Critic networks take the current state and path decision action as inputs, respectively, and output a value estimate for that state-action pair. The network calculates the temporal difference error target value by taking the smaller value from the two Critic network outputs. By using the mechanism of taking the minimum value from the two Critic networks, the problem of overestimation of value in deep reinforcement learning is suppressed.

[0032] The improved TD3 network adopts a free speed training mode. In this mode, the driving efficiency reward component is used as a positive driving force to incentivize the agent to move forward, while the energy consumption economy reward component is used as a negative constraint to suppress high energy consumption operations. During the training process, the Actor policy network does not rely on a preset fixed cruising speed, but learns autonomously and optimizes the complete speed curve by maximizing the cumulative reward.

[0033] The reward function R consists of a weighted sum of six components, and its complete expression is: ; In the formula, R is the total reward value of the current control step size; to These are the weighting coefficients for each reward component; Security bonus portion; Bonus points are awarded for driving efficiency; This is the weight of the energy consumption economic incentive; For comfort bonus portion; To avoid reward points at road boundaries; Rewards for avoiding obstacles.

[0034] The specific calculation formulas for each reward component are as follows: The formula for calculating security bonus points is as follows: ; In the formula, Δs is the safety reference acceleration calculated using the IDM car-following model based on the vehicle's current speed, the arc length difference with the vehicle in front, and the speed difference; 'a' is the vehicle's current actual acceleration; Δs is the arc length difference between the vehicle and the adjacent vehicle in front, in meters. This is the preset minimum safe stationary distance, expressed in meters. This component includes two penalties: the first is a negative penalty when the vehicle's acceleration exceeds the safe reference acceleration, and the second is a negative penalty when the actual arc length difference is less than the minimum safe stationary distance.

[0035] Driving efficiency bonus The calculation formula is: ; In the formula, Δl is the arc length distance traveled by the vehicle along the reference path within the current control step. This component uses the actual distance traveled by the vehicle as a positive reward to incentivize the agent to actively complete the driving task and prevent it from stopping due to excessive pursuit of low energy consumption.

[0036] The formula for calculating the energy consumption economic incentive is as follows: ; In the formula, This is the preset fuel consumption penalty scaling factor; MOE is the instantaneous fuel consumption value calculated by the VT-Micro fuel consumption model. The VT-Micro model takes the vehicle speed v and acceleration a as inputs, and uses different model coefficient matrices to calculate instantaneous fuel consumption depending on the sign of the acceleration. This component guides the agent to tend towards a smooth, low-acceleration / deceleration driving style.

[0037] The formula for calculating the comfort bonus component is: ; In the formula, The preset jerk penalty coefficient is dimensionless; |jerk| is the absolute value of the longitudinal jerk, which is the first derivative of acceleration with respect to time. This component penalizes abrupt changes in acceleration, making the velocity change process smoother and improving ride comfort.

[0038] The formula for calculating the road boundary avoidance bonus is as follows: ; In the formula, The preset road boundary penalty coefficient; The potential field function value represents the road boundary, with the lateral offset of the vehicle, l, as the independent variable. The potential field function value increases sharply as l approaches the road boundary. This component constrains the vehicle to travel within the road boundary, preventing it from leaving the lane.

[0039] Obstacle avoidance reward The calculation formula is: ; In the formula, is the preset obstacle penalty coefficient; M is the total number of surrounding obstacles within the current perception range; s is the current longitudinal arc length coordinate of the vehicle; l is the current lateral offset of the vehicle, in meters; Let be the longitudinal arc length coordinate of the i-th obstacle in the Frenet coordinate system; Let be the lateral offset of the i-th obstacle in the Frenet coordinate system; The longitudinal standard deviation corresponding to the i-th obstacle is dynamically determined based on the length of the obstacle. This represents the lateral standard deviation corresponding to the i-th obstacle, dynamically determined based on the obstacle's width. This component is in the form of a two-dimensional Gaussian potential field function, generating a strong penalty signal when the vehicle approaches the center of the obstacle, guiding the vehicle to maintain a safe distance.

[0040] The weight coefficients of each component of the reward function are determined using the analytic hierarchy process (AHP). A judgment matrix is ​​constructed with safety as the highest priority, and pairwise comparisons are performed on safety, driving efficiency, energy economy, and comfort. An initial weight vector is calculated using the eigenvector method, and the consistency ratio is calculated. After confirming that the consistency ratio is less than a preset threshold (usually 0.1), each weight is calibrated with small-range perturbations under multiple simulation conditions. Indicators such as trajectory length, maximum curvature, and minimum distance to obstacles are compared to determine the final weight ratio.

[0041] The improved TD3 network employs a batch lag update mechanism with one update per round. After the agent completes a full path planning task in one round, all samples collected in that round are stored in the experience pool, and then the network parameters are updated by sampling from the experience pool. When the number of samples in the experience pool has not reached the preset minimum capacity, the original action output by the Actor network is used as the mean of a normal distribution, and random sampling is performed on this distribution and then amplitude-limited as the actual action to be executed. After completing a preset number of training rounds, a validation evaluation without noise is performed under fixed environmental conditions, and the model parameters corresponding to the highest cumulative validation reward value are saved.

[0042] The training process also incorporates a learning-from-demonstration framework, using the IDM and G-IDM models as expert policies. Each training round simultaneously collects demonstration samples generated by the expert policies and agent-explored samples, mixing them in a preset ratio and storing them in an experience pool. Furthermore, the training environment is divided into multiple levels based on the desired speed range. After a preset number of consecutive runs covering multiple levels, a batch network update is performed. This allows the value network to learn the overall performance of the policy from multiple complete trajectories covering different speed requirements, avoiding overfitting to specific speed conditions.

[0043] S30: Security Constraint Verification Receive path decision actions, perform security checks and make necessary corrections.

[0044] Establish an elliptical vehicle body contour model. The calculation formulas for the longitudinal safety half-axis 'a' and the lateral safety half-axis 'b', centered on the vehicle's center of gravity, are as follows: ; ; In the formula, a is the longitudinal safety half-axis; v represents the length of the vehicle body; v represents the current speed of the vehicle. is the preset driver braking reaction time; m is the vehicle mass; The square of the relative velocity between the vehicle and the obstacle; b is the maximum braking force of the vehicle; b is the lateral safety half-shaft; P is the preset expansion coefficient, with a value greater than one. The width of the vehicle body is given. The aforementioned longitudinal safety half-axis dynamically increases with the current speed of the vehicle and the relative speed between the vehicle and the obstacle, so that the area of ​​the elliptical safety profile adapts to different vehicle speeds. The higher the speed and the greater the relative speed, the wider the safety range.

[0045] A minimum safe distance model is established. Three minimum safe distance models are established for corner collision scenarios between the vehicle and obstacles at different locations: a first model for the vehicle and the vehicle in front in its own lane, a second model for the vehicle and the vehicle in front in the left lane, and a third model for the vehicle and the vehicle behind in the left lane. Each model uses the vehicle's speed, the obstacle's speed, the time of the corner collision, the longitudinal safe half-axis 'a', and the lateral safe half-axis 'b' as parameters to calculate the minimum safe distance required to avoid a corner collision. Each model is only applicable when the speeds of the vehicle and the corresponding obstacle meet a preset relationship.

[0046] After establishing the aforementioned geometric safety boundaries, the Action Masking safety mechanism is executed. Within each control step, the safety reference acceleration at the current moment is calculated using the IDM model based on the vehicle's current speed, the arc length difference between the vehicle and the preceding vehicle, and the speed difference. The expected acceleration of the path decision action is compared with the safety reference acceleration: if the expected acceleration is less than or equal to the safety reference acceleration, the path decision action remains unchanged; otherwise, the expected acceleration is replaced with the safety reference acceleration, and the corrected longitudinal expected velocity is obtained by integrating the replaced expected acceleration. This corrected velocity, together with the lateral offset sequence, constitutes the corrected path decision action. Simultaneously, the state and action pair that triggers the safety constraint is recorded in the experience pool of the improved TD3 network, enabling the Actor policy network to learn to recognize safety boundaries during subsequent training.

[0047] S40: Trajectory Optimization Using discrete path points in the corrected path decision-making process as reference trajectory points, the discrete path points are transformed into continuous and smooth optimal trajectories by constructing and solving a piecewise quintic polynomial quadratic programming problem.

[0048] The entire trajectory is divided into M trajectory segments along the longitudinal arc length coordinate. The lateral offset l within each trajectory segment is represented by a fifth-degree polynomial with the longitudinal arc length coordinate s as the independent variable. The expression is as follows: ; in, This represents the lateral offset corresponding to the arc length position s within the trajectory segment; s is the longitudinal arc length coordinate. to These are the six polynomial coefficients of the trajectory segment. Each of the six coefficients uniquely defines the geometry of a trajectory segment. The polynomial coefficients of all M trajectory segments are concatenated sequentially according to the segment order to form a complete coefficient vector C = The coefficient vector is the optimization variable for the quadratic programming problem.

[0049] The objective function is a weighted sum of the reference trajectory fit term and the smoothness term, and its formula is: ; Where C is the optimization variable coefficient vector; K is the total number of sampling points along the trajectory; For sampling points Substitute the value into the fifth-order polynomial to calculate the horizontal offset; is the lateral offset value of the k-th reference trajectory point; Q is the preset tracking accuracy weight matrix; R is the second derivative of the polynomial with respect to the arc length at the sampling point, reflecting the trajectory curvature; R is a preset smoothness weight matrix. The fit term guides the optimized trajectory to approach the path decision of the deep reinforcement learning network output, while the smoothness term penalizes curvature changes, guiding the generation of a trajectory with continuous and gentle curvature.

[0050] The constraints are categorized into five types. The first type is inter-segment continuity constraints, requiring adjacent trajectory segments to satisfy positional continuity, tangential continuity (equal first derivatives), and curvature continuity (equal second derivatives) at their connection points. The second type is road boundary constraints, requiring that the sum of the lateral offset calculated by the polynomial at each sampling point and the lateral safety half-axis b in S30 does not exceed half the road width. The third type is kinematic curvature constraints, requiring that the absolute value of the trajectory curvature calculated by the first and second derivatives of the polynomial at each sampling point does not exceed the maximum curvature threshold corresponding to the vehicle's physical limits. The maximum curvature threshold is determined by the vehicle's wheelbase L and maximum front wheel angle based on kinematic relationships. The fourth type is starting point state constraints, requiring that the polynomial value, first derivative, and second derivative at the starting point of the first trajectory segment are equal to the lateral offset of the vehicle's current position, the first derivative of the lateral offset with respect to the arc length, and the second derivative, respectively. The fifth type is ending point state constraints, requiring that the polynomial value and first derivative at the ending point of the last trajectory segment are equal to the planned terminal lateral offset and the first derivative of the lateral offset corresponding to the terminal direction angle.

[0051] This quadratic programming problem is a standard convex optimization problem, which can be solved using the interior point method or the effective set method. After obtaining the optimal solution of the coefficient vector, the polynomials of each trajectory segment are concatenated sequentially according to their arc lengths to obtain a continuous, smooth, optimal trajectory from the current position of the vehicle to the planned endpoint, satisfying both continuity and constraints.

[0052] S50: Multi-vehicle cooperative path adjustment When the vehicle is in a single-vehicle driving state, the optimal trajectory of S40 is directly transmitted to S60 as the target trajectory.

[0053] In multi-vehicle cooperative scenarios, V2V communication is used to acquire real-time information on the arc length position, speed, and acceleration of surrounding cooperative vehicles in the Frenet coordinate system. A constant-distance cooperative control strategy is employed to calculate the desired following distance, which is equal to the preset minimum stationary safety distance plus the preset headway multiplied by the vehicle's current speed v. The distance error is defined as the difference between the arc length position of the adjacent vehicle and the arc length position of the vehicle itself, minus the desired following distance.

[0054] When the vehicle is following another vehicle, the distance error is fed back to the aiming distance adjustment strategy. If the actual distance to the vehicle is greater than the expected following distance, the aiming distance is reduced to enhance tracking sensitivity and guide the vehicle to accelerate and approach the vehicle in front. If the actual distance to the vehicle is less than the expected following distance, the aiming distance is increased to smooth out oscillations and guide the vehicle to decelerate smoothly and increase the distance.

[0055] When the lead vehicle is the navigator, the initial expected speed corresponding to the optimal trajectory on S40 is... The formula for calculating the desired speed correction is as follows: ; in, This is the desired speed correction amount, and its value is not less than zero; The free-flow velocity of the road; This represents the number of vehicles queuing behind the vehicle at the current time of its passage through the intersection. This represents the average headway. This represents the remaining green light time at the current intersection. When there are many vehicles queuing behind and the remaining green light time is short, the correction amount is larger, and the lead vehicle increases its desired speed to help vehicles behind accelerate through the intersection. When there are no vehicles queuing behind or the green light is ample, the correction amount is zero or very small, and the lead vehicle maintains its original energy-saving speed strategy. During the current green light period at the intersection, the initial desired speed is added to the correction amount to obtain the corrected desired speed. After the green light ends, the corrected desired speed reverts to the initial desired speed according to a preset decay function.

[0056] When V2V communication has a fixed time delay τ, first-order prediction compensation is also performed. If the following vehicle receives the lag arc length position, lag velocity, and lag acceleration of the preceding vehicle at a past lag time t-τ at the current time t, a first-order Taylor expansion is used to approximate the current velocity of the preceding vehicle. The predicted value is equal to the lag velocity plus the lag acceleration multiplied by τ. This predicted value replaces the originally received lag velocity in the calculation of the expected following distance and the distance error. To ensure queue stability, the headway is greater than twice the communication delay τ to ensure that the distance error does not amplify upstream in the vehicle queue in the frequency domain.

[0057] After the coordinated adjustment is completed, the corrected expected velocity is used as the longitudinal control target, and together with the path geometry of the continuous smooth optimal trajectory, they form the target trajectory after coordinated adjustment.

[0058] S60: Adaptive Track Following The system receives the target trajectory and converts it into control commands that the vehicle can execute through the lateral and longitudinal controllers.

[0059] The lateral controller employs adaptive model predictive control (adaptive MPC controller), using the path geometry in the target trajectory as the reference path and the lateral offset and heading angle of the vehicle in the Frenet coordinate system as the tracking state variables. The predictive model is established based on a three-degree-of-freedom vehicle dynamics model and a magic formula tire model. The state variables are the vehicle's lateral velocity, longitudinal velocity, yaw angle, yaw rate, global lateral position, and global longitudinal position, while the control variable is the front wheel steering angle.

[0060] Prediction Time Domain Adaptive adjustment is a key design feature of the lateral controller. Optimal prediction time-domain data that minimizes tracking error and satisfies stability constraints at different vehicle speeds was obtained through offline simulation experiments. A fifth-order polynomial was used to fit the vehicle speed v to the optimal prediction time-domain data. The mapping relationship between them is expressed by the following formula:

[0061] in, This represents the optimal prediction time domain corresponding to vehicle speed v; v is the current longitudinal speed of the vehicle. to These are the coefficients of a fifth-order polynomial obtained by fitting offline simulation data. During online control, when the vehicle speed is below a preset low-speed threshold... Take the first preset value; when the vehicle speed exceeds the preset high-speed threshold... Take the second preset value, which is greater than the first preset value. When the vehicle speed is between the two, calculate using the above fifth-order polynomial. This allows for adaptive adjustment of the prediction time domain across the entire vehicle speed range.

[0062] The objective function of the adaptive MPC controller includes lateral position error penalty terms, heading angle error penalty terms, and control increment penalty terms. Constraints include front wheel steering angle magnitude constraints (not exceeding the physical limits of the vehicle's steering mechanism), front wheel steering angle change rate constraints, tire slip angle constraints (ensuring the tire operates within the linear range), center of gravity slip angle constraints (setting different boundaries based on road adhesion conditions), and road adhesion condition constraints. Each control cycle solves this constrained rolling optimization problem, and the first element of the optimal control sequence is taken as the front wheel steering angle control command for the current moment.

[0063] The longitudinal controller employs a fuzzy PID controller, using the corrected desired velocity in the target trajectory as the tracking target. Velocity error and its rate of change are inputs. Fuzzy inference is performed through a pre-set fuzzy rule base, outputting online proportional parameter correction, integral parameter correction, and derivative parameter correction. Each rule in the fuzzy rule base uses a condition plus conclusion form: "If the velocity error is a certain fuzzy linguistic value and the velocity error rate of change is a certain fuzzy linguistic value, then the proportional parameter correction, integral parameter correction, and derivative parameter correction are respectively the corresponding fuzzy linguistic values." Triangular membership functions are used for both input and output membership functions, and the centroid method is employed for defuzzification.

[0064] The three correction values ​​output by fuzzy inference are added to the preset initial proportional, integral, and derivative parameters respectively to obtain the optimized proportional, integral, and derivative parameters. The optimized parameters are then used to calculate the initial control quantity based on the speed error through the proportional, integral, and derivative steps. If the initial control quantity is positive, it is mapped to a throttle opening command, and the brake pressure is set to zero; if the initial control quantity is negative, it is mapped to a brake pressure command, and the throttle opening is set to zero.

[0065] Through the coordinated operation of the aforementioned lateral and longitudinal controllers, the vehicle is driven to precisely follow the target trajectory.

[0066] The implementation principle of an intelligent driving path planning method based on deep reinforcement learning in this application embodiment is as follows: A multi-dimensional state space containing acceleration guidance is constructed in the Frenet coordinate system, integrating human-like driving prior knowledge into the agent's environmental perception to improve the generalization ability to random traffic scenarios. The state space is input into an improved TD3 deep reinforcement learning network using a free velocity training mode. The Actor policy network outputs the path geometry and longitudinal velocity profile in parallel. A dual Critic network minimizes the value overestimation. A multi-objective weighted reward function, considering safety, driving efficiency, energy economy, comfort, and artificial potential field obstacle avoidance penalties, guides the agent to autonomously discover the globally optimal driving strategy in complex scenarios. A geometric safety boundary is constructed using an elliptical vehicle body contour model and a minimum safety distance model, combined with the Action... The masking hard constraint mechanism performs safety verification and correction on the network output actions, ensuring path safety from both soft penalty and hard constraint levels. Using the corrected discrete path points as a reference, a piecewise fifth-order polynomial quadratic programming problem is constructed to solve for a smooth and kinematically feasible optimal trajectory under the constraints of continuity, road boundaries, and kinematic curvature. Furthermore, the state of the cooperating vehicles is obtained through V2V communication, and the multi-vehicle cooperative path is adjusted based on a constant time-distance strategy and the desired speed correction of the lead vehicle. Finally, the adaptive MPC lateral controller dynamically adjusts the predicted time-domain output front wheel steering angle, and the fuzzy PID longitudinal controller optimizes the parameters online to output longitudinal acceleration, driving the vehicle to accurately follow the target trajectory.

[0067] Figure 1 This is a flowchart illustrating an intelligent driving path planning method based on deep reinforcement learning in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows; unless explicitly stated otherwise, there is no strict order requirement for the execution of these steps, and they can be executed in other orders; and Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0068] Based on the same technical concept, referring to Figure 2 This application also provides an intelligent driving path planning device based on deep reinforcement learning, which adopts the following technical solution: The device includes: The environmental perception module is used to receive environmental perception information, decouple vehicle motion in the Frenet coordinate system and calculate acceleration guidance, and construct a multi-dimensional state space containing acceleration guidance; the acceleration guidance is the ratio of human-like driving reference acceleration to the maximum limit value of output acceleration. The path decision module is used to input the multi-dimensional state space into the improved TD3 deep reinforcement learning network. Its Actor policy network outputs path decision actions, and the dual Critic network takes the minimum value to suppress value overestimation. The network adopts a free speed training mode, and the reward function is composed of a weighted sum of safety, driving efficiency, energy economy, comfort, and artificial potential field obstacle avoidance penalty. The path correction module is used to dynamically calculate the longitudinal and lateral safety half-axis using an elliptical vehicle body contour model and establish a minimum safety distance model. Through the Action Masking mechanism, when the acceleration of the path decision action exceeds the safety reference acceleration, it is replaced by the safety reference acceleration to obtain the corrected path decision action. The trajectory generation module is used to construct a piecewise quintic polynomial quadratic programming problem with reference to the discrete path points in the corrected path decision action, and solve it to obtain the optimal trajectory that satisfies the constraints of continuity, road boundary and kinematic curvature. The trajectory adjustment module is used to obtain the status of the cooperating vehicles through V2V communication and adjust the aiming distance or desired speed based on the constant time-distance strategy and spacing error; the lead vehicle calculates the desired speed correction based on the number of vehicles in the queue behind and the remaining time of the green light to obtain the target trajectory after the cooperative adjustment. The trajectory drive module is used to dynamically adjust the predicted time-domain output front wheel steering angle of the adaptive MPC lateral controller and optimize the output longitudinal acceleration of the fuzzy PID longitudinal controller online, driving the vehicle to follow the target trajectory.

[0069] In some embodiments, the path decision module is specifically used to improve the Actor policy network of the TD3 deep reinforcement learning network to take a multi-dimensional state space as input and output a path decision action, which includes a lateral offset sequence of multiple path points uniformly distributed along the reference path and a longitudinal desired velocity. The dual-Critic network consists of a first Critic network and a second Critic network with the same network structure but different initial parameters. They estimate the action value of the path decision action in the current state, and the minimum of the two values ​​is used to calculate the temporal difference error target value to suppress the overestimation of action value by the single Critic network. The temporal difference error target value is jointly determined by the immediate reward at the current step size, the reward discount factor, the next action output by the target Actor network, and the value estimate of the next action by the target Critic network. The free speed training mode refers to a mode where driving efficiency reward components are used as positive drivers and energy consumption economy reward components are used as negative constraints. During the training process, the Actor policy network autonomously optimizes the speed curve instead of following a preset fixed cruising speed. The reward function includes: a safety reward component composed of the penalty for the difference between the current safety reference acceleration and the actual acceleration, and the penalty for the difference between the actual arc length difference between the vehicle and the adjacent vehicle and the preset minimum stationary safety distance. The safety reference acceleration is calculated by the intelligent driving following model; a driving efficiency reward component is the arc length distance traveled by the vehicle along the reference path within the current control step; an energy economy reward component is the negative value of the instantaneous fuel consumption logarithm calculated by the fuel consumption penalty scaling factor multiplied by the VT-Micro fuel consumption model; a comfort reward component is the negative value of the acceleration penalty coefficient multiplied by the absolute value of the longitudinal acceleration; and an artificial potential field obstacle avoidance penalty component including road boundary penalty and obstacle penalty. The road boundary penalty is determined by multiplying the road boundary penalty coefficient by the road boundary potential field function value. A penalty is generated when the vehicle laterally deviates close to the road boundary. The obstacle penalty is determined by multiplying the obstacle penalty coefficient by the sum of the two-dimensional Gaussian potential field function values ​​of all surrounding obstacles. The longitudinal and lateral standard deviations of the two-dimensional Gaussian potential field function are dynamically determined according to the length and width dimensions of each obstacle. The weight coefficients of each component in the reward function are determined using the analytic hierarchy process (AHP). A judgment matrix is ​​constructed with safety as the highest priority. The initial weight vector is calculated using the eigenvector method. After the consistency ratio is confirmed to be less than the preset threshold by the consistency check, simulation calibration is performed to obtain the final weight ratio.

[0070] In some embodiments, the path correction module is specifically used for the elliptical body contour model with the vehicle's center of gravity as the center. The longitudinal safety half-axis is composed of the sum of the half-length of the body, the distance traveled during the driver's braking reaction time, and the braking distance corresponding to the relative speed. The lateral safety half-axis is obtained by multiplying the half-width of the body by a preset expansion coefficient. The longitudinal safety half-axis dynamically increases with the increase of the vehicle's current speed and the relative speed between the vehicle and the obstacle, so that the elliptical body contour model adaptively represents the vehicle's safety range at different vehicle speeds. Based on the relative positional relationship between the vehicle and the obstacle, a first minimum safe distance model is established between the vehicle and the vehicle in front in the same lane, a second minimum safe distance model is established between the vehicle and the vehicle in front in the left lane, and a third minimum safe distance model is established between the vehicle and the vehicle behind in the left lane. Each minimum safe distance model uses the speed difference between the vehicle and the corresponding obstacle, the time of corner collision, the longitudinal safety half-axis, and the lateral safety half-axis as parameters to calculate the minimum safe distance required to avoid a corner collision; Within each control step, based on the Intelligent Driving Follow Model (IDM), the current safe reference acceleration is calculated according to the vehicle's current speed, the arc length difference between the vehicle and the adjacent vehicle, and the speed difference. The expected acceleration in the path decision action is compared with the safe reference acceleration. If the expected acceleration is greater than the safe reference acceleration, the expected acceleration is replaced by the safe reference acceleration as the expected acceleration in the corrected path decision action. At the same time, the state and action pair that triggers the safety limit is recorded in the experience pool of the improved TD3 deep reinforcement learning network, so that the Actor policy network can learn to avoid generating high-risk actions that exceed the safety boundary in subsequent training.

[0071] In some embodiments, the trajectory generation module is specifically used as a reference trajectory point in the corrected path decision action, dividing the entire trajectory into M trajectory segments along the longitudinal arc length coordinate. Each trajectory segment is represented by a fifth-order polynomial with the longitudinal arc length coordinate as the independent variable, and the fifth-order polynomial expression of the i-th trajectory segment consists of six polynomial coefficients. The optimization variable is a coefficient vector formed by sequentially concatenating the polynomial coefficients of all trajectory segments. The objective function is composed of a weighted sum of a reference trajectory fit term and a smoothness term. The reference trajectory fit term is the sum of squares of the differences between the lateral offset calculated by the polynomial at each sampling point within the trajectory segment and the lateral offset of the corresponding reference trajectory point. The smoothness term is the sum of squares of the second derivatives of the polynomial at each sampling point with respect to the longitudinal arc length coordinate. The priority between the reference trajectory fit term and the smoothness term is adjusted by a preset following accuracy weight matrix and a smoothness weight matrix. The constraints of the fifth-order polynomial expression include: positional continuity constraints where adjacent trajectory segments have equal polynomial values ​​at the connection point, tangent continuity constraints where the first derivative is equal, and curvature continuity constraints where the second derivative is equal; the road boundary constraints require that the sum of the lateral offset calculated by the polynomial at each sampling point and the lateral safety half-axis does not exceed half the road width. The kinematic curvature constraint requires that the absolute value of the trajectory curvature calculated by the polynomial at each sampling point does not exceed the maximum curvature threshold corresponding to the minimum turning radius of the vehicle; the starting state constraint requires that the polynomial value, first derivative, and second derivative at the starting point of the first trajectory segment are equal to the lateral offset of the vehicle's current position, the first derivative and second derivative of the lateral offset with respect to the arc length; the ending state constraint requires that the polynomial value and first derivative at the ending point of the last trajectory segment are equal to the planned terminal lateral offset and the first derivative of the lateral offset corresponding to the terminal direction angle. The quadratic programming problem is a convex optimization problem. It is solved by the interior point method or the effective set method to obtain the optimal solution of the coefficient vector. The polynomials of each trajectory segment are spliced ​​together according to the arc length to obtain the optimal trajectory from the current position of the vehicle to the planned endpoint, which satisfies the constraints of continuity, road boundary and kinematic curvature.

[0072] In some embodiments, the trajectory adjustment module is specifically used to obtain the arc length position, speed and acceleration information of surrounding cooperative vehicles in the Frenet coordinate system in real time through V2V communication; adopt a constant time-distance cooperative control strategy, define the expected following distance as the sum of the preset minimum stationary safety distance and the time distance to the front of the vehicle multiplied by the current speed of the vehicle; and define the distance error as the difference between the arc length position of the adjacent vehicle and the arc length position of the vehicle itself minus the expected following distance. When the vehicle is following another vehicle, the distance error is fed back to the pre-aiming distance. If the actual distance is greater than the expected following distance, the pre-aiming distance is reduced to enhance the following sensitivity, so that the vehicle accelerates to approach the vehicle in front. If the actual distance is less than the expected following distance, the pre-aiming distance is increased to smooth out the oscillation, so that the vehicle decelerates smoothly to increase the distance. When the vehicle is the lead vehicle, the initial expected speed is calculated, the remaining green light time at the current intersection is obtained, and the number of vehicles in the queue behind the vehicle when it passes the target intersection is obtained. The expected speed correction is calculated based on the free-flow speed of the road, the initial expected speed, and the remaining green light time. During the current green light period at the intersection, the initial expected speed is added to the expected speed correction amount to obtain the corrected expected speed; when the green light at the target intersection ends, the corrected expected speed will revert to the initial expected speed according to a preset decay function. The coordinated adjustment of the target trajectory is a complete spatiotemporal trajectory formed by using the corrected expected velocity as the longitudinal control target and the path geometry of the optimal trajectory.

[0073] In some embodiments, when there is a fixed time delay in V2V communication, after acquiring the motion state information of surrounding cooperative vehicles, a first-order prediction compensation mechanism is used to correct the received lag state: if the following vehicle receives the lag arc length position, lag speed, and lag acceleration of the preceding vehicle at time t minus τ at time t, then the speed of the preceding vehicle at the current time is approximated by a first-order Taylor expansion, and the predicted value is the sum of the products of the lag speed and the lag acceleration multiplied by the communication delay τ; the predicted value is used to replace the original received lag speed and is substituted into the constant time-distance cooperative control strategy to calculate the expected following distance and distance error; Among them, the headway is greater than twice the communication delay τ to meet the queue stability condition that the platoon spacing error does not amplify upstream along the vehicle queue in the frequency domain, and to ensure that the target trajectory after coordinated adjustment remains stable under the communication delay condition.

[0074] In some embodiments, the trajectory driving module is specifically used to adapt the MPC lateral controller to take the path geometry in the cooperatively adjusted target trajectory as the reference path and the lateral offset and heading angle of the vehicle in the Frenet coordinate system as the following state. The predictive model of the adaptive MPC lateral controller is based on a three-degree-of-freedom vehicle dynamics model and a magic formula tire model. The lateral velocity, longitudinal velocity, yaw angle, yaw rate, global lateral position, and global longitudinal position of the vehicle are used as state variables, and the front wheel steering angle is used as the control variable. The mapping relationship between vehicle speed and optimal prediction time domain is fitted by a fifth-order polynomial. When the vehicle speed is lower than the preset low-speed threshold, the prediction time domain takes the first preset value. When the vehicle speed is higher than the preset high-speed threshold, the prediction time domain takes the second preset value, and the second preset value is greater than the first preset value. When the vehicle speed is between the two, the prediction time domain value is calculated by the fifth-order polynomial, so as to realize the adaptive adjustment of the prediction time domain in the full vehicle speed range. The objective function of the adaptive MPC lateral controller includes a lateral position error penalty term, a heading angle error penalty term, and a control increment penalty term. The constraints include front wheel steering angle magnitude constraint, front wheel steering angle change rate constraint, tire slip angle constraint, center of gravity slip angle constraint, and road adhesion condition constraint. In each control cycle, the objective constrained rolling optimization problem is solved, and the first element of the optimal control sequence is taken as the front wheel steering angle control command at the current moment. The fuzzy PID longitudinal controller takes the corrected expected speed in the target trajectory after collaborative adjustment as the following target, and takes the speed error and speed error change rate between the corrected expected speed and the actual speed of the vehicle as input. It performs fuzzy inference through a preset fuzzy rule base and outputs the proportional parameter correction amount, integral parameter correction amount and derivative parameter correction amount. These are superimposed with the preset initial proportional parameter, initial integral parameter and initial derivative parameter to obtain the optimized proportional parameter, integral parameter and derivative parameter. The initial control quantity is calculated based on the speed error using the optimized proportional, integral, and derivative parameters. When the initial control quantity is positive, it is mapped to a throttle opening command and the brake pressure is set to zero. When the initial control quantity is negative, it is mapped to a brake pressure command and the throttle opening is set to zero, thus obtaining the longitudinal acceleration control command. The vehicle is driven to follow the target trajectory after coordinated adjustment by front wheel steering angle control commands and longitudinal acceleration control commands.

[0075] This application also discloses a control device.

[0076] Specifically, the control device includes a memory and a processor, with the memory storing a computer program that can be loaded and executed by the processor to implement the aforementioned deep reinforcement learning-based intelligent driving path planning method.

[0077] This application also discloses a computer-readable storage medium.

[0078] Specifically, the computer-readable storage medium stores a computer program that can be loaded and executed by a processor, such as the intelligent driving path planning method based on deep reinforcement learning described above. The computer-readable storage medium includes, for example, various media that can store program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0079] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for intelligent driving path planning based on deep reinforcement learning, characterized in that, include: Receive environmental perception information, decouple vehicle motion in Frenet coordinate system and calculate acceleration guidance amount, and construct a multi-dimensional state space containing the acceleration guidance amount; The acceleration guide value is the ratio of the human-like driving reference acceleration to the maximum limit value of the output acceleration; The multidimensional state space is input into the improved TD3 deep reinforcement learning network. Its Actor policy network outputs path decision actions, and the dual Critic network takes the minimum value to suppress value overestimation. The network adopts a free speed training mode, and the reward function is composed of a weighted sum of safety, driving efficiency, energy economy, comfort, and artificial potential field obstacle avoidance penalty. An elliptical vehicle body profile model is used to dynamically calculate the longitudinal and lateral safety half-axis, and a minimum safety distance model is established. Through the Action Masking mechanism, when the acceleration of the path decision action exceeds the safety reference acceleration, the safety reference acceleration is used instead to obtain the corrected path decision action. Using the discrete path points in the modified path decision action as a reference, a piecewise fifth-order polynomial quadratic programming problem is constructed, and the optimal trajectory that satisfies the constraints of continuity, road boundary and kinematic curvature is obtained by solving it. The vehicle obtains the status of the cooperating vehicles through V2V communication, and adjusts the aiming distance or desired speed based on the constant time-distance strategy and spacing error; the lead vehicle calculates the desired speed correction based on the number of vehicles in the queue behind and the remaining time of the green light, and obtains the target trajectory after the cooperative adjustment. The adaptive MPC lateral controller dynamically adjusts the predicted time-domain output front wheel steering angle, and the fuzzy PID longitudinal controller optimizes the parameters online to output longitudinal acceleration, driving the vehicle to follow the target trajectory.

2. The intelligent driving path planning method based on deep reinforcement learning according to claim 1, characterized in that, The process of inputting the multidimensional state space into the improved TD3 deep reinforcement learning network, where the Actor policy network outputs path decision actions and the dual Critic network takes the minimum value to suppress overestimation of value, includes: The improved TD3 deep reinforcement learning network's Actor policy network takes the multi-dimensional state space as input and outputs the path decision action, which includes a lateral offset sequence of multiple path points uniformly distributed along the reference path and a longitudinal desired velocity. The dual-Critic network comprises a first Critic network and a second Critic network with identical network structures but different initial parameters. They estimate the action value of the path decision action in the current state, and the minimum of the two values ​​is used to calculate the temporal difference error target value to suppress the overestimation of action value by the single Critic network. The temporal difference error target value is jointly determined by the immediate reward at the current step size, the reward discount factor, the next action output by the target Actor network, and the value estimate of the next action by the target Critic network. The free speed training mode refers to a mode where driving efficiency reward components are used as positive drivers and energy consumption economy reward components are used as negative constraints. During the training process, the Actor policy network autonomously optimizes the speed curve instead of following a preset fixed cruising speed. The reward function includes: a safety reward component composed of a penalty for the difference between the current safety reference acceleration and the actual acceleration, and a penalty for the difference between the actual arc length difference between the vehicle and the adjacent vehicle and the preset minimum stationary safety distance. The safety reference acceleration is calculated by the intelligent driving following model. The driving efficiency reward component is the arc length distance traveled by the vehicle along the reference path within the current control step. The energy economy reward component is the negative value of the instantaneous fuel consumption logarithm calculated by the VT-Micro fuel consumption model multiplied by the fuel consumption penalty scaling factor. The comfort reward component is the negative value of the acceleration penalty coefficient multiplied by the absolute value of the longitudinal acceleration. The artificial potential field obstacle avoidance penalty component includes road boundary penalty and obstacle penalty. The road boundary penalty is determined by multiplying the road boundary penalty coefficient by the road boundary potential field function value. A penalty is generated when the vehicle deviates laterally and approaches the road boundary. The obstacle penalty is determined by multiplying the obstacle penalty coefficient by the sum of the two-dimensional Gaussian potential field function values ​​of all surrounding obstacles. The longitudinal and lateral standard deviations of the two-dimensional Gaussian potential field function are dynamically determined according to the length and width dimensions of each obstacle. The weight coefficients of each component in the reward function are determined using the analytic hierarchy process (AHP). A judgment matrix is ​​constructed with safety as the highest priority. The initial weight vector is calculated using the eigenvector method. After the consistency ratio is confirmed to be less than a preset threshold by a consistency check, simulation calibration is performed to obtain the final weight ratio.

3. The intelligent driving path planning method based on deep reinforcement learning according to claim 2, characterized in that, The method employs an elliptical vehicle body contour model to dynamically calculate the longitudinal and lateral safety half-axes, establishing a minimum safety distance model. Through an Action Masking mechanism, when the acceleration of the path decision action exceeds the safety reference acceleration, it is replaced by the safety reference acceleration, resulting in a corrected path decision action, including: The elliptical vehicle body profile model is centered on the vehicle's center of gravity. The longitudinal safety half-axis is composed of the sum of the vehicle's half-length, the distance traveled during the driver's braking reaction time, and the braking distance corresponding to the relative speed. The lateral safety half-axis is obtained by multiplying the vehicle's half-width by a preset expansion coefficient. The longitudinal safety half-axis dynamically increases with the increase of the vehicle's current speed and the relative speed between the vehicle and the obstacle, so that the elliptical vehicle body profile model adaptively represents the vehicle's safety range at different vehicle speeds. Based on the relative positional relationship between the vehicle and the obstacle, a first minimum safe distance model is established between the vehicle and the vehicle in front in the same lane, a second minimum safe distance model is established between the vehicle and the vehicle in front in the left lane, and a third minimum safe distance model is established between the vehicle and the vehicle behind in the left lane. Each of the minimum safe distance models calculates the minimum safe distance required to avoid a corner collision using the speed difference between the vehicle and the corresponding obstacle, the time of the corner collision, the longitudinal safety half-axis, and the lateral safety half-axis as parameters. Within each control step, based on the Intelligent Driving Follow Model (IDM), the current safe reference acceleration is calculated according to the vehicle's current speed, the arc length difference between the vehicle and the adjacent vehicle, and the speed difference. The expected acceleration in the path decision action is compared with the safe reference acceleration. If the expected acceleration is greater than the safe reference acceleration, the expected acceleration is replaced by the safe reference acceleration as the expected acceleration in the corrected path decision action. At the same time, the state and action pair that triggers the safety limit is recorded in the experience pool of the improved TD3 deep reinforcement learning network, so that the Actor policy network learns to avoid generating high-risk actions that exceed the safety boundary in subsequent training.

4. The intelligent driving path planning method based on deep reinforcement learning according to claim 1, characterized in that, The step of constructing a piecewise quintic polynomial quadratic programming problem, using discrete path points in the modified path decision action as references, and solving it to obtain the optimal trajectory that satisfies the constraints of continuity, road boundaries, and kinematic curvature, includes: In the corrected path decision action, the discrete path points are used as reference trajectory points. The entire trajectory is divided into M trajectory segments along the longitudinal arc length coordinate. Each trajectory segment is represented by a fifth-order polynomial with the longitudinal arc length coordinate as the independent variable, which represents the lateral offset within the segment. The fifth-order polynomial expression of the i-th trajectory segment consists of six polynomial coefficients. The optimization variable is a coefficient vector formed by concatenating the polynomial coefficients of all trajectory segments in sequence. The objective function is composed of a weighted sum of a reference trajectory fit term and a smoothness term. The reference trajectory fit term is the sum of squares of the differences between the lateral offset calculated by the polynomial at each sampling point within the trajectory segment and the lateral offset of the corresponding reference trajectory point. The smoothness term is the sum of squares of the second derivatives of the polynomial at each sampling point with respect to the longitudinal arc length coordinate. The priority between the reference trajectory fit term and the smoothness term is adjusted by a preset following accuracy weight matrix and a smoothness weight matrix. The constraints of the fifth-order polynomial expression include: positional continuity constraints where adjacent trajectory segments have equal polynomial values ​​at the connection point, tangent continuity constraints where the first derivative is equal, and curvature continuity constraints where the second derivative is equal; the road boundary constraints require that the sum of the lateral offset calculated by the polynomial at each sampling point and the lateral safety half-axis does not exceed half the road width. The kinematic curvature constraint requires that the absolute value of the trajectory curvature calculated by the polynomial at each sampling point does not exceed the maximum curvature threshold corresponding to the minimum turning radius of the vehicle; the starting state constraint requires that the polynomial value, first derivative, and second derivative at the starting point of the first trajectory segment are equal to the lateral offset of the vehicle's current position, the first derivative and second derivative of the lateral offset with respect to the arc length; the ending state constraint requires that the polynomial value and first derivative at the ending point of the last trajectory segment are equal to the planned terminal lateral offset and the first derivative of the lateral offset corresponding to the terminal direction angle. The quadratic programming problem is a convex optimization problem. It is solved by the interior point method or the effective set method to obtain the optimal solution of the coefficient vector. The polynomials of each trajectory segment are spliced ​​together according to the arc length to obtain the optimal trajectory from the current position of the vehicle to the planned endpoint, which satisfies the constraints of continuity, road boundary and kinematic curvature.

5. The intelligent driving path planning method based on deep reinforcement learning according to claim 4, characterized in that, The system obtains the status of the cooperative vehicle through V2V communication and adjusts the aiming distance or desired speed based on a constant time-distance strategy and spacing error. The lead vehicle calculates the desired speed correction based on the number of cars in the queue behind and the remaining time of the green light, thus obtaining the coordinated target trajectory, including: The system acquires real-time information on the arc length position, speed, and acceleration of surrounding cooperative vehicles in the Frenet coordinate system via V2V communication; adopts a constant time-distance cooperative control strategy, defining the desired following distance as the sum of the preset minimum stationary safety distance and the time distance to the front of the vehicle multiplied by the current speed of the vehicle itself; and defines the distance error as the difference between the arc length position of the adjacent vehicle and the arc length position of the vehicle itself, minus the desired following distance. When the vehicle is following another vehicle, the distance error is fed back to the pre-aiming distance. If the actual distance is greater than the expected following distance, the pre-aiming distance is reduced to enhance the following sensitivity, so that the vehicle accelerates to approach the vehicle in front. If the actual distance is less than the expected following distance, the pre-aiming distance is increased to smooth out the oscillation, so that the vehicle decelerates smoothly to increase the distance. When the vehicle is the lead vehicle, the initial expected speed is calculated, the remaining green light time at the current intersection and the number of vehicles in the queue behind the vehicle when it passes the target intersection are obtained, and the expected speed correction is calculated; the expected speed correction is calculated based on the road free-flow speed, the initial expected speed and the remaining green light time. During the current green light period at the intersection, the initial expected speed is added to the expected speed correction amount to obtain the corrected expected speed; when the green light at the target intersection ends, the corrected expected speed is reduced back to the initial expected speed according to a preset decay function; The target trajectory after collaborative adjustment is a complete spatiotemporal trajectory formed by using the corrected desired velocity as the longitudinal control target and the path geometry of the optimal trajectory.

6. The intelligent driving path planning method based on deep reinforcement learning according to claim 5, characterized in that, When the V2V communication has a fixed time delay, after obtaining the motion state information of the surrounding cooperating vehicles, a first-order prediction compensation mechanism is used to correct the received lag state: if the following vehicle receives the lag arc length position, lag speed and lag acceleration of the preceding vehicle at time t minus τ at time t, then the speed of the preceding vehicle at the current time is approximated by a first-order Taylor expansion, and the predicted value is the sum of the product of the lag speed and the lag acceleration multiplied by the communication delay τ; The predicted value is used to replace the original received hysteresis speed and is substituted into the constant time-distance cooperative control strategy to calculate the expected following distance and the distance error. The headway is greater than twice the communication delay τ to satisfy the queue stability condition that the platoon spacing error does not amplify upstream along the vehicle queue in the frequency domain, thus ensuring that the target trajectory after coordinated adjustment remains stable under communication delay conditions.

7. The intelligent driving path planning method based on deep reinforcement learning according to claim 5, characterized in that, The adaptive MPC lateral controller dynamically adjusts the predicted time-domain output front wheel steering angle, and the fuzzy PID longitudinal controller optimizes the parameters online to output longitudinal acceleration, driving the vehicle to follow the target trajectory, including: The adaptive MPC lateral controller uses the path geometry in the target trajectory after collaborative adjustment as the reference path and the lateral offset and heading angle of the vehicle in the Frenet coordinate system as the following state. The predictive model of the adaptive MPC lateral controller is established based on a three-degree-of-freedom vehicle dynamics model and a magic formula tire model. The lateral velocity, longitudinal velocity, yaw angle, yaw rate, global lateral position, and global longitudinal position of the vehicle are used as state variables, and the front wheel steering angle is used as the control variable. The mapping relationship between vehicle speed and optimal prediction time domain is fitted by a fifth-order polynomial. When the vehicle speed is lower than the preset low-speed threshold, the prediction time domain takes the first preset value. When the vehicle speed is higher than the preset high-speed threshold, the prediction time domain takes the second preset value, and the second preset value is greater than the first preset value. When the vehicle speed is between the two, the prediction time domain value is calculated by the fifth-order polynomial, so as to realize the adaptive adjustment of the prediction time domain in the full vehicle speed range. The objective function of the adaptive MPC lateral controller includes a lateral position error penalty term, a heading angle error penalty term, and a control increment penalty term. The constraints include front wheel steering angle magnitude constraint, front wheel steering angle change rate constraint, tire slip angle constraint, center of gravity slip angle constraint, and road adhesion condition constraint. In each control cycle, the objective constrained rolling optimization problem is solved, and the first element of the optimal control sequence is taken as the front wheel steering angle control command at the current moment. The fuzzy PID longitudinal controller takes the corrected expected speed in the target trajectory after collaborative adjustment as the following target, and takes the speed error and speed error change rate between the corrected expected speed and the actual speed of the vehicle as input. It performs fuzzy inference through a preset fuzzy rule base and outputs the proportional parameter correction amount, integral parameter correction amount and derivative parameter correction amount. These are superimposed with the preset initial proportional parameter, initial integral parameter and initial derivative parameter to obtain the optimized proportional parameter, integral parameter and derivative parameter. The initial control quantity is calculated based on the speed error using the optimized proportional, integral, and derivative parameters. When the initial control quantity is positive, it is mapped to a throttle opening command and the brake pressure is set to zero. When the initial control quantity is negative, it is mapped to a brake pressure command and the throttle opening is set to zero, thus obtaining the longitudinal acceleration control command. The vehicle is driven to follow the target trajectory after coordinated adjustment by the front wheel steering angle control command and the longitudinal acceleration control command.

8. An intelligent driving path planning device based on deep reinforcement learning, characterized in that, The device includes: The environmental perception module is used to receive environmental perception information, decouple vehicle motion in the Frenet coordinate system and calculate acceleration guidance amount, and construct a multi-dimensional state space containing the acceleration guidance amount; the acceleration guidance amount is the ratio of human-like driving reference acceleration to the maximum limit value of output acceleration. The path decision module is used to input the multi-dimensional state space into the improved TD3 deep reinforcement learning network. Its Actor policy network outputs path decision actions, and the dual Critic network takes the minimum value to suppress value overestimation. The network adopts a free speed training mode, and the reward function is composed of a weighted sum of safety, driving efficiency, energy economy, comfort, and artificial potential field obstacle avoidance penalty. The path correction module is used to dynamically calculate the longitudinal and lateral safety half-axis using an elliptical vehicle body contour model and establish a minimum safety distance model. Through the Action Masking mechanism, when the acceleration of the path decision action exceeds the safety reference acceleration, it is replaced by the safety reference acceleration to obtain the corrected path decision action. The trajectory generation module is used to construct a piecewise fifth-order polynomial quadratic programming problem with reference to the discrete path points in the modified path decision action, and solve it to obtain the optimal trajectory that satisfies the constraints of continuity, road boundary and kinematic curvature. The trajectory adjustment module is used to obtain the status of the cooperating vehicles through V2V communication and adjust the aiming distance or desired speed based on the constant time-distance strategy and spacing error; the lead vehicle calculates the desired speed correction based on the number of vehicles in the queue behind and the remaining time of the green light to obtain the target trajectory after the cooperative adjustment. The trajectory driving module is used to dynamically adjust the predicted time-domain output front wheel steering angle of the adaptive MPC lateral controller and optimize the output longitudinal acceleration of the fuzzy PID longitudinal controller online, driving the vehicle to follow the target trajectory.

9. A control device, characterized in that, The device includes: A memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1 to 7.

Images