Automatic driving decision-making method for intersection without lane constraint based on proximal policy optimization

Abstract

The application relates to the technical field of automatic driving, in particular to an automatic driving decision-making method for a lane-constraint-free intersection based on proximal policy optimization, which comprises the following steps: a typical intersection model without signal control is constructed, and a lane-constraint-free driving area is arranged in the central area of the intersection; a driving function of an automatic driving vehicle is designed to guide the automatic driving vehicle to realize safe and efficient passing under the condition of lane constraint; a proximal policy optimization algorithm is designed to construct a policy probability density function, and an acceleration change and a steering angle change are obtained by sampling the policy probability density function, so that the automatic driving vehicle realizes lane-constraint-free driving and collision avoidance. The application can control the automatic driving vehicle to realize lane-constraint-free driving in the central area of the typical intersection without signal control, and effectively improves the space utilization and passing capacity of the central area of the intersection.

Description

Technical Field

[0001] This invention relates to the fields of computer, traffic big data, and autonomous driving trajectory planning, specifically to an autonomous driving decision-making method for lane-constrained intersections based on near-end strategy optimization. Background Technology

[0002] With the development of intelligent connected vehicles and autonomous driving technologies, traffic control in complex urban traffic environments has become a key research focus. Currently, most intersections still rely on traffic lights and fixed lane divisions to organize traffic flow. However, traffic light control methods tend to result in longer waiting times for vehicles at intersections, reducing traffic efficiency; fixed lane divisions limit the space utilization of the central area of ​​the intersection and are difficult to adapt to dynamically changing traffic demands.

[0003] For future autonomous driving scenarios, autonomous and cooperative passage of vehicles at intersections without traffic lights remains a core problem to be solved. Existing research mostly adopts time slot allocation-based or rule-based decision-making methods, but these methods have shortcomings in handling high-dimensional state information, vehicle interaction relationships, and collision avoidance, making it difficult to achieve globally optimal traffic organization results. Especially in multi-lane intersection scenarios, vehicle travel paths are diverse, and traditional methods often cannot balance safety and efficiency.

[0004] Therefore, there is an urgent need for a method that can effectively enable autonomous vehicles to pass through intersections without traffic lights, thereby improving the space utilization of the central area of ​​the intersection and achieving efficient passage while ensuring safety. Summary of the Invention

[0005] The purpose of this invention is to provide an autonomous driving decision-making method for lane-constrained intersections based on near-end strategy optimization. This method can provide real-time driving decisions for autonomous vehicles, enabling them to make full use of the spatial resources in the center area of ​​the intersection, thereby reducing energy consumption and improving the safety of vehicle passage.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] R1 constructs a typical intersection model without traffic light control, and sets up a driving area without lane constraints in the central area of ​​the intersection;

[0008] R2 designs the driving functions for autonomous vehicles to guide them to achieve safe and efficient passage under lane-constraint-free conditions. The driving functions include:

[0009] Distance-progress drive ,in , These represent the coordinates of the vehicle from the previous time point to the target exit lane at the current time point. The Euclidean distance;

[0010] Steering angle progress drive ,in , Used to represent the vehicle's driving direction angle between the previous moment and the current moment. Indicates the direction angle of the target exit lane;

[0011] Energy consumption penalty ,in This represents the vehicle's acceleration, and this term is used to represent the integral of the vehicle's acceleration value from the initial moment to the current moment;

[0012] Collision Penalty ,in This indicates the one-time penalty value applied after a collision.

[0013] Collision warning penalty ,in Indicates the estimated time of collision between the vehicle and another vehicle;

[0014] Total driving value at time t ,in Represents the weight of each item;

[0015] R3 designs a near-end policy optimization algorithm to construct a policy probability density function, and obtains the acceleration change and steering angle change by sampling the policy probability density function, thereby controlling the autonomous vehicle to achieve lane-free driving and collision avoidance.

[0016] The near-end policy optimization algorithm trains the neural network by performing gradient descent on the loss function;

[0017] The neural network consists of fully connected layers and activation functions, using the high-dimensional state vector of the autonomous vehicle. As input, the output is the coefficient of the policy probability density function corresponding to the current state. Estimation of cumulative driving values ;

[0018] The expression for the neural network is:

[0019]

[0020] in, It is a fully connected layer. It is the ReLU activation function;

[0021] The high-dimensional state vector This includes information about the vehicle itself and other vehicles. The information about the vehicle itself includes its x-coordinate, y-coordinate, driving direction angle, velocity scalar v, and acceleration scalar. x-coordinate of the target exit lane Longitudinal coordinates of the target exit lane Target exit lane direction angle There are a total of eight parameters. The information about other vehicles includes the horizontal coordinate, vertical coordinate, and driving direction angle of the surrounding vehicles, totaling 3n parameters, where n represents the number of surrounding vehicles.

[0022] The loss function is constructed based on:

[0023]

[0024] in, It refers to the ratio of the probability of the new strategy to the probability of the old strategy. This represents a cropping operation. Represents target value. Represents the relative difference between good and bad decisions. Represents the discount rate. Represents the cutting factor;

[0025] The process of obtaining the change in acceleration and the change in steering angle is as follows: ,in It is a coefficient vector. These are sample vectors sampled from the policy probability density function, which is expressed as follows:

[0026] ;

[0027] Lane-free driving refers to the autonomous vehicle calculating the total driving value based on the current state within each step period T, then using the total driving value to guide the execution of the near-end strategy optimization algorithm to generate a corresponding driving strategy, and finally controlling the autonomous vehicle to execute the strategy. The execution process is as follows:

[0028] .

[0029] Step R1 specifically involves:

[0030] The lane-free driving area is a shared space within the center area of ​​the intersection. This shared space does not have a fixed driving path and allows autonomous vehicles to adjust their driving path autonomously based on real-time traffic conditions and the location of the target exit, provided that safety is ensured.

[0031] The lane-constrained intersection autonomous driving decision-making method based on near-end strategy optimization provided in the above technical solution has the following advantages compared with the prior art:

[0032] 1. By setting up a driving zone without lane constraints in the center of the intersection, autonomous vehicles can flexibly adjust their driving paths, making full use of intersection space resources, alleviating traffic congestion, and improving traffic efficiency.

[0033] 2. Based on the near-end strategy optimization algorithm, autonomous vehicles can make decisions based on their own status and information about surrounding vehicles, and adjust their acceleration and driving direction in real time to achieve dynamic response and flexible passage. Attached Figure Description

[0034] Figure 1 This is a flowchart of an autonomous driving decision-making method for lane-constrained intersections based on near-end strategy optimization, as described in an embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram illustrating the driving process of an autonomous vehicle without lane constraints, as described in an embodiment of the present invention. Detailed Implementation

[0036] To make the objectives and advantages of this invention clearer, the invention will be specifically described below with reference to embodiments. It should be understood that the following text is merely used to describe one or more specific embodiments of the invention and does not strictly limit the scope of protection specifically claimed by the invention.

[0037] Next, we will combine the appendix Figure 1 Appendix Figure 2 The present invention will be further analyzed and explained.

[0038] like Figure 1 As shown in the figure, one embodiment of the lane-constrained intersection autonomous driving decision-making method based on near-end strategy optimization in this application has the following specific steps:

[0039] R1 constructs a typical intersection model without traffic light control, and sets up a driving area without lane constraints in the central area of ​​the intersection;

[0040] The lane-free driving area is a shared space within the center area of ​​the intersection. This shared space does not have a fixed driving path and allows autonomous vehicles to adjust their driving path autonomously based on real-time traffic conditions and the location of the target exit, provided that safety is ensured.

[0041] R2 designs the driving functions for autonomous vehicles to guide them to achieve safe and efficient passage under lane-constraint-free conditions. The driving functions include:

[0042] Distance-progress drive ,in , These represent the coordinates of the vehicle from the previous time point to the target exit lane at the current time point. The Euclidean distance is used to guide autonomous vehicles toward the target exit lane;

[0043] Steering angle progress drive ,in , Used to represent the vehicle's driving direction angle between the previous moment and the current moment. This indicates the direction angle of the target exit lane, and its purpose is to guide autonomous vehicles to align with the target direction angle.

[0044] Energy consumption penalty ,in This represents the vehicle's acceleration. This item is used to represent the integral of the vehicle's acceleration value from the initial moment to the current moment. Its purpose is to avoid frequent acceleration and deceleration of autonomous vehicles, thereby reducing energy consumption.

[0045] Collision Penalty ,in This represents a one-time penalty value applied after a collision, with the aim of preventing collisions between autonomous vehicles.

[0046] Collision warning penalty ,in It indicates the estimated collision time between the vehicle and other vehicles, with the aim of providing early warning of potential risks and improving the safety of autonomous vehicles;

[0047] Total driving value at time t ,in This represents the weight of each item, with the aim of balancing efficiency and security.

[0048] R3 designs a near-end policy optimization algorithm to construct a policy probability density function, and obtains the acceleration change and steering angle change by sampling the policy probability density function, thereby controlling the autonomous vehicle to achieve lane-free driving and collision avoidance.

[0049] The near-end policy optimization algorithm trains the neural network by performing gradient descent on the loss function;

[0050] The neural network consists of fully connected layers and activation functions, using the high-dimensional state vector of the autonomous vehicle. As input, the output is the coefficient of the policy probability density function corresponding to the current state. Estimation of cumulative driving values ;

[0051] The expression for the neural network is:

[0052]

[0053] in, It is a fully connected layer. It is the ReLU activation function;

[0054] The high-dimensional state vector This includes information about the vehicle itself and other vehicles. The information about the vehicle itself includes its x-coordinate, y-coordinate, driving direction angle, velocity scalar v, and acceleration scalar. x-coordinate of the target exit lane Longitudinal coordinates of the target exit lane Target exit lane direction angle There are a total of eight parameters. The information about other vehicles includes the horizontal coordinate, vertical coordinate, and driving direction angle of the surrounding vehicles, totaling 3n parameters, where n represents the number of surrounding vehicles.

[0055] The loss function is constructed based on:

[0056]

[0057] in, It refers to the ratio of the probability of the new strategy to the probability of the old strategy. This represents a cropping operation. Represents target value. Represents the relative difference between good and bad decisions. This represents the discount rate, and its value ranges from 0.9 to 0.99. This represents the cropping factor, which ranges from 0 to 0.5.

[0058] The process of obtaining the change in acceleration and the change in steering angle is as follows: ,in It is a coefficient vector, and its values ​​can be... To ensure that the output results are reasonable, These are sample vectors sampled from the policy probability density function, which is expressed as:

[0059] ;

[0060] like Figure 2 As shown, lane-free driving refers to the autonomous vehicle calculating the total driving value based on the current state within each step period T, then using the total driving value to guide the execution of the near-end strategy optimization algorithm, thereby generating a corresponding driving strategy, and finally controlling the autonomous vehicle to execute the strategy. The strategy execution process is as follows:

[0061]

[0062] The step period T can be 0.1 seconds.

[0063] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. For those skilled in the art, after learning the contents described in the present invention, several equivalent changes and substitutions can be made without departing from the principle of the present invention. These equivalent changes and substitutions should also be considered to fall within the protection scope of the present invention.

Claims

1. A decision-making method for lane-constrained intersections based on near-end strategy optimization, characterized in that, Includes the following steps: R1 constructs a typical intersection model without traffic light control, and sets up a driving area without lane constraints in the central area of ​​the intersection; R2 designs the driving functions for autonomous vehicles to guide them to achieve safe and efficient passage under lane-constraint-free conditions. The driving functions include: Distance-progress drive ,in , These represent the coordinates of the vehicle from the previous time point to the target exit lane at the current time point. The Euclidean distance; Steering angle progress drive ,in , Used to represent the vehicle's driving direction angle between the previous moment and the current moment. Indicates the direction angle of the target exit lane; Energy consumption penalty ,in This represents the vehicle's acceleration, and this term is used to represent the integral of the vehicle's acceleration value from the initial moment to the current moment; Collision Penalty ,in This indicates the one-time penalty value applied after a collision. Collision warning penalty ,in Indicates the estimated time of collision between the vehicle and another vehicle; Total driving value at time t ,in Represents the weight of each item; R3 designs a near-end policy optimization algorithm to construct a policy probability density function, and obtains the acceleration change and steering angle change by sampling the policy probability density function, thereby controlling the autonomous vehicle to achieve lane-free driving and collision avoidance. The near-end policy optimization algorithm trains the neural network by performing gradient descent on the loss function; The neural network consists of fully connected layers and activation functions, using the high-dimensional state vector of the autonomous vehicle. As input, the output is the coefficient of the policy probability density function corresponding to the current state. Estimation of cumulative driving values ; The high-dimensional state vector This includes information about the vehicle itself and other vehicles. The information about the vehicle itself includes its x-coordinate, y-coordinate, driving direction angle, velocity scalar v, and acceleration scalar. x-coordinate of the target exit lane Longitudinal coordinates of the target exit lane Target exit lane direction angle There are a total of eight parameters. The information about other vehicles includes the horizontal coordinate, vertical coordinate, and driving direction angle of the surrounding vehicles, totaling 3n parameters, where n represents the number of surrounding vehicles. The loss function is constructed based on: in, It refers to the ratio of the probability of the new strategy to the probability of the old strategy. This represents a cropping operation. Represents target value. Represents the relative difference between good and bad decisions. Represents the cutting factor; The process of obtaining the change in acceleration and the change in steering angle is as follows: ,in It is a coefficient vector. These are sample vectors sampled from the policy probability density function, which is expressed as follows: ； Lane-free driving refers to the autonomous vehicle calculating the total driving value based on the current state within each step period T, then using the total driving value to guide the execution of the near-end strategy optimization algorithm to generate a corresponding driving strategy, and finally controlling the autonomous vehicle to execute the strategy. The execution process is as follows: 。 2. The lane-constrained intersection autonomous driving decision-making method based on near-end strategy optimization according to claim 1, characterized in that, The lane-free driving area is a shared space within the center area of ​​the intersection. This shared space does not have a fixed driving path and allows autonomous vehicles to adjust their driving path autonomously based on real-time traffic conditions and the location of the target exit, provided that safety is ensured.

Images