Lane-constraint-free intersection automatic driving decision-making method based on near-end strategy optimization

Abstract

The invention relates to the technical field of automatic driving, in particular to a lane-constraint-free intersection automatic driving decision-making method based on near-end strategy optimization, and the method comprises the following steps: constructing a typical intersection model without signal lamp control, and setting a lane-constraint-free driving region in a central region of the intersection; designing a driving function of the automatic driving vehicle for guiding the automatic driving vehicle to realize safe and efficient passing under the condition of no lane constraint; a near-end strategy optimization algorithm is designed to construct a strategy probability density function, and the strategy probability density function is sampled to obtain an acceleration variable quantity and a steering angle variable quantity, so that the automatic driving vehicle is controlled to realize lane-constraint-free driving and collision avoidance. According to the method, the automatic driving vehicle can be controlled to realize lane-constraint-free driving in the typical intersection center area without signal lamp control, and the space utilization rate and the traffic capacity of the intersection center area are effectively improved.

Description

Technical Field

[0001] This invention relates to the fields of computer, traffic big data, and autonomous driving trajectory planning, specifically to an autonomous driving decision-making method for lane-constrained intersections based on near-end strategy optimization. Background Technology

[0002] With the development of intelligent connected vehicles and autonomous driving technologies, traffic control in complex urban traffic environments has become a key research focus. Currently, most intersections still rely on traffic lights and fixed lane divisions to organize traffic flow. However, traffic light control methods tend to result in longer waiting times for vehicles at intersections, reducing traffic efficiency; fixed lane divisions limit the space utilization of the central area of ​​the intersection and are difficult to adapt to dynamically changing traffic demands.

[0003] For future autonomous driving scenarios, autonomous and cooperative passage of vehicles at intersections without traffic lights remains a core problem to be solved. Existing research mostly adopts time slot allocation-based or rule-based decision-making methods, but these methods have shortcomings in handling high-dimensional state information, vehicle interaction relationships, and collision avoidance, making it difficult to achieve globally optimal traffic organization results. Especially in multi-lane intersection scenarios, vehicle travel paths are diverse, and traditional methods often cannot balance safety and efficiency.

[0004] Therefore, there is an urgent need for a method that can effectively enable autonomous vehicles to pass through intersections without traffic lights, thereby improving the space utilization of the central area of ​​the intersection and achieving efficient passage while ensuring safety. Summary of the Invention

[0005] The purpose of this invention is to provide an autonomous driving decision-making method for lane-constrained intersections based on near-end strategy optimization. This method can provide real-time driving decisions for autonomous vehicles, enabling them to make full use of the spatial resources in the center area of ​​the intersection, thereby reducing energy consumption and improving the safety of vehicle passage.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] R1 constructs a typical intersection model without traffic light control, and sets up a driving area without lane constraints in the central area of ​​the intersection;

[0008] R2 designs the driving functions for autonomous vehicles to guide them to achieve safe and efficient passage under conditions without lane constraints;

[0009] R3 designs a near-end policy optimization algorithm to construct a policy probability density function, and obtains the acceleration change and steering angle change by sampling the policy probability density function, thereby controlling the autonomous vehicle to achieve lane-free driving and collision avoidance.

[0010] The specific steps of R1 are as follows:

[0011] The lane-free driving area is a shared space within the center area of ​​the intersection. This shared space does not have a fixed driving path and allows autonomous vehicles to adjust their driving path autonomously according to real-time traffic conditions and the location of the target exit, provided that safety is ensured.

[0012] Step R2 specifically involves:

[0013] The driving function of the autonomous vehicle balances efficiency and safety, and specifically includes:

[0014] Distance-progress-driven r l =l t-1 -l t , where l t-1 l t These represent the coordinates (x, y) of the vehicle from the previous time point to the target exit lane at the current time point. o ,y o The Euclidean distance of the vehicle is used to guide the autonomous vehicle to the target exit lane.

[0015] Steering angle progress drive r a =|angle t-1 -angle o |-|angle t -angle o |, where angle t-1 , angle t Used to represent the vehicle's driving direction angle between the previous and current moments, angle o This indicates the direction angle of the target exit lane, and its purpose is to guide autonomous vehicles to align with the target direction angle.

[0016] Energy consumption penalty This represents the integral of the vehicle's acceleration value from the initial moment to the current moment. Its purpose is to avoid frequent acceleration and deceleration of autonomous vehicles, thereby reducing energy consumption.

[0017] Collision Penalty Where σ represents the one-time penalty value given after a collision, the purpose of which is to prevent autonomous vehicles from colliding with each other;

[0018] Collision warning penalty r w =-e -TTCTTC stands for the estimated time of collision between the vehicle and another vehicle. Its purpose is to provide early warning of potential risks and improve the safety of autonomous vehicles.

[0019] Total driving value at time t Here, λ represents the weight of each item, and its purpose is to balance efficiency and security.

[0020] Step R3 specifically involves:

[0021] The near-end policy optimization algorithm trains the neural network by performing gradient descent on the loss function;

[0022] The neural network consists of fully connected layers and activation functions, using the high-dimensional state vector s of the autonomous vehicle. t As input, the output is an estimate of the policy probability density function coefficients β1, β2 and the cumulative driving value V(s) corresponding to the current state. t );

[0023] The expression for the neural network is:

[0024] β1,β2,V(s t ) = FC(ReLU(FC(ReLU(FC(s) t )))))

[0025] Where FC(·) is a fully connected layer and ReLU(·) is the ReLU activation function;

[0026] The high-dimensional state vector s t This includes information about the vehicle itself and other vehicles. The vehicle's information includes its x-coordinate, y-coordinate, driving direction angle, speed scalar v, acceleration scalar α, and the x-coordinate of the target exit lane. o The ordinate of the target exit lane is y. o Target exit lane direction angle o There are a total of eight parameters. The information about other vehicles includes the horizontal coordinate, vertical coordinate, and driving direction angle of the surrounding vehicles, totaling 3n parameters, where n represents the number of surrounding vehicles.

[0027] The loss function is constructed based on:

[0028] loss = -E t [min(r t (θ)A t ,clip(r t (θ), 1-ε, 1+ε)A t ]+E t [(V(s t )-V * ) 2 ]

[0029] Where, r t (θ) represents the ratio of the probability of the new strategy to the probability of the old strategy, clip(·) = min(max(r)). r (θ), 1-ε), 1+ε) represent the clipping operation. Represents target value. The relative difference between good and bad decisions is represented by γ, where γ represents the discount rate and ε represents the trimming coefficient.

[0030] The process of obtaining the changes in acceleration and steering angle is (Δα, Δangle). T = b·[x-(1,1) T ], where b is the coefficient vector, and x is the sample vector sampled from the policy probability density function, the expression of which is:

[0031] Lane-free driving refers to the autonomous vehicle calculating the total driving value based on the current state within each step period T, then using the total driving value to guide the execution of the near-end strategy optimization algorithm to generate a corresponding driving strategy, and finally controlling the autonomous vehicle to execute the strategy. The execution process is as follows:

[0032]

[0033] The lane-constrained intersection autonomous driving decision-making method based on near-end strategy optimization provided in the above technical solution has the following advantages compared with the prior art:

[0034] 1. By setting up a driving zone without lane constraints in the center of the intersection, autonomous vehicles can flexibly adjust their driving paths, making full use of intersection space resources, alleviating traffic congestion, and improving traffic efficiency.

[0035] 2. Based on the near-end strategy optimization algorithm, autonomous vehicles can make decisions based on their own status and information about surrounding vehicles, and adjust their acceleration and driving direction in real time to achieve dynamic response and flexible passage. Attached Figure Description

[0036] Figure 1 This is a flowchart of an autonomous driving decision-making method for lane-constrained intersections based on near-end strategy optimization, as described in an embodiment of the present invention.

[0037] Figure 2 This is a schematic diagram illustrating the driving process of an autonomous vehicle without lane constraints, as described in an embodiment of the present invention. Detailed Implementation

[0038] To make the objectives and advantages of this invention clearer, the invention will be specifically described below with reference to embodiments. It should be understood that the following text is merely used to describe one or more specific embodiments of the invention and does not strictly limit the scope of protection specifically claimed by the invention.

[0039] Next, we will combine the appendix Figure 1 Appendix Figure 2 The present invention will be further analyzed and explained.

[0040] like Figure 1 As shown in the figure, one embodiment of the lane-constrained intersection autonomous driving decision-making method based on near-end strategy optimization in this application has the following specific steps:

[0041] R1 constructs a typical intersection model without traffic light control, and sets up a driving area without lane constraints in the central area of ​​the intersection;

[0042] The lane-free driving area is a shared space within the center area of ​​the intersection. This shared space does not have a fixed driving path and allows autonomous vehicles to adjust their driving path autonomously according to real-time traffic conditions and the location of the target exit, provided that safety is ensured.

[0043] R2 designs the driving functions for autonomous vehicles to guide them to achieve safe and efficient passage under conditions without lane constraints;

[0044] The driving function of the autonomous vehicle balances efficiency and safety, and specifically includes:

[0045] Distance-progress-driven r l =l t-1 -l t , where l t-1 l t These represent the coordinates (x, y) of the vehicle from the previous time point to the target exit lane at the current time point. o ,y o The Euclidean distance of the vehicle is used to guide the autonomous vehicle to the target exit lane.

[0046] Steering angle progress drive r a =|angle t-1 -angle o |-|angle t -angle o |, where angle t-1 , angle t Used to represent the vehicle's driving direction angle between the previous and current moments, angle o This indicates the direction angle of the target exit lane, and its purpose is to guide autonomous vehicles to align with the target direction angle.

[0047] Energy consumption penalty This represents the integral of the vehicle's acceleration value from the initial moment to the current moment. Its purpose is to avoid frequent acceleration and deceleration of autonomous vehicles, thereby reducing energy consumption.

[0048] Collision Penalty Where σ represents the one-time penalty value given after a collision, the purpose of which is to prevent autonomous vehicles from colliding with each other;

[0049] Collision warning penalty r w =-e -TTC TTC stands for the estimated time of collision between the vehicle and another vehicle. Its purpose is to provide early warning of potential risks and improve the safety of autonomous vehicles.

[0050] Total driving value at time t Here, λ represents the weight of each item, and its purpose is to balance efficiency and security.

[0051] R3 designs a near-end policy optimization algorithm to construct a policy probability density function, and obtains the acceleration change Δα and steering angle change Δangle by sampling the policy probability density function, thereby controlling the autonomous vehicle to achieve lane-free driving and collision avoidance.

[0052] The near-end policy optimization algorithm trains the neural network by performing gradient descent on the loss function;

[0053] The neural network consists of fully connected layers and activation functions, using the high-dimensional state vector s of the autonomous vehicle. t As input, the output is an estimate of the policy probability density function coefficients β1, β2 and the cumulative driving value V(s) corresponding to the current state. t );

[0054] The expression for the neural network is:

[0055] β1,β2,V(s t ) = FC(ReLU(FC(ReLU(FC(s) t )))))

[0056] Where FC(·) is a fully connected layer and ReLU(·) is the ReLU activation function;

[0057] The high-dimensional state vector s t This includes information about the vehicle itself and other vehicles. The vehicle's information includes its x-coordinate, y-coordinate, driving direction angle, speed scalar v, acceleration scalar α, and the x-coordinate of the target exit lane. o The ordinate of the target exit lane is y. o Target exit lane direction angleo There are a total of eight parameters. The information about other vehicles includes the horizontal coordinate, vertical coordinate, and driving direction angle of the surrounding vehicles, totaling 3n parameters, where n represents the number of surrounding vehicles.

[0058] The loss function is constructed based on:

[0059] loss = -E t [min(r t (θ)A t ,clip(r t (θ), 1-ε, 1+ε)A t ]+E t [(V(s t )-V * ) 2 ]

[0060] Where, r t (θ) represents the ratio of the probability of the new strategy to the probability of the old strategy, clip(·) = min(max(r)). t (θ), 1-ε), 1+ε) represent the clipping operation. Represents target value. The relative difference between good and bad decisions is represented by γ, which represents the discount rate, ranging from 0.9 to 0.99, and ε, which represents the trimming coefficient, ranging from 0 to 0.5.

[0061] The process of obtaining the changes in acceleration and steering angle is (Δα, Δangle). T = b·[x-(1,1) T ], where b is a coefficient vector, and its value can be b = (0.1, 5). T To ensure the output is reasonable, x is a sample vector sampled from the policy probability density function, which is expressed as:

[0062]

[0063] like Figure 2 As shown, lane-free driving refers to the autonomous vehicle calculating the total driving value based on the current state within each step period T, then using the total driving value to guide the execution of the near-end strategy optimization algorithm, thereby generating a corresponding driving strategy, and finally controlling the autonomous vehicle to execute the strategy. The execution process is as follows:

[0064]

[0065] The step period T can be 0.1 seconds.

[0066] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. For those skilled in the art, after learning the contents described in the present invention, several equivalent changes and substitutions can be made without departing from the principle of the present invention. These equivalent changes and substitutions should also be considered to fall within the protection scope of the present invention.

Claims

1. A decision-making method for lane-constrained intersections based on near-end strategy optimization, characterized in that, Includes the following steps: R1 constructs a typical intersection model without traffic light control, and sets up a driving area without lane constraints in the central area of ​​the intersection; R2 designs the driving functions for autonomous vehicles to guide them to achieve safe and efficient passage under conditions without lane constraints; R3 designs a near-end policy optimization algorithm to construct a policy probability density function, and obtains the acceleration change and steering angle change by sampling the policy probability density function, thereby controlling the autonomous vehicle to achieve lane-free driving and collision avoidance.

2. The lane-constrained intersection autonomous driving decision-making method based on near-end strategy optimization according to claim 1, characterized in that, The lane-free driving area is a shared space within the center area of ​​the intersection. This shared space does not have a fixed driving path and allows autonomous vehicles to adjust their driving path autonomously based on real-time traffic conditions and the location of the target exit, provided that safety is ensured.

3. The lane-constrained intersection autonomous driving decision-making method based on near-end strategy optimization according to claim 1, characterized in that, The driving function of the autonomous vehicle balances efficiency and safety, and specifically includes: Distance-progress-driven r l =l t-1 -l t , where l t-1 l t These represent the coordinates (x, y) of the vehicle from the previous time point to the target exit lane at the current time point. o ,y o The Euclidean distance of ). Steering angle progress drive r a =|angle t-1 -angle o |-|angle t -angle o |, where angle t-1 , angle t Used to represent the vehicle's driving direction angle between the previous and current moments, angle o Indicates the direction angle of the target exit lane; Energy consumption penalty This represents the integral of the vehicle's acceleration from the initial moment to the current moment; Collision Penalty Where σ represents the one-time penalty value given after a collision; Collision warning penalty r w =-e -TTC TTC stands for the estimated time of collision between the vehicle and another vehicle. Total driving value at time t Where λ represents the weight of each item.

4. The lane-constrained intersection autonomous driving decision-making method based on near-end strategy optimization according to claim 1, characterized in that, The near-end policy optimization algorithm trains the neural network by performing gradient descent on the loss function; The neural network consists of fully connected layers and activation functions, using the high-dimensional state vector s of the autonomous vehicle. t As input, the output is an estimate of the policy probability density function coefficients β1, β2 and the cumulative driving value V(s) corresponding to the current state. t ); The high-dimensional state vector s t This includes information about the vehicle itself and other vehicles. The vehicle's information includes its x-coordinate, y-coordinate, driving direction angle, speed scalar v, acceleration scalar α, and the x-coordinate of the target exit lane. o The ordinate of the target exit lane is y. o Target exit lane direction angle o There are a total of eight parameters. The information about other vehicles includes the horizontal coordinate, vertical coordinate, and driving direction angle of the surrounding vehicles, totaling 3n parameters, where n represents the number of surrounding vehicles. The loss function is constructed based on: loss=-E t [min(r t (i)A t ,clip(r t (θ),1-ε,1+ε)A t )]+E t [(V(s t )-V * ) 2 ] Where, r t (θ) represents the ratio of the new policy probability to the old policy probability, clip(·) represents the clipping operation, V * Representing target value, A t The relative difference between good and bad decisions is represented by ε, which represents the clipping coefficient. The process of obtaining the changes in acceleration and steering angle is (Δα, Δangle). T = b·[x-(1,1) T ], where b is the coefficient vector, and x is the sample vector sampled from the policy probability density function, the expression of which is: Lane-free driving refers to the autonomous vehicle calculating the total driving value based on the current state within each step period T, then using the total driving value to guide the execution of the near-end strategy optimization algorithm to generate a corresponding driving strategy, and finally controlling the autonomous vehicle to execute the strategy. The execution process is as follows:

Images