Side collision avoidance driving decision-making method for commercial vehicles in low-speed urban environment

Abstract

The present invention discloses a method for making decisions on lateral collision avoidance driving for commercial vehicles in a low-speed urban environment. First, a hardware-in-the-loop driving simulation platform is used to construct an urban traffic scene, and safe driving behaviors under different driving conditions and driving conditions are simulated and collected. Secondly, a data set aggregation algorithm is used to simulate the safe driving behavior of human drivers through imitation learning. Finally, a proximal strategy optimization algorithm is used to further learn the lateral collision avoidance strategy through unsupervised learning, so as to achieve high-level decision output on the lateral collision avoidance driving behavior of commercial vehicles. The method proposed in the present invention can simulate the safe driving behavior of human drivers, and takes into account the impact of factors such as visual blind spots and types of traffic participants on driving safety, providing a more reasonable and effective collision avoidance driving strategy for large commercial vehicles, and realizing lateral collision avoidance driving decisions for commercial vehicles in a low-speed urban environment.

Description

Technical Field

[0001] The invention relates to a driving decision method for an operating vehicle, and in particular to a lateral collision avoidance driving decision method for an operating vehicle in a low-speed urban environment, and belongs to the technical field of automobile safety. Background Art

[0002] Among traffic accidents involving commercial vehicles in urban environments, accidents caused by visual blind spots account for the highest proportion. The reason is that due to the influence of many factors such as the long body of the commercial vehicle, the high driving position, the large difference between the inner and outer wheels, and the large right turning radius, when the vehicle turns, especially turns to the right, a crescent-shaped dynamic visual blind spot will be formed, which is very likely to collide with or even run over pedestrians and non-motor vehicles in the blind spot. Therefore, the right side of the commercial vehicle is one of the most dangerous areas of all visual blind spots, and it is also the main area for serious safety accidents such as lateral collisions and runovers. In an urban traffic environment with a large number of traffic participants, especially when the vehicle is driving at a low speed (starting, turning right, etc.), how to avoid lateral collisions caused by the visual blind spots of commercial vehicles has become a core issue in ensuring road transportation safety.

[0003] If the driver can be warned before a collision or runover accident occurs and be reminded to slow down or turn, the frequency of traffic accidents caused by visual blind spots can be greatly reduced or the damage caused can be mitigated. Therefore, in the low-speed urban environment with mixed traffic of vehicles and non-vehicles, studying effective and reliable lateral collision avoidance driving decision-making methods for commercial vehicles is important for reducing the frequency of lateral collisions and improving road traffic safety.

[0004] In general, although the current methods can play a certain warning role, they still have certain shortcomings in terms of the effectiveness and reliability of lateral collision avoidance, and do not involve research on lateral collision avoidance driving strategies that provide specific driving suggestions such as driving speed and whether to turn. In particular, there is a lack of research on effective and reliable lateral collision avoidance driving decision-making methods for commercial vehicles in low-speed urban environments. Summary of the invention

[0005] Purpose of the invention: In order to realize the lateral collision avoidance driving decision of commercial vehicles in low-speed urban environments and ensure the safety of vehicle operation, the present invention proposes a lateral collision avoidance driving decision method for commercial vehicles in low-speed urban environments for large commercial vehicles such as large passenger buses, trucks, and urban logistics vehicles. This method can simulate the safe driving behavior of human drivers, and takes into account the impact of factors such as driving conditions, visual blind spots, and types of traffic participants on driving safety. It can provide more reasonable and effective collision avoidance driving strategies for large commercial vehicles, thereby ensuring the safety of commercial vehicle operation. At the same time, this method does not need to consider complex vehicle dynamics equations and vehicle body parameters. The calculation method is simple and clear, and the lateral collision avoidance decision strategy of large commercial vehicles can be output in real time. The sensors used are low in cost, which is convenient for large-scale promotion.

[0006] Technical solution: To achieve the purpose of the present invention, the technical solution adopted by the present invention is: a method for lateral collision avoidance driving decision-making of commercial vehicles in low-speed urban environments. First, a hardware-in-the-loop driving simulation platform is used to construct an urban traffic scene to simulate and collect safe driving behaviors under different driving conditions and driving conditions. Secondly, through imitation learning, a data set aggregation algorithm is used to simulate the safe driving behavior of human drivers. Finally, through unsupervised learning, a proximal strategy optimization algorithm is used to further learn the lateral collision avoidance strategy to achieve high-level decision output for the lateral collision avoidance driving behavior of commercial vehicles. The method proposed in the present invention can simulate the safe driving behavior of human drivers, and takes into account the impact of factors such as visual blind spots and types of traffic participants on driving safety, providing large commercial vehicles with a more reasonable and effective collision avoidance driving strategy, and realizing lateral collision avoidance driving decisions for commercial vehicles in low-speed urban environments. Specifically includes the following three steps:

[0007] Step 1: Use the driving simulation platform to build urban traffic scenarios

[0008] In order to reduce the frequency of lateral collision accidents of commercial vehicles caused by factors such as visual blind spots and improve the driving safety of commercial vehicles, the present invention proposes a lateral collision avoidance driving decision-making method for commercial vehicles in a low-speed urban environment. The applicable scenario is: when an commercial vehicle is driving in a low-speed urban environment and there are other traffic participants (motor vehicles, non-motor vehicles or pedestrians) on its left or right side, in order to avoid lateral collision accidents, an effective and reliable lateral collision avoidance driving strategy should be provided to the driver.

[0009] According to the scenario described above, firstly, the driving simulation platform is used to construct the urban traffic scenario, covering straight roads, curves and intersections, and setting up highly random traffic flows and traffic participants. Secondly, multiple drivers use the driving simulator (steering wheel, accelerator and brake pedal) to control the operating vehicles and collect safe driving behaviors under eight driving conditions, including lane change, lane keeping, vehicle following, left turn, right turn, acceleration, deceleration and constant speed. Finally, based on the collected safe driving behaviors, a safe driving behavior database D is constructed.

[0010] Step 2: Use imitation learning methods to simulate the driver's safe driving behavior

[0011] As a more advanced behavior cloning method, Dataset Aggregation (DAgger) can actively select strategies from the safe driving behavior database and more easily match the safe driving behavior of human drivers in the subsequent training process, which has a higher imitation learning ability. Therefore, the present invention uses the DAgger algorithm to simulate the safe driving behavior of human drivers. The safe driving behavior database D continuously aggregates new datasets D at each time step i. i , the specific training process is as follows:

[0012] Sub-step 1: Initialize parameter φ;

[0013] Sub-step 2: Initialize strategy π;

[0014] Sub-step 3: Perform a loop of N time steps, each loop includes sub-steps 3.1 to 3.5, specifically:

[0015] Sub-step 3.1: Update the strategy using the following formula:

[0016]

[0017] In the formula, π i represents the strategy at the i-th moment, π * represents the expert strategy, β i represents the parameters of the soft update of the strategy at the i-th iteration, represents the optimal strategy at the i-th moment;

[0018] Substep 3.2: Using π i Sampling expert trajectories;

[0019] Sub-step 3.3: Output is given by π i The data set D consists of the strategies visited and the actions given by the experts i ={(S t ,π * (S t ))}, St represents the state space at time t;

[0020] Sub-step 3.4: Aggregate the dataset: D←D∪D i ;

[0021] Sub-step 3.5: Train the policy on dataset D in represents the optimal strategy at time i+1;

[0022] Sub-step 4: Finally, return to the optimal strategy at time N+1

[0023] Step 3: Use unsupervised learning methods to further learn collision avoidance strategies

[0024] In actual driving decision-making tasks, driving decisions based on imitation learning lack sufficient generalization capabilities and are difficult to effectively and accurately handle driving conditions and driving conditions that are not covered in the safe driving behavior database. In order to further improve the effectiveness and reliability of lateral collision avoidance decisions, the decision network needs to be further trained. As an unsupervised learning method, deep reinforcement learning can obtain an understanding of the traffic environment through continuous exploration and trial and error, and use the rewards of environmental feedback to guide the improvement of the policy network, thereby maximizing the return. The proximal policy optimization algorithm draws on the trust region policy optimization algorithm and achieves a new balance between sampling efficiency, algorithm performance, and the complexity of implementation and debugging by using first-order optimization. Therefore, the present invention uses the proximal policy optimization algorithm to construct an anti-collision decision model, and trains the anti-collision decision model based on step two.

[0025] First, the side collision avoidance decision problem of commercial vehicles is converted into a Markov decision process under a certain reward function, which can be described as (S, A, P, R). Among them, S is the state space, A is the driving action, P represents the state transition probability caused by the uncertainty of the target vehicle's motion, and R is the reward function. Secondly, the basic parameters of the Markov decision process are defined, specifically:

[0026] (1) Establishing the state space

[0027] First, the state space is constructed using the motion state information of the vehicle and the relative motion state information of the vehicle and surrounding traffic participants:

[0028]

[0029] In the formula, p x ,p y Respectively represent the lateral position and longitudinal position of the vehicle, in meters, v x ,v yThey represent the lateral speed and longitudinal speed of the vehicle, in meters per second, and a x ,a y They represent the lateral acceleration and longitudinal acceleration of the vehicle, respectively, in meters per second squared, and θ s Indicates the heading angle of the vehicle in degrees. Respectively represent the relative distance and relative speed between the vehicle and the jth traffic participant around it, in meters and meters per second, respectively. Among them, j = 1, 2, 3, 4, 5, 6, respectively represent the traffic participants in front, left front, left rear, rear, right rear, and right front. Considering that the number of traffic participants in actual traffic scenes is not fixed, when the sensor observes i (i is less than j) traffic participants, the last j rows of the state space are filled with zeros.

[0030] (2) Establishing the action space

[0031] In order to output high-level driving decisions, the present invention defines the action space as discrete lateral actions and longitudinal actions.

[0032] A t =[a1,a2,a3,a4,a5,a6] (3)

[0033] In the formula, A t represents the action space at time t, a1, a2, a3 represent left turn, right turn and straight ahead respectively, a4, a5, a6 represent acceleration, deceleration and keeping the speed constant respectively.

[0034] (3) Establishing the reward function

[0035] In order to quantitatively evaluate the quality of the collision avoidance strategy, the present invention establishes a collision avoidance reward function that takes into account the impact of traffic participant types on driving safety:

[0036]

[0037] In the formula, R t represents the reward function at time t, x min_1 ,x min_2 represents the lateral safety distance threshold, in meters. In the present invention, x min_1 =2,x min_2 =2.5.

[0038] In addition, negative feedback is applied to the decision that causes a side collision, that is, when the output decision strategy leads to a side collision, the reward value obtained at the current moment is subtracted by 50.

[0039] Secondly, the constructed collision avoidance decision model is trained, which includes the following sub-steps:

[0040] Sub-step 1: Initialize the policy parameters θ0 and value function parameters φ0;

[0041] Sub-step 2: Perform a loop of T time steps, each loop includes sub-steps 2.1 to 2.5, specifically:

[0042] Sub-step 2.1: Run policy π in the environment k =π(θ k );

[0043] Sub-step 2.2: Calculate the optimal reward value at time t

[0044] Sub-step 2.3: Based on the current value function Compute an estimate of the advantage function

[0045] Sub-step 2.4: Update the strategy using the following formula:

[0046]

[0047] In the formula, θ k+1 represents the policy network parameters at time k+1, ε represents the hyperparameter, and π θ represents a policy network with parameter θ, and clip(·) represents a truncation function. Truncated in [1-ε,1+ε], τ represents the hyperparameter that determines the soft update amplitude, argmax(·) represents the variable that maximizes the objective function, Represents the advantage value of the state-action pair.

[0048] Sub-step 2.5: Update the value function using the following formula:

[0049]

[0050] In the formula, φ k+1 represents the value function parameter at time k+1, V φ (S t ) represents the state space S t The value function below.

[0051] Finally, after the training of the collision avoidance decision model is completed, the motion state information of the vehicle and the relative motion state information of the vehicle and surrounding traffic participants are input into the collision avoidance decision model, and driving suggestions such as acceleration, deceleration, and lane change can be output, thus realizing effective and reliable lateral collision avoidance driving decisions for large commercial vehicles.

[0052] Beneficial effects: Compared with the general driving decision-making method, the method proposed by the present invention is more effective and reliable, which is specifically reflected in:

[0053] (1) The method proposed in the present invention can simulate the safe driving behavior of human drivers, provide a more reasonable and safe lateral collision avoidance decision-making strategy for large commercial vehicles, realize the lateral collision avoidance driving decision of commercial vehicles in low-speed urban environments, and ensure the operating safety of commercial vehicles.

[0054] (2) The method proposed in the present invention comprehensively considers the impact of factors such as driving conditions, visual blind spots, and types of traffic participants on driving safety, and sets a refined reward function for different types of traffic participants, thereby realizing lateral collision avoidance driving decisions under different driving conditions and further improving the effectiveness and reliability of the decision.

[0055] (3) The method proposed in the present invention does not need to consider complex vehicle dynamics equations and vehicle body parameters. The calculation method is simple and clear, and can output the side collision avoidance strategy of large-scale operating vehicles in real time. The sensor cost used is low, which is convenient for large-scale promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is the technical roadmap of the present invention. DETAILED DESCRIPTION

[0057] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0058] The present invention proposes a vehicle lateral collision avoidance driving decision-making method in a low-speed urban environment for large commercial vehicles such as large passenger buses, trucks, and urban logistics vehicles. First, a hardware-in-the-loop driving simulation platform is used to construct an urban traffic scene to simulate and collect safe driving behaviors under different driving conditions and driving conditions. Secondly, through imitation learning, a data set aggregation algorithm is used to simulate the safe driving behavior of human drivers. Finally, through unsupervised learning, a proximal strategy optimization algorithm is used to further learn the lateral collision avoidance strategy to achieve high-level decision output for the lateral collision avoidance driving behavior of commercial vehicles. The method proposed in the present invention can simulate the safe driving behavior of human drivers, and takes into account the impact of factors such as visual blind spots and types of traffic participants on driving safety, providing large commercial vehicles with a more reasonable and effective collision avoidance driving strategy, and realizing the lateral collision avoidance driving decision of commercial vehicles in a low-speed urban environment. The technical route of the present invention is as follows: Figure 1 As shown, the specific steps are as follows:

[0059] Step 1: Use the driving simulation platform to build urban traffic scenarios

[0060] In order to reduce the frequency of lateral collision accidents of commercial vehicles caused by factors such as visual blind spots and improve the driving safety of commercial vehicles, the present invention proposes a lateral collision avoidance driving decision-making method for commercial vehicles in a low-speed urban environment. The applicable scenario is: when an commercial vehicle is driving in a low-speed urban environment and there are other traffic participants (motor vehicles, non-motor vehicles or pedestrians) on its left or right side, in order to avoid lateral collision accidents, an effective and reliable lateral collision avoidance driving strategy should be provided to the driver.

[0061] According to the scenario described above, firstly, the driving simulation platform is used to construct the urban traffic scenario, covering straight roads, curves and intersections, and setting up highly random traffic flows and traffic participants. Secondly, multiple drivers use the driving simulator (steering wheel, accelerator and brake pedal) to control the operating vehicles and collect safe driving behaviors under eight driving conditions, including lane change, lane keeping, vehicle following, left turn, right turn, acceleration, deceleration and constant speed. Finally, based on the collected safe driving behaviors, a safe driving behavior database D is constructed.

[0062] Step 2: Use imitation learning methods to simulate the driver's safe driving behavior

[0063] As a more advanced behavior cloning method, Dataset Aggregation (DAgger) can actively select strategies from the safe driving behavior database and more easily match the safe driving behavior of human drivers in the subsequent training process, which has a higher imitation learning ability. Therefore, the present invention uses the DAgger algorithm to simulate the safe driving behavior of human drivers. The safe driving behavior database D continuously aggregates new datasets D at each time step i. i , the specific training process is as follows:

[0064] Sub-step 1: Initialize parameter φ;

[0065] Sub-step 2: Initialize strategy π;

[0066] Sub-step 3: Perform a loop of N time steps, each loop includes sub-steps 3.1 to 3.5, specifically:

[0067] Sub-step 3.1: Update the strategy using the following formula:

[0068]

[0069] In the formula, π i represents the strategy at the i-th moment, π * represents the expert strategy, β i represents the parameters of the soft update of the strategy at the i-th iteration, represents the optimal strategy at the i-th moment;

[0070] Substep 3.2: Using π i Sampling expert trajectories;

[0071] Sub-step 3.3: Output is given by π i The data set D consists of the strategies visited and the actions given by the experts i ={(S t ,π * (S t ))}, S t represents the state space at time t;

[0072] Sub-step 3.4: Aggregate the dataset: D←D∪D i ;

[0073] Sub-step 3.5: Train the policy on dataset D in represents the optimal strategy at time i+1; Sub-step 4: Finally, return the optimal strategy at time N+1

[0074] Step 3: Use unsupervised learning methods to further learn collision avoidance strategies

[0075] In actual driving decision-making tasks, driving decisions based on imitation learning lack sufficient generalization capabilities and are difficult to effectively and accurately handle driving conditions and driving conditions that are not covered in the safe driving behavior database. In order to further improve the effectiveness and reliability of lateral collision avoidance decisions, the decision network needs to be further trained. As an unsupervised learning method, deep reinforcement learning can obtain an understanding of the traffic environment through continuous exploration and trial and error, and use the rewards of environmental feedback to guide the improvement of the policy network, thereby maximizing the return. The proximal policy optimization algorithm draws on the trust region policy optimization algorithm and achieves a new balance between sampling efficiency, algorithm performance, and the complexity of implementation and debugging by using first-order optimization. Therefore, the present invention uses the proximal policy optimization algorithm to construct an anti-collision decision model, and trains the anti-collision decision model based on step two.

[0076] First, the side collision avoidance decision problem of commercial vehicles is converted into a Markov decision process under a certain reward function, which can be described as (S, A, P, R). Among them, S is the state space, A is the driving action, P represents the state transition probability caused by the uncertainty of the target vehicle's motion, and R is the reward function. Secondly, the basic parameters of the Markov decision process are defined, specifically:

[0077] (1) Establishing the state space

[0078] First, the state space is constructed using the motion state information of the vehicle and the relative motion state information of the vehicle and surrounding traffic participants:

[0079]

[0080] In the formula, p x ,p y Respectively represent the lateral position and longitudinal position of the vehicle, in meters, v x ,v y They represent the lateral speed and longitudinal speed of the vehicle, in meters per second, and a x ,a y They represent the lateral acceleration and longitudinal acceleration of the vehicle, respectively, in meters per second squared, and θ s Indicates the heading angle of the vehicle in degrees. Respectively represent the relative distance and relative speed between the vehicle and the jth traffic participant around it, in meters and meters per second, respectively. Among them, j = 1, 2, 3, 4, 5, 6, respectively represent the traffic participants in front, left front, left rear, rear, right rear, and right front. Considering that the number of traffic participants in actual traffic scenes is not fixed, when the sensor observes i (i is less than j) traffic participants, the last j rows of the state space are filled with zeros.

[0081] (2) Establishing the action space

[0082] In order to output high-level driving decisions, the present invention defines the action space as discrete lateral actions and longitudinal actions.

[0083] A t =[a1,a2,a3,a4,a5,a6] (3)

[0084] In the formula, A t represents the action space at time t, a1, a2, a3 represent left turn, right turn and straight ahead respectively, a4, a5, a6 represent acceleration, deceleration and keeping the speed constant respectively.

[0085] (3) Establishing the reward function

[0086] In order to quantitatively evaluate the quality of the collision avoidance strategy, the present invention establishes a collision avoidance reward function that takes into account the impact of traffic participant types on driving safety:

[0087]

[0088] In the formula, R t represents the reward function at time t, x min_1 ,x min_2 represents the lateral safety distance threshold, in meters. In the present invention, x min_1 =2,x min_2 =2.5.

[0089] In addition, negative feedback is applied to the decision that causes a side collision, that is, when the output decision strategy leads to a side collision, the reward value obtained at the current moment is subtracted by 50.

[0090] Secondly, the constructed collision avoidance decision model is trained, which includes the following sub-steps:

[0091] Sub-step 1: Initialize the policy parameters θ0 and value function parameters φ0;

[0092] Sub-step 2: Perform a loop of T time steps, each loop includes sub-steps 2.1 to 2.5, specifically:

[0093] Sub-step 2.1: Run policy π in the environment k =π(θ k );

[0094] Sub-step 2.2: Calculate the optimal reward value at time t

[0095] Sub-step 2.3: Based on the current value function Compute an estimate of the advantage function

[0096] Sub-step 2.4: Update the strategy using the following formula:

[0097]

[0098] In the formula, θ k+1 represents the policy network parameters at time k+1, ε represents the hyperparameter, and π θ represents a policy network with parameter θ, and clip(·) represents a truncation function. Truncated in [1-ε,1+ε], τ represents the hyperparameter that determines the soft update amplitude, argmax(·) represents the variable that maximizes the objective function, Represents the advantage value of the state-action pair.

[0099] Sub-step 2.5: Update the value function using the following formula:

[0100]

[0101] In the formula, φ k+1 represents the value function parameter at time k+1, V φ (S t ) represents the state space S t The value function below.

[0102] Finally, after the training of the collision avoidance decision model is completed, the motion state information of the vehicle and the relative motion state information of the vehicle and surrounding traffic participants are input into the collision avoidance decision model, and driving suggestions such as acceleration, deceleration, and lane change can be output, thus realizing effective and reliable lateral collision avoidance driving decisions for large commercial vehicles.

Claims

1. The lateral collision avoidance driving decision-making method for commercial vehicles in low-speed urban environments. First, the urban traffic scenario is constructed using the hardware-in-the-loop driving simulation platform to simulate and collect safe driving behaviors under different driving conditions and driving conditions; Secondly, through imitation learning, the data set aggregation algorithm is used to simulate the driver's safe driving behavior; finally, through unsupervised learning, the proximal strategy optimization algorithm is used to further learn the lateral collision avoidance strategy, provide collision avoidance driving strategies for large operating vehicles, and realize lateral collision avoidance driving decisions for operating vehicles in low-speed urban environments; the characteristics are: Step 1: Use the driving simulation platform to build urban traffic scenarios When a commercial vehicle is driving in a low-speed urban environment, there are other traffic participants on its left or right side, including motor vehicles, non-motor vehicles or pedestrians; First, a driving simulation platform is used to build urban traffic scenarios, including straight roads, curves, and intersections, and to set up highly random traffic flows and traffic participants. Secondly, multiple drivers controlled the operating vehicles using a driving simulator, which had a steering wheel, accelerator and brake pedals, and collected safe driving behaviors under eight driving conditions: lane change, lane keeping, vehicle following, left turn, right turn, acceleration, deceleration and constant speed. Finally, based on the collected safe driving behaviors, a safe driving behavior database D is constructed; Step 2: Use imitation learning methods to simulate the driver's safe driving behavior Use data set aggregation algorithms to simulate safe driving behaviors of human drivers; The safe driving behavior database D continuously aggregates new datasets D at each time step i i , the specific training process is as follows: Sub-step 1: Initialize parameter φ; Sub-step 2: Initialize strategy π; Sub-step 3: Perform a loop of N time steps, each loop includes sub-steps 3.1 to 3.5, specifically: Sub-step 3.1: Update the strategy using the following formula: In the formula, π i represents the strategy at the i-th moment, π * represents the expert strategy, β i represents the parameters of the soft update of the strategy at the i-th iteration, represents the optimal strategy at the i-th moment; Substep 3.2: Using π i Sampling expert trajectories; Sub-step 3.3: Output is given by π i The data set D consists of the strategies visited and the actions given by the experts i ={(S t ,π * (S t ))},S t represents the state space at time t; Sub-step 3.4: Aggregate the dataset: D←D∪D i ; Sub-step 3.5: Train the policy on dataset D in represents the optimal strategy at time i+1; Sub-step 4: Finally, return to the optimal strategy at time N+1 Step 3: Use unsupervised learning methods to further learn collision avoidance strategies The collision avoidance decision model is constructed using the proximal policy optimization algorithm, and the collision avoidance decision model is trained based on step 2. First, the lateral collision avoidance decision problem of the operating vehicle is converted into a Markov decision process under a certain reward function, which is described as (S, A, P, R). Among them, S is the state space, A is the driving action, P represents the state transition probability caused by the uncertainty of the target vehicle's motion, and R is the reward function. Secondly, the basic parameters of the Markov decision process are defined, specifically: (1) Establishing the state space First, the state space is constructed using the motion state information of the vehicle and the relative motion state information of the vehicle and surrounding traffic participants: In the formula, S t represents the state space at time t, p x ,p y Respectively represent the lateral position and longitudinal position of the vehicle, in meters, v x ,v y They represent the lateral speed and longitudinal speed of the vehicle, in meters per second, and a x ,a y They represent the lateral acceleration and longitudinal acceleration of the vehicle, respectively, in meters per second squared, and θ s Indicates the heading angle of the vehicle, in degrees; Respectively represent the relative distance and relative speed between the vehicle and the jth traffic participant around it, in units of meters and meters per second, respectively; where j = 1, 2, 3, 4, 5, 6, representing the traffic participants in front, left front, left rear, rear, right rear, and right front, respectively; considering that the number of traffic participants in actual traffic scenes is not fixed, when the sensor observes i traffic participants, the last j rows of the state space are filled with zeros, where i is less than j; (2) Establishing the action space In order to output high-level driving decisions, the action space is defined as discrete lateral and longitudinal actions; <h2 style=";text-align:left;direction:ltr">A<h2 style=";text-align:left;direction:ltr"> t <h2 style=";text-align:left;direction:ltr"> (3) [a1,a2,a3,a4,a5,a6] In the formula, A t represents the action space at time t, a1, a2, a3 represent left turn, right turn and straight ahead respectively, a4, a5, a6 represent acceleration, deceleration and keeping the speed constant respectively; (3) Establishing the reward function In order to quantitatively evaluate the pros and cons of the collision avoidance strategy, an anti-collision reward function is established that takes into account the impact of traffic participant types on driving safety: In the formula, R t represents the reward function at time t, x min_1 ,x min_2 Indicates the lateral safety distance threshold in meters, x min_1 =2,x min_2 =2.5; It represents the relative distance between the vehicle and the jth traffic participant around it, in meters; In addition, negative feedback is applied to the decision that causes a side collision, that is, when the output decision strategy leads to a side collision, the reward value obtained at the current moment is subtracted by 50; Secondly, the constructed collision avoidance decision model is trained, which includes the following sub-steps: Sub-step 1: Initialize the policy parameters θ0 and value function parameters φ0; Sub-step 2: Perform a loop of T time steps, each loop includes sub-steps 2.1 to 2.5, specifically: Sub-step 2.1: Run policy π in the environment k =π(θ k ), θ k represents the policy network parameters at time k; Sub-step 2.2: Calculate the optimal reward value at time t Sub-step 2.3: Based on the current value function V φk , calculate the estimate of the advantage function Sub-step 2.4: Update the strategy using the following formula: In the formula, θ k+1 represents the policy network parameters at time k+1, ε represents the hyperparameter, and π θ represents the output value of the policy network with parameter θ, T represents the time step of the update, Denote the parameter as θ k The policy network output value, D k represents the data in the dataset, which consists of the driving strategy output at time k and the actions given by the expert. clip(·) represents the truncation function. Truncated in [1-ε,1+ε], τ represents the hyperparameter that determines the soft update amplitude, argmax(·) represents the variable that maximizes the objective function, Represents the advantage value of the state-action pair; Sub-step 2.5: Update the value function using the following formula: In the formula, φ k+1 represents the value function parameter at time k+1, V φ (S t ) represents the state space S t The value function under ; Finally, after the training of the collision avoidance decision model is completed, the motion state information of the vehicle and the relative motion state information of the vehicle and surrounding traffic participants are input into the collision avoidance decision model, and driving suggestions for acceleration, deceleration and lane change are output, thus realizing effective and reliable lateral collision avoidance driving decisions for large commercial vehicles.

Images