A forward collision avoidance driving decision-making method for commercial vehicles in open interference scenarios

Abstract

The present invention discloses a forward collision avoidance driving decision-making method for commercial vehicles in open interference scenarios. First, the safe driving behavior of human drivers is extracted to construct a "human-like" driving behavior database for learning forward collision avoidance driving decisions. Secondly, a behavior cloning algorithm is used to learn forward collision avoidance driving strategies from the "human-like" driving behavior database. Finally, a generative adversarial imitation learning algorithm is used to continue learning driving strategies to obtain forward collision avoidance driving strategies with a highly human-like level. The method proposed in the present invention can simulate the safe driving behavior of human drivers, and takes into account the influence of factors such as visual blind spots, sudden obstacles, and different driving conditions on forward collision avoidance decisions, and outputs advanced decision-making strategies such as acceleration, deceleration, and steering, thereby realizing forward collision avoidance driving decisions for commercial vehicles in open interference scenarios.

Description

Technical Field

[0001] The invention relates to a driving decision method for an operating vehicle, and in particular to a forward collision avoidance driving decision method for an operating vehicle facing an open interference scenario, and belongs to the technical field of automobile safety. Background Art

[0002] Operating vehicles undertake tasks such as highway logistics and passenger transportation, and are the "main force" of transportation, the forerunner of the national economy. As a key link in multimodal transport, operating vehicles are an important tool for achieving inter-provincial and inter-regional transportation, and their technical level and safety status are important indicators of road transport service capabilities. At present, operating vehicles still face many thorny problems in safe transportation, which are particularly reflected in the two aspects of proneness to accidents and low safety.

[0003] Specifically, due to the characteristics of high center of mass, large load capacity, long operation time, and wide visual blind spots of operating vehicles, coupled with the complex road conditions and traffic conditions in my country, traffic accidents occur frequently. Especially in open interference environments such as cities, sudden encounters with obstacles often occur. If forward collision avoidance operations such as emergency lane change or emergency braking are performed, operating vehicles are very likely to roll over due to instability. In addition, since there are static visual blind spots in front of the front of the operating vehicle, near the right front wheel, and near the bottom of the right rearview mirror, when the vehicle turns right, the front of the vehicle is very likely to collide with pedestrians and non-motor vehicles in the visual blind spot. It can be seen that the front of the front of the operating vehicle is one of the most dangerous areas of all visual blind spots, and it is also the main area for serious safety accidents. Therefore, in an open urban traffic environment with multiple traffic targets, how to avoid forward collisions caused by factors such as visual blind spots and sudden encounters with obstacles is a key issue to ensure the driving safety of operating vehicles and improve transportation efficiency.

[0004] At present, the active development of autonomous driving technology has become an important means of ensuring vehicle operation safety that is widely recognized at home and abroad. As a key link in achieving high-quality autonomous driving, driving decisions determine the rationality and safety of autonomous driving of commercial vehicles. If the driver can be given a danger warning 1.5 seconds before a traffic accident occurs and a reliable and effective collision avoidance decision-making strategy is provided, the frequency of traffic accidents caused by forward collisions can be greatly reduced or the damage caused by them can be reduced. Therefore, studying the forward collision avoidance driving decision-making method of commercial vehicles in open interference scenarios is of great significance to reducing the frequency of collision accidents.

[0005] To this end, many patents and literature have studied anti-collision driving decisions, but they are mainly aimed at passenger vehicles. Compared with passenger vehicles, commercial vehicles are more likely to roll over due to instability during emergency lane changes or emergency braking, and have longer braking distances and braking times. Therefore, the anti-collision decision-making method for passenger vehicles cannot be directly applied to commercial vehicles. On the other hand, existing patents have studied the anti-collision decision-making of commercial vehicles, such as a heavy-duty commercial vehicle anti-collision warning method that comprehensively considers front and rear obstacles (patent number: ZL202011512720.8), an adaptive anti-collision graded warning method based on forward obstacle recognition (patent number: ZL202010436618.8), and a large commercial vehicle anti-collision decision-making method that considers road adhesion conditions (publication number: CN113753026A), etc., but these methods are all oriented to highway scenarios.

[0006] Different from the highway scenario where only the rear-end collision prevention problem is considered, in the open interference and mixed traffic of vehicles and non-vehicles in urban traffic environment, forward collision avoidance decision-making should consider the following three aspects: (1) Rear-end collisions should be avoided; (2) Collisions with traffic participants approaching laterally in front of the vehicle should be avoided, that is, collisions with sudden obstacles should be avoided; (3) During the vehicle turning process (especially right turning), collisions with other traffic participants in the visual blind spot should be avoided. Therefore, the collision avoidance decision-making method for commercial vehicles in highway scenarios cannot be directly applied to the open interference urban traffic environment.

[0007] In general, in an open urban traffic environment with multiple traffic interferences, the existing forward collision avoidance decision-making methods have shortcomings such as low reliability and poor accuracy. There is still a lack of forward collision avoidance driving decision-making methods that can provide specific driving suggestions such as driving actions and driving paths, especially a lack of reliable forward collision avoidance driving decision-making methods for operating vehicles that take into account the impact of visual blind spots and sudden obstacles. Summary of the invention

[0008] Purpose of the invention: In order to realize the forward collision avoidance driving decision of commercial vehicles in open interference scenarios and ensure the driving safety of vehicles, the present invention proposes a forward collision avoidance driving decision method for commercial vehicles in open interference scenarios for automatic driving commercial vehicles such as heavy goods vehicles and heavy trucks. This method can simulate the safe driving behavior of human drivers, and consider the impact of factors such as visual blind spots, sudden obstacles, and different driving conditions on forward collision avoidance decisions, and output advanced decision strategies such as acceleration, deceleration, and steering. At the same time, this method does not need to consider complex vehicle dynamics equations and vehicle body parameters, the calculation method is simple and clear, and the forward collision avoidance driving strategy of large commercial vehicles can be output in real time. The sensors used are low in cost, which is convenient for large-scale promotion.

[0009] Technical solution: To achieve the purpose of the present invention, the technical solution adopted by the present invention is: a forward collision avoidance driving decision-making method for commercial vehicles in open interference scenarios. First, the safe driving behavior of human drivers is extracted to construct a "human-like" driving behavior database for forward collision avoidance driving decision-making learning. Secondly, the forward collision avoidance driving strategy is learned from the "human-like" driving behavior database using a behavioral cloning algorithm. Finally, the driving strategy is continued to be learned using a generative adversarial imitation learning algorithm to obtain a forward collision avoidance driving strategy with a highly human-like level. Specifically, the following steps are included:

[0010] Step 1: Build a “human-like” driving behavior database

[0011] In order to achieve driving decisions comparable to those of human drivers, the present invention collects safe driving behaviors of human drivers under different driving conditions and driving conditions through actual road tests and driving simulations, and then constructs a "human-like" driving behavior database. Specifically, it includes the following five sub-steps:

[0012] Sub-step 1: Use millimeter-wave radar, visual sensors, Beidou sensors and inertial sensors to build a multi-dimensional target information synchronous acquisition system.

[0013] Sub-step 2: In a real urban environment, multiple drivers take turns driving an operating vehicle equipped with a multi-dimensional target information synchronous acquisition system.

[0014] Sub-step 3: Analyze the relevant data of the driver's lane changing, lane keeping, vehicle following, acceleration and deceleration and other safe driving behaviors, focusing on the following and lane changing behaviors that are common in urban vehicle driving and prone to traffic accidents, and extract multi-source heterogeneous description data of each typical driving behavior, such as position, speed, acceleration, relative distance and relative speed to surrounding traffic participants, etc.

[0015] Sub-step 4: In order to simulate the safe driving behavior of human drivers under extreme conditions such as dangerous conditions and edge scenarios, a driving simulation scenario based on hardware-in-the-loop simulation is constructed (covering various road network structures such as straight roads, curves, and intersections, as well as various types of traffic participants such as commercial vehicles, passenger cars, non-motor vehicles, and pedestrians). The constructed urban traffic scenarios include the following two categories: (1) During the driving process of the vehicle, there are traffic participants approaching laterally in front of the vehicle (i.e., sudden obstacles); (2) During the turning process of the vehicle, there are other traffic participants in the vehicle's visual blind spot.

[0016] The driver drives the operating vehicle in the simulation scene through a real controller to collect information such as the vehicle's position, speed, acceleration, relative distance and relative speed to surrounding traffic participants.

[0017] Sub-step 5: Based on the data collected from the real urban environment and the driving simulation environment, a "human-like" driving behavior database for forward collision avoidance driving decision learning is constructed, which can be specifically expressed as:

[0018]

[0019] In the formula, X represents a binary group covering states and actions, that is, the constructed driving behavior database, (s j ,a j ) represents the “state-action” pair at time j, s j represents the state at time j, a j represents the action at time j, i.e. the human driver’s action based on state s j The action taken, n is the number of "state-action" pairs in the database.

[0020] Step 2: Use the behavior cloning algorithm to learn safe driving behaviors from the "human-like" driving behavior database

[0021] In order to learn driving behaviors from a driving behavior database efficiently and quickly, strategy optimization and behavior learning are often performed through imitation learning. Imitation learning mainly includes four types of methods: behavior cloning, inverse reinforcement learning, imitation learning from observations, and probabilistic reasoning. As a relatively simple and direct supervised learning method, the behavior cloning algorithm can improve the policy network by learning expert knowledge from sample data, making the decision of the policy network more like human driving behavior. Therefore, the present invention uses the behavior cloning algorithm to learn safe driving behaviors in a "human-like" driving behavior database.

[0022] Firstly, a driving decision network based on behavior cloning is constructed, which contains three fully connected layers FC1, FC2 and FC3, each containing 64 neurons.

[0023] Secondly, based on the driving behavior database constructed in step 1 The driving decision network is trained with the following goal:

[0024]

[0025] In the formula, π θ (s i ) represents the deterministic action output at time i.

[0026] The driving decision network is considered as a multi-category classifier and trained using supervised learning. The goal is to make the policy network make the same decisions as human experts. The specific training process is as follows:

[0027] Sub-step 1: Action a in the driving behavior database jAfter one-bit efficient encoding, the output vector can be expressed as

[0028] Sub-step 2: Perform a loop of T time steps until the algorithm converges. Each loop includes sub-steps 2.1 to 2.2, specifically:

[0029] Sub-step 2.1: Perform uniform random sampling from the sequence numbers {1, 2, ..., n};

[0030] Substep 2.2: Let the parameters of the current policy network be θ now . j , As input, the gradient is calculated through back propagation, and then the parameter θ is updated using the following formula;

[0031]

[0032] In the formula, θ new represents the updated parameters θ, π(·|s j θ now ) represents a random policy network, represents the gradient function of the neural network loss function with parameter θ, and H represents the vector With the random policy network π(·|s j θ now ) is the cross entropy of .

[0033] Step 3: Use the generative adversarial imitation learning algorithm to learn the forward collision avoidance driving strategy

[0034] Although supervised learning-based methods such as behavioral cloning can learn the safe driving behavior of human drivers, they cannot traverse all traffic scenarios and driving conditions. Once a state that does not appear in the driving behavior database is encountered, the effectiveness and reliability of the decision will be greatly affected. Considering that generative adversarial imitation learning draws on the ideas of reinforcement learning and generative adversarial networks, it can be learned through samples generalized from the driving behavior database, which has a certain effect on improving the performance of collision avoidance decisions. Therefore, the present invention uses generative adversarial imitation learning to continue the forward collision avoidance driving strategy based on the decision model obtained by training in step two.

[0035] First, we define the basic parameters of the generative adversarial imitation learning network.

[0036] (1) Define the state space

[0037] In order to describe the motion state of the vehicle and nearby traffic participants, the present invention uses time series data and occupancy grid graph to construct the state space. The specific description is as follows:

[0038] S t=[S1(t),S2(t),S3(t)] (4)

[0039] In the formula, S t represents the state space at time t, S1(t) represents the state space related to the time series data at time t, S2(t), S3(t) represent the state space related to the occupancy grid map at time t.

[0040] First, the motion state of the vehicle is described using continuous position, velocity, acceleration, and heading information:

[0041] S1(t)=[p x ,p y ,v x ,v y ,a x ,a y ,θ s ] (5)

[0042] In the formula, p x ,p y Respectively represent the lateral position and longitudinal position of the vehicle, in meters, v x ,v y They represent the lateral speed and longitudinal speed of the vehicle, in meters per second, and a x ,a y They represent the lateral acceleration and longitudinal acceleration of the vehicle, respectively, in meters per second squared, and θ s Indicates the heading angle of the vehicle in degrees.

[0043] Secondly, in order to describe the relative motion state between the vehicle and the surrounding traffic participants, the present invention rasterizes the road area, divides it into a number of p×q grid areas, and abstracts the road area and vehicle targets into a grid map; wherein p represents the length of the grid area, and q represents the width of the grid area; the constructed grid maps include two categories: an “existence” grid map S2(t) for describing the relative position relationship and a “speed” grid map S3(t) for describing the relative speed relationship.

[0044] The "presence" grid map contains four attributes, including grid coordinates, whether there is a vehicle, the type of the corresponding vehicle, and the distance from the left and right lane lines. Among them, the grid without traffic participants is set to "0", and the grid with traffic participants is set to "1". The position distribution of this grid and the grid where the vehicle is located is used to describe the relative distance between the two vehicles. Similarly, the "speed" grid map contains three attributes, including grid coordinates, the speed of the corresponding vehicle, and the distance from the left and right lane lines. Among them, the grid without traffic participants is set to "0", and the value in the grid with traffic participants represents the relative speed in meters per second.

[0045] (2) Define the action space

[0046] In order to output high-level driving decisions, the present invention defines the action space as discrete lateral actions and longitudinal actions.

[0047] A t =[a left ,a straight ,a right ,a accel ,a remain ,a decel ] (6)

[0048] In the formula, A t represents the action space at time t, a left ,a straight ,a right They represent left turn, straight ahead and right turn respectively. accel ,a remain ,a decel They represent acceleration, constant speed, and deceleration respectively.

[0049] Secondly, construct a generative adversarial imitation learning network. The specific process is as follows:

[0050] (1) Constructing the generator network

[0051] Let the generator network be π(A t |S t ; θ), where θ represents the parameters of the generator network; the input of the generator network is the state space, and the output is the vector f = π(·|s; θ), the dimension of the output vector f is 6, and each of its elements corresponds to an action in the action space, indicating the probability of executing the action.

[0052] (2) Constructing the discriminator network

[0053] The discriminator is denoted as D(S t ,A t ; φ), where φ represents the parameters of the discriminator network; the input of the discriminator network is the state space, and the output is the vector Output vector The dimension is 6, each element of which corresponds to an action, and an element is recorded as:

[0054]

[0055] In the formula, Close to 1 means that the action comes from the driving behavior database. A value close to 0 indicates that the action was generated by the generator network.

[0056] Finally, in order to make the trajectories generated by the generator network as good as those in the driving behavior database, the generative adversarial imitation learning network is trained, which includes the following sub-steps:

[0057] Sub-step 1: Initialize the generator network parameters θ0 and the discriminator network parameters φ0;

[0058] Sub-step 2: Perform a loop of N time steps until the algorithm converges. Each loop includes sub-steps 2.1 to 2.5, specifically:

[0059] Sub-step 2.1: Uniformly sample a trajectory from the “human-like” driving behavior database constructed in step 1, denoted as:

[0060]

[0061] In the formula, τ real represents the trajectory sampled from the driving behavior database, represents the state space at time m, represents the driving action at time m;

[0062] Sub-step 2.2: Using the generator network π(a|s; θ now ) controls the agent to interact with the environment and obtains a trajectory, recorded as:

[0063]

[0064] In the formula, τ fake represents the trajectory generated by the generator, represents the state space at time n, represents the driving action at time n;

[0065] Sub-step 2.3: Use the discriminator to evaluate whether the decision of the generator network is realistic:

[0066]

[0067] In the formula, u t represents the return value at time t, represents the state space generated by the generator at time t, represents the action space executed by the generator at time t, φ now Represents the parameters of the current discriminator network;

[0068] Sub-step 2.4: Set τ fake and u1,u2,...u n As input, update the parameters of the generator network using the following formula:

[0069]

[0070] In the formula, θ new represents the updated generator network parameters, Δ is the hyperparameter, Represents the objective function, which can be calculated by the following formula:

[0071]

[0072] In the formula, θ now Represents the parameters of the current generator network;

[0073] Sub-step 2.5: Set τ real and τ fake As input, update the parameters of the discriminator network using the following formula:

[0074]

[0075] In the formula, φ new represents the updated discriminator network parameters, φ now represents the parameters of the current discriminator network, η represents the learning rate, represents the gradient of the discriminator network, F(τ real ,τ fake ; φ now ) represents the loss function, which can be calculated by the following formula:

[0076]

[0077] When the training is completed, the state space information is used as input to output advanced driving strategies such as left turn, right turn, acceleration and deceleration, realizing the parameters of the forward collision avoidance driving decision network for operating vehicles in open interference scenarios.

[0078] Beneficial effects: Compared with the general driving decision-making method, the method proposed by the present invention is more effective and reliable, which is specifically reflected in:

[0079] (1) The method proposed in the present invention can imitate the safe driving behavior of human drivers, provide a more reasonable and safe forward collision avoidance driving strategy for operating vehicles, realize the forward collision avoidance driving decision of operating vehicles in open interference scenarios, and effectively ensure the driving safety of vehicles.

[0080] (2) The method proposed in the present invention comprehensively considers the impact of factors such as visual blind spots, sudden obstacles, and different driving conditions on forward collision avoidance decisions, achieves safe driving decisions under different traffic environments and driving conditions, and further improves the effectiveness and reliability of forward collision avoidance decisions.

[0081] (3) The method proposed in the present invention does not need to consider complex vehicle dynamics equations and vehicle body parameters. The calculation method is simple and clear, and the forward collision avoidance driving strategy of operating vehicles can be output in real time. The sensors used are low in cost, which is convenient for large-scale promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 It is the technical roadmap of the present invention;

[0083] Figure 2 It is a schematic diagram of the driving decision network structure based on behavior cloning designed by the present invention;

[0084] Figure 3 It is a schematic diagram of the generator network structure designed by the present invention;

[0085] Figure 4 It is a schematic diagram of the discriminator network structure designed by the present invention. DETAILED DESCRIPTION

[0086] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0087] The present invention proposes a forward collision avoidance driving decision-making method for commercial vehicles in an open urban traffic environment with multiple traffic target interference. First, the safe driving behavior of human drivers is extracted to construct a "human-like" driving behavior database for learning forward collision avoidance driving decisions. Secondly, the behavior cloning algorithm is used to learn the forward collision avoidance driving strategy from the "human-like" driving behavior database. Finally, the generative adversarial imitation learning algorithm is used to continue learning the driving strategy to obtain a forward collision avoidance driving strategy with a highly human-like level. The method proposed in the present invention can simulate the safe driving behavior of human drivers, and takes into account the influence of factors such as visual blind spots, sudden obstacles, and different driving conditions on forward collision avoidance decisions, and outputs advanced decision-making strategies such as acceleration, deceleration, and steering, thereby realizing forward collision avoidance driving decisions for commercial vehicles in open interference scenarios. The technical route of the present invention is as follows: Figure 1 As shown, the specific steps are as follows:

[0088] Step 1: Build a “human-like” driving behavior database

[0089] In order to achieve driving decisions comparable to those of human drivers, the present invention collects safe driving behaviors of human drivers under different driving conditions and driving conditions through actual road tests and driving simulations, and then constructs a "human-like" driving behavior database. Specifically, it includes the following five sub-steps:

[0090] Sub-step 1: Use millimeter-wave radar, visual sensors, Beidou sensors and inertial sensors to build a multi-dimensional target information synchronous acquisition system.

[0091] Sub-step 2: In a real urban environment, multiple drivers take turns driving an operating vehicle equipped with a multi-dimensional target information synchronous acquisition system.

[0092] Sub-step 3: Analyze the relevant data of the driver's lane changing, lane keeping, vehicle following, acceleration and deceleration and other safe driving behaviors, focusing on the following and lane changing behaviors that are common in urban vehicle driving and prone to traffic accidents, and extract multi-source heterogeneous description data of each typical driving behavior, such as position, speed, acceleration, relative distance and relative speed to surrounding traffic participants, etc.

[0093] Sub-step 4: In order to simulate the safe driving behavior of human drivers under extreme conditions such as dangerous conditions and edge scenarios, a driving simulation scenario based on hardware-in-the-loop simulation is constructed (covering various road network structures such as straight roads, curves, and intersections, as well as various types of traffic participants such as commercial vehicles, passenger cars, non-motor vehicles, and pedestrians). The constructed urban traffic scenarios include the following two categories: (1) During the driving process of the vehicle, there are traffic participants approaching laterally in front of the vehicle (i.e., sudden obstacles); (2) During the turning process of the vehicle, there are other traffic participants in the vehicle's visual blind spot.

[0094] The driver drives the operating vehicle in the simulation scene through a real controller to collect information such as the vehicle's position, speed, acceleration, relative distance and relative speed to surrounding traffic participants.

[0095] Sub-step 5: Based on the data collected from the real urban environment and the driving simulation environment, a "human-like" driving behavior database for forward collision avoidance driving decision learning is constructed, which can be specifically expressed as:

[0096]

[0097] In the formula, X represents a binary group covering states and actions, that is, the constructed driving behavior database, (s j ,a j ) represents the state-action pair at time j, s j represents the state at time j, a j represents the action at time j, i.e. the human driver’s action based on state s j The action taken, n is the number of "state-action" pairs in the database.

[0098] Step 2: Use the behavior cloning algorithm to learn safe driving behaviors from the "human-like" driving behavior database

[0099] In order to learn driving behaviors from a driving behavior database efficiently and quickly, strategy optimization and behavior learning are often performed through imitation learning. Imitation learning mainly includes four types of methods: behavior cloning, inverse reinforcement learning, imitation learning from observations, and probabilistic reasoning. As a relatively simple and direct supervised learning method, the behavior cloning algorithm can improve the policy network by learning expert knowledge from sample data, making the decision of the policy network more like human driving behavior. Therefore, the present invention uses the behavior cloning algorithm to learn safe driving behaviors in a "human-like" driving behavior database.

[0100] Firstly, a driving decision network based on behavior cloning is constructed, which contains three fully connected layers FC1, FC2 and FC3, each containing 64 neurons.

[0101] Secondly, based on the driving behavior database constructed in step 1 The driving decision network is trained with the following goal:

[0102]

[0103] In the formula, π θ (s i ) represents the deterministic action output at time i.

[0104] The driving decision network is considered as a multi-category classifier and trained using supervised learning. The goal is to make the policy network make the same decisions as human experts. The specific training process is as follows:

[0105] Sub-step 1: Action a in the driving behavior database j After one-bit efficient encoding, the output vector can be expressed as

[0106] Sub-step 2: Perform a loop of T time steps until the algorithm converges. Each loop includes sub-steps 2.1 to 2.2, specifically:

[0107] Sub-step 2.1: Perform uniform random sampling from the sequence numbers {1, 2, ..., n};

[0108] Sub-step 2.2: Let the parameters of the current policy network be θ now . j , As input, the gradient is calculated through back propagation, and then the parameter θ is updated using the following formula;

[0109]

[0110] In the formula, θ new represents the updated parameters θ, π(·|s j θnow ) represents a random policy network, represents the gradient function of the neural network loss function with parameter θ, and H represents the vector With the random policy network π(·|s j θ now ) is the cross entropy of .

[0111] Step 3: Use the generative adversarial imitation learning algorithm to learn the forward collision avoidance driving strategy

[0112] Although supervised learning-based methods such as behavioral cloning can learn the safe driving behavior of human drivers, they cannot traverse all traffic scenarios and driving conditions. Once a state that does not appear in the driving behavior database is encountered, the effectiveness and reliability of the decision will be greatly affected. Considering that generative adversarial imitation learning draws on the ideas of reinforcement learning and generative adversarial networks, it can be learned through samples generalized from the driving behavior database, which has a certain effect on improving the performance of collision avoidance decisions. Therefore, the present invention uses generative adversarial imitation learning to continue the forward collision avoidance driving strategy based on the decision model obtained by training in step two.

[0113] First, we define the basic parameters of the generative adversarial imitation learning network.

[0114] (1) Define the state space

[0115] In order to describe the motion state of the vehicle and nearby traffic participants, the present invention uses time series data and occupancy grid graph to construct the state space. The specific description is as follows:

[0116] S t =[S1(t),S2(t),S3(t)] (4)

[0117] In the formula, S t represents the state space at time t, S1(t) represents the state space related to the time series data at time t, S2(t), S3(t) represent the state space related to the occupancy grid map at time t.

[0118] First, the motion state of the vehicle is described using continuous position, velocity, acceleration, and heading information:

[0119] S1(t)=[p x ,p y ,v x ,v y ,a x ,a y ,θ s ] (5)

[0120] In the formula, p x ,p yRespectively represent the lateral position and longitudinal position of the vehicle, in meters, v x ,v y They represent the lateral speed and longitudinal speed of the vehicle, in meters per second, and a x ,a y They represent the lateral acceleration and longitudinal acceleration of the vehicle, respectively, in meters per second squared, and θ s Indicates the heading angle of the vehicle in degrees.

[0121] Secondly, in order to describe the relative motion state between the vehicle and the surrounding traffic participants, the present invention rasterizes the road area, divides it into a number of p×q grid areas, and abstracts the road area and vehicle targets into a grid map; wherein p represents the length of the grid area, and q represents the width of the grid area; the constructed grid maps include two categories: an “existence” grid map S2(t) for describing the relative position relationship and a “speed” grid map S3(t) for describing the relative speed relationship.

[0122] The "presence" grid map contains four attributes, including grid coordinates, whether there is a vehicle, the type of the corresponding vehicle, and the distance from the left and right lane lines. Among them, the grid without traffic participants is set to "0", and the grid with traffic participants is set to "1". The position distribution of this grid and the grid where the vehicle is located is used to describe the relative distance between the two vehicles. Similarly, the "speed" grid map contains three attributes, including grid coordinates, the speed of the corresponding vehicle, and the distance from the left and right lane lines. Among them, the grid without traffic participants is set to "0", and the value in the grid with traffic participants represents the relative speed in meters per second.

[0123] (2) Define the action space

[0124] In order to output high-level driving decisions, the present invention defines the action space as discrete lateral actions and longitudinal actions.

[0125] A t =[a left ,a straight ,a right ,a accel ,a remain ,a decel ] (6)

[0126] In the formula, A t represents the action space at time t, a left ,a straight ,a right They represent left turn, straight ahead and right turn respectively. accel ,a remain ,a decel They represent acceleration, constant speed, and deceleration respectively.

[0127] Secondly, construct a generative adversarial imitation learning network. The specific process is as follows:

[0128] (1) Constructing the generator network

[0129] Let the generator network be π(A t |S t ; θ), where θ represents the parameters of the generator network; the input of the generator network is the state space, and the output is the vector f = π(·|s; θ), the dimension of the output vector f is 6, and each of its elements corresponds to an action in the action space, indicating the probability of executing the action.

[0130] (2) Constructing the discriminator network

[0131] The discriminator is denoted as D(S t ,A t ; φ), where φ represents the parameters of the discriminator network; the input of the discriminator network is the state space, and the output is the vector Output vector The dimension is 6, each element of which corresponds to an action, and an element is recorded as:

[0132]

[0133] In the formula, Close to 1 means that the action comes from the driving behavior database. A value close to 0 indicates that the action was generated by the generator network.

[0134] Finally, in order to make the trajectories generated by the generator network as good as those in the driving behavior database, the generative adversarial imitation learning network is trained, which includes the following sub-steps:

[0135] Sub-step 1: Initialize the generator network parameters θ0 and the discriminator network parameters φ0;

[0136] Sub-step 2: Perform a loop of N time steps until the algorithm converges. Each loop includes sub-steps 2.1 to 2.5, specifically:

[0137] Sub-step 2.1: Uniformly sample a trajectory from the “human-like” driving behavior database constructed in step 1, denoted as:

[0138]

[0139] In the formula, τ real represents the trajectory sampled from the driving behavior database, represents the state space at time m, represents the driving action at time m;

[0140] Sub-step 2.2: Using the generator network π(a|s; θ now ) controls the agent to interact with the environment and obtains a trajectory, recorded as:

[0141]

[0142] In the formula, τ fake represents the trajectory generated by the generator, represents the state space at time n, represents the driving action at time n;

[0143] Sub-step 2.3: Use the discriminator to evaluate whether the decision of the generator network is realistic:

[0144]

[0145] In the formula, u t represents the return value at time t, represents the state space generated by the generator at time t, represents the action space executed by the generator at time t, φ now Represents the parameters of the current discriminator network;

[0146] Sub-step 2.4: Set τ fake and u1,u2,...u n As input, update the parameters of the generator network using the following formula:

[0147]

[0148] In the formula, θ new represents the updated generator network parameters, Δ is the hyperparameter, Represents the objective function, which can be calculated by the following formula:

[0149]

[0150] In the formula, θ now Represents the parameters of the current generator network;

[0151] Sub-step 2.5: Set τ real and τ fake As input, update the parameters of the discriminator network using the following formula:

[0152]

[0153] In the formula, φ new represents the updated discriminator network parameters, φ now represents the parameters of the current discriminator network, η represents the learning rate, represents the gradient of the discriminator network, F(τ real,τ fake ; φ now ) represents the loss function, which can be calculated by the following formula:

[0154]

[0155] When the training is completed, the state space information is used as input to output advanced driving strategies such as left turn, right turn, acceleration and deceleration, realizing the parameters of the forward collision avoidance driving decision network for operating vehicles in open interference scenarios.

Claims

1. A forward collision avoidance driving decision-making method for commercial vehicles in open interference scenarios. First, the safe driving behaviors of human drivers are extracted to construct a "human-like" driving behavior database for forward collision avoidance driving decision-making learning. Secondly, the behavior cloning algorithm is used to learn the forward collision avoidance driving strategy from the "human-like" driving behavior database; finally, the generative adversarial imitation learning algorithm is used to continue learning the driving strategy, and a forward collision avoidance driving strategy with a highly human-like level is obtained; the characteristics are: Step 1: Build a "human-like" driving behavior database Through actual road tests and driving simulation, the safe driving behaviors of human drivers under different driving conditions and driving conditions are collected, and then a "human-like" driving behavior database is constructed; specifically, it includes the following five sub-steps: Sub-step 1.1: Use millimeter wave radar, visual sensor, Beidou sensor and inertial sensor to build a multi-dimensional target information synchronous acquisition system; Sub-step 1.2: In a real urban environment, multiple drivers sequentially drive an operating vehicle equipped with a multi-dimensional target information synchronous acquisition system; Sub-step 1.3: Analyze the relevant data of the driver's various safe driving behaviors, including lane change, lane keeping, vehicle following, acceleration and deceleration, and extract multi-source heterogeneous description data of each typical driving behavior, including position, speed, acceleration, relative distance to surrounding traffic participants, and relative speed; Sub-step 1.4: Build a driving simulation scenario based on hardware-in-the-loop simulation, covering a variety of road network structures such as straight roads, curves, and intersections, as well as multiple types of traffic participants such as commercial vehicles, passenger cars, non-motor vehicles, and pedestrians. The constructed urban traffic scenarios include the following two categories: (1) During the driving process of the vehicle, there will be traffic participants approaching laterally in front of the vehicle, that is, sudden obstacles; (2) When the vehicle is turning, there are other traffic participants in the vehicle's blind spot; The driver drives the operating vehicle in the simulation scene through a real controller to collect information about the vehicle's position, speed, acceleration, relative distance and relative speed to surrounding traffic participants; Sub-step 1.5: Based on the data collected from the real urban environment and the driving simulation environment, a "human-like" driving behavior database for forward collision avoidance driving decision learning is constructed, which is specifically expressed as: In the formula, X represents a binary group covering states and actions, that is, the constructed driving behavior database, (s j ,a j ) represents the "state-action" pair at time j, s j represents the state at time j, a j represents the action at time j, i.e. the human driver’s action based on state s j The action taken, n represents the number of "state-action" pairs in the database; Step 2: Use the behavior cloning algorithm to learn safe driving behaviors from the "human-like" driving behavior database The behavior cloning algorithm is used to learn safe driving behaviors from the "human-like" driving behavior database. First, a driving decision network based on behavior cloning is constructed. The network contains three fully connected layers FC1, FC2 and FC3, each containing 64 neurons. Secondly, based on the driving behavior database constructed in step 1 The driving decision network is trained with the following goal: In the formula, π θ (s i ) represents the deterministic action output at time i, The driving decision network is regarded as a multi-category classifier, and the classifier is trained using supervised learning. The goal is to make the decision made by the policy network the same as that of human experts. The specific training process is as follows: Sub-step 2.1: Action a in the driving behavior database j Perform one-bit efficient encoding, and the output vector is represented as Sub-step 2.2: Perform a loop of T time steps until the algorithm converges; each loop includes sub-steps 2.2.1 to 2.2.2, specifically: Sub-step 2.2.1: Perform uniform random sampling from the sequence numbers {1, 2, ..., n}; Sub-step 2.2.2: Let the parameters of the current policy network be θ now ; will s j , As input, the gradient is calculated through back propagation, and then the parameter θ is updated using the following formula; In the formula, θ new represents the updated parameters θ, π(·|s j θ now ) represents a random policy network, represents the gradient function of the neural network loss function with parameter θ, and H represents the vector With the random policy network π(·|s j θ now )’s cross entropy; Step 3: Use the generative adversarial imitation learning algorithm to learn the forward collision avoidance driving strategy Based on the decision model trained in step 2, the forward collision avoidance driving strategy is continued using generative adversarial imitation learning, including sub-steps 3.1 to 3.3: Sub-step 3.1: Define the basic parameters of the generative adversarial imitation learning network, including sub-steps 3.1.1 to 3.1.2: Sub-step 3.1.1: Define the state space The state space is constructed using time series data and occupancy grid graphs; the specific description is as follows: S t =[S1(t),S2(t),S3(t)] (4) In the formula, S t represents the state space at time t, S1(t) represents the state space related to the time series data at time t, S2(t) represents the state space related to the relative position relationship of the occupied grid at time t, and S3(t) represents the state space related to the relative speed relationship of the occupied grid at time t; First, the motion state of the vehicle is described using continuous position, velocity, acceleration, and heading information: S1(t)=[p x ,p y ,v x ,v y ,a x ,a y ,θ s ] (5) In the formula, p x ,p y Respectively represent the lateral position and longitudinal position of the vehicle, in meters, v x ,v y They represent the lateral speed and longitudinal speed of the vehicle, in meters per second, and a x ,a y They represent the lateral acceleration and longitudinal acceleration of the vehicle, respectively, in meters per second squared, and θ s Indicates the heading angle of the vehicle, in degrees; Secondly, the road area is rasterized and divided into several p×q grid areas, and the road area and vehicle targets are abstracted into grid maps; where p represents the length of the grid area and q represents the width of the grid area; the constructed grid maps include two types: the "existence" grid map S2(t) used to describe the relative position relationship and the "speed" grid map S3(t) used to describe the relative speed relationship; The "existence" grid map contains four attributes, including grid coordinates, whether there is a vehicle, the type of the corresponding vehicle, and the distance from the left and right lane lines; the grid without traffic participants is set to "0", and the grid with traffic participants is set to "1". The position distribution of this grid and the grid where the vehicle is located is used to describe the relative distance between the two vehicles; the "speed" grid map contains three attributes, including grid coordinates, the speed of the corresponding vehicle, and the distance from the left and right lane lines; the grid without traffic participants is set to "0", and the value in the grid with traffic participants represents the relative speed in meters per second; Sub-step 3.1.2: Define the action space The action space is defined as discrete horizontal and vertical actions: A t =[a left ,a straight ,a right ,a accel ,a remain ,a decel ] (6) In the formula, A t represents the action space at time t, a left ,a straight ,a right They represent left turn, straight ahead and right turn respectively. accel ,a remain ,a decel They represent acceleration, uniform speed and deceleration respectively; Sub-step 3.2: Construct a generative adversarial imitation learning network, specifically including sub-steps 3.2.1 to 3.2.2: Sub-step 3.2.1: Build the generator network Let the generator network be π(A t |S t ; θ), where θ represents the parameters of the generator network; the input of the generator network is the state space, and the output is the vector f = π(·|S t ; θ), the dimension of the output vector f is 6, each element of which corresponds to an action in the action space and represents the probability of executing the action; Sub-step 3.2.2: Build the discriminator network The discriminator is denoted as D(S t ,A t ; φ), where φ represents the parameters of the discriminator network; the input of the discriminator network is the state space, and the output is the vector Output vector The dimension is 6, each element of which corresponds to an action, and an element is recorded as: In the formula, Close to 1 means that the action comes from the driving behavior database. Close to 0 means the action is generated by the generator network; Sub-step 3.3: Training the Generative Adversarial Imitation Learning Network Initialize the generator network parameters θ0 and the discriminator network parameters φ0; Next, perform a loop of N time steps until the algorithm converges; each loop includes substeps 3.3.1 to 3.3.5, specifically: Sub-step 3.3.1: Uniformly sample a trajectory from the "human-like" driving behavior database constructed in step 1, denoted as: In the formula, τ real represents the trajectory sampled from the driving behavior database, represents the state space at time m, represents the driving action at time m; Sub-step 3.3.2: Use the generator network π(a|s; θ now ) controls the agent to interact with the environment and obtains a trajectory, recorded as: In the formula, τ fake represents the trajectory generated by the generator, represents the state space at time n, represents the driving action at time n; Sub-step 3.3.3: Use the discriminator to evaluate whether the decision of the generator network is realistic: In the formula, u t represents the return value at time t, represents the state space generated by the generator at time t, represents the action space executed by the generator at time t, φ now Represents the parameters of the current discriminator network; Sub-step 3.3.4: Set τ fake and u1,u2,...u n As input, update the parameters of the generator network using the following formula: In the formula, θ new represents the updated generator network parameters, Δ is the hyperparameter, represents the objective function, which is calculated by the following formula: In the formula, θ now Represents the parameters of the current generator network; Sub-step 3.3.5: Set τ real and τ fake As input, update the parameters of the discriminator network using the following formula: In the formula, φ new represents the updated discriminator network parameters, φ now represents the parameters of the current discriminator network, η represents the learning rate, represents the gradient of the discriminator network, F(τ real ,τ fake ; φ now ) represents the loss function, which is calculated by the following formula: When the training is completed, the state space information is used as input and the high-level driving strategy is output, realizing the parameters of the forward collision avoidance driving decision network for commercial vehicles in open interference scenarios.

Images