A behavior decision system for autonomous vehicles based on enhanced perception

Abstract

The present invention discloses an enhanced perception-based autonomous driving vehicle behavior decision system, which relates to the field of autonomous driving technology, and includes an enhanced perception module and a behavior decision module; the enhanced perception module is used to construct a spatiotemporal graph based on real vehicles and ghost vehicles around the autonomous driving vehicle, collect a state environment image data set of real vehicles around the autonomous driving vehicle, construct the ghost vehicle based on the state environment image data set, and predict the future states of all vehicles in the spatiotemporal graph in parallel based on the spatiotemporal graph, wherein the ghost vehicle refers to a vehicle that is virtually constructed without a target direction position point around the autonomous driving vehicle; the behavior decision module is used to obtain lane change decisions and speed decisions based on the future state, based on deep reinforcement learning prediction and hybrid reward methods. The present invention is widely applicable, and has stable high reliability in the prediction accuracy of future trajectories; and predicting the future states of all target vehicles in parallel improves prediction efficiency.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an autonomous driving vehicle behavior decision system based on enhanced perception. Background Art

[0002] Due to the development of urbanization and the popularity of private cars, most cities in the world are plagued by traffic congestion. Generally speaking, the road environment and driver behavior are the main causes of traffic congestion. Among them, lane construction and road merging in the road environment are prone to cause traffic jams. In addition, the driver's bad driving behavior (such as sudden braking and forced lane changes) is an important factor leading to traffic congestion and even traffic accidents. Due to some shortcomings of human drivers such as limited vision and delayed reaction, this kind of man-made traffic congestion is more frequent than the congestion caused by the road environment. Among them, in high-density traffic scenes, slight traffic fluctuations caused by the driver's bad driving behavior are prone to produce a "domino effect", which spreads to the entire traffic flow and eventually leads to serious traffic congestion. In order to avoid this phenomenon, the driver needs to stay calm and concentrate at all times to maintain a safe distance between vehicles. This is extremely challenging for human drivers.

[0003] With the rapid development of vehicle automation technology, the goal of unmanned driving can be achieved when most vehicles on the road in the future are self-driving cars. Traditional methods have proved that self-driving cars can maintain a constant distance from surrounding vehicles with the help of adaptive cruise control and lane change models. However, these methods involve a set of rule matching algorithms, and these algorithms require tedious manual debugging and testing experience. As the scenes of self-driving cars become more and more complex, these methods are difficult to achieve excellent driving performance. Due to the flexible reward design and superior optimization effect, a large number of works have used reinforcement learning-based methods to complete vehicle behavior decisions in self-driving scenarios. These reinforcement learning-based methods mainly optimize the driving safety, efficiency and comfort of self-driving cars, but do not consider its impact on other vehicles around and the final traffic conditions. Obviously, if the self-driving car makes behavioral decisions only based on its own state without considering the driving conditions of surrounding vehicles, it is easy to cause serious traffic congestion or even accidents. Recently, researchers have proposed a prediction plus search framework to make discrete behavioral decisions for self-driving cars. It ideally discretizes speed changes into three behaviors: acceleration, deceleration and speed maintenance, but does not consider the difference in the magnitude of acceleration and deceleration. The discrete speed change behavior makes this method unable to fully consider the degree of impact on traffic flow.

[0004] In general, existing autonomous driving decision-making algorithms are unable to effectively reduce the impact of autonomous vehicles on the vehicles around them. To achieve this goal, autonomous vehicles need to predict the trajectories of surrounding vehicles to make better decisions. Recently, many researchers have used RNN-based encoder-decoder architectures to predict future vehicle trajectories. However, directly using these learning-based trajectory prediction models usually leads to unsatisfactory prediction results for the following reasons:

[0005] (1) They have poor applicability because they assume that the states of all surrounding conventional vehicles are always observable. In reality, the perception capabilities of autonomous vehicles will be limited by sensors, such as detection range limitations and occlusion problems.

[0006] (2) Their prediction accuracy of future trajectories decreases over time, and only the first few predicted states are reliable.

[0007] (3) Their inference process is time-consuming because they do not support parallel prediction and require separate computation for each predicted vehicle.

[0008] Therefore, directly using the trajectory prediction model to enhance the traditional perception module will not only reduce the perception efficiency, but also introduce additional errors to the downstream decision-making module. Summary of the invention

[0009] The present invention provides an autonomous driving vehicle behavior decision system based on enhanced perception, which can alleviate the above-mentioned problems.

[0010] In order to alleviate the above-mentioned problems, the technical solution adopted by the present invention is as follows:

[0011] The present invention provides an autonomous driving vehicle behavior decision system based on enhanced perception, comprising an enhanced perception module and a behavior decision module; the enhanced perception module is used to construct a space-time graph according to real vehicles and ghost vehicles around the autonomous driving vehicle, collect a state environment image data set of real vehicles around the autonomous driving vehicle, construct the ghost vehicle according to the state environment image data set, and predict the future states of all vehicles in the space-time graph in parallel according to the space-time graph, wherein the ghost vehicle refers to a vehicle that is virtually constructed but does not exist at a target direction position point around the autonomous driving vehicle; the behavior decision module is used to obtain lane change decisions and speed decisions according to the future states based on deep reinforcement learning prediction and a hybrid reward method.

[0012] In a preferred embodiment of the present invention, the enhanced perception module includes an image data acquisition module and a graph-based state prediction model;

[0013] The image data acquisition module is used to collect a state environment image dataset of real vehicles around the autonomous driving vehicle;

[0014] The state prediction model includes a ghost vehicle building module, a spatiotemporal graph building module, and a graph attention mechanism prediction module with LSTM;

[0015] The ghost vehicle construction module is used to construct the ghost vehicle according to the state environment image dataset;

[0016] The spatiotemporal graph construction module is used to construct a spatiotemporal graph based on real vehicles and ghost vehicles around the autonomous driving vehicle;

[0017] The graph attention mechanism prediction module is used to predict the future states of all vehicles in the spatiotemporal graph in parallel according to the spatiotemporal graph;

[0018] The behavior decision module includes a mixed reward function and a prediction model based on deep reinforcement learning;

[0019] The hybrid reward function and the prediction model based on deep reinforcement learning are used to obtain lane change decisions and speed decisions based on the future state based on deep reinforcement learning and hybrid reward methods.

[0020] In a preferred embodiment of the present invention, the target direction position point includes a first target direction position point and a second target direction position point;

[0021] There are six first target direction position points, including left front, front, right front, left rear, rear and right rear position points of the autonomous driving vehicle;

[0022] For each first target direction position, its left front, front, right front, left rear, rear and right rear position points are all second target direction position points.

[0023] In a preferred embodiment of the present invention, the method in which the graph attention mechanism prediction module predicts the future states of all vehicles in the spatiotemporal graph includes:

[0024] S1, aggregating vehicle nodes with different importance scores in the spatiotemporal graph, and for each vehicle node, updating vehicle node state information by interacting with the vehicles around it;

[0025] S2. For each vehicle in the spatiotemporal graph, inputting the vehicle node state information at different time steps into the LSTM;

[0026] S3. For each vehicle in the space-time graph, LSTM captures its temporal dependency from the vehicle node state information at different time steps, and LSTM outputs the future states of all vehicles in the space-time graph in parallel.

[0027] In a preferred embodiment of the present invention, in step S1, for each time-space diagram Each vehicle The calculation formula of the importance score α is as follows:

[0028]

[0029] Among them, φ1 and φ2 are two fully connected layers, Represents the historical information of each vehicle;

[0030] For each vehicle in the spatiotemporal graph, the update formula of its vehicle node state information is as follows:

[0031]

[0032] Among them, φ3 represents a fully connected layer, and * indicates that the historical information of each vehicle is multiplied by its importance score.

[0033] In a preferred embodiment of the present invention, in step S3, for each vehicle in the time-space diagram, the calculation formula of its time dependency is as follows:

[0034]

[0035] Among them, W l represents the learning parameters of the LSTM layer, and Represents vehicle status information at two different times.

[0036] In a preferred embodiment of the present invention, in step S3, for each vehicle in the time-space diagram, the calculation formula for its future state is as follows:

[0037]

[0038] Among them, φ4 represents a fully connected layer, b4 represents the bias term of the fully connected layer, Represents vehicle C in the space-time graph i future state information.

[0039] In a preferred embodiment of the present invention, the x network in the prediction model based on deep reinforcement learning is used to output candidate actions, the Q network is used to evaluate the quality of all candidate actions according to the mixed reward function, and the best candidate action is used as the lane change decision and speed decision;

[0040] The lane change decision is discretized into changing lanes to the left, not changing lanes, and changing lanes to the right, and the speed decision is to maintain a continuous speed action space.

[0041] In a preferred embodiment of the present invention, the hybrid reward function includes a safety reward function, an efficiency reward function, a comfort reward function and an impact reward function on the following vehicle.

[0042] The safety reward function is

[0043]

[0044] in, For the safety bonus, the TTC indicates how long it will take for two vehicles to collide if they maintain their current speed. is a threshold of TTC. If two vehicles have collided, the autonomous vehicle will receive a penalty value of -3.

[0045] The efficiency reward function is

[0046]

[0047] in, is the efficiency bonus value, A t+1 .v represents the speed of the autonomous vehicle, v max and v min Indicates the maximum and minimum speeds of the autonomous vehicle;

[0048] The comfort reward function is

[0049]

[0050] in, is the comfort bonus value, A t .a and A t-1 .a represents the acceleration value of the autonomous vehicle at the current moment and the previous moment, 2a ′ Indicates the maximum acceleration change of the autonomous vehicle;

[0051] The reward function for the impact on the following vehicle is:

[0052]

[0053] in, is the comfort bonus value, and They represent the speed values ​​of the following vehicle C5 at the current moment and the next moment, respectively, and 2a ′ *Δt represents the maximum speed change of the autonomous vehicle between two consecutive time steps, v thr It is a speed threshold used to determine whether the vehicle behind the autonomous vehicle is affected.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] (1) The present invention has better applicability. It constructs a ghost vehicle to fill the problem of missing vehicle detection caused by sensor occlusion and line of sight. The perception ability of the autonomous driving vehicle is not limited by the sensor and can better deal with some emergencies, such as sudden vehicle cutting in and emergency braking.

[0056] (2) The prediction accuracy of the present invention for future trajectories will not decrease over time, and has stable high reliability;

[0057] (3) The present invention adopts a parallel prediction model based on the graph attention mechanism to predict the future states of all target vehicles in parallel, thereby improving the prediction efficiency.

[0058] (4) The present invention ensures the safety, efficiency and comfort requirements of the autonomous driving vehicle when making decisions, and greatly reduces the impact of the autonomous driving vehicle's emergency speed and lane changes on the traffic behind, thereby improving the traffic efficiency and stability of the entire traffic flow.

[0059] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the embodiments of the present invention are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0061] Figure 1 The overall architecture diagram of the behavioral decision framework for enhancing perception of the present invention;

[0062] Figure 2 The network structure diagram of the module LST-GAT for enhancing perception of the present invention;

[0063] Figure 3 This is a network structure diagram of the BP-DQN module for behavior decision making in the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0065] Please refer to Figure 1 to Figure 3The present invention discloses an autonomous driving vehicle behavior decision system based on enhanced perception, including an enhanced perception module and a behavior decision module.

[0066] The enhanced perception module includes an image data acquisition module and a graph-based state prediction model (LST-GAT). The state prediction model here includes a ghost vehicle construction module, a spatiotemporal graph construction module, and a graph attention mechanism prediction module with LSTM.

[0067] The behavior decision module includes a mixed reward function and a prediction model based on deep reinforcement learning.

[0068] like Figure 1 As shown, during the implementation of the system of the present invention, the process of making behavior decisions for the autonomous driving vehicle is as follows:

[0069] First, the enhanced perception module obtains the state environment image dataset of real vehicles around the autonomous driving vehicle through the on-board sensors (image data acquisition module).

[0070] In the present invention, six vehicles are selected from all observable vehicles around the autonomous vehicle, namely

[0071] C tar (C tar ={C1,C2,…,C6}).

[0072] The six vehicles {C1, C2, …, C6} are respectively located at the six first target direction positions: left front, front, right front, left rear, rear and right rear of the autonomous driving vehicle.

[0073] Because we need to predict the status of the six vehicles {C1, C2, …, C6} at the next moment, we also need the information of the six vehicles around them, which are the vehicles at the six target direction positions: left front, front, right front, left rear, rear and right rear, that is, the information of the vehicles at the second target direction position.

[0074] There is usually a problem of missing vehicles. We have summarized the following three situations of missing vehicles:

[0075] 1) Missing range: The limited sensor detection range results in the vehicle not being detected;

[0076] 2) Missing occlusion: Mutual occlusion between vehicles leads to inaccurate sensor information;

[0077] 3) Intrinsic missingness, which is irrelevant to the sensor. Neighbors on one side may be missing on the far left or right side of the road.

[0078] Therefore, for these missing vehicles, the system of the present invention utilizes a ghost vehicle construction module to construct a ghost vehicle according to the state environment image data set to complete the missing vehicles at the target direction position points.

[0079] Finally, the spatiotemporal graph construction module in the system of the present invention constructs a spatiotemporal graph G(t) based on the real vehicles and ghost vehicles around the autonomous vehicle.

[0080] like Figure 2 As shown, the system of the present invention inputs the spatiotemporal graph G(t) into the graph attention mechanism prediction module of LSTM to predict the future states of all vehicles in the spatiotemporal graph in parallel, as follows:

[0081] 1) Aggregate vehicle nodes with different importance scores in the spatiotemporal graph. For each vehicle node, it calculates the importance score of the vehicle node and updates the vehicle node status information through interaction with the vehicles around it.

[0082] For each moment in time Each vehicle The calculation formula of the importance score α is as follows:

[0083]

[0084] Among them, φ1 and φ2 are two fully connected layers, Represents the historical information of each vehicle;

[0085] Next, this importance score is used to update the status information of each vehicle node. For each vehicle in the spatiotemporal graph, the update formula of its vehicle node status information is as follows:

[0086]

[0087] Among them, φ3 represents a fully connected layer, and * indicates that the historical information of each vehicle is multiplied by its importance score.

[0088] 2) For each vehicle in the space-time graph, its vehicle node state information at different time steps is input into LSTM. LSTM captures its temporal dependency from its vehicle node state information at different time steps, and LSTM outputs the future states of all vehicles in the space-time graph in parallel.

[0089] The present invention uses LSTM to learn the time dependency of the historical information of vehicles. For each vehicle in the space-time graph, the calculation formula of its time dependency is as follows:

[0090]

[0091] Among them, W lrepresents the learning parameters of the LSTM layer, and Represents vehicle status information at two different times.

[0092] For each vehicle in the space-time graph, the calculation formula for its future state is as follows:

[0093]

[0094] Among them, φ4 represents a fully connected layer, b4 represents the bias term of the fully connected layer, Represents vehicle C in the space-time graph i future state information.

[0095] Here, the present invention predicts the future states of all target vehicles in parallel to improve the efficiency of the prediction model.

[0096] Next, if Figure 3 As shown, the system of the present invention inputs the future state as an enhanced state into the behavior decision module. The behavior decision module enables the autonomous vehicle to learn to perform excellent lane change decisions and speed decisions through a prediction model (BP-DQN) based on deep reinforcement learning and a hybrid reward function, as follows:

[0097] In the system of the present invention, after the future state is input into BP-DQN, the x network of BP-DQN is used to output candidate actions, and the Q network is used to evaluate the quality of all candidate actions according to the mixed reward function, and the best candidate action is used as the lane change decision and speed decision.

[0098] In the system of the present invention, the lane change decision is discretized into three actions: changing lanes to the left, not changing lanes, and changing lanes to the right; the speed decision maintains a continuous action space with a value of -3m / s 2 Up to 3m / s 2 between.

[0099] In the system of the present invention, the mixed reward function includes a safety reward function, an efficiency reward function, a comfort reward function and a reward function for the impact on the following vehicle. This part plays an important role in evaluating the actions of the autonomous driving vehicle and guiding the learning process of the autonomous driving vehicle.

[0100] For the safety bonus value, the system of the present invention uses the TTC of the autonomous driving vehicle to calculate the value, and the calculation formula is as follows:

[0101]

[0102] in, For the safety bonus, the TTC indicates how long it will take for two vehicles to collide if they maintain their current speed. It is a threshold of TTC. If two vehicles have collided, the autonomous vehicle will receive a penalty value of -3.

[0103] For the efficiency bonus value, the system of the present invention uses the speed of the autonomous driving vehicle to calculate the efficiency bonus value, and the calculation formula is as follows:

[0104]

[0105] in, is the efficiency bonus value, A t+1 .v represents the speed of the autonomous vehicle, v max and v min Indicates the maximum and minimum speed of the autonomous vehicle.

[0106] For the bonus value of comfort, the system of the present invention uses the change of acceleration of the autonomous driving vehicle to calculate the value, and the calculation formula is as follows:

[0107]

[0108] in, is the comfort bonus value, A t .a and A t-1 .a represents the acceleration value of the autonomous vehicle at the current moment and the previous moment, 2a ′ Represents the maximum acceleration change of the autonomous vehicle.

[0109] For the reward value of the impact on the following vehicle, the system of the present invention uses the speed change of the following vehicle C5 of the autonomous driving vehicle to calculate the reward value, and the calculation formula is as follows:

[0110]

[0111] in, is the comfort bonus value, and They represent the speed values ​​of the following vehicle C5 at the current moment and the next moment, respectively, and 2a ′ *Δt represents the maximum speed change of the autonomous vehicle between two consecutive time steps, v thr It is a speed threshold used to determine whether the vehicle behind the autonomous vehicle is affected.

[0112] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An autonomous driving vehicle behavior decision system based on enhanced perception, characterized in that: It includes an enhanced perception module and a behavior decision module; the enhanced perception module is used to construct a spatiotemporal graph based on real vehicles and ghost vehicles around the autonomous driving vehicle, collect a state environment image dataset of real vehicles around the autonomous driving vehicle, construct the ghost vehicle based on the state environment image dataset, and predict the future states of all vehicles in the spatiotemporal graph in parallel based on the spatiotemporal graph, wherein the ghost vehicle refers to a virtually constructed vehicle that does not exist at a target direction position point around the autonomous driving vehicle; The behavior decision module is used to obtain lane change decisions and speed decisions according to the future state based on deep reinforcement learning prediction and hybrid reward method.

2. The autonomous driving vehicle behavior decision system based on enhanced perception according to claim 1, characterized in that: The enhanced perception module includes an image data acquisition module and a graph-based state prediction model; The image data acquisition module is used to collect a state environment image dataset of real vehicles around the autonomous driving vehicle; The state prediction model includes a ghost vehicle building module, a spatiotemporal graph building module, and a graph attention mechanism prediction module with LSTM; The ghost vehicle construction module is used to construct the ghost vehicle according to the state environment image dataset; The spatiotemporal graph construction module is used to construct a spatiotemporal graph based on real vehicles and ghost vehicles around the autonomous driving vehicle; The graph attention mechanism prediction module is used to predict the future states of all vehicles in the spatiotemporal graph in parallel according to the spatiotemporal graph; The behavior decision module includes a mixed reward function and a prediction model based on deep reinforcement learning; The hybrid reward function and the prediction model based on deep reinforcement learning are used to obtain lane change decisions and speed decisions based on the future state based on deep reinforcement learning and hybrid reward methods.

3. The autonomous driving vehicle behavior decision system based on enhanced perception according to claim 2, characterized in that: The target direction position point includes a first target direction position point and a second target direction position point; There are six first target direction position points, including left front, front, right front, left rear, rear and right rear position points of the autonomous driving vehicle; For each first target direction position, its left front, front, right front, left rear, rear and right rear position points are all second target direction position points.

4. The autonomous driving vehicle behavior decision system based on enhanced perception according to claim 3, characterized in that: The method in which the graph attention mechanism prediction module predicts the future states of all vehicles in the spatiotemporal graph includes: S1, aggregating vehicle nodes with different importance scores in the spatiotemporal graph, and for each vehicle node, collecting information of six vehicles around it and calculating the state information of the vehicle node through a neural network; S2. For each vehicle in the spatiotemporal graph, inputting the vehicle node state information at different time steps into the LSTM; S3. For each vehicle in the space-time graph, LSTM captures its temporal dependency from the vehicle node state information at different time steps, and LSTM outputs the future states of all vehicles in the space-time graph in parallel.

5. The autonomous driving vehicle behavior decision system based on enhanced perception according to claim 4, characterized in that: In step S1, for each time-space diagram Each vehicle The calculation formula of the importance score α is as follows: Among them, φ1 and φ2 are two fully connected layers, Represents the historical information of each vehicle; For each vehicle in the spatiotemporal graph, the update formula of its vehicle node state information is as follows: Among them, φ3 represents a fully connected layer, and * indicates that the historical information of each vehicle is multiplied by its importance score.

6. The autonomous driving vehicle behavior decision system based on enhanced perception according to claim 5, characterized in that: In step S3, for each vehicle in the space-time graph, the time dependency is calculated as follows: Among them, W l represents the learning parameters of the LSTM layer, and Represents vehicle status information at two different times.

7. The autonomous driving vehicle behavior decision system based on enhanced perception according to claim 6, characterized in that: In step S3, for each vehicle in the space-time graph, the calculation formula for its future state is as follows: Among them, φ4 represents a fully connected layer, b4 represents the bias term of the fully connected layer, Represents vehicle C in the space-time graph i future state information.

8. The autonomous driving vehicle behavior decision system based on enhanced perception according to claim 7, characterized in that: The x network in the prediction model based on deep reinforcement learning is used to output candidate actions, and the Q network is used to evaluate the quality of all candidate actions according to the mixed reward function, and the best candidate action is used as the lane change decision and speed decision; The lane change decision is discretized into changing lanes to the left, not changing lanes, and changing lanes to the right, and the speed decision is to maintain a continuous speed action space.

9. The autonomous driving vehicle behavior decision system based on enhanced perception according to claim 8, characterized in that: The mixed reward function includes a safety reward function, an efficiency reward function, a comfort reward function and an impact reward function on the following vehicle. The safety reward function is in, For the safety bonus, the TTC indicates how long it will take for two vehicles to collide if they maintain their current speed. is a threshold of TTC. If two vehicles have collided, the autonomous vehicle will receive a penalty value of -3. The efficiency reward function is in, is the efficiency bonus value, A t+1 .v represents the speed of the autonomous vehicle, v max and v min Indicates the maximum and minimum speeds of the autonomous vehicle; The comfort reward function is in, is the comfort bonus value, A t .a and A t-1 .a represents the acceleration value of the autonomous vehicle at the current moment and the previous moment, 2a ′ Indicates the maximum acceleration change of the autonomous vehicle; The reward function for the impact on the following vehicle is: in, is the comfort bonus value, and They represent the speed values ​​of the following vehicle C5 at the current moment and the next moment, respectively, and 2a ′ *Δt represents the maximum speed change of the autonomous vehicle between two consecutive time steps, v thr It is a speed threshold used to determine whether the vehicle behind the autonomous vehicle is affected.

Images