True value generation method and device of decision planning algorithm, storage medium and equipment

Abstract

The invention discloses a true value generation method and device of a decision planning algorithm, a storage medium and equipment, and relates to the field of intelligent driving. The method comprises the steps that at least one empty map scene is obtained, and the empty map scene comprises a road topological structure of a road section extracted from a static map; simulating the traffic flow in the empty map scene according to the traffic data containing the driving behavior of the real driver, and obtaining a simulation scene corresponding to the empty map scene; based on a tree search algorithm and the state space and decision space at the current moment, the state space at the future moment is simulated in the simulation scene, multiple decision planning paths of the vehicle are obtained, the state space comprises the driving state of the vehicle and the states of other traffic objects, and the decision space comprises a decision set of the vehicle; and taking the decision planning path with the highest evaluation in the plurality of decision planning paths as a true value of the decision planning algorithm. According to the invention, the truth value required by the decision planning algorithm can be obtained through an analogue simulation technology.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and more specifically, to a method, device, storage medium and equipment for generating a truth value of a decision-making planning algorithm. Background Art

[0002] Autonomous driving decision planning is the core link of the autonomous driving system. Its main task is to understand the current environment, predict the dynamic behavior of other traffic participants, and generate an optimal driving path based on this while ensuring safety, comfort and driving efficiency. This involves a series of continuous decisions, such as vehicle speed control, steering angle adjustment, lane changing and other operations.

[0003] When evaluating or optimizing a decision planning algorithm, it is necessary to compare the true value of the decision planning path with the predicted value. At present, the method for obtaining the true value of the decision planning path mainly includes: collecting the decision planning paths of multiple expert drivers in multiple driving scenarios as the true value. However, this method not only requires a lot of manpower, but also may have the problem that the expert-level decision planning paths of expert drivers cannot be collected in some special scenarios with high complexity. Summary of the invention

[0004] The present application provides a method, device, storage medium and equipment for generating a truth value of a decision planning algorithm, which can solve the problem that the method of obtaining the truth value through an expert driver not only requires a lot of manpower, but also may not be able to collect the expert-level decision planning path of the expert driver in some special scenarios with high complexity.

[0005] The specific technical solutions are as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for generating a truth value of a decision planning algorithm, the method comprising:

[0007] Acquire at least one empty map scene, wherein the empty map scene includes a road topology structure of a road segment extracted from a static map;

[0008] Simulating the traffic flow in the empty scene according to the traffic data including the driving behavior of real drivers, and obtaining a simulation scene corresponding to the empty scene;

[0009] Based on the tree search algorithm, the state space at the current moment and the decision space, the state space at the future moment is simulated in the simulation scene, and multiple decision planning paths of the ego vehicle are obtained, wherein the state space includes the driving state of the ego vehicle and the states of other traffic objects, and the decision space includes the decision set of the ego vehicle;

[0010] The decision planning path with the highest evaluation among the multiple decision planning paths is used as the true value of the decision planning algorithm.

[0011] It can be seen from the above scheme that the embodiment of the present application can construct at least one simulation scene containing traffic flow by means of simulation, and search out multiple decision planning paths by means of a tree search algorithm, and select the optimal decision planning path from them as the true value of the decision planning algorithm. It can be seen that compared with taking the decision planning path of the expert driver as the true value, the embodiment of the present application can not only be separated from the expert driver, but also use the advantages of simulation to obtain the decision planning path of special scenes with high complexity, thereby not only improving the efficiency of truth value generation, but also broadening the scene range of truth value generation.

[0012] In a first possible implementation manner of the first aspect, based on a tree search algorithm, a state space at a current moment, and a decision space, simulating a state space at a future moment in the simulation scenario, and obtaining multiple decision planning paths of the vehicle include:

[0013] Based on the state space at the current moment and the decision space, simulating in the simulation scene the traffic flow after the vehicle selects each decision in the decision space, determining the state space at the next moment, and using the state space at the next moment as the new state space at the current moment;

[0014] Continue to execute the step based on the state space at the current moment and the decision space, simulate the traffic flow after the self-vehicle selects each decision in the decision space in the simulation scene, determine the state space at the next moment, until the state space at the target moment is obtained, and generate a decision tree, wherein the decision tree includes a state node layer and a decision node layer, the state node layer includes a state space root node layer and at least one state space child node layer, the time difference between each two adjacent state node layers is the same, and a decision node layer is included between each two adjacent state node layers, and each node in the decision node layer represents a decision in the decision space;

[0015] Each decision planning path from the root node to the leaf node in the decision tree is traversed to obtain multiple decision planning paths of the vehicle.

[0016] In a second possible implementation manner of the first aspect, the method further includes:

[0017] In the process of generating the decision tree, determining whether the state space represented by each state space node in the newly generated state space sub-node layer satisfies a preset risk state;

[0018] When the state space represented by the state space node to be judged satisfies the preset risk state, the state space node to be judged is pruned.

[0019] Through the above scheme, it can be seen that the embodiment of the present application can significantly reduce the size of the decision tree and improve the efficiency of the search by pruning the state space nodes that meet the preset risk state. At the same time, it can also ensure that the final decision path is safe and reasonable, thereby guiding the decision planning of the unmanned driving system.

[0020] In a third possible implementation manner of the first aspect, simulating the traffic flow in the empty scene according to the traffic data including the driving behavior of the real driver to obtain the simulation scene corresponding to the empty scene includes:

[0021] Performing intelligent driving model (IDM) parameter identification on traffic data containing real driver driving behavior to obtain at least one parameter value that can reflect the real driver driving behavior;

[0022] A virtual driver having similar behavior to the real driver is simulated in the empty graph scene using the at least one parameter value to generate a traffic flow in the empty graph scene, and the empty graph scene including the traffic flow is determined as the simulation scene corresponding to the empty graph scene.

[0023] It can be seen from the above scheme that the embodiment of the present application can utilize IDM parameter identification to simulate a virtual driver with similar behavior to a real driver in a simulation environment, thereby generating a simulation scene that is close to or equivalent to the real traffic flow, thereby improving the accuracy of the true value of the decision-making planning algorithm.

[0024] In a fourth possible implementation manner of the first aspect, simulating the traffic flow in the empty scene according to traffic data including the driving behavior of real drivers to obtain a simulation scene corresponding to the empty scene includes:

[0025] The traffic flow is simulated in the empty map scene based on a pre-trained decision planning model to obtain the simulation scene corresponding to the empty map scene, wherein the decision planning model is trained based on the traffic data containing the driving behavior of real drivers.

[0026] It can be seen from the above scheme that the embodiment of the present application can utilize the decision-making planning model trained based on traffic data of real drivers' driving behavior to simulate a simulation scene that is close to or equivalent to the real traffic flow in a simulation environment, thereby improving the accuracy of the true value of the decision-making planning algorithm.

[0027] In a fifth possible implementation manner of the first aspect, taking the decision planning path with the highest evaluation among the multiple decision planning paths as the true value of the decision planning algorithm includes:

[0028] For each of the decision-making planning paths, determining the comfort of the vehicle on the decision-making planning path according to the driving state of the vehicle at each moment;

[0029] Determining the safety of the vehicle on the decision-making planning path by judging whether the vehicle complies with traffic rules and whether it can make correct responses when faced with emergencies;

[0030] By judging whether the decision-making planning path meets the preset driving requirements, determining the rationality of the vehicle on the decision-making planning path;

[0031] Performing weighted calculation on the comfort, the safety and the rationality to obtain an evaluation value of the decision-making planning path;

[0032] The decision planning path with the highest evaluation value is selected as the true value of the decision planning algorithm.

[0033] It can be seen from the above scheme that the embodiment of the present application can comprehensively evaluate the decision-making planning path from multiple aspects such as comfort, safety and rationality, thereby improving the accuracy of the true value of the decision-making planning algorithm.

[0034] In a second aspect, an embodiment of the present application provides a truth value generation device for a decision planning algorithm, the device comprising:

[0035] An acquisition unit, configured to acquire at least one empty map scene, wherein the empty map scene includes a road topology structure of a road segment extracted from a static map;

[0036] A simulation unit, used to simulate the traffic flow in the empty scene according to the traffic data including the driving behavior of real drivers, and obtain a simulation scene corresponding to the empty scene;

[0037] A search unit, configured to simulate a state space at a future time in the simulation scene based on a tree search algorithm, a state space at a current time, and a decision space, and obtain a plurality of decision-making planning paths of the ego vehicle, wherein the state space includes a driving state of the ego vehicle and states of other traffic objects, and the decision space includes a decision set of the ego vehicle;

[0038] A determination unit is used to use the decision planning path with the highest evaluation among the multiple decision planning paths as the true value of the decision planning algorithm.

[0039] In a first possible implementation manner of the second aspect, the searching unit includes:

[0040] A generation module is used to simulate, in the simulation scenario, the traffic flow after the self-vehicle selects each decision in the decision space based on the state space at the current moment and the decision space, determine the state space at the next moment, use the state space at the next moment as the new state space at the current moment, and continue to execute the steps of simulating, in the simulation scenario, the traffic flow after the self-vehicle selects each decision in the decision space based on the state space at the current moment and the decision space, determine the state space at the next moment, until the state space at the target moment is obtained, and generate a decision tree, wherein the decision tree includes a state node layer and a decision node layer, the state node layer includes a state space root node layer and at least one state space child node layer, the time difference between each two adjacent state node layers is the same, a decision node layer is included between each two adjacent state node layers, and each node in the decision node layer represents a decision in the decision space;

[0041] The traversal module is used to traverse each decision planning path from the root node to the leaf node in the decision tree to obtain multiple decision planning paths of the vehicle.

[0042] In a second possible implementation manner of the second aspect, the searching unit further includes:

[0043] A judgment module, used for judging whether the state space represented by each state space node in the newly generated state space sub-node layer satisfies a preset risk state during the process of generating the decision tree;

[0044] The pruning module is used to prune the state space node to be judged when the state space represented by the state space node to be judged meets the preset risk state.

[0045] In a third possible implementation manner of the second aspect, the simulation unit includes:

[0046] an identification module, used to perform intelligent driving model (IDM) parameter identification on traffic data containing real driver driving behavior, and obtain at least one parameter value that can reflect the real driver driving behavior;

[0047] The first simulation module is used to simulate a virtual driver having similar behavior to the real driver in the empty picture scene by using the at least one parameter value to generate a traffic flow in the empty picture scene, and determine the empty picture scene containing the traffic flow as the simulation scene corresponding to the empty picture scene.

[0048] In a fourth possible implementation manner of the second aspect, the simulation unit includes:

[0049] The second simulation module is used to simulate the traffic flow in the empty map scene based on a pre-trained decision planning model to obtain the simulation scene corresponding to the empty map scene, wherein the decision planning model is trained based on the traffic data containing the driving behavior of real drivers.

[0050] In a fifth possible implementation manner of the second aspect, the determining unit includes:

[0051] A first determination module is used to determine, for each of the decision-making planning paths, the comfort of the vehicle on the decision-making planning path according to the driving state of the vehicle at each moment;

[0052] A second determination module is used to determine the safety of the vehicle on the decision-making planning path by judging whether the vehicle complies with traffic regulations and whether it can make a correct response when facing an emergency;

[0053] A third determination module is used to determine the rationality of the vehicle on the decision-making planning path by judging whether the decision-making planning path meets the preset driving requirements;

[0054] A weighting module, used for performing weighted calculation on the comfort, the safety and the rationality to obtain an evaluation value of the decision-making planning path;

[0055] The fourth determination module is used to select the decision planning path with the highest evaluation value as the true value of the decision planning algorithm.

[0056] It can be seen from the above scheme that the embodiment of the present application can construct at least one simulation scene containing traffic flow by means of simulation, and search out multiple decision planning paths by means of a tree search algorithm, and select the optimal decision planning path from them as the true value of the decision planning algorithm. It can be seen that compared with taking the decision planning path of the expert driver as the true value, the embodiment of the present application can not only be separated from the expert driver, but also use the advantages of simulation to obtain the decision planning path of special scenes with high complexity, thereby not only improving the efficiency of truth value generation, but also broadening the scene range of truth value generation.

[0057] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any possible implementation manner of the first aspect.

[0058] In a fourth aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0059] one or more processors;

[0060] The processor is coupled to a storage device, and the storage device is used to store one or more programs;

[0061] When one or more programs are executed by one or more processors, the electronic device implements the method described in any possible implementation manner of the first aspect.

[0062] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer or a processor, the computer or the processor executes the method described in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work.

[0064] Figure 1 A flowchart of a method for generating a truth value of a decision planning algorithm provided in an embodiment of the present application;

[0065] Figure 2 An example diagram of a decision tree provided in an embodiment of the present application;

[0066] Figure 3 A block diagram of a truth value generation device for a decision planning algorithm provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of the structure of an electronic device or computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0069] It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The terms "including" and "having" in the embodiments of the present application and the accompanying drawings and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0070] Figure 1 The present invention is a flowchart of a method for generating a truth value of a decision planning algorithm. The method can be applied to electronic devices or computer devices, and can be applied to terminals or servers. Specifically, the method can include the following steps:

[0071] S110: Acquire at least one empty image scene.

[0072] The empty map scene includes the road topology structure of the road segment extracted from the static map. These road topology structures cover a variety of road condition changes, such as divergence and confluence areas, intersections, etc., forming a static scene model containing rich information.

[0073] In practical applications, when mining empty map scenes, it is necessary to mine some road sections where accidents of autonomous vehicles are prone to occur. Therefore, some road sections are selected as empty map scenes based on environmental factors, emergencies and other factors.

[0074] In practical applications, specific scenes can be divided in detail from multiple perspectives, including different environmental factors (such as road type, weather conditions, traffic flow, etc.) and various possible emergencies (such as pedestrians crossing, vehicle lane changes, emergency braking, etc.). After obtaining multiple specific scenes, the road topology structure can be extracted from the static map corresponding to each specific scene as the corresponding empty map scene.

[0075] S120: Simulate the traffic flow in the empty scene according to the traffic data including the driving behavior of the real driver, and obtain a simulation scene corresponding to the empty scene.

[0076] After obtaining an empty scene that only includes the road topology, the traffic flow in the empty scene can be simulated based on the traffic data containing the real driver's driving behavior, so that the empty scene is closer to the real traffic scene. The traffic flow includes the driving trajectories of the vehicle, other vehicles, pedestrians and other obstacles over a period of time, etc. The traffic flow is a dynamically changing traffic flow that can change with different decisions in subsequent simulations.

[0077] Methods for simulating traffic flow to obtain simulation scenarios include:

[0078] Method 1: Perform IDM (Intelligent Driver Model) parameter identification on traffic data containing the driving behavior of real drivers to obtain at least one parameter value that can reflect the driving behavior of real drivers; use the at least one parameter value to simulate a virtual driver with similar behavior to the real driver in an empty picture scene to generate traffic flow in the empty picture scene, and determine the empty picture scene containing the traffic flow as the simulation scene corresponding to the empty picture scene.

[0079] IDM is a mathematical model that describes vehicle driving behavior, which includes a set of parameters that can affect vehicle acceleration. Therefore, IDM parameter identification can be performed on traffic data containing real driver driving behavior to obtain at least one parameter value that can reflect the real driver driving behavior, and the at least one parameter value includes acceleration, speed, position, etc.

[0080] It can be seen from the above scheme that the embodiment of the present application can utilize IDM parameter identification to simulate a virtual driver with similar behavior to a real driver in a simulation environment, thereby generating a simulation scene that is close to or equivalent to the real traffic flow, thereby improving the accuracy of the true value of the decision-making planning algorithm.

[0081] Method 2: Based on the pre-trained decision-making planning model, traffic flow is simulated in the empty map scene to obtain the simulation scene corresponding to the empty map scene.

[0082] The decision planning model is trained based on traffic data containing real driver driving behaviors. The decision planning model can be a model trained based on a supervised learning algorithm or a model trained based on a reinforcement learning algorithm.

[0083] A supervised learning algorithm is a machine learning algorithm that requires a set of labeled training data. In this embodiment, a large amount of historical traffic data of real drivers can be used as training data, where each piece of historical traffic data contains the driver's decision, the corresponding driving environment, and the driving trajectory is annotated. Through training, the decision planning model can learn to make decisions similar to those of real drivers in a given driving environment, so that real traffic flow can be simulated. Unlike supervised learning, reinforcement learning does not require labeled data, but gradually learns the best strategy through interaction with the environment. In this process, some rewards and penalties can be set to guide the model to form reasonable driving behavior. Through training, the decision planning model can learn to make effective decisions in a complex driving environment, thereby simulating a traffic flow close to reality.

[0084] It can be seen from the above scheme that the embodiment of the present application can utilize the decision-making planning model trained based on traffic data of real drivers' driving behavior to simulate a simulation scene that is close to or equivalent to the real traffic flow in a simulation environment, thereby improving the accuracy of the true value of the decision-making planning algorithm.

[0085] S130: Based on the tree search algorithm, the state space and decision space at the current moment, the state space at the future moment is simulated in the simulation scene, and multiple decision planning paths of the ego vehicle are obtained.

[0086] The state space includes the driving state of the ego vehicle and the state of other traffic objects, and the decision space includes the decision set of the ego vehicle. The ego vehicle can be a virtual unmanned vehicle, and other traffic objects include other vehicles, pedestrians, obstacles, etc. Each decision in the decision set includes lateral decisions and longitudinal decisions. Lateral decisions involve path selection, including left lane change, right lane change, and straight driving. Longitudinal decisions involve vehicle speed control, including acceleration to cope with overtaking or traffic flow changes, deceleration to avoid obstacles or comply with speed limits, and maintaining the current speed cruise. Lane changes often occur on multi-lane roads, which may be caused by obstacles ahead, overtaking needs, or turn preparation; straight driving means moving forward in the current lane.

[0087] Tree search algorithms include MCTS (Monte Carlo Tree Search), depth-first search, etc.

[0088] A specific implementation method of this step S130 includes: based on the state space and decision space at the current moment, simulating the traffic flow after each decision in the self-vehicle selection decision space in the simulation scene, determining the state space at the next moment, and using the state space at the next moment as the new state space at the current moment; continuing to execute the step of "based on the state space and decision space at the current moment, simulating the traffic flow after each decision in the self-vehicle selection decision space in the simulation scene, and determining the state space at the next moment" until the state space at the target moment is obtained, and generating a decision tree, wherein the decision tree includes a state node layer and a decision node layer, the state node layer includes a state space root node layer and at least one state space child node layer, the time difference between each two adjacent state node layers is the same, and a decision node layer is included between each two adjacent state node layers, and each node in the decision node layer represents a decision in the decision space; traversing each decision planning path from the root node to the leaf node in the decision tree to obtain multiple decision planning paths of the self-vehicle.

[0089] The target time is the time when the preset duration is reached from the current time.

[0090] For example, Figure 2 As shown, s 0is the state space at time 0, representing the root node of the decision tree, located at the root node layer of the state space, a 1 、a 2 …a n Represents each decision in the decision space, a 1 、a 2 …a n Located at the decision node layer, each decision can lead to one of several next states with corresponding transition probabilities, including s 10 、s 11 、s 12 、s 13 、s 14 、s 15 , and s 10 、s 11 、s 12 、s 13 、s 14 、s 15 It is the state space at the first moment, located at the state space sub-node layer, and so on, until the state space at the target moment is obtained, and a leaf node is obtained.

[0091] S140: Taking the decision planning path with the highest evaluation among the multiple decision planning paths as the true value of the decision planning algorithm.

[0092] After obtaining multiple decision planning paths, the score of each decision planning path can be comprehensively evaluated from multiple perspectives, and the decision planning path with the highest score can be selected as the true value of the decision planning algorithm.

[0093] In one implementation, a comprehensive evaluation can be performed from three aspects: comfort, safety, and rationality. Specifically, for each decision-making planning path, the comfort of the vehicle on the decision-making planning path is determined according to the driving status of the vehicle at each moment; the safety of the vehicle on the decision-making planning path is determined by judging whether the vehicle complies with traffic rules and whether it can make a correct response when facing emergencies; the rationality of the vehicle on the decision-making planning path is determined by judging whether the decision-making planning path meets the preset driving requirements; weighted calculations are performed on comfort, safety, and rationality to obtain the evaluation value of the decision-making planning path; and the decision-making planning path with the highest evaluation value is selected as the true value of the decision-making planning algorithm.

[0094] When the driving state of the self-vehicle is a sudden change in acceleration or a sudden turn, a lower comfort evaluation value will be assigned. When the driving state of the self-vehicle is smooth driving or smooth turning, a higher comfort evaluation value will be assigned. When the self-vehicle obeys traffic rules and can respond correctly to emergencies, a higher safety evaluation value will be assigned. When the self-vehicle does not obey traffic rules or cannot respond correctly to emergencies, a lower safety evaluation value will be assigned. For example, overtaking on a road where overtaking is not allowed, or failing to maintain a sufficient safety distance during emergency braking, a lower safety evaluation value will be assigned. When the decision-making planning path meets the preset driving requirements, a higher rationality evaluation value will be assigned. When the decision-making planning path does not meet the preset driving requirements, a lower rationality evaluation value will be assigned. The preset driving requirements include the shortest path and properly sharing the road with other traffic participants.

[0095] The truth value generation method of the decision planning algorithm provided in the embodiment of the present application can construct at least one simulation scene including traffic flow by means of simulation, and search out multiple decision planning paths by means of a tree search algorithm, and select the best decision planning path from them as the truth value of the decision planning algorithm. It can be seen that compared with taking the decision planning path of the expert driver as the truth value, the embodiment of the present application can not only be separated from the expert driver, but also use the advantages of simulation to obtain the decision planning path of special scenes with high complexity, thereby not only improving the efficiency of truth value generation, but also broadening the scene range of truth value generation.

[0096] In one possible implementation, in the process of generating a decision tree, it can be determined whether the state space represented by each state space node in the latest generated state space sub-node layer satisfies a preset risk state; when the state space represented by the state space node to be judged satisfies the preset risk state, the state space node to be judged is pruned; when the state space represented by the state space node to be judged does not satisfy the preset risk state, the state space node to be judged is not pruned.

[0097] Among them, the preset risk status includes at least one of the vehicle leaving the road, the vehicle colliding with other objects, and violating traffic regulations.

[0098] The embodiment of the present application can significantly reduce the size of the decision tree and improve the efficiency of the search by pruning the state space nodes that meet the preset risk state. At the same time, it can also ensure that the final decision path is safe and reasonable, thereby guiding the decision planning of the unmanned driving system.

[0099] Corresponding to the above method embodiment, another embodiment of the present application provides a truth value generating device for a decision planning algorithm, such as Figure 3 As shown, the device comprises:

[0100] An acquisition unit 210 is configured to acquire at least one empty map scene, wherein the empty map scene includes a road topology structure of a road segment extracted from a static map;

[0101] The simulation unit 220 is used to simulate the traffic flow in the empty scene according to the traffic data including the driving behavior of the real driver, and obtain the simulation scene corresponding to the empty scene;

[0102] A search unit 230 is used to simulate the state space at a future time in the simulation scene based on a tree search algorithm, the state space at a current time, and the decision space, and obtain multiple decision planning paths of the ego vehicle, wherein the state space includes the driving state of the ego vehicle and the states of other traffic objects, and the decision space includes the decision set of the ego vehicle;

[0103] The determination unit 240 is used to use the decision planning path with the highest evaluation among the multiple decision planning paths as the true value of the decision planning algorithm.

[0104] In a possible implementation, the search unit 230 includes:

[0105] A generation module is used to simulate, in the simulation scenario, the traffic flow after the self-vehicle selects each decision in the decision space based on the state space at the current moment and the decision space, determine the state space at the next moment, use the state space at the next moment as the new state space at the current moment, and continue to execute the steps of simulating, in the simulation scenario, the traffic flow after the self-vehicle selects each decision in the decision space based on the state space at the current moment and the decision space, determine the state space at the next moment, until the state space at the target moment is obtained, and generate a decision tree, wherein the decision tree includes a state node layer and a decision node layer, the state node layer includes a state space root node layer and at least one state space child node layer, the time difference between each two adjacent state node layers is the same, a decision node layer is included between each two adjacent state node layers, and each node in the decision node layer represents a decision in the decision space;

[0106] The traversal module is used to traverse each decision planning path from the root node to the leaf node in the decision tree to obtain multiple decision planning paths of the vehicle.

[0107] In a possible implementation, the searching unit 230 further includes:

[0108] A judgment module, used for judging whether the state space represented by each state space node in the newly generated state space sub-node layer satisfies a preset risk state during the process of generating the decision tree;

[0109] The pruning module is used to prune the state space node to be judged when the state space represented by the state space node to be judged meets the preset risk state.

[0110] In a possible implementation, the simulation unit 220 includes:

[0111] an identification module, used to perform intelligent driving model (IDM) parameter identification on traffic data containing real driver driving behavior, and obtain at least one parameter value that can reflect the real driver driving behavior;

[0112] The first simulation module is used to simulate a virtual driver having similar behavior to the real driver in the empty picture scene by using the at least one parameter value to generate a traffic flow in the empty picture scene, and determine the empty picture scene containing the traffic flow as the simulation scene corresponding to the empty picture scene.

[0113] In a possible implementation, the simulation unit 220 includes:

[0114] The second simulation module is used to simulate the traffic flow in the empty map scene based on a pre-trained decision planning model to obtain the simulation scene corresponding to the empty map scene, wherein the decision planning model is trained based on the traffic data containing the driving behavior of real drivers.

[0115] In a possible implementation, the determining unit 240 includes:

[0116] A first determination module is used to determine, for each of the decision-making planning paths, the comfort of the vehicle on the decision-making planning path according to the driving state of the vehicle at each moment;

[0117] A second determination module is used to determine the safety of the vehicle on the decision-making planning path by judging whether the vehicle complies with traffic regulations and whether it can make a correct response when facing an emergency;

[0118] A third determination module is used to determine the rationality of the vehicle on the decision-making planning path by judging whether the decision-making planning path meets the preset driving requirements;

[0119] A weighting module, used for performing weighted calculation on the comfort, the safety and the rationality to obtain an evaluation value of the decision-making planning path;

[0120] The fourth determination module is used to select the decision planning path with the highest evaluation value as the true value of the decision planning algorithm.

[0121] The truth value generation device of the decision planning algorithm provided in the embodiment of the present application can construct at least one simulation scene including traffic flow by means of simulation, and search out multiple decision planning paths by means of a tree search algorithm, and select the best decision planning path as the truth value of the decision planning algorithm. It can be seen that compared with taking the decision planning path of the expert driver as the truth value, the embodiment of the present application can not only be separated from the expert driver, but also use the advantages of simulation to obtain the decision planning path of special scenes with high complexity, thereby not only improving the efficiency of truth value generation, but also broadening the scene range of truth value generation.

[0122] Based on the above method embodiments, another embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any of the above embodiments is implemented.

[0123] Based on the above method embodiment, another embodiment of the present application provides an electronic device or a computer device, such as Figure 4 As shown, including:

[0124] One or more processors 310;

[0125] The processor 310 is coupled to a storage device 320, and the storage device 320 is used to store one or more programs;

[0126] When the one or more programs are executed by the one or more processors 310, the electronic device or computer device implements the method described in any of the above embodiments.

[0127] Based on the above method embodiments, another embodiment of the present application provides a vehicle, which includes the device as described in any of the above embodiments, or includes the electronic device as described above.

[0128] Based on the above embodiments, another embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer or a processor, the computer or the processor executes the method described in any of the above embodiments.

[0129] The above device embodiment corresponds to the method embodiment and has the same technical effect as the method embodiment. For specific description, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For specific description, please refer to the method embodiment part, which will not be repeated here. It can be understood by those skilled in the art that the accompanying drawings are only schematic diagrams of one embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present application.

[0130] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating a truth value of a decision planning algorithm, characterized in that: The method comprises: Acquire at least one empty map scene, wherein the empty map scene includes a road topology structure of a road segment extracted from a static map; Simulating the traffic flow in the empty scene according to the traffic data including the driving behavior of real drivers, and obtaining a simulation scene corresponding to the empty scene; Based on the tree search algorithm, the state space at the current moment and the decision space, the state space at the future moment is simulated in the simulation scene, and multiple decision planning paths of the ego vehicle are obtained, wherein the state space includes the driving state of the ego vehicle and the states of other traffic objects, and the decision space includes the decision set of the ego vehicle; The decision planning path with the highest evaluation among the multiple decision planning paths is used as the true value of the decision planning algorithm.

2. The method according to claim 1, characterized in that Based on the tree search algorithm, the state space and decision space at the current moment, the state space at the future moment is simulated in the simulation scenario, and multiple decision planning paths of the ego vehicle are obtained, including: Based on the state space at the current moment and the decision space, simulating in the simulation scene the traffic flow after the vehicle selects each decision in the decision space, determining the state space at the next moment, and using the state space at the next moment as the new state space at the current moment; Continue to execute the step based on the state space at the current moment and the decision space, simulate the traffic flow after the self-vehicle selects each decision in the decision space in the simulation scene, determine the state space at the next moment, until the state space at the target moment is obtained, and generate a decision tree, wherein the decision tree includes a state node layer and a decision node layer, the state node layer includes a state space root node layer and at least one state space child node layer, the time difference between each two adjacent state node layers is the same, and a decision node layer is included between each two adjacent state node layers, and each node in the decision node layer represents a decision in the decision space; Each decision planning path from the root node to the leaf node in the decision tree is traversed to obtain multiple decision planning paths of the vehicle.

3. The method according to claim 2, characterized in that The method further comprises: In the process of generating the decision tree, determining whether the state space represented by each state space node in the newly generated state space sub-node layer satisfies a preset risk state; When the state space represented by the state space node to be judged satisfies the preset risk state, the state space node to be judged is pruned.

4. The method according to claim 1, characterized in that: The traffic flow in the empty scene is simulated according to the traffic data including the driving behavior of the real driver, and the simulation scene corresponding to the empty scene is obtained, including: Performing intelligent driving model (IDM) parameter identification on traffic data containing real driver driving behavior to obtain at least one parameter value that can reflect the real driver driving behavior; A virtual driver having similar behavior to the real driver is simulated in the empty graph scene using the at least one parameter value to generate a traffic flow in the empty graph scene, and the empty graph scene including the traffic flow is determined as the simulation scene corresponding to the empty graph scene.

5. The method according to claim 1, characterized in that The traffic flow in the empty scene is simulated according to the traffic data including the driving behavior of the real driver, and the simulation scene corresponding to the empty scene is obtained, including: The traffic flow is simulated in the empty map scene based on a pre-trained decision planning model to obtain the simulation scene corresponding to the empty map scene, wherein the decision planning model is trained based on the traffic data containing the driving behavior of real drivers.

6. The method according to any one of claims 1 to 5, characterized in that The decision planning path with the highest evaluation among the multiple decision planning paths is used as the true value of the decision planning algorithm, including: For each of the decision-making planning paths, determining the comfort of the vehicle on the decision-making planning path according to the driving state of the vehicle at each moment; Determining the safety of the vehicle on the decision-making planning path by judging whether the vehicle complies with traffic rules and whether it can make correct responses when faced with emergencies; By judging whether the decision-making planning path meets the preset driving requirements, determining the rationality of the vehicle on the decision-making planning path; Performing weighted calculation on the comfort, the safety and the rationality to obtain an evaluation value of the decision-making planning path; The decision planning path with the highest evaluation value is selected as the true value of the decision planning algorithm.

7. A truth value generating device for a decision planning algorithm, characterized in that: The device comprises: An acquisition unit, configured to acquire at least one empty map scene, wherein the empty map scene includes a road topology structure of a road segment extracted from a static map; A simulation unit, used to simulate the traffic flow in the empty scene according to the traffic data including the driving behavior of real drivers, and obtain a simulation scene corresponding to the empty scene; A search unit, configured to simulate a state space at a future time in the simulation scene based on a tree search algorithm, a state space at a current time, and a decision space, and obtain a plurality of decision-making planning paths of the ego vehicle, wherein the state space includes a driving state of the ego vehicle and states of other traffic objects, and the decision space includes a decision set of the ego vehicle; A determination unit is used to use the decision planning path with the highest evaluation among the multiple decision planning paths as the true value of the decision planning algorithm.

8. The device according to claim 7, characterized in that The search unit comprises: A generation module is used to simulate, in the simulation scenario, the traffic flow after the self-vehicle selects each decision in the decision space based on the state space at the current moment and the decision space, determine the state space at the next moment, use the state space at the next moment as the new state space at the current moment, and continue to execute the steps of simulating, in the simulation scenario, the traffic flow after the self-vehicle selects each decision in the decision space based on the state space at the current moment and the decision space, determine the state space at the next moment, until the state space at the target moment is obtained, and generate a decision tree, wherein the decision tree includes a state node layer and a decision node layer, the state node layer includes a state space root node layer and at least one state space child node layer, the time difference between each two adjacent state node layers is the same, a decision node layer is included between each two adjacent state node layers, and each node in the decision node layer represents a decision in the decision space; The traversal module is used to traverse each decision planning path from the root node to the leaf node in the decision tree to obtain multiple decision planning paths of the vehicle.

9. The device according to claim 8, characterized in that The search unit also includes: A judgment module, used for judging whether the state space represented by each state space node in the newly generated state space sub-node layer satisfies a preset risk state during the process of generating the decision tree; The pruning module is used to prune the state space node to be judged when the state space represented by the state space node to be judged meets the preset risk state.

10. The device according to claim 7, characterized in that The simulation unit comprises: an identification module, used to perform intelligent driving model (IDM) parameter identification on traffic data containing real driver driving behavior, and obtain at least one parameter value that can reflect the real driver driving behavior; The first simulation module is used to simulate a virtual driver having similar behavior to the real driver in the empty picture scene by using the at least one parameter value to generate a traffic flow in the empty picture scene, and determine the empty picture scene containing the traffic flow as the simulation scene corresponding to the empty picture scene.

11. The method according to claim 7, characterized in that The simulation unit comprises: The second simulation module is used to simulate the traffic flow in the empty map scene based on a pre-trained decision planning model to obtain the simulation scene corresponding to the empty map scene, wherein the decision planning model is trained based on the traffic data containing the driving behavior of real drivers.

12. The device according to any one of claims 7 to 11, characterized in that The determining unit comprises: A first determination module is used to determine, for each of the decision-making planning paths, the comfort of the vehicle on the decision-making planning path according to the driving state of the vehicle at each moment; A second determination module is used to determine the safety of the vehicle on the decision-making planning path by judging whether the vehicle complies with traffic regulations and whether it can make a correct response when facing an emergency; A third determination module is used to determine the rationality of the vehicle on the decision-making planning path by judging whether the decision-making planning path meets the preset driving requirements; A weighting module, used for performing weighted calculation on the comfort, the safety and the rationality to obtain an evaluation value of the decision-making planning path; The fourth determination module is used to select the decision planning path with the highest evaluation value as the true value of the decision planning algorithm.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

14. An electronic device, characterized in that: The electronic device comprises: one or more processors; The processor is coupled to a storage device, and the storage device is used to store one or more programs; When the one or more programs are executed by the one or more processors, the electronic device implements the method according to any one of claims 1 to 6.

Images