Method for planning optimal driving behavior for a vehicle that is at least partially autonomous

Abstract

The disclosure includes a method (100) for planning optimal driving behavior for a vehicle (10) that is at least partially autonomous, comprising the following steps: - Generating (102) at least one conditional cost volume (14, 14-1, 14-2) for at least one future scene development (12, 13) based on sensor data (8), wherein the at least one conditional cost volume (14, 14-1, 14-2) is designed as an evaluation function; - Evaluation (104) of a number of potential trajectories (20) using the conditional cost volumes (14, 14-1, 14-2); - Selecting (106) an optimal trajectory (22) from the number of potential trajectories (20) based on the evaluation performed, a probability of occurrence (24) for at least one future scene development (12, 13) and route information (15) for the vehicle (10).

Description

[0001] The present invention relates to a method for planning optimal driving behavior for a vehicle that is at least partially autonomous. State of the art

[0002] One possible goal of autonomous driving for a vehicle is to use sensor data to control the vehicle in such a way that a defined destination is reached as quickly, comfortably and safely as possible, for example without causing collisions or violating traffic rules.

[0003] This driving task can be subdivided into the subtasks of perception, prediction, planning, and control. The task of perception is to extract relevant information from sensor data, such as the position of objects (other vehicles or road users), identify lane markings, and recognize traffic signs, etc. Since the detected objects are usually dynamic obstacles, their future position must then be predicted to avoid collisions. Building on this, the task of planning is to generate a trajectory, which is then controlled in the control subtask.

[0004] One specific method among AI-based approaches to trajectory planning is the "End-to-End Interpretable Neural Motion Planner" (NMP) published by Uber ATG. This end-to-end approach spans the aforementioned subtasks of perception, prediction, and planning. Based on lidar point clouds and HD maps containing information about the road layout, etc., it generates both a prediction for other road users and a cost volume. This cost volume assigns a cost value to each possible position in the field of view for every point in time within the planning horizon, indicating how favorable (or unfavorable) it is to be at that location at that time. Subsequently, various trajectories that the vehicle is theoretically capable of executing are evaluated based on this cost volume. The optimal trajectory is then passed to a controller, which is responsible for guiding the vehicle along this plan.

[0005] One weakness of NMP, however, is that the plan generation relies solely on sensor data and map information. Consequently, the trajectory is selected without considering the desired route. While NMP enables collision-free driving, it does not offer the option of taking a specific navigation destination into account.

[0006] The original end-to-end approach encompasses all the aforementioned subtasks and generates the cost volume based on lidar point clouds and HD maps, which contain information about the road layout, etc. Generating a cost volume from other intermediate representations is also possible.

[0007] Another problem with the existing approach is the marginalization in the generation of cost volume and prediction. This stems from the following: in many situations, various behavioral options are possible for the ego-vehicle. For example, it may be equally permissible to stay in the current lane and to change to another lane. Additionally, other road users can act and react in different ways. Since the NMP only generates one cost volume or one prediction, this marginalizes all possible ego intentions and all developments within the overall scene.

[0008] However, the lack of differentiation between various scene developments means that an interaction between an ego-plan of the ego-vehicle and the behavior of other road users is currently only inadequately considered in the planning of the ego-vehicle's driving behavior.

[0009] It is therefore the object of the present invention to provide a solution by means of which the planning for the driving behavior of an at least partially autonomous vehicle can be improved. Disclosure of the invention

[0010] This task is solved by a procedure for planning optimal driving behavior for a vehicle that is at least partially autonomous and has the characteristics of independent claims.

[0011] According to a first aspect, the disclosure relates to a method for planning optimal driving behavior for an at least partially autonomous vehicle, which comprises the following steps: In a first step, at least one conditional cost volume is generated for at least one future scene development based on sensor data, whereby the at least one conditional cost volume is designed as an evaluation function.

[0012] In the context of the present invention, scene development is understood as a possible driving behavior of the vehicle, which can be implemented taking into account various parameters such as the environment. Furthermore, scene development considers the potential behavior of all road users in the vicinity of the ego-vehicle. Therefore, scene development can also be understood as a prediction of the behavior of all road users involved.

[0013] In a second step, an evaluation of a number of potential trajectories (= candidate trajectories) is carried out using the conditional cost volume / the cost volume.

[0014] In a third step, an optimal trajectory is selected from the number of potential trajectories based on the evaluation performed, a probability of occurrence for at least one future scene development, and route information for the vehicle.

[0015] The method according to the invention makes it possible to distinguish between different possible scenario developments for a vehicle during the planning of the vehicle's behavior. This advantageously increases safety and comfort for road users while simultaneously expanding the range of scenarios through which a vehicle can navigate autonomously. This leads to improved driving behavior for a vehicle that is at least partially autonomous. A single cost volume can cover various road scenarios. This means that different cost volumes can be generated for different scenario developments, or a single cost volume can cover one or more scenario developments.

[0016] Furthermore, the method according to the invention achieves an expansion of the cost volume representation in such a way that the previously described safety-critical marginalization can be prevented. This allows for an increase in comfort and safety in the described scenarios.

[0017] The following describes some basic aspects of the present invention: To determine safe trajectories for a vehicle, conditioned to different scene developments, several cost volumes and predictions are generated. Conditioning is achieved by linking the extracted scene information to a latent scene variable. From this, a cost volume and a prediction are then generated for each variable. The goal of the training is to optimize the parameters of the scene variables so that different cost volumes result, leading to different ego plans. These different ego plans are caused either by different ego intentions or by the intentions of other road users, requiring different responses.Consequently, when planning vehicle behavior, it is not permissible to select the trajectory that incurs the lowest costs across all scenarios, as this would mean that the planner could influence the behavior of all other actors to their advantage. Therefore, when selecting a trajectory, the planner considers not only the costs for a given scenario, but also the costs across all scenarios, the probability of the scenario occurring, and the alignment of the trajectory and route.

[0018] One possible implementation of the process involves generating the cost volumes using a machine learning model. This achieves the advantage of improved trajectory planning.

[0019] One possible design of the procedure envisages that the evaluation of the number of potential trajectories is carried out using the following approach: s(t)=α1⋅maxi1ccostvolume(t,i)+α2⋅1ccollision(t)+α3⋅logp(i)+α4⋅1crouting(t,R) This improves vehicle behavior in an efficient manner.

[0020] Here, s(t) describes the suitability of the trajectory t. Furthermore, c costvolume(t,i) the evaluation of t based on the cost volume of modes i and c collision(t) A cost term that penalizes a collision within a given time on the marginalized prediction (= combined prediction from all scene developments), i.e., on a weighted sum of all conditional cost volumes. p(i) indicates the probability of scene development i occurring. This is estimated by the NMP as an additional outcome. 1 / c Routing (t, R) describes a cost term for how well t follows the desired route R. Factors such as distance, progress, or orientation relative to the route can be used for this purpose.

[0021] The optimal trajectory should be safe for all scenario developments and have a low risk of collision. However, it may be poor in very unlikely road scenarios, i.e., too slow or uncomfortable.

[0022] One possible implementation of the procedure involves determining an optimal trajectory based on calculated deviation information compared to the provided route information. This can improve the accuracy of planning the optimal vehicle behavior, resulting in improved vehicle handling.

[0023] One possible implementation of the procedure involves generating all conditional cost volumes for the respective future roadway scenarios using the same machine learning model. This achieves improved and efficient generation as well as quantitative comparability of all conditional cost volumes.

[0024] One possible implementation of the method involves evaluating a trajectory using a cost volume not only based on future positions along the trajectory, but also taking into account at least one of the following pieces of information: speed, orientation (each at the same future point in time as the position). In this case, it is also referred to as an extended cost volume. In this way, the conditional cost volume is extended to provide information about how desirable it is to be at a defined location at a defined speed at a certain time.

[0025] According to a second aspect, the disclosure relates to a computer program containing machine-readable instructions which, when executed on one or more computers and / or computer instances, cause the computer(s) or computer instances to execute the method according to the invention.

[0026] According to a third aspect, the disclosure concerns a machine-readable data carrier and / or download product containing the computer program.

[0027] According to a fourth aspect, the disclosure relates to one or more computers and / or computer instances with the computer program, and / or with the machine-readable data carrier and / or the download product.

[0028] Further measures improving the invention are described in more detail below, together with a description of preferred embodiments of the invention, with reference to figures. Examples of implementation

[0029] It shows: Fig. 1 Schematic flowchart of procedure 100 for planning optimal driving behavior for a vehicle that is at least partially autonomous 10; Fig. 2 Exemplary illustrations of roadway scenarios for vehicles, disregarding interaction; Fig. 3 exemplary illustrations of roadway scenarios for vehicles, taking interaction into account; Fig. 4 Schematic representation of an architecture for implementing the method 100 for planning optimal driving behavior for an at least partially autonomous vehicle 10 according to an embodiment of the present invention; Fig. 5 Schematic representation of an architecture for implementing the method 100 for planning optimal driving behavior for an at least partially autonomous vehicle 10 according to an embodiment of the present invention; Fig. 6 Schematic representation of possible roadway scenarios for vehicles taking into account interaction for the implementation of method 100 for planning optimal driving behavior for an at least partially autonomous vehicle 10 according to an embodiment of the present invention; and Fig. 7 Schematic representation of possible roadway scenarios for vehicles for the implementation of the method 100 for planning optimal driving behavior for an at least partially autonomous vehicle 10 according to an embodiment of the present invention.

[0030] Fig. Figure 1 shows a schematic flowchart of procedure 100 for planning optimal driving behavior for a vehicle that is at least partially autonomous.

[0031] In a first step 102, at least one conditional cost volume 14, 14-1, 14-2 is generated for at least one future scene development 12, 13 based on sensor data 8, wherein the at least one conditional cost volume 14, 14-1, 14-2 is designed as an evaluation function. The sensor data 8 could contain data from camera, lidar, or ultrasound data, which were generated by corresponding sensors of the vehicle 10 and / or by external sensor sources located outside the vehicle 10.

[0032] Optionally, the conditional cost volumes 14, 14-1, 14-2 are generated using a machine learning model.

[0033] In a second step, 104, an evaluation of a number of potential trajectories 20 takes place. The trajectories can be provided, for example, by a sampler and are evaluated using cost volume.

[0034] Optionally, the evaluation of a trajectory using a cost volume can be based not only on the future positions along the trajectory, but can also take into account at least one of the following pieces of information: speed, orientation (each at the same future point in time as the position). In this case, it is also referred to as an extended cost volume.

[0035] In a third step 106, a conditional optimal trajectory 22 is selected from the number of potential trajectories 20 based on the evaluation performed, a probability of occurrence 24 for at least one future scene development 12, 13 and route information 15 for the vehicle 10.

[0036] Optionally, the selection 106 of a conditional optimal trajectory 22 is carried out depending on a determined deviation information 28 to the provided route information 15.

[0037] Optionally, the number of potential trajectories (20) can be evaluated using the following approach: s(t)=α1⋅maxi1cconstvolume(t,i)+α2⋅1ccollsion(t)+α3⋅log p(i)+α4⋅1crouting(t,R) The following is an explanation of the components of the above formula: s(t) represents the suitability of the trajectory t. c costvolume(t,i) stands for an evaluation of t based on the cost volume of fashion i. c collision(t) is a cost term that penalizes a collision within a given time on the marginalized prediction (= combined prediction from all scene developments). p(i) represents the probability of scene development i occurring. This is estimated by the NMP as an additional output. 1 / c Routing (t, R) describes a cost term for how well t follows the desired route R. Factors such as distance, progress, or orientation relative to the route can be used for this purpose.

[0038] When evaluating trajectories, it is required that they be collision-free for all scene developments over a short horizon (e.g., 2 seconds). Over a longer horizon, however, it is only necessary that they are suitable for a given scene development. Furthermore, scene developments that allow the ego-vehicle to follow its route should be favored.

[0039] Furthermore, according to the present invention, for each scene development, for example, for a first scene development 12, a corresponding first conditional cost volume 14-1 is generated, and for a second scene development 13, a corresponding second conditional cost volume 14-2 is generated. Or, put another way, each conditional cost volume corresponds to a future scene development.

[0040] Optionally, the generation of the cost volumes 14, 14-1, 14-2 for the respective potential trajectories 20 can be performed using a machine learning model. Optionally, all generated conditional cost volumes 14, 14-1, 14-2 can be generated using the same machine learning model.

[0041] The vehicle 10 may have vehicle or assistance functions that allow the vehicle to be controlled at least partially autonomously.

[0042] Fig. Figure 2 shows an exemplary illustration of roadway scenarios for vehicles, disregarding interaction.

[0043] In the Fig. Figure 2 presents an exemplary state-of-the-art scenario in which the lack of consideration for interaction prevents a desired lane change from being implemented. To account for the interaction between the ego-plan and the behavior of other road users, the prediction and cost calculation must be conditioned on the ego-vehicle's intention. However, in many scenarios, there is a one-to-one relationship between the ego-intention and the intentions of other actors.

[0044] For example, consider following a vehicle in front. The intention to accelerate is strongly correlated with whether the vehicle in front accelerates. If the driver's intention to accelerate is assumed to be a given, it automatically follows that the vehicle in front will also accelerate. However, this obscures the causal link, which can lead to a safety-critical decision.

[0045] For this reason, the present invention distinguishes not between ego intentions, but rather between scene developments or road scenarios that lead to different ego plans of the vehicle. This means that several conditional cost volumes are generated for evaluating possible trajectories, with each cost volume describing a different scene development. This could, for example, involve information on whether a lane change can be executed or not.

[0046] By enabling the planner, based on the inventive method, to distinguish between different possible scene developments for the vehicle, safety and comfort can be increased, and at the same time the number of scenarios through which a vehicle can navigate autonomously can be increased.

[0047] In the Fig. 2 and Fig. Figure 3 shows the same initial situation at time t=0. The autonomously driving vehicle 10 is shown in the upper lane and intends to change to the lane below, which is already occupied by other vehicles. In the Fig. 2. The planning does not consider the impact of the autonomous vehicle's behavior on other road users. Therefore, the vehicle must assume that both vehicles will continue in their lane. Since the distance between the two vehicles is too small, a lane change is not possible.

[0048] In the Fig. Figure 3 presents a scenario that takes into account that an initiating lane change by the autonomous vehicle 10 causes the following vehicle to decelerate, thus maintaining a safe distance. This allows the lane change to be carried out.

[0049] To determine safe trajectories that correspond to the various scene developments, as in the Fig. As shown in Figure 3, once conditioned, several conditional cost volumes and predictions are generated. Conditioning is achieved by linking the extracted scene information to a latent scene variable. From this, a cost volume and a prediction are then generated for each variable. The goal of the training is to optimize the parameters of the scene variables so that different conditional cost volumes result, leading to different ego plans.

[0050] Different ego plans arise either from differing ego intentions or from the intentions of other road users, requiring different responses. Consequently, it is not permissible to select the trajectory that incurs the lowest costs across all possible scenarios, as this would imply that the planner could manipulate the behavior of all other actors to their advantage.

[0051] For this reason, when selecting trajectories, the planner considers not only the costs for a given scene development, but also the costs across all scene developments, as well as the probability of the scene development occurring and the agreement between trajectory and route.

[0052] A key challenge in generating conditional cost volumes is ensuring that they reflect different scene developments while remaining quantitatively comparable. Furthermore, the training only provides the necessary ground truth for each scene development.

[0053] To ensure comparability, all conditional cost volumes are generated using the same CNN. Furthermore, in each training step, only the cost volume for which the resulting trajectory exhibits the smallest deviation from the ground truth is optimized. Since the latent tensors are trainable parameters, this ensures that they describe different scene developments leading to various ego plans.

[0054] Fig. Figure 4 shows a schematic representation of an architecture for implementing the method 100 for planning optimal driving behavior for an at least partially autonomous vehicle 10 according to an embodiment of the present invention.

[0055] In the block diagram of the Fig. The result is an optimal trajectory 42 or the best trajectory 42 for a possible scene development. Input data can be, for example, map data 30, which is transformed into latent features 32 via a CNN backbone 31. Using K-trained intention embeddings 34, which are fed into a device 33, various probability scenarios 35, CNN head cost volumes 36, and CNN head predictions 37 are then generated in order to generate corresponding conditional cost volumes 39 with the K-predictions 38. These various cost volumes 39 then lead, together with route information 40, to a corresponding trajectory selection 41, from which the best trajectory 42 is finally selected for the vehicle 10.

[0056] Fig. Figure 5 shows a schematic representation of an architecture for implementing the method 100 for planning optimal driving behavior for an at least partially autonomous vehicle 10 according to an embodiment of the present invention.

[0057] The system's basic architecture corresponds to the original NMP. From the input data 50, such as map data, a latent representation 53 is extracted by a CNN backbone network 51. From this representation, both the future position 59 of objects (= CNN head prediction) 54 is decoded, and the cost volume 57 for trajectory evaluation is generated. Unlike the original NMP, the input data 50 consists of detections rather than lidar data, in addition to HD maps.

[0058] The process for determining the best trajectory from a selection of trajectories follows the original NMP. For example, circular and straight trajectories, as well as clothoids with various parameters, are evaluated, and those with the lowest costs are selected. The use of trajectories with different shapes is possible.

[0059] In an optional extension (extended cost volume), the cost volume 57 also takes into account further dimensions of the trajectories 56 to be evaluated, such as velocity and orientation. For this to work, these additional dimensions must also be discretized in order to maintain the tensor representation of the cost volume.

[0060] While smaller discretization levels lead to a smaller discretization error, the number of parameters increases significantly. To keep the discretization error low despite using larger discretization levels, linear interpolation can be used to determine costs in selected dimensions.

[0061] Choosing coarser discretization levels simultaneously addresses the problem that during training, usable ground truth is only available for very few states (consisting of location, time, velocity, orientation, etc.). This problem becomes more serious the finer the discretization levels are.

[0062] Furthermore, the following should be noted: In many traffic situations, several behavioral patterns are possible and Both modes should be equally desirable. For example, it may be permissible to avoid an obstacle on the left or right. However, the model must evaluate all modes within the same cost range. In probabilistic terms, this representation represents the marginalization of a multimodal distribution. As a result, all waypoints (combination of position and time) are determined. Waypoints belonging to at least one desirable mode are valued at low cost. However, by representing all modes within the same cost volume, information about which waypoints belong to the same mode is lost. Consequently, not only are trajectories that accurately describe the modes evaluated as having low costs, but also those that represent a combination of these modes. The following are exemplary scenarios in which this behavior critically impacts safety and comfort can be affected.

[0063] Thus, according to the inventive method, a representation of a cost volume assigns a scalar cost value to each state or scene development consisting of position and time. This velocity-agnostic representation is therefore unable to distinguish between different physical states without errors.

[0064] To address this problem and prevent the marginalization described above, this representation must be extended to include further dimensions in order to adequately describe a state. According to the aforementioned inventors, speed is of particular importance in this regard.

[0065] The cost volume is extended from a 3D representation to a 4D representation, which provides information about how desirable it is to be at a certain location at a certain time and at a certain speed. Formally, the 3D cost volume is a tensor of dimension T×W×H, where T corresponds to the number of time steps in the planning horizon and W and H to the number of spatial discretization levels in the longitudinal and transverse directions, respectively. The 4D cost volume can be represented accordingly as a tensor of dimension T×V×W×H, where V describes the number of discretization levels with respect to speed.

[0066] It is also possible to extend the system to include other dimensions such as the vehicle's orientation (yaw angle) or 2D speed (longitudinal and lateral direction).

[0067] Fig. 6 and Fig. Figure 7 shows a schematic representation of possible roadway scenarios for vehicles taking into account interaction for the implementation of the method 100 for planning optimal driving behavior for an at least partially autonomous vehicle 10 according to an embodiment of the present invention.

[0068] In both Fig. 6 and Fig. Figure 7 shows a situation in which marginalization can lead to problems. The triangles and rhombuses represent waypoints of a feasible mode. Fig. 6 corresponds to the triangular mode for braking before the pedestrian crossing and the diamond mode for accelerating over the crossing before the pedestrian reaches it.

[0069] In the Fig. In section 7, the triangles and rhombuses each correspond to modes of circumventing the obstacle on the left and right sides, respectively. Since both modes are feasible, the triangular and rhombus-shaped waypoints will have low cost values. The circles represent an impermissible mode resulting from a combination of the triangular and rhombus-shaped modes. Because it consists only of waypoints of feasible modes, it is also assigned a low cost value. The result is in Fig. 6 the collision with the pedestrian and in Fig. 7 with the obstacle.

Claims

[1] Method (100) for planning an optimal driving behavior for an at least partially autonomous vehicle (10) with the following steps: - generating (102) at least one conditional cost volume (14, 14-1, 14-2) for at least one future scene development (12, 13) on the basis of sensor data (8), wherein the at least one conditional cost volume (14, 14-1, 14-2) is designed as an evaluation function; - evaluation (104) of a number of potential trajectories (20) by means of the conditional cost volumes (14, 14-1, 14-2); - selecting (106) an optimal trajectory (22) from the number of potential trajectories (20) on the basis of the evaluation carried out, a probability of occurrence (24) for the at least one future scene development (12, 13) and route information (15) for the vehicle (10). [2] Method (100) according to one of the preceding claims, wherein the cost volumes (14, 14-1, 14-2) are generated by means of a machine learning model. [3] Method (100) according to one of the preceding claims, wherein all conditional cost volumes for the respective future roadway scenarios are generated with the same machine learning model. [4] Method (100) according to one of the preceding claims, wherein the evaluation of the number of potential trajectories (20) is carried out by the following approach: s(t)=α1⋅maxi1ccostvolume(t,i)+α2⋅1ccollision(t)+α3⋅log p(i)+α4 ⋅1crouting(t,R) with s(t) for a suitability of the trajectory t, c costvolume(t,i) as an evaluation of t based on the cost volume of mode i, c collision(t)as a cost term that penalizes a collision within a given time on the marginalized prediction (= combined prediction from all scene developments), p(i) as a probability of occurrence of scene development i and 1 / c Routing (t, R) as a cost term that describes how well t follows the desired route R. [5] Method (100) according to one of the preceding claims, wherein the determination (106) of the optimal trajectory (22) is carried out as a function of a determined deviation information (28) for the provided route information (15). [6] Method according to one of the preceding claims, wherein the evaluation of a trajectory by means of a cost volume is not only carried out on the basis of the future positions on the trajectory, but additionally takes into account at least one of the following information: speed, orientation (in each case at the same future time as the position). [7] A computer program containing machine-readable instructions which, when executed on one or more computers and / or computer instances, cause the computer or computer instances to carry out the method according to any one of claims 1 to 6. [8] Machine-readable data carrier and / or download product with the computer program according to claim 7. [9] One or more computers and / or compute instances with the computer program according to claim 7, and / or with the machine-readable data carrier and / or download product according to claim 8.

Images