Method for predicting and / or planning trajectories for a participant, method for training a model and system for controlling a vehicle

Abstract

The invention proposes a computer-implemented method for predicting and / or planning at least one trajectory (20) for at least one actor (1) of a traffic scenario, wherein the at least one trajectory (20) is generated for the at least one actor (1) using a pre-trained model based on aggregated scenario-specific information, wherein each trajectory (20) is represented by a sequence of positions / time points (21, 22-25). According to the invention, for the trajectory (20) of the actor (1) a position / time point (22-25) is generated which has a predefined distance to a common reference point (21), in particular to the current position (21) of the actor (1). The invention also relates to a method for training a model and a system for controlling a vehicle.

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for predicting and / or planning at least one trajectory for at least one participant in a traffic scenario, wherein at least one trajectory is generated for the at least one participant based on aggregated scenario-specific information and utilizing a pre-trained model, wherein each trajectory is represented by a sequence of distance / time points.

[0002] Furthermore, the present invention also relates to a method for training a model and a system for controlling a vehicle.

[0003] The preferred application of this invention is autonomous driving. However, other applications are also conceivable, such as in the field of robotics. Background Technology

[0004] The task of autonomous driving is to control the vehicle solely based on sensor data (such as radar, lidar, and RGB cameras) and other aggregated, scene-specific information (such as map information, traffic condition information, weather, and road condition information) to reach the destination as quickly, comfortably, and safely as possible. Here, the vehicle should comply with all traffic rules as much as possible, avoiding collisions with other road users or objects, and refraining from engaging in any potential collisions. This task can be further subdivided into the following subtasks: perception, prediction, planning, and control.

[0005] The first task of perception is to aggregate sensor data and other scene-specific information describing a given traffic scene. Typically, but not always, perception involves not only collecting data but also processing it to extract relevant information from the large amount of aggregated sensor data and other scene-specific information. This includes identifying and classifying static and dynamic objects and traffic participants, and determining the position and motion state of identified objects or participants. Then, based on the extracted relevant information, an environmental model of the traffic scene can be generated, for example.

[0006] The task of prediction is to predict or estimate the future development of a traffic scenario based on aggregated scenario-specific information, and, where necessary, by utilizing environmental models. Typically, this involves predicting the future behavior of each identified and observed participant individually, more precisely, in the form of at least one trajectory that the observed participant might follow. Information about the past development of the traffic scenario can also be considered, such as the location and status of other traffic participants collected in the past within the observation area, as well as infrastructure information. However, since the goals of the observed participants are inherently unknown, predicting a single behavior, i.e., a unique trajectory, is usually insufficient. Instead, a number of—typically 6-10—potential future trajectories are predicted for each participant.

[0007] The predicted results are used in the planning process to determine a target-oriented and as safe as possible strategy for the autonomous vehicle. Then, the controller or trajectory adjuster translates the corresponding trajectory into control commands for the vehicle's actuators to implement the planned trajectory.

[0008] It is known that an artificial intelligence-based model trained for the corresponding task is used for the prediction and / or the planning.

[0009] In the context of this invention, the trajectory is represented by a sequence of distance / time points, that is, a sequence of location information and associated time information. Typically, this is represented by the absolute position in a defined coordinate system, such as a global coordinate system, or possibly the vehicle's local coordinate system. Used as location information. The associated time information is usually a timestamp, which indicates a point in time when the participant was or would be located at the corresponding location.

[0010] In addition to location and time information, trajectory data can also include several other pieces of information that describe how or how the participants involved are moving in space.

[0011] One frequently used piece of additional spatial information is orientation, i.e., the participant's orientation in space, expressed in the form of yaw, pitch, and roll angles. In particular, the yaw angle allows for the inference of the direction of motion. Furthermore, trajectory data can include information about the participant's spatial scale, for example, in the form of a bounding box. This information is especially important for collision avoidance.

[0012] Trajectory data typically also includes the participant’s motion data, such as velocity information, acceleration information, and information about angular velocity.

[0013] Finally, it should be mentioned that trajectory data can also include information about the probability or uncertainty of each predicted route / time point, which can then be taken into account, for example, during planning.

[0014] Typically, the prediction and planning method discussed here generates a trajectory with a pre-given number of waypoints at fixed time intervals. The trajectory length is then derived from the product of the corresponding participant's speed and the planning time domain (typically 3 to 8 seconds). Accordingly, the predicted or planned trajectory length varies significantly depending on the corresponding participant's speed.

[0015] This has frequently proven problematic in practice. At high speeds, the distances between individual distances / time points on a planned trajectory can become so large that information about the precise direction of the trajectory is insufficient. At slow speeds or when stationary, the distances / time points are close together. In this case, the planned trajectory is very short and provides only a small look-ahead distance in meters. This proves particularly problematic when adjusting lateral vehicle motion, as this adjustment requires a minimal look-ahead distance. This problem is especially pronounced during stopping and subsequent starts. Overall, the time-discrete trajectory presentation makes it difficult to interpret planned maneuvers, as it is impossible to discern the intended direction of the strategy when stationary. Summary of the Invention

[0016] This invention proposes measures that improve the interpretability of predicted and / or planned trajectories in relation to the intentions of the corresponding participants and simplify the safe implementation or adjustment of the planned trajectories.

[0017] According to the invention, this is achieved by generating distance / time points for the participant's trajectory, these distances / time points having a predetermined distance to a common reference point, and in particular a predetermined distance to the participant's current location.

[0018] Therefore, according to the invention, “location discrete” trajectories are generated that always have a pre-given minimum distance length for a given number of distances / time points and thus guarantee a minimum look-ahead distance in meters. This proves particularly advantageous when moving slowly and stationary, such as at traffic lights, because the trajectories in these cases also reflect the participant’s predicted or planned intentions.

[0019] In a preferred variant of the prediction or planning method according to the invention, the number of distances / time points for generating the participant's trajectory is pre-defined. This allows for a very simple pre-definition of the minimum length of the trajectory to be generated.

[0020] As previously mentioned, different coordinate systems can be used for location information related to distance / time points. In an advantageous variant of the method according to the invention, the distance / time points of the participant's trajectory are generated relative to a pre-given reference line. For example, the reference line can be a route pre-given by the navigation system to reach the destination and running along the center of the lane of the roadway. Then, when using a correspondingly trained model, a trajectory running near the reference line, i.e., the center of the lane, is preferably generated. Therefore, such models have a bias towards preferentially generating trajectories along the lane lines.

[0021] The use of "location-discrete" trajectories in behavior prediction and planning must be considered when training the model for this purpose. Therefore, the present invention also relates to a computer-implemented method for training a model for predicting and / or planning at least one trajectory for at least one participant in a traffic scenario, the method comprising the following steps: a. Provide a training dataset with training samples, wherein each training sample includes at least: - Training input generated based on aggregated scene-specific information, and - The target trajectory represented by a sequence of distance / time points; b. Apply the training input of at least one training sample to the model to be trained to generate at least one training output trajectory with a pre-given number of distances / time points; c. Evaluate the training output trajectory in relation to the target trajectory of the training samples; and d. Modify the model based on the evaluation of the training output trajectory.

[0022] The type of training samples, and especially the type of training input, depends on the configuration of the model to be trained. Therefore, unprocessed scene-specific information, such as raw sensor data, can be used as training input, or preprocessed and / or analyzed scene-specific information, such as an environment model generated based on aggregated scene-specific information, can be used as training input.

[0023] The target trajectory is usually provided in the form of a time-discrete trajectory, that is, in the form of a sequence of distance / time points at equal time intervals.

[0024] According to the present invention, the model to be trained generates distances / time points to a common reference point at predetermined distances for the training output trajectory. To evaluate the training output trajectory, the spatial and temporal distances to the target trajectory are determined for each distance / time point of the training output trajectory. These two distances are then combined in a loss function so that the model can be modified in subsequent steps to optimize the loss function.

[0025] It should be noted that the various distances / time points of the training output trajectory and the various distances / time points of the target trajectory typically do not correspond to each other, as they differ in both location and time information. According to the present invention, the training output trajectory is evaluated based on the spatial and temporal distances from each distance / time point to the target trajectory.

[0026] To determine these distances, in an advantageous extension of the training method according to the invention, the distance / time points of the target trajectory are interpolated. The spatial and temporal distances between the distance / time points of the training output trajectory and the target trajectory can then be easily determined by comparing the distance / time points of the training output trajectory with the interpolated distance / time points of the target trajectory that have the same distance to a reference point as the distance / time points of the training output trajectory.

[0027] Therefore, during training, each distance / time point of the training output trajectory is assigned to a specific point on the target trajectory. Then, for the loss function, the spatial and temporal distances for each such pair of points are determined, for example, using the Huber loss function. The Huber loss function is a combination of Mean Squared Error (MSE) and Mean Absolute Error (MAE). It is more robust to outliers than MSE, but is still differentiable, which is important for training neural networks.

[0028] Spatial and temporal distances can also be defined as the difference between two points. Here, for example, thresholding with rectified linear units (ReLU) can ensure that the temporal distance between destinations monotonically increases. This means that each subsequent destination is in the future, not in the past. This assumption is particularly meaningful in the context of behavioral prediction and planning.

[0029] In an advantageous configuration of the training method according to the invention, spatial distance and temporal distance are weighted differently in the loss function.

[0030] Because the "location discrete" trajectory representation of this invention allows the deviation between the training trajectory and the target trajectory to be simply divided into a lateral profile represented by location information at distance / time points and a longitudinal profile or velocity profile represented by time information at distance / time points, these contributions can be considered differently in the total loss. Therefore, by appropriately selecting the parameters of the loss function, either the lateral profile or the longitudinal profile can be given greater importance.

[0031] Finally, the present invention also relates to a computer-implemented system for controlling a vehicle, the system comprising at least: a. The perception layer, which is used to detect and preprocess scene-specific information; b. A planning model for planning at least one trajectory for the vehicle, wherein the planning model is designed and trained to generate "location-discrete" trajectories according to the present invention; and c. A control module for adjusting the trajectory generated by the planning model into the actuators of the vehicle.

[0032] In an advantageous extension of the invention, the system further includes a prediction model designed and trained to predict “location discrete” trajectories for at least one additional participant in a traffic scenario according to the invention. Attached Figure Description

[0033] The embodiments and advantageous extensions of the present invention will be explained in more detail below with reference to the accompanying drawings.

[0034] Figure 1 This section describes the concept of behavior prediction and / or planning based on trajectories, which are represented by sequences of distances / time points with equal time intervals—as is common in the prior art.

[0035] Figure 2 This invention explains the concept of "location discrete" trajectory representation on which the method for behavior prediction and / or planning according to the present invention is based. Detailed Implementation

[0036] exist Figure 1 Vehicle 1 is shown, or more precisely, schematically shown in a bird's-eye view. Furthermore, trajectory 10 is shown, which is planned for vehicle 1 using methods from the prior art. For this purpose, a sequence of five distance / time points 12 to 16 is generated for a planning time domain with a pre-given duration; these distance / time points, together with the starting point 11, represent trajectory 10. The boundary condition for planning is that distance / time points 11, 12 to 16 are equidistant in time, i.e., these time points are pre-given, and the two location coordinates x and y of the distance points are used as the degrees of freedom for the planning. In this planning, the speed of vehicle 1 traveling on trajectory 10 must be adjusted such that the time intervals between each distance / time point 11, 12 to 16 are of equal length. Because in the example shown here, the spatial distance between distance / time points 11, 12 to 16 continuously decreases with increasing distance from the starting point 11, vehicle 1 should correspondingly reduce its speed while traveling on trajectory 10.

[0037] Figure 1This indicates that when planning based on trajectories with equidistant distances between distances and time points, the lower the speed of the vehicles involved, the smaller the look-ahead distance in meters.

[0038] exist Figure 2 Similarly, vehicle 1 and trajectory 20 are shown in the bird's-eye view. Here, the trajectory is represented by a starting point 21 and a sequence of four distance / time points 22 to 25. Trajectory 20 is planned using the method according to the invention, based on aggregated scene-specific information, and utilizing a pre-trained planner model. Accordingly, distance / time points 22 to 25 each have a predetermined distance to a common reference point 21. In this embodiment, the current position of vehicle 1 is used as reference point 21. This reference point is also the starting point 21 of trajectory 20. Here, the predetermined distances of distance / time points 22 to 25 to the common reference point 21 are indicated by the fact that each distance / time point 22 to 25 lies on one of four circles 2 to 5 drawn centered on reference point 21.

[0039] Therefore, according to the present invention, the planner model has only one degree of freedom when generating waypoints. The second degree of freedom is to derive the time information for reaching each waypoint through regression. Accordingly, the planning time domain of the trajectory generated according to the present invention depends largely on the speed of the corresponding participant or vehicle, while the distance length is completely independent of the participant's speed, but is determined essentially by the predefined distances from each waypoint / time point of the trajectory to a common reference point.

[0040] Figure 2 This demonstrates that, through planning based on the “location discreteness” trajectory according to the present invention, a forward distance exceeding the minimum distance can be guaranteed independently of the speed of the corresponding participants.

[0041] In the embodiments shown here, distance / time points Distance points from 22 to 25 This is obtained by regression relative to reference line 30. Here, the reference line could be, for example, a route suggestion from a navigation system. Reference line 30 also originates from starting point 21 and is represented by a sequence of reference points 32 to 35. These reference points are also located on one of the four circles 2 to 5, and therefore have predetermined distances to the common reference point 21 for the distance / time points 22 to 25 of trajectory 20. Here, in order to determine the distance / time points... For the distance points 22 to 25, the angular deviations were obtained by regression analysis of each reference point 32 to 35 that have the same distance to reference point 21. .

[0042] Alternatively, the distance point in the distance / time point can also be given as the arc length to the reference point, or as a 2D coordinate with the additional loss of the point on the matching circle. Given, or as 2D coordinates with additional normalization. This additional normalization is given to ensure that the corresponding point lies on the matching circle.

Claims

1. A computer-implemented method for predicting and / or planning at least one trajectory (20) for at least one participant (1) in a traffic scenario, wherein, Based on aggregated scene-specific information, and using a pre-trained model, at least one trajectory (20) is generated for the at least one participant (1), wherein each trajectory (20) is represented by a sequence of distance / time points (21, 22-25). The feature is that the trajectory (20) of the participant (1) is generated with the following distance / time points (22-25), which have a predetermined distance to a common reference point (21), and in particular to the current position (21) of the participant (1).

2. The method according to claim 1, characterized in that, The number of distances / time points (22-25) generated for the trajectory (20) of the participant (1) is given in advance.

3. The method according to claim 1 or 2, characterized in that, The distance / time points (22-25) of the trajectory (20) of the participant (1) are generated relative to a pre-given reference line (30).

4. A computer-implemented method for training a model, said model for implementing the method according to any one of the preceding claims for predicting and / or planning at least one trajectory for at least one participant in a traffic scenario. in, The method includes the following steps: a. Provide a training dataset with training samples, wherein each training sample includes at least: Training input generated based on aggregated scene-specific information, and The target trajectory is represented by a sequence of distance / time points; b. Apply the training input of at least one training sample to the model to be trained to generate at least one training output trajectory with a pre-given number of distances / time points; c. Evaluate the training output trajectory in relation to the target trajectory of the training samples; and d. Modify the model based on the evaluation of the training output trajectory; Its features are, The model to be trained generates a path / time point from the training output trajectory to a common reference point at a predetermined distance. To evaluate the training output trajectory, the spatial and temporal distances to the target trajectory are determined for each distance / time point of the training output trajectory. Combine these two distances in the loss function, and The model is modified in terms of optimizing the loss function.

5. The method according to claim 4, characterized in that, Interpolate the distance / time points of the target trajectory to determine the spatial distance and the temporal distance for each distance / time point of the training output trajectory.

6. The method according to claim 5, characterized in that, The spatial and temporal distances between the distance / time points of the training output trajectory and the distance / time points of the target trajectory are determined by comparing the distance / time points of the training output trajectory with the distance / time points of the interpolated target trajectory, wherein the distance / time points of the training output trajectory have the same distance from the reference point.

7. The method according to any one of claims 4 to 6, characterized in that, The spatial distance and the temporal distance are weighted differently in the loss function.

8. A computer-implemented system for controlling a vehicle, the system comprising at least: a. The perception layer, which is used to detect and preprocess scene-specific information; b. A planning model for planning at least one trajectory for the vehicle, wherein the planning model is designed to perform the method according to any one of claims 1 to 3, and the planning model is trained using the method according to any one of claims 4 to 7; and c. A control module for adjusting the trajectory generated by the planning model into the actuators of the vehicle.

9. The system of claim 8, further comprising a prediction model for predicting at least one trajectory of at least one other participant in a traffic scene, wherein, The prediction model is designed to perform the method according to any one of claims 1 to 3, and the prediction model is trained using the method according to any one of claims 4 to 7.

Images