Neighbor determination for learning-based trajectory prediction models
A hybrid method using rule-based and learning-based approaches to select relevant neighbors for trajectory prediction in autonomous vehicles addresses the limitations of Euclidean distance-based models, enhancing predictive accuracy and safety.
Patent Information
- Application Number
- PCT/EP2026/058816
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-06
- Filing Date
- 2026-03-26
- Publication Date
- 2026-11-12
AI Technical Summary
Existing learning-based trajectory prediction models for autonomous vehicles on highways face limitations in selecting relevant neighbors for trajectory planning, as Euclidean distance at the current time point may not accurately reflect future interactions, particularly with high-speed agents and static infrastructure, leading to reduced predictive accuracy.
A hybrid approach combining rule-based and learning-based methods, where rule-based predictions are used to select relevant neighbors based on future distances and interactions, improving the selection of k nearest neighbors for learning-based models, considering factors like speed and trajectory endpoints.
Enhances the accuracy of neighbor selection in trajectory prediction, ensuring that relevant agents and infrastructure are considered, thereby improving the safety and efficiency of automated vehicle navigation.
Smart Images

Figure EP2026058816_12112026_PF_FP_ABST
Abstract
Description
[0001] Mercedes-Benz Group AG
[0002] Neighbor determination for learning-based trajectory prediction models
[0003] The invention relates to a method and a device for planning a future driving trajectory for a motor vehicle.
[0004] Highly automated vehicles that navigate independently in traffic alongside other road users typically plan and maintain their immediately preceding trajectory automatically. Trajectory planning is closely linked to predicting the immediately preceding future behavior of other road users. Therefore, the future movements of other road users, in the sense of traffic agents (vehicles, pedestrians, etc.), are predicted, while the future movement of the automated vehicle is planned. Reliable and safe motion planning is only possible if both of these components (prediction of other behavior and planning of the vehicle's own trajectory) are of high quality. For this purpose, it is advisable to use learning-based methods for both prediction and planning, i.e., machine learning methods, particularly those based on artificial neural networks.Types of such predictions include trajectory prediction and occupancy prediction. Trajectory prediction typically predicts multiple trajectories (temporal sequences of states) per agent, i.e., one additional road user.
[0005] Different model architectures can be used to train a model during a training process and to make predictions about the behavior of other road users in the sense of other agents from these given input data or to find a future trajectory for the automated vehicle.
[0006] Graph Neural Networks are suitable for the model architectures; see in particular the following publications:
[0007] - J. Schmidt, J. Jordan, F. Gritschneder and K. Dietmayer, "CRAT-Pred: Vehicle Trajectory Prediction with Crystal Graph Convolutional Neural Networks and Multi-HeadSelf-Attention," 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA, 2022, pp. 7799-7805, doi: 10.1109 / ICRA46639.2022.9811637. - T. Monninger et al., "SCENE: Reasoning About Traffic Scenes Using Heterogeneous Graph Neural Networks," in IEEE Robotics and Automation Letters, vol. 8, no. 3, pp. 1531-1538, March 2023, doi: 10.1109 / LRA.2023.3234771.
[0008] - J. Schmidt, P. Huissel, J. Wiederer, J. Jordan, V. Belagiannis and K. Dietmayer, "RESET: Revisiting Trajectory Sets for Conditional Behavior Prediction", 2023 IEEE Intelligent Vehicles Symposium (IV), Anchorage, AK, USA, 2023, pp. 1-8, doi:
[0009] 10.1109 / IV55152.2023.10186587.
[0010] Transformer architectures can also be used for this purpose; this is described in particular in the following publication:
[0011] - Jiquan Ngiam, Benjamin Caine, Vijay Vasudevan, Zhengdong Zhang, Hao-Tien Lewis Chiang, Jeffrey Ling, Rebecca Roelofs, Alex Bewley, Chenxi Liu, Ashish Venugopal, David Weiss, Ben Sapp, Zhifeng Chen, Jonathon Shlens, "Scene Transformer: A unified architecture for predicting multiple agent trajectories", ICLR 2022, https: / / doi.org / 10.48550 / arXiv.2106.08417
[0012] Among graph-based approaches, there are forms that either restrict the number of surrounding agents considered for the prediction of a given agent to agents within a fixed Euclidean radius, or to the nearest Euclidean neighbors.
[0013] An example of the first type of restriction can be found in the following publication:
[0014] Z. Zhou, L. Ye, J. Wang, K. Wu and K. Lu, "HiVT: Hierarchical Vector Transformer for Multi-Agent Motion Prediction," 2022 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 8813-8823, doi:
[0015] 10.1109 / CVPR52688.2022.00862.
[0016] An example of the second type of restriction can be found in the following publication:
[0017] Zhejun Zhang, Alexander Liniger, Christos Sakaridis, Fisher Yu, Luc Van Gool, "Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding". NeurlPS 2023. https: / / doi.org / 10.48550 / arXiv.2310.12970
[0018] For this purpose, the Euclidean distance at time T=0, i.e., the most recent observation, is always used. This also applies to the integration of the k_stat entities of the static infrastructure. Possible output formats of such approaches include, for example, the predicted trajectories of the surrounding agents or the planned future trajectory of the automated vehicle for which the trajectory prediction is performed.
[0019] In general, such models can be summarized as follows:
[0020] - The input is a representation of the environment, typically consisting of the past movement of surrounding dynamic agents and information about the static infrastructure, e.g., road geometries, traffic lights, etc. Information about the agents is usually provided by perception components; information about the static infrastructure can also be supplied by perception or come from a map, often a high-definition (HD) map.
[0021] The representation of this input information is often fully vectorized or mixed. In a fully vectorized representation, both the agents and information about the static infrastructure are vectorized as polylines. In a mixed representation, information about the static infrastructure is presented as a raster map. This raster map then depicts the static infrastructure of the traffic scene from above.
[0022] A learning-based model (comprising an encoder and a trajectory decoder) processes this input information. For raster maps, CNNs or vision transformers can be used, for example. For vectorized information, recurrent neural networks (RNNs) and / or graph neural networks (GNNs) can be used, for example. The encoder converts the given representation of the environment into a latent representation. The decoder predicts multiple trajectories per agent.
[0023] Such trajectory prediction with a vector-based approach for other road users, also referred to as agents due to the abstraction mentioned above and hereafter, is thus often based, according to the state of the art, on k-nearest neighbor logic. For this purpose, each agent (or rather its features, primarily its respective past trajectory) is first encoded into a latent feature vector. Each agent then forms a node in a graph. Each node of the graph representing an agent is connected only to the k Euclidean nearest other nodes via edges of the graph. The Euclidean distance between two agents is calculated based on T=0. T=0 denotes the last observed time, i.e., the most recent observation.During the subsequent graph convolution using a graph neural network (the same computational procedure could also be called a "transformer"), the following occurs: Information is exchanged between the agent accounts connected by edges under consideration. This information exchange is learning-based. During a training process of the trajectory prediction model, an error function is used to determine which information from the surrounding k agents is incorporated into the latent representation of a specific agent. This is generally referred to as "social interactions." The graph neural network, with its convolution, is used to update the features of an agent (i.e., a node in the graph) so that the relevant information from the k agents surrounding that specific agent is also included.Limiting the number of neighboring agents considered for predicting an agent's trajectory to k is necessary to ensure a consistently low algorithm runtime for predicting the trajectories of other agents, even with a variable and potentially large number of them. The assumption that only the Euclidean agents closest at time T=0 are relevant is a simple heuristic that requires relatively little computational effort.
[0024] The integration of the static infrastructure follows an analogous procedure. The entities of the static infrastructure are also first encoded and transformed into a graph structure. Edges are only created between an agent and the k_stat nearest entities of the static infrastructure. k_stat can be a different number than k (k being used here for the agents). The nearest entities of the static infrastructure are again defined by the Euclidean distance of the respective agent at time T=0.
[0025] In an application of such trajectory prediction for highway driving, however, one can expect to encounter other agents, typically traveling at high speeds. To complicate matters further, the prediction horizon, i.e., the target prediction time, of the learning-based prediction model is often chosen to be around 5 seconds. At 130 km / h, this means that an agent can travel up to 181 m in this time. Although the Euclidean distance at time T=0 is a simple heuristic for determining the k or k_stat nearest neighbors, it is questionable whether this is a suitable heuristic for highways. This is illustrated below with a few examples:
[0026] - If a slow-moving vehicle is 200 m ahead of the agent being predicted, and the agent is approaching at high speed (e.g., 150 km / h), the slow-moving vehicle is highly relevant despite its considerable distance. Only by including this vehicle can a good prediction be made for the agent being predicted. To achieve a good prediction, the slow-moving vehicle should be among the nearest neighbors (K) despite its considerable distance. - Vehicles in oncoming traffic, separated by a guardrail, can be partially detected by vehicle perception. These vehicles may be Euclidean close to the agent being predicted, but due to the physical barrier, they have no influence whatsoever. They should not be among the nearest neighbors (K), despite their close proximity.
[0027] - Static infrastructure typically includes entities located behind the agent being predicted, but which have no influence on its future movement. They should not be among the nearest neighbors (k_stat).
[0028] DE 102022 102214 A1 further relates to a method for operating an autonomous vehicle, wherein the method comprises: receiving sensor data by at least one processor, which is associated with an environment in which an autonomous vehicle operates; determining an agent set comprising multiple agents located in the environment by the at least one processor based on the sensor data; determining an interaction parameter, which specifies a prediction of the interaction of a corresponding agent with the autonomous vehicle, by the at least one processor for each agent of the agent set; determining a primary agent set by the at least one processor based on the interaction parameters, wherein the primary agent set is a subset of the agent set; and generating a trajectory for the autonomous vehicle by the at least one processor based on the primary agent set.
[0029] The object of the invention is to improve the planning of a prior driving trajectory of a motor vehicle.
[0030] The invention is defined by the features of the independent claims. Advantageous further developments and embodiments are the subject of the dependent claims.
[0031] A first aspect of the invention relates to a method for operating a motor vehicle, comprising the steps of:
[0032] - Detecting the environment of the motor vehicle and a number M agents in the environment by means of a sensor unit of the motor vehicle, wherein the M agents include other road users;
[0033] - Performing a first trajectory prediction for each i-th of the M agents using a rule-based approach, where i is a running index with i=1,2,...,M, where the rule-based approach includes an extrapolation of a driving motion of a respective i-th agent on possible driving lanes; - Selecting, for each i-th of the M agents, a respective number of k relevant agents from the number of remaining M-1 agents based on a distance condition, according to which the k relevant remaining agents have the smallest future distances to the respective i-th agent, where the distances are determined using the first trajectory prediction;
[0034] - Performing a second trajectory prediction for each i-th of the M agents using a learning-based approach with a pre-trained machine learning model, where the second trajectory prediction for the respective i-th agent only considers the k further agents identified as relevant for the respective i-th agent; and
[0035] - Planning a future driving trajectory of the motor vehicle at least on the basis of the trajectories predicted by the second trajectory prediction of all M agents, and automatic execution of the driving trajectory by an actuator control on the motor vehicle.
[0036] The pre-trained machine learning model for executing the second trajectory prediction is advantageously trained in such a way that for the respective i-th agent only the respective k further agents determined to be relevant for the respective i-th agent are taken into account.
[0037] The vehicle's surroundings are detected at least by means of a sensor unit within the vehicle itself, such as a camera unit. However, LIDAR, RADAR, ultrasonic sensors, and other sensors can also be used individually or in combination.
[0038] Furthermore, the vehicle's surroundings can be captured using a digital map, from which information about static infrastructure can be extracted. This information can be used in agent models for other road users as well as for planning the vehicle's own trajectory.
[0039] Particularly for safety reasons, a classical rule-based prediction is typically performed in addition to the learning-based prediction in a system with a learning-based approach that uses a pre-trained machine learning model, especially an artificial neural network. The classical rule-based prediction assigns a lane to each agent to be predicted and then extrapolates their movement along the lane using their current speed projected into the future. If the lane splits or the initial assignment is ambiguous, an extrapolation is performed along various lane combinations. If different extrapolation possibilities exist (e.g., due to a lane split or because an agent is in the process of changing lanes), the classical prediction also generates multiple trajectory predictions.These are also referred to as trends. Both approaches (rule-based and learning-based) have their strengths and are therefore combined in one system.
[0040] - The classic, rule-based approach is easy to interpret and validate. - The learning-based approach is also able to learn complex relationships (e.g., the interactions between agents) from the data and then recognize them in the application and correctly incorporate them into the prediction.
[0041] Both approaches process the same input information and deliver predictions independently. For example, each approach could provide two different trajectory predictions per agent (these are referred to as "modes"; one mode of the classic, rule-based approach could be lane following, another could be lane changing). The predictions from both approaches are then combined. With two predictions per approach, this would result in four trajectory predictions. These predictions are subsequently used during the planning of the automated vehicle's behavior to determine a safe and comfortable driving trajectory.
[0042] Instead of running classical, rule-based, and learning-based prediction independently, a sequential approach is proposed. Here, classical prediction is executed first. This extrapolates along the possible lane combinations and, for multiple valid options, yields several modes of motion (i.e., trajectories) for a given agent. Crucially, the resulting trajectories from the classical approach can be directly used in the learning-based prediction to improve the selection of the k nearest neighbors. In general terms, the classical approach provides N_i trajectory predictions for a given agent ie{1 <i<M}, mit insgesamt M Agenten in der Verkehrsszene.
[0043] To use the learning-based prediction approach, a selection of the relevant k agents from the M other agents must be made for agent i. Here, agent i and a surrounding agent j are assigned 1 <j<M-1 wegen j i einDistanzmaß berechnet, das insbesondere auf dem Endpunkt einer jeweiligen prädizierten Trajektorie basiert. Es werden die klassischen Prädiktionen genutzt, um zu bestimmen, wie nahe sich zwei Agenten (unter Berücksichtigung aller prädizierten Moden) möglicherweise kommen können. Für den Agent i werden dann die k anderen Agenten ausgewählt, die die niedrigste Distanz haben. Dies wird für jeden der M Agenten durchgeführt, jeder prädizierte Agent i kann also unterschiedliche k Nachbarn haben.
[0044] In an extended form, the approach doesn't just rely on x and y coordinates; the endpoint can be an entire state. This state can also include, for example, velocities, accelerations, or angles. Specifically, this means that the selection of the k nearest neighbors for agent i in the learning-based prediction no longer corresponds to a Euclidean distance at time T=0, but rather to an estimated minimum distance between the states at the end of the prediction horizon (e.g., in 5 seconds), estimated using the classical prediction method that is required anyway.
[0045] Since classical, rule-based prediction is required anyway, this involves virtually no additional computational effort. However, the selection of the k nearest neighbors is drastically improved. All conceivable examples, some of which are mentioned in the problem statement, are thus improved; in particular, slow vehicles ahead would always be taken into account because classical prediction is based on extrapolating the instantaneous speed along the lanes.
[0046] The planning of the vehicle's own trajectory is preferably carried out by another machine learning model whose input comprises the predicted trajectories of the agents. In an extension, a similar principle can be used for learning-based planning of the vehicle's anticipated trajectory. These methods also typically incorporate the social interactions between agents (especially in relation to the automated vehicle). The selection of the k relevant other agents can also be based on classical prediction, as described above.
[0047] Furthermore, the principle described above can also be applied to approaches that do not necessarily always have to select k other agents or k_stat entities of the static infrastructure, but rather always select the entities that lie within a certain Euclidean distance at time T=0. For these, too, instead of the Euclidean distance to T=0, the estimated minimum distance of the states at the end of the prediction horizon (e.g., in 5 seconds) can be used (estimated here using the classical prediction required anyway).
[0048] The planner is provided with additional predictions for planning the upcoming trajectory, supplementing the rule-based approach. In the simplest case, for example, instead of two predictions as with the rule-based approach alone, four are now provided, and different reactions are triggered depending on these. For instance, the classic rule-based predictions could always be used when a strong reaction to them is expected, e.g., in the case of heavy braking. Otherwise, the learning-based approaches could be used.
[0049] It is an advantageous effect of the invention that an improved selection of k nearest neighbors of further agents is made in a nearest neighbor classification for each specific agent.
[0050] This principle can also be applied to the selection of static infrastructure entities used for predicting an agent's behavior. As described earlier, there are also fully vectorized, machine learning-based approaches that process static infrastructure entities using a graph structure. Here, too, a preselection of the relevant k_stat static infrastructure entities for an agent i takes place, based on the Euclidean distance at time T=0. This approach also has limitations, particularly for applications on highways: For example, lanes behind an agent might be selected, while distant lanes might not be selected, even though they lie in the direction of travel and could be reached within the next 5 seconds at a speed of 130 km / h. This reduces the predictive accuracy.Therefore, rule-based predictions can be used again to calculate the smallest distances to the surrounding entities of the static infrastructure for various journals in the future (e.g., at T=1s, T=2s, ... T=5s). Then, for an agent i, the k_stat entities with the lowest distance (regardless of the journal) are selected.
[0051] According to an advantageous embodiment, the number k is automatically selected depending on at least one of the following: the driving speed of one or more of the M-1 agents, the driving speed of the motor vehicle, or the traffic density of the M-1 agents. According to a further advantageous embodiment, the planning of the upcoming driving trajectory of the motor vehicle is carried out based on the first trajectory prediction and on the basis of the second trajectory prediction.
[0052] According to another advantageous embodiment, the respective future distances between the respective i-th agent and each further of the remaining M-1 agents are related to each end of a trajectory of the respective i-th agent predicted by the rule-based approach.
[0053] According to a further advantageous embodiment, the relevance of each remaining M-1 agent is determined on the basis of a distance measure, wherein the distance measure is determined: i) on the basis of a future end of a trajectory of the respective i-th agent predicted by the rule-based approach, and ii) on the basis of at least one further location that lies between a beginning and the end of the trajectory predicted by the rule-based approach.
[0054] In this form, the distance measure is not determined using the endpoint of the predicted trajectory (i.e., after, for example, 5 seconds), but also at intermediate intervals (e.g., 1 s, 2 s, ...). Thus, the smallest distances to other agents are calculated for the various magazines in the future. Subsequently, the k agents with the lowest distance measure to agent i, regardless of the time, can be selected.
[0055] According to a further advantageous embodiment, each of the M-1 agents is additionally classified as relevant based on at least one of the following parameters:
[0056] Relative velocity between the i-th agent and each of the other M-1 agents, accelerations and / or angles between the i-th agent and each of the other M-1 agents.
[0057] According to a further advantageous embodiment, a first quantity is the future distance and at least one further quantity is used, with the quantities being weighted.
[0058] According to another advantageous embodiment, the agents comprise elements of the stationary infrastructure.
[0059] According to another advantageous embodiment, it is automatically determined whether the anticipated driving trajectory of the motor vehicle is planned on the basis of the first trajectory prediction or on the basis of the second trajectory prediction.
[0060] Another aspect of the invention relates to a device for a motor vehicle, comprising a sensor unit designed to detect the environment of the motor vehicle with a number M agents in the environment, wherein the M agents comprise other road users; and comprising a computing unit designed and configured for this purpose:
[0061] - to perform a first trajectory prediction for each i-th of the M agents using a rule-based approach, where i is a running index with i=1,2,...,M, where the rule-based approach includes an extrapolation of a driving motion of a respective i-th agent on possible driving lanes;
[0062] -for each i-th of the M agents, to select a respective number of k relevant agents from the number of remaining M-1 agents based on a distance condition, according to which the k relevant remaining agents have the smallest future distances to the respective i-th agent, whereby the distances are determined using the first trajectory prediction;
[0063] - to perform a second trajectory prediction for each i-th of the M agents using a learning-based approach with a pre-trained machine learning model, wherein the second trajectory prediction for the respective i-th agent only considers the respective k of the other agents M-1 determined to be relevant for the respective i-th agent; and
[0064] - to plan a future driving trajectory of the motor vehicle at least on the basis of the trajectories of all M agents predicted by the second trajectory prediction, and to automatically execute the driving trajectory by means of an actuator control on the motor vehicle.
[0065] Advantages and preferred further developments of the proposed device result from an analogous and substantive transfer of the above statements made in connection with the proposed method.
[0066] Further advantages, features and details will become apparent from the following description, in which - possibly with reference to the drawing - at least one embodiment is described in detail.
[0067] Figure 1 shows a method for planning a forward driving trajectory according to an embodiment of the invention.
[0068] Fig. 2: An exemplary traffic situation from a bird's-eye view, in which the method of Fig. 1 is applied.
[0069] Fig. 3: The traffic situation of Fig. 2 in a further calculation state.
[0070] Fig. 4: A vehicle with a device for planning a forward driving trajectory.
[0071] The representations in the figures are schematic and not to scale.
[0072] Fig. 1 shows a method for operating a motor vehicle 1, comprising the steps of: - detecting S1 an environment of the motor vehicle 1 and a number M agents 2 in the environment by means of a sensor unit of the motor vehicle 1, wherein the M agents 2 comprise other road users;
[0073] - Performing S2 a first trajectory prediction for each i-th of the M agents 2 using a rule-based approach, where i is a running index with i=1,2,... ,M, where the rule-based approach includes an extrapolation of a driving motion of a respective i-th agent 2 on possible driving lanes;
[0074] - Select S3, for each i-th of the M agents 2, a respective number of k relevant agents from the number of the remaining M-1 agents 2 based on a distance condition, according to which the k relevant remaining agents 2 have the smallest future distances to the respective i-th agent 2, where the distances are determined using the first trajectory prediction;
[0075] - Execute S4 a second trajectory prediction for each i-th of the M agents 2 using a learning-based approach with a pre-trained machine learning model, wherein the second trajectory prediction for the respective i-th agent 2 only considers the respective k further agents 2 determined to be relevant for the respective i-th agent 2; and
[0076] - Planning S5 of a future driving trajectory of the vehicle 1, at least based on the trajectories of all M agents 2 predicted by the second trajectory prediction, and automatic execution of the driving trajectory by an actuator control on the vehicle 1. Fig. 2 shows an exemplary and schematic traffic situation from a bird's-eye view. The goal is to perform automatic trajectory planning for the vehicle 1. For this purpose, a large number of M other road users, referred to as agents 2, are detected by the sensor unit 3 of the vehicle 1. As can be seen in the progression from Fig. 2 to Fig. 3, one agent 2 is selected for each of the M agents 2 and the sequence is rotated using the index i. While, for example, M = four in Fig. 2, the road user in the lower right is selected first as the i-th agent 2.This leaves M-1 = three potential additional agents 2, from which the k nearest neighbors are selected as relevant for performing a trajectory prediction for the i-th agent 2. This is done as described in the procedure shown in Fig. 1. The result for the considered i-th road user agent 2, k = two, is shown as an example. Fig. 3 depicts the same traffic situation, but the i-th agent 2 is the one shown in the lower left of the image. The k relevant agents 2 are then determined from the remaining three agents 2, where k = one in this example. Only the additional agents 2 in the group of k relevant agents are used for the prediction with the learning-based approach for each i-th agent 2. This procedure is carried out for all four of the additional agents 2, so that each of these four agents 2 is agent i once.
[0077] Fig. 4 shows motor vehicle 1 with a device for trajectory planning, comprising a sensor unit 3, designed to detect the environment of the motor vehicle 1 with a number M agents 2 in the environment, wherein the M agents 2 comprise other road users; and comprising a computing unit 4, which is designed and configured for this purpose:
[0078] - to perform a first trajectory prediction for each i-th of the M agents 2 using a rule-based approach, where i is a running index with i=1,2,... ,M, where the rule-based approach includes an extrapolation of a driving motion of a respective i-th agent 2 on possible driving lanes;
[0079] -for each i-th of the M agents 2, to select a respective number of k relevant agents from the number of the remaining M-1 agents 2 based on a distance condition, according to which the k relevant remaining agents 2 have the smallest future distances to the respective i-th agent 2, whereby the distances are determined using the first trajectory prediction;
[0080] - to perform a second trajectory prediction for each i-th of the M agents 2 using a learning-based approach with a pre-trained machine learning model, wherein the second trajectory prediction for the respective i-th agent 2 only considers the respective k further agents 2 identified as relevant for the respective i-th agent 2; and
[0081] - to plan a future driving trajectory of the motor vehicle 1 at least on the basis of the trajectories of all M agents 2 predicted by the second trajectory prediction, and to automatically execute the driving trajectory by means of an actuator control on the motor vehicle 1.
[0082] Although the invention has been further illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived from them by a person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned as examples are truly only examples and are not to be understood in any way as limiting, for example, the scope of protection, the possible applications, or the configuration of the invention.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without leaving the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description.
Claims
Mercedes-Benz Group AG Patent claims 1. Computer-implemented method for operating a motor vehicle (1), comprising the steps: - Detection (S1) of the environment of the motor vehicle (1) and a number M agents (2) in the environment by means of a sensor unit of the motor vehicle (1), wherein the M agents (2) comprise other road users; - Performing (S2) a first trajectory prediction for each i-th of the M agents (2) using a rule-based approach, where i is a running index with i=1,2,...,M, where the rule-based approach includes an extrapolation of a driving motion of a respective i-th agent (2) on possible driving lanes; - Selections (S3), for each i-th of the M agents (2), of a respective number of k relevant from the number of the respective M-1 remaining agents (2) based on a distance condition, according to which the k relevant remaining agents (2) have the smallest future distances to the respective i-th agent (2), where the distances are determined by means of the first trajectory prediction; - Executing (S4) a second trajectory prediction for each i-th of the M agents (2) using a learning-based approach with a pre-trained machine learning model, wherein the second trajectory prediction for the respective i-th agent (2) only considers the respective k further agents (2) identified as relevant for the respective i-th agent (2); and - Planning (S5) a future driving trajectory of the motor vehicle (1) at least on the basis of the trajectories of all M agents (2) predicted by the second trajectory prediction, and automatic execution of the driving trajectory by an actuator control on the motor vehicle (1).
2. Method according to claim 1, where the number k is automatically chosen depending on at least one of the following: A driving speed of one or more of the M-1 agents (2), a driving speed of the motor vehicle (1), a traffic density of the M-1 agents (2).
3. Method according to any one of the preceding claims, wherein the planning of the motor vehicle's (1) trajectory ahead is based on the first trajectory prediction and on the second trajectory prediction.
4. Method according to any one of claims 1 to 3, wherein the respective future distances between the respective i-th agent (2) and each further of the remaining M-1 agents (2) are related to each end of a trajectory of the respective i-th agent (2) predicted by the rule-based approach.
5. Method according to any one of claims 1 to 3, wherein the relevance of each remaining M-1 agent (2) is determined on the basis of a distance measure, wherein the distance measure is determined: i) on the basis of a future end of a trajectory of the respective i-th agent (2) predicted by the rule-based approach, and ii) on the basis of at least one further location that lies between a beginning and the end of the trajectory predicted by the rule-based approach.
6. Method according to any one of the preceding claims, where each of the M-1 agents (2) is additionally classified as relevant based on at least one of the following quantities: relative velocity between the i-th agent (2) and the respective of the other M-1 agents (2), accelerations and / or angles between the i-th agent (2) and the respective of the other M-1 agents (2).
7. Method according to claim 6, where a first quantity is the distance in the future and at least one other quantity is used, with the quantities being weighted.
8. Method according to any one of the preceding claims, wherein the agents (2) comprise elements of the stationary infrastructure.
9. Method according to any one of the preceding claims, where it is automatically determined whether the upcoming driving trajectory of the motor vehicle (1) is planned on the basis of the first trajectory prediction or on the basis of the second trajectory prediction.
10. Device for a motor vehicle (1), comprising a sensor unit (3) designed to detect the environment of the motor vehicle (1) with a number M agents (2) in the environment, wherein the M agents (2) comprise other road users; and comprising a computing unit (4) designed and configured to: - perform a first trajectory prediction for each i-th of the M agents (2) using a rule-based approach, wherein i is a running index with i=1 ,2, ... ,M, wherein the rule-based approach comprises an extrapolation of a driving motion of a respective i-th agent (2) on possible lanes; -for each i-th of the M agents (2) to select a respective number of k relevant agents from the number of the remaining M-1 agents (2) on the basis of a distance condition, according to which the k relevant remaining agents (2) have the smallest future distances to the respective i-th agent (2), whereby the distances are determined by means of the first trajectory prediction; - to perform a second trajectory prediction for each i-th of the M agents (2) using a learning-based approach with a pre-trained machine learning model, wherein the second trajectory prediction for the respective i-th agent (2) only considers the respective k further agents (2) identified as relevant for the respective i-th agent (2); and - to plan a future driving trajectory of the motor vehicle (1) at least on the basis of the trajectories of all M agents (2) predicted by the second trajectory prediction, and to automatically execute the driving trajectory by a
