Method for predicting influence of road user on at least one other road user and method for operating vehicle
By analyzing traffic scenarios and calculating impact scores between traffic participants, the problem of accurate prediction of traffic participants' mutual influence in autonomous vehicle environments is solved, and more accurate trajectory prediction and safety control are achieved .
Patent Information
- Application Number
- CN202380050124.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-07-05
- Filing Date
- 2023-06-21
- Publication Date
- 2025-02-14
AI Technical Summary
The prior art is difficult to accurately predict the mutual influence between traffic participants in an autonomous vehicle environment, affecting the accuracy of vehicle trajectory prediction and safety control.
By using trained artificial neural networks, traffic scenarios are analyzed and impact score values between traffic participants are calculated. This method quantifies the impact of each traffic participant on other traffic participants by comparing the deviations between the real trajectory and the simulated trajectory, and uses these score values for trajectory prediction and safety control.
Accurate identification of traffic participants and prediction of their future trajectories in the autonomous vehicle environment is achieved, improving the reliability and safety of vehicle functions, especially in collision warning and avoidance systems.
Smart Images
Figure CN119452404A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for predicting the influence of a traffic participant on at least one other traffic participant according to the preamble of claim 1 .
[0002] The invention also relates to a method for operating a vehicle. Background Art
[0003] DE 10 2021 005 625.6 describes a method for predicting vehicle trajectories in an autonomous vehicle environment with the aid of a trained artificial neural network. Here, the degree of interaction between vehicles is determined with the aid of a machine-trained attention-based interaction algorithm. With the aid of the interaction algorithm, individual vehicles in a set of vehicles in the environment of the autonomous vehicle are identified as being relevant to trajectory prediction, and if the degree of interaction of each vehicle with at least one of the vehicles whose trajectory is to be predicted exceeds a predetermined limit value, the vehicle is selected for trajectory prediction. In a subsequent learning step, the trajectory prediction algorithm is trained only with the vehicles selected as being relevant to trajectory prediction, and the trajectory prediction performed with the aid of the trajectory prediction algorithm is performed only for the vehicles selected as being relevant to trajectory prediction. In addition, a method for automatically operating an autonomous vehicle is described, wherein the trajectory of the vehicle in the environment of the autonomous vehicle is predicted, and during the automated operation of the autonomous vehicle, the predicted trajectory is taken into account during the automated lateral and / or longitudinal control of the autonomous vehicle. Summary of the invention
[0004] The object of the present invention is to provide a novel method for predicting the influence of a road user on at least one other road user and a novel method for operating a vehicle.
[0005] According to the invention, this object is achieved by a method for predicting the influence of a road user on at least one other road user having the features of claim 1 and a method for operating a vehicle having the features of claim 6 .
[0006] Advantageous embodiments of the invention are the subject matter of the dependent claims.
[0007] In a method for predicting the influence of a traffic participant on at least one other traffic participant by analyzing a traffic scene with the aid of a trained artificial neural network, according to the invention, the training of the neural network is performed with the aid of recorded traffic scenes. In this case, these traffic scenes have a plurality of traffic participants and are marked with a score value, which represents the influence exerted by one traffic participant by the other traffic participants. The deviation between two trajectories of a traffic participant is determined, and on this basis a corresponding score value is calculated for the one traffic participant for the other traffic participant. In this case, one of the two trajectories is the acquired real trajectory, i.e. the trajectory that the one traffic participant actually travels in the corresponding recorded traffic scene. The other of the two trajectories is a simulated trajectory, which is determined in a simulation and represents the trajectory that the one traffic participant would travel in the same traffic scene if the other traffic participant were not present.
[0008] For automated vehicles operating at levels 2 to 5 of automation according to standard SAE J3016, as well as active collision avoidance systems according to NCAP, traffic participants must be accurately identified in the automated vehicle's environment. In order to plan safe and collision-free trajectories for the automated vehicle, the future trajectories of the surrounding traffic participants must also be correctly predicted. Here, the influence of each traffic participant on the others must be intrinsically understood in order to take into account interaction dependencies.
[0009] In order to quantify these interactions, this method is used to establish a scalar influence metric in the form of a fractional value and a method for learning this influence metric.
[0010] The method makes it possible to express influence scores particularly advantageously at the level of interaction between at least two road users. Score values can be extracted from any traffic scenario so that they can be used as labels and / or road conditions. The method implements a learning-based method in particular, which uses the generated score values as labels and learns predictions of the score values in real time based on the distribution of surrounding road users and optional infrastructure, such as lanes, traffic regulations, etc.
[0011] In one possible embodiment of the method, the deviation between two trajectories is determined by means of an average displacement error and / or a final displacement error. The minimum average displacement error indicates how far each calculated position of the corresponding trajectory is on average from its true position. The minimum final displacement error indicates the deviation of the prediction from the true trajectory for each last prediction step.
[0012] In another possible embodiment of the method, the influence of a traffic participant on only one other traffic participant is determined based on the score value of the traffic participant. That is, the score value is regarded as a relative value in order to describe the relative influence of a traffic participant on another specific traffic participant.
[0013] In another possible embodiment of the method, the influence of a traffic participant on all other traffic participants in the corresponding traffic scene is determined based on the score value of the traffic participant. In other words, the score value is regarded as an absolute value in order to describe the global influence of a traffic participant on the traffic scene.
[0014] In another possible embodiment of the method, the trained artificial neural network uses a map-free method, which uses dynamic information about other road users as input information. Alternatively or additionally, the trained artificial neural network uses a scene graph, which uses all available information, including information about static infrastructure from a map. Both options, either individually or together, lead to efficient and successful training.
[0015] In a method for operating a vehicle, according to the invention, the influence of a road user on at least one other road user predicted in the above-described method or embodiment is used to perform a vehicle function, thereby achieving particularly reliable operation of the vehicle function.
[0016] In one possible embodiment of the method for operating a vehicle, the probability of a collision between the automated vehicle and surrounding traffic participants is determined using a collision warning and / or collision avoidance system of the automated vehicle with the aid of a predicted influence of surrounding traffic participants on the automated vehicle. Due to the use of the predicted influence, this determination is particularly efficient and reliable.
[0017] In another possible embodiment of the method for operating a vehicle, the predicted influence is used as a heuristic method to limit the search space to at least one relevant road user during the automated path finding of the vehicle. This allows a particularly efficient operation of the corresponding vehicle function.
[0018] In another possible embodiment of the method for operating a vehicle, the predicted influences are used as input variables for a trajectory prediction method and the interaction intensity between pairs of road users is modeled. This allows for particularly accurate trajectory predictions. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Embodiments of the present invention are explained in more detail below with reference to the accompanying drawings.
[0020] in:
[0021] Figure 1A top view of the traffic situation is schematically shown;
[0022] Figure 2 A top view of another traffic situation is schematically shown;
[0023] Figure 3 A top view of another traffic situation is schematically shown;
[0024] Figure 4 Schematically shows Figure 2 and 3 the trajectories of vehicles in the traffic situation, and
[0025] Figure 5 Schematic diagram of the graph structure and the use of graph-based artificial neural network processing
[0026] Information in the graph structure.
[0027] In all the figures, parts which correspond to one another are provided with the same reference numerals. DETAILED DESCRIPTION
[0028] Figure 1 The figure shows a top view of the traffic situation at a T-junction. Here, a traffic participant V1 in the form of a vehicle is on the main road, and two other traffic participants V2 and V3, also in the form of vehicles, stop one after the other at the stop line S. In this case, traffic participant V1 has a great influence on traffic participant V2, because traffic participant V1 is the reason for traffic participant V2 to stop. Similarly, traffic participant V2 also has a great influence on traffic participant V3, otherwise traffic participant V3 would have driven all the way to the stop line S.
[0029] In order to quantify the interactions between the traffic participants V1 to V3 , an influence measure, also referred to as a score value, and a method for learning the influence measure are provided.
[0030] In order to determine the influence of one road user V1 to V3 on another road user V1 to V3 , a deviation of the trajectory T1 , T2 is determined, which the one road user V1 to V3 would have traveled if the other road user V1 to V3 under consideration had not been present.
[0031] like Figure 2 It is shown that the road user V2 stops at the stop line S in the driving trajectory T1 due to the approaching road user V1 .
[0032] In order to quantify the influence of traffic participant V1 on traffic participant V2 using this influence measure, Figure 3The same traffic situation without the road user V1 is shown in . In this case, the road user V2 does not stop at the stop line S, but follows the trajectory T2 and turns to the right.
[0033] The deviation between the two trajectories T1, T2 is correspondingly large, so according to the influence measure, the influence of traffic participant V1 on traffic participant V2 is also large. This is plausible because traffic participant V2 does interact with traffic participant V1 and will not continue to drive until traffic participant V1 passes the intersection.
[0034] For the trajectory t of j when i exists, and the trajectory t of j when i does not exist, use the distance size / distance measurement value D according to
[0035] e i,j =D(t j|i ,t j|i ) (1)
[0036] Calculate the influence measure e for determining the influence of traffic participant i on traffic participant j i,j .
[0037] As distance measure, all influence measures that allow comparison of two time series can be used, such as the so-called average displacement error (ADE) or the so-called final displacement error (FDE), depending on
[0038]
[0039] and
[0040]
[0041] Furthermore, the influence can be calculated in absolute or relative terms. Relative describes the influence of traffic participant i on traffic participant j, and absolute describes the influence of traffic participant i on all surrounding traffic participants:
[0042]
[0043] If the influence measure in the form of the mean displacement error is applied to Figure 2 and 3 The traffic scene shown in the figure will result in Figure 4 , in which there are trajectories T1 , T2 , and the trajectories T1 , T2 have positions POS1 to POS4 , POS1 ′ to POS4 ′ every second.
[0044] Deviations d2 to d4 are formed between the positions POS1 to POS4, POS1' to POS4' of the trajectories T1, T2, which are still zero at the positions POS1, POS1', but increase with each subsequent position POS2 to POS4, POS2' to POS4'. The reason for this is that the road user V2 stops at the stop line S when traveling on the trajectory T1, but turns right when traveling on the trajectory T2.
[0045] That is, two trajectories T1 , T2 of the road user V2 are required for calculating the influence. One trajectory T1 is used for the situation where the road user V1 is present, and the other trajectory T2 is used for the situation where the road user V1 is not present.
[0046] The recorded data here only covers the cases where the road user V1 is present. Other cases require so-called interventions, ie interventions in the environment that are not possible later. For this, see, for example, "Judea Pearl: Causality: Models, Reasoning, and Inference".
[0047] However, in order to still obtain a trajectory T2 for this case, the traffic scene is re-simulated and the environment is retained except for the traffic participant V1. The trajectory planning module can now reprocess the same traffic scene, but without the traffic participant V1. This allows intervention in the simulation.
[0048] For a recorded traffic scene, this method makes it possible to quantify the influence of each road participant V1 to V3 on each other road participant V1 to V3 in the scene. This score value is now used as a label in a learning-based method, the purpose of which is to predict the influence.
[0049] Any input can be used in this case. For example, a so-called map-free method can be used, which uses only the dynamic information DI of the surrounding road users V1 to V3 as input information.
[0050] But it is also possible to use a scene graph, which uses all available information, including information about static infrastructure from the map. Figure 5 , it is shown that this is done specifically with the aid of a graph structure GS and that the information in the graph structure GS is processed with a graph-based artificial neural network N. The graph structure GS contains a plurality of nodes K1 to Km, which are connected by edges E1 to En.
[0051] Here, static information SI, dynamic information DI, semantic information SEI and relationship information RI are transmitted as input information of the traffic scene into the graph structure GS.
[0052] A form of learning-based method, in particular a graph-based artificial neural network N, is then used, wherein the output information AI provides information about the influence of a road participant V1 to V3 on at least one other road participant V1 to V3.
[0053] In this way, the influence of one road user V1 to V3 on another road user V1 to V3 can be predicted by a learning-based method. This information can be used in a collision warning, in a trajectory prediction method or in a trajectory planning method.
Claims
1. A method for predicting the influence of a traffic participant (V1 to V3) on at least one other traffic participant (V1 to V3) by analyzing a traffic scene with the aid of a trained artificial neural network (N), characterized in that - training the artificial neural network (N) with the aid of recorded traffic scenarios, wherein the traffic scenarios have a plurality of traffic participants (V1 to V3) and are assigned score values which represent the influence exerted by one traffic participant (V1 to V3) on the other traffic participants (V1 to V3); - Determine the deviation (d2 to d4) between two trajectories (T1, T2) of one road user (V1 to V3) and, based on this, determine the deviation (d2 to d4) of two trajectories (T1, T2) of another road user (V1 to V3) Calculating a corresponding score value for the one traffic participant (V1 to V3); - one of the two trajectories (T1, T2) is the acquired true trajectory (T1), That is, the actual driving trajectory of the one traffic participant (V1 to V3) in the recorded corresponding traffic scene; and The other of the two trajectories (T1, T2) is a simulated trajectory (T2), which is determined in a simulation and represents the trajectory that the one traffic participant (V1 to V3) would travel in the same traffic scenario if the other traffic participant (V1 to V3) were not present.
2. The method according to claim 1, characterized in that The deviation (d2 to d4) between the two trajectories (T1, T2) is determined by means of: - mean displacement error, and / or - Final displacement error.
3. The method according to claim 1 or 2, characterized in that: On the basis of the score value of a road user (V1 to V3), the influence of the road user (V1 to V3) on only one other road user (V1 to V3) is determined.
4. The method according to any one of the preceding claims, characterized in that On the basis of the score value of a road participant (V1 to V3), the influence of the road participant (V1 to V3) on all other road participants (V1 to V3) in the respective traffic scene is determined.
5. The method according to any one of the preceding claims, It is characterized in that The trained artificial neural network (N) - using a map-free method which uses the dynamic information (DI) of the other road users (V1 to V3) as input information, and / or - Employ a scene graph that uses all available information, including information about static infrastructure from maps.
6. A method of operating a vehicle, wherein: A vehicle function is performed using the predicted influence of a road user (V1 to V3) on at least one other road user (V1 to V3) according to the method of one of the preceding claims.
7. The method according to claim 6, wherein: Based on the predicted influence of surrounding traffic participants (V1 to V3) on the autonomous driving vehicle, the probability of a collision between the autonomous driving vehicle and the surrounding traffic participants (V1 to V3) is determined with the aid of a collision warning and / or collision avoidance system of the autonomous driving vehicle.
8. The method according to claim 6 or 7, wherein: The predicted influence is used as a heuristic in order to restrict the search space to at least one relevant road user (V1 to V3) during the automated path finding of the vehicle.
9. The method according to any one of claims 6 to 8, wherein: The predicted influences are used as input variables of the trajectory prediction method and the interaction intensity between pairs of traffic participants (V1 to V3) is modeled.