Method and corresponding device for evaluating possible trajectories
Through collaborative recommendation services and neural networks to evaluate the static and dynamic characteristics of the traffic environment, identify traffic participants and evaluate trajectories, solve the problem of insufficient comprehensive and complex environment processing in the existing technology, and achieve rapid and reliable trajectory evaluation and planning .
Patent Information
- Application Number
- CN202010625817.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-07-03
- Filing Date
- 2020-07-02
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-07-02
AI Technical Summary
The existing technology fails to fully cover all possible trajectory options when evaluating traffic conditions, resulting in trajectory planning being too cautious or dangerous, and category-based systems are functionally limiting when facing a large number of possibilities, making it difficult to deal with the complex traffic environment and agents’ mutual influence. .
Adopting collaborative recommendation services to identify traffic participants by detecting static and dynamic features of traffic environments, and evaluating possible trajectories with adaptive recommendation services and neural networks, provides trajectory options that comply with traffic rules, avoiding all possible situations Rules create and calculate explosive growth.
Achieving rapid and reliable assessment of possible trajectories of traffic participants in real traffic environments provides comfortable and safe trajectory planning, avoiding overly cautious or dangerous planning, and without explicit development rules or heuristics.
Smart Images

Figure CN112185148B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method and a corresponding device for evaluating possible trajectories of traffic participants in a traffic situation. Background Art
[0002] For reliable trajectory planning of at least partially automated vehicles in real traffic situations, it is essential to evaluate the trajectory options of other road users. If another vehicle crosses into a shared lane, for example, the at least partially automated vehicle may have to reduce its speed. If a pedestrian wants to cross the street in front of the automated vehicle, the vehicle may have to come to a complete stop.
[0003] Known classification models have a fixed number of categories. These categories each represent a special trajectory of a traffic participant for an intersection type and are highly specific to this case. In this case, for example, the general problem is reduced to a fixed set of options, heuristic methods are used, or machine learning techniques are used to calculate a specific trajectory. Summary of the invention
[0004] The problem with the solutions described in the prior art is that not all options for possible traffic situations are covered, which can lead to overly cautious planning or, on the other hand, overly dangerous planning. It is not possible to use predefined environment type variants to reflect a large number of possible real-world scenarios.
[0005] In the case where the number of trajectories of different classes is too large, systems for trajectory planning based on division into classes quickly reach their functional limits due to the large number of possibilities. This is especially true if the classes are difficult to distinguish and therefore may not be correctly annotated. This is because the agents of other agents influence each other, including the actions of the automated ego vehicle. This leads to a combinatorial explosion of possible alternatives, and creating a set of rules for all possible solutions is beyond practical possibilities.
[0006] Creating heuristics, i.e. a set of learned rules that can be applied in a specific situation and that may depend on the location or scenario, also quickly becomes unfeasible as the number of different stop locations and scenarios increases.
[0007] When using state-of-the-art machine learning approaches to estimate agent trajectories (i.e. the trajectories of traffic participants), these trajectories are generated, for example, by reinforcement learning, recurrent neural networks or imitation learning.
[0008] The problem with this approach is that the trained model must receive and understand all necessary inputs and must then be able to output one or more different trajectories or intentions that correspond to traffic regulations. However, it is difficult to further process a large number of calculated trajectories without evaluating the individual trajectories. Machine learning systems generated in this way are not reliable enough and are difficult to validate.
[0009] The problem with approaches that simplify trajectory estimation by defining a limited set of possible categories is that not all options are covered, which can lead to overly cautious or dangerous planning. This is because the planning system must be able to obtain all options of other agents as much as possible and weight them according to their probability and impact in order to be able to plan a comfortable and safe trajectory.
[0010] According to the invention, a method, a device, a computer program product and a computer-readable storage medium for evaluating possible trajectories of traffic participants in a traffic environment are described, which at least partially have the above-mentioned effects. Advantageous embodiments are also the subject of the invention.
[0011] The invention is based on the knowledge that a possible trajectory of a road user in and out of a particular traffic environment in terms of traffic technology and traffic regulations can be particularly suitably evaluated with the aid of a recommendation service and the probability of the observed trajectory can be displayed therefrom.
[0012] In this context, such an evaluation is to be understood as a recommendation of a trajectory to the traffic environment, which can be said to use the recommendation service and of which the traffic participant is part. The basic assumption in the case of a recommendation service is that similar users behave similarly. The evaluation of these users is implicitly derived by using the trajectory and the metric.
[0013] The assessment of the user is implicitly derived from his future behavior by comparing possible trajectories with actually observed trajectories. This is repeated millions of times over a long period of time for a large number of traffic situations. The advantage of this method is that multiple assessments with almost the same probability can be made without ambiguity, for example by predicting the average of two trajectories using a different type of method, which is disadvantageous and can lead to impossible trajectories.
[0014] According to one aspect of the method for evaluating possible trajectories of traffic participants in a traffic environment, the traffic environment is detected in one step having static and dynamic characteristics.
[0015] In a further step of the method, at least one road user is identified. In particular, this identification can be performed with the aid of detected dynamic features of the traffic situation.
[0016] In particular, road users may be identified with the aid of detected dynamic features of the traffic situation.
[0017] In a further step of the method, at least one possible trajectory is determined for the at least one road user in the traffic situation.
[0018] In a further step of the method, the at least one possible trajectory determined for the at least one road user is evaluated with the aid of the adapted recommendation service and the detected traffic environment.
[0019] Here, a possible trajectory of a traffic participant is a trajectory that the respective traffic participant can follow while complying with traffic regulations. However, in particular, it can also be a trajectory that the respective traffic participant could follow. Here, the environment is described with respect to the respective traffic participant. For example, the environment can be described with the traffic participant as the center.
[0020] Here, a recommendation service is a method whose purpose is to predict and quantify the intensity of a user's interest in an object so as to recommend to the user exactly the object that the user may be most interested in from a set of all existing objects.
[0021] Collaborative recommendation services, also called collaborative filters, recommend objects that are of most interest to users with similar evaluation behavior (i.e. similar users). No additional knowledge about the objects themselves is required for this. Thus, such collaborative recommendation services are able to recommend just complex objects without having to "understand" the objects themselves. The recommendation service evaluates objects based on previously considered user-object combinations based on metrics with the aid of given information about the current use.
[0022] Collaborative filter systems come in many forms, but many common systems can be simplified into two steps:
[0023] - Searching for users that share the same evaluation pattern as the active user, ie the user for which the prediction is made.
[0024] - using the evaluations of the users found in the search to calculate a prediction for the active users.
[0025] This belongs to the category of user-based collaborative filtering. The specific application is the user-defined K-nearest neighbor algorithm (K- Nachbar-Algorithmus).
[0026] The advantage of the method for evaluating possible trajectories of traffic participants is that there are no restrictions on the traffic environment. No rules or heuristic methods need to be developed explicitly. Such an evaluation of possible trajectories can be done quickly, almost in real time.
[0027] The static and dynamic characteristics of the traffic environment may relate to details of static and dynamic objects, but the characteristics may also relate to the entire object itself, for example as a whole.
[0028] The identification of the road users can take place with the aid of the detected dynamic features of the traffic situation.
[0029] The detected traffic environment having static and dynamic features means that features are detected at specific moments in time at which information can be obtained in order to perform the method.
[0030] An adapted recommendation service is a recommendation service that is designed and tested to perform specific tasks in terms of its structure, i.e., the type or manner of recommendation service, and in terms of its special configuration, for example, by training a neural network or by constructing a K-nearest neighbor method with the aid of training data or training vectors, etc. An exemplary structure or adaptation of such a recommendation service will be described in more detail.
[0031] According to one aspect, the traffic environment has the following dynamic and static features or objects of the traffic situation: speed, speed difference, distance of traffic participants from each other, distance of traffic participants from the environment, a bird's eye view of traffic participants and the traffic situation, direction of movement or orientation of traffic participants relative to the environment.
[0032] Furthermore, the traffic environment has the following dynamic and static features or objects of the traffic situation: traffic regulations, in particular locally relevant traffic regulations, stop lines, stop lines, traffic lines, lane markings.
[0033] Furthermore, the traffic environment has the following dynamic and static features or objects of the traffic situation: traffic light status, geographic map data, lanes, intersection structure, general T-junctions and traffic routes; and additionally a summary description of the traffic environment, such as: general traffic conditions, traffic flow, traffic congestion, free driving roads.
[0034] In this case, the road users are not only vehicles in the general sense, but also all road users (such as in particular also pedestrians) and their positions, orientations and speeds.
[0035] According to one aspect, it is proposed to detect at least one feature of the traffic environment by means of a sensor. This can be a static or dynamic feature, whereby also temporary features or time courses can be detected.
[0036] According to one aspect, it is proposed that the traffic environment is detected by determining at least one dynamic feature at at least two temporally consecutive points in time. The accuracy of the method can be significantly increased if temporally consecutive information is available in each individual evaluation step to describe the traffic environment.
[0037] According to one aspect, it is proposed that the traffic environment is characterized by at least one traffic regulation and / or at least one characteristic of at least one traffic light device and / or at least one static or dynamic traffic sign. These additional descriptions describe the traffic environment in more detail and thus allow a better assessment of possible trajectories.
[0038] According to one aspect, it is proposed to detect at least one feature of the traffic environment by means of an imaging sensor. Such an imaging sensor allows very accurate recognition and classification of at least a part of the traffic environment, which in turn can have a positive influence on the accuracy of the assessment.
[0039] According to one aspect, it is proposed to detect at least one feature of the traffic environment by means of sensors of the ego vehicle. Advantageously, this type of detection of the traffic environment focuses on particularly relevant parts related to the ego vehicle.
[0040] According to one aspect, it is proposed to detect at least one feature of the traffic environment by means of a fixed sensor. For example, such a sensor can transmit data, for example to the ego vehicle, at a busy or complex intersection, which enables the ego vehicle to improve its assessment of the possible trajectories, in particular of other road users.
[0041] According to one aspect, it is proposed to detect at least one dynamic feature of the traffic environment by means of a sensor. Such a sensor can increase the reliability compared to other possibilities for detecting dynamic characteristics. For example, a radar sensor can directly determine the speed of a traffic participant. However, dynamic features or objects can also be derived, for example, from specific image processing of an imaging sensor.
[0042] According to one aspect, it is proposed that at least some of the static and / or dynamic characteristics of a traffic situation of at least one road participant are wirelessly transmitted to other road participants of the traffic situation.
[0043] As a result, a larger overview of the traffic situation can be obtained through the interaction of a plurality of road users, and, for example, a traffic environment or a traffic situation that is difficult for some road users to see can be provided.
[0044] According to one aspect, it is proposed that the recommendation service be adapted to evaluate possible trajectories using a large number of corresponding combinations of evaluated observed trajectories, different traffic conditions and the at least one possible trajectory.
[0045] This means that the recommendation service can be constructed based on observed trajectories, which are combined with corresponding traffic environments to adapt the recommendation system. Observed trajectories are trajectories that have traveled in a specific traffic environment in the real world and can be represented and drawn in a variety of ways.
[0046] According to one aspect, it is proposed that the recommendation service is a collaborative recommendation service and has a neural network with at least one convolutional layer, or has a recurrent neural network, or is based on a K-nearest neighbor method.
[0047] According to one aspect, it is proposed that the neural network has a first autoencoder for detecting the environment and a second autoencoder for detecting the possible trajectories. By dividing the neural network in the described manner, training can be made simpler.
[0048] An "autoencoder" is understood to be an artificial neural network that makes it possible to learn specific patterns contained in input data. An autoencoder is used to generate a compressed or noise-free representation of the input data by extracting important features (such as specific categories) from a generalized background, respectively.
[0049] Autoencoders use three or more layers:
[0050] Input layer, e.g. a 2D image.
[0051] • Multiple significantly smaller layers that form the encoding for data reduction.
[0052] An output layer, the dimension of which corresponds to the dimension of the input layer, i.e., each output parameter in the output layer has the same meaning as the corresponding parameter in the input layer.
[0053] Alternatively, the traffic environment and the possible trajectories may also be directly input into a convolutional neural network (CNN) to obtain an evaluation.
[0054] According to another aspect, a vector of a K-nearest neighbor method is formed by means of a large number of corresponding combinations of detected traffic environments and observed trajectories. The K-nearest neighbor method provides a parameter-free and easy-to-implement approach for evaluating new objects based on previously explicitly evaluated and stored objects.
[0055] According to one aspect, it is proposed to detect at least one static feature of the traffic environment with the aid of a geographical map. This is a simple method for detecting static features relevant to traffic. The method can be further improved using a high-precision geographical map.
[0056] According to one aspect, it is proposed that the traffic environment is detected by converting a spatial portion of the traffic environment into a two-dimensional reference system based on a top view of the traffic environment.
[0057] A top view or plan view is a two-dimensional orthogonal projection of a graphical representation of spatial conditions. In contrast to a bird's eye view, the top view is a planar representation from which the dimensions of an object can be measured.
[0058] According to one aspect, it is proposed that the detected traffic environment includes a previous trajectory of at least one road user.
[0059] According to a further aspect, it is proposed that at least one dynamic feature of the traffic environment of at least one road user is detected in its previous course for a sufficiently long time to be able to estimate therefrom a further course of the trajectory.
[0060] According to one aspect, it is proposed that the detected traffic environment includes a speed of at least one road user.
[0061] According to one aspect, it is proposed that the detected traffic environment includes a direction of movement of at least one road user.
[0062] According to one aspect, it is proposed to determine at least one possible trajectory by means of optimization of a cost function and / or by means of a search-based method and / or a machine learning method. By means of a plurality of methods for determining the possible trajectories, the method can be adapted to local conditions.
[0063] Possible trajectories can be determined by heuristic methods for lane keeping, for speed regulation, and based on the street orientation. For example, possible trajectories are determined for all possible lanes and additional trajectories are additionally considered for different traffic light circuits.
[0064] Standard methods can create possible trajectories based on the optimization of a cost function, either based on search or through machine learning. The calculation of trajectories through optimization creates trajectories that deviate as little as possible from the best possible properties (such as driving comfort, distance to traffic participants, and compliance with traffic regulations) either iteratively or directly. In short, search-based methods iteratively search for the best trajectory for its current environment by searching for a connection from A to B. Machine learning methods can learn to imitate trajectories that have already been seen and generate new trajectories based on these trajectories.
[0065] Alternative trajectory calculation functions are, for example, based on optimization and search for a trajectory that minimizes a given cost function; map-based methods, for example, simply follow the lane; deep learning methods such as "imitation learning", "reinforcement learning" create trajectories with the help of machine learning; physics-based methods determine the trajectory based on the dynamics of the vehicle and are mainly used for short approximations.
[0066] In particular, the trajectory maps the position of the object over time, but can also alternatively or additionally be represented as a speed specification over time and / or an acceleration specification over time.
[0067] When determining the trajectory, all possibilities for changing the driving direction due to lanes and lane merging can be taken into account. In particular, the possible trajectories of each traffic participant can be determined with the aid of an accurate map representation of the corresponding environment, including intersections, T-junctions, street layouts and / or current traffic regulations.
[0068] According to one aspect, it is proposed to determine at least one possible trajectory taking into account driving comfort on the trajectory and / or distances to other road users and / or compliance with traffic regulations. In particular, in the case of an at least partially automated vehicle, the vehicle uses the described method, for example, to determine a trajectory to be traveled, which serves to improve safety in a traffic situation or traffic environment.
[0069] According to one aspect, it is proposed to determine at least one possible trajectory by means of a map representation of the traffic environment. Here, all possibilities of changing the driving direction according to lanes and lane merging and intersections, current traffic regulations are to be considered.
[0070] According to one aspect, a method for creating a recommendation service for evaluating possible trajectories is proposed.
[0071] In this case, a plurality of corresponding combinations of detected traffic environments, observed trajectories and at least one possible trajectory are determined in one step for a plurality of different traffic environments by repeating the following steps:
[0072] In one step, the traffic environment is detected having at least one static feature and at least one dynamic feature.
[0073] In a further step, at least one road participant in the traffic situation is identified.
[0074] In a further step, an observed trajectory of the at least one road user is detected.
[0075] In a further step, at least one possible trajectory is determined for the at least one road user in the traffic environment.
[0076] Then, the recommendation service is subsequently adapted using a plurality of corresponding combinations of the detected traffic environment, the observed trajectory and the at least one possible trajectory by means of deviations of the observed trajectory from the at least one possible trajectory.
[0077] In this case, sensor data for detecting the traffic environment may be detected with the aid of the fused sensor data.
[0078] According to one aspect, it is proposed that, in order to evaluate the observed trajectory-traffic environment combination in the detected traffic environment, the difference between the possible trajectory of the traffic participant and the observed trajectory is calculated with the aid of a metric. By applying the metric to the result of the deviation of the observed trajectory from the possible trajectory, the trajectory used can be quantitatively described and stored for further calculations.
[0079] For the training data, all trajectories are explicitly evaluated with the aid of a metric. The recommendation service will then implicitly try to minimize the metric through the evaluation according to the similarity metric.
[0080] For example, a system determines a series of possible trajectories based on a current street map. All possible changes of direction based on existing lanes and possible lane merges are calculated. A series of characteristics of the traffic environment, such as relative distances, speeds, number of traffic participants, relationships of traffic participants, traffic rules, stop lines, traffic lines, overhead views of traffic participants, etc., are then used as input to the recommendation service to estimate the probabilities of the possible trajectories. Offline, i.e. without affecting the active prediction system, the possible trajectories of all traffic participants are compared with the observed trajectories. With the help of differences between lane positions, speed deviations, etc., each possible trajectory can be evaluated for multiple time steps.
[0081] These evaluations are used to train a machine learning system to
[0082] a) Estimate the intention as a regression problem,
[0083] b) compare the probabilities of two possible trajectories by classifying the possible trajectory with the higher probability, and
[0084] c) State the estimated probability for each possible trajectory.
[0085] This trained or adapted recommender system (also called rating estimator) is then used online to estimate the probabilities of different intents.
[0086] According to one aspect, it is proposed that the metric for calculating the difference between a possible trajectory and an observed trajectory of a road user determines the distance of a waypoint on the respective trajectory.
[0087] According to another aspect, the metric calculates the absolute differences of the waypoints on the corresponding trajectory at three different times after the moment of detecting the traffic environment, i.e. the absolute differences of the possible trajectory and the observed trajectory, and adds them up. The three different times can be located, for example, 1 second, 2 seconds or 3 seconds after the determination of the possible trajectory and can be obtained by weighing costs and benefits.
[0088] According to a further aspect, it is proposed that the measure is a function of the difference of the waypoints on the respective trajectory and / or the difference of the speed and / or acceleration at the waypoints.
[0089] In particular, the differences in position, velocity and acceleration can be calculated at three points in time.
[0090] This relationship can be described by the following formula 1:
[0091]
[0092] Where M(t k , t) represents the metric of trajectory tk, where:
[0093] t k = possible trajectory; t = observed trajectory;
[0094] v(t k , T) is the trajectory t at time T: (t1, t2, t3) k The speed at which
[0095] x(t k , T) is the trajectory t at time T: (t1, t2, t3) k location;
[0096] a(t k , T) is the trajectory t at time T: (t1, t2, t3) k The acceleration at .
[0097] The variable t_k represents the kth trajectory, t represents the observed trajectory, t1, t2, t3 represent the three time steps, v represents the velocity, x represents the position, and a represents the acceleration.
[0098] According to one aspect, the recommendation service is a collaborative recommendation service.
[0099] According to one aspect, it is proposed that the adaptation or training of the recommendation system is performed offline.
[0100] According to one aspect, it is proposed that the recommendation service evaluates the at least one possible trajectory by means of a neural network, which has been trained using a large number of evaluated observed trajectory-traffic environment combinations of road users.
[0101] This allows the trained neural network to directly provide a scalar at its output as a metric for evaluating possible trajectories (Maβzahl).
[0102] Neural networks provide a framework for many different algorithms for machine learning, collaboration, and processing of complex data inputs. Such neural networks learn to perform tasks based on examples without the need to typically program the neural network with task-specific rules.
[0103] Such neural networks are based on a collection of connected units or nodes called artificial neurons. Each connection can transmit a signal from one artificial neuron to another. The artificial neuron that receives the signal can process it and then forward the signal with modifications if necessary.
[0104] In the case of conventional implementations of neural networks, the signal is a real number at the connection of an artificial neuron, and the output of the artificial neuron is calculated by a nonlinear function of the sum of the inputs of the artificial neuron. The connections of the artificial neurons typically have weights that adapt as the learning progresses. The weights increase or decrease the strength of the signal at the connection. The artificial neuron may have a threshold value so that the signal is output only when the total signal exceeds the threshold value. Typically, a large number of artificial neurons are combined into layers. Different layers may perform possibly different types of transformations on their inputs. The signal migrates from the first layer (i.e., the input layer) to the last layer (i.e., the output layer), possibly after passing through these layers multiple times.
[0105] The architecture of the artificial neural feedforward network may be one configured in such a way that it receives a single data pattern in its input stage, e.g., according to an image, and provides an output value, e.g., a classification vector or a scalar, containing an approximation based on previous training, depending on the problem posed. The feedforward network passes each neuron exactly once.
[0106] Multilayer Perceptron (MLP) belongs to the family of feedforward artificial neural networks. Basically, MLP consists of at least three layers of neurons: an input layer, an intermediate layer (hidden layer) and an output layer. This means that all neurons of the network are divided into layers, where neurons of a given layer always get their input only from a single previous layer and forward their output likewise only to one fixed layer. Therefore, there are no connections that skip a layer or are activated multiple times. With the exception of the input layer, the different layers consist of neurons that are usually subjected to nonlinear activation functions and connected to neurons of the next layer.
[0107] A feedback neural network (RNN) is a neural network that, compared to a feedforward network, also has connections from neurons in one layer to neurons in the same or previous layer. Here, this structure is particularly suitable for discovering temporal coding information in data.
[0108] As a supplement to the description of the feedforward neural network above, the structure of the artificial neural convolutional network (Convolutional Neural Network) consists of one or more convolutional layers, followed by a pooling layer if necessary. The layer sequence can be used with or without a normalization layer (e.g., batch normalization), a zero padding layer, a dropout layer, and an activation function (e.g., a rectified linear unit ReLU, a sigmoid function, a tanh function, or a softmax function). In principle, these units can be repeated any number of times until they are repeated enough times and are then called a deep convolutional neural network. After repeating several blocks consisting of convolutional layers and pooling layers, the CNN can be ended with one (or more) fully connected layers (similar to the architecture of an MLP).
[0109] For the K-nearest neighbor method, during adaptation or training, vectors with information from environment detection, maps for autonomous driving, and information about possible trajectories are stored in a vector space.
[0110] The vector space is expanded by the length of the input vector, which, for example, has 80 entries for environmental detection and 20 entries for trajectories, which will expand the vector space to R^100. R here represents a real number, and R can be reduced according to the dimension of each entry, for example to natural numbers for discrete valued or binary inputs.
[0111] To evaluate a new environment detection-track combination represented by a vector, the new vector is compared with the existing vectors by means of a metric (e.g., the Euclidean norm). The existing "k" nearest vectors are added to the evaluation. The estimate is evaluated as a weighted sum of the estimates of the K nearest neighbors. As weights, for example, the same weights or weights selected inversely to the distance can be used.
[0112] This means that the detection of the traffic environment (including static information from the map material) is compressed into a vector with the help of an autoencoder. In addition, the directly calculated features for the environment (e.g. traffic light status, right-of-way rules, speed limits) are stored in a separate vector. The autoencoder provides a vector of size 256, i.e. a feature vector 64. In addition, the trajectory is represented with the help of the vector compressed by the autoencoder and the feature vector. The sizes of these vectors are 64 and 24 respectively. The combined vector of the combination of environment detection and trajectory consists of 408 entries and spans a vector space of this size. For each vector entry, there is an evaluation based on the similarity of the actually traveled trajectory to the possible predicted trajectory during training. During inference, i.e. during online evaluation, the evaluations of the three nearest neighbors of the predicted trajectory for a given environment are arithmetic averaged.
[0113] In order to optimize the running time, the expanded vector space can be compressed by means of a principal component analysis and then divided up by means of a “tree” so that the next entry can be found more quickly.
[0114] For the recommendation service, the features of the traffic environment with the associated observed trajectories are converted into a feature space and stored. For example, the most likely trajectory is determined based on the distance in the feature space by a K-nearest neighbor method. Here, instead of taking the next category, the category whose features are closest to the current trajectory is taken.
[0115] According to one aspect, it is proposed that the possible trajectories be evaluated repeatedly, for example cyclically, thereby making it possible to react to changes in the traffic situation.
[0116] For example, as soon as one of the asynchronously determined traffic circumstances has been calculated, a new evaluation is performed.
[0117] A method for planning a trajectory to be traveled by at least one traffic participant in a traffic environment is proposed, wherein in one step at least one possible trajectory is determined for each traffic participant in the traffic environment. In a further step, at least one of a plurality of possible trajectories is evaluated for each traffic participant according to the above method. For example, one possible trajectory for each accessible lane and a plurality of additional situations. In a further step, the trajectory to be traveled by the at least one traffic participant in the traffic situation is determined with the aid of all evaluated possible trajectories of other traffic participants.
[0118] According to one aspect, it is proposed to transfer the evaluation of possible trajectories to a trajectory planning system. This then enables the trajectory planning system to plan the trajectory to be traveled.
[0119] According to one aspect, it is proposed that, as a function of the evaluation of the trajectory, at least one of the vehicles calculates its own trajectory based on all available information.
[0120] The described method can be used for behavior planning, trajectory planning, motion planning and as an emergency braking assistant or driver assistance system.
[0121] From the evaluation of the possible trajectories, probabilities can be calculated, and the planning of the self-trajectory to be pursued can then be carried out using the possible trajectories identified in this way.
[0122] A system for suggesting a self-trajectory to be traveled is proposed, which is designed to carry out a method according to the above method and to suggest a trajectory to be traveled to one of the traffic participants based on an evaluation of the trajectories of other traffic participants. The trajectory to be traveled is the trajectory that the traffic participant wants to use to reach his destination.
[0123] A device is proposed which is designed to carry out the above method.
[0124] A computer program is proposed, which comprises instructions which, when the program is executed by a computer, cause the computer to carry out the above-described method.
[0125] A machine-readable storage medium is proposed, on which the described computer program is stored. BRIEF DESCRIPTION OF THE DRAWINGS
[0126] The embodiments of the present invention are Figures 1 to 3 and described in more detail below.
[0127] Figure 1 A street intersection with possible trajectories of traffic participants is shown;
[0128] Figure 2 A T-junction with two road users and possible trajectories is shown; and
[0129] Figure 3 The steps of a method for evaluating possible trajectories are shown. DETAILED DESCRIPTION
[0130] exist Figure 1 , a traffic environment 100 is schematically depicted in the form of a top view of a multi-lane street intersection. A traffic participant 110 is schematically depicted, and various possible trajectories of the traffic participant 110 are depicted by dashed lines a to f. In this case, multiple trajectories can also be considered for at least one lane. Traffic participants that are known earlier are only represented by thin lines of their observed trajectories.
[0131] Figure 2Another traffic environment 200 is schematically depicted from a top view of a T-junction or T-junction with a self-vehicle 210 located in front of a stop line 211, which does not have the right of way. From the perspective of the self-vehicle 210, a traffic participant 220 comes from the left, and its vehicle 220 is at a slight angle to the straight direction on the right-of-way street. Three possibilities for continuing to travel are obtained under the situation of the traffic participant 220, and a possible trajectory is drawn for each possibility. For turning into a cross street, it is trajectory 230, for continuing to travel in a straight direction, it is trajectory 250, and for turning around on the right-of-way street, it is trajectory 240. All possible changes of direction are calculated based on existing lanes and possible mergers. In addition, the trajectory to be traveled by the self-vehicle 210 that plans to turn left to the right-of-way street is drawn with a dotted line. The times t1, t2, and t3 are represented by black dots on the possible trajectories 230, 240, and 250 of the traffic participant 220. Furthermore, an observed trajectory 270 of the traffic participant 220 is plotted, and the times t1 , t2 , t3 are represented by small squares on the observed trajectory 270 to describe the steps of adapting the recommendation service.
[0132] According to the method for evaluating possible trajectories of traffic participants in a traffic environment, a traffic environment 200 with its static features of a T-junction is detected in one step with the aid of a geographic map (e.g., a top view and dynamic features of a self-vehicle 210 and traffic participants 220). A series of features, such as the relative distance of vehicles 210, 220: barely a vehicle length, the number of traffic participants: here two, the spatial positions of these traffic participants relative to one another: here almost perpendicular to one another, traffic rules: here streets with right of way and stop lines, are provided as inputs for the recommendation service to evaluate all possible trajectories 230, 240, 250.
[0133] In a further step, a traffic participant 220 is identified, for example, by means of detected dynamic features of the traffic situation 200. In a further step, a possible trajectory 230, 240, 250 of at least one traffic participant 220 in the traffic situation 200 is determined. In a further step, a possible trajectory 230, 240, 250 of at least one traffic participant 220 in the traffic situation 200 is evaluated by means of a recommendation service. Figure 2 As can be seen, possible trajectories 230 , 240 , 250 are determined, in particular taking into account driving comfort and compliance with traffic regulations on the trajectories.
[0134] As already described above, the recommendation service evaluates possible trajectories 230 , 240 , 250 with the aid of a large number of evaluated observed trajectory-traffic environment combinations of road users.
[0135] When the road user 220 continues his path, it can be determined that Figure 2 Observed trajectory 270 indicated in , to obtain other trajectory-traffic environment combinations for training the recommendation service. The observed trajectory 270 can be evaluated using the possible trajectories 230, 240, 250 by means of a metric, in which waypoints on the corresponding trajectory are determined at three times and the distances between the waypoints from the observed trajectory 270 and the waypoints from the corresponding possible trajectories 230, 240, 250 are measured respectively. The sum of these values yields a metric for evaluation according to formula 1.
[0136] The recommendation service used has an adapted, in particular trained, neural network, wherein the recommendation service evaluates the at least one possible trajectory by means of a neural network that has been trained using a large number of evaluated observed trajectory-traffic environment combinations of road users. The neural network has a first autoencoder for detecting the environment and a second autoencoder for detecting the possible trajectory.
[0137] The output of environment detection is used as the input of the autoencoder to represent the current traffic conditions around the ego vehicle and traffic participants.
[0138] To this end, the output of the environment detection is partially converted to a top-down view with the help of a 3D transformation and correspondingly represented in 2D as image planes. These image planes are then compressed as part of the autoencoder with the help of convolutional layers.
[0139] Additionally, information such as traffic regulations, distances between traffic participants, speeds, traffic light states and other properties that are difficult to visualize are added as vectors to the next layer of the autoencoder. The possible trajectories are also processed into a 2D visualization including map material in a separate autoencoder by means of a 3D projection and supplemented with additional information vectors.
[0140] The outputs of the two autoencoders are then combined and fed into a system of multiple layers at dense and normalized layers. The output of the system is a scalar estimate of the likely trajectory of the current environment. An additional dropout layer randomly silences some neurons in the neural network to reduce the possibility of overfitting.
[0141] The visualization of different properties of the environment detection is fed as input to an autoencoder with a set base structure. The environment is defined in a (6, 224, 224) tensor, which is defined as a 6-color channel image with a 224 pixel x 224 pixel resolution structure as a visualization of the environment. The different color channels represent static structure, map information, dynamic structure, speed, current right-of-way lane, and traffic light stage.
[0142] The basic structure of the convolutional network was set to Resnet-34 with an output vector of 2000 units. In addition, we added 96 units with additional information about the environment, such as speed limits, right-of-way rules, exact distances and lane relevance. The trajectories are visualized as two-channel images. One channel visualizes the map material and the other channel visualizes possible trajectories. The visualization is encoded into a vector of 512 units with the help of the autoencoder of Resnet-34. Additionally, one-dimensional speed steps, acceleration steps and orientation steps are added in a vector of 72 units.
[0143] The output of the autoencoder with information vectors, i.e., 2096 and 584 units, is imported as a vector into a dense feedforward network with scalar outputs. The structure of the network is as follows.
[0144] Layer (Type) Initial form parameter# enter (2680,1) 0 Intensive 21 (4096,1) 4096*2680 quit (4096,1) 0 Intensive 2 (2048,1) 2048*4096 Batch Normalization (2048,1) 0 Intensive 3 (1028,1) 1028*2048 Intensive 4 (512,1) 512*1028 Output (1) 512
[0145] Table 1 describes the feed-forward network behind the autoencoder. The input is an input vector of 2680 units.
Claims
1. A method for evaluating possible trajectories of traffic participants in a traffic environment, comprising the following steps: Detecting traffic environments with static and dynamic characteristics (S1); identifying at least one traffic participant in the detected traffic environment (S2); determining at least one possible trajectory for at least one identified traffic participant in the traffic environment (S3); By means of the adapted / trained recommendation service and the detected traffic environment, the at least one possible trajectory determined for the at least one identified traffic participant is evaluated based on similar behaviors of similar users (S4), in, The recommendation service is adapted to evaluate possible trajectories by means of a large number of corresponding combinations of evaluated observed trajectories, different traffic environments and the at least one possible trajectory.
2. The method according to claim 1, wherein: The recommendation service is a collaborative recommendation service and has a neural network with at least one convolutional layer, or has a recurrent neural network, or is based on a K-nearest neighbor method.
3. The method according to claim 1, wherein: The recommendation service is a collaborative recommendation service and has a neural network with at least one convolutional layer, or has a recurrent neural network with a first autoencoder for detecting the environment and a second autoencoder for detecting the possible trajectories.
4. The method according to claim 1, wherein: The recommendation service is a collaborative recommendation service and is based on a K-nearest neighbor method, whose vector is formed by means of a large number of corresponding combinations of detected traffic environments and observed trajectories.
5. The method according to any one of the preceding claims 1 to 4, wherein: The at least one possible trajectory is determined with the aid of a geographical map of the traffic environment.
6. The method according to any one of the preceding claims 1 to 4, wherein: The traffic environment is detected by converting a spatial portion of the traffic environment into a two-dimensional reference system based on a top view of the traffic environment.
7. The method according to any one of the preceding claims 1 to 4, wherein: The detected traffic environment includes the previous trajectory of at least one road user.
8. The method according to any one of the preceding claims 1 to 4, characterized in that At least one possible trajectory is determined by means of an optimization of a cost function and / or by means of a search-based method and / or a machine learning method.
9. A method for creating a recommendation service for evaluating possible trajectories based on similar behaviors of similar users, comprising the following steps: for a large number of different traffic environments, determining a large number of corresponding combinations of detected traffic environments, observed trajectories, and at least one possible trajectory by repeating the following steps: detecting the traffic environment having at least one static characteristic and at least one dynamic characteristic; identifying at least one traffic participant of the traffic situation in the detected traffic environment; detecting an observed trajectory of the identified at least one traffic participant; determining at least one possible trajectory for at least one identified traffic participant of the traffic environment; and By means of the deviation of the observed trajectory from the at least one possible trajectory, the recommendation service is adapted using a plurality of corresponding combinations of the detected traffic environment, the observed trajectory and the at least one possible trajectory.
10. The method according to claim 9, wherein: In order to evaluate the observed trajectory-traffic environment combination in the detected traffic environment, the difference between the possible trajectory of the road user and the observed trajectory is calculated with the aid of a metric.
11. A method for planning a trajectory to be travelled by at least one traffic participant in a traffic environment, comprising the following steps: determining at least one possible trajectory for each traffic participant in the traffic environment; According to the method according to any one of claims 1 to 10, a plurality of possible trajectories are evaluated for each road participant; and the trajectory to be traveled by the at least one road participant in the traffic situation is determined with the aid of all evaluated possible trajectories of other road participants.
12. A device comprising a computer and a memory, in which a computer program containing program instructions is stored, which, when the computer program is executed by the computer, cause the computer to perform the method according to any one of claims 1 to 11.
13. Computer program product comprising a computer program containing program instructions which, when executed by a computer, cause the computer to carry out the method (10) according to any one of claims 1 to 11.
14. A machine-readable storage medium having stored thereon a computer program containing program instructions which, when executed by a computer, cause the computer to implement the method (10) according to any one of claims 1 to 11.
Citation Information
Patent Citations
Method for selecting optimizing the trajectory
CN107010053A
Journey time estimation method based on auxiliary supervised learning
CN109035761A
Evaluation framework for predicted trajectories in autonomous driving vehicle traffic prediction
CN109937343A
Method and device for predicting trajectories of a motor vehicle
DE102016203522A1
Prediction of a road user's behavior
DE102017212629A1
