Prediction of behavior of road users

Abstract

One aspect of the invention relates to an apparatus for predicting the behavior of a road user, wherein the apparatus is configured to: provide at least one hypothesis about the behavior of the road user, provide a hidden Markov model for each hypothesis, wherein the hidden Markov model includes two hidden states for the corresponding hypothesis, wherein one hidden state indicates that the road user follows the hypothesis, and the other hidden state indicates that the road user does not follow the hypothesis, and wherein possible observations of the hidden Markov model for the corresponding hypothesis characterize at least one feature of the road user, and predict the behavior of the road user based on the hidden states of the hidden Markov model for the at least one hypothesis.

Description

Technical Field

[0001] This invention relates to an apparatus and method for predicting the behavior of road users. Background Technology

[0002] Within the scope of this document, the term "autonomous driving" can be understood as driving with automatic longitudinal or lateral guidance, or autonomous driving with automatic longitudinal and lateral guidance. The term "autonomous driving" includes autonomous driving with any level of automation. Exemplary levels of automation are driver assistance, partial automation, highly automated driving, or fully automated driving. These levels of automation are defined by the Federal Highway Research Institute (BASt) (see BASt publication "Research Report," version 11 / 2012). In driver assistance, the driver continuously performs longitudinal or lateral guidance, while the system takes over other functions within certain limits. In partial automation (TAF), the system takes over longitudinal and lateral guidance for a period of time and / or under specific circumstances, where the driver must continuously monitor the system as in driver assistance. In highly automated driving (HAF), the system takes over longitudinal and lateral guidance for a period of time without the driver continuously monitoring the system; however, the driver must be able to take over vehicle guidance for a certain period. In fully automated driving (VAF), the system automatically manages driving in all situations for a specific application scenario where a driver is no longer required. The four levels of automation defined by BASt correspond to SAE Levels 1 through 4 in the SAE J3016 standard (SAE - Society of Automotive Engineers). For example, Highly Automated Implementation (HAF) according to BASt corresponds to Level 3 in the SAE J3016 standard. Furthermore, SAE J3016 specifies SAE Level 5 as the highest level of automation, which is not included in the BASt definition. SAE Level 5 corresponds to fully autonomous driving, where the system can automatically handle all situations like a human driver throughout the entire driving process; generally, a driver is no longer needed.

[0003] In order to achieve high-quality automated driving, it is necessary to predict the behavior of road users around the automated vehicle.

[0004] Several methods are known to address this, which utilize artificial intelligence, particularly neural networks. However, these methods often suffer from a conflict of objectives: the conflict between the most accurate prediction of road user behavior and the minimum resource requirements needed to determine the prediction outcome. Summary of the Invention

[0005] The purpose of this invention is to provide an apparatus and method for predicting the behavior of road users with high quality and low resource requirements.

[0006] This objective is achieved through the features of the independent claim. Advantageous embodiments are described in the dependent claims. It should be noted that additional features of a claim dependent on the independent claim, either in the absence of features of the independent claim or only in combination with a subset of features of the independent claim, can constitute a separate invention independent of all combinations of features of the independent claim, which can be the subject of the independent claim, divisional application, or subsequent application. This also applies to the technical theories described in the specification, which can form an invention independent of the features of the independent claim.

[0007] The first aspect of the invention relates to a device for predicting the behavior of road users, particularly motor vehicles as road users.

[0008] Devices for predicting road user behavior, particularly components of driver assistance systems or driving systems for automated motor vehicles, and providing the predicted road user behavior to the driver assistance system or driving system as information for mobility planning and mobility control.

[0009] The behavior of road users includes, in particular, at least one decision made by a road user in resolving their driving task, such as a road user's decision about driving in or out of a lane.

[0010] The device is configured to provide at least one hypothesis about the behavior of a road user. Here, the hypothesis precisely describes the possible behaviors of the road user. In other words, the hypothesis defines a scenario over a finite timeframe: what decisions the road user will make when facing a driving task.

[0011] Furthermore, the device is configured to provide a hidden Markov model for each hypothesis.

[0012] Hidden Markov Models contain exactly two hidden states for each hypothesis: one hidden state indicates that the road user follows the hypothesis, and the other hidden state indicates that the road user does not follow the hypothesis.

[0013] Here, the present invention is based on the understanding that the algorithmic complexity of Hidden Markov Models is limited to only two hidden states, because this typically increases quadratically with the total number of states.

[0014] The corresponding hypothetical two hidden states describe logically opposite possibilities, thus completing the picture regarding the road user's actual course of action. The road user either follows the hypothesis or does not. Therefore, the road user is always in one of two states.

[0015] The possible observations of the corresponding hypothetical Hidden Markov Model characterize at least one feature of the road user. There exists a correlation, in the form of a probability density function, between the representation of at least one feature and its hidden state, such that each representation of the feature represents a specific probability that the corresponding hidden state currently exists.

[0016] At least one characteristic, particularly an absolute characteristic of the road user, can be determined by observing the road user.

[0017] Alternatively, the at least one characteristic, particularly the relative characteristic of the road user, can be determined by observing the relationship between the road user and a reference object, such as the relationship with the lane, another road user, or the vehicle itself.

[0018] The device is configured to predict the behavior of road users based on the hidden states of at least one hypothetical Hidden Markov Model.

[0019] In particular, predictions of road user behavior are made by determining the most likely hidden state of a hidden Markov model that assumes at least one hypothesis about the behavior of road users.

[0020] Here, predicting the behavior of road users may include, for example, establishing at least one hypothesis and determining the most likely hidden state of the Hidden Markov Model. Alternatively, predicting the behavior of road users may also simply involve determining the most likely hidden state of the Hidden Markov Model.

[0021] By determining the most likely hidden state of the Hidden Markov Model, it is possible to determine whether road users follow the hypothesis for each hypothesis, i.e., whether their behavior conforms to the hypothesis.

[0022] In the case of multiple assumptions and multiple Hidden Markov Models (HMMs), the most probable state of multiple HMMs can be equivalent to the road user following the corresponding assumptions. In this case, or conversely, in the case where the most probable state cannot be explicitly determined for any HMM, the road user's behavior can be classified as unclassifiable, which in driver assistance systems may, for example, lead to an increase in the safe distance from the corresponding road user.

[0023] In an advantageous implementation, at least one characteristic of the road user is a quantifiable characteristic of the road user. In particular, a characteristic is quantifiable if its representation can be converted into a measurable quantity and value. For example, a characteristic is quantifiable if its representation can be expressed numerically without substantially or even any loss of information content. Thus, for example, a road user's trajectory is not a quantifiable characteristic.

[0024] In another advantageous implementation, the possible observations of the correspondingly assumed Hidden Markov Model characterize at least two independent features. Specifically, features are independent if the probability of the hidden state can be inferred from the representation of the features regardless of the representations of other features.

[0025] For example, it can be assumed that for a road user turning right at an intersection, the distance from the road user to the center of the lane is independent of the angular distance of the lane orientation. This assumption may not always be true in practice, but it greatly limits the complexity of the model.

[0026] In another advantageous implementation, possible observations of the correspondingly assumed Hidden Markov Model include the distance from the road user to the center of the lane in which the road user is located as a feature.

[0027] The present invention is based on the understanding that, for certain behaviors of road users, the distance from the road user to the center of the lane where the road user is located is very precise. Therefore, information about hypothetical scenarios, such as whether the road user is traveling along or turning from the lane, can be obtained from the distance from the road user to the center of the lane.

[0028] In another advantageous implementation, possible observations of the correspondingly assumed Hidden Markov Model include the deviation between the road user's orientation and the orientation of the lane in which the road user is located as a feature.

[0029] The present invention is based on the understanding that, for certain behaviors of road users, the deviation between the road user's orientation and the orientation of the lane in which the road user is located is very precise. Therefore, from the deviation between the road user's orientation and the orientation of the lane in which the road user is located, information about assumptions can be obtained, such as whether the road user is driving along the lane or turning from the lane.

[0030] Here, the difference between the orientation of the road user and the orientation of the lane, particularly the yaw angle, indicates the deviation.

[0031] In another advantageous implementation, possible observations of the correspondingly assumed Hidden Markov Model include the activation of the driving direction indicator lights of road users as a feature.

[0032] Here, the present invention is based on the understanding that the activation of the driving direction indicator is also very precise for certain behaviors of road users. Therefore, information regarding the hypothesis that a road user is turning left or right from the lane can be obtained from the activation of the driving direction indicator.

[0033] In addition, the basic modeling can also take into account, for example, road users signaling right but not turning right with a very low probability, or road users turning without signaling.

[0034] In another advantageous implementation, possible observations of the correspondingly assumed Hidden Markov Model include features characterizing the right-of-way behavior of road users. Features characterizing the right-of-way behavior of road users include, for example, the deceleration of road users, particularly when road users are forced to yield the right-of-way due to the current right-of-way situation.

[0035] To simplify the model, it is assumed that the above features are conditionally independent of each other given the hidden state, and therefore these features are particularly suitable for combination in Hidden Markov Models.

[0036] In another advantageous embodiment, the device is configured to determine the traffic conditions in which the road user is located, and to determine at least one hypothesis about the behavior of the road user based on the traffic conditions.

[0037] A second aspect of the invention relates to a method for predicting the behavior of road users.

[0038] One step in this method is to provide at least one hypothesis about the behavior of road users.

[0039] Another step of the method is to provide a hidden Markov model for each hypothesis, wherein the hidden Markov model includes two hidden states for the corresponding hypothesis, one hidden state indicating that the road user follows the hypothesis and the other hidden state indicating that the road user does not follow the hypothesis, and wherein the possible observations of the hidden Markov model for the corresponding hypothesis characterize at least one feature of the road user.

[0040] Another step in the method is to predict the behavior of road users based on the hidden states of at least one hypothetical Hidden Markov Model.

[0041] The above-described embodiments of the apparatus according to the first aspect of the invention are also applicable in a corresponding manner to the method according to the second aspect of the invention. Advantageous embodiments of the method according to the invention not explicitly stated herein and in the claims correspond to advantageous embodiments of the apparatus according to the invention described above or in the claims. Attached Figure Description

[0042] The present invention will now be described with reference to the accompanying drawings and embodiments. Wherein:

[0043] Figure 1 An exemplary traffic situation is shown.

[0044] Figure 2 An embodiment of the device according to the invention is shown, and

[0045] Figure 3 Another embodiment of the device according to the invention is shown. Detailed Implementation

[0046] Figure 1 An exemplary traffic situation is illustrated. In this situation, road user VT moves toward the intersection, where they have three different behavioral options: turn left, proceed straight through the intersection, or turn right.

[0047] The first assumption h1 regarding the behavior of road user VT is that road user VT turns left.

[0048] The second assumption h2 regarding the behavior of road user VT is that road user VT travels straight through the intersection.

[0049] The third assumption h3 regarding the behavior of road user VT is that road user VT turns right.

[0050] Figure 2 An embodiment of a device PV for predicting the behavior of a road user VT according to the present invention is shown.

[0051] The device PV is configured to provide at least one hypothesis h1, h2, h3 regarding the behavior of road user VT. Specifically, the device is configured to determine the traffic situation in which road user VT is located, and based on that traffic situation, determine at least one hypothesis h1, h2, h3 regarding the behavior of road user VT. For example, if road user VT... Figure 1 If the lanes moving towards the intersection are shown, then the lanes leaving the intersection can all be assumed as h1, h2, and h3.

[0052] In addition, the device PV is configured to provide a hidden Markov model for each hypothesis h1, h2, h3.

[0053] Hidden Markov models include two hidden states s1,1-s3,2 for each hypothesis. Specifically, for hypothesis h1, the hidden states are s1,1 and s1,2; for hypothesis h2, the hidden states are s2,1 and s2,2; and for hypothesis h3, the hidden states are s3,1 and s3,2.

[0054] One of these hidden states, such as hidden states s1,1, s2,1 and s3,1, represents that road user VT follows the corresponding assumptions h1, h2 and h3, respectively.

[0055] Another hidden state among these hidden states, namely hidden states s1,2, s2,2 and s3,2, respectively, indicates that road user VT does not follow the corresponding assumptions h1, h2 and h3.

[0056] Each Hidden Markov Model (HMM) includes predetermined probabilities for transitions between its two states. Thus, for example, suppose the HMM h1 has a probability of ps12 for transitioning from state s1,1 to state s1,2 and a probability of ps11 for transitioning from state s1,2 to state s1,1. Suppose the HMM h2 has a probability of ps22 for transitioning from state s2,1 to state s2,2 and a probability of ps21 for transitioning from state s2,2 to state s2,1. Suppose the HMM h3 has a probability of ps32 for transitioning from state s3,1 to state s3,2 and a probability of ps31 for transitioning from state s3,2 to state s3,1.

[0057] These probabilities are, for example, 50%.

[0058] Furthermore, the corresponding Hidden Markov Model also includes possible observations b1-b6. These possible observations b1-b6 characterize at least one feature of the road user VT, particularly a quantifiable feature of the road user VT.

[0059] Here, it is assumed that the possible observations b1-b4 of the hidden Markov models of h1, h2, and h3 represent at least two features m1 and m2 that are conditionally modeled independently of each other.

[0060] For example, the possible observations b1 and b2 of the hidden Markov model assuming h1 represent the distance from road user VT to the center of the lane where road user VT is located. Furthermore, the possible observations b3 and b4 of the hidden Markov model assuming h2 represent the deviation between the orientation of road user VT and the orientation of the lane where road user VT is located.

[0061] For each hidden state s1,1-s3,2, there exists a certain probability p11,1-p32,2 of actually observing the corresponding possible observation result b1-b6.

[0062] In addition, the device PV is also configured to predict the behavior of road user VT based on the hidden states s1,1-s3,2 of a hidden Markov model of at least one hypothesis h1, h2, h3.

[0063] Figure 3 Another embodiment of the device PV for predicting the behavior of road user VT according to the present invention is shown.

[0064] The device PV is configured to provide at least one hypothesis h1 for the behavior of road user VT, and a hidden Markov model is provided for the hypothesis h1.

[0065] The Hidden Markov Model includes two hidden states s1,1 and s1,2 for hypothesis h1. One hidden state s1,1 indicates that road user VT follows hypothesis h1, and the other hidden state s1,2 indicates that road user VT does not follow hypothesis h1.

[0066] Suppose that the possible observations b1-b4 of the Hidden Markov Model of h1 characterize at least one feature of the road user VT. Here, it is assumed that the possible observations b1-b4 of the Hidden Markov Model of h1 characterize two independent feature sets m1 and m2, wherein feature set m1 includes features b1 and b2, and feature set m2 includes features b3 and b4.

[0067] For example, suppose that possible observations b1 and b2 of the hidden Markov model of h1 characterize the activation of the driving direction indicator of road user VT, and suppose that possible observations b3 and b4 of the hidden Markov model of h1 describe the characteristics characterizing the behavior of road user VT in yielding the right-of-way.

[0068] Alternatively, b1 represents the distance to the centerline, b2 represents the difference in orientation from the lane, b3 represents the turn signal status in the current time step, and b4 represents the right-of-way characteristic, which is calculated, for example, by at least one traffic rule and acceleration or braking behavior.

[0069] For clarity, and Figure 2 In comparison, Figure 3 The hidden Markov model does not show the transition probabilities between hidden states s1,1 and s1,2, nor does it show the probabilities of possible observations b1-b4 in hidden states s1,1 and s1,2.

[0070] The device PV is configured to predict the behavior of road user VT based on the hidden states s1,1 and s1,2 of a hidden Markov model with hypothesis h1.

Claims

1. A device (PV) for predicting the behavior of a road user (VT), wherein the device (PV) is configured to: • Determine the traffic conditions in which the road user (VT) is located; • Determine multiple potential behaviors of the road user (VT) based on the traffic conditions; • For each of the plurality of potential behaviors of the road user (VT), a hypothesis (h1, h2, h3) is provided respectively. • A Hidden Markov Model (HMM) is provided for each hypothesis (h1, h2, h3), wherein the HMM comprises two hidden states (s1,1-s3,2) for the corresponding hypothesis, wherein one hidden state (s1,1; s2,1; s3,1) indicates that the road user (VT) follows the hypothesis (h1, h2, h3), and the other hidden state (s1,2; s2,2; s3,2) indicates that the road user (VT) does not follow the hypothesis (h1, h2, h3), and wherein the possible observations (b1-b6) of the HMM for the corresponding hypothesis (h1, h2, h3) characterize at least one feature of the road user (VT), and • Based on the hidden states (s1,1-s3,2) of the Hidden Markov Model of the assumptions (h1, h2, h3), determine the predicted behavior of the road user (VT) from the plurality of potential behaviors.

2. The device (PV) according to claim 1, wherein at least one characteristic of the road user (VT) is a quantifiable characteristic of the road user (VT).

3. The apparatus (PV) according to any one of the preceding claims, wherein the possible observations (b1-b4) of the hidden Markov model of the corresponding assumptions (h1, h2, h3) characterize at least two independent feature sets (m1, m2).

4. The device (PV) according to any one of the preceding claims, wherein the possible observations (b1-b6) of the hidden Markov model of the corresponding assumptions (h1, h2, h3) include the following as features: the distance from the road user (VT) to the center of the lane where the road user (VT) is located.

5. The device (PV) according to any one of the preceding claims, wherein the possible observations (b1-b6) of the hidden Markov model of the corresponding assumptions (h1, h2, h3) include the following as features: the deviation of the orientation of the road user (VT) from the orientation of the lane in which the road user (VT) is located.

6. The device (PV) according to any one of the preceding claims, wherein the possible observations (b1-b6) of the hidden Markov model of the corresponding assumptions (h1, h2, h3) include the following as features: activation of the driving direction indicator of the road user (VT).

7. The apparatus (PV) according to any one of the preceding claims, wherein the possible observations (b1-b6) of the hidden Markov model of the corresponding assumptions (h1, h2, h3) include: The characteristics that characterize the behavior of the road user (VT) in yielding the right-of-way.

8. A method for predicting the behavior of road users (VTs), wherein the method includes the following steps: • Determine the traffic conditions in which the road user (VT) is located; • Determine multiple potential behaviors of the road user (VT) based on the traffic conditions; • For each of the plurality of potential behaviors of the road user (VT), a hypothesis (h1, h2, h3) is provided respectively. • A Hidden Markov Model (HMM) is provided for each hypothesis (h1, h2, h3), wherein the HMM comprises two hidden states (s1,1-s3,2) for the corresponding hypothesis, wherein one hidden state (s1,1; s2,1; s3,1) indicates that the road user (VT) follows the hypothesis (h1, h2, h3), and the other hidden state (s1,2; s2,2; s3,2) indicates that the road user (VT) does not follow the hypothesis (h1, h2, h3), and wherein the possible observations (b1-b6) of the HMM for the corresponding hypothesis (h1, h2, h3) characterize at least one feature of the road user (VT), and • Based on the hidden states (s1,1-s3,2) of the Hidden Markov Model of the assumptions (h1, h2, h3), determine the predicted behavior of the road user (VT) from the plurality of potential behaviors.

Images