Predictive contextual behavior modeling of road users

Abstract

The invention relates to a system for a vehicle (0), comprising a computing module (1) having a neural network and an input data interface (3) designed to provide past movements of other road users and current sensor data, wherein a first (5) and a second (7) sub-network make a location-related or a time-related prediction of the future behavior of other road users, and the input data interface (3) carries out reinforcement learning with the aid of an anomaly detection unit (9) for adapting weightings of a machine learning attention mechanism.

Description

[0001] 24-1771

[0002] 1

[0003] Predictive contextual behavior modeling of road users

[0004] The invention relates to a system for an automated vehicle for predicting a traffic scenario, and a method for predicting a traffic scenario to determine a driving strategy for an automated vehicle.

[0005] In this document, the term "automated driving" or "automated vehicle control" refers to driving with automated longitudinal and / or lateral control. Automated driving can, for example, involve extended periods of driving on the highway or time-limited driving during parking maneuvers. The term "automated driving" encompasses automated driving at any level of automation. Examples of automation levels include assisted, partially automated, conditionally automated, highly automated, and fully automated driving (each with an increasing degree of automation). The five automation levels mentioned above correspond to SAE Levels 1 to 5 of the SAE J3016 standard (SAE - Society of Automotive Engineering) as of April 30, 2021. In assisted driving (SAE Level 1), the system performs longitudinal or lateral control in specific driving situations.In partially automated driving (SAE Level 2), the system takes over longitudinal and lateral control in certain driving situations, although the driver must continuously monitor the system, as with assisted driving. In conditionally automated driving (SAE Level 3), the system takes over longitudinal and lateral control in certain driving situations without the driver needing to continuously monitor the system; however, the driver must be able to take over vehicle control within a certain timeframe if requested by the system. In highly automated driving (SAE Level 4), the system takes over vehicle control in certain driving situations, even if the driver does not respond to a request to intervene, thus eliminating the driver as a fallback option. In fully automated driving (SAE Level 5), the system can perform all aspects of the dynamic driving task under any road and environmental conditions that a human driver can also handle.

[0006] Challenges in automated driving include, for example, adapting to dynamic conditions and ensuring safety. This requires predictive capabilities in a system to anticipate the movement of other road users, even under changing and unpredictable environmental conditions. The use of static data and 24-1771

[0007] Using two predefined algorithms from simpler forecasting models, which may not adapt efficiently to sudden changes in the environment such as accidents or weather fluctuations, can lead to unreliable or inaccurate predictions.

[0008] The object of the invention is to provide a more robust and adaptable predictive model of the future traffic situation for an automated vehicle.

[0009] The invention is defined by the features of the independent claims. Advantageous further developments and embodiments are the subject of the dependent claims.

[0010] A first aspect of the invention relates to a system for a vehicle, comprising a computing module with a stored and pre-trained artificial neural network from the class of graph neural networks, and comprising an input data interface designed to provide information to the artificial neural network at least about past movements of other road users and about a currently sensor-detected environment of the vehicle, including other road users, wherein the artificial neural network comprises a first subnetwork and a second subnetwork, wherein the first subnetwork is designed to make a location-based prediction based on the information provided at the input data interface, and wherein the second subnetwork is designed to make a time-based prediction of the future behavior of the other road users.and wherein the input data interface is configured to perform reinforcement learning during vehicle operation, which uses the predictive accuracy of the future behavior of other road users and / or the accuracy of a derived driving strategy of the vehicle to adjust weights of a machine learning attention mechanism of the artificial neural network, and which uses the result of an anomaly detection unit to detect anomalies relating to the past movements of other road users and / or to the currently sensor-detected environment of the vehicle, wherein input variables of the machine learning attention mechanism are based in particular on the information provided at the input data interface.

[0011] The purpose of the system is to predict the future behavior of at least one other road user in order to derive a driving strategy for the vehicle based on this prediction. The future behavior of one or more other road users preferably includes not only the individual movement of the road user, but also, more generally, their interaction with other road users and thus also with the vehicle itself in the broadest sense. At a minimum, however, the 24-1771

[0012] 3. Future behavior in its estimated form represents a future trajectory of the respective other road user.

[0013] The vehicle's driving strategy is a broad term and can be implemented as an intervention in the manual driving controls of a manually driven vehicle, thus functioning as a driver assistance system (SAE Level 1 or 2), for example, initiating emergency braking. However, in more highly automated vehicles such as SAE Level 3 or higher, the driving strategy can also include path planning, particularly trajectory planning. In the case of path or trajectory planning, complex decisions must be made compared to simpler interventions in manual driving, similar to those a human driver would make. This can include decisions about which lane to occupy, overtaking maneuvers, yielding or claiming the right-of-way, interaction with other road users, and much more.

[0014] Predicting the future behavior of at least one other road user, and in particular all other road users in the immediate vicinity of the vehicle, serves to determine the vehicle's driving strategy. Once a prediction of future behavior has been made, methods known in the prior art can be used to derive the driving strategy, which is why this aspect will not be discussed further here.

[0015] The system primarily uses historical data from the past movements of other road users, such as the journeys of other vehicles, but also current sensor data describing the vehicle's surroundings. This information is processed in an artificial neural network from the class of graph neural networks to predict future behavior. A graph neural network (also called a "graph neural network," abbreviated GNN) is a class of artificial neural networks for processing data that can be represented as a graph.

[0016] The artificial neural network is preferably a multi-layer GNN and can comprise one or more graph convolutional networks or one or more graph attention networks, which use so-called attention layers to implement an attention mechanism from the field of machine learning. Using methods of reinforcement learning, the weights of the attention mechanism are adjusted, effectively weighting the use of the different data sources in their input to the artificial neural network, as described in [24-1771].

[0017] 4

[0018] Data sources include at least information about the past movements of other road users as well as current information from the sensor data relating to the vehicle's surroundings. The artificial neural network is pre-trained when it is deployed in the vehicle, with pre-training preferably carried out using a supervised or semi-supervised learning method, for which common backpropagation and stochastic gradient methods can be applied.

[0019] The past movements of other road users preferably include at least one of the following: speed, direction, route information, and driving behavior metrics. The past movements of other road users are recorded over arbitrary time periods, meaning they are not necessarily in the immediate or even wider vicinity of the vehicle. The movements of these other road users, recorded in historical journeys, are used primarily for pattern recognition and time series analysis, particularly for anomaly detection. Furthermore, the artificial neural network is used to predict the future movements of other road users in the vicinity of the vehicle.

[0020] The current information about the environment as currently perceived by the sensors, i.e., the real-time data, is preferably obtained from one of the following vehicle sensors: camera unit, radar unit, or lidar unit. However, external data can also be received from data streams via a communication interface, such as weather data, traffic data, or data from a V2X communication interface. Furthermore, contextual data, such as current weather conditions, traffic information, and information about other road users nearby, can be obtained from a central database via vehicle-to-everything (V2X) communication.

[0021] To facilitate reinforcement learning, particularly through the computing module, anomaly detection is performed by an anomaly detection unit. This unit can also be part of the computing module. The anomaly detection uses integrated data as input, specifically patterns derived from the past movements of other road users and evaluated time series analyses, as well as current sensor information from the vehicle and, optionally, additional contextual information, obtained primarily via a communication interface. The anomaly detection can utilize algorithms such as Isolation Forest or One-Class Support Vector Machines, or a Local Outlier Factor algorithm (24-1771).

[0022] 5. Real-time data processing is advantageously performed in the anomaly detection unit, for example using sliding windows and / or streaming analytics, for example Apache Kafka and / or Apache Flink.

[0023] If the anomaly detection unit detects an anomaly, it is taken into account in the reinforcement learning process. Ideally, the system should refrain from adjusting the weights in the attention mechanism when one type of anomaly is detected, while deliberately using other types of anomalies to adjust the weights. An example of the former is consistent false-positive information from multiple vehicle sensors, which is detected through plausibility analysis and can be used to prevent weight adjustments. Other anomalies that require a response, such as a wrong-way driver on a highway, should be deliberately included in the weight adjustment to allow for appropriate action.Another possibility is to consider not only safety-related anomalies, but also comfort anomalies that affect driving comfort in one's own vehicle.

[0024] As described above, the fundamental purpose of reinforcement learning is to adjust the weights of an attention mechanism within the artificial neural network, thereby weighting different pieces of information from the input interface differently. These weights correspond to the so-called policies of reinforcement learning, while a reward function, the so-called "reward," is determined by the quality of the predicted behavior of at least one other road user, or even by the quality of the driving strategy derived from that predicted behavior.Quality criteria such as maintaining a distance from other road users, avoiding crossing a solid lane marking, avoiding driving over a curb, complying with traffic rules, avoiding collisions, avoiding warning signals such as light signals or horn signals from other road users, or other factors can be used.

[0025] The system thus advantageously integrates information about the past movements of other road users as well as real-time data using information about the vehicle's surroundings, which is primarily captured by vehicle sensors, but can also include contextual data from a database. While the past movements of other road users are preferentially processed and interpreted in order to identify trends using time series analyses and determined patterns, thus enabling the optimal determination of a driving strategy, 24-1771

[0026] 6

[0027] Real-time data is used and embedded in the context of the current situation. This allows changing environmental conditions to be taken into account in real time. The attention mechanism of the artificial neural network plays a key role in this. The artificial neural network implicitly models complex dynamics and interactions between other road users and the vehicle itself. However, deviations from typical scenarios are detected by anomaly detection to account for events such as accidents or sudden weather changes in real time, enabling a near-real-time response.

[0028] The advantage lies in using multiple data sources to improve prediction accuracy and responsiveness. These data sources include current sensor data and, in another embodiment, externally generated contextual data, past movements of road users, and the results of anomaly detection to consider environmental changes in real time and to predict and adapt to traffic situations. This integration of data sources aims to enable safer and more efficient navigation by dynamically synthesizing these different data types in real time for the decision-making of an automated system. Real-time data integration allows for the dynamic consideration of changes in environmental conditions, such as weather changes, traffic anomalies, or other factors, in real time.Furthermore, learning can be achieved from new situations or anomalies, and behavior can be adapted based on continuous data input. The modeling of complex interactions takes this into account accordingly.

[0029] In summary, the system uses the following logical components:

[0030] 1. Use of historical data: The system uses time series analysis to interpret historical vehicle trajectory data. This data helps identify patterns and trends that predict the future behavior of autonomous vehicles under similar circumstances.

[0031] 2. Dynamic contextual integration: Context information such as real-time weather conditions, time of day, and unexpected traffic events are dynamically integrated to improve the accuracy of predictions. This ensures that the vehicle's reactions are adapted to current and changing environmental conditions.

[0032] 3. Graph Neural Network (GNN) Architecture: A special GNN architecture is used to model the complex dynamics and interactions between multiple vehicles. This allows the system to make predictions based on real-time data. 24-1771

[0033] 7. Adjust from various sources to ensure that the vehicle responds appropriately to the immediate driving environment.

[0034] 4. Anomaly Detection: The system includes specially developed algorithms for anomaly detection, enabling the system to identify and respond to deviations from normal patterns. These deviations may be due to sudden natural events, accidents, or other anomalies affecting the driving environment.

[0035] 5. Real-time responsiveness: The system features an adaptive response mechanism that adjusts vehicle predictions and behavior in real time as soon as new data is received. This component is crucial for maintaining the safety and efficiency of navigation, especially under unpredictable conditions.

[0036] According to an advantageous embodiment, the first subnetwork is designed to perform the location-based prediction using a graph convolutional network.

[0037] According to another advantageous embodiment, the second subnetwork is configured to perform the time-related prediction using a Long Short-Term Memory network. Alternatively, a Transformer network can be used instead of the Long Short-Term Memory network.

[0038] According to another advantageous embodiment, the reinforcement learning includes a deep Q-learning algorithm.

[0039] According to a further advantageous embodiment, the computing module is designed to link the information about a currently sensor-detected environment of the vehicle by means of data fusion and / or to enrich it with context data for the artificial neural network.

[0040] Data fusion integrates multiple data sources, particularly diverse sensors, for example, using a Kalman filter, a Bayesian network, or Dempster-Shafer theory. Contextualization can be achieved using external data sources and corresponding APIs to obtain real-time updates.

[0041] According to a further advantageous embodiment, the computing module is configured to process the information about past movements of other road users provided at the input data interface for the artificial neural network by means of time series analysis and / or pattern recognition. 24-1771

[0042] 8

[0043] Time series analysis can be performed using methods such as Autoregressive Integrated Moving Average (ARIMA), Long Short-Term Memory (LSTM), and / or time series forecasting using the Prophet algorithm. Pattern recognition can employ clustering techniques like K-means or the so-called "Density-based spatial clustering of applications with noise" (DBSCAN), as well as dimensionality reduction methods such as "Principal component analysis" (PCA). Pattern recognition can be performed in the vehicle's computer module; alternatively, it can be performed in a central processing unit that receives the past movements of other road users, potentially even at significantly earlier times than when the vehicle is currently traversing a traffic scenario.The vehicle then receives the already recognized patterns and results of the time series analysis from the central computing unit, the central server, via a corresponding communication interface.

[0044] According to a further advantageous embodiment, the anomaly detection unit is configured to perform anomaly detection using a spatiotemporal convolutional neural network pre-trained using a combination of one-class support vector machines and isolation forest methods. Another possible method includes a local outlier factor (LOF) algorithm.

[0045] According to another advantageous embodiment, the system further comprises a central server designed to receive information about past movements of other road users and to apply Dynamic Time Warping and / or Multi-Scale Adaptive Multivariate State Estimation techniques for pattern recognition.

[0046] According to a further advantageous embodiment, at least some of the information is location-related, at least regarding past movements of other road users.

[0047] The spatial orientation of the past movements of other road users links their movements to a context and / or a traffic situation. For example, if there is an exit on a motorway, the observed pattern of vehicles remaining on the motorway or using the exit is plausible, whereas on a motorway section without an exit, leaving the motorway would represent an anomaly.

[0048] Another aspect of the invention relates to a method for predicting a

[0049] Traffic scenarios for determining a driving strategy for a vehicle, based on a 24-1771

[0050] 9

[0051] The input data interface provides information, at least about past movements of other road users and about the currently sensor-detected environment of the vehicle, including other road users, to a stored and pre-trained artificial neural network from the class of graph neural networks. A first subnetwork of the artificial neural network makes a location-based prediction based on the provided information, and a second subnetwork makes a time-based prediction of the future behavior of the other road users. Reinforcement learning is performed during vehicle operation.which uses a predictive accuracy regarding the future behavior of other road users and / or a predictive accuracy of a derived driving strategy of the vehicle to adjust weights of a machine learning attention mechanism of the artificial neural network, and which uses the result of an anomaly detection by an anomaly detection unit with respect to the past movements of other road users and / or with respect to the currently sensor-detected environment of the vehicle, wherein input variables of the machine learning attention mechanism are based in particular on the information provided at the input data interface.

[0052] Advantages and preferred further developments of the proposed procedure result from an analogous and substantive transfer of the above statements made in connection with the proposed system.

[0053] Further advantages, features and details will become apparent from the following description, in which - possibly with reference to the drawing - at least one embodiment is described in detail.

[0054] They show:

[0055] Fig. 1: The application of a system according to an embodiment of the invention for predicting a traffic scenario in an exemplary traffic situation.

[0056] Fig. 2: A method for predicting a traffic scenario to determine a driving strategy for a vehicle according to an embodiment of the invention.

[0057] Fig. 1 shows a vehicle 0 in an exemplary traffic situation with other road users. Vehicle 0 is a highly automated motor vehicle with SAE Level 'Three' or higher, meaning that more complex driving maneuvers are possible without human input, in particular longitudinal and lateral control is active, which 24-1771

[0058] 10

[0059] The vehicle receives commands in both dimensions according to an automatically determined driving strategy. In this example traffic situation, vehicle 0 is on a road with oncoming traffic, and vehicles ahead are either remaining in their current lane or turning right. To determine a safe and appropriate automatic driving strategy, vehicle 0 needs to predict the future behavior of other road users at any given time for the near future. This can be important in the current traffic situation, but it becomes even more relevant in intersection situations or on highways with heavy traffic on multiple parallel lanes.Vehicle 0 is equipped with an onboard computing module 1, which receives a variety of sensor data from vehicle 0 via an input data interface 3. This data is then used to make future predictions using an artificial neural network from the graph neural network class. The artificial neural network is structured to model the dynamic interactions between multiple road users. It analyzes very large datasets from vehicle sensors and external sources to predict potential scenarios and suggest optimal navigation routes. All collected data is processed in real time by the artificial neural network.The computing module 1 accesses data from a central server 11, to which numerous other road users have transmitted their movement and status data (including, among other things, the activation of a turn signal or horn) in the past, some of which may be some time ago. A time series analysis and pattern recognition are performed on the central server 11, and the respective results are transmitted to vehicle 0. This allows an anomaly detection unit 9 in vehicle 0 to detect anomalies based on the data from the central server 11 and the data currently being acquired by vehicle 0 about its surroundings. These anomalies are then addressed by a reinforcement learning method implemented at the input data interface 3.This can include detecting a sudden stop by a vehicle ahead due to an accident or recognizing unexpected pedestrian movements. The anomaly detection algorithm specifically utilizes machine learning techniques to improve its detection capabilities over time. Reinforcement learning serves to establish a feedback mechanism that verifies the quality of the vehicle 0's automated driving and adjusts the weights of a machine learning attention mechanism, effectively weighting the various information sources for the artificial neural network in computing module 1. This allows certain real-time data from information about the currently detected environment of vehicle 0 or data from the central server 11 to be considered with different priorities relative to each other. The weights a. t ' of 24-1771

[0060] 11

[0061] Machine learning attention mechanisms are determined for each information source according to the following relationship: where x t vectorially specifies the states at time t, in particular encompassing past movements h t other road users and current sensor data about the vehicle's surroundings. Furthermore, Q(xt, at) describes the Q-value, i.e., the quality score of an action a of vehicle 0, while an action, for example, is the weighting of the information provided by input interface 3 in the machine learning

[0062] The attention mechanism depends on the current state xr. The variables / and j describe running indices. The total weighted input of the artificial neural network is then: where zr describes the aggregated input vector which the artificial neural network uses for prediction at time t.

[0063] The artificial neural network is further characterized by the fact that it comprises a first subnetwork 5 and a second subnetwork 7. While the first subnetwork 5 performs a location-based prediction to estimate the future movement paths of other road users using a graph convolutional network, the second subnetwork 7 serves as a long short-term memory network for the time-based prediction of the respective position and / or other behavior of each other road user in the relevant vicinity of vehicle 0. The prediction of the first subnetwork 5 is described by the following relationship, calculated by the graph convolutional network layer with a local node embedding vector h. v with v from the set V of other road users: where o is an activation function, N(v) describes the set of neighbors with node v, W is a weighting matrix, b is a bias, and / is a local running index. The subsequent time-related prediction is computed using a Long Short-Term Memory (LSTM) network in the second subnetwork 7: 24-1771

[0064] 12

[0065] Fig. 2 shows a method for predicting a traffic scenario to determine a driving strategy for a vehicle 0, wherein information about at least past movements of other road users and about a currently sensor-detected environment of the vehicle 0, including other road users, is provided from an input data interface 3 to a stored and pre-trained artificial neural network from the class of graph neural networks S1, wherein a location-based prediction is made S2 by means of a first subnetwork 5 of the artificial neural network based on the provided information, and wherein a time-based prediction of the future behavior of the other road users is made S3 by means of a second subnetwork 7 of the artificial neural network, and wherein reinforcement learning is carried out via the input data interface 3 during operation of the vehicle 0 S4.which uses a predictive accuracy regarding the future behavior of other road users and / or a predictive accuracy of a derived driving strategy of vehicle 0 to adjust weights of a machine learning attention mechanism of the artificial neural network and which uses the result of an anomaly detection unit 9 for anomaly detection S5 with respect to the past movements of other road users and / or with respect to the currently sensorially detected environment of vehicle 0, wherein input variables of the machine learning attention mechanism are based on the information provided at the input data interface 3.

[0066] Although the invention has been further illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived from them by a person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned as examples are truly only examples and are not to be understood in any way as limiting, for example, the scope of protection, the possible applications, or the configuration of the invention.

Claims

Patent claims 1. System for a vehicle (0), comprising a computing module (1) with a stored and pre-trained artificial neural network from the class of graph neural networks, and comprising an input data interface (3) designed to provide information to the artificial neural network about at least past movements of other road users and about a currently sensor-detected environment of the vehicle (0) including other road users, wherein the artificial neural network comprises a first subnetwork (5) and a second subnetwork (7), wherein the first subnetwork (5) is designed to make a location-based prediction based on the information provided at the input data interface (3), and wherein the second subnetwork (7) is designed to make a time-based prediction of the future behavior of the other road users, and wherein the input data interface (3) is designed toto perform reinforcement learning during operation of the vehicle (0), which uses a quality of prediction about the future behavior of other road users and / or a quality of a derived driving strategy of the vehicle (0) to adjust weights of a machine learning attention mechanism of the artificial neural network and which uses the result of an anomaly detection of an anomaly detection unit (9) with respect to the past movements of other road users and / or with respect to the currently sensorily detected environment of the vehicle (0).

2. System according to claim 1, wherein the first subnetwork (5) is configured to perform the location-based prediction using a graph convolutional network.

3. System according to one of the preceding claims, wherein the second subnetwork (7) is configured to perform the time-related prediction using a Long Short-Term Memory network.

4. System according to any of the preceding claims, wherein the reinforcement learning comprises a Deep-Q Learning algorithm.

5. System according to any one of the preceding claims, wherein the computing module (1) is designed to link the information about a currently sensor-detected environment of the vehicle (0) by means of data fusion and / or to enrich it with context data for the artificial neural network.

6. System according to one of the preceding claims, wherein the computing module (1) is configured to process the information on past movements of other road users provided at the input data interface (3) for the artificial neural network by means of a time series analysis and / or a pattern recognition.

7. System according to any of the preceding claims, wherein the anomaly detection unit (9) is configured to perform the anomaly detection using a spatiotemporal convolutional neural network pre-trained using a combination of one-class support vector machines and isolation forest methods.

8. System according to one of the preceding claims, further comprising a central server (11) designed to receive information about past movements of other road users and to apply Dynamic Time Warping and / or Multi-Scale Adaptive Multivariate State Estimation techniques for pattern recognition.

9. System according to one of the preceding claims, wherein at least some of the information is location-related, at least regarding past movements of other road users.

10. Method for predicting a traffic scenario to determine a driving strategy for a vehicle (0), wherein information about at least past movements of other road users and about a currently sensor-detected environment of the vehicle (0) including other road users is provided from an input data interface (3) to a stored and pre-trained artificial neural network from the class of graph neural networks (S1), wherein a location-based prediction is made based on the provided information by means of a first subnetwork (5) of the artificial neural network (S2) and wherein a time-based prediction is made by means of a second subnetwork (7) of the artificial neural network. of the future behavior of other road users (S3), and wherein during operation of the vehicle (0) reinforcement learning is performed (S4), which uses a quality of prediction about the future behavior of other road users and / or a quality of a derived driving strategy of the vehicle (0) to adjust weights of a machine learning attention mechanism of the artificial neural network and which uses the result of an anomaly detection unit (9) for anomaly detection (S5) regarding the past movements of other road users and / or regarding the currently sensorially detected environment of the vehicle (0).

Images