Methods, devices and computer programs

Abstract

A method for determining information about one or more influencing factors that lead to a determination of a vehicle's trajectory, the method comprising: Training (110) a machine learning model using a reinforcement learning approach, wherein the reinforcement learning approach is based on a reward function, the reward function being based on a plurality of components, and wherein the machine learning model is trained to determine the trajectory of the vehicle, the trajectory comprising a plurality of driving maneuvers; Determine (120), for each driving maneuver of the trajectory, at least one component of the plurality of components of the reward function with a dominant influence on the reward function, wherein the dominant influence is characterized by the fact that it led to the selection of the driving maneuver from a plurality of possible driving maneuvers in the training of the machine learning model; Select (130) for each driving maneuver of the trajectory that includes at least one component as an influencing factor for the selection of the driving maneuver; and Determine (140) the information about the one or more influencing factors based on the influencing factors of the driving maneuvers of the trajectory.

Description

[0001] Exemplary embodiments of the present invention relate to a method, a device and a computer program for determining information about one or more influencing factors, a method, a device and a computer program for training a machine learning model, and to a method, a device and a computer program for a vehicle.

[0002] A key focus in automotive research is the control of autonomous vehicles. In many cases, this involves the use of a machine learning model trained to select a driving maneuver based on the current driving situation, ensuring safe and comfortable vehicle operation. The decision regarding which maneuver to execute is typically influenced by numerous factors, such as the distance to other vehicles, road conditions, other road users, traffic flow, and the vehicle's destination. Based on these factors, and a multitude of other potential influences, the machine learning model generally determines the driving maneuver to be performed.

[0003] In many cases, it is difficult to understand how the machine learning model arrived at its decision. Many models do not offer sufficient means to ensure the unambiguous traceability and documentation of the actions performed by the autonomous vehicle. Such information would be desirable in many cases, however, to evaluate the decisions of the respective vehicle, for example, after accidents. For traceability and error detection, it can be particularly advantageous to consider not only a single step, such as starting with the actuator signal and environment model, but also further information, such as sensor signals that influenced it or the actuator signals preceding it.In some cases, autonomous vehicles may also be required to ensure that every decision they make – especially if it is safety-relevant – is clearly traceable and documented. These decisions can, for example, be documented in databases.

[0004] DE 10 20217 217 443 A1 discloses a method and system for providing training data for machine learning for a control model of an automatic vehicle control system. DE 10 2018 206 805 B3 discloses a method for predicting the future movement of an object using two machine learning systems.

[0005] Therefore, there is a need to create an improved concept that can identify and document the reasoning behind decisions made by machine learning models for controlling vehicles.

[0006] The system and the procedure address this need in accordance with the independent claims.

[0007] Examples of implementation are based on the understanding that a machine learning model can be trained to classify the decisions of another machine learning model. To train this second model, a functionality can be incorporated into the training of the second model that documents why a decision was made. In this case, this can be achieved using a reward function of the second model. The second model can then be trained using a so-called reinforcement learning approach. This approach generally involves comparing different variants and determining, based on the reward function, which variant is preferable. The reward function consists of several components, each representing different objectives for the machine learning model.The functionality for documenting the decision can now determine which component of the reward function was decisive. This information can be documented and subsequently used to train the machine learning model, which can then be used to attribute the decision to the respective component of the reward function. Thus, this machine learning model can now be used to explain, even in real-world driving situations, why another machine learning model (or a similarly implemented model) selected a particular driving maneuver. These implementation examples thus provide a method for ensuring the traceability of autonomous vehicle actions, for example, to meet legal documentation requirements.At least some implementations achieve this by extending the trajectory planning with a classification component that makes the reason for the selected decision unambiguous and traceable, for example, based on an environmental model. At least some implementations further extend the environmental model so that it allows for a traceable dependency on sensor data and previous actuator actions. Furthermore, some implementations document the decision in a database, for example, on a Hadoop cluster (a group of computers running a Hadoop environment).

[0008] Embodiments of the present disclosure provide a method for determining information about one or more influencing factors that lead to the determination of a vehicle's trajectory. The method includes training a machine learning model using a reinforcement learning approach. The reinforcement learning approach is based on a reward function. The reward function is based on a plurality of components. The machine learning model is trained to determine the vehicle's trajectory. The trajectory comprises a plurality of driving maneuvers. The method further includes determining, for each driving maneuver of the trajectory, at least one component of the plurality of components of the reward function that has a dominant influence on the reward function.The dominant influence is characterized by its ability to select a specific driving maneuver from a plurality of possible maneuvers during the training of the machine learning model. The procedure further involves selecting, for each driving maneuver in the trajectory, at least one component as an influencing factor for the maneuver selection. The procedure also includes determining information about these one or more influencing factors based on the influencing factors of the driving maneuvers in the trajectory. By performing this procedure, it is possible to identify, for each driving maneuver, which component of the reward function was decisive for the maneuver selection. This information can then be used to generate another machine learning model that can be used to predict this component (in real-world driving situations).

[0009] For example, the multiple components can represent multiple objectives for the trajectory. The reward function, for instance, can represent a compromise between different objectives, such as a compromise between driving maneuvers with as few sudden speed changes as possible and a time to react to a traffic event, or a compromise between driving speed and comfort. The multiple components of the reward function can represent this compromise. By determining the component of the reward function that had a dominant influence on the choice of driving maneuver, it is therefore possible to ascertain which objective led to the selection of that maneuver.

[0010] In at least some embodiments, the majority of components can be based on several elements from the group consisting of vehicle speed, vehicle safety, vehicle fuel consumption, vehicle ride comfort, the number of lane changes, the number and / or intensity of braking maneuvers, and the number / intensity of acceleration maneuvers. These elements represent, for example, quantifiable values ​​that can be used to implement the reward function in selecting one of the driving maneuvers.

[0011] For example, for each component of the plurality of components, a value can be calculated based on the selected driving maneuver. At least one component can be selected based on how much its value differs from another value calculated based on a different driving maneuver from the plurality of possible driving maneuvers. Large relative changes usually indicate that one component, and therefore one objective, has had a dominant influence on the choice of driving maneuver.

[0012] The information about one or more influencing factors can, for each driving maneuver, include information about a dependency between the chosen maneuver and at least one of the components. This information can then be used to train another machine learning model to derive this dependency even in previously unknown situations.

[0013] In at least some embodiments, position and / or movement data from a plurality of traffic objects can be used as input data for the machine learning model. The method can include determining the sensitivity of the selected driving maneuver and / or the component with a dominant influence on the reward function to changes in the position and / or movement data of the plurality of traffic objects. For example, several variants can be simulated to understand why the machine learning model chose this driving maneuver (and not another), or how the (simulated) environment would have had to be different to produce a different result.

[0014] For example, the majority of traffic objects can include one or more elements from the group consisting of a vehicle, a pedestrian, a traffic sign, a signal, a roadway, and a roadway change. These traffic objects can influence the choice of driving maneuver.

[0015] Embodiments of the present disclosure further provide a (further) method for training a machine learning model. This method can, for example, use the information about the one or more influencing factors generated in the previous method to train the machine learning model so that it is capable of predicting, in arbitrary driving situations (outside of simulation), which component of a reward function, and thus which objective, has had a dominant influence on the selection of a driving maneuver. In other words, the information about the one or more influencing factors can have been determined based on the above method. The method includes training the machine learning model using a supervised learning approach. Position data and / or motion data of a plurality of traffic objects are used as input data.The input data also includes information about a plurality of possible driving maneuvers. Furthermore, the input data includes information about a trajectory determined based on this plurality of possible driving maneuvers. The trajectory encompasses a plurality of driving maneuvers. Information about one or more influencing factors that led to the determination of the trajectory based on this plurality of possible driving maneuvers is used as the desired output value for training the machine learning model. The trajectory is determined by another machine learning model. This second machine learning model is trained using a reinforcement learning approach. A reward function of the reinforcement learning approach is based on a plurality of components.The information about the one or more influencing factors indicates, for each driving maneuver of the trajectory, which at least one component of the plurality of components of the reward function had a dominant influence on the selection of the driving maneuver for determining the trajectory. This machine learning model can now be used, for example, in vehicles or in an evaluation outside the vehicle, to make a statement about which component of the reward function, and thus which objective, led to the selection of the respective driving maneuver from a plurality of driving maneuvers.

[0016] In other words, the machine learning model can be trained to determine, for each driving maneuver out of a plurality of driving maneuvers along a vehicle's trajectory, at least one component of the reward function that had a dominant influence on the choice of maneuver. This facilitates a subsequent analysis of the decisions made by the rest of the machine learning model.

[0017] The method can further include using the machine learning model to determine at least one component of the reward function that had a dominant influence on the choice of driving maneuver. This allows a statement to be made about which component of the reward function (likely) led to the choice of driving maneuver.

[0018] Exemplary embodiments establish a method for a vehicle. The method includes the use of a first machine learning model to determine a trajectory of the vehicle. The trajectory comprises a plurality of driving maneuvers. The first machine learning model is trained using a reinforcement learning approach. A reward function of the reinforcement learning approach is based on a plurality of components. The method further includes the use of a second machine learning model to determine, for each driving maneuver of the plurality of driving maneuvers, at least one component of the reward function that had a dominant influence on the selection of the driving maneuver. The second machine learning model is trained according to the (further) method for training the first machine learning model.By determining at least one component of the reward function, it may be possible to understand how the first machine learning model arrived at its selection of driving maneuvers for the trajectory.

[0019] For example, the process can involve providing, for each driving maneuver (or the majority of driving maneuvers), information about at least one component of the reward function and information about the driving maneuver itself to a server. This allows the server, for instance, to store information about at least one component of the reward function to ensure the traceability of the vehicle's executed driving maneuvers, thus facilitating subsequent analysis of the decisions made at a central location.

[0020] Alternatively or additionally, the procedure can include providing information about at least one component of the reward function via an output module of the vehicle. For example, the owner of the vehicle can be informed why the vehicle selected a particular driving maneuver.

[0021] Further embodiments provide a program with program code for carrying out at least one of the methods, if the program code is executed on a computer, a processor, a control module or a programmable hardware component.

[0022] Embodiments of the present disclosure further provide a device for determining information about one or more influencing factors that lead to the determination of a vehicle's trajectory. The device comprises one or more computing modules and one or more memory modules. The device is configured to train a machine learning model using a reinforcement learning approach. The reinforcement learning approach is based on a reward function. The reward function is based on a plurality of components. The machine learning model is trained to determine the vehicle's trajectory. The trajectory comprises a plurality of driving maneuvers. The device is configured to determine, for each driving maneuver of the trajectory, at least one component of the plurality of components of the reward function that has a dominant influence on the reward function.The dominant influence is characterized by its ability to select a specific driving maneuver from a plurality of possible maneuvers during the training of the machine learning model. The device is designed to identify, for each driving maneuver in the trajectory, at least one component as an influencing factor for the maneuver selection. The device is designed to determine information about these one or more influencing factors based on the influencing factors of the driving maneuvers in the trajectory. The device can then identify, for each driving maneuver, which component of the reward function was decisive for the maneuver selection. This information can then be used to generate another machine learning model that can predict this same component (in real-world driving situations).

[0023] Embodiments of the present disclosure further provide a device for training a machine learning model. The device comprises one or more computing modules and one or more memory modules. The device is configured to train the machine learning model using a supervised learning approach. Position and / or motion data of a plurality of traffic objects are used as input data. Information about a plurality of possible driving maneuvers is also used as input data. Furthermore, information about a trajectory determined based on the plurality of possible driving maneuvers is used as input data. The trajectory comprises a plurality of driving maneuvers. Information about one or more influencing factors that led to the determination of the trajectory based on the plurality of possible driving maneuvers is used as the desired output value for training the machine learning model.The trajectory is determined by another machine learning model. This additional machine learning model is trained using a reinforcement learning approach. A reward function within this approach is based on multiple components. Information about the one or more influencing factors indicates, for each driving maneuver in the trajectory, which at least one component of the reward function had a dominant influence on the selection of that maneuver. This machine learning model can then be used, for example, in vehicles or in an external evaluation, to determine which component of the reward function, and thus which objective, led to the selection of a particular driving maneuver from among a number of possible maneuvers.

[0024] Embodiments of the present disclosure further provide a device for a vehicle. The device comprises one or more computing modules and one or more memory modules. The device is configured to utilize a first machine learning model to determine a trajectory of the vehicle. The trajectory comprises a plurality of driving maneuvers. The first machine learning model is trained using a reinforcement learning approach. A reward function of the reinforcement learning approach is based on a plurality of components. The device is configured to utilize a second machine learning model to determine, for each driving maneuver of the plurality of driving maneuvers, at least one component of the reward function that exerted a dominant influence on the selection of the driving maneuver.The second machine learning model is trained according to the (further) procedure, or by the corresponding device, for training the first machine learning model. By determining at least one component of the reward function, it may be possible to understand how the first machine learning model arrived at its selection of driving maneuvers for the trajectory.

[0025] Examples of implementation are explained in more detail below with reference to the accompanying figures. These show: Fig. 1a shows a flowchart of an embodiment of a method for determining information about one or more influencing factors; Fig. 1b shows a block diagram of an embodiment of a device for determining information about one or more influencing factors; Fig. Figure 2a shows a flowchart of an exemplary implementation of a method for training a machine learning model; Fig. Figure 2b shows a block diagram of an embodiment of a device for training a machine learning model; Fig. 3a shows a flowchart of an embodiment of a method for a vehicle; and Fig. Figure 3b shows a block diagram of an exemplary embodiment of a device for a vehicle.

[0026] Several embodiments are now described in more detail with reference to the accompanying drawings, in which some of these embodiments are illustrated. For the sake of clarity, the thickness dimensions of lines, layers, and / or regions may be exaggerated in the figures.

[0027] Fig. Figure 1a shows a flowchart of an exemplary implementation of a method for determining information about one or more influencing factors that lead to the determination of a vehicle's trajectory. The method includes training a machine learning model using a reinforcement learning approach. The reinforcement learning approach is based on a reward function. The reward function is based on a plurality of components.

[0028] The machine learning model is trained to determine the vehicle's trajectory. The trajectory comprises a plurality of driving maneuvers. The procedure further includes determining, for each driving maneuver of the trajectory, at least one component of the plurality of components of the reward function with a dominant influence on the reward function. The dominant influence is characterized by the fact that it led to the selection of the driving maneuver from a plurality of possible driving maneuvers in the training of the machine learning model. The procedure further includes selecting, for each driving maneuver of the trajectory, at least one component as an influencing factor for the selection of the driving maneuver. The procedure further includes determining, based on the influencing factors of the driving maneuvers of the trajectory, the information about the one or more influencing factors.

[0029] Fig. Figure 1b shows a block diagram of an embodiment of a corresponding device 10 for determining information about one or more influencing factors. The device 10 comprises one or more computing modules 14 and one or more storage modules 16, which are coupled to the one or more computing modules. The device also optionally includes an interface 12 for communication with other devices, in particular for receiving and / or outputting data. The interface 12 can also be coupled to the one or more computing modules 14. The device 10 is configured to perform the method of Fig. 1a. In particular, the one or more computing modules 14 can be configured to execute the procedure of Fig. 1a to execute, for example in conjunction with interface 12 and / or the one or more memory modules 16.

[0030] The following description refers to the procedure of Fig. 1a as well as on the device of Fig. 1b.

[0031] At least some embodiments relate to a method, a device, and a computer program for determining information about the one or more influencing factors that lead to the determination of a vehicle's trajectory. This method, device, and computer program can, for example, be configured to implement a first component of the present disclosure—the training of a machine learning model to provide a trajectory for a vehicle, additionally outputting metadata about the determined trajectory in the form of information about the one or more influencing factors. A trajectory can be described as a combination of multiple driving maneuvers; that is, a trajectory comprises multiple driving maneuvers, such as lane changes, braking, acceleration, turning, etc.

[0032] Such a machine learning model is suitable, for example, for autonomous vehicles with automation capabilities of levels 3 to 5. Machine learning can refer to algorithms and statistical models that computer systems can use to perform a specific task without explicit instructions, instead of relying on models and inference. For example, instead of a rule-based transformation of data, machine learning can use a data transformation derived from an analysis of historical and / or training data. The autonomous vehicle's trajectory planning (implemented by the machine learning model) is extended by a classifier that determines the reason for choosing a particular driving maneuver. This classifier provides information about one or more influencing factors.Such a classifier can be used, for example, to explain why the machine learning model (during training) selected a particular driving maneuver for the trajectory. For instance, a lane change (selected driving maneuver) might have been chosen due to a slow-moving vehicle in front (influencing factor), braking (selected driving maneuver) due to vulnerable road users at the roadside (influencing factor), and so on. The classifier can be determined by training a machine learning model for trajectory planning using reinforcement learning, similar to known approaches for training machine learning models for trajectory planning. In other words, the process involves training the machine learning model using a reinforcement learning approach. The machine learning model is trained to determine the vehicle's trajectory.The basis for this is, for example, a simulation on a server, which is fed with artificial and / or real data. In other words, training the machine learning model can involve simulating a multitude of traffic situations using artificially generated and / or real data. Each traffic situation can be based on a multitude of traffic objects. Position data (i.e., information about a relative or absolute static position) and / or motion data (i.e., information about relative or absolute position changes) of a multitude of traffic objects can be used as input data for the machine learning model. The position data and / or motion data of the multitude of traffic objects can be used to model the environment of a vehicle during the training of the machine learning model.The machine learning model can be trained to determine the vehicle's trajectory by considering the plurality of traffic objects. These plurality of traffic objects can include one or more elements from the group consisting of a vehicle, a pedestrian, a traffic sign, a signal (such as a traffic light or an adaptive traffic sign), a roadway, and a roadway change. In other words, the plurality of traffic objects can include one or more vehicles, one or more pedestrians, one or more traffic signs, one or more signal systems, one or more roadways and / or lanes, and / or one or more roadway changes.

[0033] In some implementation examples, the machine learning model is trained using a reinforcement learning approach. Reinforcement learning is a group of machine learning algorithms. In reinforcement learning, one or more software actors (so-called "software agents") are trained to perform actions in a simulated environment. Based on these actions, a reward is calculated according to a reward function. In other words, the reinforcement learning approach is based on a reward function. This reward function is based on a plurality of components. For example, the reward function can be designed to evaluate different objectives. These objectives can be represented by the multiple components of the reward function.In other words, multiple components can represent multiple trajectory objectives. These components can be based on several elements of the group, including vehicle speed, vehicle safety, vehicle fuel consumption, vehicle ride comfort, the number of lane changes, the number and / or intensity of braking maneuvers, and the number / intensity of acceleration maneuvers. The objectives can be defined accordingly. In other words, reinforcement learning can utilize a cost functional that can consider different criteria regarding trajectory planning, such as higher speed, higher safety, higher CO2 efficiency, and comfort (e.g., through reduced jerkiness).The reward function can, for example, represent one or more of the following objectives: achieving the highest possible (average) speed of the vehicle, achieving the highest possible safety of the vehicle, achieving the lowest possible fuel consumption of the vehicle, achieving the highest possible driving comfort of the vehicle, achieving the fewest possible lane changes of the vehicle, achieving the fewest possible braking maneuvers, achieving the lowest possible intensity of braking maneuvers, achieving the fewest possible acceleration maneuvers, and achieving the lowest possible intensity of acceleration maneuvers.Reinforcement learning is based on training one or more software agents to select actions in such a way as to increase the cumulative reward, resulting in software agents that become better at the task they are given (which can be demonstrated by increasing rewards based on the reward function).

[0034] Unlike other reinforcement learning-based approaches, this reinforcement learning in the simulation is enhanced by metadata, analyzing why the cost functional improves with the proposed maneuver. For example, the following relationships can be identified: - A higher speed is achieved because there is no vehicle in front. - Increased safety is achieved because a greater safety distance to the vehicle in front is chosen. - Higher CO2 efficiency is achieved because a higher acceleration is chosen to reach a traffic light.

[0035] This can be achieved by determining, for each driving maneuver in the trajectory, which component of the reward function, and thus also which objective among the plurality of objectives, led to the selection of the respective driving maneuver. In other words, the procedure involves determining, for each driving maneuver in the trajectory, at least one component of the plurality of components of the reward function with a dominant influence on the reward function. More precisely, the procedure can involve determining, for each driving maneuver in the trajectory, at least one component of the plurality of components of the reward function that led to a dominant influence on the reward function in the selection of the driving maneuver for the trajectory. The dominant influence is characterized by the fact that it led to the selection of the driving maneuver from a plurality of possible driving maneuvers in the training of the machine learning model.Two or more components can certainly exert a dominant influence, for example, components with a linear relationship, such as between braking / acceleration and comfort, or between distance and safety. At least one component among the plurality of components can be selected, for instance, based on how much its value changes when a particular driving maneuver is chosen compared to other driving maneuvers. For example, a component value can be calculated for each component based on the selected driving maneuver. Combining these components (along with a weighting function) yields the value of the reward function.At least one component can be selected based on how much its value changes compared to another value of the component, calculated based on a different driving maneuver from the majority of possible driving maneuvers. In other words, the at least one component that changes most (positively) in the selected driving maneuver compared to one or more other driving maneuvers (such as the second-best driving maneuver (as judged by the reward function) or a majority of other driving maneuvers) can be selected. This at least one component can then be selected as the influencing factor for this driving maneuver.Furthermore, the procedure can include determining at least one traffic object that is taken into account by at least one component of the reward function (i.e., that is responsible for a significant proportion (e.g., at least 10% of the value of the respective component of the reward function) of the value of the component of the reward function).

[0036] The procedure further includes determining information about one or more influencing factors based on the influencing factors of the driving maneuvers of the trajectory. The information about the one or more influencing factors corresponds to the metadata mentioned previously. The determined dependencies (influencing factors) can, for example, be stored as a key-value pair, such as part of the information about the one or more influencing factors. Thus, the information about the one or more influencing factors for each driving maneuver can include information about a dependency between the selected driving maneuver and at least one component (e.g., as a key-value pair).Furthermore, information about the one or more influencing factors for each driving maneuver can provide information about a dependency between the chosen driving maneuver, which includes at least one component of the reward function, and at least one traffic object that is considered by the at least one component of the reward function. This information can now be incorporated into the method, the device, and the computer program of the [system / device]. Fig. 2a and / or 2b can be used to train the (further) machine learning model.

[0037] In some implementations, the sensitivity of the decision can also be determined. In other words, for each selected driving maneuver, it can be determined how the decision would have differed had the situation been (slightly) different. For this purpose, position data (i.e., information about a relative or absolute static position) and / or motion data (i.e., information about relative or absolute position changes) of a plurality of traffic objects can be used as input data for the machine learning model. The method can include determining the sensitivity of the selected driving maneuver and / or the component with a dominant influence on the reward function to a change in the position data and / or motion data of the plurality of traffic objects.To this end, each selection of a driving maneuver based on a plurality of variants of position and / or motion data can be simulated to determine the required change for the machine learning model to select a different driving maneuver. Furthermore, information about the one or more influencing factors for each driving maneuver can include information about the sensitivity of the selected maneuver and / or the component with a dominant influence on the reward function to a change in the position and / or motion data of the plurality of traffic objects.

[0038] The following discussion focuses on machine learning models in general, but also specifically on the machine learning model of Fig. 1a / 1b and the machine learning models of Fig. Reference is made to sections 2a to 3b. Machine learning algorithms are typically based on a machine learning model. In other words, the term "machine learning algorithm" can refer to a set of instructions that can be used to create, train, or use a machine learning model. The term "machine learning model" can refer to a data structure and / or a set of rules that represents the learned knowledge (e.g., based on the training performed by the machine learning algorithm). In some examples, the use of a machine learning algorithm may imply the use of an underlying machine learning model (or multiple underlying machine learning models). The use of a machine learning model may imply that the machine learning model and / or the data structure / set of rules that constitutes the machine learning model is / are trained by a machine learning algorithm.

[0039] For example, the machine learning model could be an artificial neural network (ANN). ANNs are systems inspired by biological neural networks, such as those found in a retina or brain. ANNs comprise a plurality of interconnected nodes and a plurality of connections, called edges, between the nodes. There are typically three types of nodes: input nodes, which receive input values; hidden nodes, which are connected (only) to other nodes; and output nodes, which provide output values. Each node can represent an artificial neuron. Each edge can transmit information from one node to another. The output of a node can be defined as a (nonlinear) function of its inputs (e.g., the sum of its inputs). The inputs of a node can be used in the function based on a "weight" assigned to the edge or the node providing the input.The weight of nodes and / or edges can be adjusted during the learning process. In other words, training an artificial neural network can involve adjusting the weights of the nodes and / or edges of the artificial neural network, i.e., to achieve a desired output for a given input.

[0040] Alternatively, the machine learning model can be a support vector machine, a random forest model, or a gradient boosting model. Support vector machines (i.e., support vector networks) are supervised learning models with associated learning algorithms that can be used to analyze data (e.g., in classification or regression analysis). Support vector machines can be trained by providing an input with a plurality of training input values ​​belonging to one of two categories. The support vector machine can be trained to assign a new input value to one of the two categories. Alternatively, the machine learning model can be a Bayesian network, which is a probabilistic directed acyclic graphical model. A Bayesian network can represent a set of random variables and their conditional dependencies using a directed acyclic graph.Alternatively, the machine learning model can be based on a genetic algorithm, which is a search algorithm and heuristic technique that imitates the process of natural selection.

[0041] Interface 12 (and / or corresponding interfaces 22; 32 of devices 20; 30 of the Fig. 2b and / or 3b) can, for example, correspond to one or more inputs and / or one or more outputs for receiving and / or transmitting information, such as in digital bit values, based on a code, within a module, between modules, or between modules of different entities.

[0042] In exemplary embodiments, the one or more computing modules 14 (and / or corresponding computing modules 24; 34 of the devices 20; 30 of the Fig. 2b and / or 3b) correspond to one or more arbitrary controllers or processors, or a programmable hardware component. For example, the one or more computing modules 14 can also be implemented as software programmed for a corresponding hardware component. Alternatively, the one or more computing modules 14 can be implemented as programmable hardware with appropriately adapted software. Any processor, such as digital signal processors (DSPs), can be used. The examples are not limited to a specific type of processor. Any processor, or even multiple processors, are conceivable for implementing the one or more computing modules 14.

[0043] The one or more memory modules 16 (and / or corresponding memory modules 26; 36 of the devices 20; 30 of the Fig. 2b and / or 3b) may, for example, include at least one element of the group consisting of computer-readable storage medium, magnetic storage medium, optical storage medium, hard disk, flash memory, floppy disk, random access memory, programmable read only memory (PROM), erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), and network storage.

[0044] Fig. Figure 2a shows a flowchart of an exemplary implementation of a method for training a machine learning model. This machine learning model is a machine learning model that, for example, uses the method of Fig. 1a. Information obtained about one or more influencing factors for training the machine learning model is used to ultimately estimate, for any trajectories provided by a machine learning model based on the same (or similar) reward function, the component of the reward function that had a dominant influence on the selection of a driving maneuver for the trajectory. The procedure involves training the machine learning model using a supervised learning approach. The input data for training the machine learning model are: - Position data and / or movement data of a multiple traffic objects, - Information about a plurality of possible driving maneuvers, and - Information about a trajectory determined based on a plurality of possible driving maneuvers is used. The trajectory comprises a plurality of driving maneuvers. Information about one or more influencing factors that led to the determination of the trajectory based on the plurality of possible driving maneuvers is used as the desired output value for training the machine learning model (for example, information about the one or more influencing factors that are determined by the method, device, or computer program of Fig. 1a / 1b are provided. The trajectory is determined by another machine learning model. This additional machine learning model is trained using a reinforcement learning approach. A reward function of the reinforcement learning approach is based on a plurality of components. The information about the one or more influencing factors indicates, for each driving maneuver of the trajectory, which at least one component of the plurality of components of the reward function had a dominant influence on the selection of the driving maneuver for determining the trajectory.

[0045] Fig. Figure 2b shows a block diagram of an exemplary embodiment of a corresponding device for training the machine learning model. The device 20 comprises one or more computing modules 24 and one or more memory modules 26, which are coupled to the one or more computing modules. The device also optionally includes an interface 22 for communication with other devices, in particular for receiving and / or outputting data. The interface 22 can also be coupled to the one or more computing modules 24. The device 20 is configured to perform the method of Fig. 2a. In particular, the one or more computing modules 24 can be configured to implement the procedure of Fig. 1a to be executed, for example in conjunction with the interface 22 and / or the one or more memory modules 26.

[0046] The following description refers to the procedure of Fig. 2a as well as the corresponding device of Fig. 2b.

[0047] The method, the apparatus and the corresponding computer program of the Fig. Sections 2a to 2c now refer to the training of a machine learning model that can be used to determine (predict) which component of a reward function led to the determination of a specific trajectory, based on possible driving maneuvers and position and / or movement data of a plurality of traffic objects. The machine learning model can, for example, be a neural network that may include at least one hidden layer and a soft-max layer (a soft-max function, or normalized exponential function) as its output. This machine learning model can, for example, be trained as a classifier. In other words, a (second) classifier can be trained based on the previous results. This is implemented as a supervised learning algorithm.

[0048] As previously mentioned, machine learning can refer to algorithms and statistical models that computer systems can use to perform a specific task without explicit instructions, rather than relying on models and inference. For example, instead of a rule-based transformation of data, machine learning can use a transformation of data derived from an analysis of historical and / or training data. For instance, the content of images can be analyzed using a machine learning model or algorithm. To enable the machine learning model to analyze the content of an image, it can be trained using training images as input and training content information as output.By training the machine learning model with a large number of training images and / or training sequences (e.g., words or sentences) and associated training content information (e.g., labels or annotations), the machine learning model "learns" to recognize the content of the images, so that the content of images not included in the training data can be recognized using the machine learning model. The same principle can also be used for other types of data: By training a machine learning model using training sensor data and a desired output, the machine learning model "learns" a conversion between the data and the output, which can be used to provide an output based on non-training sensor data provided to the machine learning model. The provided data (e.g.,Sensor data, metadata, and / or image data can be preprocessed to obtain a feature vector, which is used as input for the machine learning model. Such an approach is used in the training of the machine learning model of [missing information]. Fig. 2a / 2b is being followed.

[0049] Machine learning models can be trained using training input data. The examples above demonstrate how to implement the Fig. 2a / 2b describes a training method called "supervised learning." In supervised learning, the machine learning model is trained using a plurality of training samples, where each training sample can include a plurality of input values ​​and a plurality of desired output values; that is, each input value is associated with a desired output value. By providing both input data values ​​and desired output values, the machine learning model "learns" which output value to provide based on an input value that is similar to the input values ​​provided during training. Supervised learning can be based on a supervised learning algorithm (for example, a classification algorithm, a regression algorithm, or a similarity algorithm).Classification algorithms can be used when the outputs are limited to a finite set of values ​​(categorical variables), i.e., the input is classified as one of the limited set of values. In practical examples, the machine learning model can be trained using a classification algorithm.

[0050] The following input data can be used as input values ​​for training the machine learning model: - the simulated dynamic vehicle environment, in particular the position, speed and class of the surrounding objects (such as the position data and / or movement data of the majority of traffic objects) - the possible maneuvers, for example lane changes, stopping at the stop line (such as information about the majority of possible driving maneuvers) - the classic trajectory planning as a series of edges with gradients (for example, information about the trajectory determined based on the majority of possible driving maneuvers).

[0051] The training data (the input data) can be the same data used to train the reinforcement learning system. Fig. 1a / 1b were used. This can also result in the Ground Truth (literal translation "ground truth"), since the scenarios are known in advance.

[0052] The desired output value of the training is information about the one or more influencing factors that led to the determination of the trajectory based on the plurality of possible driving maneuvers. This information indicates, for each driving maneuver in the trajectory, which at least one component of the plurality of components of the reward function had a dominant influence on the selection of the driving maneuver for determining the trajectory. In other words, the information about the one or more influencing factors can specify a relationship between the chosen driving maneuver and the at least one component of the reward function for each driving maneuver in a trajectory.This makes it possible to train the machine learning model to identify the component of the reward function that exerted a dominant influence on the reward function and thus led to the selection of the driving maneuver. The machine learning model can therefore be trained to determine, for each driving maneuver within a plurality of driving maneuvers along a vehicle's trajectory, at least one component of the reward function that exerted a dominant influence on the selection of the driving maneuver.

[0053] Furthermore, information about the one or more influencing factors for each driving maneuver of a trajectory can establish a relationship between the chosen driving maneuver, which includes at least one component of the reward function, and at least one traffic object considered by that component. This allows the machine learning model to be trained to identify the traffic object that, in relation to the component of the reward function, led to the selection of the driving maneuver. In other words, the classifier can identify the environmental object(s) (traffic object(s)) that led to the improvement of the respective cost functional.The machine learning model can therefore be trained to determine, for each driving maneuver of a plurality of driving maneuvers of a vehicle's trajectory, at least one component of the reward function that has had a dominant influence on the selection of the driving maneuver, and to determine at least one traffic object that is taken into account by the at least one component of the reward function.

[0054] In an advantageous implementation, the classifier is analyzed with regard to the sensitivity of the input signals and the maneuver. For example, it can investigate whether the maneuver would have been decided differently if the pedestrian had been traveling 1 m / s faster. Information about the one or more influencing factors for each driving maneuver can include information about the sensitivity of the chosen driving maneuver and / or the component with a dominant influence on the reward function to a change in the position data and / or movement data of the majority of traffic objects.The machine learning model can be trained to determine, for each driving maneuver, the sensitivity of the selected driving maneuver and / or the component with a dominant influence on the reward function to a change in the position data and / or movement data of the majority of traffic objects (based on the previously specified input data, and using the information about the sensitivity of the selected driving maneuver and / or the component with a dominant influence on the reward function to a change in the position data and / or movement data of the majority of traffic objects as the desired output value).The machine learning model can therefore be trained to determine, for each driving maneuver of the majority of driving maneuvers in a vehicle's trajectory, the sensitivity of the selected driving maneuver and / or the component with a dominant influence on the reward function to a change in the position data and / or movement data of the majority of traffic objects.

[0055] Furthermore, the procedure, as further explained in Fig. As shown in Figure 2a, a benefit of the machine learning model 220 is to determine at least one component of the reward function that had a dominant influence on the choice of driving maneuver. For this purpose, instead of the training input data, "real" input data, i.e., input data from a vehicle, can be applied to the inputs of the machine learning model, and the output of the machine learning model can display at least one component of the reward function, and optionally the corresponding traffic objects and the sensitivity.

[0056] Fig. Figure 3a shows a flowchart of an embodiment of a method for a vehicle 300. The method includes a benefit 310 of a first machine learning model to determine a trajectory of the vehicle. The machine learning model can be similar to the machine learning model of Fig. 1a / 1b can be implemented, whereby the determination of information about the one or more influencing factors can be omitted. In particular, the first machine learning model can be based on the same or a similar reward function as the machine learning model of Fig. 1a / 1b. The trajectory comprises a plurality of driving maneuvers. The first machine learning model is trained using a reinforcement learning approach. The reward function of the reinforcement learning approach is based on a plurality of components. The procedure further includes the use of a second machine learning model to determine, for each driving maneuver of the plurality of driving maneuvers, at least one component of the reward function that had a dominant influence on the selection of the driving maneuver. The second machine learning model is designed according to the procedure, the device, and / or the computer program of the Fig. 2a / 2b trained. In some embodiments, the method can be carried out by the vehicle, for example by a device for the vehicle, as described in Fig. Figure 3b shows that, alternatively, the procedure can be executed at least partially outside the vehicle, for example by a server. In this case, using the first machine learning model can include obtaining information about the trajectory, where the trajectory was determined by the first machine learning model. In this case, the trajectory can be determined and stored by a vehicle and retrieved from the vehicle by the entity outside the vehicle.

[0057] Fig. Figure 3b shows a block diagram of an embodiment of a corresponding device 30 for the vehicle 300. Fig. Figure 3b further shows the vehicle 300 comprising the device 30. The device 30 comprises one or more computing modules 34 and one or more storage modules 36, which are coupled to the one or more computing modules. The device also optionally includes an interface 32 for communication with other devices, in particular for receiving and / or outputting data. The interface 32 can also be coupled to the one or more computing modules 34. The device 30 is configured to perform the method of Fig. 3a. In particular, the one or more computing modules 34 can be configured to perform the procedure of Fig. 3a to be executed, for example in conjunction with the interface 32 and / or the one or more memory modules 36.

[0058] The following description refers to both the procedure of Fig. 3a as well as the device and the vehicle of the Fig. 3b.

[0059] The method, the apparatus and the corresponding computer program of the Fig. 3a / 3b now refer to the use of the machine learning model, which is implemented through the method, device and / or computer program of the Fig. 2a / 2b was generated. Two components are provided: Firstly, an initial machine learning model is used to determine the trajectory. This initial machine learning model can, for example, be based on the machine learning model of the Fig. 1a / 1b. Alternatively, the first machine learning model can correspond to a machine learning model based on the reward function of the machine learning model of Fig. 1a / 1b, or based on a similar reward function. Position data (i.e., information about a relative or absolute static position) and / or motion data (i.e., information about relative or absolute position changes) of multiple traffic objects can be used as input data for the first machine learning model. Based on the position and / or motion data, the machine learning model can be trained to determine the trajectory using the majority of driving maneuvers. The machine learning model is trained using the reinforcement learning approach with the help of the reward function. Consequently, the machine learning model can be trained to determine the trajectory in accordance with the reward function.This can be ensured by training the machine learning model to determine trajectories that achieve particularly good scores in the evaluation by the reward function.

[0060] Furthermore, the second machine learning model is used to determine, for each of the majority of driving maneuvers, at least one component of the reward function that had a dominant influence on the selection of the driving maneuver. This second machine learning model is now implemented according to the method, the device, and / or the computer program of the Fig. 2a / 2b is generated. This machine learning model, for example, extends the autonomous production vehicle to include the classification component, enabling it to uniquely determine at any given time the reason for a maneuver decision.

[0061] The maneuver decision, including a timestamp and dependencies, can be transmitted via a communication unit to the backend of the vehicle manufacturer or ODM (On-Demand Mobility) provider. In other words, the process can involve providing (e.g., transmitting) the trajectory to a server. Furthermore, the process can involve providing (e.g., transmitting) information about at least one component of the reward function and information about the maneuver to the server for each or the majority of driving maneuvers. Additionally, the environmental model that led to the generation of the trajectory can be transmitted. This environmental model can be extended to reflect a traceable dependency on sensor data and previous actuator actions.In exemplary implementations, the position and / or movement data of the majority of traffic objects can be transmitted to the server. This data can be correlated with the transmitted trajectory, making it clear which position / movement data led to the selection of a driving maneuver. Furthermore, the position and / or movement data can be transmitted over a period of time, and not just at a single point in time, so that it is clear how the positions and movements of the road users have developed. In an advantageous embodiment, the sensitivity of the decision is determined in the backend. This can be implemented similarly to how it is done in connection with [the following]. Fig. 1a to 2b were described.

[0062] In some embodiments, the decision can be presented to the technically interested driver. Consequently, the method can include providing information about at least one component of the reward function via an output module of the vehicle. This output module can, for example, correspond to a screen in the vehicle. The information about at least one component of the reward function can be displayed, for example, by showing for each driving maneuver what objective the respective component of the reward function represents, such as "the driving maneuver was performed to improve comfort, safety, fuel consumption, or similar aspects." In some embodiments, at least one traffic object from the plurality of traffic objects that has influenced the component can also be displayed.For example, it can be reported that a driving maneuver was carried out because a vehicle ahead is driving too slowly, or because a pedestrian was detected at the edge of the road.

[0063] Although some aspects have been described in connection with a system or device, it is understood that these aspects also represent a description of the corresponding process, so that a functional feature, block, or component of a device is also to be understood as a corresponding process step or as a feature of a process step. Similarly, aspects described in connection with or as a process step also represent a description of a corresponding functional feature, block, detail, or feature of a corresponding system.

[0064] Another embodiment is a computer program for carrying out at least one of the methods described above, provided the computer program runs on a computer, a processor, or a programmable hardware component. Another embodiment is a digital storage medium that is machine- or computer-readable and that contains electronically readable control signals which can interact with a programmable hardware component to execute one of the methods described above.

[0065] Depending on specific implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be carried out using a digital storage medium, for example a floppy disk, DVD, Blu-ray disc, CD, ROM, PROM, EPROM, EEPROM or FLASH memory, hard disk or other magnetic or optical storage medium, on which electronically readable control signals are stored that can interact with, or interact with, a programmable hardware component in such a way that the respective method is carried out.

[0066] In general, embodiments of the present invention can be implemented as a program, firmware, computer program, or computer program product with program code or as data, wherein the program code or data is / are effective in carrying out one of the methods when the program runs on a processor or a programmable hardware component. The program code or data can, for example, also be stored on a machine-readable medium or data carrier. The program code or data can be in the form of source code, machine code, bytecode, or other intermediate code, among others.

[0067] A program according to one embodiment can implement one of the methods during its execution, for example, by reading memory locations or writing data to them, thereby potentially triggering switching operations or other processes in transistor structures, amplifier structures, or other electrical, optical, magnetic, or otherwise operating components. Similarly, by reading a memory location, a program can acquire, determine, or measure data, values, sensor values, or other information. Therefore, by reading from one or more memory locations, a program can acquire, determine, or measure quantities, values, measured values, and other information, and by writing to one or more memory locations, it can initiate, trigger, or execute an action, as well as control other devices, machines, and components. Reference symbol list 10 Device for determining information about one or more influencing factors 12 Interface 14 Calculation module 16 memory modules 20 Device for training a machine learning model 22 Interface 24 Calculation module 26 memory module 30 Device for a vehicle 32 interface 34 Calculation module 36 memory module 110 Training a machine learning model 120 Determine at least one component of a reward function 130 Selecting at least one component 140 Determining information about one or more influencing factors 210 Training a machine learning model 220 Benefits of the machine learning model 300 vehicles 310 Benefits of a first machine learning model 320 Benefits of a second machine learning model 330 Providing information about at least one component of a reward function for a server 340 Providing information about at least one component of a reward function via an output module

Claims

[1] A method for determining information about one or more influencing factors that lead to a determination of a vehicle's trajectory, the method comprising: Training (110) a machine learning model using a reinforcement learning approach, wherein the reinforcement learning approach is based on a reward function, the reward function being based on a plurality of components, and wherein the machine learning model is trained to determine the trajectory of the vehicle, the trajectory comprising a plurality of driving maneuvers; Determine (120), for each driving maneuver of the trajectory, at least one component of the plurality of components of the reward function with a dominant influence on the reward function, wherein the dominant influence is characterized by the fact that it led to the selection of the driving maneuver from a plurality of possible driving maneuvers in the training of the machine learning model; Select (130) for each driving maneuver of the trajectory that includes at least one component as an influencing factor for the selection of the driving maneuver; and Determine (140) the information about the one or more influencing factors based on the influencing factors of the driving maneuvers of the trajectory. [2] The method according to claim 1, wherein the plurality of components represents a plurality of objectives for the trajectory. [3] The method according to one of claims 1 or 2, wherein the plurality of components are based on several elements of the group consisting of a vehicle speed, a vehicle safety, a vehicle consumption, a vehicle ride comfort, a number of vehicle lane changes, a number and / or intensity of vehicle braking maneuvers, and a number / intensity of vehicle acceleration maneuvers. [4] The method according to one of claims 1 to 3, wherein for each component of the plurality of components a value of the component is calculated based on the selected driving maneuver, wherein the at least one component is selected based on how much the value of the component changes compared to another value of the component calculated based on another driving maneuver of the plurality of possible driving maneuvers. [5] The method according to any one of claims 1 to 4, wherein the information about the one or more influencing factors for each driving maneuver includes information about a dependency between the selected driving maneuver and the at least one component. [6] The method according to any one of claims 1 to 5, wherein position data and / or movement data of a plurality of traffic objects are used as input data for the machine learning model, the method comprising determining a sensitivity of the selected driving maneuver and / or component with a dominant influence on the reward function to a change in the position data and / or movement data of the plurality of traffic objects. [7] A method for training a machine learning model, the method encompassing: Training (210) the machine learning model using a supervised learning approach, using the following as input data: - Position data and / or movement data of a multiple traffic objects, - Information about a plurality of possible driving maneuvers, and - Information about a trajectory determined based on the plurality of possible driving maneuvers, wherein the trajectory comprises a plurality of driving maneuvers, wherein information about one or more influencing factors that led to a determination of the trajectory based on the plurality of possible driving maneuvers is used as the desired output value for training the machine learning model, wherein the trajectory is determined by another machine learning model, wherein the further machine learning model is trained using a reinforcement learning approach, wherein a reward function of the reinforcement learning approach is based on a plurality of components, where the information about the one or more influencing factors for each driving maneuver of the trajectory indicates which at least one component of the plurality of components of the reward function has had a dominant influence on the selection of the driving maneuver for determining the trajectory. [8] The method according to claim 7, wherein the machine learning model is configured to determine, for each driving maneuver of a plurality of driving maneuvers of a trajectory of a vehicle, at least one component of the reward function that has had a dominant influence on the selection of the driving maneuver. [9] The method according to one of claims 7 or 8, wherein the information about the one or more influencing factors was determined based on the method according to one of claims 1 to 7. [10] A method for a vehicle comprising: Use (310) of a first machine learning model to determine a trajectory of the vehicle, wherein the trajectory comprises a plurality of driving maneuvers, wherein the first machine learning model is trained using a reinforcement learning approach, wherein a reward function of the reinforcement learning approach is based on a plurality of components; and Use (320) of a second machine learning model to determine, for each driving maneuver of the plurality of driving maneuvers, at least one component of the reward function that has had a dominant influence on the selection of the driving maneuver, wherein the second machine learning model is trained according to one of claims 7 to 9. [11] The method according to claim 10, wherein the method comprises providing (330) for each driving maneuver of the plurality of driving maneuvers, information about the at least one component of the reward function and information about the driving maneuver to a server, and / or wherein the method comprises providing (340) information about at least one component of the reward function via an output module of the vehicle. [12] A program comprising program code for performing at least one of the methods according to any one of claims 1 to 11, when the program code is executed on a computer, a processor, a control module or a programmable hardware component. [13] A device (10) for determining information about one or more influencing factors that lead to a determination of a vehicle's trajectory, the device comprising one or more computing modules (14) and one or more storage modules (16), wherein the device is configured to perform the method according to any one of claims 1 to 6. [14] A device (20) for training a machine learning model, the device comprising one or more computing modules (24) and one or more memory modules (26), wherein the device is configured to perform the method according to any one of claims 7 to 9. [15] A device (30) for a vehicle (300), the device comprising one or more computing modules (34) and one or more storage modules (36), wherein the device is configured to perform the method according to one of claims 10 or 11.

Images