Method, computer program product and motor vehicle for training at least one algorithm for a control device of a motor vehicle

Abstract

A method for training at least one algorithm for a control device of a motor vehicle, the control device being used to implement an autonomous driving function, wherein the algorithm is trained by a self-learning neural network, the method comprising the following steps: a) providing a computer program product module for the autonomous driving function, wherein the computer program product module contains the algorithm to be trained and the self-learning neural network; b) providing at least one indicator and a reward function; c) embedding the computer program product module in a simulation environment to simulate at least one important relevant traffic situation, and training the self-learning neural network by simulating emergency scenarios and determining the indicator (M) until the self-learning neural network is satisfied. until a first quality standard (G1) is met; d) embedding the trained computer program product module into the control device of the motor vehicle to simulate important relevant traffic situations, and training the self-learning neural network by simulating emergency scenarios and determining the indicator (M) until a second quality standard (G2) is met, wherein, e), (i) if the indicator (M) in step d) is worse than the first quality standard (G1), the method continues from step c), or (ii) if the indicator (M) in step d) is better than the first quality standard (G1) but worse than the second quality standard (G2), the method continues from step d).

Description

Technical Field

[0001] A method for training at least one algorithm for a control unit of a motor vehicle, a computer program product, and a motor vehicle are described herein, wherein the control unit is used to implement an autonomous driving function while intervening in an aggregate of the motor vehicle. Background Art

[0002] Methods, computer program products and motor vehicles of the type mentioned at the outset are known in the prior art. In the past few years, the first autonomous motor vehicles have reached the level of series production. Autonomous motor vehicles must react independently to unknown traffic situations with maximum safety based on various predefined requirements (such as the destination and compliance with common traffic regulations). Since traffic reality is highly complex due to the unpredictability of the behavior of traffic participants, it is almost impossible to program corresponding control devices of motor vehicles using conventional methods and rules.

[0003] On the other hand, it is known to use machine learning or artificial intelligence methods to develop algorithms that, on the one hand, can react more appropriately to critical traffic situations than conventional algorithms. On the other hand, artificial intelligence can be used to further develop algorithms in everyday life through continuous learning.

[0004] DE 10 2015 007 493 A1 discloses a method for training a decision algorithm based on machine learning for use in a control device of a motor vehicle, wherein the decision algorithm determines output data to be considered for controlling the operation of the motor vehicle and a reliability value describing the reliability of the output data based on input data describing a current operating state and / or a current driving condition, and is trained based on a basic training data set before use in a motor vehicle, wherein, in the event that the reliability value is below a threshold value, the input data on which the determination of the output data assigned to the reliability value is based are stored as evaluation input data and are displayed to a human evaluator at a later time, and then, evaluation output data corresponding to the output data is received through the evaluator's operational input, and the decision algorithm is trained based on an improved training data set formed by the evaluation input data and the assigned evaluation output data.

[0005] Hallerbach, Xia, Eberle & Koester (April 3, 2018) "Simulation-based Identification of Critical Scenarios for Cooperative and Automated Vehicles" (SAE 2018-01-1066) describes a series of auxiliary tools for the development of emergency scenarios based on simulation. The process includes the simulation of the dynamic behavior of the motor vehicle as well as the simulation of the traffic situation and the simulation of the cooperative behavior of virtual traffic participants. The identification of emergency situations is based on indicators (e.g., safety indicators or traffic quality indicators).

[0006] A disadvantage of the known method is that the development of algorithms for autonomously driven motor vehicles up to series production level is complex and time-consuming.

[0007] The object is therefore to expand a method, a computer program product and a motor vehicle of the type mentioned at the outset for training at least one algorithm for a control device of a motor vehicle in such a way that autonomous driving functions can be implemented in an autonomously driven motor vehicle more quickly and with higher quality than before. Summary of the invention

[0008] This object is achieved by a method according to the invention for training at least one algorithm for a control device of a motor vehicle, a computer program according to the invention, and a motor vehicle according to the invention. Further embodiments and developments are the subject matter of the technical solution of the invention.

[0009] A method for training at least one algorithm for a control device of a motor vehicle is described below, wherein the control device is configured to implement an autonomous driving function using the at least one algorithm based on input data while intervening in a unit of the motor vehicle, wherein the algorithm is trained by a self-learning neural network, the method comprising the following steps:

[0010] a) providing a computer program product module for an autonomous driving function, wherein the computer program product module comprises an algorithm to be trained and a self-learning neural network;

[0011] b) providing at least one indicator and a reward function for an autonomous driving function;

[0012] c) embedding the computer program product module into a simulation environment for simulating at least one traffic situation of importance for the autonomous driving function, wherein the simulation environment is based on map data of the real environment and on a digital vehicle model of the motor vehicle,

[0013] Training a self-learning neural network by simulating emergency scenarios and determining the quality until a first quality criterion (Gütemaβ) is met, wherein the quality is the result of a quality function of at least one indicator;

[0014] d) embedding the trained computer program product module into a control unit of a motor vehicle to simulate traffic situations that are relevant for an autonomous driving function, wherein the simulation is performed in a simulation environment based on map data of a real environment,

[0015] Training the self-learning neural network by simulating emergency scenarios and obtaining quality until a second quality standard is met, wherein the second quality standard is stricter than the first quality standard;

[0016] e) (i) If the quality in step d) is worse than the first quality standard, the method continues from step c)

[0017] Continue, or

[0018] (ii) if the quality in step d) is better than the first quality standard but worse than the second quality standard,

[0019] The method then continues with step d).

[0020] With the aid of the above-described method, an algorithm developed by means of a self-learning neural network for implementing an autonomous driving function can be developed more quickly and more safely than with the aid of conventional methods.

[0021] Since the system is trained in a purely virtual environment in an early step, the algorithm can already reach a certain maturity before the self-learning neural network can adapt the algorithm in the next step to the more complex situations caused by real motor vehicles in a safer virtual environment. Increased complexity such as variations in sensor input signals from real sensors, delays in the signal chain, temperature dependencies and similar phenomena.

[0022] By introducing a quality criterion for the algorithm, with the aid of which the determined indicator is measured, a long learning process can be avoided if the algorithm is not suitable for the higher reality level in step d), by temporarily resetting the learning process to a less complex full simulation in step c) and further developing the algorithm there.

[0023] Corresponding indicators can be, for example: the average number of accidents per route, the number of dangerous situations per route, the number of violations of traffic rules per route, etc. From these indicators, the quality can be determined, which is measured with the aid of quality standards. A stricter quality standard then means, for example, fewer accidents per route, fewer dangerous situations per route, etc. Only when the quality standard is no longer exceeded can further training be carried out in the next stage. This can prevent unstable algorithms from requiring long learning times and can achieve higher quality algorithms earlier.

[0024] A first possible further implementation configuration is provided:

[0025] f) simulating traffic situations that are important for the autonomous driving function in a mixed reality environment, and training the self-learning neural network by simulating critical scenarios and determining the quality until a third quality standard is met, wherein the third quality standard is stricter than the second quality standard, wherein:

[0026] g) If the quality in step f) is worse than the second quality standard, the method continues from step e).

[0027] According to this embodiment, in a next step, the algorithm can be expanded by a self-learning neural network in the context of mixed reality, in which the danger to traffic participants is minimized. The learning process can also be accelerated by checking the quality based on quality criteria and returning to an earlier stage of algorithm development if necessary.

[0028] Another possible further implementation configuration is as follows:

[0029] h) simulating traffic conditions that are important for the autonomous driving function in a real environment, and training the self-learning neural network by simulating critical scenarios and evaluating the quality until a fourth quality standard is met, wherein the fourth quality standard is stricter than the third quality standard, wherein:

[0030] i) If the quality in step h) is worse than the third quality standard, the method continues with step g), or if the quality in step h) is worse than the second quality standard, the method continues with step e).

[0031] According to this embodiment, in the next step, the algorithm can be expanded in a real environment by means of a self-learning neural network. At this point in time, it can be assumed that the algorithm is sufficiently stable so that safety in road traffic is no longer at risk. The learning process can also be accelerated by checking the quality and, if necessary, returning to an earlier stage of the algorithm development.

[0032] Another possible further embodiment provides that the computer program product module is activated for use in road traffic if the indicator meets a fourth quality criterion.

[0033] At this point in time it can be assumed that the algorithm is stable enough for use in normal road traffic.

[0034] Another possible further development provides that method steps f) and / or h) are performed by a safety driver.

[0035] This makes it possible to further reduce the risk to other road users, since the safety driver is instructed to always take over control of the autonomously driven motor vehicle within a short period of time.

[0036] Another possible further embodiment provides that the indicator has the following criteria: accident and / or collision time and / or braking time and / or required delay per unit route

[0037] The corresponding indicators are easy to obtain.

[0038] Another possible further implementation configuration is that the neural network is based on "reinforcement learning ( Lernen) method for learning.

[0039] Reinforcement learning or reinforcement learning represents a family of machine learning methods in which an agent (a self-learning neural network here) continuously learns a strategy in order to maximize the reward obtained. In this case, the agent is not given a pre-information about which action is best in which situation, but receives a reward at a certain moment, which may also be negative. Based on the reward, the agent approximates a utility function, which describes what value a certain state or a certain action has. With the help of corresponding learning methods, the self-learning neural network can continuously further develop the algorithm.

[0040] Another possible further embodiment provides that the neural network tries variants of existing algorithms according to a random principle.

[0041] This makes it possible to test various strategies leading to a desired result in the high-dimensional space of applied algorithms.

[0042] A first independent subject matter relates to a device for training at least one algorithm for a control device of a motor vehicle, wherein the control device is configured to implement an autonomous driving function using at least one algorithm based on input data while intervening in a unit of the motor vehicle, wherein the algorithm is trained by a self-learning neural network, wherein the device is configured to perform the following steps:

[0043] a) providing a computer program product module for an autonomous driving function, wherein the computer program product module comprises an algorithm to be trained and a self-learning neural network;

[0044] b) providing at least one metric and reward function for an autonomous driving function;

[0045] c) embedding the computer program product module into a simulation environment for simulating at least one traffic situation of importance for the autonomous driving function, wherein the simulation environment is based on map data of the real environment and on a digital vehicle model of the motor vehicle,

[0046] Training a self-learning neural network by simulating emergency scenarios and finding quality until a first quality criterion is met, wherein the quality is a result of a quality function of at least one indicator;

[0047] d) embedding the trained computer program product module into a control unit of a motor vehicle in order to simulate traffic situations that are relevant for an autonomous driving function, wherein the simulation is performed in a simulation environment based on map data of a real environment,

[0048] Training the self-learning neural network by simulating emergency scenarios and obtaining indicators until a second quality standard is met, wherein the second quality standard is stricter than the first quality standard;

[0049] e) (i) If the quality in step d) is worse than the first quality standard, the method continues from step c)

[0050] Continue, or

[0051] (ii) if the quality in step d) is better than the first quality standard but worse than the second quality standard,

[0052] The method then continues with step d).

[0053] A first possible further embodiment provides that the device is further configured to:

[0054] f) simulating traffic situations that are important for the autonomous driving function in a mixed reality environment, and training the self-learning neural network by simulating critical scenarios and determining the quality until a third quality standard is met, wherein the third quality standard is stricter than the second quality standard, wherein:

[0055] g) If the quality in step f) is worse than the second quality standard, the method continues from step e).

[0056] Another possible further implementation configuration is that the device is also configured to:

[0057] h) simulating traffic conditions that are important for the autonomous driving function in a real environment, and training a self-learning neural network by simulating critical scenarios and determining the quality until a fourth quality standard is met, wherein the fourth quality standard is stricter than the third quality standard, wherein if the quality in step h) is worse than the third quality standard, the method continues from step g), or if the quality in step h) is worse than the second quality standard, the method continues from step e).

[0058] Another possible further development provides that the device is further configured to enable the computer program product module for use in road traffic if the quality meets a fourth quality criterion.

[0059] Another possible further development provides that the device is also designed so that method steps f) and / or h) can be performed by a safety driver.

[0060] Another possible further embodiment provides that the device is further configured to use the following criteria as indicators: accident and / or collision time and / or braking time and / or required delay per unit of route.

[0061] Another possible further embodiment provides that the neural network is designed to learn according to a “reinforcement learning” method.

[0062] Another possible further embodiment provides that the neural network is designed to try variants of existing algorithms according to the principle of randomness.

[0063] Another independent subject relates to a computer program product having a computer-readable storage medium on which instructions are embedded, which are implemented when these instructions are executed by a computing unit, which is configured to implement the method according to the technical solution of the present invention.

[0064] A first further development of the computer program product provides that a computer program product module of the type mentioned above has these instructions.

[0065] A further independent subject matter relates to a motor vehicle having a computing unit and a computer-readable storage medium, wherein a computer program product of the aforementioned type is stored on the storage medium.

[0066] A first further development provides that the computing unit is a component of the control device.

[0067] Another further embodiment provides that the computing unit is networked with the environmental sensors. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Further features and details are derived from the following description, in which at least one embodiment is described in detail (with reference to the drawings, if necessary). The features described and / or shown in the drawings form the subject matter individually or in any meaningful combination, if necessary also independently of the technical solution of the invention, and in particular additionally can also be the subject matter of one or more separate applications. Identical, similar and / or functionally identical components are provided with the same reference numerals. Schematically shown here:

[0069] Figure 1 A motor vehicle is shown, the motor vehicle being configured for autonomous driving;

[0070] Figure 2 Shown for Figure 1 Computer program products for motor vehicles; and

[0071] Figure 3 A flow chart of the method is shown. DETAILED DESCRIPTION

[0072] Figure 1 A motor vehicle 2 is shown which is designed for autonomous driving.

[0073] The motor vehicle 2 has a motor vehicle control device 4, which has a computing unit 6 and a memory 8. A computer program product is stored in the memory 8, which is described in particular below in connection with Figure 2 and Figure 3 The computer program product is described in more detail.

[0074] The motor vehicle control device 4 is connected to a series of environmental sensors, which allow the current position of the motor vehicle 2 and the corresponding traffic conditions to be detected. These environmental sensors include: environmental sensors 10, 12 at the front of the motor vehicle 2, environmental sensors 14, 16 at the rear of the motor vehicle 2, a camera 18 and a GPS module 20. Depending on the configuration, other sensors, such as wheel speed sensors, acceleration sensors, etc., can be provided, which are connected to the motor vehicle control device 4.

[0075] During operation of the motor vehicle 2, the computing unit 6 loads the computer program product stored in the memory 8 and executes it. Based on the algorithm and the input signals, the computing unit 6 decides on the control of the motor vehicle 2, which can be achieved by intervening in the steering device 22, the engine control device 24 and the brake device 26, which are each connected to the motor vehicle control device 4.

[0076] Figure 2 A computer program product 28 having computer program product modules 30 is shown.

[0077] The computer program product 30 has a self-learning neural network 32, which trains an algorithm 34. The self-learning neural network 32 learns according to a reinforcement learning method, i.e., the neural network 32 tries to correspond to one or more criteria (Kriterien or ) obtains a reward for improved performance, i.e., a reward for the improvement of algorithm 34.

[0078] Algorithm 34 may essentially include a complex filter having a matrix of values ​​(usually referred to as weights) that define a filter function, which determines the performance of algorithm 34 as a function of input variables (which are currently recorded by environmental sensors 10 to 20) and generates control signals for controlling motor vehicle 2.

[0079] The quality of the algorithm 34 is monitored by another computer program product module 36, which monitors the input variables and output variables, determines indicators therefrom and controls the adherence to the quality based on the indicators by means of functions. At the same time, the computer program product module 36 can give the neural network 32 negative rewards as well as positive rewards.

[0080] Figure 3 A flow chart showing the method is shown.

[0081] In a first step, a computer program product module and a learning environment are provided.

[0082] In a purely virtual environment, both the vehicle as a model and the environment are provided virtually. The model of the vehicle corresponds to the later real model in terms of its parameters, sensors, driving characteristics and performance. The model of the environment is based on map data of the real environment in order to make the model as realistic as possible.

[0083] In this purely virtual environment, training continues until quality G M Better than the pre-given quality standard G1. Quality G M This results from a quality function G(M), which is a function of at least one indicator M. The respective indicator M can be a criterion such as the accident and / or collision time and / or braking time per unit of route and / or can have similar measured variables, such as required delay, lateral acceleration, undershooting of a safety distance, violation of applicable traffic regulations, etc.

[0084] As long as the quality G M If it is not enough to exceed the first quality standard G1, the training continues.

[0085] Only when the quality G MOnly when it is so high that the first quality criterion G1 is exceeded, the switch is made to the next phase of the training, in which the computer program product is transferred to the motor vehicle control unit 4 of the real motor vehicle and further trained there.

[0086] The training is carried out in a virtual environment based on a real motor vehicle. By using a real motor vehicle, which in some cases has different properties compared to the virtual model of the motor vehicle from the first training phase, the algorithm 34 can be further developed in such a way that the algorithm can take into account the properties of the real motor vehicle 2. For example, the differences may be due to the use of real sensors, which may have different signal heights, noise, etc.

[0087] The quality function G(M) is always monitored during training. The goal is the quality G M Better than the second quality standard G2. The second quality standard G2 is stricter than the first quality standard G1.

[0088] When switching to the real motor vehicle 2, it may occur that: Quality G M Below the first quality standard G1. In this case, the algorithm 34 switches back to the purely virtual environment and continues training until it exceeds the first quality standard G1 and continues training with the real motor vehicle 2.

[0089] Only when the quality G M Only when it is no longer lower than the second quality standard G2 can the training be continued in the next step.

[0090] Then, we switch to a partly real, partly virtual environment, in which we continue the previously described principle. If the quality function is below the threshold of the second quality criterion G2, the method is reset to the previous training step. If the quality function is even below the threshold of the first quality criterion G1, the method is reset to the initial training step.

[0091] The same principle continues in the next step by training the neural network in a real environment. This and the previous steps can be performed by safety drivers who can quickly switch back to manual driving mode in an emergency.

[0092] As long as the quality G M Better than the fourth G4, it is possible to enable algorithm 34 for free traffic.

[0093] Although the subject matter has been described and elaborated in detail by way of example, the present invention is not limited to the disclosed examples, and other variations can be derived therefrom by those skilled in the art. Therefore, it is apparent that there are multiple variations possible. It is also apparent that the exemplary embodiments mentioned only illustrate examples, which should not be construed as limitations on the scope of protection, application possibilities or configurations of the present invention in any way. On the contrary, the previous description and the accompanying drawings enable those skilled in the art to specifically implement these exemplary embodiments, wherein, in the case of understanding the disclosed inventive concept, those skilled in the art can, for example, make various changes in the functions or arrangements of the various elements mentioned in the exemplary embodiments, without departing from the scope of protection defined by the claims and their legal equivalents (e.g., further elaboration in the specification).

[0094] Reference numerals list

[0095] 2Motor Vehicles

[0096] 4Motor vehicle control equipment

[0097] 6 computing units

[0098] 8 Memory

[0099] 10 Environmental Sensors

[0100] 12 Environmental Sensors

[0101] 14 Environmental Sensors

[0102] 16 Environmental Sensors

[0103] 18 Camera

[0104] 20 GPS module

[0105] 22 Steering device

[0106] 24 Motor control device

[0107] 26 Braking device

[0108] 28 Computer program products

[0109] 30 Computer program product modules

[0110] 32 Neural Networks

[0111] 34 Algorithms

[0112] 36 Computer program product modules

[0113] G(M) quality function

[0114] G M quality

[0115] G1 first quality standard

[0116] G2 Second Quality Standard

[0117] G3 Third Quality Standard

[0118] G4 Fourth Quality Standard

[0119] M-Indicator

Claims

1. A method for training at least one algorithm (34) for a control device (4) of a motor vehicle (2), wherein: The control device (4) is configured to implement an autonomous driving function based on input data by using the at least one algorithm (34) while intervening in a unit (22, 24, 26) of the motor vehicle (2), wherein the algorithm (34) is trained by a self-learning neural network (32), the method comprising the following steps: a) providing a computer program product module (28) for the autonomous driving function, wherein the computer program product module (28) contains the algorithm (34) to be trained and the self-learning neural network (32); b) providing at least one metric (M) and a reward function for said autonomous driving function; c) embedding the computer program product module (28) in a simulation environment for simulating at least one traffic situation of importance for the autonomous driving function, wherein the simulation environment is based on map data of a real environment and on a digital vehicle model of the motor vehicle (2), By simulating emergency scenarios and obtaining quality (G M ) to train the self-learning neural network (32) until a first quality standard (G1) is met, wherein the quality (G M ) is the result of a quality function (G(M)) of said at least one indicator (M); d) When the quality (G M ) exceeds a first quality standard (G1), embedding the trained computer program product module (28) into a control device (4) of a real motor vehicle (2) to simulate traffic situations that are relevant and important for the autonomous driving function, the training being carried out in a virtual environment based on a real motor vehicle, wherein the simulation is carried out in a simulation environment based on map data of the real environment, By simulating emergency scenarios and obtaining the quality (G M ) to train the self-learning neural network (32) until a second quality standard (G2) is met, wherein the second quality standard (G2) is stricter than the first quality standard (G1); wherein, e) (i) If the quality (G M ) is worse than the first quality standard (G1), The method then continues from step c), or (ii) if the quality (G M ) is better than the first quality standard (G1) but worse than the second quality standard (G2), the method continues from step d), f) simulating traffic situations that are relevant to the autonomous driving function in a mixed reality environment and by simulating critical scenarios and determining the quality (G M ) to train the self-learning neural network (32) until a third quality standard (G3) is met, wherein the third quality standard (G3) is stricter than the second quality standard (G2), wherein, g) If the quality (G M ) is worse than the second quality standard (G2), the method continues from step e), h) simulating traffic situations that are relevant to the autonomous driving function in a real environment, and by simulating critical scenarios and determining the quality (G M ) to train the self-learning neural network (32) until a fourth quality standard (G4) is met, wherein the fourth quality standard (G4) is stricter than the third quality standard (G3), wherein, i) If the quality (G M ) is worse than the third quality standard (G3), the method continues from step g), or if the quality (G M ) is worse than the second quality standard (G2), the method continues from step e), Wherein, when the quality (G M ) when the fourth quality standard (G4) is met, the computer program product module (28) is enabled for use in road traffic.

2. The method according to claim 1, wherein: The method steps f) and / or h) are performed by a safety driver.

3. The method according to claim 1 or 2, wherein: The indicator (M) has the following criteria: accident and / or collision time and / or braking time and / or required delay per unit route.

4. The method according to claim 1 or 2, wherein: The neural network (32) learns according to a "reinforcement learning" method.

5. The method according to claim 1 or 2, wherein: The neural network (32) tries variations of existing algorithms based on a random principle.

6. A computer program product having a computer-readable storage medium (8) on which instructions are embedded, which are implemented when the instructions are executed by a computing unit (6), and the computing unit (6) is configured to implement the method according to any of the preceding claims.

7. The computer program product according to claim 6, wherein: The computer program product module (28) according to any one of claims 1 to 5 comprises the instructions.

8. A motor vehicle (2) comprising a computing unit (6) and a computer-readable storage medium (8), wherein: A computer program product according to claim 6 or 7 is stored on the storage medium (8).

9. The motor vehicle (2) according to claim 8, wherein: The computing unit (6) is a component of the control device (4).

10. A motor vehicle according to claim 8 or 9, wherein: The computing unit (6) is networked with environmental sensors (10, 12, 14, 16, 18).

Images