Method for training a machine learning model for a detection application in a vehicle

Abstract

The invention relates to a method (100) for training a machine learning model for a detection application in a vehicle (1), comprising the following steps: - providing (101) a training data set, wherein the training data set represents a traffic environment, wherein the training data set comprises identifiers for objects in the traffic environment, - selecting (102) a partial range of the identifiers in the training data set on the basis of at least one characteristic of the detection application, wherein the detection application is applied as part of a system function of the vehicle (1) and the at least one characteristic is specific to the system function, - training (103) the machine learning model for the detection application using the selected partial range of the identifiers. The invention also relates to a computer program, to a device and to a storage medium for this purpose.

Description

[0001] Description

[0002] title

[0003] Method for training a machine learning model for a detection application in a vehicle

[0004] The invention relates to a method for training a machine learning model for a detection application in a vehicle. The invention further relates to a computer program, a device, and a storage medium for this purpose.

[0005] State of the art

[0006] Active learning refers to the training of machine learning models using "labeled" data, i.e., data for which the output of the machine learning model has already been determined prior to training. Training a machine learning model specifically involves statistically optimizing the model to a target function tailored to the application. In active learning, machine learning models are generally trained using all available labels.

[0007] A specific application of a machine learning model is so-called three-dimensional object recognition. In this case, the labels are, for example, three-dimensional "bounding boxes," i.e., 3D cuboids that enclose the relevant objects (e.g., vehicles, pedestrians, trucks, cyclists, etc.).

[0008] In driver assistance applications, the output of an object detector can be used as an input signal for system functions such as adaptive cruise control (ACC) or automatic emergency braking (AEB). In state-of-the-art active learning, all available labels are typically used to optimize the detection application, regardless of the scenario. Furthermore, system functions usually utilize detection applications trained on the entire field of view, even though the system function generally does not require the entire field of view.

[0009] Disclosure of the invention

[0010] The invention relates to a method with the features of claim 1, a computer program with the features of claim 8, a device with the features of claim 9, and a computer-readable storage medium with the features of claim 10. Further features and details of the invention will become apparent from the respective dependent claims, the description, and the drawings. Features and details described in connection with the method according to the invention naturally also apply in connection with the computer program, the device, and the computer-readable storage medium according to the invention, and vice versa, so that a reciprocal reference is always possible with regard to the disclosure of the invention.

[0011] The invention relates in particular to a method for training a machine learning model for a detection application in a vehicle, comprising the following steps, wherein the steps can be performed repeatedly and / or sequentially. The machine learning model can be a neural network, in particular a deep neural network. The detection application is in particular an object detection, preferably a detection of three-dimensional objects.

[0012] In a first step, a training dataset is preferably provided, wherein the training dataset represents a traffic environment. The traffic environment is, for example, a highway, a country road, or a scene of urban or city traffic. The training dataset further preferably includes identifiers for objects in the traffic environment. Within the scope of the present invention, identifiers can also be referred to and understood as "labels." Labels in the training of a machine learning model are, in particular, data points that represent desired responses or results for the input data. They are, for example, a component of the supervised learning of a machine learning model. In the context of supervised learning, labels can serve to show the machine learning model during the training process what output is expected when certain inputs are processed.This allows the machine learning model to learn to identify relationships between the input data and the corresponding labels and to make predictions or decisions for new, unknown data. Labels can take various forms, depending on the type of task for which the machine learning model is being trained. In a classification task, for example, labels are the categories or classes to which the input data belongs. In a detection application within the scope of the present invention, the labels could be, for example, "Vehicle in the ego lane," "Vehicle in the adjacent lane," "Pedestrian," or "Traffic light." In this case, the labels are specifically three-dimensional "bounding boxes," i.e., three-dimensional cuboids that enclose objects such as vehicles, pedestrians, trucks, or cyclists.

[0013] In a further step, a subset of the identifiers in the training dataset is preferably selected based on at least one characteristic of the detection application. The detection application is used, in particular, within the framework of a system function of the vehicle, and the at least one characteristic is specific to this system function. The characteristic can, for example, specify that only vehicles on an ego lane of the vehicle are relevant for the system function. An ego lane is, in particular, the lane in which the vehicle is located. In this case, the subset would only include identifiers for objects that are assigned to vehicles on the vehicle's ego lane. For an overtaking assistant as a system function, for example, the vehicle's neighboring lane could also be relevant.

[0014] In a further step, the machine learning model is preferably trained for the detection application using the selected subset of the labels. This selected subset allows for targeted training, enabling the detection application to be applied more precisely within the system's functionality and optimized accordingly. The training is primarily conducted as supervised training. During training, the machine learning model adjusts model parameters, such as weights in a neural network, to maximize its predictive accuracy. Hyperparameters of the machine learning model, such as the learning rate or the tree depth, can also be adjusted during this process.The machine learning model can then be tested with a validation dataset to evaluate the generalizability of the machine learning model and to adjust the hyperparameters accordingly.

[0015] Furthermore, it is advantageous if the selection process, especially the selection of the sub-area, is automated and includes the following step:

[0016] Applying a user-defined algorithm designed to perform the selection based on at least one characteristic.

[0017] The user-defined algorithm is, for example, a rule-based selection of relevant markers by classifying the available boxes into objects inside and outside the ego lane using a lane recognition algorithm. Alternatively, the selection can also be made by an existing (production-ready) system or by an existing corresponding algorithm.

[0018] It may be advantageous for the procedure to further include the following step:

[0019] Weighting of the labels in an objective function of the machine learning model, where the selected sub-area is assigned a higher weighting than the remaining labels.

[0020] This allows the machine learning model to be specifically trained on the identifiers relevant to the vehicle's system function. This advantageously enables high accuracy to be achieved in a subsequent inference phase of the machine learning model. For example, the process may further include the following step:

[0021] Defining an additional auxiliary task for the machine learning model, whereby the labels of the selected sub-area are detected within the framework of the additional auxiliary task.

[0022] This can have the advantage of allowing the machine learning model to be trained in a more differentiated way, thereby achieving higher accuracy in recognizing the labels in the selected sub-area. The technique of using auxiliary tasks can be applied to improve the performance of the machine learning model on the main task, which in this case is the detection application. Auxiliary tasks can be trained in parallel with the main task, encouraging the machine learning model to learn internal representations that are useful for both the main and auxiliary tasks. By adding auxiliary tasks, a machine learning model can be encouraged to extract richer and more general features from the input data. These features can then be useful for both the main and auxiliary tasks.Auxiliary tasks can also serve as a form of regularization by preventing the machine learning model from focusing too heavily on the specific properties of the main task and potentially overfitting. The need to perform well on the auxiliary task as well can force the machine learning model to find more general solutions. Furthermore, training with auxiliary tasks allows a machine learning model to learn to organize its internal representations in a way that not only improves performance but also enhances the interpretability of the model dynamics.

[0023] Another possibility is that the auxiliary task is executed in parallel or sequentially, particularly iteratively, to the detection application. In other words, the auxiliary task can be performed simultaneously with the detection application, such as a distance control, or the auxiliary task can be performed first in defined iterations, followed by the distance control, or vice versa. An iterative application is also conceivable, i.e., multiple sequential executions of the auxiliary task and the detection application. As a further advantage, the method can also include the following step:

[0024] Using the trained machine learning model in an inference phase for the detection application within the framework of the vehicle's system function.

[0025] This allows the advantages of the machine learning model training according to the invention to be exploited with regard to the detection application, which has been optimized for the system function. For example, the specific training according to the invention can provide higher precision in detection, particularly in the detection of objects relevant to the system function.

[0026] It is also advantageous if the system function includes lane keeping assist, lane change assist, adaptive cruise control (ACC), or autonomous emergency braking (AEB). This list is not exhaustive, so other system functions that include a detection application are also conceivable.

[0027] It is possible that the trained machine learning model according to the invention is used in an inference phase in a vehicle. The vehicle can be, for example, a motor vehicle and / or passenger vehicle and / or autonomous vehicle. The vehicle can have vehicle equipment, for example, for providing an autonomous driving function and / or a driver assistance system. The vehicle equipment can be designed to control the vehicle at least partially automatically and / or accelerate and / or brake and / or steer.

[0028] The machine learning model is specifically trained for classification and / or object detection. Accordingly, the training can result in a trained machine learning model that can be used for classification and / or object detection. Its use, and thus its inference capabilities, can be implemented, for example, in a vehicle. The input data points can be, for example, pixels from image data or be based on them, in order to perform the classification and / or object detection of the data points based on the pixels. The input data can include sensor and / or image data, which at least partially results from acquisition with a sensor, preferably a camera sensor, and / or which has been at least partially synthesized, thus specifically replicating the real-world data of a sensor.Specifically, the values ​​of image points, preferably pixels, in the image data can represent the environment of a sensor and / or a vehicle and / or a traffic scene. A classification, preferably image classification and / or object detection, based on these values ​​can be provided. This enables, for example, the detection of objects within the traffic scene. The classification can also take the form of semantic segmentation (i.e., pixel- or area-based classification) and / or object detection. The image data can be, for example, images from a radar sensor and / or an ultrasonic sensor and / or a LiDAR sensor and / or a thermal imaging camera. Accordingly, the images can also be presented as radar images and / or ultrasonic images and / or thermal images and / or LiDAR images.

[0029] The invention also relates to a computer program, in particular a computer program product, comprising instructions which, when executed by a computer, cause the computer to execute the method according to the invention. Thus, the computer program according to the invention offers the same advantages as those described in detail with reference to a method according to the invention.

[0030] The invention also relates to a data processing device configured to execute the method according to the invention. This device can, for example, be a computer that executes the computer program according to the invention. The computer can have at least one processor for executing the computer program. Alternatively, a non-volatile data storage medium can be provided in which the computer program is stored and from which the processor can read the computer program for execution. The invention can also relate to a computer-readable storage medium that contains the computer program according to the invention and / or includes instructions that, when executed by a computer, cause it to execute the method according to the invention. The storage medium can be, for example, a data storage device such as a hard drive and / or non-volatile memory and / or a memory card.The storage medium can, for example, be integrated into the computer.

[0031] Furthermore, the method according to the invention can also be implemented as a computer-implemented method.

[0032] Further advantages, features, and details of the invention will become apparent from the following description, in which exemplary embodiments of the invention are described in detail with reference to the drawings. The features mentioned in the claims and in the description can each be essential to the invention individually or in any combination. The drawings show:

[0033] Fig. 1 is a schematic visualization of a method, a vehicle, a device, a storage medium and a computer program according to exemplary embodiments of the invention.

[0034] Fig. 1 schematically depicts a method 100, a vehicle 1, a device 10, a storage medium 15 and a computer program 20 according to exemplary embodiments of the invention.

[0035] Fig. 1 shows, in particular, an embodiment of a method 100 for training a machine learning model for a detection application in a vehicle 1. In a first step 101, a training dataset is provided, wherein the training dataset represents a traffic environment and includes identifiers for objects in the traffic environment. In a second step 102, a subset of the identifiers in the training dataset is selected depending on at least one characteristic of the detection application, wherein the detection application is used within the framework of a system function of the vehicle 1 and the at least one characteristic is specific to the system function. In a third step 103, the machine learning model for the detection application is trained using the selected subset of identifiers.

[0036] Relevant scenarios, or labels, can be derived from the system function requirements, either manually or automatically. Furthermore, performance optimization for relevant scenarios is achieved by categorizing existing labels into subcategories of important and unimportant labels for a specific system function.

[0037] During training, detections in the relevant scenarios can be improved as follows: A higher weighting of the selected labels can be applied to the objective function. Furthermore, an additional auxiliary task can be defined within which the selected labels are detected.

[0038] The following describes an example using an automatic distance control (ACC) system. Relevant markers here are, in particular, vehicles in the vehicle's "ego" lane on highways or rural roads. The accuracy for these relevant vehicles can be increased by the method according to exemplary embodiments of the invention, although this comes at the cost of reduced detection results for other markers, such as vehicles in adjacent lanes. In this case, the auxiliary task would be, for example, the detection of vehicles in the "ego" lane, while the primary task, or detection application, remains the detection of all vehicles.

[0039] Within the scope of the present invention, adjustments to the objective function for training the machine learning model can be made depending on the characteristics of the system functions. The objective function can be adapted by reweighting the existing labels or by introducing an auxiliary task that consists of detecting the identifiers relevant to the system function. As a result, the detection application can be advantageously optimized for the system functions. The identifiers relevant to the system function are preferably selected according to the characteristics of the system functions. The selection can be made by an existing (production-ready) system, manually, or by a user-defined algorithm. The user-defined algorithm is, for example,A rule-based selection of the relevant markers is achieved by classifying the available boxes into objects inside and outside the ego lane, utilizing a lane recognition algorithm. The preceding explanation of the embodiments describes the present invention solely by way of examples. Naturally, individual features of the embodiments can be freely combined with one another, provided this is technically feasible, without departing from the scope of the present invention.

Claims

Claims 1. Method (100) for training a machine learning model for a detection application in a vehicle (1), comprising the following steps: Providing (101) a training dataset, wherein the training dataset represents a traffic environment, wherein the training dataset includes labels for objects in the traffic environment, Selecting (102) a subset of the labels in the training data set depending on at least one characteristic of the detection application, wherein the detection application is applied within the framework of a system function of the vehicle (1) and the at least one characteristic is specific to the system function, Training (103) the machine learning model for the detection application using the selected subset of the labels.

2. Method (100) according to claim 1, characterized in that the selection (102) is automated and comprises the following step: Applying a user-defined algorithm designed to perform the selection based on at least one characteristic.

3. Method (100) according to claim 1 or 2, characterized in that the method (100) further comprises the following step: Weighting of the labels in an objective function of the machine learning model, where the selected sub-area is assigned a higher weighting than the remaining labels.

4. Method (100) according to one of the preceding claims, characterized in that the method (100) further comprises the following step: Defining an additional auxiliary task for the machine learning model, wherein, within the scope of the additional auxiliary task, the markings of the selected sub-area are detected.

5. Method (100) according to claim 4, characterized in that the auxiliary task is performed in parallel or sequentially, in particular iteratively, to the detection application.

6. Method (100) according to one of the preceding claims, characterized in that the method (100) further comprises the following step: using the trained machine learning model in an inference phase for the detection application within the framework of the system function of the vehicle (1).

7. Method (100) according to one of the preceding claims, characterized in that the system function is a lane keeping assistant, a lane change assistant, an automatic distance control or an automatic braking parking assistant.

8. Computer program (20), comprising instructions which, when the computer program (20) is executed by a computer (10), cause it to execute the method (100) according to one of the preceding claims.

9. Device (10) for data processing, which is configured to carry out the method (100) according to any one of claims 1 to 7.

10. Computer-readable storage medium (15), comprising instructions which, when executed by a computer (10), cause it to to carry out steps of the method (100) according to any one of claims 1 to 7.

Images