Method for training machine learning algorithm to assign confidence value to each of plurality of object detection results

Abstract

The invention relates to a method for training a machine learning algorithm for assigning a confidence value to each of a plurality of object detection results, the method (1) having the following steps: providing training data for training the machine learning algorithm, wherein the training data has a plurality of object detection results a, each of which characterizes the same object, and reference truth information (2) about the object, and wherein each of the plurality of object detection results is based on another combination of object detection respectively generated from sensor data of at least two different sensors.

Description

Technical Field

[0001] The present invention relates to a method for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results, wherein the plurality of object detection results respectively characterize the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated based on sensor data of at least two different sensors. Background Art

[0002] Conventional motor vehicles and in particular autonomous motor vehicles have a plurality of sensors, wherein the sensor data acquired by these sensors can be used to control vehicle functions, such as drive components, vehicle control units, lighting devices or other controllable systems of the respective motor vehicle.

[0003] A sensor, also called detector, (measurement variable or measurement) receiver or (measurement) detector, is understood here to be a technical component that can qualitatively or quantitatively detect specific physical or chemical properties and / or material properties of its environment as a measurement variable.

[0004] The sensors integrated into conventional motor vehicles can be optical sensors, in particular lidar sensors, radar sensors or optical cameras, for example. The data collected by these sensors can be processed by object detection algorithms, for example, in order to detect objects, such as pedestrians and / or other motor vehicles, in the corresponding data or in a combination of these data. Depending on the objects collected, one or more vehicle functions can then be controlled, for example, safety-related actions can be initiated.

[0005] It should be noted here that different sensors have different properties and / or are designed for different situations. For example, lidar sensors or lidar measuring systems are fast and precise, but they are susceptible to dust particles, water particles or smoke particles, for example. Radar sensors, on the other hand, can be used optimally regardless of the existing weather conditions, and are also particularly suitable for measuring speed. Optical cameras also generally have a very good resolution. Therefore, a method is needed that allows objects that can be detected respectively according to the sensor data of these sensors to be processed optimally based on the current conditions in order to achieve the best possible performance.

[0006] From document DE 102009021785 B4 a method for object recognition is known, wherein the object has a plurality of representations of an abstract object property and is assigned to an object property class of a hierarchical system of object property classes stored in a first memory, wherein at least one location where the object is supposed to be located is considered by means of a plurality of sensors having a sensor group, which each respond to at least one object property and then emit a sensor signal, wherein a check is made as to whether the emitted sensor signal exceeds a threshold value predetermined for it in each case and whether a sensor signal exceeding the threshold value is accepted. The sensor properties of the accepted sensor signals are combined in pairs to form identification property pairs, wherein the entirety of the identification property pair is compared with the object property class stored in the first memory, and wherein the object is determined according to the object property class, the object property pair of which corresponds to the acquired identification property pair. Summary of the invention

[0007] It is therefore an object of the present invention to specify a method which allows objects respectively detected using sensor data from different sensors to be processed optimally based on the current conditions.

[0008] This object is achieved by a method for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results according to the features of claim 1 .

[0009] The object is also achieved by a control device for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results according to the features of claim 6 .

[0010] According to one embodiment of the present invention, the task is solved by a method for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results, wherein the plurality of object detection results respectively characterize the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated based on sensor data of at least two different sensors, and wherein the method comprises: providing training data for training the machine learning algorithm, wherein the training data has a plurality of object detection results, wherein the plurality of object detection results respectively characterize the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated based on sensor data of at least two different sensors, and wherein the training data also has Baseline truth information about the object; and training a machine learning algorithm, wherein the training of the machine learning algorithm for each of the multiple object detection results includes determining the distance between the corresponding object detection result and the baseline truth and assigning a confidence value to the corresponding object detection result based on the distance between the corresponding object detection result and the baseline truth, and for each of the multiple object detection results, respectively determining the distance from each of the other object detection results, and training the machine learning algorithm based on the confidence values ​​respectively assigned to the object detection results and the distances between the individual object detection results, so that a conclusion about the confidence value of each of the multiple object detection results can be inferred based on the distances between the multiple object detection results.

[0011] Machine learning algorithms are based on the use of statistical methods in order to train a data processing device so that it can perform a specific task without having been explicitly programmed for this purpose. The goal of machine learning is to build algorithms that can learn from data and make predictions. These algorithms create mathematical models with which data can be classified, for example.

[0012] A confidence value is also understood to be a value describing the model's estimate of the accuracy of the respective predicted value, wherein the confidence value is typically between zero and one and the higher the accuracy, the larger the confidence value.

[0013] Furthermore, ground truth information is understood to be knowledge about the exact position of the corresponding object.

[0014] Different sensors are also understood to mean sensors with different properties or sensors designed for different situations.

[0015] Object detection is also understood to be the detection of an object in data acquired by exactly one of the at least two different sensors by applying an object detection algorithm to the corresponding data.

[0016] An object detection result is also understood to combine or fuse one or more of these object detections to form a common object detection result.

[0017] If the distance or spacing between one of the object detection results and the reference truth information or the exact position of the object is small, this means that the corresponding object detection result is very accurate. On the other hand, if the distance between one of the object detection results and the reference truth information is relatively large, this means that the corresponding object detection result is relatively inaccurate.

[0018] Therefore, a machine learning algorithm is trained based on labeled training data in such a way that, based on the distances between the individual object detection results, it can draw conclusions about the possible actual position of the corresponding object and therefore also the corresponding distances of the individual object detection results from the actual position of the object, from which it can also deduce the corresponding confidence values ​​and therefore also the optimal sensor in the corresponding situation or the optimal combination of sensors in the corresponding situation.

[0019] Overall, a method is thus described which allows objects respectively detected by different sensors to be processed optimally based on the current conditions.

[0020] Here, the step of training the machine learning algorithm may include training the machine learning algorithm in such a way that the algorithm is permutation invariant.

[0021] The machine learning algorithm is permutation invariant, which means that it remains the same or independent of whether the individual input data are swapped.

[0022] As a result, the assignment of a confidence value to each of the plurality of object detection results, in particular the accuracy of the assignment, can be further improved.

[0023] Furthermore, the at least two different sensors are at least two of a lidar sensor, a radar sensor, and an optical camera.

[0024] LiDAR sensors generate precise three-dimensional information about the shape and surface properties of surrounding objects. This technology uses a laser beam within the eye-safe range to create a 3D representation of the captured environment. However, a disadvantage here is that they are easily affected by dust particles, water particles or smoke particles, for example.

[0025] A radar sensor is a light beam-based sensor that is used to detect objects such as other vehicles and pedestrians and to measure their distance from the vehicle and their relative speed. For this purpose, electromagnetic waves are emitted. Radar sensors can be used optimally regardless of the existing weather conditions and are particularly suitable for measuring speed.

[0026] With an optical camera, light enters the camera housing through a lens, which projects an image onto a sensor. Optical cameras generally have very good resolution here.

[0027] Thus, the at least two different sensors may be sensors which are typically located in a motor vehicle, for example in an autonomously driven motor vehicle.

[0028] Here, the at least two different sensors are at least two of a laser radar sensor, a radar sensor, and an optical camera, but this is only one possible implementation. More specifically, it may include at least different sensors and other sensors with different characteristics and / or sensors designed for different situations.

[0029] Utilizing another embodiment of the present invention, a method for detecting objects in sensor data of at least two different sensors is also provided, wherein the method includes providing sensor data of at least two different sensors; for each of the at least two different sensors, respectively detecting an object in the sensor data collected by the corresponding sensor to obtain object detection; generating multiple object detection results from the object detection by fusing one or more object detections; for each of the multiple object detection results, respectively assigning confidence values ​​to the corresponding object detection results by using a machine learning algorithm to assign confidence values ​​to each of the multiple object detection results, wherein the machine learning algorithm is trained to assign confidence values ​​to each of the multiple object detection results by the method described above for training the machine learning algorithm to assign confidence values ​​to each of the multiple object detection results; comparing the confidence values ​​respectively assigned to the multiple object detection results to determine the object detection result with the highest confidence value; and detecting the object based on the object detection result with the highest confidence value.

[0030] A method for detecting objects in sensor data from at least two different sensors is therefore provided, which is based on a method that enables the objects detected by the sensors to be processed optimally according to the current conditions. The method is based in particular on a machine learning algorithm, which is trained on the basis of labeled training data in such a way that it can draw conclusions about the possible actual position of the respective object and thus also the corresponding distance of the respective object detection result from the actual position of the object based on the distances between the respective object detection results, from which it can also deduce the corresponding confidence value and thus the optimal sensor in the respective case or the optimal combination of sensors in the respective case.

[0031] Another embodiment of the present invention also provides a method for controlling a controllable system based on an object detected in sensor data of at least two different sensors, wherein the method includes detecting an object in the sensor data of at least two different sensors by the method for detecting an object in sensor data of at least two different sensors described above, and controlling the controllable system based on the detected object.

[0032] A controllable system is understood here to be a system that can be controlled in such a way that, by applying suitable control signals or by applying suitable tasks or actions, the state of the system or one or more components of the system can be transformed into a new state within a limited time, in particular from a selected input state to a selected output state.

[0033] Thus, a method for controlling a controllable system is provided, which is based on a method that enables objects detected by different sensors to be processed optimally according to the current conditions. In particular, the method is based on a machine learning algorithm that is trained based on labeled training data in such a way that it can draw conclusions about the possible actual position of the respective object and thus also the corresponding distance of the respective object detection result from the actual position of the object based on the distances between the respective object detection results, from which it can also deduce the corresponding confidence value and thus also the optimal sensor in the respective case or the optimal combination of sensors in the respective case.

[0034] Controllable systems here may be, in particular, robotic systems and, for example, autonomously driven motor vehicles.

[0035] Another embodiment of the present invention also provides a control device for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results, wherein the plurality of object detection results respectively represent the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated based on sensor data of at least two different sensors, and wherein the control device has a providing unit and a training unit, and the providing unit is constructed to provide training data for training the machine learning algorithm, wherein the training data has a plurality of object detection results, wherein the plurality of object detection results respectively represent the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated based on sensor data of at least two different sensors, and wherein The training data also has reference truth information about the object; the training unit is constructed to train a machine learning algorithm, wherein for each of the multiple object detection results, the training machine learning algorithm includes determining the distance between the corresponding object detection result and the reference truth and assigning a confidence value to the corresponding object detection result based on the distance between the corresponding object detection result and the reference truth, and for each of the multiple object detection results, respectively determining the distance to each of the other object detection results, and training the machine learning algorithm in this way based on the confidence values ​​respectively assigned to the object detection results and the distances between the individual object detection results, so that a conclusion about the confidence value of each of the multiple object detection results can be inferred based on the distances between the multiple object detection results.

[0036] Thus, a control device is provided which enables an optimal processing of objects respectively detected by different sensors based on the current conditions. The control device is designed in particular to train a machine learning algorithm based on labeled training data in such a way that it can draw conclusions about the possible actual position of the respective object and thus also the respective distance of the respective object detection result from the actual position of the object based on the distances between the respective object detection results, from which it can also deduce the respective confidence value and the optimal sensor in the respective case or the optimal combination of sensors in the respective case.

[0037] Here, the training unit may be configured to train the machine learning algorithm in such a way that the machine learning algorithm is permutation-invariant. As a result, the assignment of confidence values ​​to each of the plurality of object detection results and in particular the accuracy of the assignment may be further improved.

[0038] In addition, the at least two different sensors may also be at least two of a lidar sensor, a radar sensor and an optical camera. Thus, the at least two different sensors may be sensors that are typically in a motor vehicle, such as an autonomously driven motor vehicle.

[0039] However, the at least two different sensors are at least two of a lidar sensor, a radar sensor, and an optical camera, which is only a possible implementation. More precisely, the at least two different sensors can also have other sensors with different characteristics and / or sensors designed for different situations.

[0040] According to another embodiment of the present invention, a control device for detecting an object in sensor data of at least two different sensors is provided, wherein the control device comprises a providing unit, a detecting unit, a generating unit, an allocating unit, a comparing unit and a determining unit, wherein the providing unit is configured to provide sensor data of at least two different sensors, the detecting unit is configured to detect an object in the sensor data of the corresponding sensor for each of the at least two different sensors to obtain an object detection, the generating unit is configured to generate a plurality of object detection results from the object detection by fusing one or more object detections, and the allocating unit is configured to generate a plurality of object detection results for the plurality of object detections. For each of the plurality of object detection results, a confidence value is assigned to each of the plurality of object detection results by a machine learning algorithm, wherein the machine learning algorithm is trained to assign a confidence value to each of the plurality of object detection results by the control device described above for training the machine learning algorithm to assign a confidence value to each of the plurality of object detection results, the comparison unit is constructed to determine the object detection result with the highest confidence value by comparing the confidence values ​​respectively assigned to the plurality of object detection results, and the determination unit is constructed to detect the object according to the object detection result with the highest confidence value.

[0041] Thus, a control device for detecting objects in sensor data of at least two different sensors is provided, which is based on a control device that allows optimal processing of the objects respectively detected by the sensors based on the current conditions. In particular, the control device is based on a machine learning algorithm that is trained based on labeled training data in such a way that it can draw conclusions about the possible actual position of the respective object and thus also the corresponding distance of the respective object detection result from the actual position of the object based on the distances between the respective object detection results, from which it can also deduce the corresponding confidence value and the optimal sensor in the respective case or the optimal combination of sensors in the respective case.

[0042] According to another embodiment of the present invention, a control device for controlling a controllable system based on an object detected in sensor data of at least two different sensors is provided, wherein the control device has a detection unit and a control unit, the detection unit being constructed to detect an object in the sensor data of at least two different sensors through the control device for detecting an object in the sensor data of at least two different sensors described above, and the control unit being constructed to control the controllable system based on the detected object.

[0043] Thus, a control device for controlling a controllable system is provided, which is based on a control device that allows optimal processing of objects detected by the sensors based on the current conditions. In particular, the control device is based on a machine learning algorithm that is trained based on labeled training data in such a way that it can draw conclusions about the possible actual position of the respective object and thus also the respective distance of the respective object detection result from the actual position of the object based on the distances between the respective object detection results, from which it can also deduce the respective confidence value and the optimal sensor in the respective case or the optimal combination of sensors in the respective case.

[0044] Controllable systems here can also be, in particular, robotic systems and, for example, autonomously driven motor vehicles.

[0045] The described embodiments and developments can be combined in any desired manner.

[0046] Further possible embodiments, refinements and implementations of the invention also include combinations of inventive features not explicitly mentioned which were described above or below with reference to the exemplary embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The attached drawings are intended to provide a further understanding of the embodiments of the present invention, and are used to illustrate the embodiments and to explain the principles and concepts of the present invention in conjunction with the description.

[0048] Further embodiments and many of the advantages mentioned are apparent from the drawings. The elements shown in the drawings are not necessarily shown true to scale with respect to one another.

[0049] Figure 1 A flowchart showing a method for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results according to an embodiment of the present invention; and Figure 2 A schematic block diagram of a system for detecting an object in sensor data of at least two different sensors according to an embodiment of the present invention is shown.

[0050] In the drawings, unless otherwise indicated, the same reference numerals indicate the same or functionally equivalent elements, components or assemblies. DETAILED DESCRIPTION

[0051] Figure 1 A flowchart of a method for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results 1 according to an embodiment of the present invention is shown.

[0052] Sensor data acquired by different sensors, i.e. sensors with different characteristics and / or designed for different situations, can be fused or linked to obtain information of better quality. It is known to fuse sensor data and then further process the fused sensor data. Furthermore, it is also possible to first analyze the data acquired by the individual sensors and then to link the corresponding analysis results with one another. Such a linking of analysis results is used, for example, in the field of autonomous driving, where redundancy is taken into account for safety reasons in the functions of the respective vehicles.

[0053] However, the fusion result can be affected by the state of one of the sensors or by a deterioration, which can lead to safety-critical situations when analyzing the corresponding data. In particular, an influence or deterioration of the state of one of the sensors also affects the corresponding combination or modality. Therefore, a method is needed with which a trustworthy modality can be determined.

[0054] Figure 1A method for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results 1 is shown herein, wherein the plurality of object detection results respectively characterize the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated based on sensor data of at least two different sensors, and wherein the method 1 comprises: step 2 providing training data for training the machine learning algorithm, wherein the training data has a plurality of object detection results, wherein the plurality of object detection results respectively characterize the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated based on sensor data of at least two different sensors, and wherein the training data also has information about The method comprises the steps of: step 110 comprising: training a machine learning algorithm for each of the plurality of object detection results, wherein the training of the machine learning algorithm for each of the plurality of object detection results comprises determining the distance between the corresponding object detection result and the ground truth and assigning a confidence value to the corresponding object detection result based on the distance between the corresponding object detection result and the ground truth, and determining the distance between each of the plurality of object detection results and each of the other object detection results, and training the machine learning algorithm according to the confidence values ​​respectively assigned to the object detection results and the distances between the respective object detection results, so that a conclusion about the confidence value of each of the plurality of object detection results can be inferred according to the distances between the plurality of object detection results.

[0055] Therefore, the machine learning algorithm is trained based on labeled training data so that it can draw conclusions about the possible actual position of the corresponding object and therefore the corresponding distance of the individual object detection results from the actual position of the object based on the distances between the individual object detection results. From this, the corresponding confidence value and the optimal sensor in the corresponding situation or the optimal combination of sensors in the corresponding situation can be derived.

[0056] Overall, a method 1 is thus described which allows objects respectively detected by different sensors to be processed optimally based on the current conditions.

[0057] In this case, the distance can in each case be determined, for example, based on the Mahalanobis distance.

[0058] In this case, the distance can also be determined based on features of the respective object derived from the sensor data, for example the position of the respective object and a possible size of the object and / or an orientation or inclination of the object and / or the object class.

[0059] Furthermore, a 3D tensor can be generated according to the method, wherein the first dimension of the tensor consists of vectors, the elements of which respectively characterize the distance of one of the sensor modalities. If the distance cannot be determined in one case, the corresponding vector elements can be selected or set to be as large as possible.

[0060] The machine learning algorithm may also be an artificial neural network that receives the tensor as input. Figure 1 In an implementation manner, step 3 of training the machine learning algorithm includes training the machine learning algorithm in such a way that the algorithm is permutation invariant.

[0061] In particular, machine learning algorithms can be trained here based on a max pooling process.

[0062] according to Figure 1 In an embodiment, the at least two different sensors are furthermore a lidar sensor, a radar sensor and an optical camera.

[0063] The individual modalities are in particular: object detection based solely on sensor data of a lidar sensor, a combination or fusion of object detection based on sensor data of a lidar sensor and object detection based on sensor data of a radar sensor, a combination of object detection based on sensor data of a lidar sensor and object detection based on sensor data of an optical camera, a combination of object detection based on sensor data of a lidar sensor, object detection based on sensor data of a radar sensor and object detection based on sensor data of an optical camera, object detection based solely on sensor data of a radar sensor, a combination of object detection based on sensor data of a radar sensor and object detection based on sensor data of a lidar sensor, a combination of object detection based on sensor data of a radar sensor and object detection based on sensor data of a lidar sensor, The present invention relates to a method for detecting an object based on sensor data of a radar sensor and an object detection based on sensor data of an optical camera, a method for detecting an object based on sensor data of a radar sensor and an object detection based on sensor data of a lidar sensor, a method for detecting an object based on sensor data of an optical camera only, a method for detecting an object based on sensor data of an optical camera and an object detection based on sensor data of a lidar sensor, a method for detecting an object based on sensor data of an optical camera and an object detection based on sensor data of a radar sensor, and a method for detecting an object based on sensor data of an optical camera, an object detection based on sensor data of a lidar sensor and an object detection based on sensor data of a radar sensor.

[0064] In this case, the object detection result having the highest confidence value or the most trustworthy object detection result may then be determined, wherein the object may be detected or determined based on the most trustworthy object detection result.

[0065] The determined objects can then be used, in particular, to control a controllable system, for example an autonomously driven motor vehicle.

[0066] Figure 2 A schematic block diagram of a system for detecting an object in sensor data of at least two different sensors according to an embodiment of the present invention is shown.

[0067] like Figure 2 As shown, the system 10 here has a control device for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results 11 and a control device for detecting an object in sensor data of at least two different sensors 12, wherein the control device for detecting an object in sensor data of at least two different sensors 12 is constructed to detect the object based on the machine learning algorithm trained by the control device for training the machine learning algorithm to assign a confidence value to each of a plurality of object detection results 11.

[0068] according to Figure 2 In an embodiment, the control device for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results 11 further comprises a providing unit 13 and a training unit 14, the providing unit being configured to provide training data for training the machine learning algorithm, wherein the training data comprises a plurality of object detection results, wherein the plurality of object detection results respectively characterize the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated according to sensor data of at least two different sensors, and wherein the training data also comprises reference truth information about the object; the training unit being configured to train the machine learning algorithm, wherein for each of the plurality of object detection results, training the machine learning algorithm comprises determining a distance between the corresponding object detection result and the reference truth and assigning a confidence value to the corresponding object detection result based on the distance between the corresponding object detection result and the reference truth, for each of the plurality of object detection results, determining a distance from each of the other object detection results, and training the machine learning algorithm in such a manner based on the confidence values ​​respectively assigned to the object detection results and the distances between the respective object detection results, so that a conclusion about the confidence value of each of the plurality of object detection results can be inferred based on the distances between the plurality of object detection results.

[0069] The providing unit can be, for example, a receiver which is designed to receive the corresponding data. The training unit can also be implemented, for example, based on a code which is stored in a memory and can be executed by a processor.

[0070] according to Figure 2 In an embodiment of the present invention, the training unit 14 is again designed to train the machine learning algorithm in such a way that the machine learning algorithm is permutation invariant.

[0071] Furthermore, the at least two different sensors are at least two of a lidar sensor, a radar sensor, and an optical camera.

[0072] according to Figure 2 In an embodiment of the present invention, the control device for detecting an object in sensor data of at least two different sensors 12 further comprises another providing unit 15, a detecting unit 16, a generating unit 17, an allocating unit 18, a comparing unit 19 and a determining unit 20, wherein the another providing unit is configured to provide sensor data of at least two different sensors, the detecting unit is configured to detect an object in the sensor data collected by the corresponding sensor for each of the at least two different sensors to obtain an object detection, the generating unit is configured to generate a plurality of object detection results from the object detection by fusing one or more object detections, and the allocating unit is configured to generate a plurality of object detection results for the plurality of object detections. For each of the detection results, a confidence value is assigned to each of the multiple object detection results by a machine learning algorithm, wherein a machine learning algorithm is trained to assign a confidence value to each of the multiple object detection results by a control device for training the machine learning algorithm to assign a confidence value to each of the multiple object detection results 11, the comparison unit is constructed to determine the object detection result with the highest confidence value by comparing the confidence values ​​respectively assigned to the multiple object detection results, and the determination unit is constructed to detect the object or determine the object as the detected object according to the object detection result with the highest confidence value.

[0073] The providing unit can be, for example, a receiver configured to receive corresponding sensor data. The detecting unit, the generating unit, the assigning unit, the comparing unit and the determining unit can also be implemented, for example, based on a code stored in a memory and executable by a processor.

[0074] Furthermore, the control device for detecting objects in the sensor data of at least two different sensors 12 is also designed here to simultaneously recognize and to correspondingly evaluate a plurality of objects in the respective scene.

[0075] In addition, the control device for training a machine learning algorithm to assign a confidence value to each of the multiple object detection results 11 can also be constructed to perform the above-described method for training a machine learning algorithm to assign a confidence value to each of the multiple object detection results.

Claims

1. A method for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results, wherein the plurality of object detection results respectively characterize the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated based on sensor data of at least two different sensors, and wherein the method (1) has the following steps: - providing training data for training a machine learning algorithm, wherein the training data has a plurality of object detection results, wherein the plurality of object detection results respectively characterize the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated based on sensor data of at least two different sensors, and wherein the training data also has ground truth information (2) about the object; and - training a machine learning algorithm, wherein the training of the machine learning algorithm for each of the plurality of object detection results comprises determining a distance between the corresponding object detection result and the reference truth and assigning a confidence value to the corresponding object detection result based on the distance between the corresponding object detection result and the reference truth, determining for each of the plurality of object detection results a distance from each of the other object detection results, and training the machine learning algorithm based on the confidence values ​​respectively assigned to the object detection results and the distances between the respective object detection results, such that a conclusion about the confidence value of each of the plurality of object detection results can be inferred based on the distances between the plurality of object detection results (3).

2. The method according to claim 1, wherein the step of training the machine learning algorithm (3) comprises training the machine learning algorithm in such a way that the machine learning algorithm is permutation invariant.

3. The method according to claim 1 or 2, wherein the at least two different sensors are at least two of a lidar sensor, a radar sensor, and an optical camera.

4. A method for detecting an object in sensor data of at least two different sensors, the method comprising the following steps: - providing sensor data from at least two different sensors; - for each of at least two different sensors, respectively detecting an object in sensor data of the respective sensor to obtain an object detection; - generating a plurality of object detection results from the object detections by fusing one or more object detections; - for each of the plurality of object detection results, assigning a confidence value to the corresponding object detection result by a machine learning algorithm to assign a confidence value to each of the plurality of object detection results, wherein the machine learning algorithm is trained to assign a confidence value to each of the plurality of object detection results by a method for training a machine learning algorithm to assign a confidence value to each of the plurality of object detection results according to any one of claims 1 to 3; - comparing the confidence values ​​respectively assigned to the plurality of object detection results to determine the object detection result having the highest confidence value; and - Detecting the object based on the object detection result having the highest confidence value.

5. Controlling the controllable system in dependence on an object detected in sensor data of at least two different sensors, wherein the method comprises the following steps: - detecting an object in the sensor data of at least two different sensors by means of the method for detecting an object in the sensor data of at least two different sensors according to claim 4; and -Control controllable systems based on detected objects.

6. A control device for training a machine learning algorithm to assign a confidence value to each of a plurality of object detection results, wherein the plurality of object detection results respectively represent the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated based on sensor data of at least two different sensors, and wherein the control device (11) has a providing unit (13) and a training unit (14), the providing unit being configured to provide training data for training the machine learning algorithm, wherein the training data has a plurality of object detection results, wherein the plurality of object detection results respectively represent the same object, and wherein each of the plurality of object detection results is based on another combination of object detections respectively generated based on sensor data of at least two different sensors, and wherein the training The data also has reference truth information about the object; the training unit is constructed to train a machine learning algorithm, wherein for each of the multiple object detection results, the training machine learning algorithm includes determining the distance between the corresponding object detection result and the reference truth and assigning a confidence value to the corresponding object detection result based on the distance between the corresponding object detection result and the reference truth, for each of the multiple object detection results, respectively determining the distance to each of the other object detection results, and training the machine learning algorithm in accordance with the confidence values ​​respectively assigned to the object detection results and the distances between the individual object detection results, so that a conclusion about the confidence value of each of the multiple object detection results can be inferred based on the distances between the multiple object detection results.

7. The control device according to claim 6, wherein: The training unit (14) is designed to train the machine learning algorithm in such a way that the machine learning algorithm is permutation invariant.

8. The control device according to claim 6 or 7, wherein the at least two different sensors are at least two of a lidar sensor, a radar sensor and an optical camera.

9. A control device for detecting an object in sensor data of at least two different sensors, wherein the control device (12) comprises a providing unit (15), a detecting unit (16), a generating unit (17), an allocating unit (18), a comparing unit (19) and a determining unit (20), wherein the providing unit is configured to provide sensor data of at least two different sensors, the detecting unit is configured to detect an object in the sensor data of the corresponding sensor for each of the at least two different sensors to obtain an object detection, the generating unit is configured to generate a plurality of object detection results from the object detection by fusing one or more object detections, and the allocating unit is configured to allocate a plurality of object detection results for the plurality of object detections. For each of the detection results, a confidence value is assigned to each of the multiple object detection results by a machine learning algorithm, wherein the machine learning algorithm is trained to assign a confidence value to each of the multiple object detection results by a control device for training the machine learning algorithm to assign a confidence value to each of the multiple object detection results (11) according to any one of claims 6 to 8, the comparison unit is constructed to determine the object detection result with the highest confidence value by comparing the confidence values ​​respectively assigned to the multiple object detection results, and the determination unit is constructed to detect the object according to the object detection result with the highest confidence value.

10. A control device for controlling a controllable system based on an object detected in sensor data of at least two different sensors, wherein the control device has a detection unit and a control unit, the detection unit is constructed to detect an object in the sensor data of data generated by the at least two different sensors according to the control device for detecting an object in sensor data of at least two different sensors according to claim 9, and the control unit is constructed to control the controllable system based on the detected object.

Images