Method and system for reliably detecting objects in the surroundings of a motor vehicle

Abstract

The invention relates to a method for reliably detecting objects in the surroundings of a motor vehicle, said motor vehicle having a plurality of different surroundings sensors. The method (1) has the following steps: detecting sensor data by means of at least one of the plurality of different surroundings sensors (2); for each of the at least one surroundings sensors of the plurality of different surroundings sensors, detecting objects in the corresponding detected sensor data (3); determining at least one distinctive feature during the object detection on the basis of the detected objects (4); and for each of the at least one distinctive features, checking the corresponding distinctive feature in order to reliably detect (5) objects in the surroundings of the motor vehicle.

Description

[0001] Description

[0002] title

[0003] Method and system for reliably detecting objects in the environment of a motor vehicle

[0004] The invention relates to a method for reliably detecting objects in the environment of a motor vehicle and, in particular, to a method for detecting objects in the environment of a motor vehicle, which method ensures a high detection rate and, at the same time, can reduce the number of incorrectly detected objects.

[0005] Comprehensive detection of a motor vehicle's surroundings, and in particular of objects such as other road users in the vicinity of the vehicle, forms the basis for many driver assistance systems and automated driving functions of motor vehicles. Motor vehicles typically have sensor systems designed to detect objects within their respective detection range. The individual sensor systems each comprise an environmental sensor and a processing unit designed to analyze data provided by the environmental sensor, for example, to detect objects in the data. The environmental sensors can be, for example, video cameras, radars, LIDA sensors, or ultrasonic sensors.

[0006] Each of these environmental sensors is very limited in its respective area of ​​application and may not be able to provide all the necessary information about the vehicle's surroundings, for example, depending on the installation location or current weather conditions. By combining or fusing the information from different environmental sensors, the reliability of the recognition of the vehicle's surroundings and thus also the detection of objects in the vehicle's surroundings can be increased. This, in turn, can prevent corresponding safety-critical situations when controlling a driver assistance system or a driving function based on the detected objects.

[0007] In known approaches, objects are detected separately in the sensor data collected by different environmental sensors, and the individual detection results are then merged using a fusion approach. However, particularly in complex, confusing scenarios, it is difficult to ensure a high detection rate of objects in the vehicle's environment while simultaneously keeping the number of falsely detected objects low. Consequently, there is a need for improved methods for reliably detecting objects in the environment of a motor vehicle.

[0008] From the document WO 2013 / 056966 A1, a method for checking the plausibility of sensor signals and a method and a device for outputting a trigger signal are known, wherein a first sensor element detects at least one first physical quantity and outputs it as a first sensor signal, and wherein a second sensor element detects a second physical quantity correlated to the first physical quantity and outputs it as a second sensor signal.In addition, at least the first sensor element has at least one first reliability range with an upper limit and / or a lower limit, which is related to the second physical quantity detected by the second sensor element, wherein a current value of the first physical quantity detected by the first sensor element is recognized as plausible if a current value of the second physical quantity detected by the second sensor element lies within the corresponding first reliability range of the first sensor element.

[0009] The invention is therefore based on the object of providing an improved method for reliably detecting objects in the environment of a motor vehicle. This object is achieved by a method for reliably detecting objects in the environment of a motor vehicle according to the features of patent claim 1.

[0010] The problem is also solved by a system for reliably detecting objects in the environment of a motor vehicle according to the features of patent claim 8.

[0011] Disclosure of the invention

[0012] According to one embodiment of the invention, this object is achieved by a method for reliably detecting objects in the environment of a motor vehicle, wherein the motor vehicle has a plurality of different environmental sensors, and wherein the method comprises acquiring sensor data by at least one of the plurality of different environmental sensors, for each of the at least one of the plurality of different environmental sensors, detecting objects in the corresponding acquired sensor data, determining at least one abnormality in the object detection based on the detected objects, and, for each of the at least one abnormality, checking the corresponding abnormality in order to reliably detect objects in the environment of the motor vehicle.

[0013] A sensor, which is also referred to as a detector, measured variable or measuring transducer or (measuring) sensor, is understood to be a technical component that can detect certain physical or chemical properties and / or the material properties of its environment qualitatively or quantitatively as a measured variable.

[0014] An environmental sensor is also understood to be a sensor for detecting conditions present in an environment, for example the environment of a motor vehicle.

[0015] Different environmental sensors also have different

[0016] Types of environmental sensors or environmental sensors based on different methods, for example video cameras, radars, LIDArs or ultrasonic sensors.

[0017] The fact that objects are detected in the sensor data also means that objects or things represented in the object data, regardless of their type, are recorded by an object detection or an algorithm for object detection, whereby the objects can be, for example, pedestrians, other motor vehicles, traffic control systems or other objects in the environment of a motor vehicle.

[0018] Anomalies also include object detection results that may indicate an object but should be reviewed, particularly to avoid corresponding safety-critical situations when controlling a driver assistance system or a driving function. Such an anomaly may occur, for example, if an object is detected in the sensor data acquired by an environmental sensor, but the same object was not also detected in the sensor data acquired by other, different environmental sensors.

[0019] The fact that an anomaly is checked also means that the object underlying the corresponding anomaly is verified, i.e. it is checked whether the anomaly actually represents an object or not.

[0020] This can increase the reliability and accuracy of detecting objects in the environment of a motor vehicle, which in turn can prevent corresponding safety-critical situations when controlling a driver assistance system or a driving function based on the detected objects. In particular, the detection performance of corresponding object detectors can be increased, which in turn leads to improved performance of environmental perception. This ensures a high detection rate while simultaneously reducing the number of falsely detected objects. Furthermore, blind areas of individual object detectors—that is, situations in which objects are difficult to detect by individual object detectors—can be reliably detected.

[0021] Overall, an improved method for reliably detecting objects in the environment of a motor vehicle is provided.

[0022] The method may further comprise a step of pre-processing the corresponding acquired sensor data for each of the at least one of the plurality of different environmental sensors, wherein the step of detecting objects in the corresponding acquired sensor data for each of the at least one of the plurality of different environmental sensors comprises detecting objects in the corresponding pre-processed sensor data.

[0023] The fact that the sensor data is preprocessed means that the sensor data is specifically prepared for object detection, whereby, for example, deficiencies in the corresponding sensor can be compensated.

[0024] This allows the reliability of detecting objects in the vehicle's surroundings to be further increased by optimally adapting or preparing the relevant data for object detection.

[0025] In addition, the step of determining at least one abnormality in the object detection based on the detected objects may comprise determining at least one abnormality based on the detected objects and further information describing the situation.

[0026] Information describing the situation refers to data representing the respective situation. This information may include, for example, context information or information about the values ​​of one or more vehicle parameters present in the respective situation. This can further increase the reliability of detecting objects in the vehicle's surroundings by taking into account information about the current situation, such as current weather conditions, current lighting conditions, and / or the current vehicle speed.

[0027] Furthermore, the step of determining at least one abnormality in the object detection based on the detected objects may also comprise applying a machine learning algorithm which is trained to determine abnormalities in the object detection based on information about detected objects.

[0028] Machine learning algorithms are based on the use of statistical methods to train a computer to perform a specific task without having been explicitly programmed to do so. The goal of machine learning is to construct algorithms that can learn from data and make predictions. These algorithms create mathematical models that can be used, for example, to classify data.

[0029] This allows the detection of anomalies to be further optimized and the reliability and accuracy of anomaly detection to be further improved.

[0030] The step of, for each of the at least one anomaly, checking the corresponding anomaly may further comprise applying an event camera and / or refining an object detection underlying the corresponding anomaly and checking the corresponding anomaly based on the refined object detection and / or detecting objects in sensor data acquired by another of the plurality of different environmental sensors to check the corresponding anomaly.

[0031] An event camera is also defined as an imaging sensor that reacts to local changes in brightness. Instead of delivering an image sequence at a constant frequency, event cameras only transmit information from pixels where the brightness has changed significantly. This allows for even better detection of objects in the vehicle's surroundings, especially moving objects.

[0032] Refining an object detection underlying the corresponding anomaly and subsequently checking the corresponding anomaly based on the refined object detection is further understood to mean that all available computer resources are focused or directed on resolving the anomaly, i.e. the circumstances of the corresponding data processing system itself are taken into account, alternative, possibly more precise algorithms for object detection are selected and then an attempt is made to resolve the anomaly based on the more precise algorithm, or that the object detection is adapted in such a way that inconspicuous areas or pixels that are unimportant for the corresponding anomaly are omitted or not taken into account in order to save computing time and to be able to concentrate on resolving the corresponding anomaly.

[0033] The fact that objects are detected in sensor data acquired by another of the several different environmental sensors in order to check the corresponding anomaly further means that objects detected in sensor data of one environmental sensor are verified by checking whether the corresponding objects are also detected in sensor data of at least one other environmental sensor.

[0034] This can further increase the reliability of resolving anomalies and thus also of detecting objects in the vehicle's surroundings.

[0035] In one embodiment, the plurality of different environmental sensors comprise a video sensor and / or a LIDAR sensor and / or a radar sensor and / or an ultrasonic sensor. Video sensors are image-based sensors that can record moving scenes and / or evaluate the acquired sensor data fully automatically.

[0036] LIDAR sensors emit laser beams and detect the light scattered back from an object, whereby conclusions about the corresponding object can be drawn from the scattered light.

[0037] Radar sensors emit electromagnetic waves and detect the waves reflected from an object, whereby conclusions can be drawn about the corresponding object from the reflected waves.

[0038] Ultrasonic sensors, on the other hand, emit short, high-frequency sound pulses and detect the pulses reflected from an object. The reflected pulses can be used to draw conclusions about the corresponding object.

[0039] The several different environmental sensors can therefore be sensors that are already integrated into conventional motor vehicles or are usually already present, so that the process can be implemented without the need for complex and costly modifications.

[0040] However, the fact that the plurality of different environmental sensors comprise a video sensor and / or a LIDAR sensor and / or a radar sensor and / or an ultrasonic sensor is only one possible embodiment. Rather, the different environmental sensors can also comprise additional sensors that can detect objects in the vehicle's surroundings, for example, acoustic sensors.

[0041] A further embodiment of the invention also provides a method for controlling at least one function of a motor vehicle based on objects detected in an environment of the motor vehicle, wherein the method comprises detecting objects in the environment of the motor vehicle by a method described above for reliably detecting objects in an environment of a motor vehicle and controlling the at least one function of the motor vehicle based on the detected objects.

[0042] A function of a motor vehicle is understood to mean a controllable function of the motor vehicle, for example a driver assistance system or a controllable driving function.

[0043] Thus, a method for controlling at least one function of a motor vehicle is specified, which is based on an improved method for the reliable detection of objects in the environment of a motor vehicle. In particular, this is based on a method for the reliable detection of objects in the environment of a motor vehicle, with which the reliability or accuracy when detecting objects in the environment of a motor vehicle can be increased, which in turn can avoid corresponding safety-critical situations when controlling the function of the motor vehicle, for example a driver assistance system or a driving function based on the detected objects. In particular, the recognition performance or performance of corresponding object detectors can be increased, which in turn leads to improved performance of the environment perception.This ensures a high detection rate while simultaneously reducing the number of falsely detected objects. Furthermore, blind areas of individual object detectors—that is, situations in which objects are difficult to detect by individual object detectors—can be reliably detected.

[0044] A further embodiment of the invention also provides a system for reliably detecting objects in the environment of a motor vehicle, wherein the motor vehicle has a plurality of different environmental sensors, and wherein the system is designed to acquire sensor data by at least one of the plurality of different environmental sensors, wherein the system has, for each of the at least one of the plurality of different environmental sensors, a detection unit which is designed to detect objects in the corresponding acquired sensor data, and wherein the system further comprises a determination unit which is designed to determine at least one abnormality in the object detection based on the detected objects, and a verification unit which is designed to verify the corresponding abnormality for each of the at least one abnormality,to reliably detect objects in the vicinity of the vehicle.

[0045] Thus, an improved system for the reliable detection of objects in the environment of a motor vehicle is specified. In particular, the reliability or accuracy when detecting objects in the environment of a motor vehicle can be increased, which in turn can avoid corresponding safety-critical situations when controlling a driver assistance system or a driving function based on the detected objects. In particular, the detection performance or performance of corresponding object detectors can be increased, which in turn leads to improved performance of environmental perception. This can ensure a high detection rate and simultaneously reduce the number of incorrectly detected objects. Furthermore, blind areas of individual object detectors, i.e. situations in which objects are difficult to detect by individual object detectors, can be reliably detected.

[0046] The system can further comprise, for each of the at least one of the plurality of different environmental sensors, a preprocessing unit configured to preprocess the corresponding acquired sensor data, wherein, for each of the at least one of the plurality of different environmental sensors, the detection unit is configured to detect objects in the corresponding preprocessed sensor data. This can further increase the reliability of detecting objects in the vehicle's surroundings by optimally adapting or preparing the corresponding data for object detection. Furthermore, the determination unit can be configured to determine the at least one abnormality based on the detected objects and further information describing the situation.This can further increase the reliability of detecting objects in the vehicle's surroundings by taking into account information about the current situation, such as current weather conditions, current lighting conditions and / or current vehicle speed.

[0047] Furthermore, the determination unit can be designed to determine the at least one abnormality by applying a machine learning algorithm which is trained to determine abnormalities in object detection based on information about detected objects.

[0048] Information about detected objects is understood to mean data representing the corresponding detected objects.

[0049] This allows the detection of anomalies to be further optimized and the reliability and accuracy of anomaly detection to be further improved.

[0050] The verification unit can further be configured to use an event camera for each of the at least one abnormality to verify the corresponding abnormality and / or to refine an object detection underlying the corresponding abnormality and to verify the corresponding abnormality based on the refined object detection and / or to verify the corresponding abnormality by detecting objects in sensor data acquired by another of the several different environmental sensors. This can further increase the reliability of resolving abnormalities and thus also of detecting objects in the vehicle's surroundings.

[0051] In one embodiment, the plurality of different environmental sensors comprise a video sensor and / or a LIDAR sensor and / or a radar sensor and / or an ultrasonic sensor. The plurality of different environmental sensors can thus be sensors already integrated into conventional motor vehicles or are usually already present, so that the system can be implemented without the need for complex and costly modifications.

[0052] However, the fact that the plurality of different environmental sensors comprise a video sensor and / or a LIDAR sensor and / or a radar sensor and / or an ultrasonic sensor is only one possible embodiment. Rather, the different environmental sensors can also comprise additional sensors that can detect objects in the vehicle's surroundings, for example, acoustic sensors.

[0053] A further embodiment of the invention also provides a system for controlling at least one function of a motor vehicle based on objects detected in an environment of the motor vehicle, wherein the system comprises a receiving unit for receiving information about objects detected in the environment of the motor vehicle, wherein the objects were detected by a system described above for reliably detecting objects in an environment of a motor vehicle, and a control unit which is designed to control the at least one function based on the information provided.

[0054] Information about objects detected in the surroundings of the motor vehicle is understood to mean data representing the corresponding detected objects.

[0055] Thus, a system for controlling at least one function of a motor vehicle is specified, which is based on an improved system for reliably detecting objects in the environment of a motor vehicle. In particular, this system is based on a system for reliably detecting objects in the environment of a motor vehicle, with which the reliability or accuracy in detecting objects in the environment of a motor vehicle can be increased, which in turn can avoid corresponding safety-critical situations when controlling the function of the motor vehicle, for example a driver assistance system or a driving function based on the detected objects. In particular, the detection performance or performance of corresponding object detectors can be increased, which in turn leads to improved performance of environmental perception.This ensures a high detection rate while simultaneously reducing the number of falsely detected objects. Furthermore, blind areas of individual object detectors—that is, situations in which objects are difficult to detect by individual object detectors—can be reliably detected.

[0056] In summary, the present invention provides a method for detecting objects in the environment of a motor vehicle, with which a high detection rate can be ensured and at the same time the number of incorrectly detected objects can be reduced.

[0057] The described designs and further training courses can be combined as desired.

[0058] Further possible embodiments, developments and implementations of the invention also include combinations of features of the invention described previously or below with regard to the embodiments that are not explicitly mentioned.

[0059] Short description of the drawings

[0060] The accompanying drawings are intended to provide a further understanding of embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain principles and concepts of the invention.

[0061] Other embodiments and many of the aforementioned advantages will become apparent upon review of the drawings. The elements illustrated in the drawings are not necessarily drawn to scale.

[0062] Shown are: Fig. 1 a flowchart of a method for reliable detection of

[0063] Objects in an environment of a motor vehicle, according to embodiments of the invention; and

[0064] Fig. 2 is a schematic block diagram of a system for reliably detecting objects in an environment of a motor vehicle, according to embodiments of the invention.

[0065] In the figures of the drawings, the same reference symbols designate the same or functionally equivalent elements, parts or components, unless otherwise stated.

[0066] Fig. 1 shows a flowchart of a method for reliably detecting objects in an environment of a motor vehicle 1, according to embodiments of the invention.

[0067] In particular, Fig. 1 shows a flow chart of a method for reliably detecting objects in an environment of a motor vehicle 1, according to embodiments of the invention, wherein the motor vehicle has a plurality of different environmental sensors.

[0068] Comprehensive detection of a motor vehicle's surroundings, and in particular of objects such as other road users in the vicinity of the vehicle, forms the basis for many driver assistance systems and automated driving functions of motor vehicles. Motor vehicles typically have sensor systems designed to detect objects within their respective detection range. The individual sensor systems each comprise an environmental sensor and a processing unit designed to analyze data provided by the environmental sensor, for example, to detect objects in the data. The environmental sensors can be, for example, video cameras, radars, LIDA sensors, or ultrasonic sensors.Each of these environmental sensors is very limited in its respective area of ​​application and may not be able to provide all the necessary information about the vehicle's surroundings, for example, depending on the installation location or current weather conditions. By combining or fusing the information from different environmental sensors, the reliability of the recognition of the vehicle's surroundings and thus also the detection of objects in the vehicle's surroundings can be increased. This, in turn, can prevent corresponding safety-critical situations when controlling a driver assistance system or a driving function based on the detected objects.

[0069] In known approaches, objects are detected separately in the sensor data collected by different environmental sensors, and the individual detection results are then merged using a fusion approach. However, particularly in complex, confusing scenarios, it is difficult to ensure a high detection rate of objects in the vehicle's environment while simultaneously keeping the number of falsely detected objects low. Consequently, there is a need for improved methods for reliably detecting objects in the environment of a motor vehicle.

[0070] Fig. 1 shows a method 1 which comprises a step 2 of acquiring sensor data by at least one of the plurality of different environmental sensors, a step 3 of detecting objects in the corresponding acquired sensor data for each of the at least one of the plurality of different environmental sensors, a step 4 of determining at least one abnormality in the object detection based on the detected objects, and a step 5 of checking the corresponding abnormality for each of the at least one abnormality in order to reliably detect objects in the environment of the motor vehicle.

[0071] Method 1 has the advantage of increasing the reliability and accuracy of detecting objects in the environment of a motor vehicle, which in turn can avoid corresponding safety-critical situations when controlling a driver assistance system or a driving function based on the detected objects. In particular, the detection performance of corresponding object detectors can be increased, which in turn leads to improved performance of environmental perception. This ensures a high detection rate and simultaneously reduces the number of incorrectly detected objects. Furthermore, blind areas of individual object detectors, i.e., situations in which objects are difficult to detect by individual object detectors, can be reliably detected.

[0072] Overall, an improved method 1 for reliably detecting objects in the environment of a motor vehicle is thus provided.

[0073] Thus, method 1 can be used to achieve improved environmental perception and to identify individual sensor or detection deficiencies and subsequently compensate for them accordingly.

[0074] As shown in Fig. 1, the method 1 further comprises a step 6 of pre-processing the corresponding acquired sensor data for each of the at least one of the plurality of different environmental sensors, wherein the step 3 of detecting objects in the corresponding acquired sensor data for each of the at least one of the plurality of different environmental sensors comprises detecting objects in the corresponding pre-processed sensor data.

[0075] The preprocessing of the corresponding sensor data can, for example, comprise one or more processing steps of applying a semantic segmentation, applying a shape estimator, applying a low-pass filtering or averaging, or time-windowing the acquired sensor data.

[0076] According to the embodiments of Fig. 1, step 4 of determining at least one abnormality in the object detection based on the detected objects further comprises determining at least one abnormality based on the detected objects and further information describing the situation.

[0077] The additional information describing the situation or corresponding metadata may, for example, be one or more characteristics of the speed of a detected object, current lighting conditions, current weather conditions, situational context information, for example whether a detected object is at a zebra crossing, position data of the motor vehicle, or data from a navigation system of the motor vehicle or a planned route.

[0078] According to the embodiments of Fig. 1, the step of determining at least one abnormality in the object detection based on the detected objects also comprises applying a machine learning algorithm which is trained to determine abnormalities in the object detection based on information about detected objects.

[0079] The corresponding machine learning algorithm can, for example, be a neural network that was trained based on corresponding labeled training data.

[0080] In particular, the step of determining at least one abnormality during object detection can comprise applying an occupancy grid, in particular a two-dimensional occupancy grid. A two-dimensional occupancy grid consists of several cells and, viewed from a bird's eye view, represents a two-dimensional Cartesian grid of the vehicle's surroundings, wherein each cell of the occupancy grid can be assigned corresponding coordinates, and wherein the individual detected objects, i.e. the objects detected in at least one of the sensor data acquired by at least one of the several different environmental sensors, are assigned to the corresponding coordinates or cells, i.e. occupy them. Based on the occupied cells of the occupancy grid, abnormalities during object detection or regions of interest can then be determined.For example, if objects are detected in sensor data collected by different environmental sensors, an anomaly may exist if an object is only detected in the sensor data from some of these environmental sensors, but not in the sensor data from all of these environmental sensors. However, if objects are only detected in the sensor data collected by one environmental sensor, the corresponding detected objects may represent anomalies that should be verified by detecting objects in the sensor data collected by other environmental sensors.

[0081] When checking for abnormalities, you can then focus on the corresponding region of interest.

[0082] According to the embodiments of Fig. 1, step 5 of, for each of the at least one anomaly, checking the corresponding anomaly may comprise using an event camera to check the corresponding anomaly, refining an object detection underlying the corresponding anomaly and checking the corresponding anomaly based on the refined object detection and / or detecting objects in sensor data acquired by another of the plurality of different environmental sensors.

[0083] Furthermore, threshold values ​​of an object detection system or a corresponding algorithm underlying the corresponding anomaly can also be adjusted in order to achieve improved object detection.

[0084] In addition, it is also possible to optimally or improvedly use the resources of the environmental sensors, for example by using beam-forming methods to direct the sensor perception to specific regions of interest, or by increasing the sensor sampling frequency of at least one of the multiple environmental sensors for specific individual regions of interest. Overall, Fig. 1 thus shows a method 1 in which a parallel path to classic multimodal perception approaches is provided, which acts redundantly and can tune or modulate a classic path, for example if an anomaly is determined in the objects detected by one sensor modality, or in which greater robustness is achieved by comparing the objects detected by one sensor modality with objects detected by other sensor modalities.

[0085] According to the embodiments of Fig. 1, the plurality of different environmental sensors are again a video sensor, a LIDAR sensor, a radar sensor and an ultrasonic sensor.

[0086] The corresponding reliably detected objects can then be used, for example, to control a function of the motor vehicle, especially an autonomous vehicle. Furthermore, the data from the individual environmental sensors can be fused in advance.

[0087] Controlling a function of the motor vehicle can, for example, involve issuing a corresponding warning to a driver of the motor vehicle, intervening in the longitudinal and / or lateral guidance (e.g. initiating emergency braking), or issuing a corresponding warning to other road users, for example by automatically activating a signal generator of the motor vehicle, such as a horn or a vehicle headlight.

[0088] Fig. 2 shows a schematic block diagram of a system for reliably detecting objects in an environment of a motor vehicle 10, according to embodiments of the invention.

[0089] In particular, Fig. 2 shows a schematic block diagram of a system for reliably detecting objects in the environment of a motor vehicle 10, according to embodiments of the invention, wherein the motor vehicle has a plurality of different environmental sensors 11, 12, 13, 14. According to the embodiments of Fig.2, the system 10 is designed to capture sensor data by at least one of the plurality of different environmental sensors 11, 12, 13, 14, wherein the system 10 has a detection unit 15 for each of the at least one of the plurality of different environmental sensors, which is designed to detect objects in the corresponding captured sensor data, and wherein the system 10 further has a determination unit 16, which is designed to determine at least one abnormality in the object detection based on the detected objects, and a checking unit 17, which is designed to check the corresponding abnormality for each of the at least one abnormality in order to reliably detect objects in the environment of the motor vehicle.

[0090] The detection unit, the determination unit and the verification unit can each be implemented, for example, based on a corresponding code stored in a memory and executable by a processor.

[0091] As Fig. 2 shows, the system further comprises for each of the at least one of the plurality of different environmental sensors 11, 12, 13, 14 a preprocessing unit 18 which is designed to preprocess the corresponding acquired sensor data, wherein for each of the at least one of the plurality of different environmental sensors the detection unit 15 is designed to detect objects in the corresponding preprocessed sensor data.

[0092] The preprocessing unit can, for example, be implemented based on a corresponding code stored in a memory and executable by a processor.

[0093] According to the embodiments of Fig. 2, the determination unit 16 is configured to determine the at least one abnormality based on the detected objects and further information describing the situation. Furthermore, the determination unit 16 is configured to determine the at least one abnormality by applying a machine learning algorithm that is trained to determine abnormalities in object detection based on information about detected objects.

[0094] According to the embodiments of Fig. 2, the verification unit 17 is in turn designed to use an event camera for each of the at least one anomaly to verify the corresponding anomaly, to refine an object detection underlying the corresponding anomaly and to verify the corresponding anomaly based on the refined object detection, and / or to verify the corresponding anomaly by detecting objects in sensor data acquired by another of the plurality of different environmental sensors.

[0095] The several different environmental sensors 11, 12, 13, 14 are again a video sensor 11, a LIDAR sensor 12, a radar sensor 13 and an ultrasonic sensor 14.

[0096] In addition, the system 10 according to Fig. 2 is designed to carry out a method described above for reliably detecting objects in the environment of a motor vehicle.

Claims

Claims 1 . A method for reliably detecting objects in the environment of a motor vehicle, wherein the motor vehicle has a plurality of different environmental sensors, and wherein the method (1) comprises the following steps: Acquiring sensor data by at least one of the plurality of different environmental sensors (2); For each of the at least one of the plurality of different environmental sensors, detecting objects in the corresponding acquired sensor data (3); Determining at least one abnormality in the object detection based on the detected objects (4); and For each of the at least one abnormality, checking the corresponding abnormality in order to reliably detect objects in the surroundings of the motor vehicle (5).

2. The method according to claim 1, wherein the method (1) further comprises, for each of the at least one of the plurality of different environmental sensors, a step of pre-processing the corresponding acquired sensor data (6), and wherein the step of, for each of the at least one of the plurality of different environmental sensors, respectively detecting objects in the corresponding acquired sensor data (3) comprises detecting objects in the corresponding pre-processed sensor data.

3. The method according to claim 1 or 2, wherein the step of determining at least one conspicuity in the object detection based on the detected objects (4) comprises determining at least one abnormality based on the detected objects and other information describing the situation.

4. The method according to any one of claims 1 to 3, wherein the step of determining at least one abnormality in the object detection based on the detected objects (4) comprises applying a machine learning algorithm which is trained to determine abnormalities in the object detection based on information about detected objects.

5. The method according to any one of claims 1 to 4, wherein the step of, for each of the at least one anomaly, checking the corresponding anomaly (5) comprises using an event camera to check the corresponding anomaly and / or refining an object detection underlying the corresponding anomaly and checking the corresponding anomaly based on the refined object detection and / or detecting objects in sensor data acquired by another of the plurality of different environmental sensors to check the corresponding anomaly.

6. The method according to any one of claims 1 to 5, wherein the plurality of different environmental sensors comprise a video sensor and / or a LIDAR sensor and / or a radar sensor and / or an ultrasonic sensor.

7. A method for controlling at least one function of a motor vehicle based on objects detected in an environment of the motor vehicle, the method comprising the following steps: Detecting objects in the surroundings of the motor vehicle by a method for reliably detecting objects in the surroundings of a motor vehicle according to one of claims 1 to 6; and Controlling at least one function of the motor vehicle based on the detected objects.

8. A system for reliably detecting objects in the environment of a motor vehicle, wherein the motor vehicle has a plurality of different environmental sensors, and wherein the system (10) is designed to acquire sensor data by at least one of the plurality of different environmental sensors (11, 12, 13, 14), wherein the system (10) has, for each of the at least one of the plurality of different environmental sensors (11, 12, 13, 14), a detection unit (15) designed to detect objects in the corresponding acquired sensor data, and wherein the system (10) further comprises a determination unit (16) designed to determine at least one abnormality during object detection based on the detected objects, and a verification unit (17) designed to verify the corresponding abnormality for each of the at least one abnormality,to reliably detect objects in the vicinity of the vehicle, 9. System according to claim 8, wherein the system (10) further comprises, for each of the at least one of the plurality of different environmental sensors (11, 12, 13, 14), a preprocessing unit (18) which is designed to preprocess the corresponding acquired sensor data, and wherein, for each of the at least one of the plurality of different environmental sensors (11, 12, 13, 14), the detection unit (15) is designed to detect objects in the corresponding preprocessed sensor data.

10. System according to claim 8 or 9, wherein the determination unit (16) is designed to determine the at least one abnormality based on the detected objects and further information describing the situation.

11. System according to one of claims 8 to 10, wherein the determination unit (16) is designed to determine the at least one abnormality by applying a machine learning algorithm which is trained based on information about to determine detected objects and to determine abnormalities in object detection.

12. System according to one of claims 8 to 11, wherein the checking unit (17) is designed to use an event camera for each of the at least one anomaly to check the corresponding anomaly and / or to refine an object detection underlying the corresponding anomaly and to check the corresponding anomaly based on the refined object detection and / or to check the corresponding anomaly by detecting objects in sensor data acquired by another of the plurality of different environmental sensors.

13. System according to one of claims 8 to 12, wherein the plurality of different environmental sensors (11, 12, 13, 14) comprise a video sensor (11) and / or a LIDAR sensor (12) and / or a radar sensor (13) and / or an ultrasonic sensor (14).

14. A system for controlling at least one function of a motor vehicle based on objects detected in an environment of the motor vehicle, the system comprising a receiving unit for receiving information about objects detected in the environment of the motor vehicle, the objects being detected by a system for reliably detecting objects in an environment of a motor vehicle according to one of claims 8 to 13, and a control unit which is designed to control the at least one function based on the information provided.

Images