Method for evaluating sensor data with extended object recognition

Abstract

The invention discloses a method for analyzing and processing sensor data, wherein the sensor data is obtained by scanning the surrounding environment by at least one sensor, object detection and object filtering are performed based on the sensor data, the object detection is used to obtain objects from the sensor data, wherein surface characteristics of at least one object are identified and / or surface characteristics of at least one object are obtained by accessing a database. In addition, the invention discloses a control device.

Description

Technical Field

[0001] The invention relates to a method for evaluating sensor data, wherein the sensor data are ascertained by scanning the surroundings by at least one sensor, object detection and object filtering are performed based on the sensor data, the object detection being used to ascertain objects from the sensor data. The invention also relates to a control device. Background Art

[0002] In driver assistance systems and automated driving functions, the surroundings observed by sensors must be analyzed. When using sensors commonly used in this field (such as radar sensors, lidar sensors and cameras), depending on the application, a sufficient true positive rate for important relevant objects to be identified in the surroundings can usually only be achieved if a very high false positive rate is acceptable.

[0003] However, depending on the subsequent function, a high false positive rate can significantly limit the usability of the function. In general, an increased false positive rate can be caused by ghost objects, which are generated by reflections or multiple reflections. The type and generation of ghost objects vary depending on the sensor modality. Summary of the invention

[0004] The object on which the present invention is based can be seen as to be to specify a method for evaluating sensor data with a low false positive rate.

[0005] This object is achieved by means of the respective subject matter of the independent claims. Advantageous embodiments of the invention are the subject matter of the respective dependent claims.

[0006] According to one aspect of the present invention, a method for analyzing sensor data is provided, wherein the sensor data is obtained by scanning the surroundings by at least one sensor. Object detection and object filtering are performed based on the sensor data, wherein the object detection is used to obtain objects from the sensor data, wherein surface characteristics of at least one object are identified and / or surface characteristics of at least one object are obtained based on access to a database.

[0007] According to a further aspect of the present invention, a control device is provided for coupling to at least one sensor and for reading out sensor data of the at least one sensor, wherein the control device is configured to carry out all steps of the method.

[0008] The method allows objects to be identified and the surface properties of the identified objects to be determined or estimated. The objects can also be configured as unclassified objects and do not necessarily have to be designated as defined objects. Based on the additional surface properties of the objects or targets, the accuracy of the sensor data evaluation can be increased and the reliability of the environment sensor system can be improved.

[0009] In this case, the method uses in particular a classifier that can be designed to be trained and can be trained. Furthermore, data stored in a database (eg map data) can be used to determine surface properties of objects or to assign surface properties to specific objects.

[0010] The applicability of this principle is therefore not limited to certain sensor modalities. In addition, the reliability of environmental sensors and the functions associated with them can be increased by using multimodal classifiers (based on the fusion of multiple modalities).

[0011] By using map data, the method can also be used in situations in which the necessary recognition of object properties of the vehicle is not possible or cannot be performed with sufficient reliability or in sufficient time.

[0012] The method makes it possible to reduce the occurrence of ghost objects by means of a priori knowledge about surfaces or surface properties in the surroundings or to associate ghost objects with other properties in order to improve object recognition, for example, in subsequent sensor fusion.

[0013] According to one specific embodiment, surface properties of at least one identified object are ascertained in parallel with the object detection. As a result, a classifier can be used in a targeted manner to ascertain surface properties that, for example, promote reflections in the corresponding sensor modality. In particular, it can be checked based on the ascertained surface properties of the object whether the identified object is a reflection and is therefore classified as a ghost object. Thus, the identified object can be checked directly and / or indirectly, which increases the reliability of the evaluation.

[0014] According to another specific embodiment, the result of the ascertainment of the surface properties is used in the surface filtering. The ascertained or estimated surface properties of the object can be used directly for sensor data processing, so that ghost objects are reduced and the accuracy of the object state is thus improved.

[0015] According to another embodiment, in order to determine the surface properties of at least one detected object, a comparison with a database is performed during object filtering. According to one embodiment, the database is configured as a map with stored surface properties. Thus, a further embodiment of the method can be provided, in which an already existing high-precision map is expanded, for example, with material properties and / or surface properties of stationary objects.

[0016] The object properties and / or material properties may be, for example, concrete walls, mirrors, glass facades, guardrails, soundproof walls, etc., which have an influence on the environment detection and sensor evaluation due to their surface properties. With the help of a priori knowledge about the surfaces of the relevant objects, the sensor data processing can be improved or the accuracy can be increased in the subsequent sensor data fusion. In particular, the false positive rate can be reduced without compromising the true positive rate by this method.

[0017] According to another specific embodiment, at least one ghost object in the sensor data is removed or corrected based on the ascertained object properties of at least one object. The knowledge of the surface properties of objects in the surroundings of the sensor can determine the influence of the surface properties on the sensor. For example, a deflection, refraction, diffraction or reflection of a beam of a lidar sensor or radar sensor can be ascertained based on the surface properties and can be used to plausibly check the detected objects in the sensor data analysis.

[0018] For example, when detecting the environment with a lidar sensor, a vehicle behind a guardrail can be detected, which is only detected as a reflected ghost vehicle because it is blocked by the vehicle in front. With this method, either the reflective surface of the guardrail can be identified or the reflective surface of the guardrail can be derived from the map. With this information, either the ghost object can be removed or the position of the ghost object can be corrected by triangulation.

[0019] According to another embodiment, surface characteristics of at least one identified object are identified or classified by machine learning. Thus, for example, a neural network can be used to provide a classifier that assigns possible surface characteristics to the identified object, for example, based on the shape of the identified object.

[0020] According to another embodiment, within the scope of sensor data fusion, surface properties of at least one identified object are determined by object filtering based on database access, or, within the scope of sensor data fusion, identified surface properties of at least one identified object are used by object filtering. The result of the surface estimation can be passed on to subsequent sensor data fusion in order to improve the accuracy or reduce the false positive rate there as well.

[0021] Furthermore, object filters can be used at the fusion level. Compared to filtering within the scope of sensor data processing, a higher false positive rate of the sensor can be allowed here, since filtering is then performed. Compared to object filtering at the sensor data processing level, one advantage is that the surface estimation of one sensor modality can be taken into account in the object recognition of another modality, so that, for example, the surface estimation of a video camera can improve the object recognition of a lidar sensor. This allows other sensors to be used to determine surface properties or to support the determination of surface properties. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] A preferred embodiment of the present invention is described in detail below with the aid of a highly simplified schematic diagram. It is shown here:

[0023] Figure 1 The surroundings of a vehicle with sensors are shown for illustrating a method according to one specific embodiment;

[0024] Figure 2 A schematic flow chart for illustrating a method according to another embodiment is shown;

[0025] Figure 3 A schematic flow chart for illustrating a method according to another embodiment with an object filter for accessing a database is shown;

[0026] Figure 4 FIG. 1 shows a schematic flow chart for illustrating a method according to another specific embodiment with sensor data fusion;

[0027] Figure 5 A schematic flow chart is shown for illustrating a method according to a further specific embodiment with an object filter for accessing a database within the scope of sensor data fusion. DETAILED DESCRIPTION

[0028] exist Figure 1 2 shows the surroundings of a vehicle 1 with sensors 2 , 4 for explaining a method 6 according to one specific embodiment. The figure shows, in particular, an exemplary traffic situation with two lanes in one direction.

[0029] The sensors 2 , 4 may be, for example, a lidar sensor 2 and a camera 4 .

[0030] The sensors 2 , 4 are connected to a control device 5 . The control device 5 is used to read out the sensor data and to carry out the method 6 .

[0031] In this environment, lanes are separated from one another by lane dividers 8. Lane dividers 8 have reflective surface properties.

[0032] The vehicle 1 recognizes the vehicle 10 traveling ahead, while the other vehicle 12 is detected only as a reflected ghost vehicle 12a because it is blocked by the vehicle 10 traveling ahead. Through method 6, either the reflecting surface of the lane divider 8 can be identified or the reflecting surface can be obtained from the map. Therefore, with the help of this information, either the ghost object 12a can be removed or the ghost object 12a can be placed at the correct position of the other vehicle 12 by triangulation.

[0033] Figure 2 A schematic flow chart is shown for illustrating a method 6 according to another embodiment. In particular, a sensor architecture is shown. In this case, object detection 14 is performed based on the received sensor data 13 of the detectors of the sensors 2, 4. In parallel with the object detection 14, a material / surface classification 16 is performed. Based on the surface estimation 16 and the object data, a subsequent object filter 18 can reduce the false positive rate and / or increase the true positive rate. The surface classification 16 can be implemented, for example, by a previously trained neural network.

[0034] exist Figure 3 2 shows a schematic flow chart for illustrating a method 6 according to another embodiment with an object filter 18 accessing a database 20. In this case, information about surface properties is read out from a map 20, wherein the object filter 18 is responsible for the Figure 2 The surface classification 16 shown has the same function. Thus, surface properties are assigned to the identified objects 8, 10, 12, 12a by means of the object filter 18. The result of the object filter 18 can then be processed further.

[0035] exist Figure 4 , a schematic flow chart for illustrating a method 6 according to another embodiment with sensor data fusion 22 is shown. In this case, an object filter 18 is used at the fusion level. Compared to filtering within the scope of sensor data processing, a higher false positive rate of the sensors 2, 4 can be allowed here, because object filtering 18 is then performed. Compared to object filtering at the sensor data processing level, one advantage is that the surface estimation of one sensor modality can be taken into account in the object recognition of another modality, so that, for example, the surface estimation of the video camera 4 can improve the object recognition 16 of the lidar sensor 2. The sensor data 13, which are processed by the object filtering 18 and the sensor data fusion 22 and have potential object and surface characteristics, can then be further processed. This can be used, for example, to create or update an environment model.

[0036] Figure 5A schematic flow chart for explaining a method 6 according to another specific embodiment is shown with an object filter 18 accessing a database 20 within the scope of a sensor data fusion 22. In particular, a fusion architecture is shown in which the object filter 18 accesses the database 20 at the sensor data fusion level in order to perform a surface property recognition based on the sensor data 13 of a plurality of sensors 2, 4. In this case, information about the surface properties is read from a map 20, for example.

Claims

1. A method (6) for evaluating sensor data (13), wherein: Sensor data (13) are acquired by scanning a surrounding environment by at least one sensor (2, 4), and object detection (14) and object filtering (18) are performed based on the sensor data (13), wherein the object detection is used to acquire objects (8, 10, 12, 12a) from the sensor data (13), characterized in that surface properties of at least one object (8, 10, 12, 12a) are identified or classified by machine learning and / or surface properties of at least one object (8, 10, 12, 12a) are acquired based on access to a database (20), wherein lanes in the surrounding environment are separated from each other by lane separators (8), the lane separators (8) having reflective surface properties, and based on the acquired surface properties of the lane separators (8), at least one ghost object (12a) in the sensor data (13) is removed or the position of the at least one ghost object (12a) is corrected by triangulation.

2. The method according to claim 1, wherein: Surface properties of at least one detected object (8, 10, 12, 12a) are determined in parallel with object detection (14).

3. The method according to claim 2, wherein: The result of the ascertainment of the surface properties is used in the object filtering (18).

4. The method according to claim 1, wherein: In order to determine the surface properties of at least one detected object (8, 10, 12, 12a), a comparison with the database (20) is carried out during the object filtering (18).

5. The method according to claim 1 or 4, wherein: The database (20) is configured as a map having stored surface properties.

6. The method according to any one of claims 1 to 4, wherein: Within the scope of sensor data fusion (22), surface properties of at least one identified object (8, 10, 12, 12a) are determined by object filtering (18) based on access to a database (20), or, within the scope of sensor data fusion (22), identified surface properties of at least one identified object (8, 10, 12, 12a) are used by object filtering (18).

7. A control device (5) for coupling to at least one sensor (2, 4) and for reading sensor data (13) of the at least one sensor (2, 4), wherein: The control device (5) is configured to carry out all steps of the method (6) according to any one of the preceding claims.

Images