Device and method for fusing two obstacle maps for environmental detection

Abstract

Procedure for merging two obstacle maps (1, 2), hereinafter referred to as “grid (1, 2)”, comprising the steps: - Creating (200) a first raster (1) and a second raster (2), in which respective pixels (5, 6) represent an occupancy state of corresponding environment positions, - Characterizing (300) a different occupancy state of a first pixel (5) of the first raster (1) compared to a second pixel (6) of the second raster (2) corresponding to the first pixel (5) of the first raster (1) for the purpose of identifying pixels (7) hereinafter referred to as “conflict pixels (7)”, - Determining (400) distances between pixels (5, 6, 7) of the first raster (1) and pixels (5) with a positive occupancy state within the first raster (1), and / or - Determining (500) distances between pixels (5, 6, 7) of the second raster (2) and pixels (5) with a positive occupancy state within the second raster (2), the method further comprising - Compare (600) the determined distances for the conflict pixels (7) with a predefined reference and based on the comparison result - Determine (700) a probability of a cause for the existence of the conflict pixels (7).

Description

[0001] The present invention relates to a means of transportation, a device, and a method for fusing two obstacle maps, hereinafter referred to as "grids," for environment detection and object recognition. In particular, the present invention relates to an improved analysis of the cause of differences with regard to objects contained in the grids. To detect obstacles in the vicinity of a means of transportation (hereinafter referred to as "vehicle"), sensors scan the environment multiple times without contact. This information is stored in an obstacle map. An obstacle map is a 2D or 3D representation of the vehicle's environment and is represented, for example, by a multitude of cells / pixels / voxels. The more often an obstacle is detected, the greater the probability that this obstacle actually exists.Accordingly, the probability of an obstacle's presence is reduced if the sensor does not detect an obstacle in a measurement step, a process known as "free-space derivation." This significantly reduces the number of false detections. Within the scope of the present invention, a "false detection" refers to the case where an environmental object reported by the system is not actually present. If a certain number of obstacle detections in a grid cell is exceeded—for example, if a sensor repeatedly reports an obstacle in succession and / or multiple sensors report an obstacle for the respective cell (i.e., the probability of existence is high)—the cell is considered "occupied" and reported to subsequent functions. In the prior art, the individual cells of the obstacle maps are essentially fused independently of one another.“Using occupancy grids for mobile robot perception and navigation”, A. Ilfes, Computer, Vol. 22, No. 6, PP. 46-57, 1989 proposes to model the interdependence of the cells of a grid using a Markov Random Field.

[0002] DE 10 2013 217 488 A1 discloses a method for creating an occupancy grid with compact grid planes. Information from various sensors is entered into an occupancy grid to document environmental objects. If, for the same environmental position, the first sensor detects an "occupied" state and the second sensor a "free" state, additional sensor data is used as a decision criterion to determine whether the grid cell is recognized as occupied or traversable. For example, height information, intensities from a radar or LiDAR sensor, and the states of moving objects (moving, speed, lateral velocity, acceleration, or direction of movement) are used.

[0003] Because current technology considers the individual cells of obstacle maps independently and fuses the information they contain independently, the cause of conflicts or discrepancies between cells reported as "occupied" and those reported as "free" cannot be resolved. The aforementioned use of a Markov Random Field is too computationally intensive for automotive applications and cannot uncover the causes of conflicting pixels.

[0004] Furthermore, different sensors often measure different parts of an object. For example, a laser scanner is very good at measuring reflective objects (e.g., license plates, taillights, reflectors in guardrails), whereas a radar sensor is very good at detecting metallic reflectors (e.g., wheel arches, guardrail posts). These differences in the measurement results for one and the same obstacle often do not overlap sufficiently in an obstacle map representation, depending on the inverse sensor model used, to allow them to be attributed to the same obstacle. If the (slight) offset of two such grids resulting from different sensors could be recognized, the information contained in the grids could be more accurately assigned to a single environmental object.

[0005] It is an object of the present invention to eliminate the disadvantages of the prior art identified above.

[0006] The task identified above is solved by a method for fusing two obstacle maps from an environment detection device. This fusing serves to extract different information from the respective obstacle maps, which are subsequently referred to as "rasters." First, a primary raster and a secondary raster are created, in which each pixel represents the occupancy state of corresponding environment positions. In other words, sensor information is assigned to environment positions, and the respective raster indicates whether a pixel (or cell or voxel) is likely occupied or likely free. The circumstances under which two rasters are available are irrelevant. The only important factor is that both rasters contain relevant information for the intended environment detection.Subsequently, the different occupancy states of a first pixel in the first raster are characterized in relation to a corresponding second pixel in the second raster. In other words, pixels with different occupancy states for identical surrounding positions in the rasters are identified as conflict pixels. Next, distances between pixels in the first raster and pixels with a positive occupancy state within the first raster are determined. Specifically, only for the conflict pixels can a distance to pixels with a positive occupancy state be determined and stored. For example, the distances between the centers of two considered pixels can be recorded as multiples of a pixel length or a pixel width. In this way, at least for the conflict pixels, the distance to the nearest pixel with a definite positive occupancy state is determined.In particular, it can first be checked whether conflict pixels occur in the first raster or in the second raster after the merge, and the distances can only be determined if conflict pixels are actually present. Subsequently, the determined distances for the conflict pixels can be compared with a predefined reference. For example, the reference can characterize a maximum distance that pixels of an identical environment object can or may have in the rasters. Depending on the characterized distance, probability values ​​for the respective causes of conflict pixels can be assigned to the reference. Based on the comparison result, a probability for a cause of the existence of the conflict pixels is then determined. This can be done, for example, by reading the respective probability from the predefined reference.The term "probability" for a cause is to be interpreted broadly within the scope of the present invention and also includes definitive assignments (e.g., "occupied" or "free" or "cause 1" or "cause 2"). The examination of the distances between the positively occupied pixels and the conflict pixels is sometimes very crucial for determining the cause of the conflict pixels, so the present invention represents a particularly advantageous fusion of two rasters for environment detection.

[0007] The dependent claims describe preferred embodiments of the invention.

[0008] One reason for two rasters that need to be merged is that the first raster was created at a first time point and the second raster at a subsequent second time point. For example, both rasters may have been created using one and the same sensor. This does not preclude the possibility that one of the rasters may have been created, at least partially, based on a second, different sensor, whereby the operating principles of the sensors may be identical or different. In particular, the perspectives that the sensors have on the environmental object contained in both rasters, and their detection ranges or opening angles, may differ. The method according to the invention enables a reliable assignment of the cause of different occupancy states for pixels representing identical environmental positions.In particular, a distinction can be made as to whether the cause is noise or another singularity, a time offset and / or an exit of an environmental object from at least one detection area of ​​a sensor used.

[0009] Examples of environmental sensors usable according to the invention are: ultrasonic sensor, optical camera (2-D / 3-D), thermal imaging camera, acoustic camera, laser / lidar and radar.

[0010] The predefined reference can, for example, be defined based on a sensor characteristic of a sensor used to generate at least one of the grids. This characteristic might consist of a sensor having a particularly narrow opening angle or delivering reliable or less reliable results depending on information technology-characterized circumstances. Alternatively or additionally, a motion state (e.g., velocity, acceleration, etc.) of a means of locomotion equipped according to the invention can also parameterize the predefined reference. In particular, the motion state can be crucial for aligning a sensor with a specific environmental area.

[0011] Characterizing the different occupancy states can be performed sequentially for a large number of pixels in both rasters, pixel by pixel. First, pixels corresponding to identical surrounding positions can be identified, and then their occupancy states can be compared. In this way, changes in the pixel orientations relative to their surroundings can be taken into account, which corresponds to a "calibration process."

[0012] The different occupancy states can be determined, for example, by calculating the difference between numerical values ​​(1 or 0) that characterize the occupancy state. For instance, a value other than 0 represents a conflict pixel. Alternatively or additionally, an AND / OR operation can be performed on corresponding pixels, or a Boolean operation can be carried out. Alternatively or additionally, the steps of a Dempster-Shafer fusion can be used to examine the raster for conflicts / the presence of conflict pixels.

[0013] According to a second aspect of the present invention, a device for fusing two obstacle maps (“grids”) for environmental detection is proposed. The device comprises an evaluation unit (e.g., a programmable processor, a microcontroller, a nanocontroller, or the like), a sensor input for receiving environmental sensor signals, and a data storage unit which can, for example, store instructions for executing the steps of the method according to the first aspect of the invention, as well as the predefined reference. The evaluation unit is configured to create a first grid and a second grid based on the environmental information received via the sensor input, in which respective pixels represent the occupancy state of corresponding environmental positions.Furthermore, the evaluation unit can characterize a different occupancy state of a first pixel of the first raster compared to a corresponding second pixel of the second raster to identify conflict pixels. The evaluation unit is further configured to determine distances between pixels of the first raster and pixels with a positive occupancy state. This can be done, in particular, exclusively for the conflict pixels. In accordance with the first aspect of the invention, it can first be determined whether conflict pixels are present in the first raster and / or the second raster at all, and only then, if so, the distances for a respective raster are determined.The evaluation unit can then compare the determined distances for the conflict pixels with a predefined reference and, based on the comparison result, determine the probability of a cause for the existence of the conflict pixels. The features, feature combinations, and the advantages arising therefrom correspond so clearly to those described in connection with the method according to the invention that reference is made to the above descriptions to avoid repetition.

[0014] According to a third aspect of the present invention, a means of transport is proposed, which may be, for example, a car, a van, a truck, a motorcycle, an aircraft and / or a watercraft. As part of a system for environmental detection, the means of transport includes a device according to the second aspect of the invention.

[0015] Further details, features and advantages of the invention will become apparent from the following description and the figures. These show: Fig. 1 a schematic illustration of sensory detection of environmental objects by an embodiment of a means of locomotion according to the invention; Fig. 2 a schematic representation of components of an embodiment of a device according to the invention; Fig. 3 a top view of a traffic situation of an embodiment of a means of transport according to the invention; Fig. 4, Fig. 5, Fig. 6 to Fig. 7 schematic illustrations of grids created or used according to the invention; Fig. 8 a typical traffic situation from the perspective of an exemplary embodiment of a means of transport according to the invention; Fig. 9 and Fig. 10 schematic illustrations of information contained in grids created according to the invention; and Fig. 11 A flowchart illustrating steps of an embodiment of a method according to the invention for fusing two grids of a device for environmental detection.

[0016] Fig. Figure 1 shows a car 10 as a means of transport, which has a laser scanner 8 as an environmental sensor whose detection range 16 is cone-shaped and oriented in the direction of travel. A first environmental object 13, sitting upright on the ground, falls at least partially within the detection range 16 of the laser scanner 8. A second environmental object 14 is so far from the ground that it is not detected by the laser scanner 8. A third environmental object 15 is so low that its upper edge is below the detection range 16. As a result, the second environmental object 14 and the third environmental object 15 are no longer detected by the laser scanner 8 at the distance shown. As the car 10 approaches, the detection range 16 "dives" below the second environmental object 14 and "rises" above the third environmental object 15.

[0017] Fig. Figure 2 shows components of a device 4 designed according to the invention for fusing two grids. A laser scanner 8 as the first environmental sensor and an ultrasonic sensor 9 as the second environmental sensor are connected to a sensor input 20 of an electronic control unit 11 as an evaluation unit. In this way, the electronic control unit 11 is able to create different grids representing the respective occupancy states of environmental positions and to compare distances between conflict pixels and pixels that are definitely occupied with predefined references stored in a data memory 12, as well as to determine a probability for a cause for the existence of the conflict pixels.

[0018] Fig. Figure 3 shows a traffic situation involving a passenger car 10 as a means of transport designed according to the invention, whose environmental sensors (not shown) generate a polar representation of environmental objects in the form of two guardrails 17, 18 and another vehicle 19. The positions at which the environmental objects lead to detectable reflections are marked by circles. The visibility limits due to a lack of reflections are marked by triangles.

[0019] Fig. Figure 4 shows in subdiagram a) an exemplary representation of a guardrail within a first grid 1, which was created by a first environmental sensor. Populated pixels 5 represent the area definitely occupied by the guardrail, while empty pixels 6 are derived as free space and marked accordingly. Subdiagram b) shows a second grid 2, which represents an area identical to that in subdiagram a). Compared to the first grid 1, the guardrail in the second grid 2 is represented by only a single column of populated pixels 5, which may be due to the use of different sensors and / or a resolution of the sensor(s) used that changes over time. While a change in the actual extent of the guardrail over time cannot be completely ruled out, it is unlikely based on experience.Partial diagram c) shows a characterization according to the invention of different occupancy states in the first raster 1 and in the second raster 2 by a third raster 3. In this raster 3, conflict pixels 7 are marked with dots. These represent the different information when the first raster 1 and the second raster 2 are merged.

[0020] Fig. 5 shows in subdiagram a) essentially a with Fig. 4. Partial diagram a) shows a corresponding representation of a first grid representing an environment containing a guardrail as an environment object. According to the inventive method, the distances between pixels 5, 6 of the first grid and pixels 5 with a positive occupancy state within the first grid are recorded such that each pixel 5, 6 has been assigned a numerical value which represents the distance (i.e., the shortest distance between pixel centers) to the nearest definitely occupied pixel 5 as a multiple of a pixel width. The definitely occupied pixels 5 bear the numerical value "0", while the neighboring free pixels 6 bear the respective numerical values ​​"1" - "4". Partial diagram b) shows a second grid 2, which may, for example, have been created based on the signals from another sensor.The representation of the guardrail ends in the third pixel row from the top, so that the pixels 6 arranged below the occupied pixels 5 were assigned corresponding numerical values ​​to indicate their distance from the safely occupied pixels 5. It should be noted that, for the sake of simplicity, diagonal distances were evaluated according to horizontal or vertical distances, although this does not correspond to the geometric relationships. Partial diagram c) shows the result according to the invention of a fusion of the first raster 1 and the second raster 2 into a third raster 3, wherein the conflict pixels 7 now have the distance values ​​determined in partial figure b). In comparison with . Fig. Figure 4 shows that the conflict pixels 7 have significantly different and, on average, higher distance values ​​than is typical in Fig. 4. Subdiagram c) would be the case (distance value “1” in each case). Using a predefined reference in the form of a numerical value of “1” or “2”, a cause-related probability could be determined that the conflict pixels in Fig. c) in Fig. 4 due to a measurement error, in Fig. 5, however, resulted from completely different detection ranges of the sensors used.

[0021] Fig. Figure 6 shows, in subfigure a), an environment object represented by pixels 5 that are definitely occupied, located in the center of the first grid 1. This representation can be described as a square with a side length of three pixels. Subfigure b) shows a modified representation of the environment object compared to subfigure a), such that the square of occupied pixels 5 has been "shrunk" to a side length of two pixels. Subdiagram c) highlights the conflict pixels 7, which are represented accordingly. Fig. 4, subdiagram c) show a respective distance of 1 to pixels 5 that are definitely occupied. Based on a predefined reference, it can be concluded that a similar cause to that associated with Fig. 4 must be present.

[0022] Fig. 7 shows a similar situation Fig. 6, while in subdiagram b) no environment object is detected at all, and therefore in subdiagram c) the entire area marked in red in subdiagram a) is represented as conflict pixels 7. These all have an infinite distance from pixels with certainty, since the current information does not know of any pixels with certainty. A predefined reference with a predefined probability can also be stored for this case. For example, the one in Fig. The information situation shown in section 7 is a special case of the one described in [reference to information]. Fig. The information situation shown in point 5 can be understood if in Fig. 5 only the bottom four lines are considered.

[0023] Fig. Figure 8 shows the perspective of a means of transport designed according to the invention in front of a closed barrier 21. The horizontal distance of the barrier 21 from the ground enables the “blindness” of a (not shown) laser scanner to the barrier, since its detection range lies below the barrier 21.

[0024] Fig. Figure 9 shows a data-technical representation of this information. The barrier 21 is not represented by occupied pixels. In contrast, the barrier 21 can be detected without problems, for example, by an optical front camera. Due to these different sensor data, the representation according to the invention is obtained. Fig. 10, according to which conflict pixels 7 are entered at the position of the barrier, which, due to the non-existence of the barrier in the laser scanner image, have a considerable distance from the certainly occupied pixels 5. As a result, a high probability of the laser scanner being blind with respect to the barrier can be assumed.

[0025] Fig.Figure 11 shows a flowchart illustrating the steps of a process for fusing two grids of an environmental detection device. In step 100, information regarding any environmental objects present is collected using environmental sensors. This is done using at least two environmental sensors, the first of which is configured to generate a first grid and the second to generate a second grid. In step 200, the first and second grids are created, in which each pixel represents the occupancy status of corresponding environmental positions. Each pixel is thus assigned information indicating whether the corresponding environmental position was reported as "occupied" or "free" by the associated sensor.In step 300, a difference in the occupancy state of a first pixel of the first raster compared to a second pixel of the second raster corresponding to the first pixel of the first raster with respect to its surrounding position is characterized to identify conflict pixels. This could also be described as an occupancy state comparison of respective pixels of the first and second rasters, with the differences being designated as conflict pixels. In step 400, distances between the conflict pixels within a raster and those pixels with a positive occupancy state in the same raster are determined. In the simplest case, a number of edge lengths of the pixels between the centers of positively occupied pixels and the nearest conflict pixels can be recorded. In step 500, a corresponding distance determination is performed for the second raster, provided that conflict pixels are also present in this raster.If the second grid contains no conflict pixels, the process can be accelerated by omitting further analysis of the second grid. In step 600, the determined distances for the conflict pixels are compared with a predefined reference. The predefined reference determines the probability of a specific cause for the existence of the conflict pixels. For example, the predefined reference might specify a maximum distance of one pixel or two pixels for the cause "measurement inaccuracy." A higher probability of an underlying measurement inaccuracy might be assigned to a distance of one pixel than to a distance of two pixels.For greater distances, the probability increases that one sensor has not completely imaged the surrounding object, at least in a partial area of ​​the grid, and that its information should therefore be weighted less in a reliable determination of the collision probability, at least in this area. Finally, in step 700, the probability of a cause for the existence of the conflicting pixels is determined based on the comparison result. Reference symbol list: 1 first grid 2 second grid 3 third grid 4 Device 5 occupied pixels 6 free pixels 7 conflict pixels 8 laser scanners 9 Ultrasonic sensor 10 cars 11 electronic control unit 12 Data storage devices 13, 14, 15 surrounding objects 16. Detection area 17, 18 Guardrail 19 foreign vehicle 20 Sensor input 21 barrier 100 to 700 process steps

Claims

[1] Method for merging two obstacle maps (1, 2), hereinafter referred to as “grid (1, 2)”, comprising the steps: - Creating (200) a first raster (1) and a second raster (2), in which respective pixels (5, 6) represent an occupancy state of corresponding environment positions, - Characterizing (300) a different occupancy state of a first pixel (5) of the first raster (1) compared to a second pixel (6) of the second raster (2) corresponding to the first pixel (5) of the first raster (1) for the purpose of identifying pixels (7) hereinafter referred to as “conflict pixels (7)”, - Determining (400) distances between pixels (5, 6, 7) of the first raster (1) and pixels (5) with a positive occupancy state within the first raster (1), and / or - Determining (500) distances between pixels (5, 6, 7) of the second raster (2) and pixels (5) with a positive occupancy state within the second raster (2), the method further comprising - Compare (600) the determined distances for the conflict pixels (7) with a predefined reference and based on the comparison result - Determine (700) a probability of a cause for the existence of the conflict pixels (7). [2] Method according to claim 1, wherein the first grid (1) is created at a first time and the second grid (2) is created at a subsequent second time. [3] Method according to claim 1 or 2, wherein the first grid (1) is created from data from a first environmental sensor (8) and the second grid (2) is created from data from a second environmental sensor (9). [4] Method according to claim 3, wherein the first environmental sensor (8) and the second environmental sensor (9) correspond to a respective entry from the following list: - Ultrasonic sensor - optical camera - Thermal imaging camera - Laser - Lidar - Radar. [5] Method according to claim 3 or 4, wherein the predefined reference depends - a sensor property and / or - is defined as a state of motion of a means of locomotion (10). [6] Method according to one of the preceding claims, wherein the determination of distances of pixels (5) with a positive occupancy state is carried out for all pixels (5, 6) of the first raster (1) and the second raster (2). [7] Method according to one of the preceding claims, wherein the characterization of the different occupancy state is carried out pixel by pixel. [8] Method according to one of the preceding claims, wherein the characterization of the different occupancy state - in the form of a difference calculation of numerical values ​​characterizing the occupancy status and / or - by means of a Boolean operation. [9] Device for fusing two obstacle maps (1, 2), hereinafter referred to as “grid (1, 2)”, comprising for environment detection - an evaluation unit (11) - a sensor input (20) and - a data storage device (12), wherein - the data storage (12) is set up to provide a predefined reference, - the sensor input (20) is set up to provide environmental information, and - the evaluation unit (11) is set up, based on the environmental information - to create a first grid (1) and a second grid (2) in which respective pixels (5, 6) represent an occupancy state of corresponding environment positions, - to characterize a different occupancy state of a first pixel (5) of the first raster (1) compared to a second pixel (6) of the second raster (2) corresponding to the first pixel (5) of the first raster (1) for the purpose of identifying pixels (7) hereinafter referred to as “conflict pixels (7)”, - To determine the distances between pixels (5, 6, 7) of the first raster (1) and pixels (5) with a positive occupancy state, and / or - To determine the distances between pixels (5, 6, 7) of the second grid (2) and pixels (5) with a positive occupancy state, with the evaluation unit further configured, - to compare the determined distances for the conflict pixels (7) with a predefined reference and to use the comparison result as a basis - to determine a probability for a cause for the existence of the conflict pixels (7). [10] Means of transport comprising a device according to claim 9.

Images