Using Cost Maps and Convergence Maps for Self-Localization and Mapping

Abstract

To provide a method for extracting additional qualitative and quantitative features for use in a mapping and / or localization.SOLUTION: A method is disclosed for ascertaining features in an environment of at least one mobile unit for implementation of a localization and / or mapping by a control unit. In the course of the method, sensor measurement data of the environment are received, the sensor measurement data received are transformed by an alignment algorithm into a cost function and a cost map is generated with the aid of the cost function, a convergence map is generated based on the alignment algorithm. At least one feature is extracted from the cost map and / or the convergence map and stored, the at least one feature being provided in order to optimize a localization and / or mapping. A control unit, a computer program, and a machine-readable storage medium are also described.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method for determining features in the surrounding environment of at least one mobile unit for performing self-localization and / or mapping by a control device. The present invention further relates to a control device, a computer program, and a machine-readable memory medium. [Background technology]

[0002] Vehicles with autonomous driving functions are becoming increasingly important in road traffic. Among other things, such vehicles have the potential to avoid congestion and accidents due to human error. Successful autonomous driving functions require robust self-localization and precise mapping of the surrounding environment.

[0003] To create a map for the execution of automated driving functions, features are extracted from the surrounding environment and stored in a digital map. Particularly unique or prominent features can be useful for subsequent self-localization in the digital map. Such one-off features can be, for example, road signs, one-off buildings or utility poles. Furthermore, there are many features that are repeated periodically. Repeated features can be constituted, for example, by lane boundaries or delineators. However, it is problematic for map generation or self-localization that the extraction or use of repeated features can lead to ambiguities. Summary of the Invention [Problem to be solved by the invention]

[0004] The problem underlying the present invention can be seen in proposing methods for the extraction of additional qualitative and quantitative features for use during mapping and / or self-localization. [Means for solving the problem]

[0005] This problem is solved by the subject matter of the respective independent claims. Advantageous configurations of the invention are the subject matter of the respective dependent dependent claims.

[0006] According to one aspect of the invention, a method is provided for determining features in a surrounding environment of at least one mobile unit for performing self-localization and / or mapping by a control device. In one step, sensor measurement data of the surrounding environment is received. Alternatively or in addition, already processed map data may also be received.

[0007] Based on the received sensor measurement data, a registration algorithm is created. A cost function is part of the registration algorithm and will be described in detail. A cost map is then created based on the cost function. The cost map may be formed, for example, as a discrete cost function. The cost map may be generated based on sensor measurement data determined by the mobile unit's surrounding environment sensor system.

[0008] A convergence map is then generated based on the registration algorithm. Unlike the cost map, the convergence map has multiple trajectories that the registration algorithm follows for different initial conditions. In this case, the initial or starting conditions of the registration algorithm relate to different positions and orientations of the mobile unit. Thus, the cost map illustrates the properties of the cost function and the convergence map illustrates the properties of the registration algorithm.

[0009] In a further step, at least one feature is determined and saved from the cost map and / or the convergence map, which is provided for optimizing the self-localization and / or mapping, where the at least one feature may be the occurrence of one or more minima in the cost map and / or the convergence map.

[0010] An important aspect during the mapping process is to be able to automatically extract ambiguous and / or unambiguous regions or features and use these additional information during mapping or self-localization. Through the use of cost and convergence maps, additional functions necessary to extend the features used in the digital map can be determined.

[0011] The cost map and / or convergence map may preferably be generated from sensor measurement data of video or camera sensors, stereo camera sensors, three-dimensional camera sensors, 360° camera configurations, LIDAR sensors, radar sensors, ultrasonic sensors, and the like. Furthermore, sensor measurement data from different sources may be combined or fused with each other to generate a composite cost map and / or convergence map.

[0012] Registration algorithms based on iterative or incremental methods can be used, such registration algorithms can be formed for example as so-called iterative closest point algorithms or incremental algorithms.

[0013] The cost map and the convergence map can be considered as a compact representation of the actual sensor measurement data, among others. This compact representation can be done, for example, by some minima in the map, which can be used as additional features of the map and by the self-localization unit during the operation of the mobile unit. These features can be used, for example, to restrict the self-localization unit for determining the position of the mobile unit in the digital map to areas where the convergence map has a unique minimum. In areas where the minima have a periodic nature, the self-localization unit can use an appropriate strategy. The self-localization unit can, for example, focus its calculations on areas of relatively low-cost values ​​of the cost map, which are observed in the cost map and / or the convergence map.

[0014] According to a further aspect of the present invention, a control device is provided, which is adapted to execute the method. The control device can be, for example, a control device arranged in a mobile unit or an external control device. The control device can be, for example, connectable with or integrated into a control device of a mobile unit for executing autonomous driving functions. An externally formed control device can be, for example, a server unit outside the vehicle based on cloud technology.

[0015] Moreover, according to an aspect of the invention there is provided a computer program comprising instructions which, when executed by a computer or control device, cause the computer or control device to carry out the method according to the invention. According to a further aspect of the invention there is provided a machine-readable memory medium on which is stored a computer program according to the invention.

[0016] In this case, the mobile unit may be assisted in accordance with BASt (German Road Traffic Institute) standards, partially automated, highly automated, and / or fully automated or capable of operating without a driver. The mobile unit may be embodied, inter alia, as a vehicle, for example as a passenger car, a bus, a commercial vehicle, a truck, and the like. Furthermore, the mobile unit may be formed as a robot, a drone, a helicopter, an airplane, a ship, a shuttle, a robotaxi, and the like.

[0017] According to one exemplary embodiment, several minima are extracted from the cost map and / or the convergence map as at least one feature. The cost map and / or the convergence map can thus be considered as a concise representation of the sensor measurement data. The cost function is a user- or application-defined function that is based on the position of the mobile unit and the received sensor measurement data. In the context of mapping, the cost function defines how precisely the sensor measurement data of different trips match each other for a given position of the mobile unit. This can be used, for example, during the alignment of the sensor measurement data of different trips. In the context of self-localization, the cost function defines how precisely the sensor measurement data match an existing map for a given position of the mobile unit.

[0018] The convergence map relates to the underlying properties of the used alignment algorithm, which is based on an iterative or incremental method, and defines trajectories starting from various initial conditions in the so-called Pose Space of the alignment algorithm, which may for example also comprise the position of the mobile unit.

[0019] According to a further embodiment, periodically occurring features are determined by multiple detected minima and singly occurring features by only one minima in the cost map and / or the convergence map. With this measure, the number of determined minima may provide an indication as to whether the extracted feature is periodically, i.e. repeating, or unique, i.e. singly occurring. Features represented by minima in the cost map may be repeated, for example, in time or space.

[0020] According to a further embodiment, the cost function is used to generate a two-dimensional or three-dimensional cost map. This allows differently sharp cost maps to be used depending on the computational power and memory available. This allows the cost function determined by the registration algorithm to be determined in different spatial directions and stored in memory.

[0021] According to a further exemplary embodiment, the sharpness and / or shape of the cost map is extracted and used for processing the sensor measurement data. In particular, the sharpness or steepness of the cost function within one or more minima can be determined. The shape of the minima indicated in the cost map can be used, for example, to save pre-processing steps during mapping or self-localization, such as cluster formation or simulation of environmental effects.

[0022] According to a further embodiment, the differences between the determined minima in the cost map are determined, whereby the determined differences in the shape and value of different local minima can be extracted and stored as further features.

[0023] In the following, preferred exemplary embodiments of the invention are explained in more detail on the basis of highly simplified schematic drawings. [Brief description of the drawings]

[0024] [Figure 1] FIG. 2 is a schematic flow diagram illustrating a method according to one embodiment. [Diagram 2] FIG. 13 is an example diagram of a cost map with one minimum. [Diagram 3] FIG. 13 is an example diagram of a convergence map with one minimum. [Figure 4] 1 is a schematic diagram of a mobile unit's surroundings to illustrate recurring features; [Diagram 5] FIG. 1 shows a schematic convergence map reflecting recurring features. [Figure 6]FIG. 13 shows further schematic convergence maps reflecting repeated features. [Figure 7] FIG. 7 is a schematic diagram of sensor measurement data of the radar sensor used to generate the convergence map of FIG. 6. [Figure 8] FIG. 13 shows a further schematic convergence map reflecting one-time occurrence features. [Figure 9] FIG. 9 is a schematic diagram of sensor measurement data of the radar sensor used to generate the convergence map of FIG. 8. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] In Fig. 1 a schematic flow diagram is shown to illustrate a method 1 according to an embodiment, which is used to determine features in an environment U of at least one mobile unit 2 for performing self-localization and / or mapping by a control device 4. The mobile unit 2 is equipped with a control device 4 and is formed as a vehicle capable of operating autonomously and is shown in Fig. 4.

[0026] In a first step 10 of the method 1, sensor measurement data of the surrounding environment U are received. The sensor measurement data can be determined, for example, by the surrounding environment sensor system 6 and can be received and evaluated by the control device 4. Alternatively, existing map data can be called up in advance.

[0027] In a further step 12, a registration algorithm is provided and a cost function is generated by the registration algorithm based on the received sensor measurement data.

[0028] A cost map 14 is then generated based on the registration algorithm and the cost function. In a further step, a convergence map 16 is generated based on the registration algorithm.

[0029] In a further step, at least one feature is extracted and stored 18 from the cost map 14 and / or the convergence map 16 .

[0030] At least one feature is then provided 20 for optimizing self-localization and / or mapping.

[0031] 2 shows an exemplary diagram of a cost map 14 with one local minimum 8. The minimum 8 has the smallest value and therefore within the range of the minimum 8 a mobile unit 2 can travel without risk of collision.

[0032] In Fig. 3, an exemplary diagram of a convergence map 16 with one minimum 8 is shown, which corresponds to the alignment algorithm used to generate the cost map 14 shown in Fig. 2. Points 22 indicate starting points for the iterations. Lines or trajectories 24 correspond to the directions in which the respective points 22 converge. Through the use of the convergence map 16, the performance of the evaluation algorithm can be illustrated.

[0033] 4 shows a schematic diagram of the surrounding environment U of the mobile unit 2 to illustrate the repeated features 26. The repeated features 26 can be for example lane markings or guardrail fixtures.

[0034] The mobile unit 2 is exemplarily embodied as a vehicle and comprises a control device 4. The control device 4 is data-transmittingly connected to an ambient environment sensor system 6 such that the control device 4 can receive sensor measurement data of the ambient environment sensor system 6.

[0035] The surrounding environment sensor system 6 may, for example, comprise camera sensors, radar sensors, LIDAR sensors, ultrasonic sensors, and the like, and may provide determined sensor measurement data to the control device 4 in analog or digital form.

[0036] Corresponding to the repeating features 26, FIG. 5 shows a schematic convergence map 16 reflecting the repeating features 26. Here, the number of minima 8 describes the uniqueness of the features 26. A unique feature 25 gives rise to only one minimum 8. A periodically occurring feature 26 gives rise to multiple local minima 8. This relationship is made clear in FIGS. 6-9.

[0037] Figures 7 and 9 show the sensor measurement data of a radar sensor. Figure 7 shows the sensor measurement data of an autobahn section with a large number of delineators, which are formed as repeating features 26. Figure 6 illustrates a corresponding convergence map 16 with a number of minima 8.

[0038] 9 shows the sensor measurement data of a radar sensor at an autobahn exit. The autobahn exit is a unique, i.e. one-time occurring, feature 25. The resulting convergence map 16 is shown in FIG. 8 and has only one minimum 8. Based on the number of minima 8, the uniqueness of the respective feature 25, 26 can therefore be inferred. [Explanation of symbols]

[0039] 1 method 2 Mobile Units 4. Control Equipment 6 Surrounding environment sensor system 8 Minima, local minima 10 Receiving sensor measurement data of the surrounding environment U 12. Converting received sensor measurement data into a cost function using the alignment algorithm 14 Cost Map 16 Convergence Map 18. Determining and storing at least one feature 25, 26 from the cost map 14 and / or the convergence map 16 20 Providing at least one feature 25, 26 for optimizing self-localization and / or mapping 22 points 24 Lines or tracks 25 Unique or one-time occurring characteristics 26 Periodic or recurring features U Surrounding Area

Claims

1. A method (1) for determining features (25, 26) in a surrounding environment (U) of at least one mobile unit (2) for performing self-localization and / or mapping by a control device (4), comprising: - receiving (10) sensor measurement data of said surrounding environment (U), - converting (12) the received sensor measurement data into a cost function by a registration algorithm and generating a cost map (14) based on said cost function; - generating a convergence map (16) based on said registration algorithm; determining (18) and storing at least one feature (25, 26) from said cost map (14) and / or said convergence map (16); providing (20) said at least one feature (25, 26) for optimizing self-localization and / or mapping; Equipped with In the step of determining (18), extracting some minima (8) from said cost map (14) and / or said convergence map (16) as at least one feature (25, 26); In said cost map (14) and / or said convergence map (16), periodically occurring features (26) are determined by a plurality of detected minima (8) and singularly occurring features (25) are determined by only one minimum (8), The unique feature (25) represents a spatially unique feature, The periodically occurring features (26) represent spatially repeated features. Method (1).

2. The method of claim 1 , wherein the cost function is used to generate a two-dimensional or three-dimensional cost map (14).

3. 3. The method according to claim 1 or 2, wherein the sharpness and / or shape of a minimum (8) in the cost map (14) is determined and used in processing the sensor measurement data.

4. 4. The method according to claim 1, wherein the difference between the minima (8) determined in the cost map (14) is determined.

5. A control device (4) adapted to carry out the method (1) according to any one of claims 1 to 4.

6. A computer program comprising commands which, when executed by a computer or a control device (4), cause said computer or said control device (4) to carry out the method (1) according to any one of claims 1 to 4.

7. A machine-readable memory medium on which the computer program according to claim 6 is stored.

Images