How to generate universally usable feature maps

Abstract

To provide a method of creating a universally usable digital map with reduced data usage.SOLUTION: A method of creating a digital map using a control unit is provided, the method comprising: receiving measured data of the surroundings during a measuring run; identifying a trajectory of the measuring run using a SLAM method based on the received measured data; transforming the received measured data into a coordinate system of the trajectory; using the transformed measured data to create an intensity map; and extracting features from the intensity map and storing the extracted features in a feature map. A method of carrying out localization, control unit, computer program, and machine-readable storage medium are also disclosed.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method for generating a digital map and a method for performing self-localization.Furthermore, the present invention relates to a control device, a computer program and a machine-readable recording medium. [Background technology]

[0002] Localization is an essential functional element for autonomous driving of cars and robots. It allows a vehicle or robot to know its exact position within a map and environment. Based on the localization, it can generate control commands to, for example, navigate a path or perform a task.

[0003] In particular for applications without access to GNSS data, so-called SLAM allows simultaneous self-localization and mapping. For this purpose, measurement data, for example from a LIDAR sensor, are collected and analyzed in order to generate a map. In a subsequent step, the position in the map can be determined.

[0004] A challenge with SLAM is its application in dynamic or semi-static environments. Such environments may exist, for example, in storage areas, construction sites, intralogistics, or container ports. Due to the constant movement of objects, the generated maps lose their validity after a short time. The constant updating of such maps requires a large measurement and analysis effort. In particular, the regularly updated maps need to be made available to all users, which requires an infrastructure to provide large amounts of data. Summary of the Invention [Problem to be solved by the invention]

[0005] The problem underlying the present invention can be seen as being to propose a method for generating universally usable digital maps with reduced data consumption. [Means for solving the problem]

[0006] This problem is solved by the subject matter of the independent claims. Advantageous embodiments of the invention are the subject matter of the respective dependent claims.

[0007] According to one aspect of the invention, a method for generating a digital map by a control device is provided. In one step, measurement data of an environment is received during a measurement drive, where the measurement drive can be any drive. Preferably, the measurement data can also be collected when stationary or parked by at least one sensor. The corresponding measurement data can then be received and processed by the control device.

[0008] Based on the received measurement data, SLAM is performed to determine the trajectory of the measurement run, where the vehicle localizes itself based on a series of measurement data, and each position during the measurement run forms a trajectory.

[0009] In a further step, the received measurement data is transformed into the coordinate system of the trajectory. The received measurement data may, for example, comprise a relative position and / or a relative distance with respect to the sensor. The relative coordinates can then be transformed, for example, into the absolute coordinate system of the trajectory. Such a coordinate system may, for example, be a Cartesian (rectangular) coordinate system.

[0010] The converted measurement data is used to generate an intensity map. For example, the intensity of reflected light from one or more LIDAR or radar sensors may be measured and stored in the form of a map of received light intensity.

[0011] Subsequently, features are extracted from the intensity map and stored in a feature map. The features can preferably be detected from the intensity map. This process can be performed, for example, by a pattern recognition algorithm. The pattern recognition can be performed by a neural network that has been appropriately trained in advance. The pattern recognition can be performed, for example, manually by a processor or automatically. Furthermore, the processor can disable or confirm the automatic pattern recognition.

[0012] The pattern map is preferably universally usable, in particular in a sensor-independent or sensor-dominant manner, so that features can be extracted from the different measured measurement data and used for self-localization based on the feature map.

[0013] According to a further aspect of the invention, a method for performing self-localization, in particular in a control device, is provided. In one step, measurement data and a feature map of an environment are received. The measurement data may be measured by one or more sensors. Such sensors may be, for example, camera sensors, lidar sensors, radar sensors, ultrasonic sensors, etc. In particular, the sensors may be different from the sensors used to generate the feature map.

[0014] A further step involves recognising and extracting features from the received measurement data, followed by matching at least one extracted feature with the features stored in the feature map to determine the location, and upon successful matching of at least one feature the location of the sensor or the vehicle using the sensor to make the measurements can be determined.

[0015] According to a further aspect of the present invention, a control device is provided, the control device being configured to perform the above method, where the control device may be an on-board control device integrated into a vehicle control device for performing an automatic driving function, or a device connectable to the vehicle control device. Alternatively or additionally, the control device may be configured as a control device external to the vehicle, for example a server device or cloud technology.

[0016] Furthermore, according to an aspect of the invention there is provided a computer program comprising a plurality of instructions which, when executed by a computer or control device, cause the computer or control device to carry out the method according to the invention. According to a further aspect of the invention there is provided a machine-readable storage medium having stored thereon a computer program according to the invention.

[0017] In this case, the control device may be integrated into the vehicle. In particular, at least one measurement trip may be performed in a vehicle equipped with the control device. In this case, the vehicle may be driven assisted, partially automated, highly automated and / or fully automated or driverless according to the BASt standard. According to an alternative or additional embodiment, the vehicle may be a robot, a drone, a ship, etc. This allows the method to be used on roads such as highways, country roads, urban areas, and also off-road or off-road. The method can be used in particular in buildings or halls, underground spaces, parking lots and parking garages, tunnels, etc.

[0018] The at least one sensor for measuring the measurement data may be a component of the surrounding environment sensor system or at least one sensor of the vehicle. In particular, the at least one sensor may be a lidar sensor, a radar sensor, an ultrasonic sensor, a camera sensor, an odometer, an acceleration sensor, a position sensor, etc. In particular, the sensors may be used alone or in combination with one another. Furthermore, sensors such as, for example, acceleration sensors, wheel sensors, lidar sensors, ultrasonic distance sensors, cameras, etc. may be used to perform an odometric method.

[0019] The method according to the invention allows in particular to identify and extract static features of the environment. Such features may be, for example, road signs, building geometry, curbs, roads, the arrangement and position of traffic lights, delineators, lane boundaries, buildings, containers, etc. Such features can be detected by a number of different sensors and used for self-localization. For example, features extracted from measurement data of a lidar sensor may also be detected by a camera sensor and compared with each other for self-localization. Thus, a universally usable feature map can be generated which can be used by different vehicles and machines. For example, such a feature map can be used for accurate self-localization by personal mobility, transport units, manipulators, etc.

[0020] Preferably, in a first step a marking map is generated which can then be used for self-localization tasks, in particular for self-localization and control tasks of autonomous vehicles or robots.

[0021] The features of the feature map can be geometric shapes, lines, points, etc., so they can be stored in the form of coordinates or vectors with minimal data size, which can reduce the data capacity required to provide the feature map to vehicles and robots.

[0022] According to an embodiment, the feature map is stored as a digital map or as a map layer of a digital map, which allows the feature map to be used in a particularly flexible manner, in particular existing maps can be upgraded with the feature map or implemented as a digital map with minimal storage requirements.

[0023] According to a further embodiment, the received measurement data are present as a point cloud and are assigned to a raster of cells. Preferably, the measurement data of each cell are averaged to generate the intensity map. The cells of the digital map can be, for example, pixels, groups of pixels, polygons, etc. By averaging, local inconsistencies and fluctuations of the measurements can be compensated. The measurements can in particular be constituted by reflected beams from radar and / or lidar sensors or beams detected after backscattering.

[0024] According to a further embodiment, an altitude map is generated from the received measurement data, and a weighted average is calculated from the measurement data of each cell and adjacent cells to generate the altitude map, whereby additional information can be extracted from the measured measurement data and used to generate the feature map.

[0025] According to a further embodiment, information is received from the generated elevation map and stored in the feature map to determine the elevation of the extracted features. The elevation map and the feature map can then be superimposed and corresponding attributes and information of the elevation map can be transferred to the feature map. In addition, for example the height or intensity gradient at the feature location can be inherited from the respective feature. This process is preferably performed automatically, matching each cell of the elevation map with each cell of the feature map.

[0026] According to further embodiments, the extracted features are stored in a feature map as generically identifiable features. According to advantageous embodiments, the features are extracted and stored as geometric shapes, lines, points and / or point clouds, etc. In this way, objects, signs, shapes with specific shapes or features can be extracted and used from the environmental measurement data for self-localization. In particular, this allows a large number of static features to be identified and used for precise localization even in dynamic environments. Here, the marking map can be generically used, since the features can essentially be identified independently of the sensor.

[0027] According to a further embodiment, the received measurement data are realized as position data and stored in a position diagram. Preferably, the determined position is stored in the position diagram as a new measurement value if the feature is successfully matched. The feature map can be used, for example, to determine the position of a vehicle or a robot. In this case, the respective current position is determined along a path, for example at predefined time intervals, and stored in a position diagram or position map. The position diagram can display traveled distances and trajectories. If a feature is present in the feature map, the position in the feature map can be assigned to the vehicle or sensor which measured the measurement data. This position is then stored in the position diagram as another measurement value.

[0028] According to a further embodiment, the measurement data is measured from at least one sensor different from the at least one sensor used to generate the feature map. The extracted features are preferably present in an abstracted form and may thus be universally readable or comparable. Such a form of features may be present, for example, as textual coordinates. In particular, the features in the coordinates may have a start point, an end point, an intermediate point, a direction, a length, a height, etc. This information can be stored with particularly low storage requirements and used for matching.

[0029] In the following, a preferred embodiment of the invention will be described in detail using highly simplified schematic diagrams. [Brief description of the drawings]

[0030] [Figure 1] FIG. 2 is a schematic diagram of an arrangement for explaining the method according to the present invention. [Diagram 2] FIG. 2 is a schematic diagram for explaining a method for generating a digital map according to an embodiment. [Diagram 3] FIG. 2 is a schematic diagram for explaining a method for performing self-localization according to an embodiment. [Figure 4] FIG. 13 is a schematic intensity map. [Diagram 5] FIG. 2 is a schematic elevation map; [Figure 6] FIG. 2 is a perspective view of a feature map. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] FIG. 1 is a schematic diagram of arrangement 1 for illustrating methods 2 and 4 according to the present invention.

[0032] The arrangement 1 comprises two vehicles 6, 8. Alternatively or additionally, the arrangement 1 may comprise a robot and / or further vehicles. According to the illustrated embodiment, the first vehicle 6 serves to execute the method 2 for generating a digital map, in particular a marking map. The second vehicle 8 is shown diagrammatically to clarify the method 4 for performing self-localization in the digital map.

[0033] The first vehicle 6 has a controller 10 in data communication with a machine-readable memory 12 and a sensor 14. The sensor 14 may be, for example, a LIDAR sensor 14.

[0034] By means of the lidar sensor 14, the first vehicle 6 can scan the environment U and generate measurement data. The measured measurement data can then be received and analyzed by the control device 10. The feature map generated by the control device 10 can be provided to other road users and vehicles 8 via a communication link 16. The feature map may be stored in a machine-readable storage medium 12.

[0035] The second vehicle 8 similarly comprises a control device 11. The control device 11 is connected in data communication with a machine-readable storage medium 13 and with a sensor 15. According to the present embodiment, the sensor 15 is a camera sensor 15, which is also capable of measuring and transmitting measurement data of the environment U to the control device 11. The control device 11 is capable of extracting features from the measurement data of the environment U and matching them with features from a feature map received from the control device 11 via a communication link 16.

[0036] FIG. 2 is a schematic diagram for explaining a method 2 for generating a digital map according to the embodiment.

[0037] In a first step 18, measurement data of an environment U are measured during a measurement run of the first vehicle 6 and received by the control device 10. According to an embodiment, the environment U is scanned by a lidar sensor 14.

[0038] In the following step 19, SLAM is performed using the measurement data received during the measurement drive. SLAM determines the trajectory of the first vehicle 6.

[0039] The received measurement data is transformed into the coordinate system of the trajectory (20). Alternatively, the trajectory may be transformed into the coordinate system of the measurement data. For example, the common coordinate system may be a Cartesian (rectangular) coordinate system.

[0040] The transformed measurement data is used to generate an intensity map 30 (21). Such an intensity map 30 is shown in Figure 4. In particular, the measurement data may be present as a grid map having a number of cells 31, 32. The cells 31, 32 may be organized, for example, as pixels or as groups of pixels. Each cell 31, 32 may have a local assignment, such as a GPS coordinate, corresponding to a coordinate system.

[0041] Then, the intensity is calculated for each cell 31, 32. For this purpose, the average value is calculated for all the measured values ​​in each cell 31, 32. From the average values ​​thus calculated, an intensity map 30 is generated (21).

[0042] Further, an elevation map 40 is generated (22). The elevation map 40 is generated from a weighted average, which is shown in Figure 5. A weighted average is calculated for the measurements in each cell 31 and the measurement data of the corresponding adjacent cells 32.

[0043] In a further step 23, features are extracted from the intensity map 30. This can be done, for example, by automated pattern recognition algorithms or manually by an operator. For example, transitions between light and dark areas in the intensity map 30 are considered as possible patterns. Based on the elevation map 40, a profile can be assigned to each feature.

[0044] The identified features are stored (24) in a feature map 60 according to their location in the intensity map 30. The feature map 60 is shown diagrammatically in FIG. 6, in which an exemplary lidar scan is overlaid on a number of features 62, 64, 66, including exemplary roadway markings 62, roadway boundaries 64, and other markings 66 on the ground. The feature map 60 may be stored, for example, in the machine-readable storage medium 12 and provided via the communications link 16.

[0045] 3 is a schematic diagram illustrating a method 4 for performing self-localization according to an embodiment. The method 4 is exemplarily performed by a control device 11 of a second vehicle 8.

[0046] In step 25, measurement data of the environment U is measured by the sensor 15 and transmitted to the control device 11. Furthermore, the feature map 60 is received by the control device 11 via the communication link 16. This can be performed by a position diagram locator (position measuring device) implemented in the control device 11.

[0047] In this case, the measurement data can be acquired continuously or at predetermined time intervals and received by the control device 11. Furthermore, the control device 11 may receive odometric (integrated) measurement data.

[0048] In a further step 26, features 62, 64, 66 are extracted from the received measurement data. The features 62, 64, 66 are then matched (27) with the received feature map 60. During the match, an attempt is made to find the features 62, 64, 66 detected in the vehicle on the feature map 60. The measurement data measured by odometry can then limit the search area in the feature map 60. As the feature map 60 comprises abstracted and therefore universally usable features 62, 64, 66, the measurement data measured by the camera sensor 15 can also be used for self-localization.

[0049] If a match is found between the features determined at the vehicle and the features 62, 64, 66 in the feature map 60, the position of the vehicle 8 can be corrected or updated (28).

Claims

1. A method (4) for performing self-localization in an environment (U) in which a first vehicle (6) equipped with a lidar sensor (14) and a second vehicle (8) are arranged, by a control device (11) of the second vehicle (8) connected to a control device (10) of the first vehicle (6) via a communication link (16), comprising: receiving (25) measurement data of the environment (U) from at least one sensor (15) of the second vehicle (8) continuously or at predetermined time intervals, and receiving (25) a feature map (60) generated by overlaying features (62, 64, 66) detected from the environment (U) by the LIDAR sensor (14) with the scan data of the LIDAR sensor (14) from a control device (10) of the first vehicle (6) via the communication link (16); Recognizing and extracting (26) at least one feature from the received measurement data; Matching (27) the extracted at least one feature with the features (62, 64, 66) stored in the feature map (60) to determine a current location of the second vehicle (8); and upon successful matching of said features (62, 64, 66) stored in said feature map, storing said determined current position as a new measurement in a position diagram representing the trajectory of said second vehicle (8).

2. The method of claim 1 , wherein the measurement data is measured from the at least one sensor (15) different from the lidar sensor (14) used to generate the feature map (60).

3. A control device (11) configured to implement the method (4) according to claim 1 or 2.

4. A computer program comprising a number of instructions which, when executed by a computer or control device (11), causes the computer or control device to carry out the method (4) according to claim 1 or 2.

5. A machine-readable storage medium (13) having stored thereon a computer program according to claim 4.

Images