Method and apparatus for creating digital map, computer program, storage medium
Through the combination of generative neural network and image stitching neural network, the scanning matching method is used to splice environmental images, which solves the problems of low stitching efficiency and high data collection overhead, and achieves more efficient data set generation and neural network training.
Patent Information
- Application Number
- CN202411420977.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-10-13
- Filing Date
- 2024-10-12
- Publication Date
- 2025-04-15
AI Technical Summary
When creating digital maps, it is difficult to efficiently stitch environmental images, and a large number of real data sets are required for training, resulting in high data collection overhead.
The perturbed image is generated through a generative neural network, combined with another neural network for image stitching, and the environmental image is stitched into a digital map based on the location description using the scanning matching method.
Reduces data collection overhead, and uses automatic or data-driven learning to process interference caused by sensor noise and downstream algorithm processing, generating larger data sets for neural network training.
Smart Images
Figure CN119826795A_ABST
Abstract
Description
Technical Field
[0001] The invention relates primarily to a method for creating a digital map by stitching together environment images, which respectively represent an environment image of a vehicle environment, based on position descriptions using a scan matching method, wherein the scan matching method includes at least one neural network. Summary of the invention
[0002] The method for creating a digital map comprises: a step of receiving a first training data set, wherein each first training data set represents a first image of a corresponding environment; a step of generating a second training data set by means of a generative neural network, in which a corresponding second perturbed image of the same environment is determined for each first image; and a step of training a further neural network, which is configured to stitch together the first images of the first training data set and the corresponding second images of the second training data set according to a predetermined criterion. The method also comprises: a step of receiving an environment data set, which respectively represents an environment image of a vehicle environment, wherein the environment data set comprises a position description of the corresponding vehicle environment; a step of creating a digital map, in which the environment images are stitched together by means of a scan matching method according to the position description, wherein the scan matching method comprises at least one further neural network; and a step of providing a digital map.
[0003] A digital map is understood to be a map that is present on a storage medium in the form of (map) data values. The map is constructed, for example, to include one or more map layers, wherein one map layer displays, for example, a map from a bird's-eye view (the course and position of roads, buildings, landform features, etc.). This corresponds, for example, to a map of a navigation system. Another map layer includes, for example, a radar map, wherein the environmental features included in the radar map are stored with a radar signature. Another map layer includes, for example, a lidar map, wherein the environmental features included in the lidar map are stored with a lidar signature.
[0004] In one embodiment, the map is constructed as a high-precision map. The high-precision map is particularly constructed to be suitable for navigation of automated vehicles. This can be understood, for example, in that the high-precision map is constructed to determine the high-precision position of the automated vehicle by comparing stored environmental features with detected automated vehicle sensor data values. For this purpose, the high-precision map also includes, for example, environmental features with high-precision position descriptions (coordinates). Here, the map can be understood in particular as a global precision map.
[0005] A high-precision position can be understood as a position that is so precise in a predetermined coordinate system, such as WGS84 coordinates, that it does not exceed a maximum permissible ambiguity. The maximum ambiguity can be, for example, dependent on the environment. Furthermore, the maximum ambiguity can be, for example, dependent on whether the vehicle is operated manually, partially automated, highly automated or fully automated (corresponding to one of SAE levels 1 to 5). The maximum ambiguity is essentially so small that safe operation of an automated vehicle is ensured, in particular. For fully automated operation of an automated vehicle, the maximum ambiguity is, for example, in the order of magnitude of about 10 centimeters.
[0006] The first image of the corresponding environment can be understood as an image detected by means of sensors such as video, radar, ultrasound or laser radar. Here, each first training data set can also represent different (spatially separated) environments. Usually, the environment here is understood as an area that can be detected by means of corresponding sensors. Here, for example, it can be an area at a traffic intersection or an area along a traffic route. A traffic route is understood, for example, as a single-lane or multi-lane road. The area along the traffic route is understood, for example, as the traffic route itself and the strips on the left and / or right side of the traffic route. The width of the strip area is, for example, related to the specific configuration of the environment and / or the effective range of the sensors of each vehicle. Typically, the strip includes environmental features such as traffic signs, etc. In a possible embodiment, the first image exists in a parameterized manner with the help of polygons and / or point clouds.
[0007] The position description can be understood, for example, as a position description of the vehicle, which detects the environment with the help of environmental sensor technology. Here, it is understood as a position in a predetermined coordinate system, such as WGS84 coordinates. In one embodiment, the position description can also be the position of an object in the environment, wherein the object is included in the corresponding environmental data set. Here, the position of the corresponding object can be determined based on a combination of the position of the vehicle that detects the environmental feature and the relative position of the environmental feature relative to the vehicle, and is assigned to the environmental feature. Here, in a possible embodiment, the vehicle position of the vehicle is determined with the help of a satellite-based positioning system for the vehicle and the relative position from the corresponding environmental value set.
[0008] Environmental sensing technology is understood to be at least one video sensor and / or at least one radar sensor and / or at least one lidar sensor and / or at least one ultrasonic sensor and / or at least one other sensor, which is designed to detect the surroundings of the vehicle in the form of sensor values. In a possible specific embodiment, the environmental sensing technology includes a computing unit (processor, working memory, hard disk) with suitable software, for example, and / or is connected to such a computing unit.
[0009] Here, environmental features are understood to be, for example, infrastructure features (lane boundaries, guardrails, etc.) and / or traffic signs (road signs, traffic lights, etc.) and / or architectural features (buildings, bridges, tunnels, etc.) and / or other features that can be detected with the aid of environmental sensing technology.
[0010] Creating a digital map can be understood, for example, to reintegrate environmental features respectively included by the corresponding environmental image into the base map according to their position (here: position description) and / or to remove or adapt corresponding environmental features already included in the base map. Here, the base map is understood in particular to be a digital map (as described above).
[0011] The method according to the present invention advantageously solves the task of providing a method for creating a digital map. This task is solved in particular by means of the method according to the present invention in that new, suitable map segments are created respectively according to existing map segments (here: environmental images). The neural network used for this requires a large number of suitable data sets for training, which must be collected laboriously. With the method described here, the existing data sets can be enhanced or artificially regenerated in order to use them for training a scan matching method based on machine learning. In particular, in this way, the interference caused by sensor noise and downstream algorithm processing is learned automatically or data-driven. Therefore, the method described here also reduces the cost for collecting data, because a smaller amount of real data is sufficient to generate a larger data set composed of real data and artificial data and then use it for training the neural network. This means that the existence of a few fewer map segments can be used to generate or expand a larger data set.
[0012] The generative neural network is preferably trained beforehand using training image pairs, wherein each training image pair represents two different images of a spatial scene.
[0013] A training image pair is understood to be a pair of two images of the same spatial scene, which were previously detected by means of sensors such as video, radar, ultrasound or lidar. These images are obtained, for example, in the following way: multiple vehicles with corresponding sensors detect the same spatial scene or the same location (the same environment) and the same vehicle detects the same spatial scene multiple times. In this case, the corresponding images are in particular in the form of point clouds.
[0014] A generative neural network is, for example, a so-called variational autoencoder or variational autoencoder (VAE).
[0015] Preferably, the first training data set represents the first image and / or the second data set represents the second image and / or the environment data set represents the environment image, each in the form of a point cloud.
[0016] A point cloud is understood as a digital 2D or 3D representation of a physical object or a space or a spatial scene. It consists of a large number of individual measured points, each with an x-coordinate and a y-coordinate, or an x-coordinate, a y-coordinate and a z-coordinate.
[0017] Preferably, a digital map is provided for operating the automated vehicle.
[0018] An automated vehicle is understood to be a partially automated, highly automated or fully automated vehicle according to one of the SAE levels 1 to 5 (see standard SAE J3016).
[0019] Operating an automated vehicle is understood, for example, to mean carrying out a lateral control and / or longitudinal control of the automated vehicle, wherein the lateral control and / or longitudinal control of the automated vehicle is carried out in such a way that the automated vehicle moves along a trajectory at a predetermined speed. In a possible specific embodiment, operation also includes, for example, carrying out safety-related functions ("sharpening" airbags, fastening seat belts, etc.) and / or other (driving assistance) functions.
[0020] A trajectory is understood, for example—in relation to a map—to be a line that the (automated) vehicle follows at a predetermined speed. In one embodiment, the line is, for example, a fixed point on the vehicle. In another possible embodiment, a trajectory is understood, for example, to be a driving lane that the vehicle travels along.
[0021] The device, in particular a computing unit, is configured to carry out all steps of the method for creating a digital map. A computing unit is understood to be, for example, a server or a server cluster or a cloud.
[0022] The device or computing unit comprises a processor, a working memory, a storage medium and suitable software for implementing the method according to the invention. In addition, the device comprises an interface for sending and receiving data values by means of a wired connection and / or a wireless connection, for example with corresponding devices of the vehicle (controller, communication device, environmental sensor technology, navigation system, etc.) and / or further devices external to the vehicle (server, cloud, etc.).
[0023] Furthermore, protection is claimed for a computer program comprising instructions which, when executed by a computer, cause the computer to carry out the method according to the invention for creating a digital map. In one embodiment, the computer program corresponds to software included by the second device.
[0024] Furthermore, protection is claimed for a machine-readable storage medium on which the computer program is stored.
[0025] Advantageous developments of the invention are specified in the technical solution and listed in the description. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Embodiments of the invention are shown in the drawings and explained in more detail in the following description.
[0027] In the figure:
[0028] Figure 1 An embodiment of the method of the invention for creating a digital map is presented in the form of a flow chart. DETAILED DESCRIPTION
[0029] Figure 1 An embodiment of a method 300 for creating 350 a digital map is shown.
[0030] In step 301 , method 300 begins.
[0031] In step 310, N first training data sets x are received. i ,i=1,…,N, where each first training data set x i A first image representing a corresponding environment.
[0032] In step 320, N second training data sets y are generated by means of a generative neural network. i , i = 1, ..., N, in which the corresponding second perturbation image of the same environment is determined for each first image. Therefore, for each first training data set x i Generate a corresponding second training data set y i , where the second training data set y i Represents the first training data set x i The second image of the corresponding first image.
[0033] In step 330, a further neural network is trained which is designed to stitch together first images of the first training data set and corresponding second images of the second training data set according to a predefined criterion. Stitching is understood to mean that the corresponding first image and the second image generated for this purpose are aligned with one another. The predefined criterion corresponds here, for example, to a predefined accuracy which is achieved by training the further neural network to find or learn the best possible, adapted transformation.
[0034] In step 340 , an environment data set each representing an environment image of a vehicle environment is received, wherein the environment data set includes a position description of the corresponding vehicle environment.
[0035] In step 350, a digital map is created by stitching together the environment images based on the location descriptions using a scan matching method, wherein the scan matching method includes at least the further neural network. In a possible embodiment, the scan matching method also includes a linear transformation and / or rotation of the environment images.
[0036] In step 360, a digital map is provided.
[0037] In step 370 , method 300 ends.
Claims
1. A method (300) for creating (350) a digital map, comprising: - receiving (310) a first training data set, wherein each first training data set represents a first image of a corresponding environment; - generating (320) a second training data set by means of a generative neural network, in that for each first image a corresponding second perturbed image of the same environment is determined; - training (330) a further neural network, the further neural network being configured to, splicing the first image of the first training data set and the corresponding second image of the second training data set according to a predetermined criterion; - receiving (340) environmental data sets, the environmental data sets respectively representing environmental images of a vehicle environment, wherein the environmental data sets include a location description of the corresponding vehicle environment; - creating (350) a digital map by stitching the environment images together based on the location descriptions using a scan matching method, wherein the scan matching method includes at least the further neural network; and - Providing (360) the digital map.
2. The method (300) according to claim 1, characterized in that: The generative neural network is trained in advance using training image pairs, wherein each training image pair represents two different images of a spatial scene.
3. The method (300) according to claim 1, characterized in that: The first training data set represents the first image and / or the second data set represents the second image and / or the environment data set represents the environment image, respectively represented in the form of point clouds.
4. The method (300) according to claim 1, characterized in that: The digital map is provided for operating an automated vehicle.
5. A device, in particular a computing unit, which is configured to carry out all steps of the method (300) according to one of claims 1 to 4.
6. Computer program comprising instructions which, when executed by a computer, cause the computer to perform the method (300) according to one of claims 1 to 4.
7. A machine-readable storage medium on which the computer program according to claim 6 is stored.