Generating a lane-level hd map

Abstract

The invention relates to a computing device (10) and a method for generating a lane-level HD map, the method comprising the steps of: receiving geometric perception data recorded during a single trip of a vehicle; aggregating the received geometric perception data; down-sampling locations into equally sized regions, wherein each equally sized region comprises a subset of the aggregated geometric perception data; encoding the subset of the aggregated geometric perception data into a respective latent feature space; sampling center points from a trip trajectory serving as an approximate representation of a lane center; querying left and right boundary points from the respective latent feature space with the center points to provide a lane cross-section; performing link prediction under latent feature space conditions to derive a likelihood of two center points being connected, wherein the link prediction defines a lane graph; generating a complete lane model by using the lane cross-section and its connectivity provided by the lane graph to provide a lane-level HD map.

Description

Technical Field

[0001] This invention relates to the generation of high-definition (HD) maps, particularly in the automotive field. More specifically, this invention relates to a method for generating lane-level HD maps for motor vehicles. Furthermore, this invention relates to a corresponding computer program product and a computing device. Background Technology

[0002] HD maps with accurate and up-to-date lane-level information are crucial for many driver assistance systems and autonomous driving. For example, HD maps, sensor perception, or a combination thereof are often used to locate vehicles relative to HD maps in order to plan trajectories.

[0003] However, the current HD map creation process is time-consuming and inefficient due to the numerous manual steps required. Furthermore, HD maps often lack timeliness due to lengthy manual update cycles. Typically, HD maps are generated using vehicle sensor data and GNSS systems. The vehicle sensor data only includes geometric information, lacking topological relationships between geometric perceptions, and the GNSS accuracy can result in high noise and misaligned sensor data, making it challenging to identify individual driving lanes. Summary of the Invention

[0004] Therefore, the object of the present invention is to provide an improved method for generating HD maps, a computer program product, and a computing device.

[0005] This objective is achieved by the method, computer program product, and computing device according to the independent claim. Advantageous embodiments are presented in the dependent claims.

[0006] One aspect of the present invention relates to a method for generating lane-level HD maps, the method comprising the following method steps performed by a computing device. The method includes the steps of: receiving geometric sense data recorded at a location during a single trip of each vehicle; aggregating the geometric sense data received from each single trip; downsampling the location into regions of equal size, wherein each region of equal size comprises a subset of the aggregated geometric sense data; encoding the subset of aggregated geometric sense data for each region of equal size into a respective latent feature space; and sampling a center point from a vehicle travel trajectory used as an approximate representation of the lane center. The method further includes: querying left and right boundary points from the respective latent feature spaces using the center points to provide a lane cross-section at the location of the aforementioned points; performing link prediction under latent feature space conditions to derive the probability that two center points are connected, wherein the link prediction defines the lane map; and generating a complete lane model of the location using the lane cross-section and the connectivity provided by the lane map to provide a lane-level HD map.

[0007] In other words, geometric perception data recorded during vehicle movement can be sent to a computing device to generate an HD map. Geometric perception data can be measured by the vehicle's corresponding sensor devices at the location on the road where the HD map is to be created during movement. That is, geometric perception data can be raw sensor data or features already extracted from raw sensor data. For example, geometric perception data may include images of the road or location, particularly camera images and / or LiDAR images, and / or geometric perception data may include features and / or objects extracted from raw sensor data, such as lane markings. In particular, geometric perception data may include sparse vehicle observations rather than dense sensor measurements. Furthermore, geometric perception data from subsequent vehicles can also be sent to the computing device to further update and improve the HD map.

[0008] The computing device may include one or more processors and software to perform methodological steps for generating HD maps. The goal is to derive the underlying lane map as a set of lane pairs (lane cross-sections) and their connectivity. For example, the computing device may be a personal computer, laptop computer, server, or any other data processing device that can be used for HD map generation. In particular, the computing device may be wirelessly connected to the vehicle to receive geometry perception data, wherein geometry perception data received from each individual drive can be aggregated at the computing device.

[0009] The computing device can then create an HD map using the aggregated sensing data, particularly through fully automated AI-based HD map creation. Specifically, the computing device can receive a set of driving tracks and a set of observed lane boundaries derived from the sensing data. These driving tracks and lane boundaries can be provided as polylines defined as a sequence of multipoints. For this purpose, the location or area to be mapped can be divided into equally sized regions or submaps, such that each equally sized region or submap includes a subset of the aggregated geometric sensing data. Thus, the problem can be decomposed into several smaller problems, where only a subset of data in small geographic areas (equal-sized regions) needs to be considered. This reduces computational workload, allowing the analysis results from the individual equally sized regions to be combined again at the end.

[0010] Then, subsets of aggregated geometrically sensed data for each region of equal size can be encoded into their respective latent feature spaces. The latent feature space can be a multidimensional space, where each dimension represents a latent feature or attribute of the geometrically sensed data. These features may not be directly observed but can be inferred from the geometrically sensed data through techniques such as dimensionality reduction or unsupervised learning. For example, latent features can be properties of the geometrically sensed data that represent patterns, such as road lane markings or sidewalks, and / or latent features can be semantic data, such as the color of lane markings. Specifically, this encoding into the latent feature space can be performed by pre-trained artificial intelligence, which can be pre-trained using geometrically sensed data from previous driving, particularly using features or objects extracted from camera images of vehicles traveling on the road.

[0011] Furthermore, center points can be derived from vehicle trajectories, which can approximate the lane center and / or the vehicle's actual travel path. These center points are then used as queries for left and right boundary points from their respective latent feature spaces via transformer decoding. In particular, aligned trajectories (center points) can be used as starting points for predicting lane cross-sections (lane pairs) defined by the left and right boundary points. For example, the Lane Model Transformer Network (LMT-Net) can be used as an encoder-decoder neural network architecture that performs piecewise linear encoding and predicts lane cross-sections and their connectivity.

[0012] Furthermore, link prediction, or prediction of lane connectivity, can be performed under latent feature space conditions to derive the probability that any two center points are connected. This may involve predicting the connection or link between two entities (in this case, center points) based on their latent features. These link predictions can define a lane map that includes longitudinal connectivity.

[0013] Finally, the lane-level HD map for this location can be provided by lane cross-sections (lane pairs) and lane maps. Specifically, the lane-level HD map can include longitudinal connectivity provided by the lane maps and lateral connectivity provided by the lane cross-sections. The advantage of this method is that it allows for the creation of lane-level HD maps directly from the raw vehicle sensor inputs without geometric alignment, thus improving efficiency. Furthermore, a closed-loop map learning cycle can be provided to ensure that the HD map is highly timely once sufficient new geometric perception data is available for map creation. Moreover, since the labels are created based on data-driven prior selection, the quality of the HD map can be improved by using geometric perception data from multiple vehicle drives, without requiring ground truth baseline data.

[0014] According to the implementation scheme, geometric perception data refers to features and / or objects extracted from raw sensor data of sensor devices, particularly features and / or objects extracted from the sensor devices of the corresponding vehicle. For example, the vehicle's sensor devices could be cameras or LiDAR, where raw sensor data can be analyzed to extract features and / or objects, such as detected road markings. These extracted features and / or objects can be sent to a computing device for further processing. This has the advantage that the size of the transmitted geometric perception data can be much smaller compared to the raw data, enabling efficient data upload from the vehicle and aggregation on the computing device.

[0015] In another implementation, the geometric sensing data is determined from camera data. That is, the camera data may include images of locations or roads that are analyzed to derive the geometric sensing data, particularly features and / or objects extracted from the images.

[0016] In another implementation, locations are downsampled to regions of equal size, each a 50m x 50m tile. In other words, the aggregated geometric sense data can be divided into 50m quadrants in both the horizontal and vertical directions before encoding the downsampled geometric sense data into the latent feature space. The advantage of this downsampling is that the problem is divided into smaller sub-problems, which helps in determining the HD map.

[0017] In another implementation, encoding into the latent feature space is performed by a self-attention-based transformer architecture. Self-attention-based transformer architectures are mechanisms used in machine learning, such as in computer vision tasks. Specifically, dependencies and relationships within the input sequence of geometrically sensed data can be captured by a self-attention transformer. The self-attention transformer transforms the input sequence into vectors, specifically into three vectors via a linear transformation. A weighted sum based on the similarity values ​​between the vectors can then be computed. The resulting weighted sum, along with the original input, can then be passed through a feedforward neural network to generate a latent feature space that focuses on relevant information and long-range dependencies. In particular, the self-attention-based transformer is used to inform each encoded boundary point (polyline) of information from other boundary points. The advantages of using a self-attention-based transformer architecture are its ability to identify complex patterns and dependencies and its ability to compute multiple downsampled regions of geometrically sensed data in parallel, making it computationally efficient and scalable for large datasets.

[0018] In another implementation, transformer decoding is performed by decoding from the corresponding latent feature space, querying the left and right boundary points from the center point. Transformer decoding is a neural network architecture that may include multi-layered self-attention and feedforward neural networks. Specifically, the center point can be provided to the decoder as a query, whereby the decoder can then encode the left and right boundary points, for example, based on a polyline provided by the geometrically sensed data. Here, the decoder can primarily be used for cross-attention. This enables fast and automatic analysis of the geometrically sensed data.

[0019] In another implementation, the center point is sampled at predefined distances. In other words, the aforementioned point can be sampled every x meters (e.g., every 1 meter, every 2 meters, every 5 meters, or every 10 meters), where the center point lies on the corresponding vehicle's travel trajectory. However, the interval at which the center point is sampled does not need to be fixed, and sampling can also be performed based on curvature. Thus, these center points provide an approximate representation of the lane center, and the desired resolution can be set by predefined distances.

[0020] In another implementation, link prediction for the center point is defined in the form of an adjacency matrix in the lane graph. The adjacency matrix can be a square matrix used to represent the graph, where matrix elements indicate whether pairs of nodes (or, in this case, the center point) are adjacent. The advantage of this implementation is that it allows for a simple and efficient way to verify the connectivity of the center point.

[0021] Another aspect of the invention relates to a computer program product comprising program code components for performing the method according to the preceding aspect. This computer program product can also be considered a computer program. In particular, the computer program product can be stored on a non-transitory computer-readable storage medium. That is, the non-transitory computer-readable storage medium may include at least the computer program product according to the preceding aspect.

[0022] Furthermore, the present invention relates to a computing device configured to perform the method according to the preceding aspect. The advantages and embodiments already described with respect to the method also apply to this computing device.

[0023] Other advantages, features, and details of the invention arise from the following description of embodiments and the accompanying drawings. Features and combinations of features previously mentioned in the specification, as well as features and combinations of features mentioned in the following description of the drawings and / or shown separately in the drawings, may be used not only in the combinations indicated, but also in any other combination or individually without departing from the scope of the invention. Attached Figure Description

[0024] The novel features and characteristics of this disclosure are set forth in the appended claims. The accompanying drawings, which are incorporated in and form a part of this disclosure, illustrate exemplary embodiments and, together with the description, serve to explain the principles disclosed. In the drawings, the same numerals are used throughout to denote the same features and components. Some embodiments of the systems and / or methods according to this subject matter are now described below by way of example only and with reference to the accompanying drawings.

[0025] In the attached diagram:

[0026] Figure 1 These are motor vehicles and computing devices according to exemplary embodiments;

[0027] Figure 2 It is a schematic flowchart of an implementation scheme of a method.

[0028] In the accompanying drawings, the same elements or elements having the same function are represented by the same reference numerals. Detailed Implementation

[0029] In this document, the word "exemplary" is used to mean "serving as an example, instance, or illustration." Any implementation or specific embodiment of this subject matter described herein as "exemplary" should not be construed as being more preferred or advantageous than other implementations.

[0030] While this disclosure is readily adaptable to various modifications and alternatives, specific embodiments thereof have been illustrated by way of example in the accompanying drawings and will be described in detail below. However, it should be understood that this disclosure is not intended to be limited to the specific forms disclosed; rather, it will cover all modifications, equivalents, and alternatives falling within its scope.

[0031] The term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that an arrangement, apparatus, or method that includes a list of components or steps may include not only those components or steps but also other components or steps not expressly listed or inherent to such arrangement, apparatus, or method. In other words, one or more elements in a system or apparatus that begin with "comprising" do not exclude the presence of other elements or additional elements in the system or method without further limitations.

[0032] In the following detailed description of embodiments of this disclosure, reference is made to the accompanying drawings, which form part of the invention, and specific embodiments in which this disclosure may be practiced are illustrated by way of illustration. This embodiment is described in sufficient detail to enable those skilled in the art to practice this disclosure, and it should be understood that other embodiments may be utilized and changes may be made without departing from the scope of this disclosure. Therefore, the following description should not be considered limiting.

[0033] Figure 1 A schematic system with computing device 10 and motor vehicle 12 is shown. Computing device 10 may be a server, for example on the Internet, and may be wirelessly connected to motor vehicle 12 via a data connection. Computing device 10 may be configured to generate lane-level HD maps using geometric perception data, particularly based on features and / or objects (such as lane boundaries or lane markings) inferred from sensor devices 14 of motor vehicle 12.

[0034] For example, sensor device 14 is a vehicle camera, and geometric perception data may include driving tracks and observed lane boundaries. In particular, geometric perception data can be determined at specific locations (e.g., roads that should be mapped) during the movement of vehicle 12.

[0035] The geometric perception data recorded from sensor device 14 can be transmitted to computing device 10 via transceiver device 16 of vehicle 12. For example, transceiver device 16 may be a wireless modem connected to the Internet.

[0036] Preferably, the recorded geometric perception data from multiple vehicles are sent to the computing device 10 in this manner. The computing device 10 may be configured to aggregate the geometric perception data received from multiple vehicles and downsample the aggregated geometric perception data into smaller regions or submaps of equal size, such as 50m × 50m tile sizes.

[0037] The computing device 10 can then encode the downsampled geometrically sensed data region (e.g., sensed lane boundaries) into a latent feature space, specifically via a self-attention-based transformer. Furthermore, center points can be sampled from a driving trajectory serving as an approximate representation of lane centers at predefined distances. These center points can then be used to query left and right boundary points, respectively, by transformer decoding from the latent feature space. Two lane boundary points can represent lane cross-sections or lane pairs at the location of that particular aforementioned center point. Additionally, to derive the topology, the computing device 10 can perform link prediction of center points under latent feature space conditions to deduce the probability that two center points are connected. The resulting predictions can define a lane map in the form of an adjacency matrix.

[0038] As a result, computing device 10 predicts lane cross-sections and their connectivity, which allows for the construction of a lane-level HD map of the location using topological information including longitudinal and lateral connectivity.

[0039] The generated lane-level HD map can then be returned to the vehicle 12, for example, via transceiver 16, where the vehicle 12's assistance or autonomous driving system can use the generated lane-level HD map. This method allows for the direct derivation of the lane-level HD map from the original vehicle sensor inputs without requiring geometrically aligned data, thus improving efficiency.

[0040] Figure 2 A schematic flowchart of an implementation of a method for generating lane-level HD maps is shown. In step S1, computing device 10 may receive geometric perception data recorded during a single trip of the vehicle. The geometric perception data may include features and / or objects extracted from raw sensor data of sensor device 14 (e.g., from a vehicle camera). In particular, the geometric perception data may include driving traces and observed lane boundaries provided as polylines.

[0041] In step S2, geometric perception data received from multiple vehicles can be aggregated in computing device 10.

[0042] In step S3, the aggregated geometric perception data is downsampled into smaller regions of equal size, for example, a smaller region of 50m×50m tile size.

[0043] In step S4, for example via a self-attention-based transformer, smaller regions of equal size that have been downsampled are encoded into the latent feature space.

[0044] In step S5, the center point is sampled from the vehicle's trajectory, which serves as an approximate representation of the lane center. The center point can be sampled every x meters (e.g., every 1 meter, every 2 meters, or every 5 meters).

[0045] In step S6, the left and right boundary points of the lane are determined for each center point using a latent feature space. Specifically, transformer decoding can be performed from the latent feature space to query the corresponding left and right boundary points. Thus, the lane cross-section can be provided.

[0046] In step S7, the probability of two center points being connected can be derived by performing link prediction under latent feature space conditions to determine the lane map. For example, these link predictions can be provided in the form of an adjacency matrix.

[0047] Finally, in step S8, the lane map, along with the corresponding lane cross-section, provides a lane-level HD map for that specific location.

[0048] In other words, this invention uses sparse vehicle observations instead of dense sensor measurements, which include aligned driving tracks (center points) and lane boundaries. Preprocessing steps may include generating polylines by aligning and aggregating observed lane boundaries. The aligned driving tracks can be used as starting points for predicting lane pairs (lane cross-sections) using left and right boundary points. For example, a lane model transformer network can be used as an encoder-decoder neural network architecture that performs polyline encoding and prediction of lane pairs and their connectivity. A lane graph can be formed by using the predicted lane pairs as nodes and the predicted lane connectivity as edges.

[0049] Specifically, this map learning loop can be repeated once sufficient new geometric perception data is available. This ensures that the HD map has very high timeliness. Therefore, this invention provides lane geometry and semantic map learning based on crowdsourced sensor data.

[0050] Figure Labels

[0051]

Claims

1. A method for generating lane-level high-precision maps, the method comprising the following steps performed on a computing device (10): - Receive geometric sensing data recorded at a location during a single trip of each vehicle; - Aggregate geometric perception data received from each individual drive; - The location is downsampled into regions of equal size, wherein each region of equal size comprises a subset of the aggregated geometrically sensed data; - Encode a subset of aggregated geometric perception data for each region of equal size into its respective latent feature space; - Sample the center point from the vehicle's trajectory, which is used as an approximate representation of the lane center; -Use the center point to query the left and right boundary points from their respective latent feature spaces to provide the lane cross-section at the location of the aforementioned point; - Perform link prediction under the latent feature space conditions to derive the probability that two center points are connected, wherein the link prediction defines the lane map; - A lane-level high-precision map of the location is provided by the lane cross-section and the lane diagram.

2. The method according to claim 1, wherein the geometric sensing data is features and / or objects extracted from raw sensor data of the sensor device (14).

3. The method according to claim 1 or 2, wherein the geometric sensing data is determined from camera data.

4. The method according to any one of claims 1 to 3, wherein the location is downsampled to an area of ​​equal size, which is 50m × 50m.

5. The method according to any one of the preceding claims, wherein the encoding into the latent feature space is performed by a transformer architecture based on a self-attention mechanism.

6. The method of claim 5, wherein the left boundary point and the right boundary point are queried from the aforementioned points by transformer decoding from the respective latent feature spaces.

7. The method according to any one of the preceding claims, wherein the center point is sampled at a predefined distance.

8. The method according to any one of the preceding claims, wherein the link prediction of the center point defines the lane map in the form of an adjacency matrix.

9. A computer program product comprising code components for performing the method according to any one of the preceding claims.

10. A computing device (10) wherein the computing device (10) is configured to perform the method according to any one of claims 1 to 8.

Images