An online high-definition map generation method and device for an autonomous driving task

Abstract

This application discloses an online high-precision map generation method and apparatus for autonomous driving tasks. The method includes: mapping multiple data features extracted from multi-source input data to the same channel dimension to obtain multi-source features of the same dimension; converting the image features of each vehicle-mounted multi-view camera to a bird's-eye view, fusing them to obtain the original bird's-eye view features, embedding location codes into the original bird's-eye view features and fusing them with standard map features to obtain a first fused feature; embedding location codes into satellite image features and fusing them with the first fused feature, then performing convolution enhancement to obtain a second fused feature; inputting the second fused feature into a multi-branch structured map prediction head to obtain a high-precision map that integrates multi-source information. This solves the problem that existing high-definition map generation and updating methods cannot effectively integrate complementary information from various data sources, resulting in high-definition maps whose accuracy, completeness, and efficiency cannot meet the needs of practical use. Therefore, a high-precision map with higher accuracy and efficiency and more complete information is obtained.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an online high-precision map generation method and apparatus for autonomous driving tasks. Background Technology

[0002] High-definition maps are a core support for the stable operation of autonomous driving systems, providing accurate geometric and semantic information, covering key road elements such as lane dividers, road boundaries, and pedestrian crossings. This information serves as the foundational prior knowledge for core autonomous driving tasks (including precise positioning and path planning), directly determining the positioning accuracy, path planning rationality, and operational safety of the autonomous driving system. However, real-world road environments are dynamically changing, and road elements change over time. Traditional offline map annotation methods have significant limitations, being not only costly but also time-consuming, failing to meet the real-time and accurate map information requirements of autonomous driving. Therefore, there is an urgent need to explore efficient and low-cost solutions for high-definition map construction and updating.

[0003] To address the shortcomings of traditional offline methods, existing research primarily focuses on inexpensive and readily available data sources to explore pathways for constructing high-definition maps. The core data sources fall into three categories, corresponding to three implementation methods. The first method uses images from vehicle-mounted multi-view cameras as the core data source, constructing high-definition maps online. It extracts road elements and generates map data in real-time from images collected during vehicle movement. The second method introduces standard-defined maps as prior information, utilizing their structured information such as road centerlines and lane topology connections to enhance road structure reasoning capabilities and assist in high-definition map construction from vehicle-mounted multi-view camera images. The third method incorporates satellite imagery into high-definition map construction to compensate for blind spots in vehicle-mounted observations. Leveraging its wide-area coverage, it improves the completeness of road information and enhances the comprehensiveness of map construction.

[0004] Although existing research has explored various methods for constructing high-resolution maps based on inexpensive data sources, several shortcomings remain to be addressed. Specifically, methods based on vehicle-mounted multi-view camera images are limited by viewpoint mismatch, easily leading to problems such as positional shifts and shape distortions in map data, affecting map accuracy. Methods incorporating standard-defined map priors, while enhancing road structure reasoning capabilities, still fall short in predicting road details and cannot accurately capture subtle changes in road elements. Methods incorporating satellite imagery lack the topological constraints provided by standard-defined maps, making it difficult to accurately depict road shapes in occluded areas, resulting in localized map information distortion. Furthermore, significant differences exist between vehicle-mounted multi-view camera images, standard-defined maps, and satellite imagery in terms of viewpoint dimension and coordinate system. These differences make accurate cross-source alignment difficult, hindering the effective fusion of complementary information from various data sources and limiting the accuracy, completeness, and efficiency of high-resolution map construction. Summary of the Invention

[0005] This application provides an online high-precision map generation method and apparatus for autonomous driving tasks, which solves the problem that existing high-definition map generation and updating methods cannot effectively integrate complementary information from various data sources, resulting in high-definition maps whose accuracy, completeness, and efficiency cannot meet the needs of actual use.

[0006] In a first aspect, embodiments of this application provide an online high-precision map generation method for autonomous driving tasks, comprising: preprocessing collected multi-source data to obtain multi-source input data with unified time, space, and coordinate references; extracting features from the multi-source input data and mapping the extracted multiple data features to the same channel dimension to obtain same-dimensional multi-source features; wherein, the same-dimensional multi-source features include vehicle-mounted multi-view camera image features, standard map features, and satellite image features; converting each of the vehicle-mounted multi-view camera image features to a bird's-eye view to obtain candidate bird's-eye view features from multiple perspectives, and fusing them to obtain original bird's-eye view features; embedding position codes into the original bird's-eye view features and the standard map features, and fusing them through an attention mechanism to obtain a first fusion feature that fuses real-time visual features and standard road structure information; embedding position codes into the satellite image features and fusing them with the first fusion feature through an attention mechanism, and then performing convolution enhancement to obtain a second fusion feature; inputting the second fusion feature into a multi-branch structured map prediction head, and performing fusion verification on the output results of each branch to obtain a high-precision map that fuses multi-source information.

[0007] In conjunction with the first aspect, in one possible implementation, the multi-source data includes images from an onboard multi-view camera, a standard map, and satellite images. The preprocessing of the collected multi-source data to obtain multi-source input data with unified time, space, and coordinate references includes: aligning the onboard multi-view camera images, standard map, and satellite images with the vehicle's position using the vehicle's current timestamp as the time reference; establishing a local reference coordinate system centered on the vehicle's position; cropping multiple local slices from the standard map based on the vehicle's position to cover the vehicle's current perception range; mapping the vehicle's position to the satellite image coordinate system based on the vehicle's pose information; rotating and correcting the satellite image with the mapped point as the center and according to the vehicle's heading angle to obtain local satellite image patches; and pairing the local satellite image patches with the standard map to form multi-source input data with unified time, space, and coordinate references.

[0008] In conjunction with the first aspect, in one possible implementation, the step of extracting features from the multi-source input data and mapping the extracted multiple data features to the same channel dimension to obtain same-dimensional multi-source features includes: using a convolutional neural network and a visual Transformer to independently encode vehicle-mounted multi-view camera images from each viewpoint, outputting a multi-scale image feature tensor; using a lightweight convolutional network as the backbone and a dedicated map coding network with a location embedding layer superimposed to extract prior features of road structure from a standard map; using an encoder with a global receptive field and local texture enhancement to extract fine-grained features from satellite images; and mapping the multi-scale image feature tensor, the prior features of road structure, and the fine-grained features to the same channel dimension to obtain same-dimensional multi-source features.

[0009] In conjunction with the first aspect, in one possible implementation, the step of converting the image features of each of the vehicle-mounted multi-view cameras to a bird's-eye view to obtain candidate bird's-eye view features from multiple perspectives, and fusing them to obtain the original bird's-eye view features, includes: defining a bird's-eye view planar grid centered on the vehicle's position; determining the projection area of ​​each grid point in the bird's-eye view planar grid under the vehicle's camera perspective based on the camera parameters of the vehicle-mounted cameras; determining the local viewpoint features of the vehicle-mounted multi-view camera image features in each projection area and mapping them to the bird's-eye view planar grid to obtain candidate bird's-eye view features from multiple perspectives; dividing the bird's-eye view planar grid into multiple sectors corresponding to the main viewing direction of each vehicle-mounted camera according to orientation; determining corresponding distance weights based on the distance from each grid point in the sector to the center of the sector, and determining corresponding direction weights based on the angle between the position direction of each grid point in the sector and the main viewing direction of the corresponding vehicle-mounted camera; normalizing the distance weights and the direction weights to obtain the fusion weights of each vehicle-mounted camera at the corresponding grid point; and fusing the candidate bird's-eye view features from multiple perspectives using the fusion weights to obtain the original bird's-eye view features.

[0010] In conjunction with the first aspect, in one possible implementation, the step of embedding location codes into the original bird's-eye view features and the standard map features, and fusing them through an attention mechanism to obtain a first fused feature that integrates real-time visual features and standard road structure information, includes: aligning the original bird's-eye view features and the standard map features to the same spatial dimension, then slicing them into blocks, and embedding location codes into each feature block of the slice; performing self-attention operations on the sliced ​​and encoded original bird's-eye view features to mine semantic associations between different spatial regions in the original bird's-eye view features to obtain enhanced visual features; and performing cross-attention operations using the enhanced visual features as queries and the sliced ​​and encoded standard map features as keys and values ​​to fuse the real-time visual information of the original bird's-eye view features and the standard road structure information of the standard map features to obtain the first fused feature.

[0011] In conjunction with the first aspect, in one possible implementation, the step of embedding position codes in the satellite image features and then fusing them with the first fusion feature through an attention mechanism, followed by convolutional enhancement to obtain a second fusion feature, includes: aligning the first fusion feature and the satellite image features to the same spatial dimension; then segmenting the satellite image features and embedding position codes in each feature block of the segment; estimating the local offset between the segmented encoded satellite image features and the first fusion feature based on the vehicle position and compensating and correcting it to align the spatial position of the segmented encoded satellite image features with that of the first fusion feature; using the first fusion feature as a query and the spatially aligned satellite image features as keys and values, performing cross-attention operations to fuse the road semantic information in the first fusion feature with the global information in the satellite image features to obtain an initial fusion feature; and performing convolutional enhancement processing on the initial fusion feature to strengthen its spatial continuity and semantic consistency to obtain the second fusion feature.

[0012] In conjunction with the first aspect, in one possible implementation, the multi-branch map prediction head includes a semantic segmentation branch, an instance decoding branch, a vectorization generation branch, and a fusion decoding module. The semantic segmentation branch is configured to: extract semantic features of each grid point within the bird's-eye view planar grid based on the second fusion feature, and output a multi-category semantic probability map containing multiple semantic categories after fusion. The instance decoding branch is configured to: extract instances from the semantic features in the semantic segmentation branch to distinguish instances of different semantic categories and output the topological structure information of each instance. The vectorization generation branch is configured to: convert the multi-category semantic probability map and the topological structure information into vector format to generate vector data conforming to the high-precision map standard. The fusion decoding module is configured to: perform fusion verification on the multi-category semantic probability map, the topological structure information, and the vector data to eliminate semantic conflicts and topological errors, and decode to obtain a high-precision map that integrates multi-source information.

[0013] In conjunction with the first aspect, in one possible implementation, obtaining a high-precision map that integrates multi-source information further includes: correcting the positioning error in the output high-precision map based on the vehicle's pose information; and / or comparing the road topology information in the high-precision map and the standard map to correct any offsets or misalignments; and / or filtering the high-precision map to eliminate noise and false semantic annotations; and / or performing weighted fusion of multiple consecutive high-precision maps to utilize the spatial correlation between adjacent frames to correct random errors in a single frame.

[0014] Secondly, embodiments of this application provide an online high-precision map generation device for autonomous driving tasks, comprising: an alignment module for preprocessing collected multi-source data to obtain multi-source input data with unified time, space, and coordinate references; a mapping module for extracting features from the multi-source input data and mapping the extracted data features to the same channel dimension to obtain same-dimensional multi-source features; wherein the same-dimensional multi-source features include vehicle-mounted multi-view camera image features, standard map features, and satellite image features; and a conversion module for converting each of the vehicle-mounted multi-view camera image features to a bird's-eye view to obtain multi-view candidate birds. The system employs a first fusion module to embed location codes into the original bird's-eye view features and the standard map features, and then fuses them using an attention mechanism to obtain a first fused feature that integrates real-time visual features and standard road structure information. A second fusion module embeds location codes into the satellite image features, fuses them with the first fused feature using an attention mechanism, and then performs convolutional enhancement to obtain a second fused feature. An output module inputs the second fused feature into a multi-branch map prediction head, performs fusion verification on the output results of each branch, and obtains a high-precision map that integrates multi-source information.

[0015] Thirdly, embodiments of this application provide an apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the executable instructions, it implements the method as described in the first aspect or any possible implementation of the first aspect.

[0016] Fourthly, embodiments of this application provide a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium including storage for storing a computer program or instructions that, when executed, cause the method described in the first aspect or any possible implementation of the first aspect to be implemented.

[0017] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: This application's embodiments effectively integrate real-time visual details, standard road structure constraints, and global satellite information by constructing multi-source input data with unified time, space, and coordinate references, thus overcoming the information limitations of a single data source. By converting the image features of each vehicle-mounted multi-view camera to a bird's-eye view, it eliminates the perspective distortion and spatial misalignment caused by differences in installation position and shooting angle of the vehicle-mounted multi-view cameras, ensuring that road elements from different perspectives present a consistent spatial distribution within a unified bird's-eye view grid. The first fusion feature effectively integrates the local road details perceived by the vehicle's real-time vision with the global topological constraints of the standard road structure. The second fusion feature effectively fills the blind spots in the field of view caused by occlusion in vehicle perception, improving the integrity of high-precision map information. By extracting semantic, topological, and vector information separately and fusing and verifying them through a multi-branch map prediction head, it effectively reduces the errors caused by single-branch prediction and corrects semantic conflicts and topological errors. This effectively solves the problem that existing high-definition map generation and updating methods cannot effectively integrate complementary information from various data sources, resulting in high-definition maps whose accuracy, integrity, and efficiency cannot meet practical application requirements. This results in high-precision maps with higher accuracy and more complete information, which can better meet the actual needs of autonomous driving tasks and improve the efficiency and effectiveness of online high-precision map generation. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating an online high-precision map generation method for autonomous driving tasks, provided as an embodiment of this application; Figure 2 This is a schematic diagram of the structure of an online high-precision map generation device for autonomous driving tasks provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0021] The following description of some technologies involved in the embodiments of this application is provided to aid understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, some descriptions of well-known functions and structures are omitted in the following description.

[0022] Figure 1 This is a flowchart of an online high-precision map generation method for autonomous driving tasks provided in an embodiment of this application, including steps 101 to 106. Wherein, Figure 1 This is merely one execution order shown in the embodiments of this application and does not represent the only execution order for an online high-precision map generation method for autonomous driving tasks. Where the final result can be achieved, Figure 1 The steps shown can be performed in parallel or in reverse order.

[0023] Step 101: Preprocess the collected multi-source data to obtain multi-source input data with unified time, space, and coordinate references. In this embodiment, the multi-source data includes vehicle-mounted multi-view camera images, standard maps, and satellite images. Preprocessing the collected multi-source data to obtain multi-source input data with unified time, space, and coordinate references includes: aligning the vehicle-mounted multi-view camera images, standard maps, satellite images, and vehicle position with the vehicle's current timestamp as the time reference; establishing a local reference coordinate system centered on the vehicle position; cropping multiple local slices from the standard map based on the vehicle position to cover the vehicle's current perception range; mapping the vehicle position to the satellite image coordinate system based on the vehicle's pose information; rotating and correcting the satellite image with the mapping point as the center according to the vehicle's heading angle to obtain local satellite image patches; pairing the local satellite image patches with the standard map to form multi-source input data with unified time, space, and coordinate references.

[0024] Specifically, the first step is to perform time alignment of multi-source data. Using the vehicle's current timestamp as a unified time reference, the acquisition timestamps of the onboard multi-view camera images, the update timestamps of the standard map, the capture timestamps of the satellite images, and the location timestamps corresponding to the vehicle's positioning results are obtained. Through timestamp matching and interpolation synchronization, it is ensured that the multi-source data participating in the same round of online high-precision map generation and inference come from the same or adjacent times, eliminating spatial inconsistencies caused by time misalignment. Among them, the onboard multi-view camera images are preferably six-channel surround view images, corresponding to the vehicle's front view, rear view, left front, right front, left rear, and right rear views, respectively. Each surround view image synchronously records the intrinsic parameter matrix, extrinsic parameter matrix, exposure parameters, and acquisition timestamp. The standard map is a high-confidence vector or raster map that includes the road centerline, road boundaries, lane topology, pedestrian crossings, and stop lines; the satellite images are top-view images covering the target area with a resolution higher than a preset threshold, used to supplement global road structure and fine-grained texture information.

[0025] Next, a local reference coordinate system is established centered on the vehicle's current location. This involves selecting the origin of the vehicle's real-time positioning output (either geographic or vehicle coordinates) as the center of the coordinate system. The local reference coordinate system can be a vehicle coordinate system, a drivable area coordinate system, or a unified city coordinate system, used to unify the spatial reference for subsequent standard maps, satellite imagery, and onboard multi-view camera images. This local reference coordinate system covers a preset range around the vehicle, for example, covering 60 meters forward, 30 meters backward, and 30 meters to the left and right of the vehicle, ensuring a unified spatial boundary for subsequent clipping and mapping.

[0026] Then, multiple local slices are cropped from the standard map based on the vehicle's location to cover the vehicle's current perception range. During the cropping process, the origin and resolution of the local slices are kept consistent with the local reference coordinate system, so that the local slices fully contain the road structure, lane boundaries, and intersection topology information around the vehicle, avoiding road structure breaks or information loss due to insufficient cropping.

[0027] Next, the vehicle's position is mapped to the satellite image coordinate system based on its pose information. This pose information includes longitude, latitude, altitude, heading angle, pitch angle, and roll angle. Using pre-fitted affine transformation parameters, projection transformation parameters, or perspective correction parameters, the vehicle's position in the local reference coordinate system is mapped to the satellite image coordinate system, obtaining the mapped coordinates (i.e., the mapped point) of the vehicle's position in the satellite image. Error correction is then used to correct mapping deviations, eliminating coordinate offsets introduced by satellite image shooting angle, terrain undulations, and projection methods, ensuring that the accuracy of the vehicle's mapped position meets the requirements of subsequent registration.

[0028] Subsequently, the satellite image is rotated and corrected according to the vehicle's heading angle, using the mapped point as the center, to generate a local satellite image patch. Specifically, the satellite image is rotated based on the corrected mapped point on the satellite image and the vehicle's real-time heading angle, ensuring that the main direction of the satellite image aligns with the vehicle's direction of travel. Then, the rotated satellite image is cropped according to the spatial range of the local reference coordinate system, resulting in a local satellite image patch centered on the vehicle's position, oriented in the same direction as the vehicle's heading, and covering a preset area. The local satellite image patch has the same spatial scale and orientation as the local slice of the standard map, providing a spatial alignment basis for subsequent feature fusion.

[0029] Finally, local satellite image patches are paired and integrated with local tiles of the standard map to form multi-source input data with unified temporal, spatial, and coordinate references. Spatial registration is performed on the local satellite image patches and local tiles of the standard map to align key structural features such as road centerlines, road boundaries, and intersection outlines. Combining the temporal alignment results completed earlier, the vehicle-mounted multi-view camera images, local tiles of the standard map, and local satellite image patches are used as a complete set of multi-source input data for subsequent steps, ensuring the consistency and reliability of the input data throughout the online high-precision map generation process.

[0030] Step 102: Extract features from the multi-source input data and map the extracted features to the same channel dimension to obtain multi-source features of the same dimension; wherein, the multi-source features of the same dimension include features from vehicle-mounted multi-view camera images, standard map features, and satellite image features. In this embodiment, a convolutional neural network and a visual Transformer are used to encode the vehicle-mounted multi-view camera images independently from one view to the next, outputting a multi-scale image feature tensor; a lightweight convolutional network is used as the backbone, with a dedicated map encoding network superimposed with a location embedding layer to extract prior features of road structure from the standard map; a global receptive field and local texture enhancement encoder are used to extract fine-grained features from the satellite image; the multi-scale image feature tensor, prior features of road structure, and fine-grained features are mapped to the same channel dimension to obtain multi-source features of the same dimension.

[0031] Specifically, based on the imaging characteristics of the six surround-view images in the vehicle-mounted multi-view camera imagery, a hybrid coding network combining convolutional neural networks and visual Transformers (a deep learning model), or a combination thereof, is used to perform view-by-view independent feature encoding on the surround-view images of each viewpoint. The convolutional neural network is responsible for extracting the low-level edges, textures, and local geometric features of the surround-view images from each viewpoint (the six views of the vehicle-mounted camera), while the visual Transformer is used to model the long-range contextual dependencies within the surround-view images of each viewpoint. After view-by-view independent encoding, the output contains multi-scale image feature tensors with different resolutions and semantic levels. The tensor dimensions cover spatial height, spatial width, and the number of feature channels, fully preserving the spatial details and semantic information of the surround-view images from each viewpoint.

[0032] Furthermore, the standard map contains structured information such as road centerlines, road boundaries, lane connections, pedestrian crossings, and stop lines. A dedicated map coding network with a lightweight convolutional network as the backbone and overlaid location embedding layers is used for feature extraction. The lightweight convolutional network efficiently captures road structural features such as the road skeleton, topological orientation, and functional area distribution. The location embedding layers endow these road structural features with spatial location information, enhancing the spatial correlation of road structural features, and ultimately outputting prior features of the road structure with strong prior constraints.

[0033] Satellite imagery offers a superior top-down perspective, providing insights into lane markings, intersection shapes, median strip outlines, road textures, and environmental cues beyond occluded areas. An encoder that balances global receptive field and local texture enhancement is employed for feature extraction. The global receptive field branch captures large-scale road layouts and network topology, while the local texture enhancement branch strengthens fine-grained information such as lane line textures, curb edges, and intersection details. The fusion of these two branches outputs richly detailed and structurally complete fine-grained features.

[0034] Finally, channel dimension transformation operations are performed on the multi-scale image feature tensor, road structure prior features, and fine-grained features of satellite images, respectively. The number of channels is adjusted through 1×1 convolutional layers or fully connected layers to map the three types of features with different modalities and dimensions to a preset unified channel dimension. The spatial information of the three types of features is preserved during the mapping process. After the channel dimension is unified, a multi-source feature with the same dimension is obtained, which includes features from vehicle multi-view camera images, standard map features, and satellite image features. This provides feature input with consistent dimension and complementary information for subsequent cross-modal cross-attention fusion.

[0035] Step 103: Convert the image features of each vehicle-mounted multi-view camera to a bird's-eye view to obtain candidate bird's-eye view features from multiple perspectives, and fuse them to obtain the original bird's-eye view features. In this embodiment, a bird's-eye view planar grid centered on the vehicle position is defined; the projection area of ​​each grid point in the bird's-eye view planar grid under the vehicle-mounted camera's view is determined based on the camera parameters of the vehicle-mounted camera; the local viewpoint features of the vehicle-mounted multi-view camera image features in each projection area are determined and mapped to the bird's-eye view planar grid to obtain candidate bird's-eye view features from multiple perspectives; the bird's-eye view planar grid is divided into multiple sectors corresponding to the main viewing direction of each vehicle-mounted camera according to their orientation; the corresponding distance weight is determined according to the distance from each grid point in the sector to the center of the sector, and the corresponding direction weight is determined according to the angle between the position direction of each grid point in the sector and the main viewing direction of the corresponding vehicle-mounted camera; the distance weight and direction weight are normalized to obtain the fusion weight of each vehicle-mounted camera at the corresponding grid point; the candidate bird's-eye view features from multiple perspectives are fused using the fusion weight to obtain the original bird's-eye view features.

[0036] Specifically, a bird's-eye view planar grid centered on the vehicle's location is first constructed. Using the vehicle's current location as the origin, a two-dimensional grid, the bird's-eye view planar grid, is established on a horizontal ground plane. This grid covers a preset area around the vehicle, exemplarily set to 60 meters forward, 30 meters backward, and 30 meters to the left and right. The grid cell size can be set to 0.5m × 0.5m or 1m × 1m depending on accuracy requirements, to uniformly carry the feature information projected from various viewpoints.

[0037] Next, camera parameters for each vehicle-mounted camera are extracted, including intrinsic and extrinsic parameter matrices, mounting height, pitch angle, yaw angle, roll angle, and field of view, to construct a camera projection model. For each grid point in the bird's-eye view planar grid, an inverse perspective mapping algorithm is used to calculate the projection area of ​​that grid point on the corresponding vehicle-mounted camera's imaging plane, establishing a one-to-one correspondence from the bird's-eye view space to the camera image space, providing a geometric basis for subsequent feature mapping.

[0038] Then, the panoramic image features of each viewpoint in the vehicle-mounted multi-view camera image features are mapped to the bird's-eye view planar grid to generate candidate bird's-eye view features from multiple views. Specifically, for each viewpoint, based on the aforementioned projection area, local viewpoint features within the projection area are extracted from the panoramic image features of the corresponding viewpoint. Using interpolation sampling or learnable mapping methods, the local viewpoint features are shifted pixel by pixel and filled into the corresponding positions on the bird's-eye view planar grid, generating a complete candidate bird's-eye view feature map independently for each viewpoint.

[0039] Then, based on the installation position and main viewing direction of each vehicle-mounted camera, the entire bird's-eye view plane grid is divided into angle partitions. For example, the bird's-eye view plane grid is divided into six sectors: front view, rear view, left front, right front, left rear, and right rear. Each sector corresponds to the main viewing coverage area of ​​a vehicle-mounted camera. The sector boundaries are determined based on the field of view angle and installation angle of the vehicle-mounted camera to ensure that each sector matches the observation area of ​​the vehicle-mounted camera one by one.

[0040] Next, for any grid point within each sector, the Euclidean distance from that grid point to the geometric center of its sector is calculated. The smaller the Euclidean distance, the closer the grid point is to the main viewing center of the vehicle camera, and the higher the distance weight is assigned. At the same time, the angle between the azimuth direction of the grid point relative to the sector center and the corresponding main viewing direction of the vehicle camera is calculated. The smaller the angle, the more consistent the observation direction, and the higher the direction weight is assigned. The distance weight and direction weight both range from (0,1) and are used to characterize the reliability contribution of different local viewpoint features in spatial location.

[0041] Subsequently, the distance and orientation weights are normalized to obtain the fusion weights for each vehicle camera. The distance and orientation weights of all vehicle cameras corresponding to the same grid point are multiplied to obtain the initial joint weights. The initial joint weights of all vehicle cameras at that grid point are then globally normalized to ensure that the sum of all fusion weights at the same grid point is 1, eliminating the weight overflow problem in multi-view overlapping areas. Finally, the fusion weights of each vehicle camera at that grid point are obtained.

[0042] Finally, for each grid point in the bird's-eye view planar grid, the candidate bird's-eye view features corresponding to each vehicle camera are multiplied by the fusion weight corresponding to that grid point and then weighted and summed. This global weighted fusion is completed by traversing all grid points. Then, channel compression and local smoothing are performed through one or more convolutional fusion modules to eliminate misalignment, ghosting, and feature conflicts between viewpoints. The final output is the original bird's-eye view features that are spatially continuous, semantically consistent, and detailed.

[0043] Step 104: Embed location codes into the original bird's-eye view features and standard map features, and fuse them through an attention mechanism to obtain a first fused feature that integrates real-time visual features and standard road structure information. In this embodiment, the original bird's-eye view features and standard map features are aligned to the same spatial dimension, then segmented, and location codes are embedded in each feature block of the segment. Self-attention operation is performed on the original bird's-eye view features after segmentation and encoding to mine the semantic associations between different spatial regions in the original bird's-eye view features, thereby obtaining enhanced visual features. The enhanced visual features are used as queries, and the segmented and encoded standard map features are used as keys and values, respectively, to perform cross-attention operation to fuse the real-time visual information of the original bird's-eye view features and the standard road structure information of the standard map features, thereby obtaining the first fused feature.

[0044] Specifically, the original bird's-eye view features and the standard map features are first aligned to the same spatial dimension. The original bird's-eye view features are generated by fusing candidate bird's-eye view features from multiple perspectives, while the standard map features are obtained by encoding structured map data. The initial spatial resolution and size of the two may be inconsistent. Through bilinear interpolation or nearest neighbor interpolation algorithms, the smaller features are upsampled and the larger features are downsampled to make the spatial height and spatial width of the original bird's-eye view features and the standard map features completely consistent, thus achieving spatial dimension alignment and ensuring that the spatial positions of subsequent tiling operations correspond one-to-one.

[0045] Next, the spatially aligned original bird's-eye view features and standard map features are slicing operations. The aligned original bird's-eye view features and standard map features are uniformly divided into multiple non-overlapping feature blocks according to a preset block size. For example, the feature block size is set to 16×16 pixels, and blocks are sequentially sliced ​​along the height and width directions of the original bird's-eye view features or standard map features. Each type of feature generates a feature block sequence composed of multiple spatially ordered feature blocks. The slicing process maintains the spatial adjacency relationship between feature blocks, providing ordered input for subsequent location encoding and attention calculation.

[0046] Then, location codes are embedded in each feature block. For the feature block sequences of the original bird's-eye view features and the feature block sequences of the standard map features after slicing, a unique spatial location identifier is assigned to each feature block. A learnable location code or a fixed trigonometric function location code is used to generate a location code vector that matches the number of feature blocks. The location code vector is added element-wise to the feature vector of the corresponding feature block, so that each feature block contains both its own semantic information and spatial location information. This enhances the spatial location perception ability of the features in the feature block and avoids spatial location confusion during attention calculation.

[0047] Next, self-attention operations are performed on the feature block sequence of the original bird's-eye view features embedded with location encoding. The self-attention mechanism models the semantic dependencies between different spatial regions of the feature map by calculating the association weights between any two feature blocks in the feature block sequence. Using the feature block sequence of the original bird's-eye view features as input, query vectors, key vectors, and value vectors are generated respectively. Attention weights between each feature block are calculated through matrix operations, and the value vectors are weighted and summed to output a feature block sequence with enhanced global semantic association, i.e., enhanced visual features. This process can effectively capture the contextual associations between road areas, intersection areas, and background areas, strengthening the global semantic expressive power of real-time visual features.

[0048] Finally, using the enhanced visual features (also in the form of a feature block sequence) output by self-attention as the query vector, and the feature block sequence of the standard map features embedded with location encoding as the key vector and value vector respectively, the cross-attention weight between the enhanced visual features and the standard map features is calculated to characterize the degree of matching between real-time visual information and standard road structure information. The value vectors of the feature block sequence of the standard map features are weighted and summed using the cross-attention weights, injecting prior structural information such as road centerlines, road boundaries, and lane topology connections from the standard map into the enhanced visual features. The output is a feature block sequence that simultaneously contains real-time visual details and standard road structure constraints. After feature recombination, a complete first fusion feature is obtained, providing intermediate feature input for structural enhancement of detailed information from subsequent fusion of satellite images.

[0049] Step 105: After embedding position codes in the satellite image features, they are fused with the first fusion feature through an attention mechanism, and then convolutional enhancement is performed to obtain the second fusion feature. In this embodiment, the first fusion feature and the satellite image features are aligned to the same spatial dimension. The satellite image features are then segmented, and position codes are embedded in each feature block of the segment. The local offset between the segmented encoded satellite image features and the first fusion feature is estimated based on the vehicle position and compensated and corrected to align the spatial position of the segmented encoded satellite image features with that of the first fusion feature. The first fusion feature is used as the query, and the spatially aligned satellite image features are used as the key and value. Cross-attention operation is performed to fuse the road semantic information in the first fusion feature with the global information in the satellite image features to obtain the initial fusion feature. The initial fusion feature is then subjected to convolutional enhancement to strengthen its spatial continuity and semantic consistency to obtain the second fusion feature.

[0050] Specifically, the first fusion feature is aligned with the satellite image feature to the same spatial dimension. The first fusion feature is a feature that combines real-time visual data with standard road structure from a bird's-eye view, while the satellite image feature is a global fine-grained feature extracted from a top-down view. Since the original resolution and spatial scale of the two differ, interpolation transformation is used to upsample the lower-resolution feature and downsample the higher-resolution feature to make the spatial height and width of the first fusion feature and the satellite image feature completely consistent, ensuring that subsequent segmentation, registration, and fusion operations are performed at a unified spatial scale.

[0051] Next, the spatially aligned satellite image features are segmented and embedded using positional encoding. The satellite image features are uniformly divided into multiple non-overlapping feature blocks of a preset size, exemplarily using 16×16 pixel blocks, sequentially divided along the height and width directions of the satellite image features to form a spatially ordered sequence of feature blocks. A unique spatial position index is assigned to each feature block of the satellite image feature, and a corresponding position vector is generated using learnable positional encoding or trigonometric function positional encoding. The position vector is then added element-wise to the feature vector of the feature block of the satellite image feature, ensuring that each feature block simultaneously contains fine-grained texture information and spatial position information, improving the spatial perception accuracy of subsequent attention calculations.

[0052] Local offsets between feature blocks of satellite image features and the first fused feature are estimated and compensated to achieve precise spatial alignment. Due to the influence of satellite image shooting angle, terrain undulation, projection distortion, and vehicle pose error, there are local displacement deviations between feature blocks of satellite image features and the first fused feature. Using the real-time vehicle position as a reference, the local offset between the feature block sequence of satellite image features and the first fused feature is predicted. This local offset can be represented as a dense displacement field or a grid-level offset parameter. Based on the predicted local offset, bilinear interpolation or grid deformation correction algorithms are used to resample and transform the feature block sequence of satellite image features to compensate for spatial misalignment. This ensures that the corrected feature block sequence of satellite image features corresponds one-to-one with the first fused feature in spatial position, eliminating the influence of registration error on the fusion effect.

[0053] Using the first fused feature as the query vector, and the feature block sequence of the spatially aligned satellite image features as the key vector and value vector respectively, the cross-attention weight between the first fused feature and the feature block sequence of the satellite image features is calculated. The magnitude of the cross-attention weight represents the degree of matching and association between the road semantic region in the first fused feature and the global texture region in the satellite image features. The value vector of the satellite image features is weighted and summed using the cross-attention weight, and global fine-grained information such as lane markings, intersection contours, median strip shapes, and road extension boundaries in the satellite image is injected into the first fused feature to compensate for the lack of information caused by occlusion of the vehicle view and missing local observations, and the initial fused feature with global information is output.

[0054] While the initial fused features incorporate information from multiple sources, they may suffer from spatial discontinuities, semantic inconsistencies, and blurred edges. A multi-layer convolutional network is employed to enhance these initial fused features. Shallow convolutions strengthen edge details and local textures, while deep convolutions optimize global semantic relationships and spatial continuity. This convolutional enhancement process filters noise from the initial fused features, smooths feature boundaries, and enhances semantic consistency in road areas. The final output is a second fused feature that simultaneously possesses real-time visual details, standard road structure constraints, and fine-grained global satellite information, providing highly robust and complete feature input for subsequent high-precision map generation.

[0055] Step 106: Input the second fusion feature into the multi-branch map prediction head, perform fusion verification on the output results of each branch, and obtain a high-precision map fused with multi-source information. In this embodiment, the multi-branch map prediction head includes a semantic segmentation branch, an instance decoding branch, a vectorization generation branch, and a fusion decoding module; the semantic segmentation branch is configured to: extract the semantic features of each grid point in the bird's-eye view planar grid based on the second fusion feature, and output a multi-class semantic probability map containing multiple semantic categories after fusion; the instance decoding branch is configured to: extract instances from the semantic features in the semantic segmentation branch to distinguish instances of different semantic categories, and output the topological structure information of each instance; the vectorization generation branch is configured to: convert the multi-class semantic probability map and topological structure information into vector format to generate vector data that conforms to the high-precision map standard; the fusion decoding module is configured to: perform fusion verification on the multi-class semantic probability map, topological structure information, and vector data, eliminate semantic conflicts and topological errors, and decode to obtain a high-precision map fused with multi-source information.

[0056] Specifically, the semantic segmentation branch employs a fully convolutional network structure, containing multiple deconvolutional or upsampling layers, to restore the second fused features to the original spatial resolution of the bird's-eye view's planar grid. Then, using the second fused features with restored resolution as input, fine-grained semantic features for each grid point are extracted and enhanced layer by layer. The feature channels correspond to preset high-precision map semantic categories such as lane lines, road boundaries, pedestrian crossings, stop lines, traffic dividers, and intersection areas. The features of each channel are normalized using a softmax activation function, outputting the probability value of each grid point belonging to different semantic categories, forming a multi-category semantic probability map containing complete spatial distribution information, accurately representing the pixel-level distribution of various road elements under the bird's-eye view.

[0057] The instance decoding branch extracts instance-level features and outputs the topological structure information of each instance. It reuses the intermediate semantic features of the semantic segmentation branch and overlays them with the instance-aware branch network to perform instance-based modeling of the semantic features. Based on the category distribution of the multi-class semantic probability map, a non-maximum suppression algorithm is used to filter target regions, distinguishing different independent instances within the same semantic category, such as different lane line instances, different intersection area instances, and different pedestrian crossing instances. For each independent instance, topological attributes such as contour key points, connected component relationships, geometric boundary orientation, and adjacent instance connection relationships are extracted, outputting structured instance topological information and clarifying the spatial morphology and topological associations of each road element instance.

[0058] The vectorization generation branch converts multi-class semantic probability maps and topological information into high-precision map standard vector data. Based on the multi-class semantic probability maps, it extracts pixel-level contour boundaries of various road elements and combines this with the topological information of each instance to determine the continuous direction and inflection point locations of the instances. Algorithms such as polyline fitting, spline curve smoothing, and keypoint simplification are used to convert the raster-format probability distribution information into a sequence of coordinate points in vector format. Following high-precision map industry standards, the vector data is structured and encoded to generate vector data containing elements such as road centerlines, lane boundaries, pedestrian crossing outlines, stop line segments, and intersection connections, meeting the requirements of autonomous driving systems for high-precision map vector formats.

[0059] The fusion decoding module simultaneously receives three types of output: multi-category semantic probability maps, topological structure information, and vector data, and performs multi-level consistency checks. First, semantic consistency is checked by comparing the pixel categories of the multi-category semantic probability maps with the category labels of the topological structure information, correcting semantic misclassification, category confusion, and other conflicts. Next, topological rationality is checked, verifying the connection relationships between instances, the geometric constraints of adjacent elements, and the topological connectivity of intersections, eliminating errors such as topological breaks, overlaps, and logical contradictions. Finally, vector accuracy is checked, verifying the matching degree between vector coordinates and raster contours, smoothing vector polylines, correcting coordinate offsets, and eliminating redundant noise. After verification and correction, the multi-category semantic probability maps, topological structure information, and vector data are decoded to generate a high-precision map that combines pixel-level semantic accuracy, instance-level topological integrity, and vector-level format standardization, achieving effective fusion of multi-source information and comprehensive optimization of output quality.

[0060] In the embodiments of this application, the high-precision map obtained by fusing multi-source information can also: correct the positioning error in the output high-precision map based on the vehicle's pose information; and / or compare the road topology information in the high-precision map and the standard map to correct the offset or misalignment; and / or perform filtering processing on the high-precision map to eliminate noise and false semantic annotations; and / or perform weighted fusion of high-precision maps of multiple consecutive frames to utilize the spatial correlation between adjacent frames to correct random errors in a single frame.

[0061] Specifically, this application can also correct the positioning error of the generated high-precision map based on the vehicle's pose information. The local coordinate system of the high-precision map is transformed and matched with the world coordinate system corresponding to the vehicle's pose information. The spatial deviation between the key points of road elements in the high-precision map and the vehicle's positioning reference is calculated. Based on the deviation, the high-precision map is translated, rotated, and scaled in whole or in part to compensate for the positioning error introduced by sensor drift, coordinate mapping error, and pose estimation deviation, thereby improving the matching accuracy between the high-precision map and the real geographic space.

[0062] Secondly, spatial offsets or misalignments can be corrected by comparing road topology information between high-definition maps and standard maps. Standard maps possess high-confidence prior topological information such as road centerline orientation, lane connectivity, intersection topological connectivity, and road boundary geometric constraints. Key topological features such as topological nodes, connecting edges, and intersection structures of corresponding areas in the high-definition and standard maps are extracted. Topological feature matching relationships are established, and the spatial offset and angular misalignment between corresponding topological features are calculated. This identifies and corrects problems in the high-definition map caused by occlusion, feature loss, and fusion bias, such as road curvature offsets, lane line misalignments, and intersection topological deformations, ensuring the rationality and consistency of the road topology structure.

[0063] Furthermore, multi-scale filtering can be applied to high-precision maps to eliminate noise and false semantic annotations. High-precision maps are susceptible to image noise, feature interference, lighting variations, and registration errors during generation, resulting in isolated noise points, fine jagged edges, disconnected small regions, and false semantic annotations. Morphological filtering algorithms are used to process the rasterized semantic map. Opening operations remove small noise points, closing operations fill tiny holes, and connected component analysis removes isolated small regions. Simultaneously, semantic rule filtering is combined to eliminate false annotations that contradict common sense based on the geometric shape, size range, and connectivity characteristics of road elements, thus purifying the semantic and geometric information of the high-precision map.

[0064] Finally, a time-weighted fusion of multiple consecutive high-definition map frames can be performed to correct for random errors in single frames. In autonomous driving scenarios, vehicles continuously collect multi-source data and generate high-definition maps frame by frame, with spatial overlap and content correlation between adjacent frames. The high-definition maps of the current frame and its preceding and following frames are selected as a time sequence, and the matching degree between each frame's high-definition map and the current vehicle's pose information is calculated as a weight; the higher the matching degree, the greater the weight. The semantic probability, geometric coordinates, and topological information of multiple high-definition maps are weighted and summed or fused by voting. By utilizing the spatial redundancy and information complementarity of adjacent frames, random errors such as breaks, missing parts, and distortions caused by instantaneous occlusion, sudden changes in illumination, and false feature detection in single frames are corrected, thereby improving the temporal stability and spatial continuity of the high-definition map.

[0065] This application's embodiments, by constructing multi-source input data with unified time, space, and coordinate benchmarks, can effectively integrate real-time visual details, standard road structure constraints, and global satellite information to generate online high-precision maps with high real-time performance, high accuracy, and strong robustness. This not only compensates for the information limitations of a single data source but also ensures the standardization of road semantics, topology, and vector format through multi-branch prediction and fusion verification mechanisms. This meets the core requirements of autonomous driving systems for dynamic updates of high-precision maps, accurate positioning, and reliable decision-making, providing crucial support for the safe operation of autonomous vehicles. By converting the image features of various onboard multi-view cameras to a bird's-eye view, the perspective distortion and spatial misalignment caused by differences in installation location and shooting angle of the onboard multi-view cameras can be eliminated, ensuring that road elements (such as lane lines, intersections, and medians) from different perspectives present a consistent spatial distribution within a unified bird's-eye view grid. This transformation not only provides a unified spatial benchmark for cross-modal fusion of real-time visual features, standard road structure features, and satellite image features, but also fully preserves the spatial topological relationships of the vehicle's surrounding environment, such as the continuous direction of lane lines, the geometric layout of intersection areas, and the relative positions of pedestrian crossings and stop lines. This lays a solid spatial consistency foundation for the accurate extraction of subsequent road semantic information and the effective fusion of global information, thereby improving the efficiency and accuracy of multi-source feature fusion. The first fusion feature effectively integrates the local road details perceived by the vehicle's real-time vision with the global topological constraints of the standard road structure, forming an intermediate feature representation that combines local dynamic information with global structural norms. This feature retains real-time changes in the vehicle's surrounding environment, such as dynamic details like temporary obstacles, lane line wear, and construction areas, while also adhering to the geometric layout and topological relationships of standard roads, such as prior constraints like the number of lanes, intersection connection methods, and road grades. This provides reliable basic feature support for subsequent accurate alignment and cross-fusion with global features from satellite images.

[0066] While this application provides the method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in this embodiment is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the methods shown in this embodiment or the accompanying drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0067] like Figure 2 As shown in the figure, this application embodiment also provides an online high-precision map generation device 200 for autonomous driving tasks. The device includes: an alignment module 201, a mapping module 202, a conversion module 203, a first fusion module 204, a second fusion module 205, and an output module 206, as detailed below.

[0068] Alignment module 201 is used to preprocess the collected multi-source data to obtain multi-source input data with unified time, space and coordinate reference. The mapping module 202 is used to extract features from the multi-source input data and map the extracted multiple data features to the same channel dimension to obtain multi-source features of the same dimension; wherein, the multi-source features of the same dimension include vehicle multi-view camera image features, standard map features, and satellite image features; The conversion module 203 is used to convert the image features of each of the vehicle-mounted multi-view cameras to a bird's-eye view, obtain candidate bird's-eye view features from multiple perspectives, and fuse them to obtain the original bird's-eye view features. The first fusion module 204 is used to embed location codes into the original bird's-eye view features and the standard map features, and fuse them through an attention mechanism to obtain a first fused feature that fuses real-time visual features and standard road structure information. The second fusion module 205 is used to embed position codes into the satellite image features, fuse them with the first fusion feature through an attention mechanism, and then perform convolution enhancement to obtain the second fusion feature. Output module 206 is used to input the second fused feature into the map prediction head of the multi-branch structure, perform fusion verification on the output results of each branch, and obtain a high-precision map that fuses multi-source information.

[0069] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0070] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0071] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, such as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of a memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.

[0072] This application also provides an apparatus, the apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the executable instructions, it implements the method described in this application.

[0073] This application also provides a non-volatile computer-readable storage medium storing a computer program or instructions thereon, which, when executed, enables the method described in this application embodiment to be implemented.

[0074] Furthermore, in the various embodiments of the present invention, each functional module can be integrated into a processing module, or each module can exist independently, or two or more modules can be integrated into a single module.

[0075] The aforementioned storage media include, but are not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD), or Memory Card. The memory can be used to store computer program instructions.

[0076] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0077] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0078] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A method for generating high-precision maps online for autonomous driving tasks, characterized in that, include: The collected multi-source data is preprocessed to obtain multi-source input data with unified time, space, and coordinate references. Feature extraction is performed on the multi-source input data, and the extracted multiple data features are mapped to the same channel dimension to obtain same-dimensional multi-source features; wherein, the same-dimensional multi-source features include vehicle-mounted multi-view camera image features, standard map features, and satellite image features; The image features of each of the vehicle-mounted multi-view cameras are transformed to a bird's-eye view to obtain candidate bird's-eye view features from multiple perspectives, which are then fused to obtain the original bird's-eye view features. Location codes are embedded in the original bird's-eye view features and the standard map features, and then fused through an attention mechanism to obtain a first fused feature that combines real-time visual features and standard road structure information. After embedding location codes into the satellite image features, they are fused with the first fusion feature through an attention mechanism, and then convolutional enhancement is performed to obtain the second fusion feature; The second fusion feature is input into the map prediction head of the multi-branch structure, and the output results of each branch are fused and verified to obtain a high-precision map that integrates multi-source information.

2. The method according to claim 1, characterized in that, The multi-source data includes images from vehicle-mounted multi-view cameras, standard maps, and satellite images; The preprocessing of the collected multi-source data to obtain multi-source input data with unified time, space, and coordinate references includes: Using the vehicle's current timestamp as the time reference, the vehicle's multi-view camera images, standard map, satellite images, and vehicle location are time-aligned. Establish a local reference coordinate system centered on the vehicle's position; Multiple local slices are cropped from the standard map based on the vehicle's location to cover the vehicle's current perception range; The vehicle's position is mapped to the satellite image coordinate system based on the vehicle's pose information; Using the mapping point as the center, the satellite image is rotated and corrected according to the vehicle's heading angle to obtain a local satellite image patch; The local satellite image patches are paired with standard maps to form multi-source input data with unified time, space, and coordinate references.

3. The method according to claim 1, characterized in that, The step of extracting features from the multi-source input data and mapping multiple pre-defined data features to the same channel dimension to obtain multi-source features of the same dimension includes: Using convolutional neural networks and visual Transformers, images from vehicle-mounted multi-view cameras are independently encoded from one view to output multi-scale image feature tensors. A lightweight convolutional network is used as the backbone, with a dedicated map coding network with a location embedding layer superimposed to extract prior features of road structure from standard maps. An encoder employing a global receptive field and local texture enhancement is used to extract fine-grained features from satellite images; The multi-scale image feature tensor, the road structure prior features, and the fine-grained features are mapped to the same channel dimension to obtain multi-source features of the same dimension.

4. The method according to claim 1, characterized in that, The step of converting the image features of each of the vehicle-mounted multi-view cameras to a bird's-eye view to obtain candidate bird's-eye view features from multiple perspectives, and fusing them to obtain the original bird's-eye view features, includes: Define a bird's-eye view planar grid centered on the vehicle's location; The projection area of ​​each grid point in the bird's-eye view planar grid is determined based on the camera parameters of the vehicle-mounted camera. The local view features of the vehicle-mounted multi-view camera image features in each projection area are determined and mapped onto the bird's-eye view plane grid to obtain candidate bird's-eye view features from multiple perspectives. The bird's-eye view grid is divided into multiple sectors corresponding to the main viewing direction of each vehicle-mounted camera according to the orientation. The distance weight is determined based on the distance from each grid point in the sector to the center of the sector, and the direction weight is determined based on the angle between the position direction of each grid point in the sector and the main viewing direction of the corresponding vehicle camera. The distance weight and the direction weight are normalized to obtain the fusion weight of each vehicle camera at the corresponding grid point. The candidate bird's-eye view features from multiple perspectives are fused using the fusion weights to obtain the original bird's-eye view features.

5. The method according to claim 1, characterized in that, The method involves embedding location codes into the original bird's-eye view features and the standard map features, and fusing them through an attention mechanism to obtain a first fused feature that integrates real-time visual features and standard road structure information, including: Align the original bird's-eye view features with the standard map features to the same spatial dimension, then divide them into blocks, and embed position codes in each feature block of the block; Self-attention operation is performed on the original bird's-eye view features after block encoding to mine the semantic associations between different spatial regions in the original bird's-eye view features and obtain enhanced visual features; The enhanced visual features are used as queries, and the standard map features after block encoding are used as keys and values. Cross-attention operation is performed to fuse the real-time visual information of the original bird's-eye view features with the standard road structure information of the standard map features to obtain the first fused feature.

6. The method according to claim 1, characterized in that, The process of embedding location codes into the satellite image features, fusing them with the first fusion feature through an attention mechanism, and then performing convolutional enhancement to obtain the second fusion feature includes: Align the first fused feature with the satellite image feature to the same spatial dimension, then cut the satellite image feature into blocks, and embed position codes in each feature block of the block; The local offset between the segmented and encoded satellite image features and the first fused feature is estimated based on the vehicle position and compensated and corrected to align the spatial positions of the segmented and encoded satellite image features and the first fused feature. Using the first fusion feature as the query and the spatially aligned satellite image features as the key and value, a cross-attention operation is performed to fuse the road semantic information in the first fusion feature with the global information in the satellite image features to obtain the initial fusion feature; The initial fused features are subjected to convolutional enhancement processing to strengthen the spatial continuity and semantic consistency of the initial fused features, thereby obtaining the second fused features.

7. The method according to claim 1, characterized in that, The multi-branch structured map prediction head includes a semantic segmentation branch, an instance decoding branch, a vectorization generation branch, and a fusion decoding module; The semantic segmentation branch is configured to: extract semantic features of each grid point in the bird's-eye view plane grid based on the second fusion feature, and output a multi-class semantic probability map containing multiple semantic categories after fusion; The instance decoding branch is configured to: extract instances from the semantic features in the semantic segmentation branch to distinguish instances of different semantic categories and output the topological structure information of each instance; The vectorization generation branch is configured to convert the multi-category semantic probability map and the topological structure information into vector format to generate vector data that conforms to the high-precision map standard. The fusion decoding module is configured to: perform fusion verification on the multi-category semantic probability map, the topological structure information and the vector data, eliminate semantic conflicts and topological errors, and decode to obtain a high-precision map that integrates multi-source information.

8. The method according to claim 1, characterized in that, The obtained high-precision map, which integrates multi-source information, also includes: The positioning error in the output high-precision map is corrected based on the vehicle's pose information. And / or, compare the road topology information in the high-precision map with that in the standard map to correct any offsets or misalignments; And / or, filter the high-precision map to eliminate noise and false semantic annotations; And / or, weighted fusion of high-precision maps from multiple consecutive frames to utilize the spatial correlation between adjacent frames and correct random errors in a single frame.

9. An online high-precision map generation apparatus for autonomous driving tasks, used to implement the method described in any one of claims 1-8, characterized in that, include: The alignment module is used to preprocess the collected multi-source data to obtain multi-source input data with unified time, space, and coordinate reference. The mapping module is used to extract features from the multi-source input data and map the extracted data features to the same channel dimension to obtain multi-source features of the same dimension; wherein, the multi-source features of the same dimension include vehicle multi-view camera image features, standard map features, and satellite image features; The conversion module is used to convert the image features of each of the vehicle-mounted multi-view cameras to a bird's-eye view, obtain candidate bird's-eye view features from multiple perspectives, and fuse them to obtain the original bird's-eye view features. The first fusion module is used to embed location codes into the original bird's-eye view features and the standard map features, and fuse them through an attention mechanism to obtain a first fused feature that fuses real-time visual features and standard road structure information. The second fusion module is used to embed position codes into the satellite image features, fuse them with the first fusion feature through an attention mechanism, and then perform convolution enhancement to obtain the second fusion feature. The output module is used to input the second fused feature into the map prediction head of the multi-branch structure, perform fusion verification on the output results of each branch, and obtain a high-precision map that fuses multi-source information.

10. An apparatus for performing an online high-precision map generation method for an autonomous driving task, characterized in that, include: processor; Memory used to store processor-executable instructions; When the processor executes the executable instructions, it implements the method as described in any one of claims 1 to 8.

Images