Three-dimensional vectorization high-definition map generation method based on a deflectable bird's-eye view feature

Abstract

The application discloses a kind of three-dimensional vectorization high-precision map generation method based on offsetable bird's eye view feature, and high-quality real-time map construction is realized using SBEVNet network improved from MapTR network.Image feature extraction module fuses high-resolution details and deep semantic information, while maintaining spatial accuracy, effectively reducing the positioning deviation caused by multi-view projection;Bird's eye view feature coding module introduces a learnable offset reference point mechanism, so that the feature can dynamically focus on the actual spatial distribution of map elements, significantly enhancing the spatial alignment accuracy and three-dimensional height perception ability;Map element decoding module uses instance mask structure modeling, which strengthens the global structural consistency and topological continuity.The application has real-time reasoning capability, and is superior to existing mainstream solutions in terms of geometric accuracy, structural integrity and complex scene robustness, and the generated map is more consistent with the real road layout, providing reliable technical support for high-precision positioning and path planning of autonomous driving.

Description

Technical Field

[0001] This invention relates to the technical field of real-time 3D vectorized high-precision map generation, and in particular to a method for generating 3D vectorized high-precision maps based on offset bird's-eye view features. Background Technology

[0002] High-precision maps of the surrounding environment are crucial foundational information for autonomous driving systems in localization, path planning, and driving decisions. Traditional navigation systems primarily rely on offline maps and satellite positioning, which struggle to adapt to dynamic changes in real-world road environments, map-to-real-world inconsistencies, and positioning drift. Furthermore, their performance is limited in uncovered areas or complex scenarios, failing to meet the safety and robustness requirements of autonomous driving systems. Therefore, online real-time high-precision map generation technology for real-world road environments has become a key research direction in autonomous driving perception systems.

[0003] Most existing online vectorized high-precision map generation methods are still limited to two-dimensional planar scenes. Their overall architecture typically includes three core modules: image feature extraction, feature encoding, and map element decoding. These methods mainly improve the detection and modeling capabilities of two-dimensional map elements by optimizing the decoder structure. However, in the feature encoding stage, they still use a planar two-dimensional spatial representation, which underutilizes the three-dimensional structural information in the real world, especially the vertical dimension information, which is not effectively modeled.

[0004] However, in real-world autonomous driving applications, the road environment is essentially a strongly structured three-dimensional space, and relying solely on a two-dimensional plane for representation leads to significant limitations. On one hand, a vehicle's perception range typically covers approximately 30 meters in front and behind, 15 meters to the left and right, and 3 meters vertically. Within this range, the road structure not only varies on the plane but also exhibits significant height differences and spatial occlusion relationships. If only a two-dimensional grid or two-dimensional feature plane is used for discretization, a large number of feature units are often required to maintain spatial resolution, leading to a rapid increase in computational complexity and storage overhead. This also places high demands on real-time inference capabilities, limiting the system's scalability for larger perception ranges and higher accuracy requirements.

[0005] On the other hand, compressing three-dimensional spatial information into a two-dimensional planar representation inevitably leads to the loss and aliasing of vertical dimensional information, making it difficult to distinguish structural information at different height levels, thus weakening the model's ability to fully depict the geometry of real roads. This limitation is particularly pronounced in complex scenarios involving bridges, tunnels, and grade-separated intersections, making it difficult for the model to accurately model key three-dimensional geometric relationships such as the height attributes of road elements, the distance of obstacles from the ground, and road clearance. In the process of path planning and safety decision-making, the system not only relies on planar accessibility judgments but also needs to accurately perceive traversable constraints in three-dimensional space; otherwise, it is prone to misjudgments in height-restricted structures or three-dimensional occlusion scenarios, thereby affecting driving safety.

[0006] Therefore, to meet the application requirements of autonomous driving in complex real-world environments, high-precision map generation methods urgently need to evolve from the traditional two-dimensional planar modeling paradigm to three-dimensional structured representation. This requires effectively integrating vertical dimensional information while preserving spatial geometric consistency, to achieve a more complete and accurate representation of the real road environment. This online vectorized high-precision map generation method with three-dimensional perception capabilities has significant research and engineering value for improving the environmental understanding, path planning reliability, and overall safety of autonomous driving systems. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and propose a method for generating high-precision 3D vectorized maps based on offset bird's-eye view features. This method can be applied to complex 3D road scenarios for autonomous driving. While ensuring the ability to express high-precision maps, it can effectively improve the real-time inference efficiency and training efficiency of the network and enhance the scalability of the perception range.

[0008] To achieve the above objectives, the technical solution provided by this invention is as follows: a method for generating high-precision 3D vectorized maps based on offset bird's-eye view features. This method utilizes the SBEVNet network to achieve high-quality real-time generation of high-precision 3D vectorized maps. The SBEVNet network is an improved MapTR network, which improves the image feature extraction module, bird's-eye view feature encoding module, and map element decoding module of the original MapTR network. The improvement to the image feature extraction module is: adopting a high-resolution, high-semantic image feature fusion structure, which retains the spatial details of high-resolution images while fusing deep semantic information to reduce spatial errors in the multi-view projection and bird's-eye view conversion process. The improvement to the bird's-eye view feature encoding module is: replacing the fixed-position sampled bird's-eye view features with learnable offset reference points. By predicting the offset of the reference points, the bird's-eye view features can dynamically move to the spatial location of the map elements, improving the spatial alignment capability and 3D height perception capability of the bird's-eye view features to map elements. The improvement to the map element decoding module is: adopting an instance mask structure modeling mechanism, which enhances the global structural consistency and topological continuity of map elements by jointly learning the correspondence between map element instance masks and bird's-eye view features.

[0009] The specific implementation of this method includes the following steps:

[0010] 1) Collect multi-view images from the vehicle's surround-view cameras at the same time and the corresponding internal and external parameters of the cameras. Combine the vehicle's positioning information with the offline high-precision map to generate a road map around the vehicle. Convert the road map into a vectorized representation to form a vectorized high-precision map. Integrate the multi-view images, camera internal and external parameters, and vectorized high-precision map to form a dataset, and divide it into a training set and a test set.

[0011] 2) The training set is input into the SBEVNet network for training. First, data augmentation is performed on the multi-view images in the training set. Then, multi-view, multi-scale image features are extracted by the image feature extraction module and input into the bird's-eye view feature encoding module for multi-view fusion and spatial feature alignment to generate offset bird's-eye view features. Subsequently, vectorized map elements are generated by the map element decoding module, and map element instance masks are generated through the instance mask structure modeling mechanism. The generated vectorized map elements are matched with real map elements in a bipartite graph, and the map element classification loss, position loss, reference point offset loss, instance mask loss, and auxiliary supervision loss are calculated respectively. The network parameters are optimized through backpropagation, and the optimal network is obtained after multiple rounds of iterative training.

[0012] 3) Input the test set into the trained optimal network to generate a three-dimensional vectorized high-precision map of the area around the vehicle.

[0013] Furthermore, step 1) includes the following steps:

[0014] 1.1) Data acquisition: Install multiple surround view cameras at fixed positions on the vehicle, record the internal and external parameters of the cameras, set multiple cameras to sample synchronously at the same frequency, acquire multi-view images during vehicle movement, and simultaneously record vehicle pose and positioning information;

[0015] 1.2) Data preprocessing: After data acquisition, the multi-view images, camera internal and external parameters, and vehicle pose information acquired at the same time are combined into a set of data, and data with asynchronous time or obvious camera position offset are removed.

[0016] 1.3) Data annotation: Based on the vehicle's location, the road area around the vehicle is cropped from the offline high-precision map and rotated and aligned according to the vehicle's orientation to obtain the cropped and aligned road map. The road map is then converted into a vectorized high-precision map represented by an ordered set of points and annotated according to the map element categories.

[0017] 1.4) Dataset partitioning: The multi-view images, camera intrinsic and extrinsic parameters, and vectorized high-precision maps are integrated to form a dataset, which is then divided into a training set and a test set according to a preset ratio.

[0018] Furthermore, in step 2), the data augmentation includes image scaling, random brightness adjustment, random contrast adjustment, random saturation adjustment, random hue adjustment, and GridMask data augmentation; the GridMask data augmentation involves randomly generating grid occlusion regions in the input image to occlude some image information, thereby improving the diversity of training data and enhancing the robustness of the network.

[0019] Furthermore, in step 2), the image feature extraction module includes a ResNet50 network and a Feature Pyramid Network (FPN); the ResNet50 network is used to extract multi-view, multi-scale image features. , represented as subscript The scale hierarchy index represents the features of the multi-view, multi-scale image. Indicates the maximum scale. The minimum scale is represented by FPN, which is used for multi-scale feature fusion and unifies the output feature dimension. The high-resolution, high-semantic image feature fusion structure injects deep semantic features into high-resolution shallow features, as expressed by the formula:

[0020] ;

[0021] In the formula, and They represent the first and Layer fusion characteristics, Indicates an upsampling operation;

[0022] While preserving spatial details, the semantic expressive power is enhanced to reduce spatial errors in the bird's-eye view conversion process, resulting in multi-view, multi-scale image features with unified feature dimensions, which are then input into the bird's-eye view feature encoding module.

[0023] Further, in step 2), the bird's-eye view feature encoding module performs view transformation and fusion on the multi-view, multi-scale image features output by the image feature extraction module to generate offset bird's-eye view features focused on the spatial location of map elements: First, a set of learnable features represents the bird's-eye view features. A multi-layer linear network predicts the coordinates of a two-dimensional reference point in the bird's-eye view space. The coordinates of the two-dimensional reference point are bound to the bird's-eye view features. The coordinates of the two-dimensional reference point are obtained by sampling multiple heights through global self-attention to obtain the coordinates of the three-dimensional reference point. These coordinates are then projected onto the multi-view image plane through camera intrinsic and extrinsic parameters. Local image features are collected through deformable attention and then projected through a feedforward network to obtain new bird's-eye view features. The newly obtained bird's-eye view features are then processed through a multi-layer linear network to obtain new two-dimensional reference point coordinates. The bird's-eye view features are updated through a new round of projection sampling. Through multi-layer projection sampling, the bird's-eye view features gradually focus on the spatial location of map elements, thereby obtaining offset bird's-eye view features with spatial alignment capabilities. These are then input into the map element decoding module, as expressed by the formula:

[0024] ;

[0025] In the formula, Indicates the current bird's-eye view features, This indicates the updated bird's-eye view features. This indicates a global self-attention operation. This indicates a deformable attention operation. This represents a feedforward network.

[0026] Furthermore, in step 2), the map element decoding module utilizes an attention mechanism with global structure modeling capabilities to efficiently detect map elements from offset bird's-eye view features. This module is implemented using a Transformer decoder, which uses the offset bird's-eye view features obtained by the bird's-eye view feature encoding module as the key and value of the decoder, and combines them... Map element features and Each anchor point feature generates Each query corresponds to a map element anchor point; for each map element, a fixed number of queries interact with a varying number of offset bird's-eye view features associated with multiple map elements, resulting in a rearranged map. Anchor point coordinates; based on this, the instance mask structure modeling mechanism is introduced: map element queries and offset bird's-eye view features are fused element by element to predict the instance mask exclusive to each map element, so as to explicitly model the complete spatial range and global topology of each map element; the instance mask is used to spatially constrain the bird's-eye view features, so that the query only focuses on the effective area belonging to the current map element, and then combined with the distance attention mask to achieve dual enhancement of spatial nearest neighbor filtering and instance belonging constraint, which improves the interaction efficiency while strengthening the overall shape perception and structural consistency modeling of map elements such as lane lines, zebra crossings, and drivable areas.

[0027] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0028] 1. This invention replaces the traditional two-dimensional rasterization or dense feature representation method with a modeling method based on offset bird's-eye view features. While ensuring spatial expressiveness, it significantly reduces feature redundancy, thereby effectively reducing computational and storage overhead, improving the overall inference efficiency of the network, and meeting the real-time requirements of autonomous driving systems.

[0029] 2. This invention can more naturally preserve and utilize three-dimensional spatial structural information, avoiding the compression and loss of vertical dimension information in traditional two-dimensional representation, thereby improving the modeling ability of key geometric attributes such as road height, obstacle distance from the ground and road clearance, and enhancing the ability to understand the environment in complex three-dimensional traffic scenarios.

[0030] 3. This invention has stronger spatial scalability at the feature representation level, which can flexibly adapt to different sensing range and resolution requirements. When expanding the sensing range, it does not require an exponential increase in computational overhead, thereby improving the network's deployment capability and stability in large-scale scenarios. Attached Figure Description

[0031] Figure 1 This is a diagram illustrating the overall architecture of the method of the present invention.

[0032] Figure 2 This is a visualization of the input and output of the method of this invention on the Argoverse 2 dataset. The left side of the figure shows the input data for the method of this invention on the Argoverse 2 dataset, which comes from the surround view images from the vehicle-mounted sensors. The camera orientations corresponding to the surround view images in the figure, from left to right and top to bottom, are front left, front center, front right, side left, rear left, rear right, and side right. The right side of the figure shows the vectorized high-precision map generated by the method of this invention and the ground truth high-precision map provided by the dataset. As can be seen from the figure, the high-precision map generated by the method of this invention is basically consistent with the ground truth, indicating high generation quality.

[0033] Figure 3This is a visualization of the input and output of the method of this invention on the nuScenes dataset. The left side of the figure shows the input data of the method of this invention on the nuScenes dataset, which comes from the surround view images of the vehicle-mounted sensors. The camera orientations corresponding to the surround view images in the figure, from left to right and from top to bottom, are front left, front center, front right, side left, rear left, rear right, and side right. The right side of the figure shows the vectorized high-precision map generated by the method of this invention and the ground truth high-precision map provided by the dataset. As can be seen from the figure, the high-precision map generated by the method of this invention has a high degree of consistency with the ground truth annotation results, indicating that the proposed method has good accuracy and reliability in terms of map element structure recovery and spatial representation.

[0034] Figure 4 This section presents the vectorized high-precision map generation results of the method of this invention in different scenarios. Each row compares the generation effect in a specific scenario. The first three columns are the multi-view images of the input network, the fourth column is the vectorized high-precision map generated by the MapTR network, the fifth column is the vectorized high-precision map generated by the MapQR network, the sixth column is the vectorized high-precision map generated by the SBEVNet network of this invention, and the seventh column is the actual vectorized high-precision map. It can be seen that compared with MapTR and MapQR networks, the SBEVNet network exhibits better map generation results in various scenarios, verifying that the method of this invention has higher robustness than existing methods in different road scenarios.

[0035] Figure 5 This image shows the vectorized high-precision map generation results of the method of this invention under different weather conditions. Each row compares the generation effect under specific weather conditions. The first three columns are multi-view images of the input network, the fourth column is the vectorized high-precision map generated by the MapTR network, the fifth column is the vectorized high-precision map generated by the MapQR network, the sixth column is the vectorized high-precision map generated by the SBEVNet network of this invention, and the seventh column is the actual vectorized high-precision map. It can be seen that compared with MapTR and MapQR networks, the SBEVNet network exhibits better map generation results under various weather conditions, verifying that the method of this invention has higher robustness compared to existing methods under different weather conditions. Detailed Implementation

[0036] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0037] like Figures 1 to 5As shown, this embodiment discloses a method for generating 3D vectorized high-precision maps based on offset bird's-eye view features. This method utilizes the SBEVNet network to achieve high-quality real-time generation of 3D vectorized high-precision maps. The SBEVNet network is an improved MapTR network, which improves the image feature extraction module, bird's-eye view feature encoding module, and map element decoding module of the original MapTR network. The improvement to the image feature extraction module is the adoption of a high-resolution, high-semantic image feature fusion structure, which retains the spatial details of the high-resolution image while fusing deep semantic information to reduce spatial errors during the conversion between multi-view projection and bird's-eye view. The improvement to the bird's-eye view feature encoding module is the replacement of fixed-position sampled bird's-eye view features with learnable offset reference points. By predicting the offset of the reference points, the bird's-eye view features can dynamically move towards the spatial location of map elements, improving the spatial alignment capability and 3D height perception capability of the bird's-eye view features to map elements. The improvement to the map element decoding module is the adoption of an instance mask structure modeling mechanism, which enhances the global structural consistency and topological continuity of map elements by jointly learning the correspondence between map element instance masks and bird's-eye view features.

[0038] The specific implementation of this method includes the following steps:

[0039] 1) Collect multi-view images from the vehicle's surround-view cameras at the same time and the corresponding internal and external parameters of the cameras. Combine these with vehicle positioning information and offline high-precision maps to generate a road map around the vehicle. Convert the road map into a vectorized representation to form a vectorized high-precision map. Integrate the multi-view images, camera internal and external parameters, and vectorized high-precision map to form a dataset, and divide it into a training set and a test set. The specific steps include:

[0040] 1.1) Data acquisition: Install multiple surround view cameras at fixed positions on the vehicle, record the internal and external parameters of the cameras, set multiple cameras to sample synchronously at the same frequency, acquire multi-view images during vehicle movement, and simultaneously record vehicle pose and positioning information;

[0041] 1.2) Data preprocessing: After data acquisition, the multi-view images, camera internal and external parameters, and vehicle pose information acquired at the same time are combined into a set of data, and data with asynchronous time or obvious camera position offset are removed.

[0042] 1.3) Data annotation: Based on the vehicle's location, the road area around the vehicle is cropped from the offline high-precision map and rotated and aligned according to the vehicle's orientation to obtain the cropped and aligned road map. The road map is then converted into a vectorized high-precision map represented by an ordered set of points and annotated according to the map element categories.

[0043] 1.4) Dataset partitioning: The multi-view images, camera intrinsic and extrinsic parameters, and vectorized high-precision maps are integrated to form a dataset, which is then divided into a training set and a test set according to a preset ratio.

[0044] 2) The training set is input into the SBEVNet network for training. First, data augmentation is performed on the multi-view images in the training set. Then, the multi-view, multi-scale image features are extracted by the image feature extraction module and input into the bird's-eye view feature encoding module for multi-view fusion and spatial feature alignment to generate offset bird's-eye view features. Subsequently, vectorized map elements are generated by the map element decoding module, and map element instance masks are generated by the instance mask structure modeling mechanism. The generated vectorized map elements are matched with real map elements in a bipartite graph, and the map element classification loss, position loss, reference point offset loss, instance mask loss, and auxiliary supervision loss are calculated respectively. The network parameters are optimized through backpropagation, and the optimal network is obtained after multiple rounds of iterative training.

[0045] Specifically, the data augmentation includes image scaling, random brightness adjustment, random contrast adjustment, random saturation adjustment, random hue adjustment, and GridMask data augmentation; the GridMask data augmentation involves randomly generating grid occlusion regions in the input image to occlude some image information, thereby improving the diversity of training data and enhancing the robustness of the network.

[0046] Specifically, the image feature extraction module includes a ResNet50 network and a Feature Pyramid Network (FPN); the ResNet50 network is used to extract multi-view, multi-scale image features. , represented as subscript The scale hierarchy index represents the features of the multi-view, multi-scale image. Indicates the maximum scale. The minimum scale is represented by FPN, which is used for multi-scale feature fusion and unifies the output feature dimension. The high-resolution, high-semantic image feature fusion structure injects deep semantic features into high-resolution shallow features, as expressed by the formula:

[0047] ;

[0048] In the formula, and They represent the first and Layer fusion characteristics, Indicates an upsampling operation;

[0049] While preserving spatial details, the semantic expressive power is enhanced to reduce spatial errors in the bird's-eye view conversion process, resulting in multi-view, multi-scale image features with unified feature dimensions, which are then input into the bird's-eye view feature encoding module.

[0050] Specifically, the bird's-eye view feature encoding module performs view transformation and fusion on the multi-view, multi-scale image features output by the image feature extraction module to generate offset bird's-eye view features focused on the spatial location of map elements. First, a set of learnable features represents the bird's-eye view features. A multi-layer linear network predicts the coordinates of a two-dimensional reference point in the bird's-eye view space. These coordinates are then bound to the bird's-eye view features. The two-dimensional reference point coordinates are obtained by sampling multiple heights using global self-attention to obtain three-dimensional reference point coordinates. These coordinates are then projected onto the multi-view image plane using camera intrinsic and extrinsic parameters. Local image features are collected using deformable attention and then projected through a feedforward network to obtain new bird's-eye view features. The newly obtained bird's-eye view features are then processed through a multi-layer linear network to obtain new two-dimensional reference point coordinates. A new round of projection sampling updates the bird's-eye view features. Through multi-layer projection sampling, the bird's-eye view features gradually focus on the spatial location of map elements, thus obtaining offset bird's-eye view features with spatial alignment capabilities. These features are then input into the map element decoding module, as expressed by the formula:

[0051] ;

[0052] In the formula, Indicates the current bird's-eye view features, This indicates the updated bird's-eye view features. This indicates a global self-attention operation. This indicates a deformable attention operation. This represents a feedforward network.

[0053] Specifically, the map element decoding module utilizes an attention mechanism with global structure modeling capabilities to efficiently detect map elements from offset bird's-eye view features. This module is implemented using a Transformer decoder, which uses the offset bird's-eye view features obtained by the bird's-eye view feature encoding module as the key and value of the decoder, and combines them... Map element features and Each anchor point feature generates Each query corresponds to a map element anchor point; for each map element, a fixed number of queries interact with a varying number of offset bird's-eye view features associated with multiple map elements, resulting in a rearranged map. Anchor point coordinates; based on this, the instance mask structure modeling mechanism is introduced: map element queries and offset bird's-eye view features are fused element by element to predict the instance mask exclusive to each map element, so as to explicitly model the complete spatial range and global topology of each map element; the instance mask is used to spatially constrain the bird's-eye view features, so that the query only focuses on the effective area belonging to the current map element, and then combined with the distance attention mask to achieve dual enhancement of spatial nearest neighbor filtering and instance belonging constraint, which improves the interaction efficiency while strengthening the overall shape perception and structural consistency modeling of map elements such as lane lines, zebra crossings, and drivable areas.

[0054] This embodiment uses a learning rate of 6e-4, trains with 8 A800 GPUs, and sets the batch size to 4 for each GPS.

[0055] 3) Input the test set into the trained optimal network to generate a 3D vectorized high-precision map of the area around the vehicle.

[0056] The experimental results of the above-described 3D vectorized high-precision map generation method in this embodiment are described in detail below:

[0057] The SBEVNet network was evaluated on the large-scale datasets nuScenes and Argoverse2. The nuScenes dataset has an annotation frequency of 2 Hz. Each frame contains RGB images from 6 surrounding cameras, covering a 360-degree field of view. The dataset contains 1000 scenes, each containing approximately 40 frames. The dataset was divided into 28,000 frames for training and 6,000 frames for validation.

[0058] Referring to existing methods, this study considers three types of map elements—lane lines, pedestrian crossings, and drivable area boundaries—to evaluate the construction of high-resolution maps. The generated map area is defined as 30 meters in front of and behind vehicles, and 15 meters to the left and right. Average accuracy (AP) based on chamfer distance is used as the evaluation metric. Using the average accuracy at three thresholds of {0.5, 1.0, 1.5} meters, a prediction is considered a true positive (TP) only if the distance between the predicted and actual values ​​is less than a specified threshold.

[0059] The experimental results on the Argoverse2 dataset in 3D scenes are shown in Table 1 below, and the experimental results on the nuScenes dataset in 2D scenes are shown in Table 2 below. divider AP represents the average accuracy of lane lines. ped AP represents the average precision of pedestrian crossing lines. boundaryThe average accuracy of the drivable area boundary lines is represented by mAP. mAP represents the average accuracy of lane lines, sidewalks, and drivable area boundary lines, reflecting the accuracy of the vectorized high-precision map generated by the method of this invention. FPS represents how many maps the method of this invention can generate per second, reflecting the inference speed of the method of this invention. All values ​​in the FPS column were obtained from testing on the same NVIDIA A800 device. As can be seen from Tables 1 and 2, the SBEVNet network achieves the highest accuracy in generating vectorized high-precision maps with fewer parameters.

[0060] Table 1

[0061]

[0062] Table 2

[0063]

[0064] A comparison of the vectorized high-precision map generation effects of SBEVNet network and traditional methods in different scenarios. Figure 4 As shown in the figure, the fourth column represents the generation results of the MapTR network, the fifth column represents the generation results of the MapQR network, the sixth column represents the generation results of the SBEVNet network, and the seventh column represents the ground truth values ​​provided by the dataset. It can be seen that the SBEVNet network can generate accurate road maps in multiple scenarios.

[0065] A comparison of the vectorized high-precision map generation effects of SBEVNet and MapTR networks under different weather conditions. Figure 5 As shown in the figure, the fourth column represents the generation results of the MapTR network, the fifth column represents the generation results of the MapQR network, the sixth column represents the generation results of the SBEVNet network, and the seventh column represents the ground truth values ​​provided by the dataset. It can be seen that the SBEVNet network can generate accurate road maps under various weather conditions.

[0066] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for generating 3D vectorized high-precision maps based on offset bird's-eye view features, characterized in that: This method utilizes the SBEVNet network to achieve high-quality real-time generation of 3D vectorized high-precision maps. The SBEVNet network is an improved MapTR network, which enhances the image feature extraction module, bird's-eye view feature encoding module, and map element decoding module of the original MapTR network. The improvements to the image feature extraction module include: employing a high-resolution, high-semantic image feature fusion structure to retain spatial details of high-resolution images while fusing deep semantic information, thereby reducing spatial errors during multi-view projection and bird's-eye view conversion. The improvements to the bird's-eye view feature encoding module include: replacing fixed-position sampled bird's-eye view features with learnable offset reference points. By predicting the offset of the reference points, the bird's-eye view features can dynamically move towards the spatial location of map elements, improving the spatial alignment capability and 3D height perception capability of the bird's-eye view features to map elements. The improvements to the map element decoding module include: employing an instance mask structure modeling mechanism to enhance the global structural consistency and topological continuity of map elements by jointly learning the correspondence between map element instance masks and bird's-eye view features. The specific implementation of this method includes the following steps: 1) Collect multi-view images from the vehicle's surround-view cameras at the same time and the corresponding internal and external parameters of the cameras. Combine the vehicle's positioning information with the offline high-precision map to generate a road map around the vehicle. Convert the road map into a vectorized representation to form a vectorized high-precision map. Integrate the multi-view images, camera internal and external parameters, and vectorized high-precision map to form a dataset, and divide it into a training set and a test set. 2) The training set is input into the SBEVNet network for training. First, data augmentation is performed on the multi-view images in the training set. Then, multi-view, multi-scale image features are extracted by the image feature extraction module and input into the bird's-eye view feature encoding module for multi-view fusion and spatial feature alignment to generate offset bird's-eye view features. Subsequently, vectorized map elements are generated by the map element decoding module, and map element instance masks are generated through the instance mask structure modeling mechanism. The generated vectorized map elements are matched with real map elements in a bipartite graph, and the map element classification loss, position loss, reference point offset loss, instance mask loss, and auxiliary supervision loss are calculated respectively. The network parameters are optimized through backpropagation, and the optimal network is obtained after multiple rounds of iterative training. 3) Input the test set into the trained optimal network to generate a three-dimensional vectorized high-precision map of the area around the vehicle.

2. The method for generating a three-dimensional vectorized high-precision map based on offset bird's-eye view features according to claim 1, characterized in that, Step 1) includes the following steps: 1.1) Data acquisition: Install multiple surround view cameras at fixed positions on the vehicle, record the internal and external parameters of the cameras, set multiple cameras to sample synchronously at the same frequency, acquire multi-view images during vehicle movement, and simultaneously record vehicle pose and positioning information; 1.2) Data preprocessing: After data acquisition, the multi-view images, camera internal and external parameters, and vehicle pose information acquired at the same time are combined into a set of data, and data with asynchronous time or obvious camera position offset are removed. 1.3) Data annotation: Based on the vehicle's location, the road area around the vehicle is cropped from the offline high-precision map and rotated and aligned according to the vehicle's orientation to obtain the cropped and aligned road map. The road map is then converted into a vectorized high-precision map represented by an ordered set of points and annotated according to the map element categories. 1.4) Dataset partitioning: The multi-view images, camera intrinsic and extrinsic parameters, and vectorized high-precision maps are integrated to form a dataset, which is then divided into a training set and a test set according to a preset ratio.

3. The method for generating a three-dimensional vectorized high-precision map based on offset bird's-eye view features according to claim 1, characterized in that, In step 2), the data augmentation includes image scaling, random brightness adjustment, random contrast adjustment, random saturation adjustment, random hue adjustment, and GridMask data augmentation; the GridMask data augmentation is to randomly generate grid occlusion regions in the input image to occlude part of the image information, thereby improving the diversity of training data and enhancing the robustness of the network.

4. The method for generating a three-dimensional vectorized high-precision map based on offset bird's-eye view features according to claim 1, characterized in that, In step 2), the image feature extraction module includes a ResNet50 network and a Feature Pyramid Network (FPN); the ResNet50 network is used to extract multi-view, multi-scale image features. , represented as subscript The scale hierarchy index represents the features of the multi-view, multi-scale image. Indicates the maximum scale. This represents the minimum scale. FPN is used for multi-scale feature fusion and to unify the output feature dimension. The high-resolution, high-semantic image feature fusion structure injects deep semantic features into high-resolution shallow features, as expressed by the formula: ； In the formula, and They represent the first and Layer fusion characteristics, Indicates an upsampling operation; While preserving spatial details, the semantic expressive power is enhanced to reduce spatial errors in the bird's-eye view conversion process, resulting in multi-view, multi-scale image features with unified feature dimensions, which are then input into the bird's-eye view feature encoding module.

5. The method for generating a three-dimensional vectorized high-precision map based on offset bird's-eye view features according to claim 1, characterized in that, In step 2), the bird's-eye view feature encoding module performs view transformation and fusion on the multi-view, multi-scale image features output by the image feature extraction module to generate offset bird's-eye view features focused on the spatial location of map elements. First, a set of learnable features represents the bird's-eye view features. A multi-layer linear network predicts the coordinates of a two-dimensional reference point in the bird's-eye view space. The coordinates of the two-dimensional reference point are bound to the bird's-eye view features. The coordinates of the two-dimensional reference point are obtained by sampling multiple heights globally through self-attention to obtain three-dimensional reference point coordinates. These coordinates are then projected onto the multi-view image plane using camera intrinsic and extrinsic parameters. Local image features are collected through deformable attention and then projected through a feedforward network to obtain new bird's-eye view features. The newly obtained bird's-eye view features are then processed through a multi-layer linear network to obtain new two-dimensional reference point coordinates. A new round of projection sampling updates the bird's-eye view features. Through multi-layer projection sampling, the bird's-eye view features gradually focus on the spatial location of map elements, thus obtaining offset bird's-eye view features with spatial alignment capabilities. These are then input into the map element decoding module, as expressed by the formula: ； In the formula, Indicates the current bird's-eye view features, This indicates the updated bird's-eye view features. This indicates a global self-attention operation. This indicates a deformable attention operation. This represents a feedforward network.

6. The method for generating a three-dimensional vectorized high-precision map based on offset bird's-eye view features according to claim 1, characterized in that, In step 2), the map element decoding module utilizes an attention mechanism with global structure modeling capabilities to efficiently detect map elements from offset bird's-eye view features. This module is implemented using a Transformer decoder, which uses the offset bird's-eye view features obtained by the bird's-eye view feature encoding module as the key and value of the decoder, and combines them... Map element features and Each anchor point feature generates Each query corresponds to a map element anchor point; For each map element, a fixed number of queries interact with a varying number of offset bird's-eye view features associated with multiple map elements, resulting in a rearrangement. Anchor point coordinates; based on this, the instance mask structure modeling mechanism is introduced: map element queries and offset bird's-eye view features are fused element by element to predict the instance mask exclusive to each map element, so as to explicitly model the complete spatial range and global topology of each map element; the instance mask is used to spatially constrain the bird's-eye view features, so that the query only focuses on the effective area belonging to the current map element, and then combined with the distance attention mask to achieve dual enhancement of spatial nearest neighbor filtering and instance belonging constraint, which improves the interaction efficiency while strengthening the overall shape perception and structural consistency modeling of map elements such as lane lines, zebra crossings, and drivable areas.

Images