Map reconstruction method based on historical map tracking, electronic equipment and storage medium

Abstract

The invention relates to a map reconstruction method based on historical map tracking, an electronic device and a storage medium, and the method comprises the steps: constructing an end-to-end tracking enhancement framework, introducing an instance-level historical map to store the trajectory information of map elements, combining a map trajectory prior module, and optimizing the tracking query of subsequent frames through the historical trajectory information, so as to improve the map reconstruction efficiency. And the time continuity and the positioning precision of map elements are improved. Meanwhile, a global geometric perception index is provided, and the integrity and consistency of map reconstruction are evaluated from a global perspective. Experiments show that the single-frame performance (mAP) and time consistency (C-mAP and G-mAP) of the method on nuScenes and Argoverse2 data sets are superior to those of the prior art, high-precision and high-stability global vector map reconstruction is realized, and reliable support is provided for long-term perception and decision-making of automatic driving. The optimal level is achieved on a mainstream map reconstruction data set, and effectiveness and leading of the method are proved.

Description

Technical Field

[0001] This invention relates to the field of map splash technology, and in particular to a map reconstruction method, electronic device, and storage medium based on historical map tracking. Background Technology

[0002] In recent years, with the rapid development of autonomous driving technology, researchers have increasingly utilized onboard sensors to construct high-definition (HD) maps. Early research focused primarily on projecting and upscaling map elements detected on the perspective view (PV) plane into three-dimensional space for map reconstruction. However, to better integrate various sensors such as panoramic cameras and LiDAR, online HD map construction methods have gradually shifted towards bird's-eye view (BEV) representation.

[0003] High-precision maps, as a core component of autonomous driving, need to provide accurate category and geometric information of road elements (such as lane lines and curbs). Existing methods mainly fall into two categories: single-frame detection and temporal modeling.

[0004] Single-frame methods (such as MapTRv2) rely on bird's-eye view (BEV) features to decode map elements. Each frame is processed independently, without utilizing temporal information. In scenarios with occlusion (such as large vehicles obscuring lane lines) or sudden changes in lighting (such as entering or exiting tunnels), the same map element may experience positional jumps or misclassifications in consecutive frames, making it difficult to simultaneously capture the global shape and local details (such as the slight curvature of lane lines) of long-distance road elements (such as a 100-meter-long median strip on a main road).

[0005] Temporal modeling methods (such as MapTracker) maintain consistency through implicit query propagation, passing query features from the previous frame as "memory" to the current frame and predicting the motion offset of the query using Motion MLP to achieve cross-frame association. However, implicit query propagation only encodes historical information through latent features and cannot accurately preserve the geometric trajectory of map elements (such as the positional change of a road edge within 10 seconds), leading to the accumulation of long-term tracking errors. Furthermore, while relying on the vehicle pose transformation matrix to align features across multiple frames, positioning errors during actual driving (such as GPS drift and accumulated errors) can cause feature alignment shifts, further compromising temporal consistency.

[0006] Furthermore, existing metrics (such as mAP based on Chamfer distance) only evaluate single-frame accuracy and lack quantitative assessments of global map continuity (such as whether the topology of the entire intersection is consistent) and integrity (such as whether long-distance lane lines are complete and unbroken).

[0007] In summary, existing HD map construction methods are insufficient in terms of stable and robust map element representation in complex urban scenarios and perception of global map continuity. There is an urgent need for a reconstruction method that can explicitly utilize historical maps, enhance temporal continuity, and evaluate from a global perspective, in order to solve or partially solve the problem of insufficient temporal consistency in high-precision map reconstruction in autonomous driving. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the existing technology by providing a map reconstruction method, electronic device, and storage medium based on historical map tracking. This aims to address the problems of insufficient temporal perception consistency and poor global geometric continuity in existing high-precision map construction methods in complex urban scenarios. By explicitly modeling historical trajectory information and fusing it with spatiotemporal priors, high-precision and highly consistent reconstruction of the global vector map is achieved, improving the stability and reliability of autonomous driving systems in dynamic environments.

[0009] The objective of this invention can be achieved through the following technical solutions:

[0010] One aspect of the present invention provides a map reconstruction method based on historical map tracking, comprising the following steps:

[0011] Obtain vehicle surround view image information and extract multi-level visual features from multi-view images;

[0012] Based on the aforementioned visual features, bird's-eye view features and perspective view features are extracted;

[0013] Construct and initialize a historical map that represents the trajectory information of the tracked instances and decays over time;

[0014] Based on the bird's-eye view features, feature perception is performed. By detecting new instances and matching already tracked instances, the historical map is updated, and a mapping relationship between historical trajectory information and corresponding tracking queries is established.

[0015] Effective pixel masks are obtained by filtering based on historical maps. Perspective view sampling features and bird's-eye view sampling features are obtained based on perspective view features and bird's-eye view features respectively. Tracking queries that integrate category embedding are used as cross-attention queries. Perspective view sampling features and bird's-eye view sampling features are used as key and value respectively to obtain tracking queries that integrate historical map trajectory information.

[0016] Map reconstruction is achieved by integrating historical map trajectory information for tracking and querying, predicting point coordinates and instance categories.

[0017] As a preferred technical solution, the process of obtaining the perspective view sampling features includes the following steps:

[0018] Historical maps are filtered based on preset map thresholds to obtain effective pixel masks;

[0019] The coordinate position embedding of the instance is obtained through 3D position encoding;

[0020] Camera embedding is obtained based on the transformation matrix from vehicle to image;

[0021] By fusing the coordinate position embedding and the camera embedding, the perspective view position encoding is obtained;

[0022] Project the effective pixel mask onto the perspective view space;

[0023] The perspective view position code is fused with the perspective view feature, and the fused feature is sampled using an effective pixel mask projected onto the perspective view space to obtain the perspective view sampled feature.

[0024] As a preferred technical solution, the process of acquiring the bird's-eye view sampling features includes the following steps:

[0025] Obtain the sine position code of the instance;

[0026] The sinusoidal position code is fused with the bird's-eye view features, and position-guided sampling is performed based on the effective pixel mask to obtain the bird's-eye view sampling features.

[0027] As a preferred technical solution, the following comprehensive performance evaluation process for map reconstruction is also included:

[0028] The vectorized real-world annotation elements in multiple consecutive frames of vehicle surround view images are projected onto a preset global coordinate system and rasterized to generate a global ground reality map.

[0029] The reconstructed map to be evaluated is obtained. For polygonal targets, the rasterized mAP index based on IoU is used for evaluation. For polyline map elements, the global prediction result is obtained by merging the tracking results. The discrete points of the instances are extracted by sampling the farthest point, the complete polyline is fitted, and the evaluation is combined with the Chamfer distance.

[0030] The final evaluation index is obtained by statistically analyzing the number of true positives and false positives through global instance matching and averaging the evaluation results of multiple categories.

[0031] As a preferred technical solution, the global instance matching includes the following steps:

[0032] Matching multiple smaller-scale real map instances based on a single predicted instance;

[0033] Invalid false positive matches are filtered out by adjusting the validity threshold.

[0034] As a preferred technical solution, the process of feature perception based on the bird's-eye view features, and updating the historical map through the detection of new instances and the matching of already tracked instances, includes the following steps:

[0035] For each instance detected at the current moment, determine whether a corresponding tracking index exists. If not, treat it as a new instance; otherwise, treat it as an already tracked instance.

[0036] Calculate the confidence score of the instance. If the confidence score exceeds the detection threshold / tracking threshold, update it using the following formula:

[0037]

[0038] In the formula, For a historical map at time t with tracking index i, Raster() is the rasterization function. Let be the vectorized map representation of instance j at time t. Let λ be the confidence score of instance j at time t, and λ be the time decay factor. For the previous frame of historical map retrieved based on tracking index i, max() means taking the maximum value;

[0039] In the next moment, the historical map is dynamically aligned based on the vehicle's latest autonomous vehicle position.

[0040] If the confidence score of a tracked instance is less than the tracking threshold, the corresponding tracking query and historical map are removed from the aligned historical map set, and the total number of tracked instances is reduced accordingly.

[0041] As a preferred technical solution, the category embedding is obtained by encoding semantic category information based on historical instances.

[0042] As a preferred technical solution, the process of extracting bird's-eye view features based on the visual features includes the following steps:

[0043] Based on the visual features, the discrete depth distribution of each pixel is predicted by depth estimation.

[0044] Based on the discrete depth distribution, the visual features are mapped to three-dimensional space to form spatial voxel features;

[0045] Based on the pre-acquired camera intrinsic and extrinsic parameters, the spatial voxel features are projected onto the bird's-eye view coordinate system, and the features from different perspectives are spatially aggregated to obtain the initial bird's-eye view features.

[0046] The initial bird's-eye view features are enhanced by convolution and positional encoding to obtain bird's-eye view features that are spatially consistent and context-aware.

[0047] In another aspect, an electronic device is provided, comprising: one or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for performing the aforementioned map reconstruction method based on historical map tracking.

[0048] In another aspect, the present invention provides a computer-readable storage medium including one or more programs executable by one or more processors of an electronic device, said one or more programs including instructions for performing the aforementioned map reconstruction method based on historical map tracking.

[0049] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0050] (1) End-to-end construction of high-precision vectorized map is realized: The present invention constructs a historical map that represents the trajectory information of the tracked instance and decays over time, and initializes it. Based on the bird's-eye view features, feature perception is performed. Through the detection of new instances and the matching of already tracked instances, the historical map is updated, and the mapping relationship between historical trajectory information and corresponding tracking query is established. By maintaining a historical map under the tracking paradigm and introducing a map prior mechanism for tracking query, the end-to-end construction of high-precision vectorized map is realized.

[0051] (2) Achieve efficient fusion of spatiotemporal priors: This invention uses instance-level historical rasterized maps to store the historical information of global instances in a fine-grained manner, solving the problem of geometric detail loss in implicit propagation, reducing redundant calculations, and improving time consistency.

[0052] (3) Achieve global evaluation: This invention provides a comprehensive performance evaluation process for map reconstruction, which directly evaluates the geometric continuity of the global scene, making up for the shortcomings of existing indicators that only focus on single frames or indirect consistency, and is more in line with the actual needs of map data collection and long-sequence perception in autonomous driving.

[0053] (4) Explicit historical trajectory modeling: This invention uses instance-level historical rasterized maps to store the historical information of global instances in a fine-grained manner, which solves the problem of geometric detail loss in implicit propagation, reduces redundant calculations, and improves time consistency. Attached Figure Description

[0054] Figure 1 This is a flowchart of the map reconstruction method based on historical map tracking in the embodiment;

[0055] Figure 2 This is a schematic diagram of the map reconstruction model framework based on historical map tracking trajectory in the embodiment;

[0056] Figure 3 This is a schematic diagram of the map trajectory prior in the embodiment;

[0057] Figure 4 This is a schematic diagram of a perception example for map reconstruction in the embodiment;

[0058] Figure 5 This is a schematic diagram of the electronic device in the embodiment. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0060] Example 1

[0061] To address the aforementioned problems in existing technologies, this embodiment provides a map reconstruction method based on historical map tracking. On one hand, it provides an end-to-end historical map tracking framework (HisTrackMap), which explicitly maintains instance-level historical rasterized maps and meticulously stores the historical trajectory information of global map instances (such as road dividers and boundaries). This reduces redundant computation while providing stable spatiotemporal priors for subsequent frames, enabling continuous tracking of map elements across frames and significantly improving the temporal consistency and construction efficiency of the global vector map. On the other hand, it provides a map trajectory prior fusion module. This module fuses historical trajectory priors with current perceived features (BEV and PV features), allowing tracking queries to inherit historical geometric details and semantic information through a cross-attention mechanism. It utilizes effective areas of the historical map to sample key features and combines location encoding to enhance spatiotemporal perception, optimizing the accuracy of inter-frame transformations and avoiding the loss of geometric details during implicit propagation. Finally, a global geometric perception evaluation index (G-mAP) was constructed, filling the gap of existing indices that only focus on single-frame or local accuracy. By projecting the local map of continuous frames onto the global coordinate system, the IoU evaluation is used for polygonal elements (such as sidewalks), and the accuracy is calculated by combining Chamfer distance for polyline elements (such as dividing lines). It also allows a single prediction to match multiple short-distance real instances, realizing the direct quantification of the continuity and integrity of the global map. This is more in line with the actual autonomous driving requirements for global map quality and provides key technical support for the stable operation of autonomous driving systems.

[0062] See Figure 1 The method includes the following steps:

[0063] S1. Extract visual features from the surround-view cameras:

[0064] Autonomous driving surround-view cameras are a set of cameras installed around a vehicle, providing a 360° panoramic view to capture environmental information such as roads, pedestrians, obstacles, and traffic signs, supporting environmental perception, near-field object detection, and local navigation. To fully utilize the image information acquired by the surround-view cameras, this method employs ResNet50 as the feature extraction network to extract multi-level visual features from multi-view images. ResNet50, through its deep residual structure, extracts shallow edge and texture information and deep global semantic information layer by layer, generating multi-scale feature representations. Shallow features preserve local details, while deep features capture global semantics. Multi-scale integration generates high-quality input features for subsequent encoder use, providing accurate feature support for tasks such as object detection and map reconstruction.

[0065] S2. Construct an encoder to obtain unified bird's-eye view features.

[0066] The encoder is built on the Lift-Splat-Shoot (LSS) architecture to extract uniform bird's-eye view (BEV) features and perspective view (PV) features from multi-view panoramic camera images.

[0067] The encoder is built on a Lift-Splat-Shoot (LSS) architecture, first extracting Perspective View (PV) features from multi-view panoramic camera images. Specifically, the input multi-view images are first processed by a convolutional neural network (CNN) backbone to extract multi-scale two-dimensional feature maps, forming PV feature representations. These PV features preserve spatial and semantic information from the camera's perspective, providing high-quality input for subsequent spatial projection. Subsequently, using the depth estimation and voxel projection modules in LSS, the PV features are mapped to the Bird's-Eye (BEV) coordinate system, thereby obtaining a unified BEV feature representation.

[0068] The process of extracting features from the bird's-eye view includes the following steps: First, using the visual features obtained from the surround-view cameras, the discrete depth distribution of each pixel is predicted through a depth estimation module. Then, the image features are mapped to three-dimensional space according to the depth probability to form spatial voxel features. Next, the voxel features are projected onto a unified BEV coordinate system through the camera's intrinsic and extrinsic parameters, and the features from different cameras are spatially aggregated. Finally, the initial BEV features are enhanced through a convolution module and position encoding to generate a BEV representation with spatial consistency and context awareness, which is used for subsequent detection and representation of map elements.

[0069] S3. Construct the historical map and initialize it.

[0070] Historical maps are maintained for the tracked instances generated during online prediction and are used to store their trajectory information. During propagation, each tracking query corresponds to a unique mapping instance and historical map, thus establishing a strict one-to-one correspondence.

[0071] For example, at the beginning of frame t This represents the set of historical maps corresponding to a series of tracking queries, defined as follows: Each of them This represents a rasterized history map of the i-th instance, which belongs to the instances tracked in the previous frame. Here are some examples. H and W are equivalent to the height and width of the bird's-eye view (BEV) feature, respectively.

[0072] S4. Dynamically update and maintain historical maps.

[0073] 41) Subsequently, the perception results at time t are used to update the historical map. For the j-th map instance, if it is determined to be a new instance (i.e., no corresponding tracking index was found, and its confidence score is...) Exceeding the detection threshold τ det Then, the rasterization function Raster(·) is used to represent its vectorized map. Perform rasterization processing and sort the rasterization results according to confidence level. Weighting is used to initialize the historical map for this instance. If the instance has been tracked (i.e., a corresponding tracking index exists, and its confidence level is...), then... Exceeding the tracking threshold τ track Then, retrieve the previous frame of the historical map based on the tracking index i. After temporal decay using a time decay factor λ, the map is fused and updated with the newly generated rasterized result at the current moment to obtain the latest historical map. Its update rules are as follows:

[0074]

[0075] This update mechanism can explicitly maintain the temporal continuity and spatial representation consistency of each map instance, ensuring that historical maps always maintain accurate perception and updates of target elements during navigation.

[0076] 42) After the update process is complete, the final historical map will be obtained. At time t, the system will synchronously update the total number of currently tracked instances. and historical map collection M t At time step t+1, to achieve cross-frame initialization, the historical map M... t The system will dynamically align the vehicle based on its latest self-positioning and generate an aligned transformation history map. This alignment operation can be represented as:

[0077]

[0078] Among them, T t+1 This represents a standard 4×4 spatial transformation matrix, depicting the spatial transformation relationship of the coordinate systems between two consecutive time frames. Warp(·) is the historical map alignment function. Through the above alignment operation, the historical perception information at time step t can be effectively transformed to the spatial reference frame at time step t+1, thus providing accurate initialization for the construction of the next frame map.

[0079] 43) It should be noted that if the confidence level of a certain tracing query is lower than the tracing threshold τ, track If this is not the case, it indicates that the instance has disappeared in the current frame or was not successfully tracked. Therefore, it is necessary to retrieve the data from the aligned historical map set. Remove its corresponding tracking query and historical map M from the middle. r And reduce the total number of tracking instances accordingly. The above operations can be formally represented as:

[0080]

[0081] Through the above mechanism, each map instance follows the process of "initialization - tracking - historical map update" to ensure that a one-to-one mapping relationship is established between its historical trajectory information and the corresponding tracking query, thereby realizing continuous perception and information transmission across time frames.

[0082] S5. Use the map trajectory prior module to optimize tracking queries.

[0083] 51) As mentioned above, to address the difficulty in capturing fine-grained feature changes during motion propagation of implicit temporal information, this step provides a "map trajectory prior fusion module," such as... Figure 3 As shown, this method explicitly integrates historical map trajectory information into the tracking query, thereby improving the efficiency of utilizing temporal priors and perceptual consistency. The historical information provides both semantic categories and spatial trajectory coordinates as priors. To enhance the semantic representation capability of the tracking query, this embodiment defines an initial category embedding CE. init ∈R 3×C (Covering three categories: dividing lines, boundaries, and sidewalks), and encoding historical instance category information as... Where N track The number of tracking queries at each time step is added to the category embedding and fused with the tracking queries to inject category priors. The expression is as follows:

[0084] Q track =Q track +CE

[0085] 52) Because historical maps have a time decay characteristic, a map threshold τ can be set. map Filter historical map M and generate effective pixel mask M val To enhance the spatial perception capability of perspective features in three-dimensional space, a three-dimensional position encoding method was adopted, further introducing coordinate position embedding PE. coords ∈R h×w×C And the ego-to-image matrix E2I∈R cams×4×4 The generated camera embedding PE cam The two are added together to form the final perspective position code PE. pv .

[0086] Where h and w represent the height and width of the PV feature map, respectively, and cams represents the number of cameras. This module achieves accurate initialization of tracking queries by fusing historical trajectory information with current perceived features, providing crucial support for stable temporal mapping.

[0087] 53) Subsequently, the effective pixel mask M is projected using the projection function Proj(·). val Projected into the perspective view space. Within this space, the perspective view feature F pv Its corresponding position code PE pv The features are then added together to form an enhanced feature representation. Subsequently, the sampling function SampledPV(·) is used to sample features at specific locations from the fused features, yielding the final perspective view sampling feature F. sampled_pv The calculation expression is:

[0088] F sampled_pv =SampledPV(Proj(M) val ,E2I),F pv +PE pv )

[0089] This operation effectively extracts local perceptual information with geometric priors and temporal consistency by combining the effective pixel locations in the historical map with the spatial features of the current frame, which is then used for feature initialization of subsequent tracking queries.

[0090] 54) Similarly, perform the corresponding operations in the bird's-eye view (BEV) space. To ensure the sampling of BEV features F sampled_bev Including sufficient position encoding information, this invention introduces sinusoidal position encoding (PE). bev Subsequently, based on the effective pixel mask M val The sampling function SampledBEV(·) is used to analyze the BEV features F that incorporate position encoding. bev +PEbev Position-guided sampling is performed to obtain the final BEV sampling features F. sampled_bev Its calculation expression is:

[0091] F sampled_bev =SampledBEV(M val ,F bev +PE bev )

[0092] Using the above methods, BEV perception features aligned with spatial location can be effectively extracted from historical maps, providing well-structured and context-rich spatial prior information for tracking queries.

[0093] 55) Next, for each map instance, its corresponding perspective view features (PV) and bird's-eye view features (BEV) have been obtained, including perceptual features obtained from historical map trajectory sampling. The corresponding F will be tracked and queried. sampled_pv and F sampled_bev Both are input into the cross-attention module CA(·) to complete the initialization operation of the tracking query in the current time frame. The specific calculation expression is as follows:

[0094] Q track =CA(Q) track ,F sampled_pv )

[0095] Q track =CA(Q) track ,F sampled_bev )

[0096] Ultimately, by explicitly utilizing historical trajectory information, this invention provides accurate spatial priors for tracking queries, avoiding redundant supervision and temporal transformation errors present in traditional implicit propagation. Furthermore, given the inconsistent number of effective pixels corresponding to different instances, this invention uniformly fills the sampled features and introduces a padding mask to ensure that each tracking query only focuses on its corresponding spatial location region, thereby achieving a dual optimization of location accuracy and feature representation.

[0097] S6. Use historical trajectory maps to perceive the environment for autonomous driving.

[0098] The tracking query uses a multilayer perceptron (MLP) as the regression head for predicting point coordinates and map instance categories, achieving effective perception of the geometric location and category information of each map element. Through end-to-end training, an integrated output from tracking queries to accurate geometric coordinates, semantic categories, and reliability scores is achieved, providing a solid foundation for high-quality reconstruction of global vector maps.

[0099] Preferably, the method further includes:

[0100] S7. Evaluate the overall performance of the map reconstruction model based on historical map tracking.

[0101] Existing map construction evaluation metrics have the following limitations: (1) The mean accuracy (mAP) of a single frame only measures local and instantaneous performance and cannot fully reflect the overall quality and consistency of the map over a continuous time period; (2) Although the consistency-aware metric C-mAP can indirectly assess temporal consistency, it cannot explicitly measure the integrity of the map. Therefore, this invention proposes a global geometry-aware evaluation metric (G-mAP) to quantitatively evaluate map construction quality from a global perspective. Specifically, firstly, a continuous N... seq Vectorized ground truth annotation elements in the frame Projected onto a unified global coordinate system and rasterized, a global ground reality map is generated. Right now:

[0102]

[0103] RasterGlobal(·) represents the process of converting local coordinates to global coordinates and then rasterizing them. Considering the truncation problem of polygonal targets (such as pedestrians) during vehicle movement, a rasterized mAP metric based on IoU is used for evaluation.

[0104] For polyline map elements (such as lane lines and road boundaries), the Merge(·) function is used to obtain the global prediction result:

[0105]

[0106] It utilizes the farthest point sampling (FPS) method to extract discrete points of instances, fits complete polylines, and calculates its accuracy using Chamfer distance. G-mAP comprehensively employs raster-based AP (for polygons) and vector-based AP (for polylines) to measure the construction quality of three core map elements (pedestrian walkways, lane lines, and boundaries), and averages the results to derive an overall evaluation index, demonstrating comprehensive and stable assessment capabilities.

[0107] In the instance matching process, a distance cost matrix is ​​introduced. Measure the average point coordinate distance between each predicted instance and the real instance, and calculate the average confidence score of the predicted instance. Set distance threshold τ dis and validity threshold τ valid It is used to control the range of true positive (TP) and false positive (FP) results.

[0108] To accommodate the diversity of instances from a global perspective, this invention optimizes the matching algorithm: allowing a predicted instance to match multiple smaller-scale real map instances to prevent them from being ignored; simultaneously, for unlabeled but perceived instances, a τ is set... valid Limiting the statistical scope of FP improves the practicality and robustness of the evaluation results.

[0109] To verify the performance of this method, the following experiments were designed.

[0110] Regarding datasets, the proposed HisTrackMap method was evaluated on two mainstream publicly available autonomous driving datasets: nuScenes and Argoverse 2. The nuScenes dataset contains 1,000 scenes, each lasting approximately 20 seconds, providing images from six synchronized RGB cameras and precise pose information. The Argoverse 2 dataset also contains 1,000 sequences, equipped with seven ring cameras, two stereo cameras, LiDAR point clouds, and map-aligned six-DOF pose data. To ensure comprehensive evaluation, experiments were conducted on both older and newer versions of these datasets.

[0111] Regarding evaluation metrics, referring to existing work, the mean average precision (mAP) is used as the main evaluation indicator, with evaluation thresholds of 0.5 meters, 1.0 meters, and 1.5 meters to calculate the accuracy performance of pedestrian walkways, lane lines, and road boundaries, respectively. Furthermore, the consistency perception index C-mAP is employed, and the global geometric perception enhancement index G-mAP proposed in this invention is introduced to achieve a comprehensive evaluation of the map construction effect.

[0112] (1) MapTRv2, a model that uses point queries for perception; (2) HRMapNet, a perception model that maintains city-level maps; (3) MGMap, a model that uses a raster structure to optimize point queries; (4) Mask2Map, a model that uses raster results for priors; (5) MapTracker, a model for a tracking paradigm.

[0113] The specific experimental results are shown in Tables 1 and 2.

[0114] Table 1. Performance (%) of different map reconstruction models on the nuScenes dataset

[0115]

[0116] Table 2. Performance (%) of different map reconstruction models on the Argoverse2 dataset

[0117]

[0118] From Table 1 and Table 2 and Figure 4 It can be seen that this method achieves better results compared to other map reconstruction models.

[0119] Example 2

[0120] This embodiment, based on Embodiment 1, provides a map reconstruction system based on historical map tracking. The main component includes the HisTrackMap model structure, see [link to documentation]. Figure 2 The model structure of HisTrackMap is shown. HisTrackMap's overall network architecture is similar to other end-to-end high-definition map building models like DETR. HisTrackMap's architecture mainly includes an encoder, a tracking decoder, and a historical map management module.

[0121] (1) The encoder extracts BEV and PV features from the multi-view image, corresponding to steps S1-S2 in Example 1.

[0122] (2) The tracking decoder (i.e., the map decoder) fuses instance-level historical maps with current perceived features through the map trajectory prior module, stacking multiple layers of deformable attention mechanisms to generate prediction results for map elements. The prediction of each layer not only includes the category information of road elements, but also covers their geometric location, ensuring a comprehensive description of map elements within the perception range. The final output is a vector map element, corresponding to step S5 in Embodiment 1.

[0123] The decoder consists of L layers, each containing a self-attention layer, a deformable cross-attention layer, and a feedforward neural network. The query used consists of two parts: a detection query and a tracking query, where the detection query Q... det Used to capture new map elements in the current frame; tracking query Q track It can inherit and continue the geometric and semantic consistency of historical maps, and combine deformable cross attention to accurately sample the current BEV / PV features, ultimately achieving stable association and reconstruction of map elements across frames, significantly improving the temporal continuity of the global vector map.

[0124] The fusion process of cross-attention mechanism and tracking query is achieved through multimodal feature sampling and spatiotemporal alignment. During PV feature sampling, the effective region M in the historical rasterized map is selected. val (through threshold τ) map The selected samples are mapped to the perspective view (PV) space using the camera extrinsic parameter matrix (E2I, describing the transformation from the vehicle coordinate system to the image coordinate system) via the projection function Proj(·). Position encoding enhancement aims to capture the spatial relationships of the sampled points and generate a sinusoidal position embedding PE. pvThis is then added to the original PV features to form the enhanced feature F. pv +PE pv In the projected PV space, with M val Centered on the mapped region, multiple key sampling points are selected using a deformable attention mechanism. Each sampling point is then surrounded by multiple neighboring points. Attention weights are calculated and aggregated to obtain F. sampled_pv This feature preserves the geometric information of the historical trajectory while incorporating the semantic details of the current frame.

[0125] During BEV feature sampling, M val Direct positioning within the BEV space, using its grid coordinates as a sampling index, achieves BEV space alignment. A sinusoidal position embedding PE based on the BEV grid is then generated. bev Adding the BEV features to obtain F bev +PE bev The location encoding incorporates both spatial coordinates and timestamp information to distinguish the same location at different times, achieving spatiotemporal location encoding. A multi-scale BEV feature pyramid is employed, assigning reference points of different scales to each query point. A deformable attention mechanism is used to sample and aggregate features from each scale, yielding the F... sampled_bev This enhances the ability to perceive map elements of different sizes and enables multi-scale feature fusion.

[0126] The cross-attention fusion mechanism includes a query-feature interaction process, which uses the initialized tracking query as the Query, F sampled_pv and F sampled_bev Using these as Key and Value respectively, multimodal information fusion is achieved through two cross-attention operations. In the cross-attention calculation, the contribution weights of PV and BEV features are dynamically adjusted; for example, the weight of BEV features is increased in occluded scenes, and the influence of PV features is enhanced in texture-rich regions. A spatiotemporal distance metric is introduced into the attention weight calculation, giving higher weights to feature points near historical trajectories, ensuring geometric continuity between the current prediction and historical trajectories, and achieving spatiotemporal consistency constraints. This fusion process, through a cross-attention mechanism combining bimodal feature sampling and spatiotemporal awareness, effectively integrates prior knowledge of historical trajectories with multi-view perception information of the current frame, providing rich and consistent feature representations for tracking queries, and significantly improving the cross-frame association accuracy and geometric localization accuracy of map elements.

[0127] For the map trajectory prior module, this module utilizes historical maps to provide refined prior information for tracking queries in subsequent frames, improving temporal smoothness. First, it extracts the semantic category information of instances from the historical map and encodes it as a category embedding, enhancing the semantic representation of the tracking query. An initial category embedding (CE) is defined. init ∈R 3×C(Covering three categories: dividing lines, boundaries, and sidewalks), and encoding historical instance category information as... And it is integrated with tracking queries, i.e., Q track =Q track +CE, assigning prior knowledge to the tracking query category; in the perspective view (PV) and bird's-eye view (BEV) feature spaces, sampling features based on effective areas of historical maps, with a map threshold τ. map Filter historical map M, obtain valid pixels to generate M val It focuses on key areas. By fusing cross-attention mechanism with tracking query, it combines historical trajectory information with current perception features to optimize the accuracy of cross-frame tracking.

[0128] (3) The historical map management module dynamically updates historical trajectory information to ensure cross-frame consistency. This architecture constructs an end-to-end process of historical trajectory storage, spatiotemporal prior fusion, and global consistency optimization, reducing computational redundancy and improving temporal continuity, corresponding to steps S3-S4 in Implementation Example 1.

[0129] To clearly store past perception results, an instance-level historical rasterized map was designed. It can record the historical information of different global map instances (such as road dividers and boundaries) in detail. For newly detected instances (with confidence exceeding the detection threshold), its vector coordinates are converted into raster format and the historical map is initialized. For tracked instances (with confidence exceeding the tracking threshold), the historical map is updated by combining a time decay factor. This preserves effective historical information while incorporating the latest perception results, ensuring the continuity of historical trajectories.

[0130] The map reconstruction method based on historical map tracking achieves high-precision and high-consistency reconstruction of global vector high-precision maps through three core modules and an end-to-end process.

[0131] Considering that existing metrics (such as single-frame mAP) only focus on instantaneous accuracy within a single frame, making it difficult to evaluate the geometric continuity and integrity of the global map in a continuous sequence, a global geometric perception evaluation metric (G-mAP) is proposed.

[0132] The global geometric perception evaluation index first projects local map elements (including ground truth labels and predicted results) from multiple consecutive frames onto a unified global coordinate system through coordinate system transformation, thereby constructing a global ground truth map and a global predicted map containing complete scene information. Based on this, differentiated evaluation strategies are adopted for different types of map elements. For polygonal elements (such as sidewalks), the spatial coverage accuracy is evaluated by calculating the Intersection over Union (IoU) ratio between the predicted and ground truth regions. For polyline elements (such as road dividers and boundaries), the average accuracy (AP) is calculated by combining the Chamfer distance (which measures the average distance between the predicted and ground truth point sets). Simultaneously, an innovative global instance matching mechanism is introduced, allowing a single predicted instance to be matched with multiple short-distance ground truth instances (e.g., a long predicted divider can correspond to multiple short ground truth dividers), avoiding misjudgments caused by local truncation or segmented labeling. Finally, the evaluation results of the three types of elements (dividers, boundaries, and sidewalks) are averaged to achieve a comprehensive quantification of the geometric consistency, continuity, and completeness of the global map, better aligning with the requirements for global map quality in long-distance navigation during actual autonomous driving.

[0133] Example 3

[0134] Based on the foregoing embodiments, this embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for executing the map reconstruction method based on historical map tracking as described in Embodiment 1.

[0135] like Figure 5 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The method described herein. Of course, in addition to software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0136] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0137] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0138] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A map reconstruction method based on historical map tracking, characterized by, Includes the following steps: Obtain vehicle surround view image information and extract multi-level visual features from multi-view images; Based on the aforementioned visual features, bird's-eye view features and perspective view features are extracted; Construct and initialize a historical map that represents the trajectory information of the tracked instances and decays over time; Based on the bird's-eye view features, feature perception is performed. By detecting new instances and matching already tracked instances, the historical map is updated, and a mapping relationship between historical trajectory information and corresponding tracking queries is established. Effective pixel masks are obtained by filtering based on historical maps. Perspective view sampling features and bird's-eye view sampling features are obtained based on perspective view features and bird's-eye view features respectively. Tracking queries that integrate category embedding are used as cross-attention queries. Perspective view sampling features and bird's-eye view sampling features are used as key and value respectively to obtain tracking queries that integrate historical map trajectory information. Map reconstruction is achieved by integrating historical map trajectory information for tracking and querying, predicting point coordinates and instance categories.

2. The map reconstruction method based on historical map tracking according to claim 1, characterized in that, The process of obtaining the perspective view sampling features includes the following steps: Historical maps are filtered based on preset map thresholds to obtain effective pixel masks; The coordinate position embedding of the instance is obtained through 3D position encoding; Camera embedding is obtained based on the transformation matrix from vehicle to image; By fusing the coordinate position embedding and the camera embedding, the perspective view position encoding is obtained; Project the effective pixel mask onto the perspective view space; The perspective view position code is fused with the perspective view feature, and the fused feature is sampled using an effective pixel mask projected onto the perspective view space to obtain the perspective view sampled feature.

3. The map reconstruction method based on historical map tracking according to claim 2, characterized in that, The process of acquiring the bird's-eye view sampling features includes the following steps: Obtain the sine position code of the instance; The sinusoidal position code is fused with the bird's-eye view features, and position-guided sampling is performed based on the effective pixel mask to obtain the bird's-eye view sampling features.

4. The map reconstruction method based on historical map tracking according to claim 1, characterized in that, This also includes the following comprehensive performance evaluation process for map reconstruction: The vectorized real-world annotation elements in multiple consecutive frames of vehicle surround view images are projected onto a preset global coordinate system and rasterized to generate a global ground reality map. The reconstructed map to be evaluated is obtained. For polygonal targets, the rasterized mAP index based on IoU is used for evaluation. For polyline map elements, the global prediction result is obtained by merging the tracking results. The discrete points of the instances are extracted by sampling the farthest point, the complete polyline is fitted, and the evaluation is combined with the Chamfer distance. The final evaluation index is obtained by statistically analyzing the number of true positives and false positives through global instance matching and averaging the evaluation results of multiple categories.

5. The map reconstruction method based on historical map tracking according to claim 4, characterized in that, The global instance matching includes the following steps: Matching multiple smaller-scale real map instances based on a single predicted instance; Invalid false positive matches are filtered out by adjusting the validity threshold.

6. The map reconstruction method based on historical map tracking according to claim 1, characterized in that, The process of feature perception based on the bird's-eye view features, and updating the historical map through the detection of new instances and the matching of already tracked instances, includes the following steps: For each instance detected at the current moment, determine whether a corresponding tracking index exists. If not, treat it as a new instance; otherwise, treat it as an already tracked instance. Calculate the confidence score of the instance. If the confidence score exceeds the detection threshold / tracking threshold, update it using the following formula: wherein, is the history map at time t with tracking index i, Raster() is a rasterization function, is the vectorized map representation of instance j at time t, is the confidence score of instance j at time t, and λ is a temporal decay factor, is the previous frame history map retrieved based on tracking index i, and max() denotes taking the maximum value. In the next moment, the historical map is dynamically aligned based on the vehicle's latest autonomous vehicle position. If the confidence score of a tracked instance is less than the tracking threshold, the corresponding tracking query and historical map are removed from the aligned historical map set, and the total number of tracked instances is reduced accordingly.

7. The map reconstruction method based on historical map tracking according to claim 1, characterized in that, The category embedding is obtained by encoding semantic category information based on historical instances.

8. The map reconstruction method based on historical map tracking according to claim 1, characterized in that, The process of extracting bird's-eye view features based on the aforementioned visual features includes the following steps: Based on the visual features, the discrete depth distribution of each pixel is predicted by depth estimation. Based on the discrete depth distribution, the visual features are mapped to three-dimensional space to form spatial voxel features; Based on the pre-acquired camera intrinsic and extrinsic parameters, the spatial voxel features are projected onto the bird's-eye view coordinate system, and the features from different perspectives are spatially aggregated to obtain the initial bird's-eye view features. The initial bird's-eye view features are enhanced by convolution and positional encoding to obtain bird's-eye view features that are spatially consistent and context-aware.

9. An electronic device, comprising: include: One or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for performing the map reconstruction method based on historical map tracking as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, It includes one or more programs that are executed by one or more processors of an electronic device, the one or more programs including instructions for performing the map reconstruction method based on historical map tracking as described in any one of claims 1-8.

Images