A robust online vector map construction method for spatio-temporal concise query

Abstract

The application discloses a kind of robust online vector map construction methods of space-time concise query, it is related to automatic driving perception technical field, including: the multi-view feature is fused into unified bird's eye view angle representation;Based on time correlation matching current frame and historical frame instance feature;Update query feature and optimize query representation;Using high-quality vector map construction model as teacher model, focus on instance-centered map construction model as student model, the feature of teacher network is distilled to student network adaptively;Design loss function to evaluate student model, and carry out training optimization, if not reach output requirement, then further optimize student model by middle distance perception mechanism.The application proposes to significantly improve the robustness, accuracy and efficiency of online vector map construction, especially in dynamic environment and complex scene performance, in this way, multi-view, spatiotemporal correlation and complex 3D target detection tasks can be better handled.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving perception technology, specifically to a robust online vector map construction method with spatiotemporal simplified query. Background Technology

[0002] With the rapid development of artificial intelligence technology, the requirements for autonomous driving technology in terms of perception accuracy, environmental adaptability and safety are constantly increasing. In complex dynamic scenarios, especially in driving environments with high dynamism and multi-objective interaction, achieving high-precision and real-time environmental perception and map building has become an important challenge for autonomous driving technology. Vector map building, as a method of representing environmental instances through unified geometric and semantic information, provides key support for scene understanding and decision-making in autonomous driving. However, existing technologies still have many shortcomings when dealing with dynamic environments and large-scale multi-view data.

[0003] Currently, vector map construction methods based on multi-view 2D images can achieve certain results in simple static scenes, but when dealing with complex scenes with occlusion, dynamic targets, and changing viewpoints, they often suffer from insufficient accuracy due to inadequate feature fusion. On the other hand, 3D reconstruction methods based on LiDAR and image fusion can provide higher geometric accuracy, but the data processing is complex and computationally resource-intensive, making it difficult to meet real-time requirements. Furthermore, existing methods lack the ability to consistently correlate target instances and perform spatiotemporal feature modeling during multi-frame data fusion, leading to discontinuities and accuracy degradation in map construction results in dynamic scenes. In addition, existing methods lack optimization and lightweight design for redundant queries in map construction tasks, further limiting their application in resource-constrained scenarios.

[0004] In summary, there is an urgent need for a vector map construction method that combines spatiotemporal simplified querying with efficient instance feature modeling to achieve high-precision, continuous, and real-time map construction capabilities in complex and dynamic scenarios. Furthermore, by introducing knowledge distillation and lightweight optimization mechanisms, the high demands for resource adaptability and model robustness in practical applications can be addressed, further promoting the widespread adoption and application of autonomous driving technology in real-world scenarios. Summary of the Invention

[0005] To address the aforementioned technical problems, a robust online vector map construction method with spatiotemporal simplified query is provided. This technical solution solves the problems mentioned above.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A robust online vector map construction method with spatiotemporal simplified query, characterized by comprising:

[0008] S1: Based on multi-view image preprocessing, feature extraction and geometric correction, multi-view features are fused into a unified bird's-eye view representation to describe the geometric structure and semantic details of the scene;

[0009] S2: Based on temporal correlation matching of instance features between the current frame and historical frames, and through the spatial fusion module combined with the cross-attention mechanism, the historical point set and the current point set are fused to construct a globally continuous and high-precision map representation;

[0010] S3: Based on query classification analysis, identify queries with smaller contributions and prune them, retain high-quality queries, use a self-attention module for interaction, and fuse with image features through a cross-attention mechanism to update query features and optimize query representation;

[0011] S4: In the knowledge distillation stage, a high-quality vector map construction model is used as the teacher model, and an instance-centric map construction model is used as the student model, adaptively distilling features from the teacher network to the student network.

[0012] S5: Design a loss function to evaluate the student model and perform training optimization. If the output requirements are not met, further optimize the student model through a mid-range perception mechanism.

[0013] Preferably, S1 specifically includes:

[0014] The system inputs a sequence of multi-view camera images acquired simultaneously to obtain scene data containing dynamic changes in the surrounding environment;

[0015] Input scene data and preprocess the images, including distortion correction and time synchronization, to ensure alignment between multi-view images; based on the camera's intrinsic and extrinsic parameters, perform geometric calibration on the images from each viewpoint to construct a globally consistent environment representation;

[0016] The deep learning backbone network extracts features from images from each viewpoint, transforming image data into high-dimensional feature tensors; low-order features are extracted through convolutional layers, capturing texture and edge details in the image.

[0017] High-order features are modeled through a multi-head attention mechanism, focusing on global relationships and contextual associations between different objects in the scene, generating a high-dimensional feature representation tensor for each viewpoint;

[0018] Based on the vehicle's geometric information, the extracted high-dimensional features are uniformly aligned in space through projection transformation, and geometric correction is used to project the features from multiple perspectives into a unified bird's-eye view coordinate system.

[0019] By fusing features from multiple perspectives and transforming coordinate systems, features from multiple cameras are aligned into a two-dimensional plane representation. In the BEV coordinate system, each pixel contains high-dimensional feature information generated by fusing multiple perspectives.

[0020] Preferably, S2 specifically includes:

[0021] Based on BEV features, the instance features at the current time are matched with the instance features at historical time through the instance center time association module. The feature alignment technology is used to perform association transformation on historical instances in the geometric and feature dimensions, so as to unify them with the current instance into the same semantic space.

[0022] By using a learnable temporal correlation mechanism, the matching relationship between historical instances and current instances is dynamically modeled in a high-dimensional feature space, capturing cross-temporal feature correlation changes.

[0023] Based on the instance-centric spatial fusion module, the point set features of historical frames and the point set features of the current frame are spatially aligned, and the local features of historical point sets and the current point sets are efficiently aligned and fused through cross-attention mechanism.

[0024] Spatial sampling is performed on the historical global map point set, and cross-attention operation is applied to the fusion process of detection results and historical features to strengthen the interaction relationship of instances in the spatial domain. The fusion result is optimized by curve fitting technology.

[0025] Based on the temporal correlation features and spatial fusion features, the spatiotemporal dimension is jointly modeled, and the changing trends of instances in the temporal and spatial domains are captured through multi-layer network design.

[0026] In the time dimension, based on the dynamically updated instance memory buffer, the current instance is associated in real time by combining historical information of multi-frame features to generate a unified instance identifier.

[0027] In the spatial dimension, a three-layer cross-attention mechanism is used to perform deep interactive modeling of current features and historical features, which strengthens the spatial consistency of global map features. A classification optimization strategy is combined to perform fine processing on different types of map instances. Based on the optimized instance features, a global vectorized map representation is generated, which integrates geometric, temporal and semantic information to describe the details and spatial relationships of each instance in the global map.

[0028] Preferably, S3 specifically includes:

[0029] Based on query classification analysis, queries with smaller contributions are identified and pruned step by step to remove redundant queries and optimize computational efficiency, while retaining high-quality queries that make significant contributions to map construction.

[0030] By dynamically sorting and multi-level analyzing the classification scores of queries, redundant queries with a weak impact on map construction goals are filtered out.

[0031] A progressively refined strategy is introduced during pruning. High-quality queries that are retained interact through a self-attention module, which efficiently captures the global relationships between queries.

[0032] The optimized query features are fused with historical map sampling points, and based on geometric alignment and feature enhancement strategies, the current query features and historical features are unified into the same spatial domain.

[0033] Deeply analyze the feature distribution of historical sampling points, extract representative and information-rich key points, correct the positional deviation of historical features in the current coordinate system, and enhance semantic consistency with the current query features.

[0034] Through feature enhancement strategies, the model dynamically adjusts the fusion weights of historical and current features, introduces a dynamic feature weighting mechanism, and refines the control of the fusion process based on the semantic importance and contribution of features.

[0035] By employing a recursive update strategy, the feedforward optimized features are integrated layer by layer to maintain consistency and globality in both time and space dimensions. Combined with the geometric and semantic information of historical sampling points, the generated map representation is optimized, thereby ensuring its high accuracy and smoothness in complex scenes.

[0036] Preferably, S4 specifically includes:

[0037] In the knowledge distillation stage, a model is built based on a high-quality vector map and used as a teacher model to provide high-precision feature guidance of the global map for the student model. Through temporal correlation, spatial fusion and efficient pruning, a high-quality vector map containing geometric, semantic and temporal information is generated.

[0038] During the distillation process, an adaptive weighting mechanism is used to extract high-quality global feature representations from the teacher network, including multi-frame temporal information, global geometric semantics, and spatial relationships between instances.

[0039] The features of the teacher model are divided into three parts: global features, regional features, and instance features. The knowledge transfer of each part is optimized by a specific distillation loss function. Global feature distillation guides the student model to capture the whole scene by constructing an overall context representation. Regional feature distillation helps the student network form accurate representations in key areas. Instance feature distillation focuses on improving the student model's robust detection ability for target instances.

[0040] An instance-centric alignment strategy is adopted, and features from the student model are embedded into a semantic space consistent with that of the teacher model through geometric and semantic mapping mechanisms.

[0041] In the distillation process, an adaptive loss balancing module was designed to dynamically adjust the distillation weights of global features and instance features. This module combines scene complexity, number of instances, and feature contribution to allocate weights, enabling the student model to prioritize learning key instance information in complex scenarios and maintain global consistency in simple scenarios. In addition, to address the lightweight requirements of the student model, the distillation process uses multi-layer feature compression technology to transfer the deep semantic features of the teacher model to the student model in a lightweight representation, thereby significantly reducing computational resource consumption while retaining key feature information.

[0042] Through knowledge distillation, the student model achieves efficient and robust 3D object detection by constructing key regions centered on instances without relying on complex global feature modeling.

[0043] Preferably, S5 specifically includes:

[0044] Based on the student model, a loss function is designed to comprehensively evaluate and optimize the student model, ensuring its robustness and high-precision 3D target detection capability.

[0045] The loss function includes classification loss and regression loss, which are used to evaluate the target category prediction ability of the student model and the regression accuracy of the 3D bounding box, respectively.

[0046] To assess classification accuracy, the quality focus loss function is used to calculate the deviation between the target class probability distribution predicted by the student model and the true class label.

[0047] To assess regression accuracy, a smoothed L1 loss function is used to evaluate the difference between the 3D bounding box parameters predicted by the student model and the true values.

[0048] The classification loss and regression loss are combined into a total loss through a weighted summation to guide the optimization of the student model. The formula for calculating the total loss is as follows:

[0049] L total =L cls +L reg

[0050] Based on the aforementioned loss function evaluation metrics, if the output requirements are not met, the student model is further optimized using a mid-range perception mechanism. This involves dynamically adjusting the loss weights for different targets through distance-aware response distillation, with a focus on optimizing the detection accuracy for short-to-medium range targets. The specific formula is as follows:

[0051]

[0052] Where, d i w is the distance between target i and the sensor.i δ represents the normalized weight of the target, μ represents the fixed weight of the distant target, and A, B, C and δ are adjustment parameters.

[0053] Based on the distance-aware response distillation strategy, the student model is trained and optimized multiple times, and the evaluation index of the loss function is recalculated. If L is satisfied... total A threshold requirement of ≤0.1 is considered to meet the output accuracy requirements, ensuring that the student model can efficiently perform 3D object detection tasks in resource-constrained environments, while maintaining a high level of robustness and accuracy in different complex scenarios, and finally outputting the instance-centered vector map construction results built by the student model.

[0054] Preferably, the step of calculating the deviation between the target class probability distribution predicted by the student model and the true class label using the quality focus loss function for classification accuracy specifically includes:

[0055] The formula for calculating the mass focus loss function is as follows:

[0056]

[0057] In the formula, S represents the total number of targets, and p ti and p si The predicted probabilities w for the teacher model and the student model, respectively. i For target i.

[0058] Preferably, the step of using a smoothed L1 loss function to evaluate the difference between the 3D bounding box parameters predicted by the student model and the true values ​​for regression accuracy specifically includes:

[0059] The formula for the smoothed L1 loss function is:

[0060]

[0061] In the formula, r ti and r si are the bounding box regression parameters for the teacher model and the student model, respectively, where S is the total number of targets and SmoothL1 is the smooth L1 loss function.

[0062] Preferably, if the evaluation index based on the loss function does not meet the output requirements, the student model is further optimized using a mid-range perception mechanism. This involves dynamically adjusting the loss weights for different targets through distance-aware response distillation, with a focus on optimizing the detection accuracy of short-to-medium range targets. Specifically, this includes:

[0063] The dynamic adjustment formula is as follows:

[0064]

[0065] Where, di w is the distance between target i and the sensor. i δ represents the normalized weight of the target, μ represents the fixed weight of the distant target, and A, B, C and δ are adjustment parameters.

[0066] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0067] This invention proposes to significantly improve the robustness, accuracy, and efficiency of online vector map construction, especially in dynamic environments and complex scenes. In this way, it can better handle multi-view, spatiotemporal correlation, and complex 3D object detection tasks. Attached Figure Description

[0068] Figure 1 This is a flowchart of the steps of the present invention;

[0069] Figure 2 This is a flowchart of step S2 in the present invention. Detailed Implementation

[0070] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0071] Reference Figure 1 As shown, a robust online vector map construction method with spatiotemporal simplified query is characterized by comprising:

[0072] S1: Based on multi-view image preprocessing, feature extraction and geometric correction, multi-view features are fused into a unified bird's-eye view representation to describe the geometric structure and semantic details of the scene;

[0073] S2: Based on temporal correlation matching of instance features between the current frame and historical frames, and through the spatial fusion module combined with the cross-attention mechanism, the historical point set and the current point set are fused to construct a globally continuous and high-precision map representation;

[0074] S3: Based on query classification analysis, identify queries with smaller contributions and prune them, retain high-quality queries, use a self-attention module for interaction, and fuse with image features through a cross-attention mechanism to update query features and optimize query representation;

[0075] S4: In the knowledge distillation stage, a high-quality vector map construction model is used as the teacher model, and an instance-centric map construction model is used as the student model, adaptively distilling features from the teacher network to the student network.

[0076] S5: Design a loss function to evaluate the student model and perform training optimization. If the output requirements are not met, further optimize the student model through a mid-range perception mechanism.

[0077] Reference Figure 2 As shown, S1 specifically includes:

[0078] The system inputs a sequence of multi-view camera images acquired simultaneously to obtain scene data containing dynamic changes in the surrounding environment;

[0079] Input scene data and preprocess the images, including distortion correction and time synchronization, to ensure alignment between multi-view images; based on the camera's intrinsic and extrinsic parameters, perform geometric calibration on the images from each viewpoint to construct a globally consistent environment representation;

[0080] The deep learning backbone network extracts features from images from each viewpoint, transforming image data into high-dimensional feature tensors; low-order features are extracted through convolutional layers, capturing texture and edge details in the image.

[0081] High-order features are modeled through a multi-head attention mechanism, focusing on global relationships and contextual associations between different objects in the scene, generating a high-dimensional feature representation tensor for each viewpoint;

[0082] Based on the vehicle's geometric information, the extracted high-dimensional features are uniformly aligned in space through projection transformation, and geometric correction is used to project the features from multiple perspectives into a unified bird's-eye view coordinate system.

[0083] By using multi-view feature fusion and coordinate system transformation, features from multiple cameras are aligned into a two-dimensional plane representation. In the BEV coordinate system, each pixel contains high-dimensional feature information generated by multi-view fusion.

[0084] This step unifies the features of multi-view images into a bird's-eye view coordinate system, thereby achieving comprehensive information representation from multiple perspectives and helping to improve the accuracy of global spatial information.

[0085] S2 specifically includes:

[0086] Based on BEV features, the instance features at the current time are matched with the instance features at historical time through the instance center time association module. The feature alignment technology is used to perform association transformation on historical instances in the geometric and feature dimensions, so as to unify them with the current instance into the same semantic space.

[0087] By using a learnable temporal correlation mechanism, the matching relationship between historical instances and current instances is dynamically modeled in a high-dimensional feature space, capturing cross-temporal feature correlation changes.

[0088] Based on the instance-centric spatial fusion module, the point set features of historical frames and the point set features of the current frame are spatially aligned, and the local features of historical point sets and the current point sets are efficiently aligned and fused through cross-attention mechanism.

[0089] Spatial sampling is performed on the historical global map point set, and cross-attention operation is applied to the fusion process of detection results and historical features to strengthen the interaction relationship of instances in the spatial domain. The fusion result is optimized by curve fitting technology.

[0090] Based on the temporal correlation characteristics and spatial fusion characteristics, the spatiotemporal dimensions are jointly modeled, and the changing trends of instances in the temporal and spatial domains are captured through multi-layer network design;

[0091] In the time dimension, based on the dynamically updated instance memory buffer, the current instance is associated in real time by combining historical information of multi-frame features to generate a unified instance identifier.

[0092] In the spatial dimension, a three-layer cross-attention mechanism is used to perform deep interactive modeling of current features and historical features, which strengthens the spatial consistency of global map features. A classification optimization strategy is combined to perform fine processing on different types of map instances. Based on the optimized instance features, a global vectorized map representation is generated, which integrates geometric, temporal and semantic information to describe the details and spatial relationships of each instance in the global map.

[0093] This step significantly improves the feature fusion capability between history and the current frame through the design of cross-attention mechanism and instance memory buffer, enhancing the continuity and accuracy of the system in dynamic environments.

[0094] S3 specifically includes:

[0095] Based on query classification analysis, queries with smaller contributions are identified and pruned step by step to remove redundant queries and optimize computational efficiency, while retaining high-quality queries that make significant contributions to map construction.

[0096] By dynamically sorting and multi-level analyzing the classification scores of queries, redundant queries with a weak impact on map construction goals are filtered out.

[0097] A progressively refined strategy is introduced during pruning. High-quality queries that are retained interact through a self-attention module, which efficiently captures the global relationships between queries.

[0098] The optimized query features are fused with historical map sampling points. Based on geometric alignment and feature enhancement strategies, the current query features and historical features are unified into the same spatial domain.

[0099] Deeply analyze the feature distribution of historical sampling points, extract representative and information-rich key points, correct the positional deviation of historical features in the current coordinate system, and enhance semantic consistency with the current query features.

[0100] Through feature enhancement strategies, the model dynamically adjusts the fusion weights of historical and current features, introduces a dynamic feature weighting mechanism, and refines the control of the fusion process based on the semantic importance and contribution of features.

[0101] By using a recursive update strategy, the feedforward optimized features are integrated layer by layer to ensure consistency and globality in time and space dimensions. Combined with the geometric and semantic information of historical sampling points, the generated map representation is optimized to ensure its high accuracy and smoothness in complex scenes.

[0102] By introducing a self-attention mechanism and dynamic feature weighting, the global correlation between queries is enhanced, and the consistency and globality of features are ensured through a recursive update strategy, thereby improving the accuracy of map construction.

[0103] S4 specifically includes:

[0104] In the knowledge distillation stage, a model is built based on a high-quality vector map and used as a teacher model to provide high-precision feature guidance of the global map for the student model. Through temporal correlation, spatial fusion and efficient pruning, a high-quality vector map containing geometric, semantic and temporal information is generated.

[0105] During the distillation process, an adaptive weighting mechanism is used to extract high-quality global feature representations from the teacher network, including multi-frame temporal information, global geometric semantics, and spatial relationships between instances.

[0106] The features of the teacher model are divided into three parts: global features, regional features, and instance features. The knowledge transfer of each part is optimized by a specific distillation loss function. Global feature distillation guides the student model to capture the whole scene by constructing an overall context representation. Regional feature distillation helps the student network form accurate representations in key areas. Instance feature distillation focuses on improving the student model's robust detection ability for target instances.

[0107] An instance-centric alignment strategy is adopted, and features from the student model are embedded into a semantic space consistent with that of the teacher model through geometric and semantic mapping mechanisms.

[0108] During the distillation process, an adaptive loss balancing module was designed to dynamically adjust the distillation weights of global features and instance features. This module combines scene complexity, number of instances, and feature contribution to allocate weights, enabling the student model to prioritize learning key instance information in complex scenarios and maintain global consistency in simple scenarios. In addition, to address the lightweight requirements of the student model, the distillation process uses multi-layer feature compression technology to transfer the deep semantic features of the teacher model to the student model in a lightweight representation, thereby significantly reducing the consumption of computing resources while retaining key feature information.

[0109] This step reduces the consumption of computational resources by introducing adaptive loss balancing and lightweight feature compression strategies, while maintaining high-efficiency 3D object detection performance without relying on complex global modeling.

[0110] Through knowledge distillation, the student model achieves efficient and robust 3D object detection by constructing key regions centered on instances without relying on complex global feature modeling.

[0111] S5 specifically includes:

[0112] Based on the student model, a loss function is designed to comprehensively evaluate and optimize the student model, ensuring its robustness and high-precision 3D object detection capability.

[0113] The loss function includes classification loss and regression loss, which are used to evaluate the student model’s ability to predict the target category and the regression accuracy of the 3D bounding box, respectively.

[0114] To assess classification accuracy, the quality focus loss function is used to calculate the deviation between the target class probability distribution predicted by the student model and the true class label.

[0115] To assess regression accuracy, a smoothed L1 loss function is used to evaluate the difference between the 3D bounding box parameters predicted by the student model and the true values.

[0116] The classification loss and regression loss are combined into a total loss through a weighted summation to guide the optimization of the student model. The formula for calculating the total loss is as follows:

[0117] L total =L cls +L reg

[0118] Based on the loss function evaluation metrics, if the output requirements are not met, the student model is further optimized using a mid-range sensing mechanism. This involves dynamically adjusting the loss weights for different targets through distance-aware response distillation, with a focus on optimizing the detection accuracy for short-to-medium range targets. The specific formula is as follows:

[0119]

[0120] Where, d i w is the distance between target i and the sensor. i δ represents the normalized weight of the target, μ represents the fixed weight of the distant target, and A, B, C and δ are adjustment parameters.

[0121] Based on the distance-aware response distillation strategy, the student model is trained and optimized multiple times, and the evaluation index of the loss function is recalculated. If L is satisfied... total A threshold requirement of ≤0.1 indicates that the output accuracy requirement has been met, ensuring that the student model can efficiently perform 3D object detection tasks in resource-constrained environments, while maintaining a high level of robustness and accuracy in different complex scenarios, and finally outputting the instance-centered vector map construction result built by the student model.

[0122] This step optimizes the model's detection accuracy for targets at different distances by introducing a mid-range perception mechanism and quality focus loss, while ensuring high robustness and accuracy even in resource-constrained environments.

[0123] Regarding classification accuracy, the quality focus loss function is used to calculate the deviation between the target class probability distribution predicted by the student model and the true class label. Specifically, this includes:

[0124] The formula for calculating the mass focus loss function is as follows:

[0125]

[0126] In the formula, S represents the total number of targets, and p ti and p si The predicted probabilities w for the teacher model and the student model, respectively. i For target i.

[0127] Regarding regression accuracy, the smoothed L1 loss function is used to evaluate the difference between the 3D bounding box parameters predicted by the student model and the true values, specifically including:

[0128] The formula for the smoothed L1 loss function is:

[0129]

[0130] In the formula, r ti and r si are the bounding box regression parameters for the teacher model and the student model, respectively, where S is the total number of targets and SmoothL1 is the smooth L1 loss function.

[0131] Based on the loss function evaluation metrics, if the output requirements are not met, the student model is further optimized using a mid-range sensing mechanism. This involves dynamically adjusting the loss weights for different targets through distance-aware response distillation, with a focus on optimizing the detection accuracy of short-to-medium range targets. Specifically, this includes:

[0132] The dynamic adjustment formula is as follows:

[0133]

[0134] Where, d i w is the distance between target i and the sensor. i δ represents the normalized weight of the target, μ represents the fixed weight of the distant target, and A, B, C and δ are adjustment parameters.

[0135] In summary, the advantages of this invention are as follows:

[0136] This method can achieve high-precision spatiotemporal vector map construction. Through multi-view image fusion and optimization of temporal and spatial feature association, it ensures that the details and spatial relationships of each instance in the map are fully described, thus providing an accurate environmental representation. Through multi-level attention mechanism and efficient fusion of spatial and temporal features, the system can still maintain robustness in dynamically changing environments and maintain high performance even in complex or highly disturbed scenarios.

[0137] By employing query classification analysis and stepwise pruning techniques, redundant queries are effectively reduced, computational efficiency is improved, and the system is ensured to have a high response speed in real-time applications, while reducing the consumption of computing resources.

[0138] By extracting and fusing features based on self-attention and cross-attention mechanisms, key features in various perspectives and spatiotemporal dimensions can be better captured and optimized, enhancing the map building model's adaptability to complex scenarios.

[0139] By guiding the training of student models through teacher models, efficient knowledge transfer can be achieved. Especially under resource constraints, student models can maintain high-precision target detection capabilities and effectively avoid the waste of computing resources, making them suitable for deployment on edge devices or resource-constrained environments.

[0140] The optimized student model can not only build high-precision maps but also perform efficient 3D object detection. By combining evaluation methods for classification and regression accuracy, high-quality object detection is ensured, and a distance-aware mechanism further improves the detection accuracy of short-to-medium range targets.

[0141] This method uses an adaptive loss function and multi-layer feature compression techniques to ensure that the student model can balance global consistency and instance accuracy, making it particularly suitable for efficient real-time mapping and object detection in complex environments with dynamic changes.

[0142] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A robust online vector map construction method with spatiotemporal simplified query, characterized in that, include: S1: Based on multi-view image preprocessing, feature extraction and geometric correction, multi-view features are fused into a unified bird's-eye view representation to describe the geometric structure and semantic details of the scene; S2: Based on temporal correlation matching of instance features between the current frame and historical frames, and combined with cross-attention method through spatial fusion module, the historical point set and the current point set are fused to construct a globally continuous and high-precision map representation; S3: Based on query classification analysis, identify queries with smaller contributions and prune them, retain high-quality queries, use a self-attention module for interaction, and fuse with image features through cross-attention method to update query features and optimize query representation; S4: In the knowledge distillation stage, a high-quality vector map construction model is used as the teacher model, and an instance-centric map construction model is used as the student model, adaptively distilling features from the teacher network to the student network. S5: Design a loss function to evaluate the student model and perform training optimization. If the output requirements are not met, further optimize the student model using a mid-range perception method. S2 specifically includes: Based on BEV features, the instance features at the current time are matched with the instance features at historical time through the instance center time association module. The feature alignment method is used to perform association transformation on historical instances in the geometric and feature dimensions, so as to unify them with the current instance into the same semantic space. By using a learnable temporal correlation method, the matching relationship between historical instances and current instances is dynamically modeled in a high-dimensional feature space, capturing cross-temporal feature correlation changes. Based on the instance-centric spatial fusion module, the point set features of historical frames and the point set features of the current frame are spatially aligned, and the local features of historical point sets and the current point sets are efficiently aligned and fused through cross-attention methods. Spatial sampling is performed on the historical global map point set, and cross-attention operation is applied to the fusion process of detection results and historical features to strengthen the interaction relationship of instances in the spatial domain. The fusion result is optimized by curve fitting method. Based on the temporal correlation features and spatial fusion features, the spatiotemporal dimension is jointly modeled, and the changing trends of instances in the temporal and spatial domains are captured through multi-layer network design. In the time dimension, based on the dynamically updated instance memory buffer, the current instance is associated in real time by combining historical information of multi-frame features to generate a unified instance identifier. In the spatial dimension, a three-layer cross-attention method is used to perform deep interactive modeling of current features and historical features, which strengthens the spatial consistency of global map features, and combines classification optimization strategies to perform fine processing on different types of map instances. Based on the optimized instance features, a global vectorized map representation is generated, which integrates geometric, temporal and semantic information to describe the details and spatial relationships of each instance in the global map. S4 specifically includes: In the knowledge distillation stage, a model is built based on a high-quality vector map and used as a teacher model to provide high-precision feature guidance of the global map for the student model. Through temporal correlation, spatial fusion and efficient pruning, a high-quality vector map containing geometric, semantic and temporal information is generated. During the distillation process, an adaptive weighting method is used to extract high-quality global feature representations from the teacher network, including multi-frame temporal information, global geometric semantics, and spatial relationships between instances. The features of the teacher model are divided into three parts: global features, regional features, and instance features. The knowledge transfer of each part is optimized by a specific distillation loss function. Global feature distillation guides the student model to capture the whole scene by constructing an overall context representation. Regional feature distillation helps the student network form accurate representations in key areas. Instance feature distillation focuses on improving the student model's robust detection ability for target instances. An instance-centric alignment strategy is adopted, and features from the student model are embedded into a semantic space consistent with that of the teacher model through geometric and semantic mapping methods. In the distillation process, an adaptive loss balancing module was designed to dynamically adjust the distillation weights of global features and instance features. This module combines scene complexity, number of instances, and feature contribution to allocate weights, enabling the student model to prioritize learning key instance information in complex scenarios and maintain global consistency in simple scenarios. In addition, to address the lightweight requirements of the student model, the distillation process uses a multi-layer feature compression method to transfer the deep semantic features of the teacher model to the student model in a lightweight representation, thereby significantly reducing computational resource consumption while retaining key feature information.

2. The robust online vector map construction method with spatiotemporal simplified query as described in claim 1, characterized in that, S1 specifically includes: The system inputs the synchronously acquired multi-view camera image sequence to obtain scene data containing dynamic changes in the surrounding environment; Input scene data and preprocess the images, including distortion correction and time synchronization, to ensure alignment between multi-view images; Based on the intrinsic and extrinsic parameters of the camera, geometric calibration is performed on the images from each viewpoint to construct a globally consistent environment representation; Based on a deep learning backbone network, features are extracted from images from each viewpoint, and image data is transformed into high-dimensional feature tensors. Low-order features are extracted through convolutional layers, capturing texture and edge details in the image; High-order features are modeled using a multi-head attention method, focusing on global relationships and contextual associations between different objects in the scene, generating a high-dimensional feature representation tensor for each viewpoint; Based on the vehicle's geometric information, the extracted high-dimensional features are uniformly aligned in space through projection transformation, and geometric correction is used to project the features from multiple perspectives into a unified bird's-eye view coordinate system. By using multi-view feature fusion and coordinate system transformation, features from multiple cameras are aligned into a two-dimensional plane representation. In the BEV coordinate system, each pixel contains high-dimensional feature information generated by multi-view fusion.

3. The robust online vector map construction method with spatiotemporal simplified query as described in claim 1, characterized in that, S3 specifically includes: Based on query classification analysis, queries with smaller contributions are identified and pruned step by step to remove redundant queries and optimize computational efficiency, while retaining high-quality queries that make significant contributions to map construction. By dynamically sorting and multi-level analyzing the classification scores of queries, redundant queries with a weak impact on map construction goals are filtered out. A progressively refined strategy is introduced during pruning. High-quality queries that are retained interact through a self-attention module, which efficiently captures the global relationships between queries. The optimized query features are fused with historical map sampling points, and based on geometric alignment and feature enhancement strategies, the current query features and historical features are unified into the same spatial domain. Deeply analyze the feature distribution of historical sampling points, extract representative and information-rich key points, correct the positional deviation of historical features in the current coordinate system, and enhance semantic consistency with the current query features. Through feature enhancement strategies, the model dynamically adjusts the fusion weights of historical and current features, introduces a dynamic feature weighting method, and refines the control of the fusion process based on the semantic importance and contribution of features. By employing a recursive update strategy, the feedforward optimized features are integrated layer by layer to maintain consistency and globality in both time and space dimensions. Combined with the geometric and semantic information of historical sampling points, the generated map representation is optimized, thereby ensuring its high accuracy and smoothness in complex scenes.

4. The robust online vector map construction method with spatiotemporal simplified query as described in claim 1, characterized in that, Through knowledge distillation, the student model achieves efficient and robust 3D object detection by constructing key regions centered on instances without relying on complex global feature modeling.

5. The robust online vector map construction method with spatiotemporal simplified query as described in claim 1, characterized in that, S5 specifically includes: Based on the student model, a loss function is designed to comprehensively evaluate and optimize the student model, ensuring its robustness and high-precision 3D target detection capability. The loss function includes classification loss and regression loss, which are used to evaluate the target category prediction ability of the student model and the regression accuracy of the 3D bounding box, respectively. To assess classification accuracy, the quality focus loss function is used to calculate the deviation between the target class probability distribution predicted by the student model and the true class label. To assess regression accuracy, a smoothed L1 loss function is used to evaluate the difference between the 3D bounding box parameters predicted by the student model and the true values. The classification loss and regression loss are combined into a total loss through a weighted summation to guide the optimization of the student model. The formula for calculating the total loss is as follows: ； Based on the aforementioned loss function evaluation metrics, if the output requirements are not met, the student model is further optimized using a mid-range perception method. This involves dynamically adjusting the loss weights for different targets through distance-aware response distillation, with a focus on optimizing the detection accuracy for short-to-medium range targets. The specific formula is as follows: ； in, Let i be the distance between the target i and the sensor. Normalized weights for the target For distant targets, the weights are fixed, and A, B, C, and δ are adjustment parameters.

6. The robust online vector map construction method with spatiotemporal simplified query according to claim 5, characterized in that, Based on the distance-aware response distillation strategy, the student model is trained and optimized multiple times, and the evaluation index of the loss function is recalculated. If it satisfies... A threshold requirement of 0.1 is considered to meet the output accuracy requirements, ensuring that the student model can efficiently perform 3D object detection tasks in resource-constrained environments, while maintaining a high level of robustness and accuracy in different complex scenarios, and finally outputting the instance-centered vector map construction results built by the student model.

7. The robust online vector map construction method with spatiotemporal simplified query as described in claim 5, characterized in that, Regarding classification accuracy, the quality focus loss function is used to calculate the deviation between the target class probability distribution predicted by the student model and the true class label. include: The formula for calculating the mass focus loss function is as follows: ； In the formula, S represents the total number of targets. and These are the predicted probabilities for the teacher model and the student model, respectively. Normalized weights for the target; This is the quality focus loss function.

8. The robust online vector map construction method with spatiotemporal simplified query according to claim 5, characterized in that, Regarding regression accuracy, a smoothed L1 loss function is used to evaluate the difference between the 3D bounding box parameters predicted by the student model and the true values. include: The formula for the smoothed L1 loss function is: ； In the formula, and , respectively, are the bounding box regression parameters for the teacher model and the student model, where S is the total number of targets. To smooth the L1 loss function, Normalized weights for the target.

Images