Automatic driving mapping closed-loop detection and pose evaluation method based on hyperpoint semantic graph

Abstract

The invention discloses a hyperpoint semantic graph-based automatic driving mapping closed-loop detection and pose evaluation method. The method comprises the following steps of: 1, acquiring an automatic driving data set; 2, constructing a novel superpoint semantic graph representation, and providing a semantic enhancement-based annular descriptor and a double-stage closed-loop detection process; 3, training an automatic driving data set to obtain data such as F1max, AUC, Recall (at) 1, Recall (at) 1% and the like, and comparing with other methods; and 4, performing an ablation experiment, and verifying the performance improvement degree of the improved model. According to the method, a novel superpoint semantic graph representation and semantic enhancement-based annular descriptor is provided, and a double-stage closed-loop detection process is adopted. Besides, the semantic enhanced annular descriptor can realize scene recognition in a complex urban environment, a double-stage closed-loop detection process guarantees the recall rate and effectively filters wrong closed loops, and the precision and robustness of closed-loop detection and pose evaluation in a dynamic urban environment are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision processing, in particular to a method for automatic driving mapping closed loop detection and pose evaluation based on super point semantic graph. BACKGROUND

[0002] In a complex and dynamic urban environment, autonomous vehicles may have perception confusion due to similar scenes, and the location recognition of autonomous vehicles may be affected by environmental changes or changes in viewing angle. In addition, when people and vehicles come and go, line-of-sight occlusion may occur, which may further exacerbate positioning errors and closed loop detection errors, which greatly affects the safety of the autonomous driving process.

[0003] Current SOTA methods are mainly divided into two categories: (1) pure geometric methods lacking semantic understanding; (2) deep learning solutions with limited generalization ability due to data bias or lack of interpretability. Pure geometric closed loop methods often fail in the presence of structural changes or occlusions, while semantic-enhanced deep models, although robust, require computational complexity and training data dependence.

[0004] In addition, although dense point cloud registration techniques such as point-to-point / point-to-plane ICP are accurate, they are computationally intensive and sensitive to initial values, leading to performance degradation in poor prior or high noise scenarios.

[0005] Therefore, there is an urgent need for a lightweight and efficient method to deal with complex dynamic urban environments.

[0006] The information disclosed in this BACKGROUND section is only for the purpose of increasing the understanding of the background of the present application and should not be taken as an acknowledgment or any form of suggestion that this information forms prior art that is publicly known. SUMMARY

[0007] The purpose of the present application is to provide a method for precise closed loop detection and pose evaluation of autonomous vehicles, thereby solving the existing problems of perception confusion of autonomous vehicles in similar scenes and interference of dynamic entities in complex dynamic urban environments.

[0008] To achieve the above-mentioned purpose, the present application provides a method for precise closed loop detection and pose evaluation of autonomous vehicles, comprising:

[0009] Step 1, obtain three major mainstream autonomous driving datasets;

[0010] Step 2, construct a new type of super point semantic graph (SPSG) representation, and propose a semantic-enhanced ring descriptor and a two-stage closed loop detection process;

[0011] Step 3, train the three major mainstream autonomous driving datasets using the present application to obtain F1max , AUC, Recall@1, Recall@1%, etc. data, and compared with other advanced methods;

[0012] Step 4, conduct ablation experiments to verify the performance improvement of the improved model.

[0013] Preferably, in the above technical solution, step 1 acquires three major mainstream automatic driving datasets (KITTI, Ford Campus and Apollo), covering diversified urban environments, including different levels of structural complexity, dynamic object density and closed loop frequency.

[0014] Preferably, in the above technical solution, step 2 improves the SPSG model, including:

[0015] (1) A new superpoint clustering method is adopted. In this method, the nodes in the graph represent superpoints, and the edges represent spatial adjacency. Specifically, if the minimum Euclidean distance between any point pair on two superpoints is less than a predefined threshold ∈, an edge is established between the superpoints s1 and s2. Set g1 and g2 as the centroids of superpoints s1 and s2, respectively. The centroid positions are iteratively optimized by the following steps: identify the nearest neighbor in the opposite superpoint and update the centroid accordingly until convergence. Finally, the centroid distance |g1-g2| is used instead of the superpoint distance. A surrounding sphere is constructed for each superpoint with radii r1 and r2, and distance calculation is only performed when the following conditions are met:

[0016] |g1-g2|<r1+r2+∈

[0017] To realize semantic classification, the superpoint features f i are input into a multi-layer perceptron (MLP) classifier φ_cls to generate a probability vector z1:

[0018] zi=softmax(φ(f i ))

[0019] The predicted value zi is supervised by cross-entropy loss, and the target true value is one-hot encoded label 1 (c i ):

[0020]

[0021] (2) A new super node descriptor is proposed, which can capture the relative topological relationship of each superpoint in its local neighborhood and global scene structure. Given the segmented point cloud, a undirected graph G = <U, L> is constructed: where each node U i ∈R 3 represents the centroid of the superpoint, and each edge L Euclidean adjacency relation is characterized. Each super-node is characterized by a two-component descriptor (local topological histogram descriptor (f g ∈R k )) and a global spectral descriptor (f

[0022] f=(f l |f g )

[0023] The unified representation is both transformation-invariant and semantically discriminative, thus enabling robust super-point matching in multiple scenarios.

[0024] (3) A semantic ring graph G ring is constructed. This method emphasizes the macro-level semantic structure by aggregating the super-point relations in the concentric spatial partition. This ring structure explicitly preserves the scene layout and radial semantic distribution characteristics, which remain stable and consistent when revisiting the scene even if the viewpoint changes. Each ring graph is constructed by dividing the space around the sensor into K concentric rings and counting the number of edges of each semantic class in each ring. Let Eat,k denote the number of edges of semantic class t in the kth ring of graph Garing(t = 1,...,T), then the graph similarity score between two frames is defined as follows:

[0025]

[0026] This index measures the structural matching degree of semantic edge distribution among the spatial rings. If the similarity exceeds the threshold θ1, the candidate frame is retained. To ensure that spatial consistency is prioritized over semantic similarity, the invention performs geometric verification based on rigid transformation. Let the matching super-point centroid sets of the query frame and the candidate frame be Q a ={q a i} and Q b ={q b j}, respectively. Solve the optimal transformation T ∈ SE(3) to align Q a and Q b , and the residual alignment error is calculated as follows:

[0027]

[0028] (4) After loop closure detection, a hierarchical pose optimization strategy is used to improve the accuracy of inter-frame registration. First, the RANSAC coarse registration process is performed:

[0029]

[0030] where p(·) is a robust loss function to suppress abnormal matches caused by local occlusion or dynamic objects. Then the dense superpoint ICP fine registration procedure is performed:

[0031]

[0032] This procedure benefits from the structural coherence of three-dimensional superpoints, and can effectively suppress noise and maintain spatial consistency in complex environments. In order to eliminate residual geometric errors, the sparse point pair ICP fine registration procedure is then performed:

[0033]

[0034] This procedure significantly improves the registration robustness by introducing the geometric constraint of the vertical surface, especially in weak texture areas. Finally, the global pose evaluation procedure is performed for optimization of odometry drift and loop closure consistency:

[0035]

[0036] where denotes the SE(3) pose difference, and the Σ term is the association information matrix

[0037] Preferably, in the above technical solution, step 3 trains the training set of the three major mainstream automatic driving data sets (KITTI, Ford Campus and Apollo) to obtain F1max, AUC, Recall@1, Recall@1% and other data, and compare with other advanced methods.

[0038] Preferably, in the above technical solution, step 3 inputs each sequence (00-08) (city, urban, high occlusion, etc.) scene into the trained model to obtain the test results of the model.

[0039] Preferably, in the above technical solution, step 4 performs an ablation experiment on the KITTI dataset, including training each component, from T1 to T4 (two-channel superpoint descriptor, semantic ring graph, semantic space combination, and full integration). The F1max value, RRE value and RTE of each scheme are obtained, and other methods are compared. Compared with the prior art, the present application has the following beneficial effects:

[0040] (1) The application is a method for realizing precise closed-loop detection and pose evaluation of an autonomous vehicle based on a super point semantic graph (SPSG), in order to solve the problem of perception confusion of an autonomous vehicle in a complex urban environment, point cloud mapping, semantic understanding and pose optimization are realized by using a super point semantic graph (SPSG). The method abstracts point clouds into super points that capture stable geometry and high-level semantics simultaneously, the super points form graph nodes, and the edges encode spatial relationships, thereby providing an efficient and robust reasoning mechanism for place recognition and closed-loop detection. The core innovation lies in introducing a semantic-enhanced ring descriptor, which integrates geometric features and semantic features into a compact and high-expression representation, significantly improving the place recognition ability of the system under changes in viewing angle, seasonal replacement and occlusion. By fusing semantic priors, the system avoids many defects of pure geometric methods while maintaining data efficiency and interpretability.

[0041] (2) The application is a method for realizing precise closed-loop detection and pose evaluation of an autonomous vehicle based on a super point semantic graph (SPSG), which has significantly improved performance compared to traditional SGPR and other advanced methods LCDNet, BoW3D, PADLoC, SGLC and SSC, and has shown precision and anti-interference in different scene complexity. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a general flowchart of the method for realizing precise closed-loop detection and pose evaluation of an autonomous vehicle based on a super point semantic graph (SPSG) according to the application;

[0043] Figure 2 is a work flowchart of improved closed-loop detection and pose evaluation based on SPSG;

[0044] Figure 3 is a semantic aggregation graph of SPSG;

[0045] Figure 4 is a comparison chart of translation error and rotation error of SPSG (red line) method and SGLC (blue line) method in training KITTI dataset sequence 08;

[0046] Figure 5 is a qualitative comparison chart of pose evaluation of SPSG method and SGLC method in training KITTI dataset sequence 00

[0047] Figure 6 is a comparison chart of SPSG method and SGLC method in training KITTI sequence 08 single-axis three-way trajectory comparison chart;

[0048] Figure 7 is a performance comparison result of SPSG method and other SOTA on training KITTI dataset; DETAILED DESCRIPTION

[0049] The specific embodiments of the present application are described in detail below with reference to the accompanying drawings, but the scope of protection of the present application is not limited by the specific embodiments.

[0050] Unless otherwise clearly indicated, throughout the specification and claims, the terms "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element or group of elements but not the exclusion of any other element or group of elements.

[0051] As shown in the figure, according to the precise closed-loop detection and pose evaluation method of an autonomous vehicle in the specific embodiment of the present application, the following steps are included: Figures 1 to 7

[0052] Step one, obtain mainstream autonomous vehicle data set

[0053] Considering the existing problems in the closed-loop detection and pose evaluation task of autonomous vehicles, in all existing data sets, the present application selects the mainstream autonomous vehicle data set KITTI, Ford Campus and Apollo, which covers diversified urban environments, contains different levels of structural complexity, dynamic object density and closed-loop frequency, and can fully simulate complex dynamic urban environments. The research results obtained by using this data set are also more authoritative.

[0054] Step two, build a new super-point semantic graph (SPSG) representation, and propose a ring descriptor based on semantic enhancement and a two-stage closed-loop detection process.

[0055] (1) A new super-point graph is constructed, which can take into account geometric tightness and topological adjacency. In this graph, nodes represent super-points, and edges represent spatial adjacency. Specifically, if the minimum Euclidean distance between any point pair on two super-points is less than a predefined threshold ∈, an edge is established between the super-points s1 and s2. Direct pairwise calculation will result in quadratic complexity O(N 2 ), which is difficult to bear in terms of calculation.

[0056] To alleviate this problem, this paper introduces a centroid-based iterative approximation method. Let g1 and g2 be the centroids of super-points s1 and s2, respectively. The centroid positions are iteratively optimized by the following steps: identify the nearest neighbor in the opposite super-point and update the centroid accordingly until convergence. Finally, the centroid distance |g1-g2| is used instead of the super-point distance.

[0057] In addition, to reduce unnecessary calculations, this paper uses a spatial pruning mechanism. A surrounding sphere is constructed for each super-point by radii r1 and r2, and distance calculation is only performed when the following conditions are met:

[0058] |g1-g2|<r1+r2+∈ ​

[0059] The semantic classification is achieved by mapping the superpoint feature f i An input multi-layer perceptron (MLP) classifier φ cls is implemented to generate a probability vector z i :

[0060] z i = softmax(φ(f i ))

[0061] The predicted value z i is supervised by a cross-entropy loss with the ground truth being a one-hot encoded label 1 (c i ):

[0062]

[0063] (2) A new super-node descriptor is proposed, which can capture the relative topological relationship of each superpoint in its local neighborhood and global scene structure, thereby ensuring invariance under rigid changes. Given a segmented point cloud, an undirected graph G = < U, L > is constructed: where each node U i ∈ R 3 represents the superpoint centroid, and each edge represents the Euclidean adjacency relationship. Each super-node is represented by a two-component descriptor (local topological histogram descriptor (f g ∈ R k ), global spectral descriptor (f g ∈ R k )).

[0064] Local topological histogram descriptor (f): This component models the distribution of semantic edge types within the local neighborhood and their relative distance from the current node. The distance is discretized into intervals, and the semantic edge types are classified (such as vehicle-vehicle, vehicle-building), forming a compact histogram that represents the local semantic geometry. This descriptor is robust to position perturbations and sampling density changes, making it suitable for noisy lidar data in real-world scenarios.

[0065] Global spectral descriptor (f g ∈ R k ): To capture the structural role of superpoints in the overall scene topology, the adjacency matrix A of the graph is decomposed into A = QΛQT. The first k eigenvectors are selected to construct a spectral embedding, which encodes global connectivity patterns and reflects the importance of each superpoint in the graph structure. This global descriptor complements the local histogram by integrating long-range dependencies and scene-level context cues.

[0066] The final super-node descriptor f is obtained by concatenating the two components:

[0067] f = (f 1 | f g )

[0068] This unified representation has both transformation invariance and semantic discriminability, enabling robust superpoint matching in multiple scenes

[0069] (3) Constructed the semantic ring graph G ring , which can realize robust loop closure detection in large-scale long-term environment. This method emphasizes the macro-level semantic structure by aggregating the hyperpoint relationships in the same space partition. This ring structure explicitly retains the scene layout and radial semantic distribution characteristics, which remain stable and consistent when revisiting the scene even if the view changes. Each ring graph is constructed by dividing the space around the sensor into K concentric rings and counting the number of edges of each semantic class in each ring. Let Eat,k represent the number of edges of semantic class t in the kth ring of graph Garing(t = 1,...,T), then the graph similarity score between two frames is defined as follows:

[0070]

[0071] This index measures the structural matching degree of semantic edge distribution among the space rings. If the similarity exceeds the threshold θ1, the candidate frame is retained. To ensure that spatial consistency is prioritized over semantic similarity, the invention performs geometric verification based on rigid transformation. Let the matching hyperpoint centroid sets of the query frame and the candidate frame be Q a ={q a i} and Q b ={q b j}, respectively. Solve the optimal transformation T ∈ SE(3) to align Q a and Q b , and the residual alignment error is calculated as follows:

[0072]

[0073] (4) After loop closure detection, a hierarchical pose optimization strategy is used to improve the inter-frame registration accuracy. First, perform the RANSAC coarse registration process:

[0074]

[0075] where ρ(·) is a robust loss function used to suppress abnormal matches caused by local occlusion or dynamic objects. Then perform the dense hyperpoint ICP fine registration process:

[0076]

[0077] This process benefits from the structural coherence of three-dimensional hyperpoints, effectively suppressing noise and maintaining spatial consistency in complex environments. To eliminate residual geometric errors, then perform the sparse point pair ICP fine configuration process:

[0078]

[0079] The flow significantly improves the registration robustness by introducing geometric constraints of vertical surfaces, especially in weak texture regions.

[0080]

[0081] wherein represents SE(3) pose difference, and the Σ term is the association information matrix

[0082] Step three, training the improved model and obtaining test results

[0083] The training set of (KITTI, Ford Campus and Apollo) is used for training, F1max, AUC, Recall@1, Recall@1% and other data are obtained, and comparison is made with other advanced methods.

[0084] Step four, ablation experiment on KITTI dataset includes training of each component set, from T1 to T4 (hypernode descriptor, semantic ring graph, semantic space combination, complete integration), and the F1max value, RRE value and RTE of each scheme are obtained, and comparison is made with other methods

[0085] Ablation experiments are carried out on three component integrations and complete integration to verify the effectiveness of each component and analyze the reasons for the effect of the module. Through ablation comparison of each module, the effectiveness of each module can be verified. The results are that the hypernode descriptor (T1) brings preliminary performance improvement: F1max increases by 1.67% (0.901→0.916), and the rotation error decreases by 28.45% (1.16°→0.83°), which proves the effectiveness of the fusion of local geometric histogram and global spectral feature. The subsequently integrated semantic ring graph matching (T2) further reduces the translation error by 28.57% (0.21 meters→0.15 meters) through statistical analysis of the semantic edge distribution, while maintaining a recall rate of 99.77%. The complete integration scheme (T4) achieves 0.950 F1max, 99.95% recall rate and excellent pose accuracy (0.04 meter translation error, 0.20 degree rotation error), compared with the baseline configuration (T0), which realizes: F1max increases by 5.44%, translation error decreases by 84%, and rotation error decreases by 82.76.

[0086] The present application completely covers the whole process of large-scale laser radar SLAM - from feature representation (F1 increases by 1.67% to 5.44%) to geometric verification (error decreases by 28.57% to 73.33%), and then to pose optimization (cumulative relative trajectory error improves by 84%).

[0087] The foregoing description of specific exemplary embodiments of the application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise forms disclosed, and obviously many modifications and variations are possible in light of the above teaching. It is intended that the scope of the application be limited not with this detailed description, but rather by the claims appended hereto.

Claims

1.A method for automatic driving mapping loop detection and pose evaluation based on super-point semantic graph, characterized in that, Comprise: Step 1, obtain three major mainstream automatic driving data sets; Step 2, build a new type of super point semantic graph (SPSG) representation, and propose a kind of ring descriptor based on semantic enhancement and double-stage closed loop detection process; Step 3, F1 max , AUC, Recall@1, Recall@1%, and other data obtained by training three major mainstream automatic driving data sets according to the application are compared with other advanced methods; Step 4, perform ablation experiment to verify the performance improvement degree of the improved model. 2.The method of claim 1, wherein, Step 1 obtains three major mainstream automatic driving data sets (KITTI, Ford Campus and Apollo), covering diversified urban environment, including different levels of structural complexity, dynamic object density and closed loop occurrence frequency. 3.The method of claim 1, wherein, Step 2, the improvement of SPSG model includes: (1) a new super point graph clustering method is adopted, in which the nodes in the graph represent super points and the edges represent spatial adjacency; Specifically, if the minimum Euclidean distance between any point pair on two super points is less than a predefined threshold ∈, an edge is established between the super points s1 and s2; Set g1 and g2 as the centroids of super points s1 and s2 respectively. The centroid position is iteratively optimized by the following steps: identify the nearest neighbor in the opposite super point and update the centroid accordingly until convergence. Finally, the centroid distance |g1-g2| is used instead of the super point distance; A surrounding sphere is constructed for each super point by radius r1 and r2, and distance calculation is only performed when the following condition is met: |g1-g2|<r1+r2+∈ Implementing semantic classification is achieved by computing a superpoint feature f i An input multi-layer perceptron (MLP) classifier φ cls is implemented, generating a probability vector z i : zi = softmax(φ(f i )) The predicted value zi is supervised by cross-entropy loss with the target true value being one-hot encoded label 1 (c i ): (2) A new type of super-node descriptor is proposed, which can capture the relative topological relationship of each super-node in its local neighborhood and the global scene structure. Given the segmented point cloud, an undirected graph G = <U, L> is constructed: where each node U i ∈R 3 represents the super-node centroid, and each edge represents the Euclidean adjacency relationship. Each super-node is represented by a two-component descriptor (local topological histogram descriptor f, global spectral descriptor f g ∈R k ). The final super-node descriptor f is obtained by splicing the two components: f = (f1 | f g ) This unified representation has both transformation invariance and semantic distinguishability, so it can achieve robust super point matching in multiple scenarios; (3) Constructed the semantic ring graph G ring This method emphasizes the macro-level semantic structure by aggregating the hyperpoint relations in the same concentric spatial partition. This ring structure explicitly preserves the scene layout and radial semantic distribution characteristics, which remain stable and consistent when revisiting the scene even if the viewpoint changes. Each ring graph is constructed by dividing the space around the sensor into K concentric rings and counting the number of edges of each semantic class in each ring. Let Eat,k represent the number of edges of semantic class t in the kth ring of graph Garing(t = 1,...,T), then the graph similarity score between two frames is defined as follows: The index measures the structure matching degree of semantic edge distribution among various space rings; if the similarity exceeds a threshold θ1, the candidate frame is reserved; in order to ensure that the spatial consistency is prior to the semantic similarity, the geometric verification based on rigid body transformation is implemented; the matching superpoint centroid sets of the query frame and the candidate frame are respectively Q a ={q a i} and Q b ={q b j}; the optimal transformation T is solved from SE (3) so that Q a is aligned with Q b , and the residual error alignment error calculation formula is as follows: (4) after closed loop detection, a hierarchical pose optimization strategy is adopted to improve the inter-frame registration accuracy. First, the RANSAC coarse registration process is performed: Where ρ(·) is a robust loss function used to suppress abnormal matches caused by local occlusion or dynamic objects; Then, the dense super point ICP fine registration process is performed: This process benefits from the structural coherence of three-dimensional super points, which can effectively suppress noise and maintain spatial consistency in complex environments; In order to eliminate residual geometric errors, the sparse point pair surface ICP fine configuration process is then performed: This process significantly improves the registration robustness by introducing the geometric constraint of vertical surface, especially in weak texture areas; Finally, the global pose evaluation process is performed to optimize the odometer drift and closed loop consistency: where denotes the SE(3) pose difference, and the term is the information matrix. 4.The method of claim 1, wherein, Step 3 trains the training set of three major mainstream automatic driving data sets (KITTI, Ford Campus and Apollo) to obtain F1max, AUC, Recall@1, Recall@1% and other data, and compares them with other advanced methods. 5.The method of claim 1, wherein, Step 3 inputs each sequence (00-08) (city, urban, high occlusion, etc.) scene into the trained model to obtain the test results of the model. 6.The method for precise closed-loop detection and pose evaluation of an autonomous vehicle based on an ultra-point semantic graph according to claim 1, wherein, Step 4 performs ablation experiment on KITTI dataset, including integrating each component for training, from T1 to T4 (super node descriptor, semantic ring graph, semantic space combination, and all integration); Get the F1max value, RRE value and RTE of each scheme, and compare them with other methods.

Images