A method for loop closure detection of three-dimensional point cloud using semantic topological descriptors

Abstract

The application discloses a kind of closed loop detection methods for three-dimensional point cloud using semantic topological descriptor, it is related to visual space positioning field;For the poor robustness of the algorithm of previous closed loop detection based on single information, after the semantic segmentation of point cloud, the point cloud of dynamic semantic object is discarded in data preprocessing part, to reduce dynamic effect and computing burden.The static feature points are obtained from the remaining point cloud, and the corresponding score is calculated according to the semantic features and distance distribution.By non-maximum suppression (NMS) using bird eye projection to extract nodes and construct semantic topological graph.Finally, the semantic topological graph is converted into matrix descriptor, and a two-step search strategy is used for closed loop detection, which improves the matching speed while ensuring accuracy, effectively deals with the closed loop detection work under the perspective transformation or dynamic scene, to serve the application scenarios such as unmanned driving and map construction.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of visual spatial positioning, and particularly to a three-dimensional point cloud loop closure detection method based on semantic topological descriptors. BACKGROUND

[0002] With the increasing application of indoor and outdoor robots, the continuous promotion of automatic driving and augmented reality applications, how to realize the repositioning of robots in the prior map has become a problem to be solved. Simultaneous Localization and Mapping (SLAM) technology processes the input sensor data to obtain the carrier pose and construct an environment map.

[0003] Loop closure detection, as an important part of simultaneous localization and mapping, can help robots recognize previously visited locations, correct accumulated drift errors, and construct a globally consistent map, providing accurate prior information for robots.

[0004] The document "Zhu Y, Ma Y, Chen L, et al. Gosmatch: Graph-of-semantics matching for detecting loop closures in 3d lidar data [C] / / 2020 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020: 5151-5157." extracts vehicles, trees and rods as semantic objects from the original laser three-dimensional point cloud, takes the cluster center of each object as a node, connects the nodes to construct an undirected graph, and completes the similarity matching between the loop frames through the distance histogram. The method selected semantic categories are very limited, so it cannot fully utilize semantic information, and its ability to distinguish the uniqueness of the scene is limited.

[0005] The document "Kong X, Yang X, Zhai G, et al. Semantic graph based place recognition for 3d point clouds [C] / / 2020 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020: 8216-8223." first extracts the class label of each three-dimensional point cloud using a semantic segmentation network, selects 12 types of semantic information from it, and then inputs the clustering center of the three-dimensional point cloud into the designed neural network to complete the similarity matching of the graph. This method extracts only one clustering center for all large-scale semantic three-dimensional point clouds, such as ground, buildings, lawns, etc. Therefore, the features extracted under different perspectives will be different, and the problem of object segmentation in three-dimensional point cloud data cannot be solved.

[0006] The document "Vidanapathirana K, Moghadam P, Harwood B, et al. Locus: Lidar-based place recognition using spatiotemporal higher-order pooling [C] / / 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021: 5075-5081." extracts the spatial topological relationship and time information in the scene, and finally obtains a fixed-length feature descriptor through the second-order pooling of the nonlinear transformation of multi-level features. This method does not consider that the randomness of the position of dynamic objects in the outdoor environment will cause three-dimensional point cloud occlusion, loss, etc., thereby affecting the expression of the environment and the generation of the feature descriptor.

[0007] In the existing loop closure detection technology, only the geometric features of the original three-dimensional point cloud are extracted, or the features of a single semantic object three-dimensional point cloud are focused on, and the correlation information between semantic objects is ignored. These schemes are easily affected by dynamic objects and changes in perspective. To solve the problems existing in the prior art. SUMMARY

[0008] The present application provides a three-dimensional point cloud loop closure detection algorithm using semantic topological descriptors, which extracts semantic information in the environment through semantic segmentation, and fuses the features of a single semantic object and the connection of multiple semantic objects, thereby improving the accuracy and robustness of loop closure detection.

[0009] To achieve the above object, the technical scheme adopted by the present application is:

[0010] A method for loop closure detection of three-dimensional point cloud using semantic topological descriptor, comprising the following steps:

[0011] S1: extracting the label category of three-dimensional point cloud through semantic segmentation, and discarding the specified object three-dimensional point cloud according to the semantic label of three-dimensional point cloud;

[0012] S2: performing spatial transformation on three-dimensional point cloud through convex hull projection, judging the concave-convex property of points according to the spatial distribution of three-dimensional point cloud after spatial transformation, and taking the points in the original three-dimensional point cloud space corresponding to the convex points as feature points;

[0013] S3: calculating the score of feature points according to the geometric information and semantic information of three-dimensional point cloud in the original space, dividing the grid by using the bird's eye view, selecting feature points as nodes of semantic topological graph according to the grid position, taking the maximum spatial distance between each node as the edge of semantic topological graph, and constructing the semantic topological graph;

[0014] S4: converting the semantic topological graph into a descriptor in the form of matrix, taking the radial angle of the grid graph as the starting point through a two-stage search strategy, stretching into a two-dimensional matrix, taking the semantic distance as the numerical value of the corresponding row and column in the matrix, constructing the descriptor in the form of matrix, and completing the loop closure detection.

[0015] In S1, the original three-dimensional point cloud data is obtained by laser radar, and the semantic category of each three-dimensional point cloud is obtained by a semantic segmentation network of three-dimensional point cloud data.

[0016] The semantic segmentation adopts a RangeNet++ network; in the three-dimensional point cloud after semantic segmentation, the three-dimensional point cloud data with dynamic semantic objects is discarded, and the static three-dimensional point cloud data in the scene is retained.

[0017] The three-dimensional point cloud after semantic segmentation also includes static three-dimensional point cloud data corresponding to discarded dynamic semantic objects.

[0018] The convex hull projection performs voxel down-sampling on three-dimensional point cloud, and the three-dimensional point cloud is spatially transformed through formula (1); set the original three-dimensional point cloud set P origin , the three-dimensional point cloud set P hull after projection, and the convex point set P convex , the three-dimensional point cloud in each set is represented as p o , p h and p c , the observation point center is represented as p v , and the projection formula is shown as formula (1):

[0019]

[0020] wherein:

[0021]

[0022] wherein f is a kernel function, and γ represents a coefficient factor;

[0023] The concave-convex property of the judgment point is determined, K is set as the number of extracted nearest neighbor points, K nearest neighbor points in the three-dimensional point cloud after spatial transformation are screened out to extract the convex feature points satisfying formula (3):

[0024]

[0025] wherein each three-dimensional point cloud comprises a plurality of feature points, k represents the ranking serial number of the nearest neighbor point, wherein k∈{1,2,…,K}, represents the projection point corresponding to the k serial number.

[0026] The score of the feature point is calculated, the distance distribution of the three-dimensional point cloud is set as the geometric information, and the distance between the feature point and the observation center is set as d h , the distance mean of the three-dimensional point cloud in the scene is , and the variance is σ 2 , the geometric score φ1(p h ) of the feature point is calculated through a Gaussian kernel function , as shown in formula (4):

[0027]

[0028] The more three-dimensional point clouds with the same semantic category in the vicinity of the feature point, the higher the semantic score φ2(p h ) of the feature point, and vice versa, as shown in formula (5):

[0029]

[0030] wherein α represents a coefficient factor, the geometric score φ1(p h ) and the semantic score φ2(p h ) of the feature point are multiplied to obtain the total score φ(p h ) of the feature point, as shown in formula (6):

[0031] φ(p h )=φ1(p h )*φ2(p h ) (6)

[0032] The semantic topology graph is constructed, a bird's eye view is adopted, the observation center is taken as the center, and the radius is divided into N ra plurality of concentric circles, and the N s regions are divided by the concentric circles and the radial region division lines, the three-dimensional point cloud space is divided into a plurality of grids from the top-down perspective, the feature points at different heights in the same grid are selected as the representative points of the grid by local maximum suppression, and all the representative points in the grids are taken as the nodes of the semantic topology graph; taking the grid as the basic unit of distance, the semantic distance between any two grids is multiplied by the total score of the feature points to obtain the semantic distance between the nodes of the semantic topology graph with geometric and semantic weights, the maximum semantic distance between the nodes is taken as the edge of the graph, and the semantic topology graph is constructed.

[0033] The feature points of an arbitrary grid are N i,m , wherein i is the serial number of the grid circle number, i∈{1,2,…,N r}, N r is the maximum serial number of the grid circle number; j represents the jth serial number of the grid angle division, j∈{1,2,…,N s}, N s is the maximum serial number of the grid angle division; m represents the mth serial number of the grid angle division, and m≠j; the semantic category of the grid feature point is o, max is set to be a function that makes the semantic distance maximum, the semantic geometric distance d g , and the semantic category distance d s are calculated as shown in formula (7) and formula (8):

[0034]

[0035]

[0036] , wherein o represents the semantic category of the grid feature point.

[0037] The two-step search algorithm includes a first-step search algorithm and a second-step search algorithm; the grid map with the observation center as the center is composed of a plurality of concentric circles, and is divided at the same interval in the radial direction; and the grid map is stretched into a two-dimensional matrix with the first row connected to the last row.

[0038] The first-step search algorithm converts the matrix into a vector k, and stores and searches the candidate frames of the current scene by using the structure of a k-d tree (K-demension tree);

[0039] The storage and search of the candidate frames of the current scene by using the structure of a k-d tree (K-demension tree) include that the first value of the vector k is obtained from the first row of the matrix, and the subsequent values are obtained from the next row in sequence, N o is the number of selected semantic categories, N r is the grid division circle number, d is the semantic distance of the grid feature points with the same semantic category s average value, d is the semantic distance of the grid feature points with the same semantic category g average value, all rows are generated after encoding N r +N o dimensional vector ε, as shown in equation (9):

[0040]

[0041] wherein r represents the number of rows of the matrix, o is the semantic category,

[0042] The second step search algorithm sets the matrix descriptor of the current scene as s q G, the grid number is N i , the feature vector of s q N i , ||s q N i || represents its two-norm, and s q l is the semantic category, the matrix descriptor of the candidate scene is s c G, the grid number is N i , the feature vector of s c N i , ||s q N i || represents its two-norm, and s c l is the semantic category, the value stored in each unit of the matrix descriptor is the semantic distance, the difference between the descriptors is calculated using the cosine distance, and the constraint of the semantic information is added, d(s q G, s c G) is the similarity between the matrix descriptors, and the calculation formula is shown in equation (10):

[0043]

[0044] wherein, is the similarity calculation of each row of the matrix descriptor, as shown in equation (11):

[0045]

[0046] The calculated similarity is compared with the set similarity threshold value, if the similarity is less than the set similarity threshold value, the current scene has a corresponding closed loop scene, otherwise the current scene does not have a corresponding closed loop scene, and the closed loop detection part is completed. The results are applied to the process of simultaneous localization and mapping.

[0047] Beneficial technical effects

[0048] 1. The application proposes a three-dimensional point cloud loop detection algorithm using semantic topological descriptors, aiming at the problem of poor robustness of previous loop detection algorithms based on single information, after semantic segmentation of three-dimensional point cloud, the three-dimensional point cloud of dynamic semantic objects is discarded in the data preprocessing part to reduce dynamic effect and computational burden.

[0049] 2. The application obtains static feature points from the retained three-dimensional point cloud, and calculates corresponding scores according to semantic features and distance distribution, so as to express the importance of the feature points.

[0050] 3. The application uses bird's eye projection to extract nodes and construct a semantic topological graph through non-maximum suppression (NMS); the semantic topological graph is converted into a matrix descriptor, and a two-step search strategy is used for loop detection, which improves the matching speed while ensuring the accuracy.

[0051] 4. The application can effectively deal with loop detection under view angle transformation or dynamic scene, and serve the application scenarios of unmanned driving and map construction.

[0052] 5. In order to verify that the algorithm can achieve good matching accuracy, the application tests on the outdoor dataset KITTI, there are a large number of loop scenes in KITTI00, 02, 05, 06, 07, 08 sequences, the application respectively obtains the F1 maximum score of 0.984, 0.843, 0.970, 0.991, 0.915, 0.897, in addition, in the loop scene with dynamic objects, the F1 score of the application also remains above 0.8. In order to verify that the loop scene drift error calculated by the algorithm is smaller, the application tests on KITTI00, 02, 05, 06, 07, 08 datasets, the error on the yaw angle is 0.226, 1.083, 0.492, 0.070, 0.199, 1.204, 0.546 respectively, the unit is radian. In addition, the application tests the time consumption, for each frame of three-dimensional point cloud data, the average time consumption of the semantic segmentation module is 83ms, the time consumption of the descriptor calculation is 18ms, and the time consumption of the loop scene matching is 45ms, which can meet the time consumption requirement of the actual system. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 A three-dimensional point cloud loop detection method using semantic topological descriptors is provided for the embodiment of the application;

[0054] Figure 2 A three-dimensional point cloud loop detection method using semantic topological descriptors is provided for the embodiment of the application;

[0055] Figure 3A feature point and node extraction algorithm flowchart provided for an embodiment of the present application

[0056] Figure 4 A spherical projection example diagram provided for an embodiment of the present application

[0057] Figure 5 A closed loop detection flowchart provided for an embodiment of the present application

[0058] Figure 6 A grid diagram and converted two-dimensional descriptor diagram for bird's eye projection division of the present application. DETAILED DESCRIPTION

[0059] The specific embodiments of the present application will be further described in detail below in combination with the drawings and examples.

[0060] The data source of the present example is the three-dimensional point cloud data of the traffic intersection in the KITTI data set 00 sequence.

[0061] The present embodiment proposes a closed loop detection method for three-dimensional point cloud using semantic topological descriptor, as shown in Figure 1 、 Figure 2 , including the following steps:

[0062] S1: Extract the label category of the three-dimensional point cloud through semantic segmentation, and discard the specified three-dimensional point cloud of the object according to the semantic label of the three-dimensional point cloud;

[0063] In the S1, the original three-dimensional point cloud data is obtained by the laser radar, and the semantic category of each three-dimensional point cloud is obtained by the semantic segmentation network of the three-dimensional point cloud data.

[0064] The semantic segmentation uses RangeNet++ network; in the three-dimensional point cloud after semantic segmentation, the three-dimensional point cloud data with dynamic semantic object is discarded, and the static three-dimensional point cloud data in the scene is retained.

[0065] S2: Perform spatial transformation on the three-dimensional point cloud through convex hull projection, as shown in Figure 3 、 Figure 4 , according to the spatial distribution of the three-dimensional point cloud after spatial transformation, the concave-convex property of the point is judged, and the point corresponding to the original three-dimensional point cloud space of the convex point is taken as the feature point.

[0066] The three-dimensional point cloud after semantic segmentation also includes the static three-dimensional point cloud data corresponding to the discarded dynamic semantic object.

[0067] The convex hull projection reduces the data calculation by voxel down-sampling the three-dimensional point cloud; then the three-dimensional point cloud is spatially transformed by formula (1); set the original three-dimensional point cloud set P origin , the three-dimensional point cloud set P hulland a set of convex points P convex , the three-dimensional point cloud in each set is represented as p o , p h , and p c , the observation point center is represented as p v , and the projection formula is shown in formula (1):

[0068]

[0069] wherein:

[0070]

[0071] wherein f is a kernel function, and γ represents a coefficient factor, which is set to 10000 in the embodiment;

[0072] The convexity of the judgment point is determined, K is set as the number of extracted nearest neighbor points, K nearest neighbor points in the three-dimensional point cloud after spatial transformation are screened out, and the convex feature points satisfying formula (3) are extracted:

[0073]

[0074] wherein each three-dimensional point cloud includes a plurality of feature points, k represents the ranking serial number of the nearest neighbor point, wherein k∈{1,2,…,K}, represents the projection point corresponding to the k serial number;

[0075] S3: According to the geometric information and semantic information of the three-dimensional point cloud in the original space, the score of the feature point is calculated, the bird's eye view is divided into grids, the feature point is selected as the node of the semantic topology graph according to the grid position, and the maximum spatial distance between each node is considered as the edge of the semantic topology graph, so as to construct the semantic topology graph;

[0076] S4: The semantic topology graph is converted into a descriptor in the form of a matrix, as shown in formula (5): Figure 5 , Figure 6 The two-stage search strategy is used to complete the loop closure detection.

[0077] The score of the feature point is calculated, in order to evaluate the stability of the feature point, the distance distribution of the three-dimensional point cloud is taken as the geometric information, the three-dimensional point cloud closer to the mean distance has higher stability in the scene, the distance between the feature point and the observation center is set as d h , the mean distance of the three-dimensional point cloud in the scene is , and the variance is σ 2 , the geometric score φ1(p h ) of the feature point is calculated by using the Gaussian kernel function , as shown in formula (4):

[0078]

[0079] Semantic information as high-level feature information of the environment will not be affected by the change of the perspective, in order to be more consistent with the process of human perception of the environment, the system divides the three-dimensional point cloud into foreground points and background points through semantic categories, the foreground points include rod-shaped objects, trees, signal signs and other three-dimensional point clouds of objects with clear appearance contours and indicating significance, and the background points are large-scale three-dimensional point clouds of buildings, shrubs, lawns, ground, etc. Although the background points will not be particularly concerned, they are indispensable part of expressing the environment;

[0080] The more three-dimensional point clouds of the same semantic category around the feature point, the higher the semantic score φ2(p h ) of the feature point, and vice versa, as shown in formula (5):

[0081]

[0082] Wherein, α represents a coefficient factor, multiplying the geometric score φ1(p h ) and the semantic score φ2(p h ) of the feature point to obtain the total score φ(p h ) of the feature point, as shown in formula (6):

[0083] φ(p h )=φ1(p h )*φ2(p h ) (6)

[0084] The semantic topology graph is constructed by adopting a bird's eye view, taking the observation center as the center, and dividing N r concentric circles with equal distance from near to far, and taking the front of the observation as the starting angle, and dividing N s regions in the clockwise direction with equal interval, combining the concentric circle division lines and the radial region division lines, dividing the three-dimensional point cloud space into multiple grids from the top-down perspective, selecting the feature point with the maximum score in the same grid as the representative point of the grid through local maximum suppression, and taking all the representative points in the grid as the nodes of the semantic topology graph; considering the spatial distance of the feature points, taking the grid as the basic unit of distance, multiplying the distance between any two grids with the total score of the feature points to obtain the semantic distance between the nodes of the semantic topology graph with geometric and semantic weights, taking the maximum semantic distance between the nodes as the edge of the graph, and constructing the semantic topology graph;

[0085] The feature points of any grid are set as N i,m , wherein i is the serial number of the grid circle number, i∈{1,2,…,N r}, N rThe maximum index of the raster circle number; j represents the j-th index of the raster angle division, j∈{1,2,…,N} s}, N s Let be the maximum index of the grid angle division; m represents the m-th index of the grid angle division, and m ≠ j; the semantic category of the grid feature point is o, and max is set as the function that maximizes the semantic distance, where d is the semantic geometric distance. g Distance d between semantic categories s The calculations are shown in equations (7) and (8):

[0086]

[0087]

[0088] Where o represents the semantic category of the raster feature point;

[0089] S4: Convert the semantic topology graph into a matrix form descriptor, such as Figure 5 , Figure 6 As shown, a two-stage search strategy is used to complete loop closure detection;

[0090] In step S4, the raster image is stretched into a two-dimensional matrix using the radial angle as the starting point, and a matrix-form descriptor is constructed using the semantic distance as the corresponding row and column values ​​in the matrix.

[0091] Since the robot will store a large amount of scene data during its movement, it is unreasonable to use brute-force search for loop closure detection. Instead, a two-step search algorithm from coarse to fine is adopted to significantly reduce computational costs while ensuring accuracy.

[0092] The two-step search algorithm includes a first-step search algorithm and a second-step search algorithm; the grid map with the observation center as the center is composed of multiple concentric circles, divided at radially intervals of equal angles, and stretched into a two-dimensional matrix with the beginning and end connected.

[0093] The first step of the search algorithm converts the matrix into a vector k, and uses a kd-tree (K-demension tree) structure to store and search for candidate frames of the current scene;

[0094] The method of storing and searching candidate frames for the current scene using a kd-tree structure includes: obtaining the first value of vector k from the first row of the matrix, and obtaining subsequent values ​​from the next row, setting N... o N represents the number of selected semantic categories. r Divide the grid into circles. d for raster feature points of the same semantic type s average value, d for grid feature points with the same circle numberg The average value of all rows after encoding is N r +N o The eigenvector ε is shown in equation (9):

[0095]

[0096] Wherein, r represents the number of rows of the matrix, o is the semantic category,

[0097] The second step search algorithm sets the matrix descriptor of the current scene as s q G, the number of grid circles is N i The eigenvector of s q N i , ||s q N i || represents its two norms, and s is the semantic category q l, the matrix descriptor of the candidate scene is s c G, the number of grid circles is N i The eigenvector of s c N i , ||s q N i || represents its two norms, and s is the semantic category c l, the value stored in each unit of the matrix descriptor is the semantic distance, the difference between the descriptors is calculated using the cosine distance, and the constraint of semantic information is added, d(s q G, s c G) is the similarity between the matrix descriptors, and the calculation formula is shown in equation (10):

[0098]

[0099] Wherein, The similarity of each row of the matrix descriptor is calculated as shown in equation (11):

[0100]

[0101] The calculated similarity is compared with the set similarity threshold value, if the similarity is less than the set similarity threshold value, the current scene has a corresponding closed loop scene, otherwise the current scene does not have a corresponding closed loop scene, and the closed loop detection part is completed, and the result is applied to the process of simultaneous localization and mapping.

[0102] To further verify that the technical solution adopted by the present application can achieve good matching accuracy, the present embodiment is tested on the outdoor data set KITTI. There are a large number of closed-loop scenes in KITTI00, 02, 05, 06, 07, 08 sequences. The present embodiment respectively achieves F1 maximum score of 0.984, 0.843, 0.970, 0.991, 0.915, 0.897. In addition, in the closed-loop scene with dynamic objects, the F1 score of the present embodiment also remains above 0.8. To verify that the closed-loop scene drift error calculated by the present solution is smaller, the KITTI00, 02, 05, 06, 07, 08 data sets are tested, and the error in yaw angle is 0.226, 1.083, 0.492, 0.070, 0.199, 1.204, 0.546, respectively, in radian.

[0103] Table 1. Outdoor data set KITTI test data table

[0104]

[0105] The present embodiment is experimentally tested for the time consumption. For each frame of three-dimensional point cloud data, the average semantic segmentation module consumes 83 ms, the descriptor calculation consumes 18 ms, and the closed-loop scene matching consumes 45 ms, which can meet the time consumption requirement of the actual system.

Claims

1. A method for loop closure detection of 3D point clouds using semantic topological descriptors, characterized in that, Includes the following steps: S1: Extract the label categories of the 3D point cloud through semantic segmentation, and discard the specified object 3D point cloud according to the semantic labels of the 3D point cloud; S2: The 3D point cloud is spatially transformed by convex hull projection. The concavity and convexity of the points are determined based on the spatial distribution of the 3D point cloud after the spatial transformation. The points in the original 3D point cloud space corresponding to the convex points are taken as feature points. S3: Calculate the feature point scores based on the geometric and semantic information of the 3D point cloud in the original space, divide the grid using the bird's-eye view, select the feature points as nodes of the semantic topology graph based on the grid position, and use the maximum spatial distance between each node as the edge of the semantic topology graph to construct the semantic topology graph. S4: Convert the semantic topology graph into a matrix descriptor. Using a two-step search algorithm, starting from the radial angle of the grid graph, stretch it into a two-dimensional matrix. Use the semantic distance as the value of the corresponding row and column in the matrix to construct a matrix descriptor and complete the loop closure detection. The construction of the semantic topology graph uses a bird's-eye view, with the observation center as the center and the radius divided equally from near to far. Divide the area into concentric circles, using the observed front as the starting angle, and then radially divide it at equal intervals in a clockwise direction. The 3D point cloud space is divided into multiple grids by concentric circle dividing lines and radial dividing lines, taking a top-down view. Feature points at different heights in the same grid are selected by local maximum suppression to select the feature point with the highest score as the representative point of the grid. The representative points in all grids are used as nodes of the semantic topology graph. Using the grid as the basic unit of distance, the distance between any two grids is multiplied by the total score of the feature points to obtain the semantic distance between nodes of the semantic topology graph with geometric and semantic weights. The maximum semantic distance between nodes is taken as the edge of the graph to construct the semantic topology graph. Define the feature points of any raster as ,in It is the sequence number of the grid rings. , The maximum sequence number of the grid rings; j The first grid angle division j Serial number , The maximum sequence number for the grid angle division; The first grid angle division Serial number, and The semantic category of the raster feature points is ,set up To maximize the semantic distance, the semantic geometric distance is... Distance between semantic categories The calculations are shown in equations (7) and (8): (7) (8) in, Indicates the semantic category of raster feature points. Representing feature points The score, Representing feature points semantic category; The two-step search algorithm includes a first-step search algorithm and a second-step search algorithm. The first-step search algorithm converts the matrix into a vector. The kd-tree structure is used to store and search for candidate frames in the current scene; The method of storing and searching candidate frames for the current scene using a kd-tree structure includes: vectors The first value is obtained from the first row of the matrix, and subsequent values ​​are obtained sequentially from the next row. Let... The number of selected semantic categories, Divide the grid into circles. raster feature points with the same semantic category average value, raster feature points with the same circle number Average value, generated after encoding of all rows dimensional vector As shown in equation (9): (9) in, Indicates the number of rows in the matrix. It is a semantic category; The second step of the search algorithm sets the matrix descriptor of the current scene as... The number of grid circles is The feature vector is , Represents its second norm, and its semantic categories are: The matrix descriptor for the candidate scene is The number of grid circles is The feature vector is , Represents its second norm, and its semantic categories are: In the matrix descriptor, each unit stores the semantic distance. The cosine distance is used to calculate the differences between descriptors, and semantic constraints are added. The similarity between matrix descriptors is calculated using the formula shown in equation (10): (10) in, The similarity is calculated for each row of the matrix descriptor, as shown in equation (11): (11) The calculated similarity is compared with a set similarity threshold. If the similarity is less than the set similarity threshold, then there is a corresponding closed loop scene in the current scene; otherwise, there is no corresponding closed loop scene in the current scene. The closed loop detection part is completed, and the result is applied to the simultaneous localization and mapping process.

2. The method for loop closure detection of 3D point clouds using semantic topological descriptors as described in claim 1, characterized in that, S1 includes: acquiring raw 3D point cloud data through a lidar, and obtaining the semantic category of each 3D point cloud by passing the raw 3D point cloud data through a semantic segmentation network for 3D point cloud data.

3. The method for loop closure detection of 3D point clouds using semantic topological descriptors as described in claim 2, characterized in that, The semantic segmentation uses the RangeNet++ network; in the semantically segmented 3D point cloud, 3D point cloud data with dynamic semantic objects are discarded, while static 3D point cloud data in the scene are retained.

4. The method for loop closure detection of 3D point clouds using semantic topological descriptors as described in claim 2, characterized in that, The semantically segmented 3D point cloud also includes static 3D point cloud data corresponding to discarded dynamic semantic objects.

5. The method for loop closure detection of 3D point clouds using semantic topological descriptors as described in claim 1, characterized in that, The convex hull projection performs voxel downsampling on the three-dimensional point cloud, and the three-dimensional point cloud is spatially transformed by equation (1). Set the original 3D point cloud set 3D point cloud set after projection and convex point set The corresponding 3D point cloud in each set is represented as , and The center of the observation point is represented as The projection formula is shown in equation (1): (1) in: (2) in, For kernel function, Let q represent the coefficient factor, and let q represent the original 3D point cloud set. 3D point cloud in the image.

6. The method for loop closure detection of 3D point clouds using semantic topological descriptors as described in claim 1, characterized in that, The concavity or convexity of the judgment point, let To set the number of nearest neighbors to extract, the nearest neighbors are selected from the spatially transformed 3D point cloud. Extract the convex feature points that satisfy formula (3) from the points: (3) Each 3D point cloud contains multiple feature points. Represents the sorting index of the nearest neighbor, where , Indicates correspondence The projection point of the serial number.

7. The method for loop closure detection of 3D point clouds using semantic topological descriptors as described in claim 1, characterized in that, The calculation of feature point scores uses the distance distribution of the 3D point cloud as geometric information, setting the distance between the feature points and the observation center as... The mean distance of the 3D point cloud in the scene is The variance is Through Gaussian kernel function Calculate the geometric fraction of the feature points As shown in equation (4): (4) The more 3D point clouds of neighboring points with the same semantic category a feature point has, the higher its semantic score. The higher the score, the lower the semantic score, as shown in equation (5): (5) in, Representing coefficient factors, the geometric fraction of feature points. and semantic score Multiply to obtain the total score of feature points As shown in equation (6): (6)。 8. The method for loop closure detection of 3D point clouds using semantic topological descriptors as described in claim 1, characterized in that, The two-step search algorithm includes a first-step search algorithm and a second-step search algorithm; the grid map with the observation center as the center is composed of multiple concentric circles, divided at radially intervals of equal angles, and stretched into a two-dimensional matrix with the beginning and end connected.

Images