Semantic aided vision SLAM loopback detection method based on topological graph matching

Abstract

The invention discloses a semantic auxiliary vision SLAM loopback detection method based on topological graph matching, and belongs to the field of computer vision. The implementation method comprises the following steps of: screening out candidate key frames without a common-view relationship by judging whether the common-view relationship exists between a current frame and each frame in a historical key frame sequence or not; and constructing a semantic topological graph, namely constructing nodes and undirected weighted edges in the topological graph and calculating node descriptors, and realizing topological expression of the key frame image by constructing the semantic topological graph and calculating a feature descriptor for each node in the topological graph. And calculating an affinity matrix, constructing the affinity matrix between the topological graphs by combining the similarity between the weights of the node descriptors and the edges in the topological graph of the current key frame and the topological graph of the candidate key frame, and calculating an incidence matrix between the nodes of the topological graphs at the same time. And calculating the topological graph similarity through the affinity matrix and the incidence matrix. And judging whether loopback detection succeeds or not by judging whether the topological graph similarity reaches a loopback detection threshold value or not.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and relates to a semantic-assisted visual SLAM loop detection method based on topological graph matching. Background Art

[0002] Simultaneous Localization and Mapping (SLAM) is a core technology in fields such as robotics, autonomous driving, and augmented reality. Its goal is to estimate the vehicle's position and construct a map of the environment in real time using sensor data. Visual SLAM (VSLAM), which uses cameras as its primary sensor, has become a research hotspot in recent years due to its low cost, high information density, and wide applicability. However, due to the presence of cumulative errors, methods that rely solely on front-end odometry and back-end optimization struggle to address the problem of positioning drift over long periods of time. In this context, loop closure detection, a key module of SLAM technology, identifies the similarity between the current scene and historical maps, providing constraints for back-end optimization, thereby significantly reducing positioning errors and improving global consistency. This can improve the overall map construction accuracy in real-time positioning and mapping engineering practices, allowing for the construction of more accurate maps for various application scenarios.

[0003] The key technical issue of loop closure detection is to detect the correct loop matching pairs through matching algorithms, and then eliminate accumulated errors through global optimization to improve mapping accuracy.

[0004] Traditional loop detection algorithms primarily extract geometric features from images in the SLAM tracking thread and perform loop detection through feature matching. For example, the MonoSLAM algorithm proposed by Davison et al. achieves real-time positioning using an extended Kalman filter (EKF). However, its loop detection relies on matching sparse feature points, making it difficult to handle scenes with texture loss. The introduction of bag-of-words (Bag of Words) models, such as the ORB-SLAM family of algorithms that utilize binary features (ORB) and a hierarchical bag-of-words structure, has significantly improved matching efficiency and scene discrimination. However, they still face the problem of false matches caused by insufficient feature discrimination. The LSD-SLAM algorithm proposed by Ngel et al. uses a direct method to extract image features for loop detection. However, its detection performance relies on photometric consistency between keyframes, resulting in poor performance under drastic lighting changes. The PLI-SLAM algorithm proposed by Z Teng et al. introduces line features on top of point features to assist in loop detection, but the accuracy improvement is not significant in scenes where line features are difficult to obtain.

[0005] In recent years, breakthroughs in deep learning technology have provided new insights into loop closure detection. On the one hand, deep learning-based feature point extraction and matching algorithms (such as the SuperPoint and SuperGlue algorithms) have improved the success rate of feature point extraction and matching, thereby improving loop closure detection accuracy. On the other hand, end-to-end loop closure detection algorithms based on deep learning (such as the CALC algorithm proposed by Nate Merrill et al.) can achieve end-to-end loop closure detection without ordering the given feature detection results. These deep learning-based methods can effectively improve loop closure detection accuracy, but they require additional model training and significantly increase the computational complexity of loop closure detection.

[0006] Some visual SLAM algorithms, such as the SemanticLoop algorithm proposed by Yu Junfeng et al., use semantic information to assist in loop detection. These algorithms construct a topological map based on the semantic detection results in the image frames and then implement loop detection using a topological map matching algorithm. This type of algorithm can improve loop detection accuracy by leveraging object-level semantic topological maps, but their semantic topological map construction and similarity calculation methods are overly simplistic, ignoring much of the implicit information in the topological map.

[0007] In general, the current loop detection algorithm has the following shortcomings:

[0008] 1. The traditional loop detection algorithm based on feature points has low detection accuracy and can only detect loop candidate frames with high image similarity. In addition, the loop detection accuracy drops sharply under conditions such as lighting changes and missing image texture.

[0009] 2. The loop detection algorithm based on deep learning requires high computer computing power and has poor generalization ability;

[0010] 3. The construction of the semantic topology graph and the similarity calculation method of the loop detection algorithm based on the topology graph are too simple, resulting in low detection accuracy.

[0011] The advantages of the present invention are: it can construct a topological map by introducing semantic information, combining multiple similarity calculation strategies, fully integrating semantic information with spatial topological relationships, achieving loop detection with higher precision and recall rate, and improving mapping accuracy. Summary of the Invention

[0012] In order to solve the problems of false detection and missed detection in traditional loop detection algorithms, the purpose of the present invention is to provide a semantic-assisted visual SLAM loop detection method based on topological graph matching. By introducing target detection result information, constructing a topological graph containing semantic information and combining it with a graph matching algorithm, semantic-assisted visual SLAM loop detection based on topological graph matching is realized, which can improve the accuracy of visual loop detection without significantly increasing the computational complexity.

[0013] The purpose of the present invention is achieved through the following technical solutions:

[0014] The present invention discloses a semantic-assisted visual SLAM loop detection method based on topological graph matching. In the loop detection task, it is determined that there is no loop between key frames with a common view relationship. Therefore, by judging whether the current frame has a common view relationship with each frame in the historical key frame sequence, candidate key frames without a common view relationship are screened out, the amount of calculation is reduced, and a candidate key frame set is formed. The semantic topological graph is constructed, including the construction of nodes and undirected weighted edges in the topological graph and the calculation of node descriptors. By constructing the semantic topological graph and calculating the feature descriptor for each node in the topological graph, a topological representation of the key frame image is achieved for subsequent graph matching tasks. The affinity matrix is ​​calculated, combining the similarity between the node descriptors and the edge weights in the topological graph of the current key frame and the topological graph of the candidate key frame, to construct an affinity matrix between the topological graphs, and simultaneously calculate the association matrix between the nodes of the topological graph for subsequent similarity calculation. The topological graph similarity is calculated using the affinity matrix and the association matrix. The loop detection is successful by judging whether the topological graph similarity reaches the loop detection threshold.

[0015] The semantic-assisted visual SLAM loop detection method based on topological graph matching disclosed in the present invention comprises the following steps:

[0016] S1. For the current key frame Filter all candidate key frames from the historical key frame set based on the co-viewing relationship Set the current keyframe It judges each key frame in the historical key frame sequence and finds all historical key frames that do not have a co-viewing relationship to form a candidate key frame set.

[0017] S2. For the current key frame and candidate keyframes Using the same semantic topology construction method, the current key frames are obtained respectively and candidate keyframes The semantic topology graph consists of nodes and the edge composition.

[0018] S3. For the current key frame and candidate keyframes Each node in the semantic topology graph All neighbor nodes are used separately right Descriptor to update.

[0019] S4. For the current key frame Topology and candidate keyframes Topology Assume that the number of nodes in the two topological graphs is m and n respectively. First, construct the node-level affinity matrix of the two topological graphs respectively. and edge-level affinity matrix Then based on the matrix T n and matrix T e Construct the affinity matrix of the two topological graphs

[0020] S5. According to the matrix T g and node association matrix X, calculate two topological graphs and The topological similarity s,

[0021] S6: Loop detection result determination. If the topology similarity s is greater than the threshold τ s , then the frame is determined to be a successfully matched loop candidate frame.

[0022] S7: Based on the obtained loop candidate frames, loop correction and global BA optimization are performed to offset scale drift errors and improve the system mapping accuracy.

[0023] For the current key frame in S2 and candidate keyframes Using the same semantic topology construction method, the current key frames are obtained respectively and candidate keyframes The semantic topology graph of is implemented as follows:

[0024] Key frame images Perform target detection, each detection result Construct a semantic topology node

[0025] For each target detection result Use the center of the detection frame as the observation center of the object to obtain a three-dimensional space coordinate The three-dimensional space coordinates are used as the geometric center of the observed object in the camera coordinate system. Then, combined with the semantic label of the object Construct a topological node containing semantic information, represented as After obtaining the node set in the topology graph, calculate each node The distance from all other nodes in the current topology node set. If the node With node The distance between Less than the threshold τ d , then the two nodes are considered to be neighbor nodes, and a topological edge is constructed between the two nodes. The weight of this topology edge is the Euclidean distance between two nodes. A three-dimensional vector is stored in

[0026]

[0027] in and Node and nodes The three-dimensional space coordinates of . and weights Storing topological edges Get a topological edge containing spatial information, expressed as

[0028] Complete keyframe After all nodes and edges are constructed, a complete semantic topology graph is obtained, which is represented as in and ε q Represents the set of nodes and edges respectively, n q for The number of nodes in the graph. Corresponding to the current key frame, the constructed topology graph is Corresponding to candidate keyframes The constructed topology is

[0029] All neighbor nodes are used in S3 right Descriptor To update, the implementation method is:

[0030] Set the length of each distance interval in the histogram descriptor to Δ d , then the node descriptor The dimension is where n cat is the total number of semantic tags. According to 0~(n cat -1), then the neighbor node with semantic label number l in the descriptor corresponds to the interval Initialize the topology map The descriptor of each node in is a zero vector of length dim, and then the edges and weights of all neighboring nodes are counted and the node descriptors are updated.

[0031] The method for updating node descriptors in S3 is:

[0032] For nodes Traverse all neighbor nodes For the jth neighbor node Semantic tag number is Using neighbor nodes right Descriptor To update:

[0033]

[0034] in, Indicates rounding down. After the above steps, the topology graph is Each node Construct a feature descriptor that contains neighborhood semantic information

[0035] The specific implementation method of S4 is: for the current key frame Topology and candidate keyframes Topology Assume that the number of nodes in the two topological graphs is m and n respectively, and the affinity matrices of the two topological graphs are Need to pass the node level affinity matrix and edge-level affinity matrix Constructed.

[0036] According to the matrix T n and matrix T e Construct affinity matrix T g , about to T n and T e Assign the corresponding elements in to T g The corresponding position in which T n Any element in T g A diagonal element in T e Any element in T g The four off-diagonal elements in T g The rest of the elements are 0.

[0037] Each element in the node-level affinity matrix The calculation method is

[0038]

[0039] in, and Represents the topology graph The node descriptor and topology graph of the i-th node in The node descriptor of the jth node in . Then normalize the node-level affinity matrix, that is, divide all elements in the matrix by the maximum element value to obtain the node-level affinity matrix T n .

[0040] Edge-level affinity matrix T e The dimension of depends on the dimension of the edge sets of the two topological graphs, and the maximum dimension is mn*mn. The connecting edge numbered p and topology diagram The connecting edge numbered q If both edges are real, and the edge The category labels of the connecting nodes and the edges The category labels of the connected nodes match, then the edge-level affinity matrix T e The corresponding terms in are calculated as

[0041]

[0042] otherwise,

[0043] The topology similarity calculation method in S5 is:

[0044] According to the matrix T g and node association matrix X, calculate two topological graphs and The topological similarity s is calculated as follows:

[0045] s=vec(X) T ·T g vec(X)

[0046] Among them, vec(.) means that the matrix is ​​expanded into a one-dimensional column vector by column.

[0047] The matrix T n As the cost matrix of the Hungarian algorithm, the node matching results of the two topological graphs are calculated and stored in the association matrix In the case where each element in the incidence matrix takes a value of 0 or 1, x i,j =0 indicates topology map The i-th node and the topology graph The jth node in x is not associated. i,j =1, indicating association.

[0048] The S7 implementation method uses loop closure candidate frames to perform loop correction and global bundle adjustment (BA) optimization to offset scale drift errors and improve system mapping accuracy. This improved mapping accuracy significantly improves the camera's final positioning accuracy for visual mapping and localization in large scenes, while also improving the accuracy of the constructed map.

[0049] Beneficial effects:

[0050] 1. The semantic-assisted visual SLAM loop detection method based on topological graph matching disclosed in the present invention introduces high-dimensional semantic information. Compared with traditional loop detection algorithms based on point feature or line feature matching, it can use more robust semantic information to assist loop detection tasks, thereby improving the accuracy of loop detection.

[0051] 2. The semantic-assisted visual SLAM loop detection method based on topological map matching disclosed in the present invention directly uses the classic target detection results as input and adopts a strategy based on topological map matching for loop detection. It does not require model training. It only needs to use a target detector with a pre-trained model to obtain the target detection results in the image. The method has strong generalization and strong engineering deployment advantages. It is far superior to the loop detection method using deep learning in terms of deployment simplicity and deployment equipment computing power requirements.

[0052] 3. The semantic-assisted visual SLAM loop detection method based on topology map matching disclosed in the present invention adopts a threshold-based topology map construction method, a histogram-based descriptor construction method, and an edge-level similarity calculation method combining scale similarity and angle similarity to evaluate topology map similarity. The topology map similarity calculation adopts a sparse affinity matrix. The calculation method has clear algorithm logic, a small number of calculation parameters, but high calculation accuracy, and the loop detection accuracy is better than the traditional loop detection method using topology map matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Schematic diagram of the flow chart of the semantic-assisted visual SLAM loop detection method based on topological graph matching of the present invention;

[0054] Figure 2 Scene semantic topology map based on distance threshold;

[0055] Figure 3 Schematic diagram of histogram node descriptor construction;

[0056] Figure 4 Schematic diagram of affinity matrix construction. DETAILED DESCRIPTION

[0057] In order to better illustrate the purpose and advantages of the present invention, the invention is further described below with reference to the accompanying drawings and examples.

[0058] The semantic-assisted visual SLAM loop detection method based on topological graph matching disclosed in this embodiment is mainly divided into five steps: (1) Common view relationship determination. In the loop detection task, it is considered that there is no loop between key frames with common view relationship. Therefore, by judging whether there is a common view relationship between the current frame and each frame in the historical key frame sequence, candidate key frames without common view relationship are screened out to reduce the amount of calculation. (2) Semantic topological graph construction, including the construction of nodes and undirected weighted edges in the topological graph and the calculation of node descriptors. By constructing a semantic topological graph and calculating feature descriptors for each node in the topological graph, a topological representation of the key frame image is achieved for subsequent graph matching tasks. (3) Affinity matrix calculation. Combining the similarity between the node descriptors and edge weights in the topological graph of the current key frame and the topological graph of the candidate key frame, an affinity matrix between the topological graphs is constructed, and at the same time, a correlation matrix between the nodes of the topological graph is calculated for subsequent similarity calculation. (4) Topological graph similarity calculation. The topological graph similarity is calculated using the affinity matrix and correlation matrix obtained in the previous step. (5) Loop detection result determination: whether the loop detection is successful is determined by judging whether the topological graph similarity reaches the loop detection threshold.

[0059] like Figure 1 As shown, the semantic-assisted visual SLAM loop detection method based on topological graph matching disclosed in this embodiment has the following specific steps:

[0060] S1: Determination of co-viewing relationship. For the current key frame input to the loop detection module Traverse all key frames in the historical key frame sequence. For key frames in the historical key frame sequence We determine whether this key frame exists in the current key frame In the common view key frame sequence, if it exists, no subsequent operation is performed and the next historical key frame is entered. Otherwise, the frame is used as a candidate key frame for loop closure detection and the subsequent steps are performed. Figure 4 The upper left corner shows an example of the current key frame and the filtered loop candidate frame. It can be seen that there is a certain difference in perspective between the two images. The result of the common view relationship judgment is that there is no common view relationship.

[0061] S2: semantic topology map construction. For the current key frame and candidate keyframes Using the same semantic topology construction method, the following keyframe For example.

[0062] First, for each detection result from the object detector Construct a semantic topology node respectively Specifically, since the input of the SLAM system is RGB-D information, for each target detection result o i , taking the center of the detection frame as the observation center of the object, we can get a three-dimensional space coordinate This coordinate is used as the geometric center of the observed object in the camera coordinate system. Then, combined with the semantic label of the object A topological node containing semantic information can be constructed, expressed as After obtaining the node set in the topological graph, the topological edge is constructed according to the distance threshold between the nodes. For each node Calculate the distance between it and all other nodes in the current topology node set. If the node With node The distance between Less than the threshold τ d , then the two nodes are considered to be neighbor nodes, and a topological edge is constructed between the two nodes. The weight of this topology edge is the Euclidean distance between two nodes. In addition, in order to store the orientation information of the topological edge in the three-dimensional scene, we A three-dimensional vector is stored in

[0063]

[0064] In summary, we get a topological edge containing spatial information, expressed as

[0065] The topological graph constructed according to the distance threshold is as follows Figure 2 As shown, different colors represent different category labels, and each object observation is represented by an elliptical detection box.

[0066] Complete keyframe After all nodes and edges are constructed, a complete semantic topology graph is obtained, which is represented as in and ε q Represents the set of nodes and edges respectively, n q for The number of nodes in the middle; corresponding to the current key frame, the constructed topology is Corresponding to candidate keyframes The constructed topology is

[0067] S3: Affinity matrix calculation. After completing the construction in the topology graph, in order to store neighbor node information (or called topological connection information) in the node, it is necessary to construct a semantic descriptor for each node. The histogram-based node descriptor construction strategy can construct a more sparse and dimensionally uniform descriptor vector compared to other descriptor construction algorithms, which can greatly simplify the computational complexity of the node matching process. Set the length of each distance interval in the histogram descriptor to Δ d , then the dimension of the node descriptor is where n cat is the total number of semantic tags. cat -1), then the neighbor node with semantic label number l in the descriptor corresponds to the interval For each node in the topology graph, we first initialize the descriptor of each node to a zero vector of length dim, then count the edges and weights of all neighboring nodes (nodes connected to the current node) and update the node descriptor.

[0068] like Figure 3 The diagram shows the construction of the node descriptor for the object observation represented by the green oval in the figure. Specifically, for the node We iterate over all neighbor nodes For the jth neighbor node Its semantic tag number is Using neighbor nodes right Descriptor To update:

[0069]

[0070] in, Indicates rounding down. After the above steps, the topology map is realized. Each node Construct a feature descriptor that contains neighborhood semantic information like Figure 3 As shown in the figure, for the object corresponding to the green ellipse, the obtained partial node descriptor vector is: 000001001100000001000010000000.

[0071] S4: Affinity matrix calculation. This paper uses affinity matrix to calculate the similarity of topological graphs. Topology and candidate keyframes Topology Assume that the number of nodes in the two topological graphs is m and n respectively, and the affinity matrix for evaluating the similarity between the two topological graphs is Need to pass the node level affinity matrix and edge-level affinity matrix The construction of affinity matrix is ​​as follows: Figure 4 shown.

[0072] First, construct the node-level affinity matrix T n , the node-level affinity matrix T n Each element in The calculation method is

[0073]

[0074] in, and Represents the topology graph The node descriptor and topology graph of the i-th node in After the calculation is completed, the maximum norm normalization method is used to normalize T n Normalize the matrix by dividing all elements by the maximum value.

[0075] Complete the node-level affinity matrix T n After the calculation, the affinity matrix can be directly used as the cost matrix of the Hungarian algorithm to calculate the node matching results of the two topological graphs and store them in the association matrix. In the case where each element in the incidence matrix takes a value of 0 or 1, x i,j =0 indicates topology map The i-th node and the topology graph The jth node in is not associated; otherwise, it is associated.

[0076] Then calculate the edge-level affinity matrix T e , the matrix dimension depends on the dimension of the edge set of the two topological graphs, and the maximum dimension is mn*mn. The connecting edge numbered p (corresponding to a unique connecting edge Here ij is abbreviated as p, the same below) and topological graph The connecting edge numbered q (corresponding to a unique connecting edge Here ij is abbreviated as q, the same below), if both edges exist, and the edge The category labels of the connecting nodes and the edges The category labels of the connected nodes match, then the edge-level affinity matrix T e The corresponding terms in are calculated as

[0077]

[0078] otherwise,

[0079] Calculate the node-level affinity matrix T n and edge-level affinity matrix T e After that, the topological graph-level affinity matrix T can be constructed g , just follow Figure 4 As shown in T n and T e Assign the corresponding elements in to T g The specific position in n Any element in T g A diagonal element in T e Any element in T g The four off-diagonal elements in T g The rest of the elements are 0.

[0080] S4: Topology similarity calculation. Based on the topology affinity matrix T obtained above g And the node association matrix X, we can calculate two topological graphs and The topological similarity s is calculated as follows:

[0081] s=vec(X) T ·T g vec(X)

[0082] Among them, vec(.) means that the matrix is ​​expanded into a one-dimensional column vector by column. Figure 4 In the given example, the affinity matrix and node association matrix can be calculated to have a topological similarity value of 3.9.

[0083] S5: Loop detection result determination. For the current frame and candidate keyframes If their topological graph similarity s is greater than the threshold τ s , then the frame is considered to be a successfully matched loop candidate frame, and the subsequent geometric verification screening and loop optimization process can be carried out. Figure 4 A pair of key frame matching pairs in , if the similarity value between them is greater than the threshold τ s , then it is believed that Figure 4 The second frame image in is the loop candidate frame that successfully matches the first frame image.

[0084] S6: Based on the obtained loop candidate frames, loop correction and global bundle adjustment (BA) optimization are performed to offset scale drift errors and improve the system's mapping accuracy. This method significantly improves the camera's final positioning accuracy for visual mapping and positioning in large scenes, while also improving the accuracy of the constructed map.

[0085] The above specific description further illustrates the purpose, technical solutions and beneficial effects of the invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A semantically assisted visual SLAM loop detection method based on topological graph matching, characterized by: In the loop detection task, it is determined that there is no loop between key frames that have a common view relationship. Therefore, by judging whether the current frame has a common view relationship with each frame in the historical key frame sequence, the candidate key frames that do not have a common view relationship are screened out to reduce the amount of calculation and form a candidate key frame set; Semantic topological graph construction, including the construction of nodes and undirected weighted edges in the topological graph and the calculation of node descriptors. By constructing the semantic topological graph and calculating the feature descriptor for each node in the topological graph, the topological representation of the keyframe image is achieved for subsequent graph matching tasks. Affinity matrix calculation, combining the similarity between the node descriptors and edge weights in the topological graph of the current keyframe and the topological graph of the candidate keyframe, constructs the affinity matrix between the topological graphs, and simultaneously calculates the association matrix between the nodes of the topological graph for subsequent similarity calculation. Calculate the similarity of topological graphs through affinity matrix and correlation matrix; Whether the loop detection is successful is determined by judging whether the topology similarity reaches the loop detection threshold.

2. The method according to claim 1, wherein: The following steps are included: S1. For the current key frame Filter all candidate key frames from the historical key frame set based on the co-viewing relationship Set the current keyframe Compare with each key frame in the historical key frame sequence, and find all historical key frames that do not have a co-viewing relationship to form a candidate key frame set; S2. For the current key frame and candidate keyframes Using the same semantic topology construction method, the current key frames are obtained respectively and candidate keyframes The semantic topology graph consists of nodes and the edge composition; S3. For the current key frame and candidate keyframes Each node in the semantic topology graph All neighbor nodes are used separately right Descriptor Make updates; S4. For the current key frame Topology and candidate keyframes Topology Assume that the number of nodes in the two topological graphs is m and n respectively. First, construct the node-level affinity matrix of the two topological graphs respectively. and edge-level affinity matrix Then based on the matrix T n and matrix T e Construct the affinity matrix of the two topological graphs S5. According to the matrix T g and node association matrix X, calculate two topological graphs and The topological similarity s, S6: Loop detection result determination; if the topology similarity s is greater than the threshold τ s , then the frame is determined to be a successfully matched loop candidate frame; S7: Based on the obtained loop candidate frames, loop correction and global BA optimization are performed to offset scale drift errors and improve the system mapping accuracy.

3. The method according to claim 2, wherein: For the current key frame in S2 and candidate keyframes Using the same semantic topology construction method, the current key frames are obtained respectively and candidate keyframes The semantic topology graph of , is realized by, Key frame images Perform target detection, each detection result Construct a semantic topology node For each target detection result Use the center of the detection frame as the observation center of the object to obtain a three-dimensional space coordinate The three-dimensional space coordinates are used as the geometric center of the observed object in the camera coordinate system; then, combined with the semantic label of the object Construct a topological node containing semantic information, represented as After obtaining the node set in the topology graph, calculate each node The distance from all other nodes in the current topology node set. If the node With node The distance between Less than the threshold τ d , then the two nodes are considered to be neighbor nodes, and a topological edge is constructed between the two nodes. The weight of this topology edge is the Euclidean distance between two nodes; A three-dimensional vector is stored in in and Node and nodes The three-dimensional space coordinates of and weights Storing topological edges Get a topological edge containing spatial information, expressed as Complete keyframe After all nodes and edges are constructed, a complete semantic topology graph is obtained, which is represented as in and ε q Represents the set of nodes and edges respectively, n q for The number of midpoints; Corresponding to the current key frame, the constructed topology is Corresponding to candidate keyframes The constructed topology is 4. The method according to claim 3, characterized in that: S3: using all neighboring nodes right Descriptor To update: Set the length of each distance interval in the histogram descriptor to Δ d , then the node descriptor The dimension is where n cat is the total number of semantic tags; for all semantic tags According to 0~(n cat -1), then the neighbor node with semantic label number l in the descriptor corresponds to the interval Initialize the topology map The descriptor of each node in is a zero vector of length dim, and the edges and weights of all neighboring nodes are counted, and the node descriptor is updated.

5. The method according to claim 4, wherein: The method for updating the node descriptor in step S3 is: For nodes Traverse all neighbor nodes For the jth neighbor node Semantic tag number is Using neighbor nodes right Descriptor To update: in, Indicates rounding down; after the above steps, the topology graph is realized Each node Construct a feature descriptor that contains neighborhood semantic information 6. The method according to claim 5, wherein: The specific implementation method of S4 is as follows: for the current key frame Topology and candidate keyframes Topology Assume that the number of nodes in the two topological graphs is m and n respectively, and the affinity matrices of the two topological graphs are Need to pass the node level affinity matrix and edge-level affinity matrix Constructed; According to the matrix T n and matrix T e Construct affinity matrix T g , about to T n and T e Assign the corresponding elements in to T g The corresponding position in which T n Any element in T g A diagonal element in T e Any element in T g The four off-diagonal elements in T g The rest of the elements are 0.

7. The method according to claim 6, wherein: Each element in the node-level affinity matrix The calculation method is in, and Represents the topology graph The node descriptor and topology graph of the i-th node in The node descriptor of the jth node in ; Then normalize the node-level affinity matrix, that is, divide all elements in the matrix by the maximum element value to obtain the node-level affinity matrix T n .

8. The method according to claim 7, wherein: The edge-level affinity matrix T e The dimension of the edge set depends on the dimension of the two topological graphs, and the maximum dimension is mn*mn; for the topological graph The connecting edge numbered p and topology diagram The connecting edge numbered q If both edges are real, and the edge The category labels of the connecting nodes and the edges The category labels of the connected nodes match, then the edge-level affinity matrix T e The corresponding terms in are calculated as otherwise, 9. The method according to claim 8, wherein: The topology similarity calculation method in S5 is: According to the matrix T g and node association matrix X, calculate two topological graphs and The topological similarity s is calculated as follows: s=thing(X) T ·T g ·thing(X) Among them, vec(.) means that the matrix is ​​expanded into a one-dimensional column vector by column; The matrix T n As the cost matrix of the Hungarian algorithm, the node matching results of the two topological graphs are calculated and stored in the association matrix In the case where each element in the incidence matrix takes a value of 0 or 1, x i,j =0 indicates topology map The i-th node and the topology graph The jth node in x is not associated; i,j =1, indicating association.

10. The method according to claim 9, wherein: The S7 implementation method performs loop correction and global bundle adjustment (BA) optimization based on the obtained loop candidate frames to offset scale drift errors and improve the system's mapping accuracy. By improving mapping accuracy, the camera's final positioning accuracy for visual mapping and positioning in large scenes is significantly improved, while also improving the accuracy of the constructed map.

Images