Anti-dynamic interference Slam loopback detection method and device based on FPN-SuperPoint and SuperGlue fusion, and medium

Abstract

The invention discloses an anti-dynamic interference Slam loopback detection method and device based on FPN-SuperPoint and SuperGlue fusion and a medium, and the method comprises the following steps: S1, obtaining camera image sequence data, and inputting the camera image sequence data into an FPN-SuperPoint network; further performing feature matching by using a SuperGlue network; s2, introducing a semantic-probability coupled adaptive weighted sub-pixel optimization strategy to each feature point and probability distribution information thereof in the initial feature matching pair to obtain a sub-pixel-level high-precision feature point, and updating the initial feature matching pair based on the optimized feature point; s3, eliminating matching pairs corresponding to the feature points in the dynamic region through target detection and improved Kalman filtering to obtain static feature matching pairs; s4, for the static feature matching pair output in the S3, optimizing the key frame by adopting a selective step-by-step marginalization strategy based on a sliding window; and S5, constructing a key frame sequence by using the optimized key frame, carrying out closed-loop detection based on the key frame sequence, and then carrying out drift error correction on the detected closed loop.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of mobile robot autonomous positioning and navigation, and relates to a dynamic interference resistant slam loop detection method based on FPN-SuperPoint and SuperGlue fusion, and also relates to a corresponding device and medium. BACKGROUND

[0002] With the rapid development of artificial intelligence and robot technology, intelligent unmanned systems have gradually moved from closed laboratory environments to open and complex real-world environments. SLAM technology, as the cornerstone of robot autonomous movement, path planning and environment perception, is self-evident in its importance. Traditional SLAM technology mainly relies on LiDAR or camera sensors. Visual SLAM has received widespread attention from academia and industry due to its low hardware cost, ability to obtain rich texture semantic information, and small size and low power consumption. Visual-inertial SLAM (VI-SLAM) that fuses visual sensors with IMU can effectively solve the scale uncertainty problem of monocular vision by utilizing high-frequency inertial measurement data from IMU, and significantly improves the robustness of the system under extreme conditions such as fast motion and texture loss.

[0003] Although existing mainstream VI-SLAM systems have shown excellent performance in static environments, such as VINS-Mono, VINS-Fusion, ORB-SLAM3, and MSCKF, they still have the following technical problems when facing complex dynamic scenes commonly found in the real world:

[0004] 1. Limitations of traditional feature extraction algorithms; the quality of front-end feature extraction directly determines the upper limit of the accuracy of the SLAM system. Although traditional feature extraction algorithms such as Harris, Shi-Tomasi, and FAST have fast computation speed, they have poor repeatability and localization accuracy when facing scale changes, angle rotations, and lighting changes. Although deep learning-based feature extraction methods such as SuperPoint perform well in textureless areas, the encoder design in the original network architecture has limitations. The original SuperPoint usually only uses a single level of feature maps for decoding, lacks a multi-scale feature fusion mechanism, and when the object scale changes dramatically, the feature point detection ability will decrease significantly. Moreover, the output feature point coordinates are usually integer pixel level, lacking sub-pixel level precision, which limits the upper limit of the accuracy of the back-end optimization.

[0005] 2. Localization drift caused by dynamic feature interference. Traditional SLAM algorithms are usually based on the "static world assumption," which assumes that most objects in the environment are stationary. However, in real-world applications (such as busy streets, crowded shopping malls, and offices), there are often a large number of moving objects such as pedestrians and vehicles. When the camera's field of view is filled with dynamic objects, the feature points extracted by visual odometry (VO) will include a large number of "outliers." These dynamic feature points do not satisfy epipolar geometry constraints. If they are mistakenly identified as static landmarks and included in the pose calculation, incorrect observation constraints will be generated, causing the Kalman filter state divergence or the solution of nonlinear optimization to deviate from the true value, ultimately leading to severe trajectory drift or even tracking loss.

[0006] Existing solutions such as Dyna-SLAM attempt to incorporate deep learning networks for semantic segmentation to eliminate dynamic regions. However, relying solely on object detection networks still has the following limitations: ① Missed detections and false detections: When objects are blurred, occluded, or there are drastic changes in lighting, the network is prone to missing dynamic objects as static or misclassifying static backgrounds as dynamic; ② Feature scarcity: If all potentially dynamic categories such as "people" are simply and crudely eliminated, the number of available static feature points will decrease sharply in densely populated areas, leading to system failure due to insufficient constraints.

[0007] 3. Computational efficiency bottleneck of backend optimization. To ensure global consistency, existing VI-SLAM systems typically employ a nonlinear optimization backend based on a sliding window. As the system runs, new image frames are continuously added to the window, and older frames need to be removed. To preserve the historical information constraints contained in the older frames, a marginalization strategy must be used to transform the information of the older frames into priors.

[0008] However, the marginalization process involves the Schur complement operation on the Hessian matrix; the dimension of the Hessian matrix is ​​determined by the number of state variables, and the matrix dimension expands rapidly as the number of landmarks increases; the time complexity of the Schur complement operation is typically O(n log n). On embedded platforms with limited computing resources, frequent and large-scale edge operations consume a significant amount of CPU time, causing the front-end tracing thread to block and impacting the system's real-time performance. While existing systems such as VINS-Mono employ certain strategies, they typically perform overall operations on a large matrix containing all landmarks and poses, failing to fully utilize the sparse structure of the matrix and thus limiting their efficiency improvements.

[0009] 4. Loop closure detection mismatch and inefficiency. Loop closure is a key link to eliminate cumulative drift, and the traditional method is mostly based on the Bag-of-Words (BoW) model, which converts the current frame into a bag-of-words vector and matches it with the historical key frame.

[0010] But as the running track is extended, the size of the key frame database gradually increases, and the frame-by-frame retrieval efficiency is low; more seriously, in similar corridors, repeated structures and texture similar environments, the matching based on single frame appearance is prone to "perceptual aliasing", that is, different places are mistakenly identified as the same place, which leads to loop optimization error and completely destroys the consistency of the map. SUMMARY

[0011] The purpose of the present application is to provide an anti-dynamic interference Slam loop detection method based on FPN-SuperPoint and SuperGlue fusion, and corresponding equipment and medium.

[0012] Based on the above purpose, the present application adopts the following technical scheme:

[0013] In a first aspect, the present application provides an anti-dynamic interference Slam loop detection method based on FPN-SuperPoint and SuperGlue fusion, comprising the following steps:

[0014] S1, acquire camera image sequence data, input FPN-SuperPoint feature extraction network, output feature points of image frame and corresponding feature descriptors; input the image sequence after feature extraction into the SuperGlue feature matching network to obtain initial feature matching pairs;

[0015] S2, for each feature point contained in the initial feature matching pairs output by S1, introduce a semantic-probability coupled adaptive weighted sub-pixel optimization strategy combined with feature point probability distribution information, to obtain sub-pixel level high-precision feature points, and update the initial feature matching pairs based on these optimized feature points;

[0016] S3, remove the matching pairs corresponding to the feature points in the dynamic area by target detection and improved Kalman filtering to obtain static feature matching pairs;

[0017] S4, for the static feature matching pairs output by S3, adopt a selective step-by-step marginalization strategy based on a sliding window to optimize the key frame;

[0018] S5, construct a key frame sequence using the optimized key frame, perform loop closure detection based on the key frame sequence, and then correct the drift error of the detected loop.

[0019] The FPN-SuperPoint feature extraction network includes three consecutive downsampling operation modules. Each downsampling operation module contains two cascaded CR modules and a max pooling layer, which can be represented by the function as follows: , where the function Represents a CR module, functions This represents a max pooling layer; the feature map output by the third downsampling operation module. ; Through two CR modules (represented by functions as follows) The processed output is a feature map. , The mathematical space representing the input image information Image is defined as follows: This represents the channel dimension of a single layer of the encoder; channel splicing operations are used to... and Merging them together yields the spliced ​​intermediate feature map. , Introducing convolutional layers right Perform dimensionality reduction processing and output image This serves as a shared input for subsequent use.

[0020] In step S1, the specific steps for feature matching based on the SuperGlue feature matching network are as follows:

[0021] The output image The feature point detection head and feature descriptor head are input to the decoding end, respectively. The detection head predicts the feature point positions and confidence levels, and the descriptor head calculates the feature descriptor vector. The feature point positions, confidence levels, and feature descriptor vectors are combined to construct the feature set of a single frame image. The above steps are performed on the reference frame image to be matched and the current frame image, respectively, to obtain two feature sets. The two feature sets are input into the SuperGlue network, which uses an attention-based graph neural network to perform cross-graph context aggregation and solves the optimal transmission problem using the Sinkhorn algorithm to obtain the soft allocation matrix. Extracting the soft allocation matrix Top left corner The submatrix serves as the final matching probability matrix, where the first submatrix is ​​the first matching probability matrix. Line 1 The elements of the column are denoted as matching probabilities. .

[0022] If probability If the following conditions are met, then the first element in the reference frame feature set is determined. The nth feature point and the nth feature point in the current frame feature set Each feature point is a valid matching pair: wherein, is the maximum value of all column elements in the th row of the soft assignment matrix; is the maximum value of all row elements in the th column of the soft assignment matrix; is a preset confidence threshold.

[0023] In step S2, the semantic-probability coupled adaptive weighting subpixel optimization strategy is specifically:

[0024] Constructing semantic weight: based on the FPN-SuperPoint network output feature point position and the FPN-SuperPoint feature extraction network output a dense "corner probability map" , define the reliability weight of each pixel : wherein, is the position of the th pixel, is the "corner probability value" corresponding to the pixel from the "corner probability map ", and represents the gradient of the input image at the pixel position ;

[0025] Constructing a weighted objective function to obtain the final solution form: wherein, is the square of the scale factor of the th layer of the feature pyramid; is the transpose matrix of the matrix ;

[0026] The matrix , , is defined as: . Wherein, represents the Jacobian matrix of the residual vector composed of the visual reprojection error and the IMU pre-integral error with respect to the camera pose and the road marker point coordinates at the th iteration; represents the reliability weight of the pixel in the th layer feature map.

[0027] In step S3, before removing the feature points falling in the dynamic area, dynamic determination is performed based on target detection and improved Kalman filtering, and the steps are as follows:

[0028] S31, symbol definition: define the th frame System state vector of a tracked target is a vector containing its geometric properties and its first-order derivatives: wherein, represents the center coordinates of the target bounding box, is the aspect ratio of the target bounding box, is the height of the target bounding box, is the change speed of each state respectively.

[0029] S32, state prediction: a Kalman filter is constructed for each dynamic target detection box, a state transition equation is established based on the uniform motion model, and the state vector and covariance matrix of the dynamic target detection box in the first frame are predicted to obtain the prior state estimation and the prior covariance matrix of the dynamic target detection box in the second frame;

[0030] Combined with the detection box confidence and the region tracking quality score, an adaptive measurement noise covariance matrix is constructed ; let the observation vector of the second frame be , and the corresponding measurement noise covariance matrix is dynamically adjusted: wherein, is the basic measurement noise covariance matrix; is the confidence score of the current detection box; is the region tracking quality score of the detection box; is the adjustment coefficient; The region tracking quality score

[0031] is specifically: wherein, is an indicator function; is an effective matching threshold; is a small amount to prevent the denominator from being zero; is the set of feature points falling within the detection box ; is the feature point index in the set ; is the matching probability between the feature point and its corresponding matching point in the feature matching pair;

[0032] S33, hierarchical data association: using the Hungarian matching algorithm, first construct a primary cost matrix to determine the optimal matching between the high-confidence detection box and the predicted box , and then construct a secondary cost matrix to determine the optimal matching between the low-confidence detection box ​​​optimal match between the unmatched detection frame and the prediction frame ;

[0033] the detection frame successfully matched and the prediction frame , check the value between them , if greater than a set threshold, it is considered to be successfully matched, and further fusion update is performed; the prediction frame that fails to be successfully matched is retained and discarded; if it still does not exist in the frame , it is discarded;

[0034] S34, Kalman filter fusion update: for the detection frame successfully matched in S33 and the prediction frame , the observation vector is , the prior state estimation and the prior covariance matrix calculated in S32 , the adaptive measurement noise covariance matrix is calculated, the observation residual is fused using the Kalman gain, and the prior state estimation and the prior covariance matrix are updated to obtain the posterior state estimation and the posterior covariance matrix ;

[0035] S35, hypothetical tracking mechanism: after the Kalman filter fusion update, the front end does not process the frame in a hurry, and the frame result is put into a buffer queue with a size greater than or equal to ; for the unmatched detection frame, a "hypothetical tracker" is established; if the hypothetical tracker continuously matches successfully in the subsequent, its state is upgraded to "confirmed"; if the hypothetical tracker fails to match too many times in the subsequent, it is considered to be a false detection.

[0036] In step S4, the selective step-by-step marginalization strategy is specifically:

[0037] S41, the state to be marginalized is subdivided into a sparse feature group and a dense pose group, and a blockwise Hessian matrix containing the sparse feature group, the dense pose group and the remaining state variables is constructed;

[0038] S42, the sparse feature group is marginalized, and a Schur complement operation is used to eliminate its corresponding rows and columns to obtain an intermediate information matrix about the dense pose group and the remaining state;

[0039] S43, based on the intermediate information matrix, the dense pose group is further marginalized to obtain a final prior information matrix about only the remaining state variables and the corresponding residual vector ;

[0040] S44. Based on the final prior information matrix With the final residual vector Construct information about the remaining state variables The linear prior constraint equations are: This equation transforms marginalized historical information into constraints on the remaining state, which then participate as prior terms in the sliding window optimization at the next time step.

[0041] In step S5, the loop closure detection optimization strategy based on the keyframe sequence is as follows:

[0042] S51. Define the keyframe sequence: Keyframe sequence From the current keyframe and its temporally continuous preceding Composed of keyframes: in, For the first in the sequence One keyframe; The latest keyframe; The earliest keyframe in the sequence;

[0043] S52. Sequence Similarity Measurement and Matching Strategy: A weighted fusion strategy is used to construct sequence descriptors, and the similarity score between the current keyframe sequence descriptor and historical candidate sequence descriptors is calculated. A coarse matching is used to filter out the candidate sequence set; then a fine matching is performed for each historical sequence in the candidate historical sequence set. Calculate the similarity matrix between its internal single frames and the current sequence single frames. Determine the best single-frame matching pair. ;

[0044] S53. Temporal continuity test: For the optimal single-frame matching pair Perform a time continuity test. If the matching pair passes the continuity test, then perform closed-loop optimization.

[0045] Secondly, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any step in the described anti-dynamic interference SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue.

[0046] In a third aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program, when executed by a computer processor, implements any of the steps of the FPN-SuperPoint and SuperGlue fusion-based anti-dynamic interference Slam loop detection method.

[0047] Compared with the prior art, the present application has the following advantages:

[0048] 1. The present application overcomes the positioning drift and computational bottleneck problems of traditional algorithms in complex dynamic scenes by using front-end high-robustness feature perception, dynamic interference precise removal, back-end high-efficiency edge optimization, and serialized loop detection techniques, and achieves high-precision and high-reliability state estimation in extreme environments with complex texture and dramatic scale changes, thereby improving the robustness and environmental adaptability of the SLAM system. The present application is suitable for long-term stable operation of intelligent agents such as unmanned vehicles, unmanned aerial vehicles, and service robots in actual engineering deployment, and has high practical value and broad landing prospects.

[0049] 2. In view of the problems of low feature repetition rate, easy mismatch, and limited positioning accuracy to integer pixels of existing visual front-end algorithms in dramatic scale changes and weak texture scenes, the present application constructs a fusion architecture of FPN-SuperPoint feature extraction network and SuperGlue feature matching network. First, the feature pyramid (FPN) multi-scale fusion mechanism is introduced by improving the shared encoder of the SuperPoint network, effectively solving the problem of decreased detection capability of the original SuperPoint in dramatic scale changes of objects. Then, the SuperGlue network is innovatively combined, the attention mechanism of graph neural network (GNN) is used to aggregate global spatio-temporal context information, and the traditional nearest neighbor distance metric is replaced by solving the differentiable optimal transport problem, achieving high-confidence feature association far beyond traditional algorithms in view of dramatic changes in viewpoints and dynamic occlusion scenes.

[0050] Finally, the present application introduces a semantic-probability coupled adaptive weighted sub-pixel optimization strategy, uses the corner probability map to construct a weighted constraint equation, breaks through the quantization limit of the sensor pixel grid, and realizes the full-link precision leap from feature extraction, robust matching to sub-pixel level accurate positioning.

[0051] 3. In view of the problem that traditional algorithms are prone to dynamic feature misidentification and removal, leading to feature deficiency in dynamic scenes, the present application proposes a dynamic feature removal strategy based on target detection combined with improved Kalman filtering.

[0052] In view of the problems of large amount of calculation and poor real-time performance of dynamic object data association, the application innovatively introduces a one-time and two-time hierarchical cost matrix for multi-level data association, preferentially matches high confidence detection boxes, and then secondarily matches low confidence detection boxes; this hierarchical strategy can greatly prune the full Hungarian matching problem with a complexity of , reduce the CPU occupancy rate of the front-end data association module by about 40% under the premise of ensuring high dynamic rejection rate, and meet the real-time requirements of mobile robots in complex dynamic scenes.

[0053] Meanwhile, through the hypothetical tracking mechanism, the situation that static feature points are wrongly rejected due to wall newspaper, static sculptures and the like being misjudged as dynamic objects is effectively prevented.

[0054] 4. In view of the problem that the Schur complement operation in the rear-end sliding window optimization leads to efficiency bottleneck with the increase of road markers, the application proposes a rear-end selective step-by-step marginalization strategy, which decouples the to-be-marginalized state into a sparse feature group and a dense pose group by explicitly, and performs Schur complement operation in steps by using the block diagonal matrix characteristics formed by the conditional independence between road marker points; this strategy can reduce the time complexity of the marginalization feature points from to , so that the system can still maintain long-term stable operation of a large number of road markers on embedded devices with limited computing resources.

[0055] 5. In view of the problem that the calculation efficiency is low when matching historical frames one by one in the traditional loop detection process due to the growth of the map size, the application designs a loop detection optimization strategy based on key frame sequence; in terms of search efficiency, the historical sequence is pre-screened by using sequence descriptors to quickly exclude most irrelevant historical sequences, which significantly reduces the calculation resource consumption and search time consumption of the loop detection module; in terms of detection accuracy, after single-frame matching in the best matching sequence, time continuity test is introduced, which improves the loop detection speed while ensuring the accuracy of loop detection, and effectively improves the overall performance of the system. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 it is the flowchart of the application in embodiment 1; Figure 2 it is the structure diagram of FPN-SuperPoint feature extraction network in embodiment 1, which shows the specific network layer connection mode of feature extraction, up-sampling, splicing and fusion of module 3 and module 4 feature maps; Figure 3 it is the structure diagram of FPN-SuperPoint combined with SuperGlue in embodiment 1; Figure 4 it is the flowchart of the loop detection algorithm based on key frame sequence in embodiment 1, including sequence construction, bag-of-words matching, similarity score and continuity test judgment logic; Figure 5This is a visual comparison of the trajectory of the loop scene on the same road segment in Example 1 between the method of the present invention and VINS-Fusion. Detailed Implementation

[0057] The present invention will be further described in detail below through specific embodiments, but this does not limit the scope of the present invention.

[0058] Example 1

[0059] The overall process of the dynamic interference-resistant SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue is as follows: Figure 1 As shown, it includes the following steps:

[0060] S1. Acquire camera image sequence data, input it into the FPN-SuperPoint feature extraction network, and output the feature points of the image frames and their corresponding feature descriptors; input the image sequence after feature extraction into the SuperGlue feature matching network to obtain the initial feature matching pairs.

[0061] The FPN-SuperPoint feature extraction network structure is as follows: Figure 2 As shown, it includes three consecutive downsampling operation modules. Each downsampling operation module contains two cascaded CR modules and a max pooling layer, represented by the function as follows: , where the function Represents a CR module, functions This represents a max pooling layer; the feature map output by the third downsampling operation module. ; Through two CR modules (represented by functions as follows) The processed output is a feature map. , Represents the input image information Image (the function variable used earlier). The mathematical space of the general input parameters (specifically image information) is defined as follows: In this embodiment, the encoder has a single-layer channel dimension. Channel splicing operation is used to... and Merging them together yields the spliced ​​intermediate feature map. , ,at this time, The channel dimension is expanded to (i.e., 128+128); introduce convolutional layers right Perform dimensionality reduction processing and output image , This serves as a shared input for subsequent use. The channel dimension is 128.

[0062] Furthermore, feature matching is performed based on the SuperGlue feature matching network, such as... Figure 3 As shown, in this embodiment, feature matching no longer relies on independent feature point descriptor distance metrics, but instead achieves global context awareness and optimal transmission matching of feature points by constructing a Dual-Graph Attention Interaction Network. The specific steps are as follows:

[0063] S11, Output image The feature point detection head and feature descriptor head are respectively input to the decoding end. The feature point position and confidence are predicted by the detection head, and the feature descriptor vector is calculated by the descriptor head. The feature point position, confidence and feature descriptor vector are combined to construct the feature set of a single frame image.

[0064] S12. Perform the above steps for the reference frame image (Image A) to be matched and the current frame image (Image B) respectively to obtain two sets of features:

[0065] Graph node embedding and initialization, defining the current frame image as... The reference frame image is Two feature sets were obtained based on the FPN-SuperPoint feature extraction network. Expressed as: ;in, For the current frame, the first The pixel coordinates of each feature point Its corresponding Multi-scale fusion descriptor. Feature set. Corresponding reference frame 1 feature point.

[0066] Feature set Expressed as: ;in, For the reference frame The pixel coordinates of each feature point Its corresponding Multi-scale fusion descriptor; feature set Corresponding reference frame 1 feature point.

[0067] Construct a high-dimensional node state vector to fuse geometric distribution and visual semantic information, and define a position encoding function. Feature set Initial node state is expressed as: feature set initial node state is expressed as: wherein, is a learnable projection matrix for position encoding.

[0068] S13, input two sets of feature sets into the SuperGlue network, use the graph neural network based on the attention mechanism to aggregate the cross-graph context, and solve the optimal transport problem through the Sinkhorn algorithm to obtain a soft assignment matrix :

[0069] S131, cross-graph context aggregation, construct a fully connected graph structure containing two kinds of edge attributes: intra-frame edge and inter-frame interaction edge ; adopt layer graph neural network for message passing; set the node state of the th layer as , and the state update of the th layer follows the residual aggregation mechanism; wherein, is the node state vector of the corresponding feature point in the th layer; is the node state vector of the corresponding feature point in the th layer; is the aggregated message vector, which is generated by the multi-head attention mechanism.

[0070] When calculating the inter-frame interaction, define the query vector , the key vector and the value vector : wherein, is the query vector of the feature point ; is the query projection weight matrix of the th layer; is the key vector of the feature point ; is the key projection weight matrix of the th layer; is the node state vector of the feature point in the th layer; is the value vector of the feature point ; is the value projection weight matrix of the Layer-wise value projection weight matrix.

[0071] Obtain interaction weight coefficient : where, is a normalized exponential function along the dimension where the feature point is located; is the transpose of the query vector ; is the key vector of the feature point ; is the dimension of the key vector, a regularization factor for scaling the magnitude of the dot product.

[0072] Aggregate messages as a weighted sum of value vectors:

[0073] Network layers alternate performing intra-frame aggregation (Self) and inter-frame aggregation (Cross), finally output enhanced matching features and .

[0074] S132, differentiable optimal transport matching, construct similarity score matrix : where element represents the association strength of the feature point and ; is the transpose of the final matching descriptor vector of the feature point in the current frame; is the final matching descriptor vector of the feature point in the reference frame.

[0075] Introduce slack node mechanism to handle unmatched points caused by changes in field of view and dynamic object occlusion; extend the score matrix to : where, is a learnable outlier penalty parameter; is a global background threshold; is the transpose of the reference frame outlier penalty parameter vector .

[0076] Use Sinkhorn iterative algorithm to solve the optimal transport scheme; define the allocation matrix in the logarithmic domain , perform times of row and column normalization iteration: After iterative convergence, obtain the soft allocation matrix .

[0077] S14. Matching Pair Filtering: Extracting the Soft Assignment Matrix Top left corner The submatrix serves as the final matching probability matrix, where the first submatrix is ​​the first matching probability matrix. Line 1 The elements of the column are denoted as matching probabilities. .

[0078] If probability If the following conditions are met, then the first element in the reference frame feature set is determined. The nth feature point and the nth feature point in the current frame feature set Each feature point is a valid matching pair: in, For the soft allocation matrix, the first The maximum value of all columns in a row; For the soft allocation matrix, the first The maximum value of all row elements in the column; The preset confidence threshold is 0.2 in this embodiment.

[0079] S2. For each feature point in the initial feature matching pair output by S1, combined with the feature point probability distribution information, an adaptive weighted sub-pixel optimization strategy with semantic-probabilistic coupling is introduced to obtain sub-pixel-level high-precision feature points, and the initial feature matching pair is updated based on these optimized feature points. Traditional sub-pixel optimization relies only on grayscale gradients, which are prone to generating outliers when the texture is weak or the illumination changes. This invention innovatively constructs a weighted constraint equation based on the multi-scale feature response of FPN; and introduces the corner probability map of the SuperPoint network. Constructing the weight matrix This allows the optimization process to intelligently focus on high-confidence semantic feature points, thereby breaking through the dependence of traditional geometric methods on environmental texture.

[0080] Specifically, the semantic-probabilistic coupled adaptive weighted sub-pixel optimization strategy is as follows:

[0081] Constructing semantic weights: Based on the feature point locations output by the FPN-SuperPoint network and a dense "corner probability map" output by the FPN-SuperPoint feature extraction network. Define each pixel Reliability weight : in, For the first The position of each pixel To obtain from the "corner probability map" Extracting pixels from The corresponding "corner probability value", Indicates the input image the gradient at the pixel position ;

[0082] The weighted objective function is constructed as: where, is a scale factor; is the gradient of the feature map; is the reliability weight of the pixel in the feature map of the th layer; is the transpose of the pixel coordinate vector; is the set of neighborhood pixels of the initial feature point on the feature map of the th layer.

[0083] The final solution form is obtained as: where, is the square of the scale factor of the th layer of the feature pyramid; is the transpose matrix of the matrix ;

[0084] The matrix , , is defined as: where, represents the Jacobian matrix of the residual vector composed of the visual re-projection error and the IMU pre-integral error with respect to the camera pose and the road marker point coordinates at the th iteration.

[0085] S3, by target detection and improved Kalman filtering, the matching pairs corresponding to the feature points falling in the dynamic area are removed to obtain static feature matching pairs.

[0086] Before removing the feature points falling in the dynamic area, dynamicity determination is performed.

[0087] The specific steps of dynamicity determination based on target detection and improved Kalman filtering are as follows:

[0088] S31, symbol definition: the detection box of the dynamic target (such as pedestrian, vehicle) in the image is described by the center coordinates , the aspect ratio , and the height ; the system state vector of the th tracked target in the th frame is defined as an 8-dimensional vector containing its geometric properties and its first-order derivative (change rate): where, represents the center coordinates of the target box, is the aspect ratio of the target box, The height of the target bounding box. These represent the rates of change for each of the aforementioned states.

[0089] S32, State Prediction:

[0090] For each dynamic target detection box, a Kalman filter is constructed, and a state transition equation is established based on a uniform motion model. For the ... The state vector and covariance matrix of the dynamic object detection bounding box in the frame are used to predict the result. Prior state estimation of frames and prior covariance matrix :

[0091] Assuming an extremely short inter-frame time interval Within the image plane, the motion of the target follows a uniform velocity model; using the first... Posterior state estimation of frames , No. Prior state estimation of frames Represented as:

[0092] State transition matrix Defined as: in, It is the identity matrix. The control matrix, in this model, is typically treated as zero or external inputs can be ignored. Input for the system.

[0093] Prior covariance matrix The prediction formula is: in, The process noise covariance matrix is ​​a fixed matrix, representing the inherent uncertainty of the uniform motion model (such as the small acceleration changes of the target). For the first The posterior covariance matrix of the frame; State transition matrix The transpose of .

[0094] To address the uncertainty in YOLO detection, this embodiment combines the detection box confidence score and the region tracking quality score to construct an adaptive measurement noise covariance matrix. ; Introduce the confidence scores of the detection box and SuperGlue; Let the first... The observation vector of the frame is , Direct measurement data from object detection networks (such as YOLOv8), where 'det' is an abbreviation for 'detection', representing the measured value. Indicates the center coordinates of the target box. The aspect ratio of the target bounding box. Target box height. Corresponding measurement noise covariance matrix Dynamic adjustment: Wherein, is the basic measurement noise covariance matrix, indicating the inherent measurement error of the sensor under ideal detection conditions (such as clear target, no occlusion); is the confidence score of the current detection box, the value range ; is the area tracking quality score of the detection box; is the adjustment coefficient, in the embodiment , used to control the sensitivity of confidence to measurement noise amplification;

[0095] The area tracking quality score Specifically: Wherein, is an indicator function; is an effective matching threshold; is a small amount to prevent the denominator from being zero; suppose that in the current frame, the first detection box output by YOLOv8 is , is the set of feature points falling within the detection box ; is the feature point index in the set ; is the matching probability between the feature point and its corresponding matching point in the feature matching pair;

[0096] S33, hierarchical data association: using the Hungarian matching algorithm, first construct a cost matrix to determine the optimal matching between high-confidence detection boxes and predicted boxes , and then construct a secondary cost matrix to determine the optimal matching between low-confidence detection boxes and unmatched predicted boxes .

[0097] Construct a primary cost matrix and a secondary cost matrix;

[0098] First, divide the detection boxes into high-confidence detection boxes and low-confidence detection boxes according to the confidence (0.5), first calculate the intersection over union between each high-confidence detection box and each predicted box : Further, the high-confidence detection boxes and the predicted boxes are first associated, and a cost matrix is constructed As a cost matrix, where This represents the number of high-confidence detection boxes. Number of prediction boxes: The purpose of constructing the cost matrix is ​​to find the maximum total cost. In the case of values, the data association problem is transformed into an optimal task matching problem with a cost matrix; by analyzing... The row and column elements are processed to determine... and The optimal match between them.

[0099] For predicted boxes that did not match in a single match Utilizing it with low-confidence detection boxes Constructing a quadratic cost matrix First, calculate the confidence level of each low-confidence detection box. With each unmatched prediction box Crossover ratio between: Furthermore, data association is performed between low-confidence detection boxes and unmatched predicted boxes to construct a... Cost matrix , This represents the number of low-confidence detection boxes. The number of remaining prediction boxes: Then, by processing the row and column elements, the determination can be made. and The optimal match between them.

[0100] For successfully matched detection boxes With prediction box Check between them If the value is greater than a set threshold, a match is considered successful, and further fusion and updating are performed; predicted boxes that fail to match are... The predicted frame is retained and not discarded; if in If no match is found within the frame, the frame is discarded.

[0101] S34, Kalman filter fusion update: For the successfully matched detection boxes in S33 With prediction box Using the observation vector as Prior state estimation and the prior covariance matrix calculated in S32 Adaptive measurement noise covariance matrix Calculate the observation residuals and Kalman gain, and use the Kalman gain to fuse the observation residuals to estimate the prior state. and prior covariance matrix Update the state to obtain the posterior state estimate. and posterior covariance matrix ;

[0102] S35. Hypothetical Tracking Mechanism: After fusion and updating via Kalman filtering, the front end does not rush to process the frame; it first needs to put the frame result into a container with a size greater than or equal to... In the buffer queue, the following operations are performed: For unmatched detection boxes, they are not directly marked as dynamic, but a "hypothetical tracker" is established, which is considered to be a dynamic target that has just entered the field of view, and its hypothetical tracking results are also included in the tracking; if the hypothetical tracker continues to match successfully in the next 3 frames, its status is upgraded to "confirmed", and the feature matching pairs of the corresponding regions in the past 3 frames are backtracked and removed; if the hypothetical tracker fails to match too many times in the next 5 frames, it is regarded as a false detection and deleted, but the feature matching pairs of the corresponding regions are not removed; the hypothetical tracking mechanism can effectively prevent posters on the wall, static sculptures, etc. from being misjudged as dynamic objects, and ensure the number of static feature points.

[0103] S4. For the static feature matching pairs output by S3, a selective step-by-step edge-mapping strategy based on a sliding window is used to optimize the keyframes.

[0104] Specifically, the selective step-by-step edge-out strategy based on the sliding window is as follows:

[0105] S41. State variable reorganization and segmentation: The state to be marginalized is subdivided into sparse feature groups and dense pose groups, and a segmented Hessian matrix containing sparse feature groups, dense pose groups and remaining state variables is constructed.

[0106] The state to be marginalized Further subdivided into sparse feature groups and dense pose groups The sparse feature set Including the inverse depth of all landmarks observed in the oldest frame And the speed of the oldest frame and zero bias The dense pose group Including the pose of the oldest frame .

[0107] Therefore, the normal equations are rearranged into a three-part structure: in, For each sparse feature group, there is a Hessian matrix block. Since the landmark points are conditionally independent, it is a block diagonal matrix. This represents the cross-correlation Hessian matrix block between the dense pose group and the sparse feature group; The Hessian matrix block representing the cross-correlation between the remaining state variables and the sparse feature set; Hessian matrix blocks for the cross correlation between the sparse feature group and the dense pose group; Hessian matrix blocks for the dense pose group itself; Hessian matrix blocks for the cross correlation between the remaining state variables and the dense pose group; Hessian matrix blocks for the cross correlation between the sparse feature group and the remaining state variables; Hessian matrix blocks for the cross correlation between the dense pose group and the remaining state variables; Hessian matrix blocks for the remaining state variables themselves; the remaining state variables that do not need to be marginalized and are not the main optimization objects this time; the residual error vector corresponding to the sparse feature group; the residual error vector corresponding to the dense pose group; the residual error vector corresponding to the remaining state variables.

[0108] The key property is that (corresponding to the feature point part) is a block diagonal matrix, because the observations between different landmark points are independent of each other given the camera pose.

[0109] S42, sparse feature marginalization: first marginalize the sparse feature group ; eliminate its corresponding rows and columns by using the Schur complement operation to obtain the intermediate information matrix about the dense pose group and the remaining state;

[0110] S43, dense pose marginalization: based on the intermediate information matrix, marginalize the dense pose group ; eliminate its corresponding rows and columns by using the Schur complement operation to obtain the final prior information matrix about the remaining state variables and the corresponding residual error vector ;

[0111] S44, based on the final prior information matrix and the final residual error vector , construct the linear prior constraint equation about the remaining state variables : This equation converts the marginalized historical information into a constraint on the remaining state as a prior term to participate in the sliding window optimization at the next time.

[0112] S5, form a key frame sequence by optimizing the key frame group in step S4, perform loop closure detection based on the key frame sequence, and then perform drift error correction on the detected loop closure, as shown in Figure 4 .

[0113] In step S5, the loop detection optimization strategy of the key frame sequence is:

[0114] S51, defining a key frame sequence: defining a key frame sequence with a length of , the key frame sequence consisting of the current key frame and its previous key frames in time: wherein is the key frame in the sequence; is the current latest key frame; is the earliest key frame in time in the sequence.

[0115] S52, sequence similarity measurement and matching strategy:

[0116] First, rough matching is performed, a weighted fusion strategy is adopted to construct a sequence descriptor, and the similarity score between the current key frame sequence descriptor and the historical candidate sequence descriptor is calculated, and a candidate sequence set is selected through rough matching;

[0117] Defining a sequence descriptor: constructing a sequence descriptor for representing the semantic and texture information of the entire sequence ; a weighted fusion strategy is adopted: wherein is a single frame TF-IDF (Term Frequency-Inverse Document Frequency) bag of words vector, is the key frame, is a time decay weight, usually set to to treat all frames within the sequence equally, or to adaptively adjust according to the amount of information; this operation extends the observation at a single time point to a trajectory segment description on a spacetime manifold.

[0118] The normalized norm is adopted as the distance measurement, and the score is defined as: wherein is the current key frame sequence, is the historical candidate key frame sequence, is the descriptor vector of the current key frame sequence; is the descriptor vector of the historical key frame sequence; is the norm of the current sequence descriptor vector; is the norm of the historical sequence descriptor vector.

[0119] A threshold is set, and a candidate historical sequence set with a score is selected​ This step quickly eliminates most obvious false matches using the rich information of the sequence.

[0120] Further fine matching is performed for each historical sequence in the candidate set , and a similarity matrix between the internal single frames and the single frames of the current sequence is further calculated , where represents the similarity between the th frame in the historical sequence and the th frame in the current sequence. A diagonal line path with the highest score is obtained in the matrix , and the diagonal line path represents the best alignment of the two sequences in the time flow direction; if the alignment score is significantly higher than other candidates, the historical sequence

[0121] is selected as the best matching sequence, and the best single frame matching pair in the historical sequence is determined.

[0122] S53, time continuity test: assuming that the candidate loop is detected at time ; if the loop is real, at the subsequent time , the loop should be detected within the neighborhood of ; a sliding window is constructed, and only when consecutive key frames (3-5 frames) are independently matched to a continuous trajectory in history, the time continuity test is passed; the matching that passes the continuity test is subjected to loop optimization.

[0123] As shown in the figure, the embodiment method and VINS-Fusion have the same track visualization comparison of the loop scene of the same section, the blue solid line in the figure is the track of the embodiment, and the black dashed line is the track of VINS-Fusion. The error between the length of the track obtained by the embodiment method and the actual measured track length is less than the error between the length of the track obtained by VINS-Fusion and the actual measured track length. Therefore, the embodiment method has higher positioning accuracy than VINS-Fusion, which proves the performance and robustness of the system. Figure 5 Embodiment 2

[0124] An electronic device includes a memory and a processor, the memory has a computer program stored thereon, and the processor implements any step in the anti-dynamic interference Slam loop detection method based on FPN-SuperPoint and SuperGlue fusion as described in embodiment 1 when executing the computer program.

[0125] ​​​

[0126] Further, the anti-dynamic interference Slam loop detection method based on FPN-SuperPoint and SuperGlue fusion described in embodiment 1 can be implemented as a computer software program. For example, the present embodiment includes a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the method. In such an embodiment, the computer program can be downloaded and installed from a network, and / or installed from a removable medium. When the computer program is executed by a processor, the above-mentioned functions defined in the method of the present application are performed.

[0127] Embodiment 3

[0128] A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements any of the steps of the anti-dynamic interference Slam loop detection method based on FPN-SuperPoint and SuperGlue fusion as described in embodiment 1.

[0129] The computer readable medium described in the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium that can send, propagate or transfer a program for use by or in connection with an instruction execution system, device or apparatus. The program code contained on the computer readable medium can be transmitted using any suitable medium, including but not limited to wireless, wire line, optical cable, RF, or any suitable combination of the above.

[0130] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as python, C++ or conventional procedural programming languages, or the like. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0131] The computer readable storage medium of the embodiment can be accelerated by hardware such as GPU, and the parallel computing advantage of GPU is used to accelerate the processing of any step in the implementation of the anti-dynamic interference Slam loop detection method based on FPN-SuperPoint and SuperGlue fusion as described in Embodiment 1.

[0132] In summary, the present application effectively overcomes the deficiencies in the prior art, and has high industrial utilization value. The above embodiments are intended to illustrate the essential content of the present application, but do not limit the protection scope of the present application. Those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the essence and protection scope of the technical solutions of the present application.

Claims

1. A dynamic interference-resistant SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue, characterized in that, Includes the following steps: S1. Acquire camera image sequence data, input it into the FPN-SuperPoint feature extraction network, and output the feature points of the image frames and their corresponding feature descriptors; input the feature-extracted image sequence into the SuperGlue feature matching network to obtain initial feature matching pairs; S2. For each feature point contained in the initial feature matching pair output by S1, combined with the feature point probability distribution information, an adaptive weighted sub-pixel optimization strategy of semantic-probabilistic coupling is introduced to obtain sub-pixel-level high-precision feature points, and the initial feature matching pair is updated based on these optimized feature points. S3. By using target detection and improved Kalman filtering, matching pairs corresponding to feature points falling within the dynamic region are eliminated to obtain static feature matching pairs; S4. For the static feature matching pairs output by S3, a selective step-by-step edge-mapping strategy based on a sliding window is used to optimize the keyframes. S5. Construct a keyframe sequence using the optimized keyframes, perform loop closure detection based on the keyframe sequence, and then correct the drift error of the detected loops.

2. The anti-dynamic interference SLAM loop closure detection method according to claim 1, characterized in that, The FPN-SuperPoint feature extraction network includes three consecutive downsampling operation modules, each of which contains two cascaded CR modules and a max pooling layer; the feature map output by the third downsampling module... ; The output after processing by two CR modules is a feature map. Channel splicing operation is used to... and Merging them together yields the spliced ​​intermediate feature map. Introducing convolutional layers right Perform dimensionality reduction processing and output image This serves as a shared input for subsequent use.

3. The anti-dynamic interference SLAM loop closure detection method according to claim 1, characterized in that, In step S1, the specific steps for feature matching based on the SuperGlue feature matching network are as follows: The output image The feature point detection head and feature descriptor head are input to the decoding end, respectively. The detection head predicts the feature point positions and confidence levels, and the descriptor head calculates the feature descriptor vector. The feature point positions, confidence levels, and feature descriptor vectors are combined to construct the feature set of a single frame image. The above steps are performed on the reference frame image to be matched and the current frame image, respectively, to obtain two feature sets. The two feature sets are input into the SuperGlue network, which uses an attention-based graph neural network to perform cross-graph context aggregation and solves the optimal transmission problem using the Sinkhorn algorithm to obtain the soft allocation matrix. Extracting the soft allocation matrix Top left corner The submatrix serves as the final matching probability matrix, where the first submatrix is ​​the first matching probability matrix. Line 1 The elements of the column are denoted as matching probabilities. , If probability If the following conditions are met, then the first element in the reference frame feature set is determined. The nth feature point and the nth feature point in the current frame feature set Each feature point is a valid matching pair: in, For the soft allocation matrix, the first The maximum value of all columns in a row; For the soft allocation matrix, the first The maximum value of all row elements in the column; This is the preset confidence threshold.

4. The anti-dynamic interference SLAM loop closure detection method according to claim 1, characterized in that, In step S2, the semantic-probabilistic coupled adaptive weighted sub-pixel optimization strategy is specifically as follows: Constructing semantic weights: Based on the feature point locations output by the FPN-SuperPoint network and a dense "corner probability map" output by the FPN-SuperPoint feature extraction network. Define each pixel Reliability weight : in, For the first The position of each pixel To obtain from the "corner probability map" Extracting pixels from The corresponding "corner probability value", Indicates the input image At pixel position gradient at; Construct a weighted objective function to obtain the final solution form: in, For the characteristic pyramid number The square of the layer scale scaling factor; For matrix The transpose of the matrix; matrix , , Defined as: ;in, Indicates the first The Jacobian matrix of the residual vector composed of visual reprojection error and IMU pre-integration error at each iteration, relative to the camera pose and landmark coordinates. Indicates the first Pixels in layer feature map Reliability weight.

5. The anti-dynamic interference SLAM loop closure detection method according to claim 1, characterized in that, In step S3, before removing feature points falling within the dynamic region, a dynamic determination is performed based on target detection and improved Kalman filtering. The steps are as follows: S31, Symbol Definition: Definition of the first The first frame The system state vector of the tracked target A vector containing its geometric properties and its first derivative: in, Indicates the center coordinates of the target box. The aspect ratio of the target bounding box. The height of the target bounding box. These represent the rates of change for each of the aforementioned states; S32, State Prediction: For each dynamic target detection box, a Kalman filter is constructed, and a state transition equation is established based on a uniform motion model. For the ... The state vector and covariance matrix of the dynamic object detection bounding box in the frame are used to predict the result. Prior state estimation of frames and prior covariance matrix ; An adaptive measurement noise covariance matrix is ​​constructed by combining the detection box confidence score and the region tracking quality score. Let the first The observation vector of the frame is Its corresponding measurement noise covariance matrix Dynamic adjustment: in, Based on the measurement of noise covariance matrix; The confidence score for the current detection bounding box; The region tracking quality score for the detection box; This is the adjustment coefficient; The regional tracking quality score Specifically: in, For indicator functions; To effectively match the threshold; To prevent tiny quantities with a denominator of zero; To fall into the detection frame The set of feature points inside; For set Feature point index in; For feature points Its corresponding matching point in the feature matching pair The probability of matching between them; S33. Hierarchical Data Association: Using the Hungarian matching algorithm, a cost matrix is ​​first constructed to determine high-confidence detection boxes. With prediction box The optimal match between them is determined, and then a quadratic cost matrix is ​​constructed to determine the low-confidence detection boxes. Unmatched prediction boxes The optimal match between them; For successfully matched detection boxes With prediction box Check between them If the value is greater than a set threshold, a match is considered successful, and further fusion and updating are performed; predicted boxes that fail to match are... The predicted frame is retained and not discarded; if in If no match is found within the frame, the frame is discarded. S34, Kalman filter fusion update: For the successfully matched detection boxes in S33 With prediction box Using the observation vector as Prior state estimation and the prior covariance matrix calculated in S32 Adaptive measurement noise covariance matrix Calculate the observation residuals and Kalman gain, and use the Kalman gain to fuse the observation residuals to estimate the prior state. and prior covariance matrix Update the state to obtain the posterior state estimate. and posterior covariance matrix ; S35. Hypothetical Tracking Mechanism: After fusion and updating via Kalman filtering, the front end does not rush to process the frame, but instead places the frame result into a container with a size greater than or equal to... In the buffer queue; for unmatched detection boxes, a "hypothetical tracker" is created; if the hypothetical tracker continues to match successfully in the future, its status is upgraded to "confirmed"; if the hypothetical tracker fails to match too many times in the future, it is considered a false detection.

6. The SLAM loop closure detection method against dynamic interference according to claim 1, characterized in that, In step S4, the selective step-by-step edge-out strategy specifically includes: S41. Subdivide the state to be marginalized into a sparse feature group and a dense pose group, and construct a block Hessian matrix containing the sparse feature group, the dense pose group and the remaining state variables. S42. Marginalize the sparse feature group and use the Schur complement operation to eliminate its corresponding rows and columns to obtain the intermediate information matrix about the dense pose group and the remaining state. S43. Based on the intermediate information matrix, the dense pose groups are further marginalized to obtain the final prior information matrix only concerning the remaining state variables. and the corresponding residual vector ; S44. Based on the final prior information matrix With the final residual vector Construct information about the remaining state variables The linear prior constraint equations are: This equation transforms marginalized historical information into constraints on the remaining state, which then participate as prior terms in the sliding window optimization at the next time step.

7. The SLAM loop closure detection method against dynamic interference according to claim 1, characterized in that, In step S5, the loop closure detection optimization strategy based on the keyframe sequence is as follows: S51. Define the keyframe sequence: Keyframe sequence From the current keyframe and its temporally continuous preceding Composed of keyframes: in, For the first in the sequence One keyframe; The latest keyframe; The earliest keyframe in the sequence; S52. Sequence Similarity Measurement and Matching Strategy: A weighted fusion strategy is used to construct sequence descriptors, and the similarity score between the current keyframe sequence descriptor and historical candidate sequence descriptors is calculated. A coarse matching is used to filter out the candidate sequence set; then a fine matching is performed for each historical sequence in the candidate historical sequence set. Calculate the similarity matrix between its internal single frames and the current sequence single frames. Determine the best single-frame matching pair. ; S53. Temporal continuity test: For the optimal single-frame matching pair Perform a time continuity test. If the matching pair passes the continuity test, then perform closed-loop optimization.

8. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements any step in the anti-dynamic interference SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer processor, implements any step in the dynamic interference-resistant SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue as described in any of claims 1-7.

Images