Anti-dynamic interference slam loop detection method, device and medium based on fusion of FPN-SuperPoint and SuperGlue

Abstract

The application discloses a dynamic interference-resistant slam loop detection method and device based on FPN-SuperPoint and SuperGlue fusion and a medium, wherein the method comprises the following steps: S1, acquiring camera image sequence data and inputting the FPN-SuperPoint network; further utilizing the SuperGlue network to perform feature matching; S2, introducing a semantic-probability coupled adaptive weighted subpixel optimization strategy to the initial feature matching pairs and the probability distribution information of each feature point, obtaining subpixel-level high-precision feature points, and updating the initial feature matching pairs based on the optimized feature points; S3, removing the matching pairs corresponding to the feature points in the dynamic region through target detection and improved Kalman filtering to obtain static feature matching pairs; S4, adopting a selective step-by-step marginalization strategy based on a sliding window to optimize the key frames for the static feature matching pairs output by S3; S5, constructing a key frame sequence by using the optimized key frames, performing closed loop detection based on the key frame sequence, and then correcting the drift error of the detected closed loop.

Description

Technical Field

[0001] This invention belongs to the field of autonomous localization and navigation technology for mobile robots, and relates to a dynamic interference-resistant SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue, as well as corresponding equipment and media. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, intelligent unmanned systems have gradually moved from closed laboratory environments to the open and complex real world. SLAM technology, as the cornerstone of robots' autonomous movement, path planning, and environmental perception, is of paramount importance. Traditional SLAM technologies mainly rely on LiDAR or visual sensors (Camera). Among these, visual SLAM has received widespread attention from academia and industry due to its advantages such as low hardware cost, ability to acquire rich texture and semantic information, and small size and low power consumption. Visual-Inertial SLAM (VI-SLAM), which fuses visual sensors with IMUs, effectively solves the scale uncertainty problem of monocular vision by utilizing the high-frequency inertial measurement data of the IMU, and significantly improves the robustness of the system under extreme conditions such as rapid movement and texture loss.

[0003] Although existing mainstream VI-SLAM systems have demonstrated excellent performance in static environments, such as VINS-Mono, VINS-Fusion, ORB-SLAM3, and MSCKF, they still face the following technical challenges when dealing with complex dynamic scenes prevalent in the real world:

[0004] 1. Limitations of Traditional Feature Extraction Algorithms: The quality of front-end feature extraction directly determines the upper limit of the accuracy of the SLAM system. While traditional feature extraction algorithms such as Harris, Shi-Tomasi, and FAST offer fast corner extraction, they suffer from poor repeatability and localization accuracy when faced with scale changes such as robot forward and backward motion, viewpoint rotation, and lighting variations. Although deep learning-based feature extraction methods like SuperPoint perform well in single-texture regions, their original network architecture has shortcomings in its encoder design. The original SuperPoint typically uses only a single-level feature map for decoding, lacking a multi-scale feature fusion mechanism. When the object scale changes drastically, the feature point detection capability significantly decreases, and the output feature point coordinates are usually at the integer pixel level, lacking sub-pixel precision, thus limiting the upper limit of back-end optimization accuracy.

[0005] 2. Localization drift caused by dynamic feature interference. Traditional SLAM algorithms are usually based on the "static world assumption," which assumes that most objects in the environment are stationary. However, in real-world applications (such as busy streets, crowded shopping malls, and offices), there are often a large number of moving objects such as pedestrians and vehicles. When the camera's field of view is filled with dynamic objects, the feature points extracted by visual odometry (VO) will include a large number of "outliers." These dynamic feature points do not satisfy epipolar geometry constraints. If they are mistakenly identified as static landmarks and included in the pose calculation, incorrect observation constraints will be generated, causing the Kalman filter state divergence or the solution of nonlinear optimization to deviate from the true value, ultimately leading to severe trajectory drift or even tracking loss.

[0006] Existing solutions such as Dyna-SLAM attempt to incorporate deep learning networks for semantic segmentation to eliminate dynamic regions. However, relying solely on object detection networks still has the following limitations: ① Missed detections and false detections: When objects are blurred, occluded, or there are drastic changes in lighting, the network is prone to missing dynamic objects as static or misclassifying static backgrounds as dynamic; ② Feature scarcity: If all potentially dynamic categories such as "people" are simply and crudely eliminated, the number of available static feature points will decrease sharply in densely populated areas, leading to system failure due to insufficient constraints.

[0007] 3. Computational efficiency bottleneck of backend optimization. To ensure global consistency, existing VI-SLAM systems typically employ a nonlinear optimization backend based on a sliding window. As the system runs, new image frames are continuously added to the window, and older frames need to be removed. To preserve the historical information constraints contained in the older frames, a marginalization strategy must be used to transform the information of the older frames into priors.

[0008] However, the marginalization process involves the Schur complement operation on the Hessian matrix; the dimension of the Hessian matrix is ​​determined by the number of state variables, and the matrix dimension expands rapidly as the number of landmarks increases; the time complexity of the Schur complement operation is typically O(n log n). On embedded platforms with limited computing resources, frequent and large-scale edge operations consume a significant amount of CPU time, causing the front-end tracing thread to block and impacting the system's real-time performance. While existing systems such as VINS-Mono employ certain strategies, they typically perform overall operations on a large matrix containing all landmarks and poses, failing to fully utilize the sparse structure of the matrix and thus limiting their efficiency improvements.

[0009] 4. Mismatches and inefficiency in loop closure detection. Loop closure detection is a crucial step in eliminating cumulative drift. Traditional methods are mostly based on the Bag-of-Words (BoW) model, which converts the current frame into a bag-of-words vector and performs similarity matching with historical keyframes.

[0010] However, as the running trajectory lengthens, the size of the keyframe database gradually increases, and the efficiency of frame-by-frame retrieval is low. More seriously, in environments with similar textures, such as similar corridors and repetitive structures, matching based on the appearance of a single frame is prone to "perceptual aliasing," that is, incorrectly identifying different locations as the same location, leading to closed-loop optimization errors and completely destroying the consistency of the map. Summary of the Invention

[0011] The purpose of this invention is to provide a dynamic interference-resistant SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue, as well as corresponding equipment and media.

[0012] To achieve the above objectives, the present invention adopts the following technical solution:

[0013] In a first aspect, the present invention provides a dynamic interference-resistant SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue, comprising the following steps:

[0014] S1. Acquire camera image sequence data, input it into the FPN-SuperPoint feature extraction network, and output the feature points of the image frames and their corresponding feature descriptors; input the feature-extracted image sequence into the SuperGlue feature matching network to obtain initial feature matching pairs;

[0015] S2. For each feature point contained in the initial feature matching pair output by S1, combined with the feature point probability distribution information, an adaptive weighted sub-pixel optimization strategy of semantic-probabilistic coupling is introduced to obtain sub-pixel-level high-precision feature points, and the initial feature matching pair is updated based on these optimized feature points.

[0016] S3. By using target detection and improved Kalman filtering, matching pairs corresponding to feature points falling within the dynamic region are eliminated to obtain static feature matching pairs;

[0017] S4. For the static feature matching pairs output by S3, a selective step-by-step edge-mapping strategy based on a sliding window is used to optimize the keyframes.

[0018] S5. Construct a keyframe sequence using the optimized keyframes, perform loop closure detection based on the keyframe sequence, and then correct the drift error of the detected loops.

[0019] The FPN-SuperPoint feature extraction network includes three consecutive downsampling operation modules. Each downsampling operation module contains two cascaded CR modules and a max pooling layer, which can be represented by the function as follows: , where the function Represents a CR module, functions This represents a max pooling layer; the feature map output by the third downsampling operation module. ; Through two CR modules (represented by functions as follows) The processed output is a feature map. , The mathematical space representing the input image information Image is defined as follows: This represents the channel dimension of a single layer of the encoder; channel splicing operations are used to... and Merging them together yields the spliced ​​intermediate feature map. , Introducing convolutional layers right Perform dimensionality reduction processing and output image. This serves as a shared input for subsequent use.

[0020] In step S1, the specific steps for feature matching based on the SuperGlue feature matching network are as follows:

[0021] The output image The feature point detection head and feature descriptor head are input to the decoding end, respectively. The detection head predicts the feature point positions and confidence levels, and the descriptor head calculates the feature descriptor vector. The feature point positions, confidence levels, and feature descriptor vectors are combined to construct the feature set of a single frame image. The above steps are performed on the reference frame image to be matched and the current frame image, respectively, to obtain two feature sets. The two feature sets are input into the SuperGlue network, which uses an attention-based graph neural network to perform cross-graph context aggregation and solves the optimal transmission problem using the Sinkhorn algorithm to obtain the soft allocation matrix. Extracting the soft allocation matrix Top left corner The submatrix serves as the final matching probability matrix, where the first submatrix is ​​the first matching probability matrix. Line number The elements of the column are denoted as the matching probabilities. .

[0022] If probability If the following conditions are met, then the first element in the reference frame feature set is determined. The nth feature point and the nth feature point in the current frame feature set Each feature point is a valid matching pair: in, For the soft allocation matrix, the first The maximum value of all columns in a row; For the soft allocation matrix, the first The maximum value of all row elements in the column; This is the preset confidence threshold.

[0023] In step S2, the semantic-probabilistic coupled adaptive weighted sub-pixel optimization strategy is specifically as follows:

[0024] Constructing semantic weights: Based on the feature point locations output by the FPN-SuperPoint network and a dense "corner probability map" output by the FPN-SuperPoint feature extraction network. Define each pixel Reliability weight : in, For the first The position of each pixel To obtain from the "corner probability map" Extracting pixels from The corresponding "corner probability value", Indicates the input image At pixel position gradient at;

[0025] Construct a weighted objective function to obtain the final solution form: in, For the characteristic pyramid number The square of the layer scale scaling factor; For matrix The transpose of the matrix;

[0026] matrix , , Defined as: .in, Indicates the first The Jacobian matrix of the residual vector composed of visual reprojection error and IMU pre-integration error at each iteration, relative to the camera pose and landmark coordinates. Indicates the first Pixels in layer feature map Reliability weight.

[0027] In step S3, before removing feature points falling within the dynamic region, a dynamic determination is performed based on target detection and improved Kalman filtering. The steps are as follows:

[0028] S31, Symbol Definition: Definition of the first The first frame The system state vector of the tracked target A vector containing its geometric properties and its first derivative: in, Indicates the center coordinates of the target box. The aspect ratio of the target bounding box. The height of the target bounding box. These represent the rates of change for each of the aforementioned states.

[0029] S32. State Prediction: Construct a Kalman filter for each dynamic target detection box, establish a state transition equation based on a uniform motion model, and predict the state for the first... The state vector and covariance matrix of the dynamic object detection bounding box in the frame are used to predict the result. Prior state estimation of frames and prior covariance matrix ;

[0030] An adaptive measurement noise covariance matrix is ​​constructed by combining the detection box confidence score and the region tracking quality score. Let the first The observation vector of the frame is Its corresponding measurement noise covariance matrix Dynamic adjustment: in, Based on the measurement of noise covariance matrix; The confidence score for the current detection bounding box; The region tracking quality score for the detection box; This is the adjustment coefficient;

[0031] The regional tracking quality score Specifically: in, For indicator functions; To effectively match the threshold; To prevent tiny quantities with a denominator of zero; To fall into the detection frame The set of feature points inside; For set Feature point index in; For feature points Its corresponding matching point in the feature matching pair The probability of matching between them;

[0032] S33. Hierarchical Data Association: Using the Hungarian matching algorithm, a cost matrix is ​​first constructed to determine high-confidence detection boxes. With prediction box The optimal match between them is determined, and then a quadratic cost matrix is ​​constructed to determine the low-confidence detection boxes. Unmatched prediction boxes The optimal match between them;

[0033] For successfully matched detection boxes With prediction box Check between them If the value is greater than a set threshold, a match is considered successful, and further fusion and updating are performed; predicted boxes that fail to match are... Then retain Frames predicted are not discarded; if in If no match is found within the frame, the frame is discarded.

[0034] S34, Kalman filter fusion update: For the successfully matched detection boxes in S33 With prediction box Using the observation vector as Prior state estimation and the prior covariance matrix calculated in S32 Adaptive measurement noise covariance matrix Calculate the observation residuals and Kalman gain, and use the Kalman gain to fuse the observation residuals to estimate the prior state. and prior covariance matrix Update the state to obtain the posterior state estimate. and posterior covariance matrix ;

[0035] S35. Hypothetical Tracking Mechanism: After fusion and updating via Kalman filtering, the front end does not rush to process the frame, but instead puts the frame result into a container with a size greater than or equal to... In the buffer queue; for unmatched detection boxes, a "hypothetical tracker" is created; if the hypothetical tracker continues to match successfully in the future, its status is upgraded to "confirmed"; if the hypothetical tracker fails to match too many times in the future, it is considered a false detection.

[0036] In step S4, the selective step-by-step edge-out strategy specifically includes:

[0037] S41. Subdivide the state to be marginalized into a sparse feature group and a dense pose group, and construct a block Hessian matrix containing the sparse feature group, the dense pose group and the remaining state variables.

[0038] S42. Marginalize the sparse feature group and use the Schur complement operation to eliminate its corresponding rows and columns to obtain the intermediate information matrix about the dense pose group and the remaining state.

[0039] S43. Based on the intermediate information matrix, the dense pose groups are further marginalized to obtain the final prior information matrix only concerning the remaining state variables. and the corresponding residual vector ;

[0040] S44. Based on the final prior information matrix With the final residual vector Construct information about the remaining state variables Linear prior constraint equations: This equation transforms marginalized historical information into constraints on the remaining state, which then participate as prior terms in the sliding window optimization at the next time step.

[0041] In step S5, the loop closure detection optimization strategy based on the keyframe sequence is as follows:

[0042] S51. Define the keyframe sequence: Keyframe sequence From the current keyframe and its temporally continuous preceding Composed of keyframes: in, For the first in the sequence One keyframe; The latest keyframe; The earliest keyframe in the sequence;

[0043] S52. Sequence Similarity Measurement and Matching Strategy: A weighted fusion strategy is used to construct sequence descriptors, and the similarity score between the current keyframe sequence descriptor and historical candidate sequence descriptors is calculated. A coarse matching is used to filter out the candidate sequence set; then a fine matching is performed for each historical sequence in the candidate historical sequence set. Calculate the similarity matrix between its internal single frames and the current sequence single frames. Determine the best single-frame matching pair. ;

[0044] S53. Temporal continuity test: For the optimal single-frame matching pair Perform a time continuity test. If the matching pair passes the continuity test, then perform closed-loop optimization.

[0045] Secondly, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any step in the described anti-dynamic interference SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue.

[0046] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a computer processor, implements any step in the described anti-dynamic interference SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue.

[0047] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0048] 1. This invention overcomes the positioning drift and computational bottlenecks of traditional algorithms in complex dynamic scenes by employing techniques such as front-end high-robust feature perception, precise dynamic interference removal, back-end efficient edge optimization, and serialized loop closure detection. It achieves high-precision and high-reliability state estimation in extreme environments with complex textures and drastic scale changes, thereby improving the robustness and environmental adaptability of SLAM systems. This invention is applicable to the long-term stable operation of intelligent agents such as autonomous vehicles, drones, and service robots in practical engineering deployments, and has extremely high practical value and broad application prospects.

[0049] 2. To address the issues of low feature repetition rate, susceptibility to mismatches, and limited localization accuracy (constrained by integer pixels) in existing visual front-end algorithms under conditions of drastic scale changes and weak textures: This invention constructs a fusion architecture of FPN-SuperPoint feature extraction network and SuperGlue feature matching network. First, by improving the shared encoder of the SuperPoint network and introducing the Feature Pyramid (FPN) multi-scale fusion mechanism, the problem of decreased detection capability of the original SuperPoint when the object scale changes drastically is effectively solved. Then, by innovatively combining the SuperGlue network, the attention mechanism of the Graph Neural Network (GNN) is used to aggregate global spatiotemporal context information, and the traditional nearest neighbor distance metric is replaced by solving the differentiable optimal transmission problem. This achieves high-confidence feature association far exceeding that of traditional algorithms in scenarios with large viewpoint changes and dynamic occlusion.

[0050] Finally, this invention introduces a semantic-probabilistic coupled adaptive weighted subpixel optimization strategy, which uses a corner probability map to construct a weighted constraint equation, breaking through the quantization limitation of sensor pixel grids and achieving a leap in end-to-end accuracy from feature extraction and robust matching to subpixel-level precise positioning.

[0051] 3. To address the problem that traditional algorithms are prone to misidentification and removal of dynamic features in dynamic scenarios, leading to feature scarcity, this invention proposes a dynamic feature removal strategy based on target detection combined with improved Kalman filtering.

[0052] To address the issues of high computational complexity and poor real-time performance in associating dynamic object data, this invention innovatively introduces a multi-level data association approach using primary and secondary hierarchical cost matrices. High-confidence detection boxes are prioritized for matching, followed by secondary matching of low-confidence detection boxes. This hierarchical strategy reduces the computational complexity of the original method to... The full Hungarian matching problem is significantly pruned, reducing the CPU usage of the front-end data association module by about 40% while maintaining a high dynamic rejection rate, thus meeting the real-time requirements of mobile robots in complex dynamic scenarios.

[0053] Meanwhile, the hypothetical tracking mechanism effectively prevents wall posters, static sculptures, and other objects from being misjudged as dynamic objects, thus avoiding the incorrect removal of static feature points.

[0054] 4. To address the efficiency bottleneck caused by the increasing number of landmarks in the Schur complement operation during backend sliding window optimization, this invention proposes a selective step-by-step edge-out strategy. This strategy explicitly decouples the state to be edged out into sparse feature groups and dense pose groups, utilizing the block diagonal matrix characteristics formed by the conditional independence between landmark points to perform Schur complement operations step-by-step. This strategy can reduce the time complexity of edge-out feature points from... Reduced to This enables the system to maintain stable operation of a large number of road points over a long period of time, even on embedded devices with limited computing resources.

[0055] 5. To address the problem of low computational efficiency in traditional loop closure detection processes due to map size growth leading to one-to-one matching of historical frames, this invention designs a loop closure detection optimization strategy based on keyframe sequences. Regarding retrieval efficiency, sequence descriptors are used to pre-screen historical sequences, quickly eliminating the vast majority of irrelevant historical sequences, significantly reducing the computational resource consumption and retrieval time of the loop closure detection module. Regarding detection accuracy, after single-frame matching in the optimal matching sequence, a time continuity check is introduced, improving both the speed and accuracy of loop closure detection, effectively enhancing the overall performance of the system. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the process of the present invention in Example 1; Figure 2 This is a schematic diagram of the FPN-SuperPoint feature extraction network in Example 1, showing the specific network layer connection methods for the extraction, upsampling, splicing and fusion of feature maps of module 3 and module 4; Figure 3 This is a schematic diagram of the FPN-SuperPoint combined with SuperGlue structure in Example 1; Figure 4 The flowchart of the loop closure detection algorithm based on keyframe sequences in Example 1 is shown, including the judgment logic of sequence construction, bag-of-words matching, similarity scoring and continuity test; Figure 5This is a visual comparison of the trajectory of the loop scene on the same road segment in Example 1 between the method of the present invention and VINS-Fusion. Detailed Implementation

[0057] The present invention will be further described in detail below through specific embodiments, but this does not limit the scope of the present invention.

[0058] Example 1

[0059] The overall process of the dynamic interference-resistant SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue is as follows: Figure 1 As shown, it includes the following steps:

[0060] S1. Acquire camera image sequence data, input it into the FPN-SuperPoint feature extraction network, and output the feature points of the image frames and their corresponding feature descriptors; input the image sequence after feature extraction into the SuperGlue feature matching network to obtain the initial feature matching pairs.

[0061] The FPN-SuperPoint feature extraction network structure is as follows: Figure 2 As shown, it includes three consecutive downsampling operation modules. Each downsampling operation module contains two cascaded CR modules and a max pooling layer, represented by the function as follows: , where the function Represents a CR module, functions This represents a max pooling layer; the feature map output by the third downsampling operation module. ; Through two CR modules (represented by functions as follows) The processed output is a feature map. , Represents the input image information Image (the function variable used earlier). The mathematical space of the general input parameters (specifically image information) is defined as follows: In this embodiment, the encoder has a single-layer channel dimension. Channel splicing operation is used to... and Merging them together yields the spliced ​​intermediate feature map. , ,at this time, The channel dimension is expanded to (i.e., 128+128); introduce convolutional layers right Perform dimensionality reduction processing and output image. , This serves as a shared input for subsequent use. The channel dimension is 128.

[0062] Furthermore, feature matching is performed based on the SuperGlue feature matching network, such as... Figure 3 As shown, in this embodiment, feature matching no longer relies on independent feature point descriptor distance metrics, but instead achieves global context awareness and optimal transmission matching of feature points by constructing a Dual-Graph Attention Interaction Network. The specific steps are as follows:

[0063] S11, Output image The feature point detection head and feature descriptor head are respectively input to the decoding end. The feature point position and confidence are predicted by the detection head, and the feature descriptor vector is calculated by the descriptor head. The feature point position, confidence and feature descriptor vector are combined to construct the feature set of a single frame image.

[0064] S12. Perform the above steps for the reference frame image (Image A) to be matched and the current frame image (Image B) respectively to obtain two sets of features:

[0065] Graph node embedding and initialization, defining the current frame image as... The reference frame image is Two feature sets were obtained based on the FPN-SuperPoint feature extraction network. Expressed as: ;in, For the current frame, the first The pixel coordinates of each feature point Its corresponding Multi-scale fusion descriptor. Feature set. Corresponding reference frame 1 feature point.

[0066] Feature set Expressed as: ;in, For the reference frame The pixel coordinates of each feature point Its corresponding Multi-scale fusion descriptor; feature set Corresponding reference frame 1 feature point.

[0067] Construct a high-dimensional node state vector to fuse geometric distribution and visual semantic information, and define a position encoding function. Feature set Initial node state Expressed as: Feature set Initial node state Expressed as: in, A learnable projection matrix for position encoding.

[0068] S13. Input the two feature sets into the SuperGlue network, perform cross-graph context aggregation using an attention-based graph neural network, and solve the optimal transfer problem using the Sinkhorn algorithm to obtain the soft allocation matrix. :

[0069] S131. Cross-graph context aggregation to construct a fully connected graph structure containing two types of edge attributes: Intra-frame Edge. Inter-frame Edge ;use Layered graph neural networks perform message passing; let the first layer... The node state of the layer is , No. The state update of the layer follows the residual aggregation mechanism; in, For the first Corresponding feature points in the layer The node state vector; For the first Corresponding feature points in the layer The node state vector; The aggregated message vector is generated by a multi-head attention mechanism.

[0070] When calculating inter-frame interactions, define a query vector. Key vector AND value vector : in, For feature points The query vector; For the first The query projection weight matrix of the layer; For feature points The key vector; For the first Layer bond projection weight matrix; For the first Feature points in the layer The node state vector; For feature points The value vector; For the first The layer values ​​are projected onto the weight matrix.

[0071] Obtain interaction weight coefficients : in, For the feature point The normalized exponential function performed on the dimension in which it is located; For query vector transpose; For feature points The key vector; is the dimension of the key vector, and is a regularization factor used to scale the magnitude of the dot product.

[0072] Aggregated messages Weighted sum of value vectors:

[0073] The network layers alternately perform intra-frame aggregation (Self) and inter-frame aggregation (Cross), ultimately outputting enhanced matching features. and .

[0074] S132. Differentiable optimal transmission matching, constructing a similarity scoring matrix. : Among them, elements Representing feature points and The strength of the association; For the feature points of the current frame The transpose of the final matching descriptor vector; For reference frame feature points The final matching descriptor vector.

[0075] A slack node mechanism is introduced to handle unmatched points caused by changes in the field of view and occlusion by dynamic objects; the scoring matrix is ​​expanded to... : in, These are learnable outlier penalty parameters; This is the global background threshold. The outlier penalty parameter vector for the reference frame The transpose of .

[0076] The optimal transmission scheme is solved using the Sinkhorn iterative algorithm; an allocation matrix for the logarithmic field is defined. ,conduct Second row and column normalization iteration: After iterative convergence, the soft-allocation matrix is ​​obtained. .

[0077] S14. Matching Pair Filtering: Extracting the Soft Assignment Matrix Top left corner The submatrix serves as the final matching probability matrix, where the first submatrix is ​​the first matching probability matrix. Line number The elements of the column are denoted as matching probabilities. .

[0078] If probability If the following conditions are met, then the first element in the reference frame feature set is determined. The nth feature point and the nth feature point in the current frame feature set Each feature point is a valid matching pair: in, For the soft allocation matrix, the first The maximum value of all columns in a row; For the soft allocation matrix, the first The maximum value of all row elements in the column; The preset confidence threshold is 0.2 in this embodiment.

[0079] S2. For each feature point in the initial feature matching pair output by S1, combined with the feature point probability distribution information, an adaptive weighted sub-pixel optimization strategy with semantic-probabilistic coupling is introduced to obtain sub-pixel-level high-precision feature points, and the initial feature matching pair is updated based on these optimized feature points. Traditional sub-pixel optimization relies only on grayscale gradients, which are prone to generating outliers when the texture is weak or the illumination changes. This invention innovatively constructs a weighted constraint equation based on the multi-scale feature response of FPN; and introduces the corner probability map of the SuperPoint network. Constructing the weight matrix This allows the optimization process to intelligently focus on high-confidence semantic feature points, thereby breaking through the dependence of traditional geometric methods on environmental texture.

[0080] Specifically, the semantic-probabilistic coupled adaptive weighted sub-pixel optimization strategy is as follows:

[0081] Constructing semantic weights: Based on the feature point locations output by the FPN-SuperPoint network and a dense "corner probability map" output by the FPN-SuperPoint feature extraction network. Define each pixel Reliability weight : in, For the first The position of each pixel To obtain from the "corner probability map" Extracting pixels from The corresponding "corner probability value", Indicates the input image At pixel position gradient at;

[0082] Construct a weighted objective function: in, It is the scale scaling factor; It is the gradient of the feature map; For the first Pixels in layer feature map Reliability weights; For pixels Transpose of a coordinate vector; For the initial feature point at the th The set of neighboring pixels on the layer feature map.

[0083] The final solution form is obtained as follows: in, For the characteristic pyramid number The square of the layer scale scaling factor; For matrix The transpose of the matrix;

[0084] matrix , , Defined as: .in, Indicates the first The Jacobian matrix of the residual vector, composed of the visual reprojection error and the IMU pre-integration error, relative to the camera pose and landmark coordinates at each iteration.

[0085] S3. By using target detection and improved Kalman filtering to remove matching pairs corresponding to feature points falling within the dynamic region, static feature matching pairs are obtained.

[0086] Before removing feature points that fall within the dynamic region, a dynamic determination is performed first.

[0087] The specific steps for dynamic determination based on target detection and improved Kalman filtering are as follows:

[0088] S31. Symbol Definition: Let the detection box of a dynamic target (such as a pedestrian or vehicle) in an image be defined by its center coordinates. Aspect Ratio and height Description; Definition The first frame The system state vector of the tracked target An 8-dimensional vector containing its geometric properties and its first derivative (rate of change): in, Indicates the center coordinates of the target box. The aspect ratio of the target bounding box. The height of the target bounding box. These represent the rates of change for each of the aforementioned states.

[0089] S32, State Prediction:

[0090] For each dynamic target detection box, a Kalman filter is constructed, and a state transition equation is established based on a uniform motion model. For the ... The state vector and covariance matrix of the dynamic object detection bounding box in the frame are used to predict the result. Prior state estimation of frames and prior covariance matrix :

[0091] Assuming an extremely short inter-frame time interval Within the image plane, the motion of the target follows a uniform velocity model; using the first... Posterior state estimation of frames , No. Prior state estimation of frames Represented as:

[0092] State transition matrix Defined as: in, It is the identity matrix. The control matrix, in this model, is typically treated as zero or external inputs can be ignored. Input for the system.

[0093] Prior covariance matrix The prediction formula is: in, The process noise covariance matrix is ​​a fixed matrix, representing the inherent uncertainty of the uniform motion model (such as the small acceleration changes of the target). For the first The posterior covariance matrix of the frame; State transition matrix The transpose of .

[0094] To address the uncertainty in YOLO detection, this embodiment combines the detection box confidence score and the region tracking quality score to construct an adaptive measurement noise covariance matrix. ; Introduce the confidence scores of the detection box and SuperGlue; Let the first... The observation vector of the frame is , Direct measurement data from object detection networks (such as YOLOv8), where 'det' is an abbreviation for 'detection', representing the measured value. Indicates the center coordinates of the target box. The aspect ratio of the target bounding box. The height of the target bounding box. The corresponding measurement noise covariance matrix Dynamic adjustment: in, The basic measurement noise covariance matrix represents the inherent measurement error of the sensor under ideal detection conditions (such as clear target and no obstruction). This represents the confidence score of the current detection bounding box, with a value range of [range missing]. ; The region tracking quality score for the detection box; In this embodiment, the adjustment coefficient is... This is used to control the sensitivity of the confidence level to measurement noise amplification;

[0095] The regional tracking quality score Specifically: in, For indicator functions; To effectively match the threshold; To prevent tiny quantities with a denominator of zero; let the number of values ​​output by YOLOv8 in the current frame be the first... Each detection box is , To fall into the detection frame The set of feature points inside; For set Feature point index in; For feature points Its corresponding matching point in the feature matching pair The probability of matching between them;

[0096] S33. Hierarchical Data Association: Using the Hungarian matching algorithm, a cost matrix is ​​first constructed to determine high-confidence detection boxes. With prediction box The optimal match between them is determined, and then a quadratic cost matrix is ​​constructed to determine the low-confidence detection boxes. Unmatched prediction boxes The optimal match between them.

[0097] Construct the first-order cost matrix and the second-order cost matrix;

[0098] First, the detection boxes are divided into high-confidence detection boxes according to the confidence level (0.5). With low confidence detection boxes First, calculate the confidence level of each detection box. With each prediction box Crossover ratio between: Furthermore, high-confidence detection boxes and prediction boxes are first correlated to construct a... Cost matrix As a cost matrix, where This represents the number of high-confidence detection boxes. Number of prediction boxes: The purpose of constructing the cost matrix is ​​to find the maximum total cost. In the case of values, the data association problem is transformed into an optimal task matching problem with a cost matrix; by analyzing... The row and column elements are processed to determine... and The optimal match between them.

[0099] For predicted boxes that did not match in a single match Utilizing it with low-confidence detection boxes Constructing a quadratic cost matrix First, calculate the confidence level of each low-confidence detection box. With each unmatched prediction box Crossover ratio between: Furthermore, data association is performed between low-confidence detection boxes and unmatched predicted boxes to construct a... Cost matrix , This represents the number of low-confidence detection boxes. The number of remaining prediction boxes: Then, by processing the row and column elements, the determination can be made. and The optimal match between them.

[0100] For successfully matched detection boxes With prediction box Check between them If the value is greater than a set threshold, a match is considered successful, and further fusion and updating are performed; predicted boxes that fail to match are... The predicted frame is retained and not discarded; if in If no match is found within the frame, the frame is discarded.

[0101] S34, Kalman filter fusion update: For the successfully matched detection boxes in S33 With prediction box Using the observation vector as Prior state estimation and the prior covariance matrix calculated in S32 Adaptive measurement noise covariance matrix Calculate the observation residuals and Kalman gain, and use the Kalman gain to fuse the observation residuals to estimate the prior state. and prior covariance matrix Update the state to obtain the posterior state estimate. and posterior covariance matrix ;

[0102] S35. Hypothetical Tracking Mechanism: After fusion and updating via Kalman filtering, the front end does not rush to process the frame; it first needs to put the frame result into a container with a size greater than or equal to... In the buffer queue, the following operations are performed: For unmatched detection boxes, they are not directly marked as dynamic, but a "hypothetical tracker" is established, which is considered to be a dynamic target that has just entered the field of view, and its hypothetical tracking results are also included in the tracking; if the hypothetical tracker continues to match successfully in the next 3 frames, its status is upgraded to "confirmed", and the feature matching pairs of the corresponding regions in the past 3 frames are backtracked and removed; if the hypothetical tracker fails to match too many times in the next 5 frames, it is regarded as a false detection and deleted, but the feature matching pairs of the corresponding regions are not removed; the hypothetical tracking mechanism can effectively prevent posters on the wall, static sculptures, etc. from being misjudged as dynamic objects, and ensure the number of static feature points.

[0103] S4. For the static feature matching pairs output by S3, a selective step-by-step edge-mapping strategy based on a sliding window is used to optimize the keyframes.

[0104] Specifically, the selective step-by-step edge-out strategy based on the sliding window is as follows:

[0105] S41. State variable reorganization and segmentation: The state to be marginalized is subdivided into sparse feature groups and dense pose groups, and a segmented Hessian matrix containing sparse feature groups, dense pose groups and remaining state variables is constructed.

[0106] The state to be marginalized Further subdivided into sparse feature groups and dense pose groups The sparse feature set Including the inverse depth of all landmarks observed in the oldest frame And the speed of the oldest frame and zero bias The dense pose group Including the pose of the oldest frame .

[0107] Therefore, the normal equations are rearranged into a three-part structure: in, For each sparse feature group, there is a Hessian matrix block. Since the landmark points are conditionally independent, it is a block diagonal matrix. This represents the cross-correlation Hessian matrix block between the dense pose group and the sparse feature group; The Hessian matrix block representing the cross-correlation between the remaining state variables and the sparse feature set; This represents the cross-correlation Hessian matrix block between the sparse feature set and the dense pose set; For dense pose groups, it is their own Hessian matrix block; The Hessian matrix block representing the cross-correlation between the remaining state variables and the dense pose group; This is a Hessian matrix block representing the cross-correlation between the sparse feature set and the remaining state variables; This is a Hessian matrix block representing the cross-correlation between dense pose groups and other state variables; The remaining state variables are themselves Hessian matrix blocks; The remaining state variables that do not need to be marginalized and are not the main targets of this optimization; This is the residual vector corresponding to the sparse feature set; This is the residual vector corresponding to the dense pose group; This is the residual vector corresponding to the remaining state variables.

[0108] The key nature is (Corresponding feature point part) is a block diagonal matrix because, given the camera pose, the observations between different landmarks are independent of each other.

[0109] S42. Sparse Feature Marginalization: First, marginalize the sparse feature group. The corresponding rows and columns are eliminated using the Schur complement operation to obtain an intermediate information matrix about the dense pose group and the remaining state.

[0110] S43. Dense Pose Edge Computation: Edge Computation of Dense Pose Groups Based on Intermediate Information Matrix By using the Schur complement operation to eliminate the corresponding rows and columns, we obtain the final prior information matrix with respect to the remaining state variables. and the corresponding residual vector ;

[0111] S44. Based on the final prior information matrix With the final residual vector Construct information about the remaining state variables The linear prior constraint equations are: This equation transforms marginalized historical information into constraints on the remaining state, which then participate as prior terms in the sliding window optimization at the next time step.

[0112] S5. Combine the keyframes optimized in step S4 into a keyframe sequence, perform loop closure detection based on the keyframe sequence, and then correct the drift error of the detected loops. The specific process is as follows: Figure 4 As shown.

[0113] In step S5, the loop closure detection optimization strategy for the keyframe sequence is as follows:

[0114] S51. Define a keyframe sequence: Define a sequence of length... keyframe sequence The keyframe sequence consists of the current keyframe. and its temporally continuous preceding Composed of keyframes: in, For the first in the sequence One keyframe; The latest keyframe; This is the earliest keyframe in the sequence.

[0115] S52. Sequence Similarity Measurement and Matching Strategies:

[0116] First, a coarse matching is performed, a weighted fusion strategy is used to construct sequence descriptors, and the similarity score between the current keyframe sequence descriptor and the historical candidate sequence descriptors is calculated. The candidate sequence set is then selected through coarse matching.

[0117] Define sequence descriptors: Construct sequence descriptors to represent the semantic and texture information of the entire sequence. A weighted fusion strategy is adopted: in, This is the bag-of-words vector of a single frame using TF-IDF (Term Frequency-Inverse Document Frequency). For the first One keyframe The time decay weight is usually set to 1. Treat all frames within the sequence equally, or adapt them adaptively based on the amount of information; this operation expands the observation at a single time point into a description of trajectory segments on the spatiotemporal manifold.

[0118] normalization Norm as a distance metric, score Defined as: in, For the current keyframe sequence, This is a sequence of historical candidate keyframes. This is the descriptor vector for the current keyframe sequence; This is a descriptor vector for the historical keyframe sequence; For the current sequence descriptor vector Norm; For the history sequence descriptor vector Norm.

[0119] Set threshold Filter out the scores Candidate historical sequence set This step utilizes the rich information in the sequence to quickly eliminate most obvious false matches.

[0120] Then, a finer matching process is performed for each historical sequence in the candidate set. Further, calculate the similarity matrix between its internal single frames and the current sequence single frames. ,in, express The Middle Frame and The Middle Frame similarity.

[0121] In the matrix The highest-scoring diagonal path is selected, representing the best alignment of the two sequences in the temporal direction; if this alignment score is significantly higher than other candidates, it is chosen. To determine the best matching sequence, and to identify the best single-frame matching pair within it. .

[0122] S53, Time Continuity Test: Assume that at time t... Candidate loop closure detected If the loop is true, then in the subsequent At that moment, it should be able to Loop closures were detected within the neighborhood of the loop; a set of loop closure hypotheses was constructed within a sliding window, and the hypothesis was only considered if the loops were consecutive. The time continuity test is passed only when each keyframe (3-5 frames) independently matches a continuous historical trajectory; the matches that pass the continuity test are then optimized for closed-loop operation.

[0123] Figure 5 The diagram shows a visual comparison of the loopback trajectories of the method in this embodiment and VINS-Fusion on the same road segment. The solid blue line represents the trajectory of this embodiment, and the dashed black line represents the VINS-Fusion trajectory. The error between the trajectory length obtained by the method in this embodiment and the actual measured trajectory length is smaller than the error between the trajectory length obtained by VINS-Fusion and the actual measured trajectory length. Therefore, the method in this embodiment has higher positioning accuracy than VINS-Fusion, demonstrating the performance and robustness of the system presented in this paper.

[0124] Example 2

[0125] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any step of the anti-dynamic interference SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue as described in Embodiment 1.

[0126] Furthermore, the dynamic interference-resistant SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue described in Embodiment 1 can be implemented as a computer software program. For example, this embodiment includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method. In such an embodiment, the computer program can be downloaded and installed from a network, and / or installed from a removable medium. When the computer program is executed by a processor, it performs the functions defined in the method of this application.

[0127] Example 3

[0128] A computer-readable storage medium storing a computer program that, when executed by a processor, implements any step in the dynamic interference-resistant SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue as described in Embodiment 1.

[0129] The computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0130] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Python and C++, as well as conventional procedural programming languages ​​or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0131] In this embodiment, the computer-readable storage medium can be accelerated using hardware such as a GPU. The parallel computing advantage of the GPU is used to accelerate any step in the implementation of the anti-dynamic interference SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue as described in Embodiment 1.

[0132] In summary, this invention effectively overcomes the shortcomings of the prior art and has high industrial applicability. The above embodiments are intended to illustrate the substantive content of this invention, but are not intended to limit the scope of protection of this invention. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this invention without departing from the essence and scope of protection of this invention.

Claims

1. A dynamic interference-resistant SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue, characterized in that, Includes the following steps: S1. Acquire camera image sequence data, input it into the FPN-SuperPoint feature extraction network, and output the feature points of the image frames and their corresponding feature descriptors; input the feature-extracted image sequence into the SuperGlue feature matching network to obtain initial feature matching pairs; S2. For each feature point contained in the initial feature matching pair output by S1, combined with the feature point probability distribution information, an adaptive weighted sub-pixel optimization strategy of semantic-probabilistic coupling is introduced to obtain sub-pixel-level high-precision feature points, and the initial feature matching pair is updated based on these optimized feature points. S3. By using target detection and improved Kalman filtering, matching pairs corresponding to feature points falling within the dynamic region are eliminated to obtain static feature matching pairs; S4. For the static feature matching pairs output by S3, a selective step-by-step edge-mapping strategy based on a sliding window is used to optimize the keyframes. S5. Construct a keyframe sequence using the optimized keyframes, perform loop closure detection based on the keyframe sequence, and then correct the drift error of the detected loops.

2. The anti-dynamic interference SLAM loop closure detection method according to claim 1, characterized in that, The FPN-SuperPoint feature extraction network includes three consecutive downsampling operation modules, each of which contains two cascaded CR modules and a max pooling layer; the feature map output by the third downsampling module... ; The output after processing by two CR modules is a feature map. Channel splicing operation is used to... and Merging them together yields the spliced ​​intermediate feature map. Introducing convolutional layers right Perform dimensionality reduction processing and output image. This serves as a shared input for subsequent use.

3. The anti-dynamic interference SLAM loop closure detection method according to claim 1, characterized in that, In step S1, the specific steps for feature matching based on the SuperGlue feature matching network are as follows: The output image The feature point detection head and feature descriptor head are input to the decoding end, respectively. The detection head predicts the feature point positions and confidence levels, and the descriptor head calculates the feature descriptor vector. The feature point positions, confidence levels, and feature descriptor vectors are combined to construct the feature set of a single frame image. The above steps are performed on the reference frame image to be matched and the current frame image, respectively, to obtain two feature sets. The two feature sets are input into the SuperGlue network, which uses an attention-based graph neural network to perform cross-graph context aggregation and solves the optimal transmission problem using the Sinkhorn algorithm to obtain the soft allocation matrix. Extracting the soft allocation matrix Top left corner The submatrix serves as the final matching probability matrix, where the first submatrix is ​​the first matching probability matrix. Line number The elements of the column are denoted as the matching probabilities. , If probability If the following conditions are met, then the first element in the reference frame feature set is determined. The nth feature point and the nth feature point in the current frame feature set Each feature point is a valid matching pair: in, For the soft allocation matrix, the first The maximum value of all columns in a row; For the soft allocation matrix, the first The maximum value of all row elements in the column; This is the preset confidence threshold.

4. The SLAM loop closure detection method against dynamic interference according to claim 1, characterized in that, In step S2, the semantic-probabilistic coupled adaptive weighted sub-pixel optimization strategy is specifically as follows: Constructing semantic weights: Based on the feature point locations output by the FPN-SuperPoint network and a dense "corner probability map" output by the FPN-SuperPoint feature extraction network. Define each pixel Reliability weight : in, For the first The position of each pixel To obtain from the "corner probability map" Extracting pixels from The corresponding "corner probability value", Indicates the input image At pixel position gradient at; Construct a weighted objective function to obtain the final solution form: in, For the characteristic pyramid number The square of the layer scale scaling factor; For matrix The transpose of the matrix; matrix , , Defined as: ;in, Indicates the first The Jacobian matrix of the residual vector composed of visual reprojection error and IMU pre-integration error at each iteration, relative to the camera pose and landmark coordinates. Indicates the first Pixels in layer feature map Reliability weight.

5. The anti-dynamic interference SLAM loop closure detection method according to claim 1, characterized in that, In step S3, before removing feature points falling within the dynamic region, a dynamic determination is performed based on target detection and improved Kalman filtering. The steps are as follows: S31, Symbol Definition: Definition of the first The first frame The system state vector of the tracked target A vector containing its geometric properties and its first derivative: in, Indicates the center coordinates of the target box. The aspect ratio of the target bounding box. The height of the target bounding box. These represent the rates of change for each of the aforementioned states; S32, State Prediction: For each dynamic target detection box, a Kalman filter is constructed, and a state transition equation is established based on a uniform motion model. For the ... The state vector and covariance matrix of the dynamic object detection bounding box in the frame are used to predict the result. Prior state estimation of frames and prior covariance matrix ; An adaptive measurement noise covariance matrix is ​​constructed by combining the detection box confidence score and the region tracking quality score. Let the first The observation vector of the frame is Its corresponding measurement noise covariance matrix Dynamic adjustment: in, Based on the measurement of noise covariance matrix; The confidence score for the current detection bounding box; The region tracking quality score for the detection box; This is the adjustment coefficient; The regional tracking quality score Specifically: in, For indicator functions; To effectively match the threshold; To prevent tiny quantities with a denominator of zero; To fall into the detection frame The set of feature points inside; For set Feature point index in; For feature points Its corresponding matching point in the feature matching pair The probability of matching between them; S33. Hierarchical Data Association: Using the Hungarian matching algorithm, a cost matrix is ​​first constructed to determine high-confidence detection boxes. With prediction box The optimal match between them is determined, and then a quadratic cost matrix is ​​constructed to determine the low-confidence detection boxes. Unmatched prediction boxes The optimal match between them; For successfully matched detection boxes With prediction box Check between them If the value is greater than a set threshold, a match is considered successful, and further fusion and updating are performed; predicted boxes that fail to match are... The predicted frame is retained and not discarded; if in If no match is found within the frame, the frame is discarded. S34, Kalman filter fusion update: For the successfully matched detection boxes in S33 With prediction box Using the observation vector as Prior state estimation and the prior covariance matrix calculated in S32 Adaptive measurement noise covariance matrix Calculate the observation residuals and Kalman gain, and use the Kalman gain to fuse the observation residuals to estimate the prior state. and prior covariance matrix Update the state to obtain the posterior state estimate. and posterior covariance matrix ; S35. Hypothetical Tracking Mechanism: After fusion and updating via Kalman filtering, the front end does not rush to process the frame, but instead puts the frame result into a container with a size greater than or equal to... In the buffer queue; for unmatched detection boxes, a "hypothetical tracker" is created; if the hypothetical tracker continues to match successfully in the future, its status is upgraded to "confirmed"; if the hypothetical tracker fails to match too many times in the future, it is considered a false detection.

6. The SLAM loop closure detection method against dynamic interference according to claim 1, characterized in that, In step S4, the selective step-by-step edge-out strategy specifically includes: S41. Subdivide the state to be marginalized into a sparse feature group and a dense pose group, and construct a block Hessian matrix containing the sparse feature group, the dense pose group and the remaining state variables. S42. Marginalize the sparse feature group and use the Schur complement operation to eliminate its corresponding rows and columns to obtain the intermediate information matrix about the dense pose group and the remaining state. S43. Based on the intermediate information matrix, the dense pose groups are further marginalized to obtain the final prior information matrix only concerning the remaining state variables. and the corresponding residual vector ; S44. Based on the final prior information matrix With the final residual vector Construct information about the remaining state variables The linear prior constraint equations are: This equation transforms marginalized historical information into constraints on the remaining state, which then participate as prior terms in the sliding window optimization at the next time step.

7. The SLAM loop closure detection method against dynamic interference according to claim 1, characterized in that, In step S5, the loop closure detection optimization strategy based on the keyframe sequence is as follows: S51. Define the keyframe sequence: Keyframe sequence From the current keyframe and its temporally continuous preceding Composed of keyframes: in, For the first in the sequence One keyframe; The latest keyframe; The earliest keyframe in the sequence; S52. Sequence Similarity Measurement and Matching Strategy: A weighted fusion strategy is used to construct sequence descriptors, and the similarity score between the current keyframe sequence descriptor and historical candidate sequence descriptors is calculated. A coarse matching is used to filter out the candidate sequence set; then a fine matching is performed for each historical sequence in the candidate historical sequence set. Calculate the similarity matrix between its internal single frames and the current sequence single frames. Determine the best single-frame matching pair. ; S53. Temporal continuity test: For the optimal single-frame matching pair Perform a time continuity test. If the matching pair passes the continuity test, then perform closed-loop optimization.

8. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements any step in the anti-dynamic interference SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer processor, implements any step in the dynamic interference-resistant SLAM loop closure detection method based on the fusion of FPN-SuperPoint and SuperGlue as described in any of claims 1-7.

Images