Dynamic scene robust visual SLAM method based on multi-feature collaborative optimization

Abstract

The invention discloses a dynamic scene robust vision SLAM (Simultaneous Localization and Mapping) method based on multi-feature collaborative optimization, which comprises the following steps of: acquiring an image sequence, carrying out dynamic target detection and segmentation through an instance segmentation network, generating a segmentation mask containing a dynamic region mark, and identifying and separating a dynamic object and a static background; removing feature points corresponding to the dynamic object based on the segmentation mask to obtain static feature points; carrying out pose estimation based on the static feature points, and for the key frame, carrying out feature matching with the previous key frame by minimizing a re-projection error, and solving to obtain the camera pose of each key frame; for non-key frames, performing camera pose tracking and data association on the previous frame by adopting an optical flow algorithm, and accumulating solving results to obtain pose tracks of all the non-key frames; the key frames and the non-key frames are subjected to differential processing by fusing feature matching and an optical flow algorithm, so that the calculation efficiency is remarkably improved while the positioning precision is ensured, and the real-time performance is improved.

Description

Technical Field

[0001] This invention relates to the field of robot navigation technology, and more specifically to a dynamic scene robust visual SLAM method based on multi-feature collaborative optimization. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is a core technology for robots to simultaneously locate themselves and build maps of their environment in unknown environments. It has been widely applied in fields such as home robots, autonomous vehicles, and virtual reality. With the development of camera technology, Visual SLAM (VSLAM) has attracted much attention due to its advantages such as low cost, ease of integration, low power consumption, and rich environmental information capture capabilities.

[0003] Currently, significant progress has been made in vision-based SLAM algorithms, such as MONO-SLAM and SURF-SLAM based on feature point methods, UniVIO based on direct methods, and VINS-MONO and ORBSLAM3 based on multi-sensor fusion methods. However, these traditional algorithms generally rely on a core premise: all observed targets in the working environment remain stationary, i.e., the locations of landmarks in the scene are fixed. In practical applications, this assumption is often difficult to hold. When moving objects appear in the scene, they disrupt the feature correlation between consecutive frames, leading to camera trajectory estimation errors, decreased system localization capabilities, and consequently affecting the quality and reliability of map reconstruction.

[0004] Geometric feature-based methods have inherent limitations in feature extraction capabilities. Traditional geometric methods rely on hand-designed feature extractors, which cannot accurately distinguish between dynamic and static targets in complex dynamic environments, leading to feature matching errors. Furthermore, these methods have poor environmental adaptability. Scene segmentation methods based on K-Means and optical flow are easily affected by external conditions such as changes in lighting and rapid movement, resulting in significant performance degradation under harsh conditions such as dynamic environments, rapid movement, and brightness variations.

[0005] While deep learning-based methods have improved feature representation capabilities, their enormous computational cost remains a major bottleneck. The high computational complexity of deep learning modules severely limits the real-time performance of the system, making it difficult to meet the stringent response speed requirements of practical applications. For example, methods like DynaSLAM require background inpainting to fill in areas occluded by dynamic targets, further increasing computational costs and exacerbating the real-time performance issue.

[0006] Therefore, how to achieve accurate and timely positioning and mapping in dynamic environments is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a dynamic scene robust visual SLAM method based on multi-feature collaborative optimization, which has higher pose trajectory estimation accuracy and can significantly improve computational efficiency while ensuring real-time performance.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A robust visual SLAM method for dynamic scenes based on multi-feature collaborative optimization includes the following steps:

[0010] Image sequences are acquired and dynamic object detection and segmentation are performed using an instance segmentation network. A segmentation mask containing dynamic region labels is generated to identify and separate dynamic objects from static backgrounds.

[0011] Static feature points are obtained by removing feature points corresponding to dynamic objects based on segmentation masks.

[0012] Pose estimation is performed based on static feature point pairs. For key frames, the camera pose of each key frame is obtained by minimizing the reprojection error and performing feature matching with the previous key frame. For non-key frames, the optical flow algorithm is used to track the camera pose and associate data with the previous frame, and the solution results are accumulated to obtain the pose trajectory of each non-key frame.

[0013] Preferably, the steps further include: establishing a unified objective function that includes optical flow error and feature point error, and obtaining the optimal camera pose estimation by minimizing the objective function.

[0014] Preferably, the unified objective function is:

[0015]

[0016] The optical flow error is defined as:

[0017]

[0018] Feature point error is defined as:

[0019]

[0020] Where J represents the total number of non-keyframes, I represents the total number of keyframes, and M and N represent the number of feature points in the corresponding frames, respectively.

[0021] Preferably, the steps further include: constructing a local map of the scene using keyframe images, and calculating the point clouds of dynamically segmented objects and scenes to obtain a sparse point cloud map based on the segmentation results;

[0022] An incremental strategy is used to reconstruct sparse point clouds to obtain a global map.

[0023] Preferably, the steps for constructing the local map include:

[0024] Set up a sliding window to select keyframes and create a keyframe queue;

[0025] The EPnP algorithm is used to estimate the pose of candidate keyframes. The current frame is matched with the candidate keyframes through the BOW dictionary, and initialization is performed based on the matching relationship. Feature point triangulation is used to create 3D map points, unqualified feature points are removed, and local BA optimization is performed.

[0026] Preferably, the steps further include: performing loop closure detection based on the global map and performing global optimization.

[0027] Preferably, the inter-frame similarity is calculated based on the BOW vector. When three consecutive loop closure candidate keyframes have a high similarity with the current keyframe, the loop closure candidate frame is determined. For each candidate loop closure frame, its feature points are matched with those of the current frame, and the similarity transformation matrix is ​​solved using the corresponding three-dimensional information. If there are enough interior points in the loop closure frame, Sim(3) optimization is performed to optimize the global pose graph and correct the cumulative drift error.

[0028] Preferably, before performing pose estimation, keyframes are determined according to preset screening criteria; the screening criteria include one or more of the following:

[0029] Image quality is evaluated statistically by gradient magnitude and compared with a preset quality threshold to filter blurry images;

[0030] Set minimum and maximum interval thresholds respectively to form time interval limits;

[0031] Set camera position threshold and rotation angle threshold respectively to form motion threshold limit;

[0032] Set a feature overlap rate threshold so that new keyframes and historical keyframes meet the overlap rate requirements;

[0033] A map blind spot identification condition is constructed, and a keyframe supplementation strategy is introduced. A viewpoint coverage evaluation method is used to calculate the viewpoint overlap rate between the current keyframe and existing keyframes within the sliding window. By analyzing the distribution of observation counts of feature points under different viewpoints, sparsely observed areas are identified. When the area of ​​a region observed less than twice exceeds 40% of the total field of view, a map blind spot is determined to exist, and a supplementary keyframe is inserted.

[0034] Preferably, a pre-trained Mask R-CNN is used for instance segmentation: The loss function of Mask R-CNN is defined as:

[0035] Loss TotaL =Losscls +Loss bbox +Loss mask

[0036] Among them, Loss cls Loss bbox and Loss mask These represent classification error, detection error, and segmentation error, respectively.

[0037] As can be seen from the above technical solutions, compared with existing technologies, this invention discloses a dynamic scene robust visual SLAM method based on multi-feature collaborative optimization. It integrates feature matching and optical flow algorithms to differentiate between keyframes and non-keyframes, significantly improving computational efficiency and real-time performance while maintaining positioning accuracy. Based on differentiated processing, this invention performs nonlinear joint optimization of attitude trajectories using multi-feature collaboration, resulting in a significant improvement in trajectory estimation accuracy. This invention proposes a keyframe selection strategy that can adapt to environmental changes and update in real time, ensuring the accuracy of mapping. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the process of a dynamic scene robust visual SLAM method based on multi-feature collaborative optimization provided by the present invention.

[0040] Figure 2 This is a schematic diagram of the Mask R-CNN network in an embodiment of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] like Figure 1 and Figure 2 This invention discloses a dynamic scene robust visual SLAM method based on multi-feature collaborative optimization, comprising the following steps:

[0043] S1: Acquire image sequences and perform dynamic target detection and segmentation through an instance segmentation network to generate a segmentation mask containing dynamic region labels, and identify and separate dynamic objects from static backgrounds.

[0044] S2: Based on the segmentation mask, remove the feature points corresponding to dynamic objects to obtain static feature points.

[0045] S3: Based on static feature point pairs, pose estimation is performed. For key frames, the camera pose of each key frame is obtained by minimizing the reprojection error and performing feature matching with the previous key frame. For non-key frames, the optical flow algorithm is used to track the camera pose and associate data with the previous frame, and the solution results are accumulated to obtain the pose trajectory of each non-key frame.

[0046] In this embodiment, the present invention integrates a feature point fusion optical flow tracking algorithm and adopts a differentiated processing strategy: for key frames, a feature point tracking method is used to obtain static feature point pairs through ORB descriptor calculation and feature matching; for non-key frames, the Lucas-Kanade optical flow tracking algorithm is used to achieve fast motion estimation based on pixel-level brightness consistency constraints, avoiding the overhead of descriptor calculation. The synergistic combination of the two methods significantly reduces computation time and improves overall operating efficiency. Through the matching point pairs obtained by feature matching in key frames and the pixel correspondence obtained by optical flow tracking in non-key frames, the system can use these correspondences to perform pose calculation and obtain the initial camera pose estimate for each frame. Here, "initial pose" refers to the pose estimation result before multi-feature collaborative optimization, in contrast to the subsequent nonlinear joint optimization. The obtained camera pose includes the pose obtained by feature matching in key frames and the pose trajectory accumulated by optical flow tracking in non-key frames. The two are stitched together to form a complete camera motion trajectory, providing a foundation for subsequent global optimization.

[0047] To further implement the above technical solution, it also includes S4: Based on the pose estimation achieved by the feature point fusion optical flow tracking algorithm, the present invention performs multi-feature collaborative optimization of pose through a nonlinear joint optimization strategy, establishes a unified objective function including optical flow error and feature point error, and obtains the optimal camera pose estimation by minimizing the objective function.

[0048] Specifically, the unified objective function is:

[0049]

[0050] The optical flow error is defined as:

[0051]

[0052] Feature point error is defined as:

[0053]

[0054] Where J represents the total number of non-keyframes, I represents the total number of keyframes, and M and N represent the number of feature points in the corresponding frames, respectively.

[0055] The specific derivation process of the above error is as follows:

[0056] The optical flow method combines the template image T(x) with the input image I(x) and obtains the target state variable by solving and minimizing the objective function:

[0057] ∑ x [I(w(x;b+Δb))-T(x)] 2

[0058] Where b is a small transformation of the state variable, w(x; b) is the affine transformation function between the two frames, and x = (x, y). T Let b = (b1, b2, b3, b4, b5, b6) represent the set of pixel coordinates of the image. T Let w(x; b) be the target state variable, and its expression is given by the following formula:

[0059]

[0060] The coordinates of the m-th image pixel in the transformed non-keyframe j are x j,m =(x j,m ,y j,m Then, based on the camera model, we can obtain:

[0061] sx j =kexp(ζ^)p j

[0062] Where s represents the depth of the point, k represents the intrinsic parameters of the camera, ζ represents the Lie group form of the camera position, and p represents the 3D coordinates obtained through projection.

[0063] Therefore, the error equation for non-keyframe j is obtained as follows:

[0064]

[0065] Where M represents the total number of optical flow points in non-keyframe j.

[0066] For keyframes, the reprojection error is used as the feature point error to formulate the objective function, and the camera pose is described using Lie algebras, satisfying the following relationship:

[0067]

[0068] Wherein, spatial point p n The coordinates are [X n Yn Z n ] T The pixel coordinates of the corresponding projection point are [u n v n 1] T

[0069] The feature point error of keyframe i is:

[0070]

[0071] Where N represents the total number of feature points in keyframe i.

[0072] Establish a nonlinear joint optimization process, with the unified objective function as follows:

[0073]

[0074] The optimal camera pose can be solved as follows:

[0075]

[0076] Here, J and I represent the total number of non-keyframes and keyframes, respectively, and the camera pose parameters are solved by Lie group optimization.

[0077] In this embodiment, the present invention establishes a pose optimization objective function with multi-error fusion through the matching feature point pairs obtained in step S3, unifies the modeling of optical flow tracking error and feature point reprojection error, and uses the Lie group SE(3) geometric constraint optimization framework to solve for the optimal camera pose parameters, thereby obtaining accurate camera pose estimation. The specific steps are as follows:

[0078] A joint error optimization objective function is established, which weights and fuses the optical flow error term of non-key frames and the feature point reprojection error term of key frames. The optical flow error term describes the motion continuity between adjacent frames through pixel-level brightness consistency constraints, while the feature point error term measures the geometric accuracy of pose estimation through the geometric projection relationship of 3D map points onto the 2D image plane.

[0079] The camera pose is parameterized using the Lie group SE(3). The rotation-constrained optimization problem is transformed into an unconstrained optimization problem by utilizing the linear property of the tangent space of the Lie algebra se(3). The bidirectional transformation between the Lie group manifold and the Lie algebra vector space is realized through the exponential mapping exp: se(3) → SE(3) and the logarithmic mapping log: SE(3) → se(3).

[0080] The Levenberg-Marquardt algorithm is used for iterative solution, and the damping factor λ is adaptively adjusted according to error changes. The weight allocation strategy is dynamically adjusted according to the observation quality, with high-quality feature points receiving larger weights: wi = 1 / (1 + ||ri|| 2), where ri is the reprojection error. The optimization process adopts a multi-level strategy, first performing a coarse pose estimation, and then gradually refining the parameters, usually converging in 3-5 iterations.

[0081] Furthermore, based on the above, this invention proposes S5: constructing a local map and S6: loop closure detection. In this embodiment, this invention adopts a multi-threaded parallel architecture, mainly including four core modules: object detection thread, tracking thread, local mapping thread, and loop closure detection thread.

[0082] After the raw input is processed by the global feature detector, it enters multiple parallel processing modules simultaneously. The object detection thread is responsible for instance segmentation of dynamic objects, providing dynamic masks for subsequent feature selection; the feature extraction module extracts ORB feature points from static regions; the tracking thread performs pose estimation using a feature point fusion optical flow tracking algorithm; finally, multi-feature collaborative optimization is performed by initializing the pose and the tracking results; the local mapping thread maintains the consistency of the local map; and the loop closure detection thread performs global trajectory optimization.

[0083] To further implement the above technical solution, based on the precise pose parameters output by S4 multi-feature collaborative optimization, the local mapping thread initiates the 3D map reconstruction process:

[0084] S51: Sliding Window Management: Sets up a sliding window to maintain active keyframes, forming a keyframe queue. After S3 completes the keyframe selection and confirms the new keyframe, it is added to the queue and local map optimization is triggered.

[0085] S52: Initial Pose Estimation: After multi-feature collaborative optimization in S4, the pose of the new keyframe is refined using the EPnP algorithm, and the refined result is used as the initial pose. The EPnP algorithm can obtain an accurate estimate of the camera pose by solving a system of linear equations based on known 3D-2D point correspondences. These 3D-2D correspondences are derived from existing map points observed in the current keyframe.

[0086] S53: Establishing Co-view Relationships: The similarity between the current keyframe (i.e., the new keyframe) and historical keyframes in the keyframe queue is calculated using the BOW dictionary. Keyframe pairs with shared observation regions are identified, and feature point matching relationships based on visual vocabulary are established. This feature matching (denoted as Matching Process A) is specifically used for establishing co-view relationships in local mapping, differing from the feature matching used for pose determination in S3 (denoted as Matching Process B) in both purpose and method: Matching Process A: Coarse matching based on the BOW dictionary quickly identifies co-view keyframe pairs, providing inter-frame correlations for subsequent 3D reconstruction. Matching Process B: Precise feature matching based on minimizing reprojection error is used for precise camera pose determination.

[0087] S54: Stereo Matching and 3D Reconstruction: Stereo matching is performed between keyframes that satisfy geometric constraints based on matched static feature point pairs.

[0088] The geometric constraints include: baseline constraint: b ≥ 0.1m, where b is the baseline distance between two keyframes; viewpoint constraint: θ ∈ [5°, 60°], where θ is the difference in observation angles.

[0089] Calculate the coordinates of points on a 3D map using triangulation algorithms:

[0090] p * =argminΣ||x_i-π(k_i,p)|| 2

[0091] Where p * Let x_i be the coordinates of the 3D map point to be solved; x_i is the pixel coordinate of the feature point in the i-th keyframe; π(k_i,p) represents the projection function of the 3D point p in the keyframe k_i; k_i is the camera intrinsic and extrinsic parameter matrix of the i-th keyframe.

[0092] Step 5: Map Point Quality Screening: Reliable 3D points that meet preset criteria are selected. The quality evaluation standards include: ① Reprojection error (err) < 2.0 pixels; ② Number of observations (nobs) ≥ 3; ③ Observation angle range ≥ 10°. Map points that do not meet the criteria are promptly removed, while 3D map points that meet the criteria proceed to the next optimization step.

[0093] S56: Local Bundle Adjustment Optimization: The Local Bundle Adjustment algorithm simultaneously optimizes the pose of all keyframes within the sliding window and the coordinates of 3D map points that meet quality requirements, minimizing the overall reprojection error, and finally obtaining 3D map points that meet the conditions and are globally optimal.

[0094] Sparse point cloud reconstruction employs an incremental strategy, gradually adding reliable 3D map points, optimized through local bundle adjustment, to the global map. The system maintains an octree index structure for the point cloud, supporting fast neighborhood queries and real-time visualization. Map management mechanisms include redundancy detection (merging duplicate points less than 0.05m apart), continuous quality assessment, and memory control (limiting the number of global map points to 50,000).

[0095] The relocalization mechanism is based on the bag-of-words model, pre-constructing a dictionary containing 1000 visual words. When tracking fails, the system converts the current frame into a bag-of-words vector and filters candidate relocalization frames using cosine similarity calculation. In the geometric verification stage, the EPnP algorithm is used to verify candidate frames; relocalization is confirmed as successful when the number of inliers is greater than 20.

[0096] Furthermore, S6 utilizes loop closure detection for global trajectory optimization, eliminates long-term accumulated drift errors by identifying historical access scenarios, and combines global pose graph optimization techniques to correct the entire camera trajectory, obtaining globally consistent and accurate camera pose estimation. The specific steps are as follows:

[0097] A visual similarity retrieval mechanism is established based on the bag-of-words model. The current keyframe is converted into a bag-of-words vector representation and compared with a historical keyframe database. When the bag-of-words similarity of three consecutive frames exceeds a preset threshold T, the similarity is determined. loop When =0.6, the existence of loop closure candidates is confirmed; strict geometric consistency verification is performed on the candidate loop closure frames, the correspondence between feature points between the current frame and the candidate frame is established by ORB feature matching algorithm, and the similarity transformation matrix Sim(3) is calculated by RANSAC robust estimation algorithm. The number of inliers>30 and the feature distribution uniformity index U>0.7 are used as the dual criteria for loop closure confirmation.

[0098] After loop closure confirmation, a global pose graph optimization model is constructed. Using the g2o graph optimization library, a graph structure is established with keyframe poses as nodes and inter-frame geometric constraints as edges. A joint optimization objective function is then established, which includes the original odometry constraints and the newly added loop closure constraints.

[0099] X * =Σe T ij Ω ij e ij

[0100] Where X * Optimized global pose parameters, e ij For inter-frame pose error, Ω ij This is an information matrix.

[0101] By optimizing the global graph synchronously, all keyframe pose parameters are adjusted, and the optimized trajectory is smoothed to eliminate abrupt changes, ultimately achieving globally consistent high-precision camera pose estimation and effective elimination of accumulated errors.

[0102] To further implement the above technical solution, S1 uses the Mask R-CNN instance segmentation network to perform pixel-level precise segmentation of detected moving objects.

[0103] Specifically, the object detection thread receives the image stream input and uses a pre-trained Mask R-CNN model to perform instance segmentation on keyframes. The backbone network employs a ResNet-50 deep convolutional neural network to extract multi-layer features from the input image. The feature pyramid network connects the outputs of different layers of the backbone network to generate multi-scale feature maps. The region proposal network generates candidate regions for objects based on the feature maps. The ROI alignment module performs accurate feature extraction on the candidate regions. The classification and regression head performs object classification and bounding box regression on the extracted features, outputting the detection results. The mask segmentation head processes the features in parallel, generating a pixel-level segmentation mask for each detected object.

[0104] The loss function of Mask R-CNN is defined as a weighted combination of three sub-tasks: the classification loss uses the cross-entropy function to evaluate the accuracy of target category prediction, the bounding box regression loss uses the Smooth L1 function to optimize target localization accuracy, and the segmentation loss uses the binary cross-entropy function to ensure pixel-level segmentation quality. Through end-to-end training, the network can simultaneously complete target detection, classification, and pixel-level segmentation tasks.

[0105] To ensure the adaptability of the segmentation results to dynamic environments, in the object detection thread, Mask-RCNN performs instance segmentation on each frame of the input image to ensure the mask integrity and semantic consistency of moving targets (such as pedestrians and vehicles). The segmentation results are passed to other threads through a shared memory mechanism.

[0106] Furthermore, after the dynamic mask is generated, S2 preprocesses the image to mark feature points in the dynamic region as invalid, ensuring that subsequent processing only uses reliable features from the static background region.

[0107] To further implement the above technical solution, when using feature point fusion optical flow tracking in S3, it is determined whether the current frame is a key frame. If it is not a key frame, the optical flow algorithm is used to track the camera pose and associate data, and the solution results are accumulated. If it is a key frame, the camera pose is solved by minimizing the reprojection error and performing feature matching with the previous key frame.

[0108] In this embodiment, the keyframe selection mechanism comprehensively considers multiple factors: ① Image quality assessment is achieved through gradient magnitude statistics G=∑|▽I| 2 Evaluation, setting a quality threshold T q Filtering blurry images; ② Time interval control to avoid keyframes being too dense or too sparse, setting a minimum interval T. min =0.25 seconds and maximum interval T max= 15 frames; ③ Motion threshold is based on the changes in camera displacement ||t|| and rotation angle |θ|. Keyframe insertion is triggered when ||t||>0.1m or |θ|>10°; ④ Common-view feature point analysis ensures that the new keyframe has sufficient feature overlap with the existing keyframe (overlap rate>30%); ⑤ Local map coverage assessment avoids map blind spots.

[0109] Furthermore, for non-keyframe systems, the Lucas-Kanade optical flow algorithm is employed for fast tracking. Based on the assumption of constant brightness and spatial consistency constraints, the algorithm minimizes the photometric error function through iterative optimization. Optical flow tracking avoids the computational overhead of descriptors, significantly improving processing speed. Simultaneously, it utilizes the pose information from the previous frame for motion prediction, narrowing the search range and enhancing tracking stability.

[0110] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0111] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dynamic scene robust visual SLAM method based on multi-feature collaborative optimization, characterized in that, Includes the following steps: Image sequences are acquired and dynamic target detection and segmentation are performed through an instance segmentation network to generate a segmentation mask containing dynamic region labels, thereby identifying and separating dynamic objects from static backgrounds. Based on the segmentation mask, feature points corresponding to the dynamic object are removed to obtain static feature points; Pose estimation is performed based on the static feature point pairs. For key frames, the camera pose of each key frame is obtained by minimizing the reprojection error and performing feature matching with the previous key frame. For non-key frames, the optical flow algorithm is used to track the camera pose and associate data with the previous frame, and the solution results are accumulated to obtain the pose trajectory of each non-key frame.

2. The dynamic scene robust visual SLAM method based on multi-feature collaborative optimization according to claim 1, characterized in that, The steps also include: establishing a unified objective function that includes optical flow error and feature point error, and obtaining the optimal camera pose estimation by minimizing the objective function.

3. The dynamic scene robust visual SLAM method based on multi-feature collaborative optimization according to claim 2, characterized in that, The unified objective function is: The optical flow error is defined as: Feature point error is defined as: Where J represents the total number of non-keyframes, I represents the total number of keyframes, M and N represent the number of feature points in the corresponding frames, and J(ζ) is the total loss. For optical flow error, For feature point error; x j，m Let be the coordinates of the m-th image pixel of non-keyframe j after affine transformation; s represents the depth of the point, k represents the intrinsic parameters of the camera, and ζ represents the Lie group form of the camera pose; Let m be the 3D coordinates of the m-th optical flow point in the j-th non-keyframe. Let be the 3D coordinates of the nth feature point in the i-th keyframe.

4. The dynamic scene robust visual SLAM method based on multi-feature collaborative optimization according to claim 1, characterized in that, The steps also include: constructing a local map of the scene using keyframe images, and calculating the point clouds of dynamically segmented objects and scenes to obtain a sparse point cloud map based on the segmentation results; An incremental strategy is used to reconstruct sparse point clouds to obtain a global map.

5. A dynamic scene robust visual SLAM method based on multi-feature collaborative optimization according to claim 4, characterized in that, The steps for constructing the local map include: Sliding window management: Set up a sliding window to select keyframes and create a keyframe queue; listen for the addition of new keyframes; Pose initialization and feature matching: When a new keyframe is detected, the initial pose of the keyframe is estimated using the EPnP algorithm; The similarity between the current keyframe and historical keyframes in the keyframe queue is calculated using the BOW dictionary, and a feature point matching relationship based on visual vocabulary is established. 3D Reconstruction and Quality Control: Based on matched static feature point pairs, stereo matching is performed between key frames that meet geometric constraints; The coordinates of points on the 3D map are calculated using triangulation algorithms; reliable 3D points that meet the preset conditions are selected based on quality evaluation criteria such as reprojection error, number of observations, and observation angle range. Local optimization: The pose of all keyframes and the coordinates of the filtered 3D map points within the sliding window are optimized simultaneously through the local bundle adjustment algorithm, minimizing the overall reprojection error.

6. A dynamic scene robust visual SLAM method based on multi-feature collaborative optimization according to claim 4 or 5, characterized in that, The steps also include: performing loop closure detection based on the global map and performing global optimization.

7. A dynamic scene robust visual SLAM method based on multi-feature collaborative optimization according to claim 6, characterized in that, Frame similarity is calculated based on BOW vectors. When three consecutive loop closure candidate keyframes have high similarity with the current keyframe, the loop closure candidate frame is determined. For each candidate loop closure frame, its feature points are matched with those of the current frame, and the similarity transformation matrix is ​​solved using the corresponding 3D information. If there are enough interior points in the loopback frame, perform Sim(3) optimization to optimize the global pose graph and correct the accumulated drift error.

8. A dynamic scene robust visual SLAM method based on multi-feature collaborative optimization according to claim 1, characterized in that, Before performing pose estimation, keyframes are determined based on preset filtering criteria; the filtering criteria include one or more of the following: Image quality is evaluated statistically by gradient magnitude and compared with a preset quality threshold to filter blurry images; Set minimum and maximum interval thresholds respectively to form time interval limits; Set camera position threshold and rotation angle threshold respectively to form motion threshold limit; Set a feature overlap rate threshold so that new keyframes and historical keyframes meet the overlap rate requirements; We construct map blind spot identification conditions and introduce a keyframe interpolation strategy.

9. A dynamic scene robust visual SLAM method based on multi-feature collaborative optimization according to claim 1, characterized in that, Instance segmentation is performed using a pre-trained Mask R-CNN: The loss function of Mask R-CNN is: Loss TotaL =Loss cls +Loss bbox +Loss mask Loss cls Loss bbox and Loss mask These represent classification error, detection error, and segmentation error, respectively.

Images