RGB-D visual SLAM method for indoor dynamic scene

Abstract

At present, most researches on a visual synchronous localization and mapping (VSLAM) system are established on a static scene, and a dynamic target in a real scene can seriously influence the accuracy and robustness of the VSLAM system, so that the invention provides an RGB-D visual SLAM algorithm for an indoor dynamic scene. The method comprises the following steps: (1) fusing a lightweight YOLOv8 target detection network in an ORB-SLAM3 tracking thread, performing instance segmentation on an image frame, and generating semantic information; (2) processing the semantic information by using a sliding window mechanism and a template matching strategy, identifying a dynamic target and removing dynamic feature points; and (3) a PROSAC algorithm is adopted to replace an original RANSAC algorithm to remove mismatching of feature points, and the matching performance is improved. In order to evaluate the effectiveness of the method, an experiment is carried out on a disclosed TUM RGB-D data set, and compared with an original ORB-SLAM3 and a relatively advanced dynamic DynaSLAM algorithm, the method has the advantages that the algorithm positioning precision and robustness in a dynamic environment are greatly improved, the frame rate can reach 45 fps or above under the condition of no GPU acceleration, and real-time operation can be ensured.

Description

Technical Field

[0001] The invention belongs to the technical field of visual SLAM (Simultaneous Localization and Mapping), and in particular relates to an RGB-D visual SLAM method for indoor dynamic scenes. Background Art

[0002] Simultaneous localization and mapping (SLAM) is a function in which a robot uses a single sensor or multiple sensors to build a model of the surrounding environment and estimate its own position after entering an unknown scene, thereby achieving the function of localization and mapping of the unknown environment. Currently, SLAM technology supports applications such as virtual reality (AR), autonomous driving, drones, and mobile robots. Visual SLAM based on camera sensors has attracted widespread attention from researchers because of its effective use of rich scene information and low cost. Currently, research on visual SLAM is mainly carried out under the assumption of static scenes. Visual SLAM algorithms have formed a very mature theoretical system and system architecture in static environments, such as DSO, SVO, ORB-SLAM2, and ORB-SLAM3. However, dynamic objects in real scenes will seriously affect the feature matching between image frames. This interference will lead to the loss of feature tracking and serious deviations in posture estimation in the visual odometer, thereby affecting the positioning accuracy and robustness of the system. Therefore, researchers have proposed various methods to meet this challenge.

[0003] At present, the visual SLAM methods used to eliminate the influence of dynamic objects are mainly divided into two types: geometric methods and deep learning methods. Geometry-based dynamic visual SLAM aims to remove dynamic feature points by accurately estimating motion and tracking feature points. Geometric methods usually have good real-time performance and low resource consumption, and can eliminate dynamic feature points to a certain extent, but the posture estimation in high-dynamic scenes may drift and the positioning accuracy is low. Dynamic visual SLAM based on deep learning uses semantic segmentation networks such as Mask-RCNN and YOLO to obtain prior semantic information, which can effectively segment dynamic objects in the scene. Although these methods perform well in terms of positioning accuracy, their high dependence on semantic information requires a lot of computing resources, which greatly affects the real-time performance of the system. Summary of the invention

[0004] In view of the problems of low positioning accuracy, large amount of calculation, excessive feature point elimination and poor real-time performance of the current SLAM algorithm in dynamic environment, the present invention uses lightweight YOLOv8 target detection network instance segmentation to provide semantic prior information, and utilizes sliding window mechanism and template matching to process semantic information and eliminate dynamic features. Finally, PROSAC algorithm is used to replace the original RANSAC algorithm to remove false matches, improve feature matching accuracy and speed, thereby improving the positioning accuracy and robustness of the SLAM system in dynamic environment.

[0005] To achieve the above object and other related objects, the present invention provides an RGB-D visual SLAM method for indoor dynamic scenes, the method comprising the following steps:

[0006] Step 1: The target detection thread uses the YOLOv8 network to perform target detection on the RGB-D image to generate masks and semantic information, which are then fed back to the tracking thread.

[0007] 1-1 Input the image frames of the RGB-D camera into the YOLOv8 network.

[0008] 1-2 The network generates bounding boxes, masks, class labels, and confidence scores around detected objects.

[0009] 1-3 provides this semantic information to the tracking thread as the key to subsequently distinguishing between dynamic and static objects.

[0010] Step 2: The tracking thread extracts ORB feature points from the input RGB-D image, calculates the relative pose and key frame selection between the current frame and the map, and uses the constant velocity model for tracking. The sliding window mechanism and template matching strategy are used to identify and eliminate dynamic features.

[0011] 2-1 Extract the ORB features of the image and encapsulate the image into a frame data structure for processing.

[0012] 2-2 uses a sliding window mechanism, and the window stores continuous frames of fixed length.

[0013] 2-3 Based on the semantic information and target frame obtained by the target detection thread, template matching is performed on the same target object in the two frames. If the current matching score reaches the threshold, it is judged as a dynamic object.

[0014] 2-4 Based on the prior mask information, the feature points on this dynamic object are removed, leaving only static feature points for camera pose estimation.

[0015] 2-5 Select the corresponding tracking model to solve the initial position and posture of the camera, which is divided into constant speed motion tracking model and reference frame tracking model. The constant speed motion tracking model is used in normal tracking mode, and the reference frame tracking model is mainly used for repositioning after tracking failure.

[0016] 2-6 Track the local map to explore more feature matching relationships between local map points and the current frame image, and then further optimize the camera pose based on the graph optimization model.

[0017] 2-7 Determine whether it is necessary to generate a key frame based on the key frame selection conditions, and pass the key frame to the local mapping thread.

[0018] Step 3: Use the PROSAC algorithm to eliminate feature mismatches.

[0019] 3-1 Sort the initial feature matching points according to the ratio β of the Euclidean distance. The smaller the ratio β of the Euclidean distance, the higher the image matching efficiency. β = d min / d min2 , where d min is the minimum Euclidean distance, d min2 It is the sub-minimum Euclidean distance. The better the quality of the feature matching points, the higher the matching success rate.

[0020] 3-2 The assumed sample set w and sample size n are determined by the matching point set X obtained by the feature matching algorithm.

[0021] 3-3 Form an initial sample from 3 pairs of matching points in the hypothetical sample set w, and then calculate the homography matrix H.

[0022] 3-4 Check that all initial matching points in images i1 and i2 are inliers of H. Assume that x and x' are a pair of matching points and ε is the matching error threshold. ‖Hx-x'‖ 2 ≤ε, the matching point is an inlier point, and then the inlier point is calculated.

[0023] 3-5 In order to determine whether the algorithm meets the termination condition of the algorithm, it returns to the third step and re-traverses the sample set w. If the termination condition of the termination algorithm is still not met, the sample set w increases by n+1 pairs of matching points, the sample size n increases by 1, and step 2 is repeated.

[0024] Step 4: The local mapping thread adds keyframes to the current active map, deletes redundant keyframes and map points, and uses local BA to optimize the map.

[0025] 4-1 Receive new key frames from the tracking thread.

[0026] 4-2 Process the new keyframe, calculate the bag-of-words descriptor for the keyframe and insert it into the map.

[0027] 4-3 Triangulate to generate new map points and check and merge duplicate map points.

[0028] 4-4 Perform local BA optimization to further optimize the key frame pose and map point spatial coordinates.

[0029] 4-5 Determine the redundancy of key frames and delete redundant key frames.

[0030] 4-6 Send the processed key frame to the loop detection thread.

[0031] Step 5: Perform loop detection and loop optimization by querying the bag-of-visual-words database, calculating Euclidean transformation, loop fusion, and essential graph optimization.

[0032] 5-1 Find candidate closed-loop frames in the key frame database based on bag-of-words technology.

[0033] 5-2 The current key frame and the candidate closed-loop frame are feature matched, and the Sim3 transformation of the closed-loop key frame and the current key frame is solved and optimized. Then a continuity check is performed, that is, projection matching is performed on the three common view frames of the current frame and the candidate closed-loop frame. If the number of feature matches meets the conditions, the loop is detected.

[0034] 5-3 Use pose propagation to correct the key frame pose and three-dimensional coordinates of local map points, and perform map point fusion and essential graph optimization.

[0035] 5-4 performs global BA to eliminate the accumulated trajectory error.

[0036] Compared with the prior art, the advantages of the present invention are:

[0037] A visual SLAM technology for indoor dynamic scenes uses lightweight YOLOv8 target detection network instance segmentation to provide semantic prior information, and uses sliding window mechanism and template matching to process semantic information, remove dynamic features, and then uses PROSAC algorithm to replace the original RANSAC algorithm to remove mismatches, improve feature matching speed and accuracy, thereby improving the positioning accuracy and robustness of the visual SLAM system in dynamic environments. Experiments conducted on the public data set TUM RGB-D data set show that compared with the original ORB-SLAM3 and the more advanced dynamic DynaSLAM algorithm, the algorithm positioning accuracy and robustness in dynamic environments have been significantly improved. In the absence of GPU acceleration, the frame rate can reach more than 45fps, which can ensure real-time operation. This invention avoids the shortcomings of over-reliance on deep learning, significantly improves the accuracy and real-time performance of the visual SLAM system, and provides future intelligent systems with more powerful and reliable environmental perception capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to further illustrate the contents described in the present invention, the specific embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. It should be understood that these drawings are only used as typical examples and should not be regarded as limiting the scope of the present invention.

[0039] Figure 1 System block diagram

[0040] Figure 2 This is the YOLOv8 detection map

[0041] Figure 3 Template matching result map

[0042] Figure 4 It is a dynamic feature removal map

[0043] Figure 5 RANSAC and PROSAC remove mismatched images

[0044] Figure 6 This is a comparison of the actual trajectory and estimated trajectory of the present invention and ORB-SLAM3

[0045] Figure 7 This is a comparison chart of the actual trajectory and estimated trajectory of the present invention and DynaSLAM DETAILED DESCRIPTION

[0046] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. The specific embodiments of the present invention are described below:

[0047] The embodiment of the present invention provides an RGB-D visual SLAM method for indoor dynamic scenes, and the specific implementation includes the following steps:

[0048] Figure 1 The system block diagram of the RGB-D visual SLAM method for indoor dynamic scenes is as follows:

[0049] 1. In step 1, YOLOv8 is used to detect the target in the image and generate mask and semantic information, as follows:

[0050] YOLOv8 provides a new SOTA model, which consists of an input layer, a backbone network, a fusion layer, and a detection head, which can simultaneously perform target detection and instance segmentation tasks. Therefore, we use YOLOv8 as the target detection algorithm for the SLAM system. The algorithm can achieve a detection rate of more than 30fps and has high detection accuracy. By integrating deep learning-based object detection into the visual odometry process, the overall system performance and real-time capabilities can be improved.

[0051] 1-1 As Figure 2 As shown, the public TUM RGB-D dataset is used as the input of the YOLOv8 target detection network.

[0052] 1-2 The network generates semantic information such as bounding boxes, masks, category labels, and confidence scores around the detected objects. The bounding box information of the object may include the coordinates of the upper left corner (x1, y1) and the coordinates of the lower right corner (x2, y2), and the output is saved in cv::mat format. Saving in cv::mat format can save memory space and speed up inference.

[0053] 1-3 provides semantic information to the tracking thread as the key to subsequently distinguishing dynamic and static objects.

[0054] 2. In step 2, the tracking thread processes the image and semantic information as follows:

[0055] 2-1 Extract the ORB features of the image and encapsulate the image into a frame data structure for processing.

[0056] 2-2 adopts a sliding window mechanism, and the window stores continuous frames of fixed length. As the frames progress, new frames are stored in the window, and old frames are gradually deleted, and the images are compared at regular intervals. Figure 4 The specific process of combining the sliding window mechanism and template matching for detecting dynamic target objects is shown.

[0057] 2-3 How to judge the motion state of the target object is the key to improving the positioning accuracy of VSLAM in dynamic environments. The present invention compares the similarity of the same area of ​​the image over a period of time. If the area is similar, the object in the area is marked as static, and the motion state of the object is estimated in real time using a template matching algorithm. Template matching is a technology that finds the most matching (similar) part of another template image in an image. Based on the method of measuring similarity, different template matching methods have emerged, such as variance square matching, correlation matching, standard correlation matching, etc. The present invention adopts standard correlation coefficient matching to minimize the influence of illumination on the calculation results. The standard correlation coefficient can be calculated as follows.

[0058]

[0059] Based on the semantic information and target frame obtained by the target detection thread, template matching is performed on the same target object in the two frames. If the current matching score reaches the threshold, it is judged as a dynamic object.

[0060] 2-4 sets the pixel value corresponding to the dynamic ORB feature point on the dynamic object to 1, and the pixel value corresponding to the static ORB feature point outside the dynamic object to 0, and resets the coordinates at the pixel value of 1 to (-1, -1). In the SLAM system, the point with the coordinate (-1, -1) is regarded as an error point and is eliminated. Only the static feature points are left for camera pose estimation, which effectively improves the positioning accuracy of the system.

[0061] 2-5 Select the corresponding tracking model to solve the initial position and posture of the camera, which is divided into constant speed motion tracking model and reference frame tracking model. The constant speed motion tracking model is used in normal tracking mode, and the reference frame tracking model is mainly used for repositioning after tracking failure.

[0062] 2-6 Track the local map to explore more feature matching relationships between local map points and the current frame image, and then further optimize the camera pose based on the graph optimization model.

[0063] 2-7 Determine whether it is necessary to generate a key frame based on the key frame selection conditions, and pass the key frame to the local mapping thread.

[0064] 3. In step 3, the PROSAC algorithm is used to eliminate false feature matching, as follows:

[0065] In order to solve the problems of poor stability and multiple mismatched points in image registration, most researchers use the random sample consensus (RANSAC) algorithm to optimize the matching algorithm. RANSAC regards dynamic feature points as outliers and static feature points as inliers, that is, points with reliability, randomly generates hypotheses, iteratively optimizes the best hypothesis, and classifies points into inliers and outliers according to the maximum consensus set. However, when there are many dynamic points, the inlier rate is relatively low, resulting in poor performance of RANSAC. Progressive consistent sampling (PROSAC) is an optimization of sampling in the classic RANSAC. Compared with the classic RANSAC method that samples uniformly from the entire set, the PROSAC method samples from a continuously increasing set of the best corresponding points. Therefore, the PROSAC algorithm can reduce the randomness of sampling, reduce the number of iterations, and improve the accuracy of the model.

[0066] The PROSAC process of eliminating mismatches is as follows:

[0067] 3-1 Sort the initial feature matching points according to the ratio β of the Euclidean distance. The smaller the ratio β of the Euclidean distance, the higher the image matching efficiency. β = d min / d min2 , where d min is the minimum Euclidean distance, d min2 It is the sub-minimum Euclidean distance. The better the quality of the feature matching points, the higher the matching success rate.

[0068] 3-2 The assumed sample set w and sample size n are determined by the matching point set X obtained by the feature matching algorithm.

[0069] 3-3 Form an initial sample from 3 pairs of matching points in the hypothetical sample set w, and then calculate the homography matrix H.

[0070] 3-4 Check that all initial matching points in images i1 and i2 are inliers of H. Assume that x and x' are a pair of matching points and ε is the matching error threshold. ‖Hx-x'‖ 2 ≤ε, the matching point is an inlier point, and then the inlier point is calculated.

[0071] 3-5 In order to determine whether the algorithm meets the termination condition of the algorithm, it returns to the third step and re-traverses the sample set w. If the termination condition of the termination algorithm is still not met, the sample set w increases by n+1 pairs of matching points, the sample size n increases by 1, and step 2 is repeated.

[0072] In order to verify the effect of the algorithm based on PROSAC to remove mismatched points, this paper selects the image frames of the TUM data set to conduct experimental simulation verification of the PROSAC algorithm and the traditional RANSAC algorithm. The simulation results are as follows: Figure 5 Green indicates correct matches, and red indicates incorrect matches. Both can eliminate incorrect matches, but the PROSAC algorithm can eliminate incorrectly matched feature points more effectively, especially when there is a lot of interference information in the scene. The traditional RANSAC algorithm has low accuracy, and the PROSAC algorithm performance is still very stable.

[0073] Taking the inlier rate and matching time as evaluation indicators, as shown in Table 1, the difference between RANSAC and PROSAC in the inlier rate is not much, and PROSAC is slightly better. However, in terms of matching time, since PROSAC samples from a continuously increasing set of optimal corresponding points, it effectively reduces the number of system iterations, which helps to improve the real-time and stability of visual SLAM.

[0074] Table 1 RANSAC and PROSAC eliminate mismatching effects

[0075]

[0076] 4. In step 4, the local map building thread is used to maintain the local map, as follows:

[0077] The local mapping thread calculates the map points for the incoming keyframe, removes duplicate and old map points from the local keyframe window, and then performs local BA to optimize the camera pose and map points in the current keyframe window.

[0078] 4-1 Receive new key frames from the tracking thread.

[0079] 4-2 Process the new keyframe, calculate the bag-of-words descriptor for the keyframe and insert it into the map.

[0080] 4-3 Manage local map points. Newly added map points need to pass strict inspection conditions before they can be truly saved. Map points that fail the test will be deleted to reduce the impact of external points. And triangulation is performed. Only map points that meet the conditions of parallax verification can be added to the map.

[0081] 4-4 Perform local BA optimization to further optimize the key frame pose and map point spatial coordinates.

[0082] 4-5 Determine the redundancy of key frames and delete redundant key frames.

[0083] 4-6 Send the processed key frame to the loop detection thread.

[0084] 5. In step 5, the loop detection thread is used to check the loop, as follows:

[0085] Whenever a new keyframe is selected, location recognition is started to match the keyframe with the keyframe in the Atlas map set. If the match is successful, it will be divided into two cases: (1) The loop frame is a keyframe in the current active map, loop correction is performed, and global BA optimization is started to obtain a globally consistent map; (2) The loop frame is a keyframe in the inactive map, the two maps are merged and the merged map is set as the current active map.

[0086] 5-1 Find candidate closed-loop frames in the key frame database based on bag-of-words technology

[0087] 5-2 The current key frame and the candidate closed-loop frame are feature matched, and the Sim3 transformation of the closed-loop key frame and the current key frame is solved and optimized. Then a continuity check is performed, that is, projection matching is performed on the three common view frames of the current frame and the candidate closed-loop frame. If the number of feature matches meets the conditions, the loop is detected.

[0088] 5-3 Use pose propagation to correct the key frame pose and three-dimensional coordinates of local map points, and perform map point fusion and essential graph optimization.

[0089] 5-4 performs global BA to eliminate the accumulated trajectory error and improve the global consistency of the system.

[0090] Embodiment 1:

[0091] This embodiment verifies the effect of the present invention by conducting an experimental comparison on the TUM RGB-D dataset. The computer hardware environment for the experimental test verification is Intel(R) Core(TM) i7-6700CPU, with a main frequency of 3.4GHz and 8G memory, without GPU acceleration, and the software operating environment is Ubuntu18.04 system.

[0092] The TUM dataset is a widely used dataset in the field of computer vision, created by the Computer Vision Group of the Technical University of Munich. The TUM RGB-D dataset combines color and depth information, aiming to promote the research and development of tasks such as 3D reconstruction of indoor scenes, SLAM and object recognition. The selected dataset contains 4 low-dynamic scene f3_s series and 4 high-dynamic scene f3_w series. In order to quantitatively evaluate the performance of the SLAM system, the present invention selects the widely used indicator absolute trajectory error (ATE) to evaluate the accuracy of the method. ATE is the direct difference between the estimated pose and the true pose, reflecting the overall accuracy and global consistency of the trajectory. The root mean square error (RMSE), mean error (MeanError), and standard deviation (StandardDeviation, std) values ​​are used to evaluate the accuracy and robustness of the system.

[0093] From the data in Table 2, it can be seen that compared with the original ORB-SLAM3, the RMSE of the algorithm of the present invention in low dynamic scenes is increased by more than 50% on average, and the RMSE in high dynamic scenes is increased by more than 95% on average, with a maximum increase of 98.86%, and the performance on dynamic data sets has been greatly improved. In addition, from the data in Table 3, compared with DynaSLAM, the absolute trajectory error of the algorithm of the present invention in high dynamic data sets is increased by 34.29%, 33.77%, and 36.36% respectively. It can be seen that the algorithm of the present invention has better positioning accuracy than DynaSLAM.

[0094] Table 2 Comparison of absolute trajectory error (ATE) between the present invention and ORB-SLAM3

[0095]

[0096]

[0097] Table 3 Comparison of absolute trajectory error (ATE) between the present invention and DynaSLAM

[0098]

[0099] Figure 6 The comparison of the coincidence between the estimated trajectory and the real trajectory of the algorithm of the present invention and ORB-SLAM3 under 8 dynamic data sets is shown. The black represents the real trajectory, the green represents the trajectory of the original ORB-SLAM3, and the blue represents the trajectory of the algorithm of the present invention. It can be seen that the trajectory of ORB-SLAM3 is poorly consistent and there is a lot of drift, while the trajectory of the algorithm of the present invention is very consistent with the real trajectory on all data set sequences, and the trajectory error is significantly smaller than the trajectory error of ORB-SLAM3, indicating that the positioning accuracy of the algorithm of the present invention in dynamic scenes is higher, and the accuracy and robustness of the SLAM system in dynamic scenes are effectively improved.

[0100] Figure 7 The comparison of the overlap between the estimated trajectory and the real trajectory of the algorithm of the present invention and DynaSLAM under 4 high dynamic data sets is shown. The black represents the real trajectory, the green represents the trajectory of DynaSLAM, and the blue represents the trajectory of the algorithm of the present invention. It can be seen that the estimated trajectory of the algorithm of the present invention is more consistent with the real trajectory, indicating that the positioning performance of the algorithm of the present invention is better than that of DynaSLAM.

[0101] Table 3 Real-time comparison of three algorithms

[0102]

[0103] As can be seen from Table 3, the frame rate of the algorithm of the present invention is lower than that of the original ORB-SLAM3 algorithm, but the frame rate can reach more than 45fps, which can meet the real-time performance. DynaSLAM uses Mask-RCNN, which has a large network model complexity and takes a long time, seriously affecting the real-time performance of the system. The algorithm of the present invention only uses the lightweight network YOLOV8 to provide the object target frame and mask information, and uses the PROSAC algorithm to effectively reduce the number of iterations, which takes a short time and has good real-time performance.

Claims

1. A RGB-D visual SLAM method for indoor dynamic scenes, characterized in that: The method comprises the following steps: Step 1: The target detection thread uses the YOLOv8 network to perform target detection on the RGB-D image to generate masks and semantic information, which are then fed back to the tracking thread. Step 2: The tracking thread extracts ORB feature points from the input RGB-D image, calculates the relative pose and key frame selection between the current frame and the map, and uses the constant velocity model for tracking. The sliding window mechanism and template matching strategy are used to identify and eliminate dynamic features. Step 3: Use the PROSAC algorithm to eliminate feature mismatches. Step 4: The local mapping thread generates and optimizes the local map. Step 5: The loop detection thread detects whether a closed loop occurs on the map.

2. The RGB-D visual SLAM method for indoor dynamic scenes according to claim 1, characterized in that: The step 1 is specifically as follows: 1-1 Input the image frames of the RGB-D camera into the YOLOv8 network. 1-2 The network generates bounding boxes, masks, class labels, and confidence scores around detected objects. 1-3 provides this semantic information to the tracking thread as the key to subsequently distinguishing between dynamic and static objects.

3. The RGB-D visual SLAM method for indoor dynamic scenes according to claim 1, characterized in that: The step 2 is specifically as follows: 2-1 Extract the ORB features of the image and encapsulate the image into a frame data structure for processing. 2-2 uses a sliding window mechanism, and the window stores continuous frames of fixed length. 2-3 Based on the semantic information and target frame obtained by the target detection thread, template matching is performed on the same target object in the two frames. If the current matching score reaches the threshold, it is judged as a dynamic object. 2-4 Based on the prior mask information, the feature points on this dynamic object are removed, leaving only static feature points for camera pose estimation. 2-5 Select the corresponding tracking model to solve the initial position and posture of the camera, which is divided into constant speed motion tracking model and reference frame tracking model. The constant speed motion tracking model is used in normal tracking mode, and the reference frame tracking model is mainly used for repositioning after tracking failure. 2-6 Track the local map to explore more feature matching relationships between local map points and the current frame image, and then further optimize the camera pose based on the graph optimization model. 2-7 Determine whether it is necessary to generate a key frame based on the key frame selection conditions, and pass the key frame to the local mapping thread.

4. The RGB-D visual SLAM method for indoor dynamic scenes according to claim 1, characterized in that: The step three is specifically as follows: The PROSAC process of eliminating mismatches is as follows: 3-1 Sort the initial feature matching points according to the ratio β of the Euclidean distance. The smaller the ratio β of the Euclidean distance, the higher the image matching efficiency. β = d min / d min2 , where d min is the minimum Euclidean distance, d min2 It is the sub-minimum Euclidean distance. The better the quality of the feature matching points, the higher the matching success rate. 3-2 The assumed sample set w and sample size n are determined by the matching point set X obtained by the feature matching algorithm. 3-3 Form an initial sample from 3 pairs of matching points in the hypothetical sample set w, and then calculate the homography matrix H. 3-4 Check that all initial matching points in images i1 and i2 are inliers of H. Assume that x and x' are a pair of matching points and ε is the matching error threshold. ‖Hx-x'‖ 2 ≤ε, the matching point is an inlier point, and then the inlier point is calculated. 3-5 In order to determine whether the algorithm meets the termination condition of the algorithm, it returns to the third step and re-traverses the sample set w. If the termination condition of the termination algorithm is still not met, the sample set w increases by n+1 pairs of matching points, the sample size n increases by 1, and step 2 is repeated.

5. The RGB-D visual SLAM method for indoor dynamic scenes according to claim 1, characterized in that: The step 4 is specifically as follows: 4-1 Receive new key frames from the tracking thread. 4-2 Process the new keyframe, calculate the bag-of-words descriptor for the keyframe and insert it into the map. 4-3 Triangulate to generate new map points and check and merge duplicate map points. 4-4 Perform local BA optimization to further optimize the key frame pose and map point spatial coordinates. 4-5 Determine the redundancy of key frames and delete redundant key frames. 4-6 Send the processed key frame to the loop detection thread.

6. The RGB-D visual SLAM method for indoor dynamic scenes according to claim 1, characterized in that: The step five is specifically as follows: 5-1 Find candidate closed-loop frames in the key frame database based on bag-of-words technology. 5-2 The current key frame and the candidate closed-loop frame are feature matched, and the Sim3 transformation of the closed-loop key frame and the current key frame is solved and optimized. Then a continuity check is performed, that is, projection matching is performed on the three common view frames of the current frame and the candidate closed-loop frame. If the number of feature matches meets the conditions, the loop is detected. 5-3 Use pose propagation to correct the key frame pose and three-dimensional coordinates of local map points, and perform map point fusion and essential graph optimization. 5-4 performs global BA to eliminate the accumulated trajectory error.

Images