An RGB-D vision slam method applied to indoor dynamic environment
By combining semantic segmentation networks and LK optical flow to generate dynamic object masks and eliminate dynamic feature points, the problem of low positioning accuracy and misjudgment in dynamic environments of traditional visual SLAM algorithms is solved, achieving high-precision and robust indoor positioning.
Patent Information
- Application Number
- CN202310464081.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-26
- Publication Date
- 2025-11-25
- Estimated Expiration
- 2043-04-26
AI Technical Summary
Traditional visual SLAM algorithms have low localization accuracy in dynamic environments, while deep learning methods cannot identify untrained object categories, misjudge dynamic objects, and have poor robustness.
By combining a semantic segmentation network to generate a priori dynamic object mask, the LK optical flow method and epipolar constraint algorithm are used to further detect dynamic feature points, remove dynamic regions, and extract only static feature points for pose estimation.
It improves the localization accuracy and robustness of visual SLAM in dynamic environments, is applicable to most practical indoor scenarios, overcomes the limitations of deep learning methods, and can effectively detect passively moving objects.
Smart Images
Figure CN116758112B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of simultaneous localization and mapping. Specifically relates to a RGB-D vision SLAM method applied to indoor dynamic environment. BACKGROUND
[0002] With the rapid development of artificial intelligence technology and computer technology, intelligent autonomous mobile robots have become a research hotspot in the field of robots. In recent years, China has continuously introduced industrial policies and guidelines to promote and guide the development of intelligent autonomous mobile robots. Intelligent autonomous mobile robots involve a variety of intelligent technologies, among which, the simultaneous localization and mapping technology (SLAM) is the foundation for the realization of the value of intelligent autonomous mobile robots.
[0003] Simultaneous localization and mapping technology refers to a subject equipped with a specific sensor estimating its own pose during movement without prior environmental information, while constructing a map of the surrounding environment. At present, SLAM algorithms are mainly divided into two categories according to the main body sensor used: visual SLAM algorithm using monocular camera, RGB-D camera, and binocular camera as the perception unit, and laser SLAM algorithm using laser radar as the perception unit. In recent years, due to the advantages of visual SLAM algorithm such as low deployment cost, rich environmental perception information, and high indoor positioning accuracy, it has been widely applied in the fields of intelligent autonomous mobile robots, autonomous driving, etc.
[0004] Most traditional visual SLAM algorithms are based on the assumption of static environment, while in the actual physical environment, frequent dynamic objects can greatly affect the accuracy of visual odometry pose estimation, and even cause visual positioning failure. In recent years, in view of the problem of poor positioning accuracy of visual SLAM algorithm in dynamic environment, domestic and foreign scholars combine target detection network, semantic segmentation network and other deep learning methods to exclude feature points with prior motion properties in the image, in order to reduce the influence of dynamic objects in the environment on visual positioning. However, this method has some limitations: (1) the categories that can be recognized by deep learning method are limited, when the actual environment appears a dynamic object category that has not been trained, the deep learning method cannot identify this dynamic object. (2) The subjective definition of moving objects often has gaps with the real environment, which may cause misjudgment. (3) The running effect of deep learning framework is unstable, which leads to weak robustness of the system.
[0005] Therefore, it is a technical problem that needs to be solved by those skilled in the art to provide an indoor visual SLAM method with strong robustness and applicable to most actual physical scenes. SUMMARY
[0006] The main purpose of the present application is to provide an RGB-D visual SLAM method applied to an indoor dynamic environment, which can be applied to an actual physical environment, reduce the influence of dynamic objects in the environment on the pose estimation of the visual odometry, and provide robust and high-precision positioning information.
[0007] 1. The positioning accuracy of the traditional visual SLAM algorithm is low in a dynamic environment, and the positioning fails in a high dynamic scene;
[0008] 2. The target detection model or semantic segmentation model used in the visual SLAM algorithm combined with deep learning cannot recognize the object category in the image frame that is not trained, and the feature points extracted from the object region may affect the positioning accuracy of the visual odometry;
[0009] 3. The visual SLAM algorithm combined with deep learning needs to manually distinguish the dynamic and static properties of the object, which is subjective and cannot solve the situation that the active moving object forces the passive moving object to move, such as walking with a water cup in hand. The deep learning network misjudges the water cup as a static object, which is actually a dynamic object. This misjudgment will make some dynamic feature points used for pose estimation.
[0010] In order to solve the above problems, the present application provides an RGB-D visual SLAM method applied to an indoor dynamic environment, which combines a semantic segmentation network to generate a prior dynamic object mask covering the active moving property object; then combines the LK optical flow method and the epipolar constraint algorithm to further detect the dynamic feature points in the actual environment to perfect the dynamic object mask; finally, under the shielding effect of the mask, the static feature points in the non-dynamic region are extracted for pose estimation.
[0011] The specific work flow of the present application is as follows:
[0012] Step 1: An RGB-D camera is used to collect indoor dynamic environment image data, and multiple frames of continuous original RGB images and depth images are obtained;
[0013] Step 2: A semantic segmentation network is deployed in the visual SLAM framework tracking thread to infer each frame of RGB image collected in step 1 to obtain a semantic mask image;
[0014] Step 3: Based on the semantic mask image information, the detected objects in the image are divided into dynamic objects and static objects. If there are dynamic objects in the image, step 4 is performed, otherwise the feature points of the RGB image are extracted and step 7 is performed;
[0015] Step 4: Based on the dynamic and static pixel classification information obtained in step 3, a prior dynamic object mask is generated;
[0016] Step 5: The LK optical flow method and the epipolar constraint algorithm are combined to further perfect the dynamic object mask;
[0017] Step 6, based on the dynamic object mask obtained in step 5, extract the static feature points of each frame of RGB image.
[0018] Step 7, based on the static feature points of the RGB image and the depth image, pose estimation, local mapping, loop detection are performed.
[0019] In step 2, the semantic segmentation network used is the Yolov5-seg semantic segmentation model, which can perform pixel-level semantic segmentation on 80 types of objects. The specific steps are as follows:
[0020] The image sequence collected by the RGB-D camera is transmitted into the ORB-SLAM2 tracking thread, and in this thread, the ONNX model file of the Yolov5-seg semantic segmentation network is read using the dnn module of opencv, and then the incoming RGB image is preprocessed: first, a full black 3-channel square image with pixel value 0 is made, the side length is the larger value of the incoming RGB image length and width, the original RGB image is copied to the upper left corner of the square image, then the square image is scaled to 640*640 size, and the blobFromImage function of opencv is used for pixel normalization and R, B color channel conversion. After the preprocessed picture is inferred by the network, two output results output0 and output1 are obtained, the dimension of output0 is 1*25200*177, the first 85 columns are the results of each target detection box, and the last 32 columns are the semantic mask coefficients of each target detection box, the dimension of output1 is 1*32*160*160, which is the original mask image, the mask coefficients in the target detection box are weighted and summed with the original mask, and then scaled to obtain the semantic mask image. The target detection box confidence threshold is set to 0.25, the IOU threshold is set to 0.45, non-maximum suppression is performed to eliminate redundant overlapping target detection boxes with a confidence threshold lower than 0.25, and finally the object detection box, semantic annotation, confidence, and pixel-level color semantic mask annotation are marked on the original RGB image.
[0021] The dynamic objects and static objects described in step 3 come from the 80 types of objects recognizable by Yolov5-seg, and the dynamic objects are the objects with active motion properties in the 80 types (people, cats, dogs, horses, sheep, cows, elephants, bears, giraffes), and the rest are classified as static objects.
[0022] In step 4, according to the pixel-level color semantic mask obtained in step 2, the semantic annotation, and the dynamic and static object classification method described in step 3, all pixels in the RGB image frame are divided into dynamic pixels and other pixels, the pixel value of the dynamic pixel point is set to 0, and the pixel value of the other pixel point is set to 255, thereby generating a prior dynamic object mask.
[0023] The further improvement of the prior dynamic mask method obtained in step 4 is as follows:
[0024] (1) Extract the Harris corner points of the image frame
[0025] Convert the multi-frame continuous RGB image frame into a gray image, use the Harris corner point extractor in the opencv library to extract 500 Harris corner points in the gray image frame, and use the cornerSubPix function of the opencv library to further optimize the detected Harris corner points, so that the accuracy of the Harris corner points reaches the sub-pixel level.
[0026] (2) LK optical flow tracking
[0027] LK optical flow pyramid tracking is performed on the Harris corner points extracted from the previous frame, that is, the corresponding positions of the Harris corner points in the previous frame in the current frame are obtained, the corner points that fail to track are removed, and then the Harris corner points obtained by tracking are screened twice. The first screening removes the Harris corner points whose distance to the image edge is less than 5 pixels. The second screening is based on the luminosity invariance assumption of the LK optical flow method, and the pixel point difference of the 3*3 rectangular block near the matched Harris corner points in the previous and subsequent two frames is used for judgment. If the pixel point difference of a pair of matched corner points is greater than 2120, it is determined that the pair of matched corner points is not good, and the pair of matched corner points is removed. Finally, the excellent Harris corner points remaining after the two screenings are used to calculate the fundamental matrix of the previous and subsequent two frames.
[0028] (3) Determine the dynamic point by epipolar constraint
[0029] Let the Harris corner points of the previous frame after the two screenings be P1, and the two-dimensional coordinates of P1 in the previous frame image be (u1, v1), and the homogeneous coordinates of P1 be (u1, v1, 1). The Harris corner points matched with P1 in the current frame are P2, and the two-dimensional coordinates of P2 in the current frame image are (u2, v2), and the homogeneous coordinates of P2 are (u2, v2, 1). Then the epipolar line I1 formed by the mapping of P1 point in the previous frame through the fundamental matrix F between the two frames to the current frame is obtained, where A, B, C are the coefficients in the linear expression Ax+By+C=0 of the epipolar line I1, as shown in formula (1):
[0030]
[0031] The distance D of the point P2 to the epipolar line I1 is calculated, as shown in formula (2):
[0032]
[0033] If the distance D is greater than 1 pixel, it can be considered that P2 has a moving phenomenon in the actual physical environment, and it is judged as a dynamic point, and all dynamic points in the two-dimensional coordinates of the current image frame are recorded.
[0034] (4) Perfecting dynamic object mask based on dynamic point
[0035] On the prior dynamic object mask obtained in step 4, the two-dimensional coordinates of all dynamic points are taken as the center of the circle, and the pixel value of the circular range with a radius of 10 pixels is set to 0, and the dynamic object mask is further perfected.
[0036] In step 6, under the ORB-SLAM2 tracking thread, the dynamic object mask is used to extract static feature points in the dynamic environment, and the specific steps are as follows:
[0037] First, the RGB image frame and the dynamic object mask obtained in step 5 are used to obtain the image pyramid and the mask pyramid through step-by-step scaling, and the mask is used to extract the FAST feature points in the static area on each layer of the image pyramid. In order to ensure that the FAST feature points are uniformly distributed in all parts of the image, the image on each layer of the image pyramid is uniformly divided into 30x30 pixel grids when extracting the FAST feature points on each layer, and the FAST feature points are extracted separately for each grid. Finally, a large number of FAST feature points extracted on each layer of the image pyramid are subjected to quadtree screening, and the image area with feature points is continuously divided into 4 equal parts until the number of feature points to be extracted on each layer is divided into partitions. Each partition only retains the feature point with the largest response value.
[0038] The pose estimation part in step 7 uses the static feature points obtained in step 6, and the remaining process is the same as the RGB-D working mode of ORB-SLAM2. The local mapping and loop detection threads after that are the same as the conventional ORB-SLAM2 framework.
[0039] The present application has the following advantages:
[0040] (1) The present application directly embeds the Yolov5-seg semantic segmentation network into the ORB-SLAM2 tracking thread, without the need for a separate deep learning environment, which is easy to deploy to actual devices. The semantic mask image obtained by the network inference is used to generate a prior dynamic object mask to eliminate dynamic feature points with prior properties, which significantly improves the positioning accuracy of ORB-SLAM2 in dynamic environments.
[0041] (2) The present application can be applied to most actual indoor dynamic scenes, and can monitor the dynamic points in the environment in real time based on the limit constraint, overcoming the limitation of the recognizable categories of deep learning methods, and can effectively detect the current state of passive moving objects, and has strong robustness. BRIEF DESCRIPTION OF DRAWINGS
[0042] Figure 1The application is applied to an RGB-D vision SLAM method flowchart of an indoor dynamic environment.
[0043] Figure 2 A semantic mask image generated by a semantic segmentation network inference result;
[0044] Figure 3 (a) Prior dynamic object mask (b) Further processing generated dynamic object mask; DETAILED DESCRIPTION
[0045] In order to more fully express the technical scheme of the application, the following will be further described in conjunction with the drawings and examples.
[0046] As Figure 1 shown, the specific steps of the application are as follows:
[0047] 1. Collecting RGB images and depth images
[0048] The image acquisition device uses an RGB-D camera, calibrates the camera and writes the parameters into the SLAM framework sensor parameter yaml file, and transmits the collected RGB image frame stream and depth image frame stream to the computer in real time, and the tracking thread of the SLAM framework receives the image data published by the sensor.
[0049] 2. Generating a semantic mask image
[0050] The tracking thread uses the dnn module of opencv to deploy the Yolov5-seg semantic segmentation network. The dnn module of opencv is used to read the ONNX file exported by the Yolov5-seg semantic segmentation network model, which is pre-trained on the COCO dataset and can perform pixel-level semantic segmentation on 80 common objects. The specific process is as follows:
[0051] (1) Image preprocessing
[0052] The incoming RGB image is preprocessed, first a full black 3-channel square image with pixel value of 0 is made, the side length is the larger value of the incoming RGB image, the original RGB image is placed in the upper left corner of the square image, then the picture is scaled to the expected image input size (640*640) of the Yolov5-seg model, and the blobFromImage function of opencv is used for pixel normalization and R, B color channel conversion.
[0053] (2) Post-processing of inference results
[0054] After network forward propagation, the output of the two parts of the results output0 and output1, the dimension of output0 is 1*25200*177, the first 85 columns are the results of each target detection frame, and the last 32 columns are the semantic mask coefficients of each target detection frame, and the dimension of output1 is 1*32*160*160, which is the prototype mask image. The mask coefficients in the target detection frame are weighted and summed with the prototype mask, and then scaled according to the scaling condition to obtain the semantic mask image. The target detection frame confidence threshold is set to 0.25, and the IOU threshold is set to 0.45. Non-maximum suppression is performed to eliminate redundant overlapping target detection frames with low confidence. Finally, the object detection frame with high confidence, semantic annotation, confidence, and pixel-level color semantic mask annotation are labeled on the original RGB image, and the generated semantic mask image is as shown in Figure 2 .
[0055] 3. Dynamic and static classification
[0056] According to the semantic annotation in the semantic mask image, all detected objects in the image are classified into dynamic objects and static objects. Among the 80 types of objects that can be recognized by Yolov5-seg, the animal categories with active properties (people, cats, dogs, horses, sheep, cows, elephants, bears, giraffes) are classified as dynamic objects, and the rest are classified as static objects. If the above dynamic object categories exist in the semantic mask image, step 4 is performed, otherwise, the original RGB image frame is directly subjected to ORB feature point extraction, and the subsequent processing is the same as the original ORB-SLAM2 framework.
[0057] 4. Generate prior dynamic object mask
[0058] According to the pixel-level color semantic mask obtained in step 2, the semantic annotation, and the dynamic and static object classification method described in step 3, all pixel points in the RGB image frame are divided into dynamic pixel points and other pixel points. The pixel value of the dynamic pixel point is set to 0, and the pixel value of the other pixel point is set to 255, thereby generating a prior dynamic object mask. The generated prior dynamic object mask is as shown in Figure 3 (a). This mask can shield the prior dynamic area in the image frame. When feature points are extracted from the original image, only static area feature points are retained.
[0059] 5. Dynamic object mask secondary processing
[0060] The prior dynamic object mask can eliminate the feature points on the animal to a certain extent, but in the actual physical environment, the dynamic object categories that the semantic segmentation network cannot identify are likely to occur, and the active motion property object forces the passive motion property object to move, which often occurs in the actual environment. In order to solve the above problems, the prior dynamic object mask obtained in step 4 is further improved by combining the LK optical flow method and the epipolar constraint algorithm, and the specific process is as follows:
[0061] (1) Extract the Harris corner points of the image frame
[0062] The multi-frame continuous RGB image frame is converted into a gray image, the Harris corner point extractor of the opencv library is used to extract 500 Harris corner points in the gray image frame, and the cornerSubPix function of the opencv library is used to further optimize the detected Harris corner points, so that the accuracy of the Harris corner points reaches the sub-pixel level.
[0063] (2) LK optical flow tracking
[0064] The LK optical flow pyramid tracking is performed on the Harris corner points extracted from the previous frame, that is, the corresponding positions of the Harris corner points in the previous frame in the current frame are obtained, the corner points that fail to track are eliminated, and then the Harris corner points obtained by tracking are screened twice. In the first screening, the Harris corner points with a distance less than 5 pixels from the image edge are eliminated. In the second screening, based on the luminosity invariance assumption of the LK optical flow method, the pixel point difference of the 3*3 rectangular block near the matched Harris corner points of the previous and subsequent two frames is used for judgment. If the pixel point difference of a pair of matched corner points is greater than 2120, it is determined that the pair of matched corner points is poor, and the pair of matched corner points is eliminated. Finally, the excellent Harris corner points remaining after the two screenings are used to calculate the fundamental matrix of the previous and subsequent two frames.
[0065] (3) Determine the dynamic point by epipolar constraint
[0066] P1 is the Harris corner point of the previous frame after the two screenings, the two-dimensional coordinates of P1 in the previous frame image are (u1, v1), the homogeneous coordinates of P1 are (u1, v1, 1), and the Harris corner point matched with P1 in the current frame is P2, the two-dimensional coordinates of P2 in the current frame image are (u2, v2), and the homogeneous coordinates of P2 are (u2, v2, 1). Then, the P1 point of the previous frame is mapped to the current frame through the fundamental matrix F between the two frames to form an epipolar line I1, wherein A, B and C are coefficients in the linear expression Ax+By+C=0 of the epipolar line I1, as shown in formula (1):
[0067]
[0068] The distance D of the point P2 to the epipolar line I1 is calculated, as shown in formula (2):
[0069]
[0070] If the distance D is greater than 1 pixel, it can be considered that P2 has a moving phenomenon in the actual physical environment, and it is judged as a dynamic point. All dynamic points in the two-dimensional coordinates of the current image frame are recorded.
[0071] (4) Perfecting dynamic object mask based on dynamic points
[0072] On the prior dynamic object mask obtained in step 4, the two-dimensional coordinates of all dynamic points are taken as the center of the circle, and the pixel value of the circular range with a radius of 10 pixels is set to 0, so as to further perfect the dynamic object mask. The effect is as shown in Figure 3 (b), the region of the passive moving object in the actual environment is also added to the dynamic object mask.
[0073] 6. Extracting static feature points
[0074] The RGB image frame and the dynamic object mask obtained in step 5 are used to obtain an image pyramid and a mask pyramid through step-by-step scaling. The FAST feature points of the static region are extracted using the mask on each layer of the image pyramid. In order to ensure that the FAST feature points are uniformly distributed in all parts of the image, the image on each layer of the image pyramid is uniformly divided into a 30x30 pixel grid when extracting the FAST feature points on the layer. The FAST feature points are extracted separately for each grid. Finally, all the FAST feature points extracted on each layer of the image pyramid are subjected to quadtree screening. The image region with feature points is continuously divided into four equal parts until the number of partitions of the feature points to be extracted on each layer is divided. Only the feature point with the largest response value is retained in each partition.
[0075] 7. Pose estimation and mapping
[0076] The static feature points obtained in step 6 are used for pose estimation. After extracting the static feature points, the process is the same as the RGB-D working mode of ORB-SLAM2.
[0077] The above description of the specific implementation method of the present application is only for the technical personnel in the technical field to understand the present application, and is not used to limit the scope of the present application.
Claims
1. An RGB-D visual SLAM method applied to indoor dynamic environments, characterized in that, The method includes the following steps: Step 1: Use an RGB-D camera to acquire indoor dynamic environment image data, and obtain multiple consecutive frames of raw RGB images and depth images; Step 2: Deploy a semantic segmentation network in the tracking thread of the visual SLAM framework to infer the semantic mask image for each frame of RGB image acquired in Step 1. Step 3: Based on the semantic mask image information, the detected objects in the image are divided into dynamic objects and static objects. If there are dynamic objects in the image, proceed to step 4; otherwise, extract the feature points of the RGB image and proceed to step 7. Step 4: Based on the dynamic and static pixel classification information obtained in Step 3, generate a priori dynamic object mask; Step 5: Further refine the dynamic object mask by combining the LK optical flow method and the epipolar constraint algorithm; Step 6: Based on the dynamic object mask obtained in Step 5, extract the static feature points of each frame of the RGB image. Step 7: Based on the static feature points and depth image of the RGB image, perform pose estimation, local mapping, and loop closure detection; Step 5 is as follows: (1) Extract Harris corner points from image frames Convert multiple consecutive RGB image frames into grayscale images, use the Harris corner extractor built into the OpenCV library to extract 500 Harris corners from the grayscale image frames, and further optimize the detected Harris corners using the cornerSubPix function of the OpenCV library to achieve sub-pixel accuracy. (2) LK optical flow tracking By performing LK optical flow pyramid tracking on the Harris corner points extracted from the previous frame, the corresponding positions of the Harris corner points in the current frame can be obtained. Corner points that fail to be tracked are removed. Then, the tracked Harris corner points are filtered twice. The first filter removes Harris corner points that are less than 5 pixels away from the image edge. The second filter is based on the photometric invariance assumption of the LK optical flow method. It uses the pixel difference of the 3*3 rectangular blocks near the matched Harris corner points in the two frames to judge. If the pixel difference of a pair of matching points is greater than 2120, the matching corner points are judged as poor and removed. Finally, the basic matrix of the two frames is calculated using the remaining good Harris corner points after the two filters. (3) Polar constraint judgment of dynamic points Let P1 be the Harris corner point of the previous frame after two filtering steps, with its two-dimensional coordinates (u1, v1) and homogeneous coordinates (u1, v1, 1) in the previous frame image. Let P2 be the Harris corner point of the current frame that matches it, with its two-dimensional coordinates (u2, v2) and homogeneous coordinates (u2, v2, 1) in the current frame image. Then, the polar line I1 is formed by mapping the point P1 of the previous frame to the current frame through the fundamental matrix F between the two frames. A, B, and C are the coefficients in the linear expression Ax + By + C = 0 of the polar line I1, as shown in formula (1). Calculate the distance D from point P2 to the polar line I1, as shown in formula (2): If the distance D is greater than 1 pixel, then P2 can be considered to have moved in the actual physical environment, and it is judged as a dynamic point. Record the two-dimensional coordinates of all dynamic points in the current image frame. (4) Improve dynamic object masking based on dynamic points On the prior dynamic object mask obtained in step 4, with the two-dimensional coordinates of all dynamic points as the center, the pixel values of a circular range with a radius of 10 pixels are set to 0 to further improve the dynamic object mask.
2. The RGB-D visual SLAM method for indoor dynamic environments as described in claim 1, characterized in that, The semantic segmentation network mentioned in step 2 is the Yolov5-seg semantic segmentation model, which can perform pixel-level semantic segmentation of 80 object categories. The specific steps are as follows: The image sequence acquired by the RGB-D camera is fed into the ORB-SLAM2 tracking thread. In this thread, OpenCV's DNN module is used to read the ONNX model file of the Yolov5-Seg semantic segmentation network. Then, the input RGB image is preprocessed: First, a 3-channel square image with all pixels set to 0 is created, with a side length equal to the larger of the input RGB image's length and width. The original RGB image is copied to the top left corner of the square image. Then, the square image is scaled to 640*640 pixels, and OpenCV's `blobFromImage` function is used to apply the changes. Pixel normalization and R / B color channel conversion are performed. After the preprocessed image is passed through network inference, two output results, output0 and output1, are generated. The dimension of output0 is 1*25200*177, with the first 85 columns being the result of each object detection box and the last 32 columns being the semantic mask coefficients of each object detection box. The dimension of output1 is 1*32*160*160, which is the prototype mask image. The mask coefficients in the object detection boxes are weighted and summed with the prototype mask, and then scaled according to the scaling condition to obtain the semantic mask image. The target detection box confidence threshold was set to 0.25, the IOU threshold was set to 0.45, and non-maximum suppression was performed to eliminate redundant overlapping target detection boxes with confidence levels below the confidence threshold. Finally, the object detection boxes, semantic annotations, confidence levels, and pixel-level color semantic masks were annotated on the original RGB image.
3. The RGB-D visual SLAM method for indoor dynamic environments as described in claim 1, characterized in that, The dynamic and static objects mentioned in step 3 are from the 80 categories of objects that can be recognized by YOLOv5-seg. Dynamic objects are objects with active motion properties among the 80 categories, and the rest are classified as static objects.
4. The RGB-D visual SLAM method for indoor dynamic environments as described in claim 1, characterized in that, Step 4 is as follows: Based on the pixel-level color semantic mask and semantic annotation obtained in step 2, and the dynamic and static object classification method described in step 3, all pixels in the RGB image frame are divided into dynamic pixels and other pixels. The pixel value of dynamic pixels is set to 0, and the pixel value of other pixels is set to 255, thereby generating a priori dynamic object mask.
5. The RGB-D visual SLAM method for indoor dynamic environments as described in claim 1, characterized in that, Step 6 is as follows: First, using RGB image frames and the dynamic object mask obtained in step 5, an image pyramid and a mask pyramid are obtained by progressive scaling. On each layer of the image pyramid, the corresponding mask layer of the mask pyramid is used to extract FAST feature points of static regions. To ensure that the FAST feature points are evenly distributed throughout the image, when extracting FAST feature points in each layer of the image pyramid, the image of that layer is evenly divided into 30×30 pixel grids. FAST feature points are extracted for each grid individually. Finally, a quadtree is used to filter all the FAST feature points extracted from each layer of the image pyramid. The image regions containing feature points are continuously divided into 4 equal parts until the partitions for which the desired number of feature points are to be extracted are determined. Each partition retains only the feature point with the largest response value.