Semantic dynamic SLAM processing method based on depth consistency constraint

Abstract

The invention discloses a semantic dynamic SLAM (simultaneous localization and mapping) processing method based on depth consistency constraint. According to the method, a mobile terminal collects and obtains a plurality of images and records the images as each frame of scene graph, each frame of scene graph is respectively input into a semantic segmentation thread for instance segmentation and a tracking thread for ORB feature point extraction, and different object masks of each frame of scene graph and feature points of each frame of scene graph are respectively obtained; screening the feature points of each frame of scene graph in combination with different object masks of each frame of scene graph to obtain effective feature points of each frame of scene graph; and performing pose estimation processing on the effective feature points of each frame of scene graph in sequence to obtain an initial pose of the current frame, and then constructing a dense point cloud map and a semantic dense map at the rear end. Compared with an existing dynamic SLAM processing method, the dynamic SLAM processing method has the advantages that the real-time performance is kept, meanwhile, the problem of degeneration caused by the fact that the dynamic target moves along the epipolar line is effectively solved, and the positioning robustness and the map integrity in the complex dynamic environment are improved.

Description

Technical Field

[0001] This invention relates to the field of real-time localization and mapping in computer vision, and particularly to a semantic dynamic SLAM processing method based on depth consistency constraints. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) technology serves as the core support for robots to achieve autonomous environmental perception and navigation. However, the robustness of SLAM technology in dynamic scenarios has always been a key bottleneck in the field of research. Traditional visual SLAM systems rely on static environment assumptions to build pose estimation models. However, dynamic targets such as pedestrians and vehicles that are common in real-world applications introduce a large number of abnormal feature points, leading to data association failure, accumulation of pose estimation errors, and even system crashes. To address the dynamic interference problem, existing improvement schemes mainly focus on geometric constraints, semantic segmentation, and their fusion methods, but they still have significant limitations: Geometric constraint-based methods (such as RANSAC and epipolar geometry verification) remove dynamic points through motion consistency detection. While they can handle simple motion patterns, they rely on iterative optimization in complex scenarios, leading to low computational efficiency and failing to guarantee real-time system performance. Deep learning-based methods (such as Mask R-CNN and YOLOv8) utilize pixel-level semantic priors to identify predefined dynamic objects, but they cannot detect unknown dynamic targets outside the training set, and are prone to brute-force removal at the boundary between static and dynamic objects, resulting in the loss of high-quality static feature points. Fusion methods effectively alleviate the inherent defects of traditional single-modal methods in dynamic feature discrimination. The synergistic optimization of the two not only enhances the robustness of dynamic perception but also maintains real-time performance while improving positioning accuracy through a lightweight fusion architecture. However, the dynamic perception capabilities of these algorithms are insufficient in specific environments. For non-potential dynamic objects moving along the polar direction (such as uniformly moving targets without labeled categories), their motion compensation residuals approach static features, while the semantic model cannot identify them due to missing categories, resulting in a decrease in system accuracy. Summary of the Invention

[0003] To address the problems existing in the background technology, this invention proposes a semantic dynamic SLAM processing method based on depth consistency constraints. It combines semantic and geometric information for coarse dynamic feature removal, and then uses depth consistency constraints for fine dynamic feature removal, thereby achieving more accurate camera pose estimation and effectively reducing the interference of moving objects on the visual SLAM system.

[0004] To achieve the above objectives, the technical solution of this application includes the following steps:

[0005] S1. The mobile terminal collects and obtains multiple images, which are recorded as scene images of each frame. Each scene image of each frame is input into the semantic segmentation thread for instance segmentation and the tracking thread for ORB feature point extraction, so as to obtain different object masks and feature points of each scene image of each frame.

[0006] S2. Based on the different object masks of each frame scene image, the feature points of each frame scene image are filtered to obtain the effective feature points of each frame scene image.

[0007] S3. Perform pose estimation processing on the effective feature points of each frame scene map in sequence to obtain the preliminary pose of the current frame;

[0008] S4. Combine the preliminary pose estimation of the current frame to determine the key frame. If the current frame is a key frame, add the current frame to the key frame sequence, and then continue to calculate the preliminary pose of the next frame and determine the key frame until the key frame determination of all scene graphs is completed.

[0009] S5. Perform BA optimization and loop closure detection on the keyframe sequence, and then project it into 3D space to obtain a dense point cloud map. Combine the dense point cloud map with different object masks of each frame scene map to perform semantic annotation processing to obtain a semantically dense map.

[0010] Step S2 specifically involves:

[0011] S21. Based on the different object masks of each frame scene image, perform coarse dynamic feature removal on the feature points of each frame scene image to obtain the preliminary static feature points of each frame scene image.

[0012] S22. Project the preliminary static feature points of each frame scene graph onto the previous key frame of the current frame, and then perform detailed dynamic feature removal on the preliminary static feature points of each frame scene graph to obtain the effective feature points of each frame scene graph.

[0013] Step S21 specifically involves:

[0014] S211. The feature points of each frame scene map are matched with ORB feature points of adjacent frames by the pyramid LK optical flow method, and then the RANSAC algorithm combined with the seven-point method is used to calculate the basic matrix of the current frame.

[0015] S212. Calculate the geometric offset distance from the feature point of the current frame to the corresponding epipolar line based on the fundamental matrix of the current frame. If the geometric offset distance is greater than the preset fundamental threshold, mark it as a potential dynamic point.

[0016] S213. Set thresholds according to the different object mask categories of each frame scene image, and then filter potential dynamic points according to the thresholds to obtain the preliminary static feature points of each frame scene image.

[0017] Step S22 specifically involves:

[0018] S221. Project the preliminary static feature points of each frame scene graph onto the previous keyframe of the current frame to obtain the 3D points corresponding to each frame scene graph.

[0019] S222. Reverse map the 3D points corresponding to each frame scene image to the 2D image of the key frame, and perform depth extraction processing to obtain the key frame depth value.

[0020] S223. Calculate the difference between the depth values ​​corresponding to the preliminary static feature points of each frame scene map and the depth values ​​of the key frames to obtain the depth error of each preliminary static feature point.

[0021] S224. Set a depth error threshold and judge the depth error of each preliminary static feature point according to the depth error threshold: if the depth error is greater than the depth error threshold, filter out the corresponding preliminary static feature points to obtain the effective feature points of each frame scene map.

[0022] The depth error threshold is set according to the following formula:

[0023]

[0024] Among them, X k To project a 3D point onto the previous keyframe of the current frame, z k π(X) represents the depth value corresponding to the initial static feature points of each frame of the scene graph. k ) represents the feature points that are reverse-mapped back to the keyframe 2D image, d k (π(X k )) represents the keyframe depth value of each preliminary static feature point.

[0025] The semantically dense map is obtained through the following processing:

[0026] After removing dynamic region point clouds from the dense point cloud map based on different object masks in each frame of the scene map, semantic annotation processing is then performed to construct a semantically dense map.

[0027] A storage medium storing a computer program that, when executed by a processor, implements a semantic dynamic SLAM processing method based on depth consistency constraints.

[0028] A storage medium, wherein the computer program is an instruction corresponding to implementing a semantic dynamic SLAM processing method based on depth consistency constraints.

[0029] The beneficial effects of this invention are as follows:

[0030] This invention discloses a vision-based dynamic environment SLAM method that achieves efficient removal of dynamic features by integrating deep learning and geometric constraints. The method first utilizes a YOLOv8-based instance segmentation network to obtain prior semantic information of the scene, and then combines epipolar geometric constraints for coarse-grained removal of dynamic features. Next, it innovatively introduces a depth consistency constraint strategy, which calculates the normalized residual between the observed and theoretical depth values ​​by projecting the current frame feature points onto the keyframe coordinate system, accurately identifying and removing dynamic features moving along the epipolar direction and missed detection points in edge regions. This depth consistency verification mechanism effectively solves the failure problem of traditional geometric methods under co-view degradation motion. Combined with pixel-level mask expansion technology of semantic segmentation, it significantly improves the robustness of dynamic perception while ensuring real-time performance, reduces trajectory estimation errors, and provides mobile robots with high-precision dynamic environment localization capabilities, possessing significant engineering application value. Attached Figure Description

[0031] Figure 1 This is a schematic flowchart of the method of the present invention;

[0032] Figure 2 This is a schematic diagram of the framework of the method of the present invention;

[0033] Figure 3 This is a schematic diagram of the epipolar geometry constraint;

[0034] Figure 4 This is a feature point map after the dynamic removal strategy of the present invention;

[0035] Figure 5 This is a dense point cloud map of the w_xyz sequence in the TUM dataset, generated by the method of this invention.

[0036] Figure 6 This is a semantic point cloud map of the w_xyz sequence in the TUM dataset using the method of this invention. Detailed Implementation

[0037] The present invention will be further described below with reference to the accompanying drawings and embodiments. The embodiments of the present invention include, but are not limited to, the following embodiments.

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be described in more detail below with reference to the accompanying drawings.

[0039] This embodiment provides a semantic dynamic SLAM processing method based on depth consistency constraints. This method is applied to robots, where the robot's camera can acquire images of consecutive frames.

[0040] like Figure 1 As shown, this embodiment includes the following steps:

[0041] S1. The mobile terminal collects and obtains multiple images, which are recorded as scene images of each frame. Each scene image of each frame is input into the semantic segmentation thread for instance segmentation and the tracking thread for ORB feature point extraction, so as to obtain different object masks and feature points of each scene image of each frame.

[0042] Instance segmentation involves inputting the image into a YOLOv8 network and performing segmentation as follows: For each frame of the scene image, the probability distribution of the target category is calculated using an activation function, and pixel-level prediction is optimized using a segmentation loss. Then, based on the location information of the detection boxes and the fusion with the mask, the bounding boxes of different object categories and refined segmentation mask results are output, i.e., different object masks for each frame of the scene image. These refined segmentation mask results serve as prior information for subsequent steps. Generally, the trained model can recognize common dynamic objects such as pedestrians and vehicles.

[0043] This embodiment uses the YOLOv8-seg model for instance segmentation. YOLOv8-seg is a multi-task processing method for instance segmentation and object detection, deeply integrating semantic segmentation capabilities on the basis of the classic single-stage detection framework. Its architecture consists of four parts: an input-side adaptive enhancement module, a lightweight cross-stage fusion backbone, a multi-scale feature pyramid neck, and a joint detection-segmentation prediction head. Building upon the design philosophy of the YOLOv5 series, this model introduces dynamic convolutional kernel segmentation mask decoding technology, achieving pixel-level target contour parsing through a spatially aware feature aggregation mechanism. The refined segmentation mask results here serve as prior information for subsequent processing. Generally, the trained model can identify common dynamic objects such as pedestrians and vehicles.

[0044] YOLOv8s-seg, as the version with the highest computational density in this series, employs depthwise separable convolution to compress feature map dimensions and combines it with channel reparameterization technology. This significantly improves the accuracy of mask generation for complex edges and occluded targets while maintaining real-time inference speed. Its dual-branch output structure can simultaneously output target bounding boxes and instance segmentation masks, making it particularly adept at handling deformation features of dynamic targets and contour prediction of potential moving targets. This meets the stringent requirements of visual SLAM systems for scene semantic understanding, providing pixel-level perception support for 3D reconstruction and dynamic obstacle avoidance.

[0045] S2. Combine the different object masks of each frame scene image to filter the feature points of each frame scene image to obtain the effective feature points of each frame scene image.

[0046] S21. Based on the different object masks and epipolar geometric constraints of each frame scene image, coarse dynamic feature removal is performed on the feature points of each frame scene image to obtain the preliminary static feature points of each frame scene image; that is, the ORB feature extraction results and the prior semantic information of instance segmentation are combined with epipolar geometric constraints to perform coarse dynamic feature removal, and the remaining feature points are sent to fine removal.

[0047] S211. The feature points of each frame scene map are matched with ORB feature points of adjacent frames by the pyramid LK optical flow method, and then the RANSAC algorithm combined with the seven-point method is used to calculate the basic matrix of the current frame.

[0048] S212. Calculate the geometric offset distance from the feature point of the current frame to the corresponding epipolar line based on the fundamental matrix of the current frame. If the geometric offset distance is greater than the preset fundamental threshold, mark it as a potential dynamic point.

[0049] like Figure 2 The diagram shows the epipolar geometry constraint. O1 and O2 represent the optical center of the camera, and P1 and P2 are the feature matching points corresponding to spatial point P in the previous and current frames, respectively. The dashed lines L1 and L2 are the epipolar lines in the two frames, and P' and P" are the positions of map point P after it has moved. The homogeneous coordinates corresponding to P1 and P2 are P1 = [x1, y1, 1] and P2 = [x2, y2, 1].

[0050]

[0051] Where X, Y, and Z represent line vectors. The epipolar constraint can be expressed as...

[0052]

[0053] D represents the distance between feature point P2 and its corresponding limit. Ideally, the offset distance of a static point should be 0, but in real-world environments, various noise interferences exist, generally resulting in a distance less than the threshold of 4. The threshold was obtained through multiple experimental comparisons, such as... Figure 3 As shown, when point P moves to the position of P', the corresponding offset distance will be greater than the set threshold, and it will be identified as a dynamic point.

[0054] S213. Set thresholds according to the different object mask categories in each frame of the scene image, and then filter potential dynamic points based on the thresholds to obtain preliminary static feature points for each frame of the scene image. For example, the threshold for pedestrians is generally set to 5, and the threshold for chairs is 3.

[0055] The fundamental matrix represents the geometric relationship between two image frames. For a feature point in the current frame, the corresponding epipolar line in the previous frame can be calculated using the fundamental matrix (theoretically, the matching point should lie on this line). The geometric offset distance refers to the distance between the current frame's feature point and the epipolar line in the previous frame, representing the deviation of that point from the ideal matching position. A large distance indicates a large matching error, which may be due to a moving object or a mismatch; a small distance indicates a reasonable match.

[0056] The projection of spatial point P in the previous frame is P1, and its projection in the current frame is P2. A spatial ray connecting the camera center O1 of the previous frame and point P, projected onto the image plane of the current frame, forms a straight line L2, called the epipolar line. The essence of this line is that the set of all spatial points that could potentially project onto point P1 must lie on the epipolar line L2 in the current frame.

[0057] The Pyramid LK optical flow method employs a hierarchical iterative strategy to optimize feature point matching layer by layer across images at multiple scales (pyramid levels). It calculates the optical flow vector by leveraging pixel brightness gradients within a local window, addressing the matching problem in large displacement scenarios and improving the robustness of ORB feature point matching between adjacent frames.

[0058] The RANSAC algorithm, through a random sampling consensus mechanism, robustly estimates the fundamental matrix from a noisy set of matching points, effectively eliminating the interference of dynamic points on geometric calculations.

[0059] S22. Project the preliminary static feature points of each frame's scene graph onto the previous keyframe, and then perform detailed dynamic feature removal on the preliminary static feature points of each frame's scene graph using depth consistency constraints to obtain the effective feature points of each frame's scene graph. In other words, project the preliminary static feature points of each frame's scene graph onto the corresponding keyframe, and then perform detailed dynamic feature removal using depth consistency constraints. The remaining static feature points are used for tracking.

[0060] If there is no previous keyframe, step S2 will not be performed until a previous keyframe exists.

[0061] S221. Project the preliminary static feature points of each frame scene map to the previous keyframe through camera pose transformation to obtain the 3D points corresponding to each frame scene map.

[0062]

[0063] The pixel coordinates of the feature points in the current frame are p = [u, v]. T Construct homogeneous coordinates p h =[u,v,1] T K is the intrinsic parameter matrix. The depth observation value d is obtained from the depth map of the current frame. i (p). The three-dimensional point P is restored using the following formula. c :

[0064]

[0065] S222. Reverse map the 3D points corresponding to each frame scene image to the 2D image of the key frame, and perform depth extraction processing to obtain the key frame depth value.

[0066] Specifically, combined with the pose T of the current frame i pose T of keyframe k The normalized coordinates are mapped to the keyframe coordinate system through a rigid transformation, and the 3D point X is obtained using the following formula. k :

[0067]

[0068] To quantify the depth consistency between normal frames and keyframes, it is necessary to convert the 3D point X... k Project the image onto the keyframe's image plane. Then, obtain the corresponding pixel coordinates using the projection function and extract the depth observation value at those pixel coordinates. The projection function is:

[0069]

[0070] S223. Calculate the difference between the depth values ​​corresponding to the preliminary static feature points of each frame scene map and the depth values ​​of the key frames to obtain the depth error of each preliminary static feature point.

[0071] S224. Set a depth error threshold and judge the depth error of each preliminary static feature point according to the depth error threshold: if the depth error is greater than the depth error threshold, filter out the corresponding preliminary static feature point; otherwise, do not perform any operation.

[0072] That is, the depth consistency constraint is checked by comparing the calculated depth value in the corresponding key frame with the actual depth of the projection point. If the feature point does not meet the depth consistency constraint, it is judged as a dynamic feature point and removed.

[0073] The depth error threshold is set according to the following formula:

[0074]

[0075] Where X k To project a 3D point onto the previous keyframe of the current frame, z k π(X) represents the depth value corresponding to the initial static feature points of each frame of the scene graph. k ) represents the feature points that are reverse-mapped back to the keyframe 2D image, d k (π(X k )) represents the keyframe depth value from which the projection positions of each preliminary static feature point are extracted from the depth map of the keyframe.

[0076] Ideally, if the depth observation is noise-free and the pose estimation is accurate, X k True depth z k It should be related to the keyframe depth d k (π(X k Strict consistency is required. However, noise exists in the environment, so an empirical value η is set for the rejection strategy. Based on multiple experiments, the set value is 0.5. If δ > η, the feature point is considered a dynamic feature point and is rejected. Figure 4 The image shown is a feature point distribution map after dynamic feature removal.

[0077] S3. Perform pose estimation processing on the effective feature points of each frame scene map in sequence to obtain the preliminary pose of the current frame;

[0078] S4. Combine the preliminary pose estimation of the current frame to determine the key frame. If the current frame is a key frame, add the current frame to the key frame sequence, and then continue to calculate the initial pose of the next frame and determine the key frame. If the current frame is not a key frame, continue to calculate the initial pose of the next frame and determine the key frame, until the key frame determination of all scene graphs is completed.

[0079] S5. Perform BA optimization and loop closure detection on the keyframe sequence, and then project it into 3D space to obtain a dense point cloud map. Combine the dense point cloud map with different object masks of each frame scene map to perform semantic annotation processing to obtain a semantically dense map.

[0080] This involves performing backend optimization and constructing dense point cloud maps and semantic point cloud maps based on depth information and prior information from semantic segmentation.

[0081] Semantic dense maps are obtained through the following steps:

[0082] After removing dynamic region point clouds from the dense point cloud map based on different object masks in each frame of the scene map, semantic annotation is performed to construct a semantically dense map.

[0083] Figure 2 The overall system framework can be divided into a tracking thread, a segmentation thread, and a backend optimization and dense mapping process. This embodiment introduces a coarse-to-fine multi-constraint dynamic feature removal method. First, ORB feature points are extracted and instances are segmented from the input image. Then, based on the semantic prior information obtained from the instance segmentation thread and the epipolar geometric constraints, coarse dynamic feature removal is performed. Next, fine dynamic feature point removal is performed based on depth consistency constraints to further remove the dynamic feature points mentioned above, and tracking is performed based on the remaining static points. Finally, the subsequent optimization process continues, and a dense point cloud map and a semantic point cloud map are built.

[0084] Specifically, by inputting RGB-D images from a visual sensor, a coarse culling strategy combining semantic segmentation and geometric constraints is first employed, followed by fine culling using depth consistency constraints. Backend optimization is then performed using static feature quality weights to achieve high-precision localization and semantic map construction in dynamic scenes. First, prior semantic information is obtained through the YOLOv8 instance segmentation thread, and coarse culling of dynamic features is completed using epipolar geometric constraints. Second, the remaining feature points are projected onto the keyframe coordinate system, and fine culling of dynamic features is achieved based on depth consistency constraints, with static quality weights quantified for each feature point. Finally, semantically enhanced sparse and dense point cloud maps are constructed.

[0085] The above steps realize a semantic dynamic SLAM processing method based on depth consistency constraints, which improves the localization accuracy and robustness of SLAM systems in dynamic environments and provides richer map information for robot navigation.

[0086] In S5, after tracking static feature points to obtain the initial camera pose, the local mapping thread performs local BA optimization on newly added keyframes and their co-view frames, jointly optimizing the pose of keyframes within the local window and the 3D coordinates of associated map points to ensure geometric consistency in the local region. When the loop closure detection thread identifies scene similarity between historical keyframes and the current frame based on the bag-of-words model, it optimizes the relative pose constraints between closed-loop frames through similarity transformation and triggers global pose graph optimization to minimize the weighted sum of loop closure error and odometry error, eliminating trajectory drift caused by long-term operation. The dense point cloud map is generated by fusing depth observations from multiple keyframes: first, the depth map is back-projected into 3D space, and data compression is achieved through voxel mesh downsampling and statistical outlier filtering; then, combined with the dynamic object mask output by the semantic segmentation thread, dynamic region point clouds are removed, while the static environment structure is preserved, ultimately constructing a globally consistent semantically dense map, providing a high-precision geometric and semantic foundation for path planning and scene understanding. Figure 5 The image shows the dense point cloud mapping results on the w_xyz sequence of the TUM dataset. The influence of dynamic objects has been removed, providing richer information for the robot. Figure 6 The image shows a semantic point cloud map. Items such as chairs and monitors are labeled with point clouds based on semantic prior information, providing more specific semantic information for the robot to perceive its environment.

[0087] The above technical solutions only embody the preferred technical solutions of the present invention. Any modifications that may be made by those skilled in the art to certain parts thereof embody the principles of the present invention and fall within the protection scope of the present invention.

[0088] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A semantic dynamic SLAM processing method based on depth consistency constraints, characterized in that, The method includes the following steps: S1. The mobile terminal collects and obtains multiple images, which are recorded as scene images of each frame. Each scene image of each frame is input into the semantic segmentation thread for instance segmentation and the tracking thread for ORB feature point extraction, so as to obtain different object masks and feature points of each scene image of each frame. S2. Based on the different object masks of each frame scene image, the feature points of each frame scene image are filtered to obtain the effective feature points of each frame scene image. S3. Perform pose estimation processing on the effective feature points of each frame scene map in sequence to obtain the preliminary pose of the current frame; S4. Combine the preliminary pose estimation of the current frame to determine the key frame. If the current frame is a key frame, add the current frame to the key frame sequence, and then continue to calculate the preliminary pose of the next frame and determine the key frame until the key frame determination of all scene graphs is completed. S5. Perform BA optimization and loop closure detection on the keyframe sequence, and then project it into 3D space to obtain a dense point cloud map. Combine the dense point cloud map with different object masks of each frame scene map to perform semantic annotation processing to obtain a semantically dense map.

2. The semantic dynamic SLAM processing method based on depth consistency constraints according to claim 1, characterized in that: Step S2 specifically involves: S21. Based on the different object masks of each frame scene image, perform coarse dynamic feature removal on the feature points of each frame scene image to obtain the preliminary static feature points of each frame scene image. S22. Project the preliminary static feature points of each frame scene graph onto the previous key frame of the current frame, and then perform detailed dynamic feature removal on the preliminary static feature points of each frame scene graph to obtain the effective feature points of each frame scene graph.

3. The semantic dynamic SLAM processing method based on depth consistency constraints according to claim 1, characterized in that: Step S21 specifically involves: S211. The feature points of each frame scene map are matched with ORB feature points of adjacent frames by the pyramid LK optical flow method, and then the RANSAC algorithm combined with the seven-point method is used to calculate the basic matrix of the current frame. S212. Calculate the geometric offset distance from the feature point of the current frame to the corresponding epipolar line based on the fundamental matrix of the current frame. If the geometric offset distance is greater than the preset fundamental threshold, mark it as a potential dynamic point. S213. Set thresholds according to the different object mask categories of each frame scene image, and then filter potential dynamic points according to the thresholds to obtain the preliminary static feature points of each frame scene image.

4. The semantic dynamic SLAM processing method based on depth consistency constraints according to claim 1, characterized in that: Step S22 specifically involves: S221. Project the preliminary static feature points of each frame scene graph onto the previous keyframe of the current frame to obtain the 3D points corresponding to each frame scene graph. S222. Reverse map the 3D points corresponding to each frame scene image to the 2D image of the key frame, and perform depth extraction processing to obtain the key frame depth value. S223. Calculate the difference between the depth values ​​corresponding to the preliminary static feature points of each frame scene map and the depth values ​​of the key frames to obtain the depth error of each preliminary static feature point. S224. Set a depth error threshold and judge the depth error of each preliminary static feature point according to the depth error threshold: if the depth error is greater than the depth error threshold, filter out the corresponding preliminary static feature points to obtain the effective feature points of each frame scene map.

5. The semantic dynamic SLAM processing method based on depth consistency constraints according to claim 1, characterized in that: The depth error threshold is set according to the following formula: Among them, X k To project a 3D point onto the previous keyframe of the current frame, z k π(X) represents the depth value corresponding to the initial static feature points of each frame of the scene graph. k ) represents the feature points that are reverse-mapped back to the keyframe 2D image, d k (π(X k )) represents the keyframe depth value of each preliminary static feature point.

6. The semantic dynamic SLAM processing method based on depth consistency constraints according to claim 1, characterized in that: The semantically dense map is obtained through the following processing: After removing dynamic region point clouds from the dense point cloud map based on different object masks in each frame of the scene map, semantic annotation processing is then performed to construct a semantically dense map.

7. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 8.

8. A storage medium according to claim 9, characterized in that, The computer program thereunder is an instruction that implements the method of any one of claims 1 to 8.

Images