SLAM three-dimensional modeling method suitable for complex dynamic scene

Abstract

The invention belongs to the technical field of image processing and three-dimensional modeling, and particularly relates to an SLAM three-dimensional modeling method suitable for a complex dynamic scene. By constructing a multi-modal feature hierarchical processing architecture, the system realizes technical breakthrough in three dimensions: firstly, in combination with a pixel mask pattern and a multi-view geometric method, static feature points are reserved while dynamic feature points are removed as far as possible; secondly, according to the optimal flow field mapping relation of the grid nodes, static background restoration of the shielded area is achieved by establishing the geometric constraint relation of adjacent key frames; and finally, the depth perception data and the optimized feature set are fused to construct a tightly coupled three-dimensional reconstruction model, and a closed-loop optimization framework with motion robustness is formed.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional modeling, and in particular relates to a SLAM three-dimensional modeling method suitable for complex dynamic scenes. Background Art

[0002] Although the visual perception system based on monocular / binocular cameras can achieve synchronous perception of the geometric structure and semantic features of the environment, it still faces three technical bottlenecks in practical applications: first, the feature mismatch problem caused by dynamic interference will lead to the cumulative effect of pose solution errors, which in turn affects the reliability of the downstream navigation decision module; second, the lack of a scene semantic understanding module will significantly increase the failure rate of cross-scene re-identification, causing positioning drift in similar topological environments; third, the computational load brought by the dynamic discrimination of features can easily lead to the degradation of the real-time performance of the algorithm, restricting the system's application potential on resource-constrained platforms. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, this paper proposes a SLAM 3D modeling method suitable for complex dynamic scenes, enabling efficient 3D map construction and pose tracking. First, the method combines pixel masks and multi-view geometry to maximize the removal of dynamic feature points while retaining static feature points. Secondly, based on the optimal flow field mapping relationship between grid nodes, static background restoration is achieved in obscured areas by establishing geometric constraints between adjacent keyframes. Finally, depth perception data is integrated with an optimized feature set to construct a tightly coupled 3D reconstruction model, forming a closed-loop optimization framework with motion robustness.

[0004] The technical solution adopted in the present invention is:

[0005] A SLAM three-dimensional modeling method suitable for complex dynamic scenes includes the following steps:

[0006] S1, obtain the RGB image sequence and depth image sequence of the target scene respectively through the RGB-D camera;

[0007] S2. For RGB images, an improved grid division feature point extraction method is used to extract ORB feature points, specifically including:

[0008] 1) Construct an image pyramid. Assume that the total number of feature points required is N. Use a hierarchical dynamic allocation strategy to determine the number of feature points in each layer. For the feature point allocation of the i-th layer image, C i :

[0009]

[0010] Where n is the total number of pyramid layers, and l is the inverse of the scale factor;

[0011] 2) Perform parameterized grid division on each layer of image and construct a two-dimensional grid array to control feature distribution. The division method is to adjust the factor a by row-column ratio. i To adjust the grid row and column dimension ratio, and control the number of grids per unit area through the grid density coefficient t, the number of rows and columns of the i-th layer image Rs i and Cs i They are:

[0012] Rs i =a i ×Cs i ,

[0013]

[0014] The number of feature points c extracted from a single grid of the i-th layer image i for:

[0015]

[0016] 3) Adaptive thresholding is used to determine whether a pixel in the grid image is a FAST corner point. The initial threshold T is determined based on global grayscale statistics:

[0017]

[0018] Among them, np is the total number of image pixels, I(x) represents the gray value of the pixel, and k is the average gray value of all pixels. i The preset quota is not reached c i When , the threshold is iteratively attenuated until the number of features is met;

[0019] 4) After the initial feature point extraction is completed through the adaptive threshold, a two-level screening strategy is adopted to ensure the number of features: if the total number of retained feature points is greater than C i , sort by Harris response value in descending order and keep the first c i features, if the number of feature points in the grid is less than c i , then all feature points are retained; after all grids are screened, if the total number of retained feature points is less than C i , then sort all the unretained feature points in the grid by Harris response value, and then take the largest response values ​​in the sequence to supplement the number of feature points until the total number of feature points reaches C i ;

[0020] S3. Use the YOLOX-S target detection network to detect the ORB feature and obtain the object detection frame;

[0021] S4. Obtain a binary mask image based on the depth image to remove dynamic feature points of the prior dynamic objects, specifically including:

[0022] 1) Establish pixel-level spatial correspondence through coordinate system transformation and align the depth image and RGB image;

[0023] 2) Divide the foreground and background regions: Perform grid sampling within the target detection frame. Using the center of the detection frame as the reference point, sample the image in four directions at equal intervals, sampling every m pixels until the pixel distance from the sampling point to the edge of the target detection frame is less than m. Perform median calculation and absolute deviation analysis on the depth values ​​of the sampling points. Multiply the median of the absolute deviation by the scale factor V to obtain the standard deviation σ. The 3σ criterion is used to distinguish between dynamic targets and static backgrounds. If the absolute deviation distance is less than 3σ, the depth value of the point is considered to be close to the median of the depth values ​​of the sampling point set and the point is classified into the dynamic target region. Otherwise, it is classified into the background region.

[0024] 3) Calculate the average depth of the foreground and background based on the depth values ​​of the dynamic target area and the static background area, and use the binary classification clustering method of the depth information to distinguish the dynamic target and background in the target detection frame;

[0025] 4) After dividing the pixel types, the two types of pixels are converted into a binary mask image through an image processing algorithm. Finally, the dynamic feature points of the prior dynamic object are eliminated through the binary mask image to obtain a feature map without the prior dynamic feature points.

[0026] S5. For the feature map without prior dynamic feature points, the optical flow field is first calculated to predict the position of the feature points of the previous frame in the next frame to provide an initial matching estimate. The descriptor nearest neighbor criterion is then used for feature matching. The initial matching set is probabilistically screened through a progressive consistency sampling mechanism to eliminate abnormal matching pairs and obtain a feature point matching set.

[0027] S6. For the obtained feature point matching set, use multi-view geometric constraints to check whether there are dynamic feature points that have not been recognized by the target detection network, and filter out the checked dynamic feature points to obtain a feature map without dynamic feature points;

[0028] S7. After obtaining the feature map with dynamic feature points removed, the feature extraction method of subsequent images is updated to improve the feature extraction efficiency, specifically combining the grid method and the dynamic masking strategy for feature extraction, wherein the dynamic masking strategy is specifically as follows: the image frame corresponding to the obtained feature map with dynamic feature points removed is defined as the previous frame, and for the current frame, the IMU measurement value between the current frame and the previous frame is used to predict the feature position in the current frame, and the feature position predicted by the optical flow is used as the initial position, and matching features are found in the current frame. The features that are successfully tracked are defined as stable features, and the features that fail to be tracked are defined as unstable features. In the image, a circular mask is set with the stable feature as the center, and a circular mask is also set for the area where the unstable feature is taken. Then, new features are detected from the area in the current frame that is not blocked by the circular mask; and then the feature map of the current frame with dynamic feature points removed is obtained through steps S3-S6.

[0029] S8. Perform static background restoration on the feature map with the dynamic feature points removed to obtain a static background map. Specifically, starting from the key frame to be restored, a sliding window containing 5 frames is extended before and after it. Each frame in the sliding window is defined as a candidate reference frame. For the candidate reference frames in the window, their visual features are extracted and the number M is counted. Then, feature matching is performed on the candidate reference frame and the number of matching feature points N is recorded. If the absolute difference between the two is less than 50, it is determined that the reference frame has rich image information.

[0030] In the image alignment stage, each keyframe image is aligned one by one to the frame to be repaired based on the grid optical flow field between adjacent frames. After alignment is completed, the corresponding pixels in the missing area of ​​the frame to be repaired are retrieved along the motion trajectory: when there is a single valid pixel, it is directly filled; if there are multiple matching pixels, the arithmetic mean of their pixel values ​​is calculated and used for repair;

[0031] S9. Through the collaborative registration mechanism of visual-depth dual-channel information, the restored static background and depth image are used to perform three-dimensional space joint reconstruction.

[0032] Furthermore, the specific method of using the binary classification clustering method of depth information to distinguish the dynamic target and the background in the target detection frame is:

[0033] 31) Set the depth values ​​of the two cluster centers;

[0034] 32) Traverse the depth values ​​of the pixels in the detection box and calculate the difference between the depth value of the pixel and the cluster center value;

[0035] 33) Classification based on the nearest neighbor method;

[0036] 34) Recalculate the cluster centers of the two classes;

[0037] 35) If the difference between the cluster centers is lower than the threshold, or the number of iterations exceeds the maximum number, the iteration is stopped; otherwise, the next round of iteration is performed; the specific iterative calculation process is:

[0038] Assume that the dynamic target class is class α and the static background class is class β; in the first round of iteration, the average depth value d of the dynamic target pixels is αvg Set it as the center point of the dynamic target class and set the pixel value with the largest relative depth in the previous sampling point set K Set as the cluster center of the static background class; the first round of center points are set as follows:

[0039]

[0040] Traverse the depth values ​​of the pixels in the detection box and calculate the difference between the depth value of the pixel and the cluster center value:

[0041]

[0042] where d K2α Represents the distance between the pixel value of the point and the center point of class α, d K2β Represents the distance between the pixel value of the point and the center point of class β;

[0043] Then reclassify using the nearest neighbor method and calculate the average center point depth value of the new cluster as the two cluster centers for the next round. The new center points are as follows:

[0044]

[0045] Among them, c α Contains a pixels in total, c β Contains a total of b pixels;

[0046] At this time, the difference between the two cluster centers and the number of iterations are judged. If the conditions are met, the iteration is stopped; otherwise, the iteration is restarted.

[0047] Furthermore, the specific method of probabilistically screening the initial matching set through the progressive consistency sampling mechanism is:

[0048] 1) For each feature point, find its corresponding best matching point and second best matching point, and calculate the distance dmin1 of the best matching point and the distance dmin2 of the second best matching point;

[0049] 2) Calculate the distance ratio of each pair of feature points and define the evaluation function;

[0050] 3) Arrange the feature points in descending order according to the evaluation function value, form a group of eight feature points, calculate the quality sum of each group, and sort the groups by quality;

[0051] 4) Select the 8 highest-quality matching points from the previously sorted groups and use these points to calculate the fundamental matrix F;

[0052] 5) Eliminate the selected 8 groups of matching points from the original feature point set, and calculate the corresponding projection points of the remaining feature points in the subset according to the currently estimated basic matrix;

[0053] 6) For each remaining feature point, calculate the error between it and its projection point and compare it with the set error threshold ε. If e<ε, the point is an interior point, otherwise it is an exterior point.

[0054] 7) Calculate the number of inliers t that meet the conditions and compare it with the set inlier number standard T. If t>T, update the number of inliers, otherwise return to step 4;

[0055] 8) After updating the number of inliers t, recalculate the fundamental matrix F based on the newly obtained inliers, obtain new inliers, and repeat steps 5 to 8 to enhance the reliability of the model.

[0056] Furthermore, the specific method of using multi-view geometric constraints to check whether there are dynamic feature points that are not recognized by the target detection network is:

[0057] Assume that image frame I t with i t+Δ Corresponding to the image collection at two moments, for a pair of matched static feature points k t With k t+Δ , the two points satisfy the epipolar constraint; when the camera optical center W t With W t+Δ When the spatial line of intersects the projection ray of the three-dimensional point K, the three spatial elements together define the epipolar plane with geometric constraints. The intersection line L of the plane and the two image planes is t , L t+Δ The line connecting the two optical centers forms the system baseline. In this geometric framework, the static feature points follow strict spatial constraints: their projection points k t+Δ Strictly located on the corresponding epipolar line L t+Δ In the neighborhood, dynamic feature points violate epipolar geometry constraints due to changes in three-dimensional coordinates, which is manifested as a significant increase in the Euclidean distance between matching points. By calculating the degree of deviation between the projection position of the feature point and the theoretical epipolar line, when the deviation exceeds the preset threshold δ, the feature point is determined to be a dynamic target.

[0058] The beneficial effects of the present invention are as follows: In response to the problems of low positioning accuracy and poor map robustness of existing visual SLAM systems in complex dynamic scenes, the present invention proposes an optimization algorithm based on the improved ORB-SLAM3 framework, which significantly improves the system performance in complex environments through lightweight feature extraction modules, dynamic feature filtering and static background restoration technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is an overall flow chart of the present invention.

[0060] Figure 2 It is a comparison chart of relative pose errors.

[0061] Figure 3 This is a comparison chart of absolute trajectory errors.

[0062] Figure 4 is the comparison of RPE visualization results, where (a)-(I) correspond to different methods;

[0063] Figure 5 is a schematic diagram of ATE visualization results, where (a)-(I) correspond to different methods;

[0064] Figure 6 This is a comparison chart of the average frame rate of the three algorithms. DETAILED DESCRIPTION

[0065] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0066] like Figure 1 As shown, the present invention includes the following steps:

[0067] 1. Obtain the RGB image sequence and depth image sequence of the target scene respectively according to the RGB-D camera.

[0068] 2. For RGB images, the improved grid division feature point extraction method is used to quickly extract evenly distributed high-quality ORB feature points. The specific steps are as follows:

[0069] (1) Pyramid feature allocation mechanism

[0070] In the image pyramid architecture, let the total feature point demand be N, and adopt the hierarchical dynamic allocation strategy to determine the feature quota of each layer. For the feature point quota C of the i-th layer image, i , as shown below:

[0071]

[0072] Among them, n is generally 8, indicating the total number of pyramid layers, and l is the inverse of the scale factor.

[0073] (2) Dynamic grid division strategy

[0074] Implement parameterized grid division on each level image and construct a two-dimensional grid array to control feature distribution. i To adjust the grid row and column dimension ratio, and control the number of grids per unit area through the grid density coefficient t.i and Cs i are the number of rows and columns of the i-th layer image respectively:

[0075] Rs i =a i ×Cs i

[0076]

[0077] Among them, Rs i and Cs i are the number of rows and columns of the i-th layer image respectively.

[0078] Assume that the number of feature points extracted from a single grid of the i-th layer image is c i , which is obtained by equally dividing the total quota of the level as shown in the following formula:

[0079]

[0080] (3) Adaptive threshold extraction algorithm

[0081] A dynamic threshold adjustment mechanism is proposed, and an adaptive threshold method is used to determine whether a pixel in a grid image is a FAST corner point. The initial threshold T is determined based on global grayscale statistics:

[0082]

[0083] Among them, np is the total number of image pixels, i(x) represents the gray value of the pixel, and k is the average gray value of all pixels. i The preset quota is not reached c i When , the threshold is iteratively attenuated (each time reduced to 1 / 4 of the original value) until the feature quantity requirement is met.

[0084] (4) Feature Optimization and Compensation Mechanism

[0085] After the adaptive threshold completes the initial extraction of feature points, a two-level screening strategy is used to ensure that the number of features is accurately controlled. If the total number of retained feature points is greater than C i , sort by Harris response value in descending order and keep the first c i features, if the number of feature points in the grid is less than c i , all feature points are retained.

[0086] After all grids are screened, if the total number of retained feature points is less than C i , then sort all the unretained feature points in the grid by Harris response value, and then take the largest response values ​​in the sequence to supplement the number of feature points until the total number of feature points reaches C i .

[0087] (5) Feature Update

[0088] After filtering out dynamic feature points and matching static feature points, feature masks are added to the feature detection method based on the grid division method. This can effectively eliminate unstable features such as noise and dynamic features with inconsistent motion, and significantly improve the real-time performance of the system while ensuring the uniformity of feature distribution.

[0089] Stable features and new features are indicated in red and blue, respectively. The green circle represents the optical flow prediction. Successfully tracked features turn red, while untracked features turn purple. An orange circular no-sampling zone (radius R) is established centered on the red feature to prevent repeated sampling of adjacent areas. A dynamic masking domain is constructed within the purple feature distribution to suppress the regeneration of unstable features. New feature points are detected from the unobstructed areas of the current frame.

[0090] 3. Use the YOLOX-S target detection network to detect the ORB feature and obtain the object detection frame.

[0091] 4. Obtain a binary mask image based on the depth image to remove the dynamic feature points of the prior dynamic objects.

[0092] (1) First, through coordinate system transformation, the pixel-level spatial correspondence is established to align the depth image and RGB image.

[0093] (2) Secondly, the foreground and background areas are divided: grid sampling is performed within the target detection frame. The center of the detection frame is used as the reference point, and sampling is extended equidistantly in four directions, sampling every n pixels until the pixel distance from the sampling point to the edge of the target detection frame is less than n.

[0094] (3) The median is then used to distinguish unstable outliers in the depth image information: the median of the depth values ​​of the sampling points is calculated and the absolute deviation is analyzed. The standard deviation σ is obtained by multiplying the median of the absolute deviation by the scale factor V. The 3σ criterion is used to distinguish dynamic targets from static backgrounds. If the absolute deviation distance is less than the threshold, the depth value of the point is considered to be close to the median of the depth values ​​of the sampling point set and the point is classified as a dynamic target area. Otherwise, it is classified as a background area.

[0095] (4) Then, the average depth of the foreground and background is calculated based on the depth values ​​of the dynamic target area and the static background area. The binary classification clustering method of the depth information is used to distinguish the dynamic target and background in the target detection frame. The specific method is as follows:

[0096] By constructing a bimodal characteristic analysis of the depth histogram, dynamic target pixel areas and static background areas are proposed. The depth values ​​of dynamic target pixels obey a low variance distribution (σ<0.3m), while the depth values ​​of residual static background pixels show a systematic offset due to their location at a farther spatial level, thereby achieving the distinction between dynamic target pixel areas and static background areas.

[0097] (1) Set the depth values ​​of the two cluster centers;

[0098] (2) Traverse the depth values ​​of the pixels in the detection box and calculate the difference between the depth value of the pixel and the cluster center value;

[0099] (3) Classification based on the nearest neighbor method

[0100] (4) Recalculate the cluster centers of the two classes;

[0101] (5) If the difference between the cluster centers is lower than the threshold, or the number of iterations exceeds the maximum number, the iteration is stopped; otherwise, the next round of iteration is performed.

[0102] Assume that the dynamic target class is class α and the static background class is class β. In the first round of iteration, the average depth value of the dynamic target pixels is set as the center point of the dynamic target class, and the pixel value with the maximum relative depth in the previous step's sampling point set K is set as the cluster center point of the static background class. The first round center point is set as follows:

[0103]

[0104] Traverse the depth values ​​of the pixels in the detection box and calculate the difference between the depth value of the pixel and the cluster center value;

[0105]

[0106] where d K2α Represents the distance between the pixel value of the point and the center point of class α, d K2β Represents the distance between the pixel value of the point and the center point of class β.

[0107] Then reclassify using the nearest neighbor method and calculate the average center depth value of the new cluster as the two cluster centers for the next round. The new center points are as follows:

[0108]

[0109] At this point, the difference between the two cluster centers and the number of iterations are determined, and the iteration is stopped if the conditions are met. Otherwise, the iteration is restarted.

[0110] After classifying the pixels, an image processing algorithm converts the two types of pixels directly into a binary image output, which can be used as a mask image for subsequent data processing. Morphological operations such as erosion and dilation are then used to remove noise, ensuring the smoothness and integrity of the depth image.

[0111] Finally, the dynamic feature points of the prior dynamic objects are eliminated by binarizing the mask image, and a feature map without the prior dynamic feature points is obtained.

[0112] 5. For feature maps that have been stripped of prior dynamic feature points, we first calculate the optical flow field to predict the positions of feature points from the previous image in the next image, providing an initial matching estimate and determining a search range for matching feature points. We then use the descriptor nearest neighbor criterion for feature matching. We then perform probabilistic screening of the initial matching set using the Progressive Consensus Sampling (PROSAC) algorithm, effectively eliminating anomalous matching pairs and ultimately obtaining a matching set of feature points.

[0113] The present invention proposes to use the Progressive Consistent Sampling (PROSAC) algorithm steps as follows.

[0114] This method improves the accuracy and efficiency of model fitting by gradually increasing the quality of sample selection. The algorithm steps are as follows:

[0115] (1) For each feature point, find its corresponding best matching point and second best matching point, and calculate the distance dmin1 of the best matching point and the distance dmin2 of the second best matching point.

[0116] (2) Calculate the distance ratio of each pair of feature points and define the evaluation function.

[0117] (3) Arrange the feature points in descending order according to the evaluation function value, with eight feature points as a group, calculate the quality sum of each group, and sort the groups.

[0118] (4) Select the 8 groups of matching points with the highest quality from the previously sorted groups and use these points to calculate the fundamental matrix F.

[0119] (5) Eliminate the selected 8 groups of matching points from the original feature point set, and calculate the corresponding projection points of the remaining feature points in the subset based on the currently estimated homography matrix.

[0120] (6) For each remaining feature point, calculate the error between it and its projection point and compare it with the set error threshold ε. If e<ε, the point is an interior point; otherwise, it is an exterior point.

[0121] (7) Calculate the number of inliers t that meet the conditions and compare it with the set inlier number standard T. If t>T, update the number of inliers; otherwise, return to the iterative operation.

[0122] (8) After updating the number of inliers t, recalculate the fundamental matrix F according to the newly obtained inliers, obtain new inliers, and repeat the above steps to enhance the reliability of the model.

[0123] 6. For the obtained feature point matching set, use multi-view geometric constraints to check whether there are dynamic feature points that are not recognized by the target detection network and filter them out.

[0124] Assume that image frame I t with I t+Δ Corresponding to the image collection at two moments, for a pair of matched static feature points k t With k t+Δ , the two points satisfy the epipolar constraint. t With O t+Δ When the spatial line of intersecting with the projection ray of the three-dimensional point K, these three spatial elements together define the epipolar plane with geometric constraint characteristics. The intersection line L of the plane and the two image planes t , L t+Δ The line connecting the two optical centers forms the system baseline.

[0125] In this geometric framework, static feature points follow strict spatial constraints: their projection points k t+Δ Should be strictly located on the corresponding polar line L t+Δ Within the neighborhood, dynamic feature points violate epipolar geometry constraints due to changes in their three-dimensional coordinates, which is manifested by a significant increase in the Euclidean distance between matching points.

[0126] Based on this characteristic, the present invention constructs a dynamic feature discrimination criterion: by calculating the degree of deviation between the projected position of a feature point and the theoretical epipolar line, the feature point is considered a dynamic target when the deviation exceeds a preset threshold δ. This screening mechanism based on spatial geometric constraints effectively achieves accurate identification and filtering of dynamic interference factors in the scene.

[0127] In subsequent feature matching, the system quantitatively analyzes the motion characteristics of each feature point by constructing a Euclidean distance measurement model between matching pairs of feature points. When this measurement value exceeds a preset threshold, it is identified as a dynamic feature point, enabling accurate separation of moving objects. This results in a feature map with dynamic pixels (feature points) removed.

[0128] 7. After obtaining the feature map with dynamic feature points removed, combine the grid method and dynamic masking strategy to achieve efficient feature update. That is, based on the grid division method, feature masking is added to the feature detection method. The specific steps are as follows:

[0129] In the previous frame, stable features are represented in red and newly detected features are represented in blue. When the current frame arrives, the IMU measurements between the current frame and the previous frame are used to predict the feature position in the current frame (green circle). Using the feature position predicted by the optical flow as the initial position, matching features are found in the current frame. Features that are successfully tracked are marked in red, while features that fail to be tracked are marked as unstable features (purple). To avoid duplication and aggregation of feature detection, an orange circular mask centered on the stable feature is set to prevent repeated sampling of neighboring areas; the area where the unstable feature is located is blocked with a purple circle to suppress the regeneration of unstable features. Based on the mask, new features are detected from the unmasked area in the current frame and marked in blue.

[0130] (8) In response to the above-mentioned problem that the feature points in some areas of the feature map are sparse due to the filtering out of some dynamic feature points, the present invention proposes an optimal grid mapping background restoration algorithm based on the grid flow model, which uses an inter-frame feature matching method to determine the best restoration reference frame.

[0131] The specific method is to start with the target keyframe F to be repaired and extend a sliding window containing 5 frames before and after it. For each frame in the window, its visual features are extracted and the number M is counted. Then, feature matching is performed between the candidate reference frame and the target frame, and the number of matching feature points N is recorded. If the absolute difference between the two is less than 50, the reference frame is considered to have rich image information.

[0132] During the image alignment phase, each keyframe image is aligned to the frame to be inpainted, based on the grid optical flow field between adjacent frames. After alignment, the corresponding pixels in the missing area of ​​the frame to be inpainted are retrieved along the motion trajectory. If a single valid pixel exists, it is directly filled; if multiple matching pixels exist, the arithmetic mean of their values ​​is calculated and used for inpainting.

[0133] Therefore, the static background restoration of the dynamically occluded area is achieved through the static information in the adjacent frames.

[0134] (9) Through the collaborative registration mechanism of visual-depth dual-channel information, the restored static background and depth image are jointly reconstructed into three-dimensional space. Then, a back-end nonlinear optimization correction system is introduced to iteratively solve the optimal estimation parameters of the system pose, and finally a dense point cloud map model with global consistency is established.

[0135] This paper addresses the problems of low positioning accuracy and poor map robustness of existing visual SLAM systems in complex dynamic scenes. It proposes an optimization algorithm based on the improved ORB-SLAM3 framework. By using a lightweight feature extraction module, dynamic feature filtering, and static background restoration technology, it significantly improves system performance in complex environments. The specific technical implementation results are as follows:

[0136] 1. Efficient filtering of dynamic features and accurate pose estimation

[0137] 1) Improve feature extraction and matching: Use grid partitioning method to optimize ORB feature extraction distribution, combine PROSAC algorithm to replace traditional RANSAC, and use progressive sampling strategy to quickly eliminate mismatched points, thus improving feature matching efficiency by about 30%. Figure 2 As shown in Figure 3, the pose estimation error is reduced by 60.17% to 70.48% relative to the ORB-SLAM3 error, and by 10.07% to 17.39% relative to the RDS-SLAM error. This shows that the feature matching strategy of this system is more robust in dynamic scenes.

[0138] 2) Dynamic object segmentation and filtering: Introducing the lightweight YOLOX-S semantic detection network, combined with the depth information binary clustering algorithm, to achieve pixel-level mask segmentation of dynamic objects. Dynamic feature points are further filtered out through multi-view geometric constraints (epipolar geometry), such as Figure 4 , 5 shows that in low dynamic environment, RPE and ATE are slightly reduced compared with the original system; while in high dynamic environment, RPE and ATE are greatly reduced. This shows that the performance of the present invention has been improved to a certain extent.

[0139] 2. Static background restoration and high-precision mapping

[0140] 1) Optimal Grid Mapping Inpainting: Addressing background loss caused by dynamic object occlusion, the system dynamically inpaints occluded areas based on feature matching between adjacent frames and mesh flow alignment, restoring static background information. The inpainted RGB-D data is mapped to 3D space to construct a globally consistent dense point cloud map, reducing redundant point clouds by approximately 15% compared to ORB-SLAM3.

[0141] 2) Dynamic ghosting removal: Dynamic objects in keyframes are filtered out through semantic masking. Combined with point cloud downsampling and outlier removal, this effectively eliminates the ghosting of dynamic objects in point cloud maps, improving map readability and practicality.

[0142] 3. Real-time and robustness optimization

[0143] 1) Lightweight semantic detection: The YOLOX-S network inference speed reaches 30fps. The frame rate of this system is between ORB-SLAM3 and RDS-SLAM: Compared with ORB-SLAM3, the frame rate drops by 36.19% to 47.16%, but this is because ORB-SLAM3 does not have a thread for detecting dynamic objects. Figure 6 As shown in the figure, the frame rate of the system of the present invention is improved by 8.42% to 26.65% compared with RDS-SLAM.

[0144] 4. Experimental verification and effect comparison

[0145] 1) Dataset testing: In the high-dynamic sequence (fr3_walking) of the TUM dataset, the positioning error is reduced by 63.98%, and the trajectory accuracy is significantly better than ORB-SLAM3 and Dyna-SLAM.

[0146] 2) Actual environment verification: After dynamic features are filtered out, the density of static feature points is retained to more than 85%. The constructed dense point cloud map has clear static target contours and completely eliminates dynamic interference.

Claims

1. A SLAM three-dimensional modeling method suitable for complex dynamic scenes, characterized in that: The following steps are involved: S1, obtain the RGB image sequence and depth image sequence of the target scene respectively through the RGB-D camera; S2. For RGB images, an improved grid division feature point extraction method is used to extract ORB feature points, specifically including: 1) Construct an image pyramid. Assume that the total number of feature points required is N. Use a hierarchical dynamic allocation strategy to determine the number of feature points in each layer. For the feature point allocation of the i-th layer image, C i : Where n is the total number of pyramid layers, and l is the inverse of the scale factor; 2) Perform parameterized grid division on each layer of image and construct a two-dimensional grid array to control feature distribution. The division method is to adjust the factor a by row-column ratio. i To adjust the grid row and column dimension ratio, and control the number of grids per unit area through the grid density coefficient t, the number of rows and columns of the i-th layer image Rs i and Cs i They are: Rs i =a i ×Cs i , The number of feature points c extracted from a single grid of the i-th layer image i for: 3) Adaptive thresholding is used to determine whether a pixel in the grid image is a FAST corner point. The initial threshold T is determined based on global grayscale statistics: Among them, np is the total number of image pixels, i(x) represents the gray value of the pixel, and k is the average gray value of all pixels. i The preset quota is not reached c i When , the threshold is iteratively attenuated until the number of features required is met; 4) After the initial feature point extraction is completed through the adaptive threshold, a two-level screening strategy is adopted to ensure the number of features: if the total number of retained feature points is greater than C i , sort by Harris response value in descending order and keep the first c i features, if the number of feature points in the grid is less than c i , then all feature points are retained; after all grids are screened, if the total number of retained feature points is less than C i , then sort all the unretained feature points in the grid by Harris response value, and then take the largest response values ​​in the sequence to supplement the number of feature points until the total number of feature points reaches C i ; S3. Use the YOLOX-S target detection network to detect the ORB feature and obtain the object detection frame; S4. Obtain a binary mask image based on the depth image to remove dynamic feature points of the prior dynamic objects, specifically including: 1) Establish pixel-level spatial correspondence through coordinate system transformation and align the depth image and RGB image; 2) Divide the foreground and background regions: Perform grid sampling within the target detection frame. Using the center of the detection frame as the reference point, sample the image in four directions at equal intervals, sampling every m pixels until the pixel distance from the sampling point to the edge of the target detection frame is less than m. Perform median calculation and absolute deviation analysis on the depth values ​​of the sampling points. Multiply the median of the absolute deviation by the scale factor V to obtain the standard deviation σ. The 3σ criterion is used to distinguish between dynamic targets and static backgrounds. If the absolute deviation distance is less than 3σ, the depth value of the point is considered to be close to the median of the depth values ​​of the sampling point set and the point is classified into the dynamic target region. Otherwise, it is classified into the background region. 3) Calculate the average depth of the foreground and background based on the depth values ​​of the dynamic target area and the static background area, and use the binary classification clustering method of the depth information to distinguish the dynamic target and background in the target detection frame; 4) After dividing the pixel types, the two types of pixels are converted into a binary mask image through an image processing algorithm. Finally, the dynamic feature points of the prior dynamic object are eliminated through the binary mask image to obtain a feature map without the prior dynamic feature points. S5. For the feature map without prior dynamic feature points, the optical flow field is first calculated to predict the position of the feature points of the previous frame in the next frame to provide an initial matching estimate. The descriptor nearest neighbor criterion is then used for feature matching. The initial matching set is probabilistically screened through a progressive consistency sampling mechanism to eliminate abnormal matching pairs and obtain a feature point matching set. S6. For the obtained feature point matching set, use multi-view geometric constraints to check whether there are dynamic feature points that have not been recognized by the target detection network, and filter out the checked dynamic feature points to obtain a feature map without dynamic feature points; S7. After obtaining the feature map with dynamic feature points removed, the feature extraction method of subsequent images is updated to improve the feature extraction efficiency, specifically combining the grid method and the dynamic masking strategy for feature extraction, wherein the dynamic masking strategy is specifically as follows: the image frame corresponding to the obtained feature map with dynamic feature points removed is defined as the previous frame, and for the current frame, the IMU measurement value between the current frame and the previous frame is used to predict the feature position in the current frame, and the feature position predicted by the optical flow is used as the initial position, and matching features are found in the current frame. The features that are successfully tracked are defined as stable features, and the features that fail to be tracked are defined as unstable features. In the image, a circular mask is set with the stable feature as the center, and a circular mask is also set for the area where the unstable feature is taken. Then, new features are detected from the area in the current frame that is not blocked by the circular mask; and then the feature map of the current frame with dynamic feature points removed is obtained through steps S3-S6. S8. Perform static background restoration on the feature map with the dynamic feature points removed to obtain a static background map. Specifically, starting from the key frame to be restored, a sliding window containing 5 frames is extended before and after it. Each frame in the sliding window is defined as a candidate reference frame. For the candidate reference frames in the window, their visual features are extracted and the number M is counted. Then, feature matching is performed on the candidate reference frame and the number of matching feature points N is recorded. If the absolute difference between the two is less than 50, it is determined that the reference frame has rich image information. In the image alignment stage, each keyframe image is aligned one by one to the frame to be repaired based on the grid optical flow field between adjacent frames. After alignment is completed, the corresponding pixels in the missing area of ​​the frame to be repaired are retrieved along the motion trajectory: when there is a single valid pixel, it is directly filled; if there are multiple matching pixels, the arithmetic mean of their pixel values ​​is calculated and used for repair; S9. Through the collaborative registration mechanism of visual-depth dual-channel information, the restored static background and depth image are used to perform three-dimensional space joint reconstruction.

2. A SLAM three-dimensional modeling method applicable to complex dynamic scenes according to claim 1, characterized in that, The specific method of using the binary classification clustering method of depth information to distinguish dynamic targets and backgrounds in the target detection frame is: 31) Set the depth values ​​of the two cluster centers; 32) Traverse the depth values ​​of the pixels in the detection box and calculate the difference between the depth value of the pixel and the cluster center value; 33) Classification based on the nearest neighbor method; 34) Recalculate the cluster centers of the two classes; 35) If the difference between the cluster centers is lower than the threshold, or the number of iterations exceeds the maximum number, the iteration is stopped; otherwise, the next round of iteration is performed; the specific iterative calculation process is: Assume that the dynamic target class is class α and the static background class is class β; in the first round of iteration, the average depth value d of the dynamic target pixels is αv Set it as the center point of the dynamic target class and set the pixel value with the largest relative depth in the previous sampling point set K Set as the cluster center of the static background class; the first round of center points are set as follows: Traverse the depth values ​​of the pixels in the detection box and calculate the difference between the depth value of the pixel and the cluster center value: where d K2α Represents the distance between the pixel value of the point and the center point of class α, d K2β Represents the distance between the pixel value of the point and the center point of class β; Then reclassify using the nearest neighbor method and calculate the average center point depth value of the new cluster as the two cluster centers for the next round. The new center points are as follows: Among them, c α Contains a pixels in total, c β Contains a total of b pixels; At this time, the difference between the two cluster centers and the number of iterations are judged. If the conditions are met, the iteration is stopped; otherwise, the iteration is restarted.

3. A SLAM three-dimensional modeling method applicable to complex dynamic scenes according to claim 1, characterized in that, The specific method of probabilistically screening the initial matching set through the progressive consistency sampling mechanism is: 1) For each feature point, find its corresponding best matching point and second best matching point, and calculate the distance dmin1 of the best matching point and the distance dmin2 of the second best matching point; 2) Calculate the distance ratio of each pair of feature points and define the evaluation function; 3) Arrange the feature points in descending order according to the evaluation function value, form a group of eight feature points, calculate the quality sum of each group, and sort the groups; 4) Select the 8 highest-quality matching points from the previously sorted groups and use these points to calculate the fundamental matrix F; 5) Eliminate the selected 8 groups of matching points from the original feature point set, and calculate the corresponding projection points of the remaining feature points in the subset according to the currently estimated basic matrix; 6) For each remaining feature point, calculate the error between it and its projection point and compare it with the set error threshold ε. If e<ε, the point is an interior point, otherwise it is an exterior point. 7) Calculate the number of inliers t that meet the conditions and compare it with the set inlier number standard T. If t>T, update the number of inliers, otherwise return to step 4; 8) After updating the number of inliers t, recalculate the fundamental matrix F based on the newly obtained inliers, obtain new inliers, and repeat steps 5 to 8 to enhance the reliability of the model.

4. A SLAM three-dimensional modeling method applicable to complex dynamic scenes according to claim 1, characterized in that, The specific method of using multi-view geometric constraints to check whether there are dynamic feature points that are not recognized by the target detection network is: Assume that image frame I t with I t+Δ Corresponding to the image collection at two moments, for a pair of matched static feature points k t With k t+Δ , the two points satisfy the epipolar constraint; when the camera optical center O t With O t+Δ When the spatial line of intersects the projection ray of the three-dimensional point K, the three spatial elements together define the epipolar plane with geometric constraints. The intersection line L of the plane and the two image planes is t , L t+Δ The line connecting the two optical centers forms the system baseline. In this geometric framework, the static feature points follow strict spatial constraints: their projection points k t+Δ Strictly located on the corresponding epipolar line L t+α In the neighborhood, dynamic feature points violate epipolar geometry constraints due to changes in three-dimensional coordinates, which is manifested as a significant increase in the Euclidean distance between matching points. By calculating the degree of deviation between the projection position of the feature point and the theoretical epipolar line, when the deviation exceeds the preset threshold δ, the feature point is determined to be a dynamic target.

Images