3D Gaussian SLAM Method Based on Dynamic Suppression and Incremental Optimization

Abstract

This invention discloses a 3D Gaussian SLAM method based on dynamic suppression and incremental optimization, comprising: acquiring RGB-D image frames; constructing a dynamic object mask using an integrated lightweight semantic segmentation network; simultaneously removing dynamic objects from the acquired image frames in both the color and depth domains; and initializing 3D Gaussian elements based on the dynamically contamination-suppressed image frames; iteratively optimizing and estimating the camera pose of subsequent input images by constructing a joint depth-color residual constraint; and adding a new Gaussian element to each pixel in the region requiring densification indicated by the mask to refine the scene representation of insufficiently rendered regions. The dynamic contamination suppression mechanism proposed in this invention provides an accurate static scene foundation for Gaussian element incremental optimization, while Gaussian element incremental optimization, in turn, improves the completeness and quality of map construction, thereby jointly improving the localization and mapping performance of the entire system.

Description

Technical Field

[0001] This invention relates to computer vision and robot autonomous navigation technology, specifically to a 3D Gaussian SLAM method based on dynamic suppression and incremental optimization. Background Technology

[0002] In the field of mobile robotics, SLAM (Simultaneous Localization and Mapping) technology is crucial. It enables robots to self-localize in unknown environments and perceive their surroundings, thereby creating maps and estimating their own position and orientation. Traditional SLAM systems assume that the environment is static, meaning that the landmarks used for localization are typically considered unchanging. This assumption usually does not hold true in real life because dynamic objects (such as pedestrians or vehicles) are ubiquitous, easily leading to data association errors, which in turn severely impacts the accuracy of localization and the global consistency of the map.

[0003] This challenge is even more pronounced in smart warehousing scenarios. Smart warehousing is a modern warehousing ecosystem that integrates cutting-edge technologies such as the Internet of Things, big data, and artificial intelligence, aiming to automate and intelligently manage the efficient storage and sorting of goods. However, in warehousing scenarios, the storage bins are frequently moved, rearranged, or added, forcing robots to constantly update their positioning information. Furthermore, the layout of various objects in the warehouse changes, and even new obstacles appear, requiring semiconductor intelligent mobile robots to have the ability to relocalize and update maps in real time. Simultaneously, due to the presence of numerous other dynamic AMR robots and human operations (such as forklift operations or worker activities) within the warehouse, AMR robots must be able to interact effectively with these dynamic elements while avoiding collisions.

[0004] In response, patent application CN118172514A discloses a dynamic SLAM method and system for robots in intelligent warehouse dynamic environments. It designs a dynamic object detection network to detect and segment dynamic objects based on SLAM, and proposes a scene completion network to re-complete the areas removed by the segmentation network, thus constructing a complete static scene to improve the accuracy of localization. However, this method still has some technical defects: (1) Limited integrity and information richness of map construction: This method completes the scene after removing dynamic objects, and then constructs the map based on a traditional SLAM network (such as ORB-SLAM2). The map constructed in this way is still a sparse point cloud map, which contains limited information, wastes a lot of information in image frames, and is difficult for people to understand intuitively. (2) Insufficient adaptability to dynamic scenes: In dynamic scenes, although this method processes dynamic objects through dynamic object detection and completion networks, it still relies on the traditional SLAM algorithm framework. When dealing with complex dynamic environments, it may still be affected by dynamic objects, resulting in a decrease in the accuracy and global consistency of map construction. (3) Low utilization of sensor data: This method mainly focuses on the processing of dynamic objects and the construction of static scenes. It fails to make full use of the rich data input by visual sensors (such as depth information, color information, etc.), resulting in low integration and utilization efficiency of information in the process of map construction and positioning.

[0005] Therefore, it is necessary to propose a new dynamic SLAM method for robots in intelligent warehousing scenarios to meet the high requirements of autonomous localization and mapping of intelligent mobile robots due to the dynamic and uncertain nature of the warehousing environment. Summary of the Invention

[0006] The present invention aims to at least partially solve one of the technical problems existing in the related art.

[0007] The purpose of this invention is to provide a 3D Gaussian SLAM method based on dynamic suppression and incremental optimization. In the dynamic scenario of intelligent warehousing, it effectively suppresses the pollution of the map initialization stage of the SLAM process by dynamic objects, and constructs a high-precision and globally consistent scene map, thereby improving the robot's localization and mapping performance in dynamic environments.

[0008] To achieve the above objectives, the present invention provides a 3D Gaussian SLAM method based on dynamic suppression and incremental optimization, comprising the following steps:

[0009] S1. Obtain RGB-D image frames, construct dynamic object masks using integrated lightweight semantic segmentation networks, and establish a depth-color cross-modal mask synchronization mechanism to synchronously remove dynamic objects in the acquired image frames in the color and depth domains, thus completing the dynamic contamination suppression of the image frames. Then, perform 3D Gaussian initialization based on the image frames after dynamic contamination suppression.

[0010] S2. After initialization, the camera pose of subsequent input images is iteratively optimized and estimated by constructing a joint depth-color residual constraint.

[0011] S3. After completing camera tracking and obtaining accurate camera pose estimation, static repair is performed on the color and depth values ​​of under-rendered areas using historical frames and local neighboring region information. Based on the repaired color and depth information, a new Gaussian element is added to each pixel in areas that need to be densified, thus refining the scene representation of under-rendered areas.

[0012] A further preferred embodiment of the present invention is that, in step S1, a dynamic object mask is constructed using an integrated lightweight semantic segmentation network. The specific steps are as follows:

[0013] Deploy a real-time semantic segmentation network on the input RGB image Perform dynamic object instance segmentation and generate pixel-level dynamic region binary masks. Furthermore, by integrating depth information, a binary mask is extracted to further segment the movable object region affected by dynamic objects. The dynamic object mask of the final RGB image is obtained by adding them together. , represented as:

[0014] ;

[0015] RGB image after masking Represented by pixel-by-pixel and channel-by-channel masking operations:

[0016] ;

[0017] in, Indicates the height of the input image. Indicates the width of the input image. This represents the red, green, and blue color channels of the input image; Indicates pixel position It belongs to the dynamic region. Represents pixels It belongs to the static area; Represents pixels in a planar image. It is a pixel coordinates The x-axis is... The vertical axis is denoted by .

[0018] Preferably, in step S1, a depth-color cross-modal mask synchronization mechanism is established to synchronously remove dynamic objects in the acquired image frames in both the color and depth domains. Specifically:

[0019] After setting a mask for the RGB image, the RGB image and the depth image are associated based on the timestamp. For RGB and depth images with a timestamp interval of less than 0.02 seconds, the depth value of the corresponding position in the depth image is synchronously set to zero for regions in the RGB image where the pixel value is zero, as shown below:

[0020] ;

[0021] in, For RGB images Depth images with timestamp intervals less than 0.02s and All images are real images obtained from sensors; Depth image after forcibly setting the depth value of the dynamic region to zero.

[0022] Preferably, in step S2, the camera pose is iteratively optimized and estimated for subsequent input images by constructing a joint depth-color residual constraint, specifically as follows:

[0023] Joint registration is performed using RGB and depth map information from the input image. The pose estimation results are optimized based on the residuals between the rendered RGB image, depth image, contour image, and the real depth map. A rendering quality threshold strategy is established, using only fully rendered and optimized pixel regions from the contour image for pose gradient calculation, minimizing the following loss to update the camera pose:

[0024] ;

[0025] in, This represents an RGB image rendered using the 3D Gaussian method. This represents a depth image rendered using the 3D Gaussian method. This represents the rendered outline image; and These are the weighting coefficients for depth map rendering error and color rendering error, respectively, used to measure the importance of different information sources in the optimization process; This means that rendering loss is calculated only in pixel regions where the contour image optimization is effective. express Loss function.

[0026] Preferably, in step S3, after completing camera tracking and obtaining accurate camera pose estimation, a reliable representation of the Gaussian element position distribution in the scene is established by combining the geometric information provided by the depth image, and an adaptive densification mechanism is initiated for the areas in the scene that are not fully covered by the Gaussian model.

[0027] Preferably, this adaptive densification mechanism first sets a mask to determine which pixel regions need to be densified, represented as:

[0028] ;

[0029] in, This represents an RGB image rendered using the 3D Gaussian method. This represents a depth image rendered using the 3D Gaussian method. This represents the rendered outline image; This is the RGB image after dynamic masking. A true depth image;

[0030] mask The instructions specify that new 3D Gaussian elements need to be added for densification in the following three cases:

[0031] (1) Areas where the rendered outline image is under-rendered;

[0032] (2) The area of ​​the RGB image with a pixel value of 0 after dynamic masking;

[0033] (3) The depth value of the real depth image is less than the depth value of the rendered depth image, and the error of the depth value is greater than 50 times the median depth error (MDE).

[0034] Preferably, for areas that are permanently obscured by dynamic object masks, i.e. By fusing features from multiple static scenes with local neighborhood information, dynamic object masks can be created. The coverage area restores static content in RGB and depth images in accordance with physical laws, and then densifies it using the restored color and depth information.

[0035] Preferably, in step S3, a new Gaussian element is added to each pixel in the area that needs to be densified. The color c of the Gaussian element is the RGB value of the pixel after repair, the center position μ is the coordinate of the three-dimensional space back-projected based on the depth of the repaired pixel, the opacity o is set to 0.5, and the radius r is the ratio of the true depth value to the camera focal length.

[0036] In another aspect, the present invention provides a non-transitory computer-readable storage medium having computer instructions stored thereon, which cause a computer to execute the above-described 3D Gaussian SLAM method based on dynamic suppression and incremental optimization.

[0037] In another aspect, the present invention provides an electronic device, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus, and the processor calls logical instructions in the memory to execute the aforementioned 3D Gaussian SLAM method based on dynamic suppression and incremental optimization.

[0038] In another aspect, the present invention provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer performs the aforementioned 3D Gaussian SLAM method based on dynamic suppression and incremental optimization.

[0039] Beneficial Effects: This invention employs a dynamic contamination suppression mechanism based on Gaussian initialization. Dynamic feature active isolation is implemented during the initialization phase of 3D Gaussian meta-model construction. By integrating a lightweight semantic segmentation network to construct dynamic object masks and establishing a depth-color cross-modal mask synchronization mechanism, the depth values ​​of dynamic regions are forcibly set to zero, thus suppressing dynamic contamination. The fusion of depth and semantic information ensures that the construction of the basic Gaussian model during the initialization phase is unaffected by dynamic contamination.

[0040] This invention employs a robust pose optimization mechanism based on rendering residual constraints. During the iterative optimization phase of pose estimation, a joint color-depth residual constraint is implemented to enhance the stability and accuracy of camera pose estimation. A depth-color dual-channel residual loss function is constructed, and corresponding weights are designed according to the scene to measure the importance of different information sources. A rendering quality threshold strategy is established, using only fully rendered and optimized pixel regions obtained from the contour image for pose gradient calculation, ensuring the robustness and accuracy of camera pose optimization.

[0041] This invention employs an incremental Gaussian element optimization mechanism for under-rendered areas. It addresses the rendering hole problem caused by dynamic masks during the iterative process of 3D Gaussian element density optimization. Static scene repair is performed on under-rendered areas using historical frames and local neighboring region information. The repaired color and depth information is then used to incrementally optimize the Gaussian elements in the scene. In areas requiring densification as indicated by the mask, a new Gaussian element is added to each pixel, with properties similar to those initialized with the Gaussian elements. Adding new Gaussian elements refines the scene representation of under-rendered areas.

[0042] In summary, the 3D Gaussian SLAM method based on dynamic suppression and incremental optimization of the present invention significantly improves the positioning robustness and the consistency and integrity of scene reconstruction in a dynamic environment. The proposed dynamic contamination suppression mechanism provides an accurate static scene basis for the incremental optimization of Gaussian elements, and the incremental optimization of Gaussian elements in turn improves the integrity and quality of map construction, thus jointly improving the positioning and mapping performance of the entire system. Therefore, the present invention is particularly suitable for complex working conditions such as dynamic goods handling and AGV cluster operations in intelligent warehousing, providing a high-precision environment model support for subsequent upper-layer applications such as path planning and multi-robot cooperation. Brief Description of the Drawings

[0043] Figure 1 It is a flow chart of the 3D Gaussian SLAM method based on dynamic suppression and incremental optimization.

[0044] Figure 2 It is a comparison chart of the sparse point cloud map and the 3D Gaussian map in scene representation; Figure 2 In (a), it is a comparison chart of the sparse point cloud map and the 3D Gaussian map on the fr1_desk sequence, Figure 2 In (b), it is a comparison chart of the sparse point cloud map and the 3D Gaussian map on the fr3_long_office_household sequence.

[0045] Figure 3 It is an effect diagram of the dynamic contamination suppression mechanism in the Gaussian initialization link; Figure 3 In (a), it is a 3D Gaussian map without the dynamic contamination suppression mechanism, Figure 3 In (b), it is a 3D Gaussian map after adding the dynamic contamination suppression mechanism.

[0046] Figure 4 It is a comparison chart of the rendered scene results under different loss weights; Figure 4 In (a), it is the rendered scene map when Figure 4 In (b), it is the rendered scene map when Figure 4 In (c), it is the rendered scene map when Figure 4 In (d), it is the rendered scene map when

[0047] Figure 5 It is an effect diagram of the Gaussian element incremental optimization mechanism for the under-rendered area; Figure 5 In (a), it is a 3D Gaussian map without the Gaussian element incremental optimization mechanism, Figure 5 In (b), it is a 3D Gaussian map after adding the Gaussian element incremental optimization mechanism.

[0048] Figure 6A comparison chart of the rendering results of the original method and the method of the present invention in a dynamic scene; Figure 6 (a) shows a rendering comparison of the original method (left) and the method of this invention (right) on the fr3_walking_xyz sequence. Figure 6 (b) shows a rendering comparison of the original method (left) and the method of this invention (right) on the fr3_walking_static sequence. Figure 6 (c) shows a rendering comparison of the original method (left) and the method of this invention (right) on the fr3_walking_halfsphere sequence. Figure 6 The middle (d) image shows a rendering comparison of the original method (left) and the method of the present invention (right) on the fr3_sitting_static sequence.

[0049] Figure 7 A comparison chart showing the effects of sparse point cloud maps and 3D Gaussian maps in a real-world intelligent warehousing environment; Figure 7 (a) shows the actual smart warehouse environment. Figure 7 (b) is the sparse point cloud map that was created. Figure 7 (c) is the 3D Gaussian map that was created.

[0050] Figure 8 A comparison chart of the rendering results of the original method and the method of the present invention in a real dynamic intelligent warehousing environment; Figure 8 In the middle (a), the rendering result obtained by the original method is shown. Figure 8 (b) shows the rendering result obtained using the method of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. In the description of this invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0052] The following is combined Figures 1-8 This invention describes the 3D Gaussian SLAM method based on dynamic suppression and incremental optimization, the non-transitory computer-readable storage medium, the electronic device, and the computer program product provided by this invention.

[0053] Example 1: This example provides a 3D Gaussian SLAM method based on dynamic suppression and incremental optimization, such as... Figure 1 As shown, it includes the following steps:

[0054] S1. The system input is an RGB-D image frame, and a dynamic contamination suppression mechanism is implemented during the initialization phase. To address the initialization deficiencies in dynamic interference scenarios, this embodiment proposes a cross-modal dynamic mask suppression mechanism, specifically including:

[0055] Deploy a real-time semantic segmentation network on the input RGB image Perform dynamic object instance segmentation and generate pixel-level dynamic region binary masks. Furthermore, by integrating depth information, a binary mask is extracted to further segment the movable object region affected by dynamic objects. The dynamic object mask of the final RGB image is obtained by adding them together. , represented as:

[0056] ;

[0057] RGB image after masking Represented by pixel-by-pixel and channel-by-channel masking operations:

[0058] ;

[0059] in, Indicates the height of the input image. Indicates the width of the input image. This represents the red, green, and blue color channels of the input image; Indicates pixel position It belongs to the dynamic region. Represents pixels It belongs to the static area; Represents pixels in a planar image. It is a pixel coordinates The x-axis is... The vertical axis is denoted by .

[0060] After setting a mask for the RGB image, the RGB image and the depth image are associated based on the timestamp. For RGB and depth images with a timestamp interval of less than 0.02 seconds, the depth value of the corresponding position in the depth image is synchronously set to zero for regions in the RGB image where the pixel value is zero, as shown below:

[0061] ;

[0062] in, For RGB images Depth images with timestamp intervals less than 0.02s and All images are real images obtained from sensors; Depth image after forcibly setting the depth value of the dynamic region to zero.

[0063] After completing the dynamic contamination suppression of the image frame, 3D Gaussian element initialization is performed based on the image frame after dynamic contamination suppression.

[0064] In recent years, with the emergence of 3D Gaussian technology, applying it to visual SLAM to handle dynamic problems in intelligent warehousing environments offers a novel solution. Traditional visual SLAM systems typically use sparse point clouds to represent the environment, such as ORB-SLAM2. These maps are sparse, wasting much information from image frames and are difficult for humans to understand. In contrast, the 3D Gaussian method uses anisotropic Gaussian elements to represent the environment, resulting in a map containing more information. Furthermore, 3D Gaussian technology allows for dense scene mapping, and adaptive density adjustment strategies for adding or removing 3D Gaussian elements make the scene easier for humans to understand. However, while direct map construction using 3D Gaussian technology works without scene anomalies in static scenarios, in dynamic scenes, direct 3D Gaussian mapping suffers from the problem of some 3D Gaussian elements landing on dynamic objects during initialization. The information of these Gaussian elements changes continuously with the movement of objects in the image frames, preventing the system from building a globally consistent map.

[0065] In this embodiment, 3D Gaussian techniques are used to construct a map of the scene, building a Gaussian meta-parameterized model containing 8-dimensional feature vectors, including color information representing RGB. The three parameters represent the center position information of the Gaussian element. The three parameters, and the radius of the Gaussian element. Opacity of Gaussian elements For a point in three-dimensional space Its Gaussian distribution function is expressed as:

[0066] ;

[0067] By employing differentiable rendering, the underlying Gaussian map is rendered into high-fidelity color, depth, and contour maps under arbitrary camera reference frames. The gradient of the error between the underlying scene Gaussian representation and camera parameters relative to the rendered result and the provided RGB-D frame is directly calculated, and the Gaussian and camera parameters are updated accordingly to minimize this error, achieving high-precision fitting of camera pose and scene volume representation.

[0068] The RGB image rendering process is as follows:

[0069] Given a set of 3D Gaussian units and a camera pose, sort all Gaussian units in order of near to far relative to the camera depth. Then, using alpha synthesis, sequentially blend the 2D projections of each Gaussian unit in pixel space to render an RGB image. Each pixel... The rendered color can be represented as:

[0070] ;

[0071] in Indicates the first An RGB color vector with Gaussian elements. Indicates the first A Gaussian unit in a pixel The contribution weight at each pixel is determined by the pixel's position. Let the pixel coordinates of a point in three-dimensional space be projected onto a two-dimensional pixel plane. and This represents the pixel points after projection onto a two-dimensional pixel plane. The location information and radius are expressed by the following formula:

[0072] , , ;

[0073] in It is a known camera intrinsic parameter matrix. The camera at the The extrinsic parameter information acquired during frame motion includes translation and rotation matrices. It is the known focal length of the camera. The depth of the Gaussian center in the camera coordinate system, i.e., the z-axis coordinate.

[0074] In addition to rendering the RGB image, to fully utilize the information input from the vision sensor, the system also renders the input depth image using a similar color rendering method. The rendering result can be directly compared with the input depth image and the gradient value is returned, as shown in the following formula:

[0075] ;

[0076] Simultaneously, a contour image is rendered to determine whether the information contained in a pixel is visible in the map of the current frame, expressed by the formula:

[0077] ;

[0078] For the first frame of the input image, the tracking step is skipped, and the camera pose is initialized. Since the rendered contour is empty, all pixels are used to initialize new Gaussian elements. For each pixel, a new Gaussian element is added with the following properties: color c is the RGB value of the pixel, center position μ is the coordinates of the pixel's depth backprojection into 3D space, opacity o is set to 0.5, and radius r is represented as:

[0079] ;

[0080] in For depth, The focal length of the camera is used to ensure that the radius is equivalent to 1 pixel when projected onto a 2D image through ratio calculation.

[0081] In this embodiment, a cross-modal dynamic mask suppression mechanism is used to achieve synchronous removal of dynamic objects in the color and depth domains, thus solving the problem of interference of dynamic objects on Gaussian initialization.

[0082] S2. After initialization, the camera pose of subsequent input images is iteratively optimized and estimated by constructing a joint depth-color residual constraint. Specifically:

[0083] Joint registration is performed using RGB and depth map information from the input image. The pose estimation results are optimized based on the residuals between the rendered RGB image, depth image, contour image, and the real depth map. A rendering quality threshold strategy is established, using only fully rendered and optimized pixel regions from the contour image for pose gradient calculation, minimizing the following loss to update the camera pose:

[0084] ;

[0085] in, This represents an RGB image rendered using the 3D Gaussian method. This represents a depth image rendered using the 3D Gaussian method. This represents the rendered outline image; and These are the weighting coefficients for depth map rendering error and color rendering error, respectively, used to measure the importance of different information sources in the optimization process; express Loss function. This means that rendering loss is calculated only in pixel areas where optimization is effective, in order to enhance the stability and accuracy of camera pose estimation. New input images may contain new perspectives, information not found in the map, or even content that was not previously optimized. Therefore, the above loss function design helps to improve the overall accuracy of pose estimation.

[0086] S3. After completing camera tracking and obtaining accurate camera pose estimation, a Gaussian incremental optimization mechanism is used for insufficiently rendered areas.

[0087] After completing camera tracking and obtaining accurate camera pose estimation, a reliable representation of the Gaussian pixel location distribution in the scene has been established by combining the geometric information provided by the depth image. However, for areas in the scene that are not yet fully covered by the existing Gaussian model, an adaptive densification mechanism needs to be activated to improve the reconstruction quality. This adaptive densification mechanism first sets a mask to determine which pixel regions need to be densified, represented as:

[0088] ;

[0089] mask The instructions specify that new 3D Gaussian elements need to be added for densification in the following three cases:

[0090] (1) Areas where the rendered outline image is under-rendered;

[0091] (2) The area of ​​the RGB image with a pixel value of 0 after dynamic masking;

[0092] (3) The depth value of the real depth image is less than the depth value of the rendered depth image, and the error of the depth value is greater than 50 times the median depth error (MDE).

[0093] However, the area that is permanently obscured by the dynamic object mask, i.e. The inability to trigger densification operations results in persistent geometric holes in dynamically remaining regions. To address these issues, features from multiple static scenes are fused with local neighboring region information within the dynamic object mask. The overlay area restores physically consistent static content from RGB and depth images, then densifies it using the restored color and depth information. (In the mask...) For the indicated area that needs densification, a new Gaussian element is added to each pixel, similar to the initialization phase. The color c of this Gaussian element is the RGB value of the pixel after repair, the center position μ is the coordinate of the three-dimensional space back-projected based on the depth of the repaired pixel, the opacity o is set to 0.5, and the radius r is the ratio of the true depth value to the camera focal length.

[0094] This embodiment achieves high-quality rendering of scene details in under-rendered areas by adding new Gaussian elements.

[0095] The method of this invention is compared with the method for building sparse point cloud maps, as well as the original method without dynamic pollution suppression, rendering residual constraints and Gaussian incremental optimization.

[0096] like Figure 2As shown, the differences between sparse point cloud maps and 3D Gaussian maps in scene representation are compared. Sparse point cloud maps contain less effective information, waste a lot of image frame information obtained from the sensor, and are difficult for people to understand. In contrast, 3D Gaussian maps can accurately and meticulously represent the scene, make full use of the image information input from the sensor, and are easy for people to understand.

[0097] like Figure 3 As shown, the effect of the dynamic pollution suppression mechanism in the Gaussian initialization process is demonstrated. In the initialization rendering scene without the dynamic pollution suppression mechanism, the afterimages of dynamic objects can be found. This information belonging to dynamic objects will seriously affect the rendering of the overall map and make it impossible to build a globally consistent map. After adding the dynamic pollution suppression mechanism, no information of dynamic objects will appear in the initialization scene, which effectively solves the problem of the impact of dynamic objects on map construction.

[0098] like Figure 4 As shown, the rendering results of the scene are compared under different loss weights. It can be seen that the appropriate weight coefficient has a great impact on the final result. Ensuring the accuracy of camera pose estimation is essential to further improve the scene map rendering effect.

[0099] like Figure 5 As shown in the figure, the effect of the Gaussian incremental optimization mechanism in under-rendered areas is demonstrated. It can be seen from the figure that without the introduction of the Gaussian incremental optimization mechanism, the final rendering result of the scene has rendering holes, and the occluded areas of dynamic objects cannot be effectively filled with details. After adding the Gaussian incremental optimization mechanism, the statically repaired depth-color image data is used to drive Gaussian densification, so as to achieve high-quality rendering of scene details in under-rendered areas.

[0100] like Figure 6 As shown in the figure, the 3D Gaussian rendering results of the scene before and after the improvement are displayed on the standard TUM dynamic dataset. It can be seen from the figure that through improvements in scene initialization, Gaussian compaction, and color and depth loss weights, the method proposed in this invention still has excellent rendering performance in dynamic scenes, which fully proves the effectiveness of the method of this invention.

[0101] like Figure 7 As shown in the figure, the difference between sparse point cloud maps and 3D Gaussian maps in scene representation in a real intelligent warehousing environment is evident. It can be seen from the figure that the scene represented by the sparse point cloud map contains too little information and is difficult for people to understand, while the scene rendered by the 3D Gaussian map makes full use of the information in the input image frame, and the rendered scene is clearer, more detailed and specific, and easier for people to understand.

[0102] like Figure 8As shown in the figure, the rendering effects of the original method and the improved method are compared in a real dynamic intelligent warehousing environment. It can be seen from the figure that the improved method proposed in this invention can effectively alleviate the impact of dynamic objects on the construction of a globally consistent map in a real dynamic scene, and the rendered scene is clearer.

[0103] The method of this invention was compared with the original method and similar works on the TUM dataset, as shown in Tables 1 and 2. By comparing the rendering quality metric Peak Signal-to-Noise Ratio (PSNR) and the localization performance metric Absolute Trajectory Error (ATE), the rendering quality of this invention is improved by approximately 30% compared to the original method, and the localization accuracy is improved by approximately 90%. It also shows improved rendering quality compared to similar works, fully demonstrating the superiority and necessity of the innovative method proposed in this invention. The scene maps constructed by the original method and traditional methods are severely affected by dynamic factors, resulting in low scene rendering quality. This invention utilizes a dynamic pollution suppression mechanism to effectively avoid the impact of dynamic factors on map rendering quality. Simultaneously, the added Gaussian incremental optimization mechanism further enhances the details of the rendered scene, achieving higher scene rendering quality.

[0104] Table 1. Comparison of localization and mapping performance between the original method and the method of this invention.

[0105]

[0106] Table 2. Comparison of rendering quality between the method of the present invention and similar tasks.

[0107]

[0108] This invention also compared the impact of different weight designs in rendering residual constraints on positioning performance and mapping quality. As can be seen from Table 3, a suitable weight design can bring a 92.9% improvement in positioning accuracy and a 39.9% improvement in mapping quality, respectively, which fully demonstrates the importance of rendering residual constraint design.

[0109] Table 3. Analysis of System Localization Performance and Mapping Quality under Different Loss Weights

[0110]

[0111] The dynamic contamination suppression mechanism proposed in this invention provides an accurate static scene foundation for Gaussian incremental optimization, which in turn improves the completeness and quality of map construction, thereby jointly enhancing the positioning and mapping performance of the entire system. In summary, this invention, through multi-dimensional improvements, provides an efficient and accurate 3D Gaussian SLAM solution for complex dynamic scenarios in intelligent warehousing.

[0112] Example 2: This example provides a non-transitory computer-readable storage medium storing computer instructions that cause a computer to execute a 3D Gaussian SLAM method based on dynamic suppression and incremental optimization. The method includes the following steps:

[0113] S1. Obtain RGB-D image frames, construct dynamic object masks using integrated lightweight semantic segmentation networks, and establish a depth-color cross-modal mask synchronization mechanism to synchronously remove dynamic objects in the acquired image frames in the color and depth domains, thus completing the dynamic contamination suppression of the image frames. Then, perform 3D Gaussian initialization based on the image frames after dynamic contamination suppression.

[0114] S2. After initialization, the camera pose of subsequent input images is iteratively optimized and estimated by constructing a joint depth-color residual constraint.

[0115] S3. After completing camera tracking and obtaining accurate camera pose estimation, static repair is performed on the color and depth values ​​of the under-rendered areas using historical frames and local neighboring region information. Based on the repaired color and depth information, a new Gaussian element is added to each pixel in the area that needs to be densified as indicated by the mask, thus refining the scene representation of the under-rendered areas.

[0116] Example 3: This example provides an electronic device that may include a processor, a communication interface, memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions from the memory to execute a 3D Gaussian SLAM method based on dynamic suppression and incremental optimization. This method includes the following steps:

[0117] S1. Obtain RGB-D image frames, construct dynamic object masks using integrated lightweight semantic segmentation networks, and establish a depth-color cross-modal mask synchronization mechanism to synchronously remove dynamic objects in the acquired image frames in the color and depth domains, thus completing the dynamic contamination suppression of the image frames. Then, perform 3D Gaussian initialization based on the image frames after dynamic contamination suppression.

[0118] S2. After initialization, the camera pose of subsequent input images is iteratively optimized and estimated by constructing a joint depth-color residual constraint.

[0119] S3. After completing camera tracking and obtaining accurate camera pose estimation, static repair is performed on the color and depth values ​​of the under-rendered areas using historical frames and local neighboring region information. Based on the repaired color and depth information, a new Gaussian element is added to each pixel in the area that needs to be densified as indicated by the mask, thus refining the scene representation of the under-rendered areas.

[0120] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] Example 4: This example provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a 3D Gaussian SLAM method based on dynamic suppression and incremental optimization. The method includes the following steps:

[0122] S1. Obtain RGB-D image frames, construct dynamic object masks using integrated lightweight semantic segmentation networks, and establish a depth-color cross-modal mask synchronization mechanism to synchronously remove dynamic objects in the acquired image frames in the color and depth domains, thus completing the dynamic contamination suppression of the image frames. Then, perform 3D Gaussian initialization based on the image frames after dynamic contamination suppression.

[0123] S2. After initialization, the camera pose of subsequent input images is iteratively optimized and estimated by constructing a joint depth-color residual constraint.

[0124] S3. After completing camera tracking and obtaining accurate camera pose estimation, static repair is performed on the color and depth values ​​of the under-rendered areas using historical frames and local neighboring region information. Based on the repaired color and depth information, a new Gaussian element is added to each pixel in the area that needs to be densified as indicated by the mask, thus refining the scene representation of the under-rendered areas.

[0125] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 3D Gaussian SLAM method based on dynamic suppression and incremental optimization, characterized in that, Includes the following steps: S1. Acquire RGB-D image frames, construct dynamic object masks using an integrated lightweight semantic segmentation network, and establish a depth-color cross-modal mask synchronization mechanism to synchronously remove dynamic objects in the acquired image frames in both the color and depth domains, thus completing dynamic contamination suppression for the image frame. Then, perform 3D Gaussian initialization based on the dynamically contamination-suppressed image frame. Specifically, the dynamic object mask is constructed using the integrated lightweight semantic segmentation network as follows: Deploy a real-time semantic segmentation network on the input RGB image Perform dynamic object instance segmentation and generate pixel-level dynamic region binary masks. Furthermore, by integrating depth information, a binary mask is extracted to further segment the movable object region affected by dynamic objects. The dynamic object mask of the final RGB image is obtained by adding them together. , is represented as: ； RGB image after dynamic masking Represented by pixel-by-pixel and channel-by-channel masking operations: ； in, Indicates the height of the input image. Indicates the width of the input image. This represents the red, green, and blue color channels of the input image; Indicates pixel position It belongs to the dynamic region. Represents pixels It belongs to the static area; Represents pixels in a planar image. It is a pixel coordinates The x-axis is... The vertical axis is used as the coordinate. A depth-color cross-modal mask synchronization mechanism is established to synchronously remove dynamic objects in the acquired image frames across both the color and depth domains. Specifically: After setting a mask for the RGB image, the RGB image and the depth image are associated based on the timestamp. For RGB and depth images with a timestamp interval of less than 0.02 seconds, the depth value of the corresponding position in the depth image is synchronously set to zero for regions in the RGB image where the pixel value is zero, as shown below: ； in, For use with RGB images Depth images with timestamp intervals less than 0.02s and All images are real images obtained from sensors; Depth image after forcibly setting the depth values ​​of dynamic regions to zero; S2. After initialization, the camera pose of subsequent input images is iteratively optimized and estimated by constructing a joint depth-color residual constraint. S3. After completing camera tracking and obtaining accurate camera pose estimation, static repair is performed on the color and depth values ​​of under-rendered areas using historical frames and local neighboring region information. Based on the repaired color and depth information, a new Gaussian element is added to each pixel in areas that need to be densified, thus refining the scene representation of under-rendered areas.

2. The 3D Gaussian SLAM method based on dynamic suppression and incremental optimization according to claim 1, characterized in that, In step S2, the camera pose is iteratively optimized and estimated for subsequent input images by constructing a joint depth-color residual constraint, specifically as follows: Joint registration is performed using RGB and depth map information from the input image. The pose estimation results are optimized based on the residuals between the rendered RGB image, depth image, contour image, and the ground truth image. A rendering quality threshold strategy is established, using only fully rendered and optimized pixel regions from the contour image for pose gradient calculation, minimizing the following loss to update the camera pose: ； in, This represents an RGB image rendered using the 3D Gaussian method. This represents a depth image rendered using the 3D Gaussian method. This represents the rendered outline image; and These are the weighting coefficients for depth map rendering error and color rendering error, respectively, used to measure the importance of different information sources in the optimization process; This means that rendering loss is calculated only in pixel regions where the contour image optimization is effective. express Loss function.

3. The 3D Gaussian SLAM method based on dynamic suppression and incremental optimization according to claim 1, characterized in that, In step S3, after completing camera tracking and obtaining accurate camera pose estimation, a reliable representation of the Gaussian element position distribution in the scene is established by combining the geometric information provided by the depth image. For areas in the scene that are not fully covered by the Gaussian model, an adaptive densification mechanism is initiated.

4. The 3D Gaussian SLAM method based on dynamic suppression and incremental optimization according to claim 3, characterized in that, The adaptive densification mechanism first sets a mask to determine which pixel regions need to be densified, represented as: ； in, This represents an RGB image rendered using the 3D Gaussian method. This represents a depth image rendered using the 3D Gaussian method. This represents the rendered outline image; This is the RGB image after dynamic masking. A true depth image; mask The instructions specify that new 3D Gaussian elements need to be added for densification in the following three cases: (1) Areas where the rendered outline image is under-rendered; (2) The area of ​​the RGB image with a pixel value of 0 after dynamic masking; (3) The depth value of the real depth image is less than the depth value of the rendered depth image, and the error of the depth value is greater than 50 times the median depth error (MDE).

5. The 3D Gaussian SLAM method based on dynamic suppression and incremental optimization according to claim 4, characterized in that, For areas that are permanently obscured by dynamic object masks, i.e. By fusing features from multiple static scenes with local neighborhood information, dynamic object masks can be created. The coverage area restores static content in RGB and depth images in accordance with physical laws, and then densifies it using the restored color and depth information.

6. The 3D Gaussian SLAM method based on dynamic suppression and incremental optimization according to claim 5, characterized in that, In step S3, a new Gaussian element is added to each pixel in the area that needs to be densified. The color c of the Gaussian element is the RGB value of the pixel after repair, the center position μ is the coordinate of the three-dimensional space back-projected based on the depth of the repaired pixel, the opacity o is set to 0.5, and the radius r is the ratio of the true depth value to the camera focal length.

7. A non-transitory computer-readable storage medium, characterized in that, It stores computer instructions that cause the computer to execute the 3D Gaussian SLAM method based on dynamic suppression and incremental optimization as described in any one of claims 1-5.

8. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer performs the 3D Gaussian SLAM method based on dynamic suppression and incremental optimization as described in any one of claims 1-5.

Images