Active space chunk selection and locally differentiable rendering method for large-scale gaussian spray slam

Abstract

The application discloses a kind of active space block selection and local differentiable rendering method for large-scale scene Gaussian splash SLAM, first, global Gaussian map is constructed and is divided into several space blocks, and each space block is assigned block identification, Gaussian primitive is associated to space block and is updated, and the final active space block set is screened from all the space blocks of global Gaussian map;Then extract all the Gaussian primitives belonging to any space block in the final active space block set and form local Gaussian primitive working set;Only local Gaussian primitive working set is input into differentiable rasterization renderer to execute local differentiable rendering, and local rendering result is obtained, the camera pose of current frame is tracked using local rendering result, and map optimization is carried out according to current frame and key frame;Finally, new Gaussian primitive is generated using current frame depth information and is added to global Gaussian map, and the ownership relationship between each Gaussian primitive and space block is updated.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, simultaneous localization and mapping, 3D scene reconstruction and graphics rendering, and in particular to an active spatial block selection and locally differentiable rendering method for Gaussian splash SLAM for large-scale scenes. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is one of the core technologies in computer vision and robotics. In recent years, differentiable rendering SLAM methods based on 3D Gaussian Splatting have attracted widespread attention due to their high-fidelity real-time rendering capabilities. This type of method maintains a global map composed of a large number of 3D Gaussian primitives and utilizes a differentiable rasterizer renderer to simultaneously track camera pose and optimize scene structure.

[0003] During online operation, as visual sequences are continuously input, the number of Gaussian elements in the map increases to cover new observation areas. To ensure mapping accuracy and tracking robustness, existing techniques typically require rendering, loss calculation, and gradient backpropagation of all or most Gaussian elements in the global map in each frame optimization iteration. This global optimization strategy has the following inherent drawbacks:

[0004] First, the consumption of computing and storage resources increases linearly with the map size. The memory usage is positively correlated with the number of Gaussian elements, and long-running sequences are prone to causing GPU memory overflow. At the same time, the time consumed per frame for tracking and mapping also increases continuously with the increase of the number of global Gaussian elements, which seriously restricts the real-time performance and long-term running capability of the system on resource-constrained platforms.

[0005] Second, simple spatial clipping strategies suffer from boundary omissions and inconsistent selection. Some existing works have attempted to use view frustum clipping or fixed spatial mesh clipping to narrow the optimization range. However, these methods rely on only a single geometric condition. When Gaussian elements happen to be distributed near block boundaries, or when projection points fluctuate due to depth observation noise, inconsistent activation / culling results are easily generated between adjacent frames, leading to cracks in the rendered image or tracking jitter.

[0006] Third, the dynamic topological changes of Gaussian maps render traditional index caching strategies ineffective. During optimization, Gaussian primitives frequently undergo operations such as adding and expanding uncovered areas, deleting redundant or low-opacity primitives, and splitting excessively large primitives to improve detail. This means that the global array index of Gaussian primitives is highly unstable. If the set of Gaussian indices activated in the previous frame or iteration is directly cached, problems such as index out-of-bounds errors and incorrect gradients being written back to irrelevant primitives are highly likely to occur, leading to system crashes or degradation of reconstruction quality.

[0007] In view of the above problems, how to design a low-overhead, boundary-continuous and topology-stable local optimization mechanism for Gaussian splash SLAM for large-scale scenarios has become a key technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and propose an active spatial block selection and local differentiable rendering method for Gaussian splash SLAM in large-scale scenes. Based on the current frame depth observation, camera view frustum, camera location region and neighborhood protection rules, spatial blocks related to the current frame are selected from the continuously growing global Gaussian map, and local differentiable rendering is performed only on the Gaussian within the selected blocks.

[0009] To achieve the above objectives, the technical solution specifically adopted by the present invention is as follows:

[0010] An active spatial block selection and locally differentiable rendering method for Gaussian splash SLAM in large-scale scenes includes the following steps:

[0011] Step 1: Obtain the initial frame image, initial camera pose and depth information, and construct a global Gaussian map. Divide the global Gaussian map into several spatial blocks, assign block identifiers to each spatial block, associate Gaussian elements with the spatial blocks and update them.

[0012] Step 2: Based on the camera pose and depth information of the current frame, select the final set of active spatial blocks from all spatial blocks of the global Gaussian map;

[0013] Step 3: Based on the final active spatial block set and the affiliation relationship between each Gaussian element and the spatial block at the current time, extract all Gaussian elements belonging to any spatial block in the final active spatial block set from the global Gaussian map to form a local Gaussian element working set.

[0014] Step 4: Input only the local Gaussian meta set into the differentiable rasterizer to perform local differentiable rendering, obtain the local rendering result, use the local rendering result to track the camera pose of the current frame, and optimize the map based on the current frame and key frames.

[0015] Step 5: Determine the pixel region of the new Gaussian primitive based on the difference between the local rendering result and the observation of the current frame, generate the new Gaussian primitive using the depth information of the current frame and append it to the global Gaussian map, and update the affiliation relationship between each Gaussian primitive and the spatial block.

[0016] Preferably, the filtering method is as follows: the spatial blocks hit by depth observation are included in the first set, the spatial blocks where the camera's view frustum intersects are included in the second set, the spatial block where the camera is located in the current frame is determined as the camera's home block, the spatial blocks in the first set and the camera's home block are used as seeds, and a set of neighboring protected blocks is obtained by expanding according to a preset protection radius, and the first set, the second set, the camera's home block and the set of neighboring protected blocks are merged to obtain the final active spatial block set.

[0017] Preferably, in step 1, the scene space covered by the global Gaussian map is divided into several spatial blocks according to a preset two-dimensional world coordinate plane. Based on the projection position of the center position of each Gaussian element on the two-dimensional world coordinate plane, each Gaussian element is associated with the corresponding spatial block, and the association relationship between the Gaussian element and the spatial block is recorded.

[0018] Preferably, in step 1, the update method is as follows: based on the observation of the current frame, Gaussian elements of the global Gaussian map are added, pruned, and optimized. If the number of Gaussian elements of a spatial block exceeds a preset threshold, the spatial block is subdivided into quadtrees. The relationship between Gaussian elements and spatial blocks is updated as Gaussian elements are added, pruned, and spatial blocks are subdivided.

[0019] Preferably, the spatial block hit by the depth observation is determined in the following manner:

[0020] Back-project the effective depth pixels in the current frame depth map to transform the pixel coordinates and observed depth values ​​to the world coordinate system to obtain three-dimensional world coordinate points;

[0021] Determine the inclusion relationship between the planar coordinate components of the three-dimensional world coordinate points corresponding to all valid pixels and the planar regions covered by each spatial block;

[0022] A spatial block containing at least one of the three-dimensional world coordinate points is identified as a spatial block hit by depth observation.

[0023] Preferably, the spatial blocks where the camera view frustums intersect are determined in the following manner:

[0024] Construct the projection region of the current frame camera's view frustum on the spatial division plane based on the current frame camera pose, camera intrinsic parameters, and preset maximum observation depth;

[0025] Spatial blocks where the covered planar area intersects with the projected area are identified as spatial blocks where the view frustums intersect.

[0026] Preferably, the camera-assigned block is the spatial block in which the current frame camera's planar position in the world coordinate system is located; if the current frame camera's planar position does not fall into any divided spatial block, then the spatial block closest to that planar position is selected as the camera-assigned block.

[0027] Preferably, the set of neighborhood protection blocks is determined in the following manner:

[0028] Use all spatial blocks in the first set and the camera-assigned block as seed blocks;

[0029] The spatial blocks whose shortest distance to any seed block is less than or equal to the preset protection radius are included in the neighborhood protection block set;

[0030] The preset protection radius is a first value in the map optimization stage and a second value in the camera pose tracking stage, and the second value is greater than the first value.

[0031] Preferably, in step 3, the method for extracting Gaussian elements is as follows:

[0032] Traverse each Gaussian cell in the global Gaussian map and read the spatial block identifier currently associated with that Gaussian cell;

[0033] If the spatial block identifier belongs to the final active spatial block set, then add the Gaussian index to the local Gaussian working set;

[0034] If it does not belong to the group, skip that Gaussian element.

[0035] Preferably, in step 4, the local rendering results include: a local rendering color image, a local rendering depth map, a local rendering contour map, and an existence map; the local rendering only calculates the contribution value of Gaussian elements in the local Gaussian element working set to the image pixels, and the Gaussian elements in the global Gaussian map that are not selected into the local Gaussian element working set do not participate in the rendering and gradient calculation of the current frame.

[0036] Preferably, in step 5, determining the pixel region to be added with new Gaussian pixels specifically involves:

[0037] Pixels whose pixel values ​​in the local rendering contour map are lower than a preset contour existence threshold are determined to be pixels not covered by the contour.

[0038] Pixels whose depth values ​​in the local rendering depth map are greater than the current frame's observed depth value and whose absolute difference is greater than a preset depth anomaly determination coefficient are identified as depth anomaly pixels.

[0039] The union of the pixels whose contours are not covered and the pixels with abnormal depth is taken as the pixel region to be added to the Gaussian unit.

[0040] As a preferred approach, during multiple map optimization iterations on the same keyframe, the block identifiers of the final active spatial block set are pre-cached. In each iteration, the local Gaussian working set corresponding to each iteration is re-extracted based on the cached block identifiers and the latest Gaussian elements and spatial block affiliation at the current moment, rather than directly caching or reusing the Gaussian index set of the previous iteration.

[0041] Preferably, the quadtree subdivision method is as follows: initially, the space is divided into several coarse-grained space blocks. When the number of Gaussian elements in any space block exceeds a preset load threshold, the space block is subdivided into quadtrees until the number of Gaussian elements in the space block does not exceed the preset load threshold.

[0042] This invention also provides a Gaussian splash SLAM system for large-scale scenarios, comprising:

[0043] The image acquisition module is used to acquire the initial frame image and depth information;

[0044] The global map building module is used to build a global Gaussian map and associate each Gaussian element with a spatial block in a preset spatial partitioning structure according to its center position. Each spatial block has a unique and stable block identifier.

[0045] The active spatial block selection module is used to select the final set of active spatial blocks based on the camera pose and depth information of the current frame, using the active spatial block selection and local differentiable rendering method for Gaussian splash SLAM for large-scale scenes.

[0046] The local working set extraction module is used to extract the local Gaussian working set from the global Gaussian map based on the final active spatial block set and the affiliation relationship between each Gaussian element and the spatial block.

[0047] Differentiable rendering module, used to perform differentiable rasterization rendering only on the local Gaussian working set, and output local rendering results;

[0048] The pose tracking and map optimization module is used to perform camera pose tracking and map parameter optimization based on the local rendering results.

[0049] The map extension module is used to determine the region of pixels to be added based on the difference between the local rendering results and the current frame observation, generate new Gaussian elements and update the attribution relationships;

[0050] The storage and caching module is used to cache the block identifiers of the final active spatial block set so that they can be re-parsed by the local working set extraction module in each map optimization iteration.

[0051] This invention has the following characteristics and beneficial effects:

[0052] This invention abandons the traditional approach of rendering and optimizing all global Gaussian elements. Through multiple constraints such as depth hit blocks, view frustum intersection blocks, camera affiliation blocks, and neighborhood protection blocks, each frame only selects the local spatial blocks most relevant to the current observation, and controls the number of Gaussian elements participating in rendering and gradient calculation to a level far lower than the global total.

[0053] While maintaining a similar final Gaussian scalar number of units as the original method in long-sequence online tracking and mapping processes, the peak memory usage is reduced by approximately 27.9%, and the total tracking and mapping time is reduced by approximately 35.7%. This demonstrates that the present invention effectively overcomes the bottleneck of linearly increasing resource consumption as map size grows, enabling Gaussian splash SLAM to maintain real-time responsiveness even on resource-constrained embedded platforms or in long-term continuous operation scenarios, thus possessing practical value for engineering deployment.

[0054] Unlike coarse cropping methods that rely solely on view frustums or fixed meshes, this invention introduces a neighborhood protection block set. Using depth hit blocks and camera-assigned blocks as seeds, it expands outwards to a preset protection radius, completely incorporating spatial blocks near the boundary into the active set.

[0055] This mechanism ensures that Gaussian primitives located on both sides of the block boundary are consistently activated, avoiding the "selection jump" phenomenon caused by depth noise or small pose fluctuations, and fundamentally solving the rendering discontinuity problem caused by local clipping. Simultaneously, the view frustum intersection block serves as a supplement, ensuring that existing Gaussian primitives in the current image, although not having an effective depth hit, are geometrically visible, further guaranteeing rendering integrity. Experimental data also show that this invention significantly reduces peak memory usage and processing time, while achieving slightly better reconstructed image quality (PSNR) and trajectory accuracy (ATE RMSE) than the global optimization baseline, verifying the effectiveness of the local selection strategy in ensuring accuracy.

[0056] To address the instability of the global index caused by frequent additions, pruning, and subdivision of Gaussian metagrams during optimization, this invention creatively proposes a cache space block ID instead of directly caching the Gaussian metagram index.

[0057] Since spatial block identifiers are determined solely by the spatial affiliation of Gaussian elements, and Gaussian map expansion operations do not alter the geometric boundaries of stable spatial blocks, the cached block identifier set remains valid throughout multiple mapping iterations. In each optimization iteration, the system only needs to dynamically re-parse the corresponding active Gaussian element index from the cached block identifiers based on the latest Gaussian-spatial block mapping relationship, thus completely avoiding gradient write-back errors or program crashes caused by index out-of-bounds errors. This mechanism enables the invention to converge stably even when facing drastic changes in map size, providing a reliable technical guarantee for long-term, large-scale online 3D reconstruction.

[0058] This invention further utilizes local rendering results to determine the addition of new Gaussian units, generating new Gaussians only in regions not covered by rendering or where the depth lags significantly behind the observations. Since this determination process relies solely on the local active set rather than the global map, it reduces the computational overhead of addition determination and avoids repeated generation in already sufficiently mapped regions, thus controlling the reasonable growth of the global Gaussian count. Final experiments show that, under the same sequence, the final number of Gaussians in this invention is slightly less than the baseline, verifying the efficiency of this local addition strategy.

[0059] In summary, this invention, while ensuring or even slightly surpassing the quality and trajectory accuracy of global optimization reconstruction, achieves comprehensive improvements in memory usage, computation time, system robustness, and map expansion efficiency, marking a crucial step towards the large-scale and long-term application of Gaussian splash SLAM technology in real-world scenarios. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating the overall process framework of an embodiment of the present invention.

[0061] Figure 2 This is a schematic diagram of the location of frame 700 in the global map in an embodiment of the present invention.

[0062] Figure 3 This is a magnified view of a portion of the active space block in an embodiment of the present invention. Detailed Implementation

[0063] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0064] Example 1

[0065] This embodiment provides an active spatial block selection and locally differentiable rendering method for Gaussian splash SLAM in large-scale scenes, such as Figure 1 As shown, it includes the following steps:

[0066] Step 1: Obtain the initial frame image, initial camera pose value, and depth information, and construct a global Gaussian map. Then, divide the scene space covered by the global Gaussian map according to the preset two-dimensional world coordinate plane to obtain several spatial blocks, and assign block identifiers to each spatial block. According to the projection position of the center position of each Gaussian element on the two-dimensional world coordinate plane, associate each Gaussian element with the corresponding spatial block, and record the association relationship between the Gaussian element and the spatial block.

[0067] In subsequent frame processing, the current frame image, the initial camera pose value, and the current frame depth information are acquired. Based on the current frame observations, Gaussian primitives of the global Gaussian map are added, cropped, and optimized. If the number of Gaussian primitives associated with any spatial block exceeds a preset load threshold, the spatial block is subdivided into quadtrees. The original spatial block after subdivision is no longer used as a spatial block, and the Gaussian primitives originally associated with that spatial block are reassociated to the corresponding sub-spatial blocks according to their respective 2D projection positions. The association relationship between Gaussian primitives and spatial blocks is maintained simultaneously during the addition, cropping, and spatial block subdivision processes.

[0068] Specifically, let the global Gaussian map at time t be:

[0069]

[0070] in: Indicates time Global Gaussian map; This represents the i-th 3D Gaussian primitive, which includes the center position, scale, rotation, color, opacity, and optimization parameters; is the number of global Gaussians at time t; i represents the global index of the Gaussian.

[0071] The ground plane or a predefined two-dimensional world coordinate plane is divided into several spatial blocks. Each Gaussian primitive is associated with a stable block identifier based on the spatial region to which its center position belongs.

[0072]

[0073] in: This represents the spatial block identifier of the i-th Gaussian at time t; For a moment A collection of spatial blocks.

[0074] Further, the quadtree is subdivided: initially, the space is divided into several coarse-grained space blocks. When the number of Gaussian elements in any space block exceeds a preset load threshold, the space block is subdivided into a quadtree until the number of Gaussian elements in the space block does not exceed the preset load threshold.

[0075] Specifically, in this embodiment, a quadtree hierarchical spatial structure is used to organize the blocks. Initially, coarse spatial blocks are planned. When the number of Gaussians in a spatial block exceeds a preset load threshold (250,000 in this embodiment), the system automatically subdivides the spatial block using a quadtree. Subdivision stops when the number of Gaussians in the block does not exceed the threshold. Gaussians are added, pruned, or subdivided, and the system is updated synchronously. This ensures that it remains consistent with the global Gaussian distribution. Figure 1 Therefore, the spatial scale of the block is not fixed, but determined by the local Gaussian load. During subsequent activations, dense regions can be more accurately pruned of irrelevant Gaussians, while sparse regions will not be over-segmented.

[0076] Step 2: Based on the camera pose and depth information of the current frame, select the final set of active spatial blocks from all spatial blocks of the global Gaussian map.

[0077] In this embodiment, the filtering method is as follows: spatial blocks hit by depth observation are included in the first set, spatial blocks where the camera's view frustum intersects are included in the second set, the spatial block where the camera in the current frame is located is determined as the camera's home block, and a neighboring protection block set is obtained by using the spatial blocks in the first set and the camera's home block as seeds and expanding according to a preset protection radius. The first set, the second set, the camera's home block, and the neighboring protection block set are merged to obtain the final active spatial block set.

[0078] Specifically, this embodiment does not rely solely on a single spatial condition, but instead constructs the following four types of spatial block sets:

[0079] First set: Valid pixels in the current frame depth map or depth point set. The observed depth is back-projected and transformed to the world coordinate system:

[0080]

[0081]

[0082] in, Let (u,v) be the three-dimensional coordinates of the pixel in the camera coordinate system; This pixel is a 3D point in the world coordinate system. K represents the effective observation depth of a pixel; K is the camera intrinsic parameter matrix. This is the homogeneous transformation matrix from the camera coordinate system to the world coordinate system; the superscripts c and w represent the camera coordinate system and the world coordinate system, respectively. Depth can be obtained by IGEV-Stereo stereo matching.

[0083] This embodiment uses only pixels with finite depth, greater than the minimum effective depth but less than the maximum observation depth, to suppress unreliable depth. Let the set of effective world coordinate points obtained from equation (4) be... Let R(b) be the spatial region of spatial block b. Therefore, the depth-hit block set is:

[0084]

[0085] in: Represents the set of depth-hit blocks; For any space block; For any valid world coordinate point; Representing a three-dimensional point Planar coordinate components; Represents a space block The covered planar area;

[0086] This set ensures that the scene area actually covered by the current frame depth observation is included in the working set.

[0087] Second set: Construct the current frame's frustum region based on the current camera pose, camera intrinsic parameters, and maximum viewing depth. The spatial blocks intersecting with the visual cone region constitute:

[0088]

[0089] in: Represents the set of intersecting view cones; : The projection area of ​​the camera's view cone on the spatial block dividing plane at time t;

[0090] This set ensures that existing Gaussians that may be visible in the current image are not missed simply because there is no depth hit.

[0091] Camera Attribution Block: Let the current camera's world coordinate plane position be... The camera's ownership block can be defined as the block containing that location, or the nearest occupied block to that location:

[0092]

[0093] in: The camera block at time t; This indicates the current planar position of the camera in the world coordinate system; The shortest distance from a point to a spatial region; This represents a spatial block that minimizes the distance.

[0094] This block is used to ensure that the area where the camera is located is not excluded due to lack of depth, low texture, or frustum edge conditions.

[0095] Neighborhood protection blocks: Using depth-hit blocks and camera-owned blocks as seeds, select blocks whose distance is not greater than the protection radius. Neighboring blocks:

[0096]

[0097] in: For a set of neighboring protected blocks; : Any block in the depth hit block or the camera-owned block; The shortest distance between two spatial block regions; : Preset neighborhood protection radius (this value is 2 meters when mapping, and 6 meters when tracking);

[0098] Neighborhood protection blocks are used to mitigate rendering truncation caused by spatial block boundaries, as well as selection discontinuity when depth points fall near block boundaries.

[0099] Final active space block set: Merging the above space blocks yields:

[0100]

[0101] in: The final set of active space blocks at time t;

[0102] : Depth-hit block set;

[0103] : Set of intersecting cone blocks;

[0104] : A single-element collection containing only the block to which the camera belongs;

[0105] : Neighborhood protected block set.

[0106] This formula is the core of the invention: the current frame depth observation, geometric visibility, camera local continuity, and block boundary protection jointly determine the active spatial block set of the current frame.

[0107] Step 3: Based on the final active spatial block set and the affiliation relationship between each Gaussian element and the spatial block at the current time, extract all Gaussian elements belonging to any spatial block in the final active spatial block set from the global Gaussian map to form a local Gaussian element working set.

[0108] The method for extracting Gaussian elements is as follows:

[0109] Traverse each Gaussian cell in the global Gaussian map and read the spatial block identifier currently associated with that Gaussian cell;

[0110] If the spatial block identifier belongs to the final active spatial block set, then add the index of the Gaussian to the local Gaussian working set;

[0111] If it does not belong to the group, skip that Gaussian element.

[0112] Specifically, based on the set of active spatial blocks, local Gaussian indices are extracted from the global Gaussian map:

[0113]

[0114] in: For a moment The set of active Gaussian indexes; i is the global Gaussian index; Indicates the global Gaussian count; Gaussian Corresponding space block identifier; This is the final set of active space blocks.

[0115] Step 4: Input only the local Gaussian meta-workset into the differentiable rasterizer to perform local differentiable rendering, obtain the local rendering result, use the local rendering result to track the camera pose of the current frame, and perform map optimization based on the current frame and keyframes.

[0116] The local rendering results include: local rendering color image, local rendering depth map, local rendering contour map, and existence map; the local rendering only calculates the contribution value of Gaussian elements in the local Gaussian element working set to the image pixels, and the Gaussian elements in the global Gaussian map that are not selected into the local Gaussian element working set do not participate in the rendering and gradient calculation of the current frame.

[0117] Specifically, only The Gaussian input is a differentiable rasterizer to obtain local rendering results:

[0118]

[0119] in: For Gaussian differentiable rendering functions; This represents a local Gaussian subset extracted from the global map based on the active index. The color image obtained from local rendering; This is a depth map obtained from local rendering. The outline or existence map obtained from local rendering; Let be the transformation matrix from the world coordinate system to the camera coordinate system at time t; This is the camera intrinsic parameter matrix; This indicates that the corresponding Gaussian is extracted based on the active index set.

[0120] Local rendering results are used for camera pose tracking and map optimization; global map parameters are not truncated or deleted due to local selection.

[0121] In this embodiment, the keyframes used for map optimization are obtained through the original method LSG-SLAM, so they will not be described in detail here.

[0122] Furthermore, the map optimization method is as follows: for multiple mapping iterations of the same target keyframe, direct caching is not performed. Gaussian set, and cache active space block set In the j-th mapping iteration, the index is re-obtained based on the latest Gaussian-spatial block correspondence:

[0123]

[0124] in: Let be the set of active Gaussian indices at time t and the j-th mapping iteration; j is the iteration number of the same mapping target frame; The total number of Gaussians at the j-th iteration; This is the spatial block identifier for Gaussiani at the j-th iteration; A pre-cached set of active space blocks.

[0125] Due to the addition, trimming, and subdivision of Gaussian, changes may occur. The Gaussian row index, based on this formula, avoids out-of-bounds errors, incorrect gradient write-backs, or repeated active block selection caused by caching old indexes, thereby accelerating the system's running speed.

[0126] Step 5: Determine the pixel region of the new Gaussian primitive based on the difference between the local rendering result and the observation of the current frame, generate the new Gaussian primitive using the depth information of the current frame and append it to the global Gaussian map, and update the affiliation relationship between each Gaussian primitive and the spatial block.

[0127] Specifically, the pixel region to be added with new Gaussian elements is as follows:

[0128] Pixels whose pixel values ​​in the local rendering contour map are lower than a preset contour existence threshold are determined to be pixels not covered by the contour.

[0129] Pixels whose depth values ​​in the local rendering depth map are greater than the current frame's observed depth value and whose absolute difference is greater than a preset depth anomaly determination coefficient are identified as depth anomaly pixels.

[0130] The union of the pixels whose contours are not covered and the pixels with abnormal depth is taken as the pixel region to be added to the Gaussian unit.

[0131] Specifically, by utilizing local rendering contours and depth, the pixel areas in the map that need to be expanded in the current frame are determined:

[0132]

[0133] in: This represents the Gaussian pixel mask to be added at time t. The pixel position in the image; To render the outline locally in pixels The value at; The threshold for the existence of the contour; Indicates the local rendering depth in pixels The value at; For the current frame in pixels The observation depth at that location; Indicates the depth anomaly determination coefficient; This is the set of depth errors for the currently valid depth pixels; This represents the median of the depth error set;

[0134] This formula indicates that if a local rendering does not cover a pixel, or if the local rendering depth lags significantly behind the observation depth, the region corresponding to that pixel may lack a foreground Gaussian. A new Gaussian is generated based on this mask and the observation depth through backprojection, and the new Gaussian is appended to the global map while updating its spatial block identifier.

[0135] This embodiment also includes synchronously updating the spatial block ownership relationships after Gaussian additions, trimmings, and subdivisions.

[0136] Finally, repeat the above steps to complete continuous visual SLAM.

[0137] To demonstrate the effectiveness of this invention, experiments were conducted comparing it with the existing baseline work, LSG-SLAM, in scenes of the same length. For example... Figure 2 and Figure 3 The image shows the set of active spatial blocks selected in frame 700 of the KITTI 02 scene in this embodiment. The orange curve represents the entire camera trajectory; the black dot indicates the camera position in the current target frame; the gold border indicates the block containing the camera; the red blocks are blocks directly hit by depth consistency. Blue blocks represent blocks selected only by view frustum intersection; green blocks are neighborhood protection blocks; and blocks without color are inactive blocks that do not participate in the tracking and mapping work of this frame.

[0138] The original LSG-SLAM full-map Gaussian rendering baseline was compared with the active space block local rendering scheme proposed in this paper. A comparative experiment was conducted on the online tracking and mapping process in the KITTI 02 scene, covering the first 750 frames and a cumulative trajectory length of approximately 880 meters. Both sets of experiments used the same data range, initialization method, and optimization parameters. It can be seen that when the final Gaussian number is in the tens of millions and the number is basically the same, this invention can effectively reduce peak video memory usage and accelerate the system reconstruction time. At the same time, the reconstruction quality and trajectory accuracy are slightly better than the baseline work. The results are shown in Table 1.

[0139] Table 1 shows a comparative experiment of the first 750 frames (cumulative length approximately 880 meters) in the KITTI 02 scene.

[0140]

[0141] As shown in Table 1, under the same experimental conditions of approximately 880 meters cumulative ground truth trajectory length in the first 750 frames of the KITTI 02 scene, which only included online tracking and mapping, the peak memory usage of the method of this invention decreased from 18.33 GiB to 13.22 GiB, a reduction of 27.86%; the total time for online tracking and mapping decreased from 4402.6 s to 2830.7 s, a reduction of 35.7%.

[0142] Meanwhile, the method of this invention ultimately generates 10.375M Gaussian elements, which is only 0.53% different from the original baseline's 10.431M, indicating that the gains in storage and time are not obtained by significantly reducing the map size. PSNR, SSIM, LPIPS, and ATERMSE are all at comparable levels to the original baseline. This demonstrates that this invention can reduce the resource consumption of large-scale Gaussian map online rendering and optimization processes and improve online tracking and mapping efficiency without sacrificing reconstruction quality and trajectory accuracy.

[0143] Example 2

[0144] This embodiment provides a Gaussian splash SLAM system for large-scale scenarios, including:

[0145] The image acquisition module is used to acquire the initial frame image and depth information;

[0146] The global map building module is used to build a global Gaussian map and associate each Gaussian element with a spatial block in a preset spatial partitioning structure according to its center position. Each spatial block has a unique and stable block identifier.

[0147] The active spatial block selection module is used to select the final set of active spatial blocks based on the camera pose and depth information of the current frame, using the active spatial block selection and local differentiable rendering method for Gaussian splash SLAM for large-scale scenes.

[0148] The local working set extraction module is used to extract the local Gaussian working set from the global Gaussian map based on the final active spatial block set and the affiliation relationship between each Gaussian element and the spatial block.

[0149] Differentiable rendering module, used to perform differentiable rasterization rendering only on the local Gaussian working set, and output local rendering results;

[0150] The pose tracking and map optimization module is used to perform camera pose tracking and map parameter optimization based on the local rendering results.

[0151] The map extension module is used to determine the region of pixels to be added based on the difference between the local rendering results and the current frame observation, generate new Gaussian elements and update the attribution relationships;

[0152] The storage and caching module is used to cache the block identifiers of the final active spatial block set so that they can be re-parsed by the local working set extraction module in each map optimization iteration.

[0153] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for active spatial block selection and locally differentiable rendering in Gaussian splash SLAM for large-scale scenes, characterized in that, Includes the following steps: Step 1: Obtain the initial frame image, initial camera pose value and depth information, and then construct a global Gaussian map. Divide the global Gaussian map into several spatial blocks, assign block identifiers to each spatial block, associate Gaussian elements with spatial blocks and update them. Step 2: Based on the camera pose and depth information of the current frame, select the final set of active spatial blocks from all spatial blocks of the global Gaussian map; Step 3: Based on the final active spatial block set and the affiliation relationship between each Gaussian element and the spatial block at the current time, extract all Gaussian elements belonging to any spatial block in the final active spatial block set from the global Gaussian map to form a local Gaussian element working set. Step 4: Input only the local Gaussian meta set into the differentiable rasterizer to perform local differentiable rendering, obtain the local rendering result, use the local rendering result to track the camera pose of the current frame, and optimize the map based on the current frame and key frames. Step 5: Determine the pixel region of the new Gaussian primitive based on the difference between the local rendering result and the observation of the current frame, generate the new Gaussian primitive using the depth information of the current frame and append it to the global Gaussian map, and update the affiliation relationship between each Gaussian primitive and the spatial block.

2. The method according to claim 1, characterized in that, In step 1, the scene space covered by the global Gaussian map is divided according to a preset two-dimensional world coordinate plane to obtain several spatial blocks. Based on the projection position of the center position of each Gaussian element on the two-dimensional world coordinate plane, each Gaussian element is associated with the corresponding spatial block, and the association relationship between the Gaussian element and the spatial block is recorded.

3. The method according to claim 1, characterized in that, In step 1, the update method is as follows: based on the observation of the current frame, Gaussian elements of the global Gaussian map are added, pruned and optimized. If the number of Gaussian elements of a spatial block exceeds a preset load threshold, the spatial block is subdivided into quadtrees. The relationship between Gaussian elements and spatial blocks is updated with the addition, pruning and subdivision of Gaussian elements and spatial blocks.

4. The method according to claim 1, characterized in that, The filtering method is as follows: spatial blocks hit by depth observation are included in the first set, spatial blocks where the camera's view frustum intersects are included in the second set, the spatial block where the camera is located in the current frame is determined as the camera's home block, the spatial blocks in the first set and the camera's home block are used as seeds, and a neighboring protection block set is obtained by expanding according to a preset protection radius, and the first set, the second set, the camera's home block and the neighboring protection block set are merged to obtain the final active spatial block set; The spatial blocks hit by the depth observation are determined in the following way: Back-project the effective depth pixels in the current frame depth map to transform the pixel coordinates and observed depth values ​​to the world coordinate system to obtain three-dimensional world coordinate points; Determine the inclusion relationship between the planar coordinate components of the three-dimensional world coordinate points corresponding to all valid pixels and the planar regions covered by each spatial block; A spatial block containing at least one of the three-dimensional world coordinate points is identified as a spatial block hit by depth observation; The spatial blocks where the camera view frustums intersect are determined in the following way: Construct the projection region of the current frame camera's view frustum on the spatial division plane based on the current frame camera pose, camera intrinsic parameters, and preset maximum observation depth; Spatial blocks where the covered planar area intersects with the projected area are identified as spatial blocks where the view frustums intersect.

5. The method according to claim 4, characterized in that, The camera-assigned block is the spatial block in which the current frame camera's planar position in the world coordinate system is located; if the current frame camera's planar position does not fall into any divided spatial block, then the spatial block closest to that planar position is selected as the camera-assigned block.

6. The method according to claim 4, characterized in that, The set of neighborhood protection blocks is determined in the following way: Use all spatial blocks in the first set and the camera-assigned block as seed blocks; The spatial blocks whose shortest distance to any seed block is less than or equal to the preset protection radius are included in the neighborhood protection block set; The preset protection radius is a first value in the map optimization stage and a second value in the camera pose tracking stage, and the second value is greater than the first value.

7. The method according to claim 1, characterized in that, In step 3, the method for extracting Gaussian elements is as follows: Traverse each Gaussian cell in the global Gaussian map and read the spatial block identifier currently associated with that Gaussian cell; If the spatial block identifier belongs to the final active spatial block set, then add the index of the Gaussian to the local Gaussian working set; If it does not belong to the group, skip that Gaussian element.

8. The method according to claim 1, characterized in that, In step 4, the local rendering results include: a local rendering color image, a local rendering depth map, and a local rendering contour map; the local rendering only calculates the contribution value of Gaussian elements in the local Gaussian element working set to the image pixels, and the Gaussian elements in the global Gaussian map that are not selected into the local Gaussian element working set do not participate in the rendering and gradient calculation of the current frame.

9. The method according to claim 8, characterized in that, In step 5, determining the pixel region for which new Gaussian elements are to be added specifically involves: Pixels whose pixel values ​​in the local rendering contour map are lower than a preset contour existence threshold are determined to be pixels not covered by the contour. Pixels whose depth values ​​in the local rendering depth map are greater than the current frame's observed depth value and whose absolute difference is greater than a preset depth anomaly determination coefficient are identified as depth anomaly pixels. The union of the pixels whose contours are not covered and the pixels with abnormal depth is taken as the pixel region to be added to the Gaussian unit.

10. The method according to claim 1, characterized in that, During multiple map optimization iterations on the same keyframe, the block identifiers of the final active spatial block set are pre-cached. In each iteration, the local Gaussian working set corresponding to each iteration is re-extracted based on the cached block identifiers and the latest Gaussian elements and spatial block affiliation at the current moment, instead of directly caching or reusing the Gaussian index set of the previous iteration.

11. The method according to claim 3, characterized in that, The quadtree subdivision method is as follows: initially, the space is divided into several coarse-grained space blocks. When the number of Gaussian elements in any space block exceeds a preset load threshold, the space block is subdivided into a quadtree until the number of Gaussian elements in the space block does not exceed the preset load threshold.

12. An active spatial block selection and locally differentiable rendering system for Gaussian splash SLAM in large-scale scenes, characterized in that, include: The image acquisition module is used to acquire the initial frame image and depth information; The global map building module is used to build a global Gaussian map and associate each Gaussian element with a spatial block in a preset spatial partitioning structure according to its center position. Each spatial block has a unique and stable block identifier. An active spatial block selection module is used to select a final set of active spatial blocks based on the current frame camera pose and depth information using the method described in any one of claims 1 to 11. The local working set extraction module is used to extract the local Gaussian working set from the global Gaussian map based on the final active spatial block set and the affiliation relationship between each Gaussian element and the spatial block. Differentiable rendering module, used to perform differentiable rasterization rendering only on the local Gaussian working set, and output local rendering results; The pose tracking and map optimization module is used to perform camera pose tracking and map parameter optimization based on the local rendering results. The map extension module is used to determine the region of pixels to be added based on the difference between the local rendering results and the current frame observation, generate new Gaussian elements and update the attribution relationships; The storage and caching module is used to cache the block identifiers of the final active spatial block set so that they can be re-parsed by the local working set extraction module in each map optimization iteration.

Images