Hash octree-based three-dimensional gauss simultaneous localization and mapping method and system

Abstract

The application relates to the fields of computer vision and robot technology, and discloses a three-dimensional Gauss synchronous positioning and mapping method and system based on a hash octree, which processes time-synchronized laser radar point clouds, inertial measurement data and camera images and obtains a current frame pose; a global three-dimensional Gauss map is constructed on a main memory side by using hash coarse voxels and fine-grained octrees; relevant three-dimensional Gauss points are loaded into an active sub-map on a graphics processor side according to the current frame pose, a camera view cone and a common-view key frame observation range, and dormant three-dimensional Gauss points are removed but retained in the global map; key frame insertion, replacement or marginalization are performed according to a common-view coefficient of a visible three-dimensional Gauss point set of the key frame, and a training view angle is selected in a common-view key frame set; a differentiable rendering and parameter optimization are performed on the active sub-map, and updated three-dimensional Gauss parameters and an optimizer state are written back to corresponding leaf nodes. The application limits the size of an optimization working set on the graphics processor side.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and robotics technology, and specifically relates to a method and system for three-dimensional Gaussian simultaneous localization and mapping based on hash octree. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is used to estimate sensor poses and build environmental maps in unknown or partially unknown environments. Traditional methods often employ explicit geometric representations such as point clouds, voxels, or surface elements; implicit scene representations such as neural radiation fields can produce high-quality appearance reconstruction results, but online volume rendering and network optimization usually have high computational costs. 3D Gaussian Splatting (3DGS) uses a set of differentiable anisotropic 3D Gaussian ellipsoids to represent the scene, combining explicit structure with high rendering efficiency, and is therefore increasingly used for online 3D reconstruction and SLAM.

[0003] An existing 3D Gaussian simultaneous localization and mapping (SLR) method integrates data from LiDAR, inertial measurement units (IMUs), and cameras to represent the scene using a 3D Gaussian ellipsoid and incrementally construct a 3D Gaussian map. During the Gaussian mapping process, after the current frame is identified as a keyframe, several keyframes are randomly selected from all keyframes as training views, and iterative optimization is performed on the 3D Gaussian map. In one implementation of this method, the established 3D Gaussian map is maintained on the graphics processor (GPU) side. The color, scale, rotation, and opacity parameters of the 3D Gaussian points, as well as the corresponding gradients and optimizer states, continuously occupy GPU storage space as the map size increases.

[0004] As the scope of the scenarios expands, the above organizational methods have the following problems:

[0005] (1) The established global 3D Gaussian map is always stationed on the graphics processor side. The 3D Gaussian parameters, gradients and optimizer states continue to grow, which can easily exceed the available storage space of the resource-constrained platform.

[0006] (2) Although the three-dimensional Gaussian points outside the current observation area have a weak relationship with the current local mapping, they will still occupy the storage space on the graphics processor side;

[0007] (3) Randomly selecting training viewpoints from the global keyframe set will introduce observations with weak co-viewing relationship with the current local scene, resulting in redundant training viewpoints.

[0008] Furthermore, incremental mapping requires frequent spatial deduplication of 3D Gaussian points, local point set retrieval, and updates to the current optimized point set. Without a hierarchical spatial index suitable for large-scale incremental maps, the associated access and filtering overhead can easily increase with map size. Maintaining only a fixed-size keyframe window in chronological order may also generate a large number of similar keyframes while the camera is stationary, and retain historical keyframes with weak relevance to the current area during rapid navigation or scene changes.

[0009] Therefore, there is a need for a synchronous localization and mapping method that can maintain the integrity of the global 3D Gaussian map, perform efficient spatial management of global 3D Gaussian points, organize only the 3D Gaussian points corresponding to the current effective observation area into the device-side optimization working set, and manage training keyframes based on scene co-view relationships.

[0010] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0011] To address or at least alleviate one or more of the above problems, a method and system for simultaneous localization and mapping of 3D Gaussian based on hash octrees is provided. This method can limit the size of the graphics processor-side optimization working set while maintaining the integrity of the global 3D Gaussian map, and match the keyframe selection with the actual observation relationship of the current local scene.

[0012] To achieve the above objectives, according to a first aspect of the present invention, a method for three-dimensional Gaussian synchronous localization and mapping based on a hash octree is provided, characterized in that time-synchronized lidar point clouds, inertial measurement data, and camera images are processed, and the pose of the current frame is estimated by a multi-sensor localization front end, using an anisotropic three-dimensional Gaussian ellipsoid as the scene representation primitive, comprising the following steps:

[0013] S1. Construct a global hash octree map:

[0014] On the main memory side, the coordinates of bold voxels are determined based on the center position of the three-dimensional Gaussian points. Bold voxels are managed by hash mapping, and fine-grained octrees are set up in each bold voxel to perform hierarchical spatial indexing and persistent storage of the global three-dimensional Gaussian points.

[0015] S2. Construct a visibility-driven active submap:

[0016] Based on the current frame pose, camera frustum, and the observation range of the co-view keyframe, determine the bold voxels currently participating in optimization, load the 3D Gaussian points within the corresponding bold voxels from the global hash octree map to the active sub-map on the graphics processor side, and remove the 3D Gaussian points that meet the dormancy conditions from the active sub-map so that they continue to remain in the global hash octree map.

[0017] S3. Perform common-view keyframe management:

[0018] For candidate keyframes and existing keyframes within the window, a set of visible 3D Gaussian points is determined. The common vision coefficient is calculated based on the intersection and union relationship between the sets of visible 3D Gaussian points. Based on the common vision coefficient and the pose difference between keyframes, keyframes are inserted, replaced, or marginalized to form a set of keyframes for active submap optimization.

[0019] S4. Perform active submap optimization and write-back:

[0020] A training viewpoint is selected from the set of keyframes. Differentiable rendering is performed on the 3D Gaussian points in the active sub-map on the graphics processor side, and the loss is calculated. The 3D Gaussian points to be updated are determined based on their effective contribution to the rendering results. The parameters of the 3D Gaussian points to be updated are updated. After optimization, the updated 3D Gaussian parameters and their corresponding optimizer states are written back to the corresponding leaf nodes in the global hash octree map on the main memory side.

[0021] Further, in S1, the global hash octree map includes a two-level spatial index of coarse-grained hash grid and fine-grained octree within bold voxels; the center position of the three-dimensional Gaussian point is quantized into integer bold voxel coordinates according to the preset bold voxel side length, and the integer bold voxel coordinates are used as hash keys to locate the corresponding bold voxel; within the corresponding bold voxel, the space is divided into eight sub-regions layer by layer according to the three-axis direction until the preset maximum depth is reached; each octree leaf node stores at most one three-dimensional Gaussian point, and each bold voxel maintains a reference list of its occupied leaf nodes so as to access the three-dimensional Gaussian point within the bold voxel without recursively traversing the entire octree.

[0022] Furthermore, the three-dimensional Gaussian point includes at least a center position, appearance parameters, scale parameters, rotation parameters, and opacity parameters, and includes a status identifier to characterize whether it is located in an active submap and its corresponding position in the active submap; when inserting a new three-dimensional Gaussian point into the global hash octree map, the following steps are performed on the candidate LiDAR points in sequence: coverage filtering based on the transmittance rendered by the current Gaussian model to eliminate candidate points in areas that have been fully modeled; depth deduplication based on the image projection pixels, retaining the candidate point with the smallest depth when multiple candidate points correspond to the same pixel; and spatial deduplication based on the occupancy status of the octree leaf nodes, creating and inserting a new three-dimensional Gaussian point only when the target leaf node is not occupied.

[0023] Furthermore, in S2, the active sub-map is synchronously maintained as follows: active bold voxel set, leaf node reference list corresponding one-to-one with active 3D Gaussian points, active 3D Gaussian parameter array, and optimizer state array; the leaf node reference list, 3D Gaussian parameter array, and optimizer state array correspond to the same 3D Gaussian point according to the same index.

[0024] Furthermore, the update of the active submap includes three states: reconstruction update, incremental append, and no change. When there are invisible 3D Gaussian points that meet the preset conditions and need to be removed from the active submap, a reconstruction update is performed; when there are only newly activated 3D Gaussian points, an incremental append is performed; when the active set is consistent with the parameter set on the graphics processor side, no change is performed; and the overhead caused by frequent reconstruction is reduced by at least one of the constraints of upper limit threshold, lower limit threshold, minimum update interval, and minimum removal quantity.

[0025] Furthermore, in S3, each keyframe records the 3D Gaussian points visible in the active submap through a bit set. The i-th bit of the bit set being 1 indicates that the i-th 3D Gaussian point in the active submap is located in front of the camera of the keyframe and falls within the preset view frustum of the keyframe. The co-view coefficient of keyframe A and keyframe B is represented by the Jaccard coefficient of the visible 3D Gaussian point set, that is, the co-view coefficient is the ratio of the number of intersection elements to the number of union elements of the visible 3D Gaussian point sets of the two keyframes.

[0026] Furthermore, the management of candidate keyframes includes the following steps: when the co-view coefficient between a candidate keyframe and an existing keyframe is higher than a preset redundancy threshold, the candidate keyframe replaces the corresponding redundant keyframe; otherwise, when the positional distance and line-of-sight angle between a candidate keyframe and the nearest keyframe are both lower than the corresponding thresholds, insertion is skipped; in other cases, the candidate keyframe is inserted and the co-view relationship is updated incrementally; when the maximum co-view coefficient between the newly inserted keyframe and the existing keyframes in the window is lower than a preset scene switching threshold, the nearest preset number of keyframes are retained; when the keyframe window capacity exceeds the limit, keyframe marginalization is performed; when performing keyframe marginalization, the latest keyframe, the keyframe with the lowest average co-view coefficient, and the keyframe that has an exclusive observation contribution to the active sub-map are protected; for the remaining keyframes, a marginalization score is calculated based on redundancy, co-view coefficient with the latest keyframe, normalized dwell time, and coverage impact, and keyframes whose scores meet the preset marginalization conditions are removed.

[0027] Furthermore, when selecting a training perspective from the set of keyframes, the basic sampling weight is determined based on the number of times each keyframe has been selected; the more times a keyframe has been selected, the smaller its basic sampling weight. Further, the basic sampling weight is modified based on the co-visibility between the candidate keyframe and recently selected keyframes; the lower the co-visibility between the candidate keyframe and recently selected keyframes, the larger its modified weight. The modified weights are then normalized to obtain the sampling probability of each keyframe, and the training perspective is selected according to the sampling probability.

[0028] Furthermore, in S4, after each forward differentiable rendering, a set to be updated is determined based on whether the 3D Gaussian point makes a valid synthetic contribution to at least one pixel. Backpropagation and optimizer parameter updates are performed only on the 3D Gaussian points in the set to be updated, while the parameters of 3D Gaussian points that did not participate in the current rendering remain unchanged. The optimizer state includes the first-order moment state and the second-order moment state of the adaptive moment estimation optimizer. When a 3D Gaussian point is removed from the active submap, its optimizer state is persisted to the corresponding octagonal leaf node. When the 3D Gaussian point is reactivated, the optimizer state is restored from the corresponding octagonal leaf node to continue the optimization process of the 3D Gaussian point.

[0029] To achieve the above objectives, according to a second aspect of the present invention, a three-dimensional Gaussian simultaneous localization and mapping system based on a hash octree is provided, comprising:

[0030] The data synchronization and pose estimation module is used to synchronize the lidar point cloud, inertial measurement data and camera images in time, and the pose of the current frame is estimated by the multi-sensor positioning front end.

[0031] The global map management module is used to determine the coordinates of bold voxels on the main memory side based on the center position of the three-dimensional Gaussian points, manage bold voxels with hash mapping, and set up fine-grained octrees in each bold voxel to perform hierarchical spatial indexing and persistent storage of global three-dimensional Gaussian points.

[0032] The visibility sub-map module is used to determine the currently participating bold voxels based on the current frame pose, camera frustum, and the observation range of the co-view keyframe. It loads the 3D Gaussian points within the corresponding bold voxels from the global hash octree map to the active sub-map on the graphics processor side, and removes the 3D Gaussian points that meet the dormancy conditions from the active sub-map so that they continue to remain in the global hash octree map.

[0033] The common-view keyframe management module is used to determine the set of visible 3D Gaussian points for candidate keyframes and existing keyframes in the window, calculate the common-view coefficient based on the intersection and union relationship between the sets of visible 3D Gaussian points, and perform keyframe insertion, replacement or marginalization based on the common-view coefficient and the pose difference between keyframes to form a set of keyframes for active submap optimization.

[0034] The 3D Gaussian optimization and write-back module is used to select a training viewpoint from the keyframe set, perform differentiable rendering on the 3D Gaussian points in the active sub-map on the graphics processor side and calculate the loss, determine the 3D Gaussian points to be updated based on the effective contribution of the 3D Gaussian points to the rendering results, perform parameter updates on the 3D Gaussian points to be updated, and write the updated 3D Gaussian parameters and their corresponding optimizer states back to the corresponding leaf nodes in the global hash octree map on the main memory side.

[0035] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art.

[0036] This invention combines hash coarse positioning with octree fine positioning to establish a hierarchical spatial index for global 3D Gaussian points. This enables spatial deduplication, local point set access, and Gaussian point state management to be mainly performed around local coarse voxels and their leaf nodes, reducing the need for global scanning as the map size grows.

[0037] This invention separates the global persistent map on the main memory side from the active sub-map on the graphics processor side, and loads only the 3D Gaussian points related to the current viewpoint and common keyframes into the device-side optimization working set, thereby limiting the scale of the device-side persistent 3D Gaussian and reducing optimization overhead that is irrelevant to the current local scene.

[0038] This invention manages common-view keyframes based on a set of visible 3D Gaussian points, making keyframe selection relevant to the actual observation of the current local scene, reducing highly redundant viewpoints, and updating the training keyframe set in a timely manner when the viewpoint changes rapidly or the scene switches.

[0039] This invention reduces invalid gradient calculations caused by 3D Gaussian points that are not involved in the current rendering by performing gradient updates only on the 3D Gaussian points that make a valid contribution to the current rendering.

[0040] This invention maintains a consistent correspondence between parameters and optimization state when a 3D Gaussian point enters and exits an active submap by using leaf node reference relationships and optimizer state persistence, so that a reactivated 3D Gaussian point can continue its previous optimization state.

[0041] The specific embodiments of the present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0042] The accompanying drawings, as part of this invention, are provided to further illustrate the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention, but do not constitute an undue limitation thereof. Clearly, the drawings described below are merely some embodiments; those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0043] Figure 1 This is a schematic diagram of the overall process of the three-dimensional Gaussian synchronous localization and mapping method based on hash octree in this specific embodiment.

[0044] Figure 2 This is a schematic diagram of the structure of bold voxel hash positioning, bold voxel inner octree descent, and leaf node association with three-dimensional Gaussian points in this specific embodiment.

[0045] Figure 3 This is a schematic diagram illustrating the loading, activation, write-back, and hibernation process between the global hash octree map on the main memory side and the active sub-map on the graphics processor side in this specific embodiment.

[0046] Figure 4 This is a schematic diagram illustrating the process of managing the shared keyframe set and candidate keyframe insertion, replacement, and marginalization in this specific embodiment.

[0047] Figure 5 This is a schematic diagram comparing the mapping frame rate and device-side memory usage of the method of the present invention and the comparative method in this specific embodiment.

[0048] Figure 6 This is a schematic diagram of the structure of the three-dimensional Gaussian synchronous localization and mapping system based on a hash octree in this specific embodiment. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the accompanying drawings. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0050] In the following description, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0051] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0052] In the description of the embodiments of the present invention, "main memory side" refers to the system memory side that the central processing unit can directly access, and "graphics processor side" refers to the device memory side of the graphics processor. "Active sub-map" refers to the working set of 3D Gaussian points currently participating in optimization on the graphics processor side; "dormant state" refers to the state in which the 3D Gaussian points are still stored in the global hash octree map on the main memory side but are not loaded to the graphics processor side; "co-visibility coefficient" refers to the degree of similarity between the sets of 3D Gaussian points observed in two keyframes.

[0053] Please see Figure 1 This invention provides a 3D Gaussian synchronous localization and mapping method based on a hash octree. It processes time-synchronized LiDAR point clouds, inertial measurement data, and camera images, and estimates the current frame pose using a multi-sensor localization front-end. An anisotropic 3D Gaussian ellipsoid is used as the scene representation primitive. The method includes the following steps:

[0054] S1. Construct a global hash octree map: On the main memory side, determine the coordinates of bold voxels based on the center position of the three-dimensional Gaussian points, manage bold voxels with hash mapping, and set up fine-grained octrees in each bold voxel to perform hierarchical spatial indexing and persistent storage of the global three-dimensional Gaussian points.

[0055] S2. Construct a visibility-driven active sub-map: Based on the current frame pose, camera frustum, and the observation range of the co-view keyframe, determine the bold voxels currently participating in optimization, load the 3D Gaussian points within the corresponding bold voxels from the global hash octree map to the active sub-map on the graphics processor side, and remove the 3D Gaussian points that meet the dormancy conditions from the active sub-map so that they continue to remain in the global hash octree map.

[0056] S3. Perform common-view keyframe management: Determine the visible 3D Gaussian point set for the candidate keyframe and the existing keyframe in the window respectively, calculate the common-view coefficient according to the intersection and union relationship between the visible 3D Gaussian point sets, and perform keyframe insertion, replacement or marginalization according to the common-view coefficient and the pose difference between keyframes to form a keyframe set for active submap optimization.

[0057] S4. Perform active submap optimization and write-back: Select a training viewpoint from the keyframe set, perform differentiable rendering on the 3D Gaussian points in the active submap on the graphics processor side and calculate the loss, and determine the 3D Gaussian points to be updated based on the effective contribution of the 3D Gaussian points to the rendering results, and perform parameter updates on the 3D Gaussian points to be updated; after optimization, write the updated 3D Gaussian parameters and their corresponding optimizer states back to the corresponding leaf nodes in the global hash octree map on the main memory side.

[0058] Specifically, camera images, inertial measurement data, and lidar point clouds are first synchronized in time, and the pose of the current frame is estimated by a multi-sensor positioning front-end. The positioning front-end can employ existing visual, lidar, or visual-inertial fusion positioning methods; the improvement of this invention does not focus on limiting specific pose estimation algorithms. The current frame that meets the mapping update conditions is used to generate new 3D Gaussian candidate points and update the global 3D Gaussian map.

[0059] In other embodiments, in S1, the global hash octree map includes a two-level spatial index of a coarse-grained hash grid and a fine-grained octree within a coarse voxel; the center position of the three-dimensional Gaussian point is quantized into integer coarse voxel coordinates according to a preset coarse voxel side length, and the integer coarse voxel coordinates are used as hash keys to locate the corresponding coarse voxel; within the corresponding coarse voxel, the space is divided into eight sub-regions layer by layer according to the three-axis direction until a preset maximum depth is reached; each octree leaf node stores at most one three-dimensional Gaussian point, and each coarse voxel maintains a reference list of its occupied leaf nodes so as to access the three-dimensional Gaussian point within the coarse voxel without recursively traversing the entire octree.

[0060] Specifically, please see Figure 2The global hash octree map employs a two-level spatial index combining a coarse-grained hash grid with a fine-grained octree within each coarse voxel. First, the center position of the 3D Gaussian point is quantized into integer coarse voxel coordinates according to a preset coarse voxel side length. The calculation method is as follows:

[0061]

[0062] Where p represents the center position of the three-dimensional Gaussian point, s v represents the side length of the bold element, and v represents the integer coordinates of the bold element.

[0063] In one specific implementation, a polynomial hash function can be used for the integer bold pixel coordinates (x, y, z), and its calculation method is as follows:

[0064] H(x,y,z) = ((z × 116101 + y) × 116101 + x) mod 10 10

[0065] Here, x, y, and z represent the three components of the integer bold voxel coordinates, 116101 is a prime number, and H(x,y,z) represents the hash key corresponding to the integer bold voxel coordinates. Using integer coordinates to construct the hash key avoids the precision ambiguity caused by floating-point coordinate hashing. This hash function is only one possible implementation and does not constitute a limitation on the hash implementation.

[0066] The corresponding bold element is located using the integer bold element coordinates as a hash key. Inside the bold element, starting from the root node of the octree, the next layer of child nodes is determined based on the position of the center of the three-dimensional Gaussian point relative to the center of the current node in the x, y, and z directions, and the process is repeated layer by layer until the leaf node is reached.

[0067] In one implementation, each leaf node of the octree stores at most one 3D Gaussian point, thus maintaining a one-to-one correspondence between leaf nodes and 3D Gaussian points. Each bold voxel maintains a reference list of its occupied leaf nodes to traverse the existing 3D Gaussian points within the voxel without recursively accessing a large number of empty nodes. In one specific parameter setting, the side length of the bold voxel can be set to 0.5m, the maximum depth of the octree can be set to 6, and the corresponding leaf node side length is approximately 7.8mm. The above values ​​are only used to illustrate one spatial resolution configuration; in practical applications, the side length of the bold voxel and the depth of the octree can be adjusted according to the LiDAR point cloud density, the 3D Gaussian scale, and the scene range.

[0068] In some embodiments, a 3D Gaussian point includes at least a center position, appearance parameters, scale parameters, rotation parameters, and opacity parameters, and includes a status identifier to characterize whether it is located in an active submap and its corresponding position in the active submap; when inserting a new 3D Gaussian point into the global hash octree map, the following steps are performed on the candidate LiDAR points in sequence: coverage filtering based on the transmittance rendered by the current Gaussian model to remove candidate points in areas that have been fully modeled; depth deduplication based on the image projection pixels, retaining the candidate point with the smallest depth when multiple candidate points correspond to the same pixel; and spatial deduplication based on the occupancy status of the octree leaf nodes, creating and inserting a new 3D Gaussian point only when the target leaf node is not occupied.

[0069] Specifically, please see Figure 1 When inserting a new 3D Gaussian point into the global hash octree map, the candidate LiDAR points are first projected onto the current image. For candidate points that can be projected onto the image, it is determined whether the corresponding pixel has been fully modeled by the existing 3D Gaussian points based on the transmittance or equivalent coverage obtained from the current Gaussian model rendering. In one specific implementation, the transmittance threshold can be set to 0.01. For candidate points that pass the coverage screening, they are grouped according to image pixels. When the same pixel corresponds to multiple candidate points, the candidate point with the smaller depth is retained. Then, the hash octree is used to check the occupancy status of the target leaf node, and a new 3D Gaussian point is created and inserted only when the target leaf node is not occupied.

[0070] The newly created 3D Gaussian point includes at least its center position, appearance parameters, scale parameters, rotation parameters, and opacity parameters, as well as a status identifier indicating whether it is located in an active submap and its corresponding position in the active submap. If the bold element containing the newly created 3D Gaussian point already belongs to an active submap, the 3D Gaussian point is simultaneously appended to the active submap to participate in the current optimization; otherwise, the 3D Gaussian point is only written to the global hash octree map on the main memory side and remains dormant until the bold element containing it is subsequently activated and then loaded onto the graphics processor side.

[0071] In one specific implementation, taking a third-order spherical harmonic appearance parameter and adaptive moment estimation optimizer as an example, a three-dimensional Gaussian point can include a 3D center position, a 3D basic color, 45-dimensional higher-order spherical harmonic coefficients, a 1D opacity, a 3D scale, and a 4D rotation parameter; the optimizer can store the first-order and second-order moments for each trainable dimension. The above parameter dimensions and optimizer type are only used to illustrate one specific implementation and do not limit other orders of spherical harmonic representation, other differentiable three-dimensional Gaussian parameterization methods, or other optimization algorithms.

[0072] In some embodiments, the active sub-map in S2 is synchronously maintained as follows: an active bold voxel set, a leaf node reference list corresponding one-to-one with active 3D Gaussian points, an active 3D Gaussian parameter array, and an optimizer state array; the leaf node reference list, the 3D Gaussian parameter array, and the optimizer state array correspond to the same 3D Gaussian point according to the same index.

[0073] Specifically, please see Figure 3 The main memory-side global hash octree map stores the complete 3D Gaussian map, while the GPU-side active sub-map is the working set of the global 3D Gaussian map at the current optimization moment. Based on the current frame pose, camera frustum, and the observation range of the shared keyframes, the system determines the bold voxels that need to participate in the current optimization and loads the 3D Gaussian parameters within the corresponding voxels and their corresponding optimizer states from the main memory-side global hash octree map to the GPU-side active sub-map. For 3D Gaussian points that are invisible to all keyframes within the shared window and meet preset removal conditions, they are removed from the active sub-map, but their parameters are still completely stored in the main memory-side global hash octree map; they are only marked as dormant and do not participate in the current round of device-side optimization.

[0074] It should be noted that the active submap in this embodiment is based on a set of bold voxels determined by visibility, rather than a fixed geometric region centered on the camera. Therefore, more bold voxels will be activated in open areas, while fewer bold voxels will be activated in occluded or narrow areas, so that the device-side working set matches the actual observable scene.

[0075] The active submap synchronously maintains an active bold voxel set, a leaf node reference list corresponding one-to-one with active 3D Gaussian points, an active 3D Gaussian parameter array, and an optimizer status array, wherein the leaf node references, 3D Gaussian parameters, and optimizer status correspond to the same 3D Gaussian point according to the same index; the leaf node references are used to accurately write the updated parameters back to the global hash octree map on the main memory side after optimization is completed on the device side.

[0076] In other embodiments, the update of the active submap includes three states: reconstruction update, incremental append, and no change. When there are invisible 3D Gaussian points that meet preset conditions and need to be removed from the active submap, a reconstruction update is performed. When there are only newly activated 3D Gaussian points, an incremental append is performed. When the active set is consistent with the parameter set on the graphics processor side, no change is performed. The overhead caused by frequent reconstruction is reduced by at least one of the following constraints: upper limit threshold, lower limit threshold, minimum update interval, and minimum removal quantity.

[0077] Specifically, to reduce the overhead caused by frequent reconstruction of device-side parameter tensors, the active submap performs reconstruction updates, incremental appending, or maintains the status quo based on changes in the active set. When there are invisible 3D Gaussian points that meet the removal criteria, the latest parameters and optimizer status of the 3D Gaussian points to be removed are first written back to the global hash octree map on the main memory side. Then, these 3D Gaussian points are marked as dormant, and the active parameter array and leaf node reference list are compressed. Finally, the GPU-side parameter tensor and keyframe visibility data are reconstructed. This method is called reconstruction update. When there are only newly activated 3D Gaussian points, the corresponding parameters are appended to the original active set without reconstructing the existing parameters. This method is called incremental appending. When the active set has not changed, the GPU-side data structure remains unchanged.

[0078] In one specific implementation, an upper limit, a lower limit, a minimum reconstruction interval, and a minimum batch removal quantity can be set to form hysteresis control: invisible 3D Gaussian points are collected as candidate removal objects only when the number of active 3D Gaussian points exceeds the upper limit and the minimum reconstruction interval has been reached since the last reconstruction update; a reconstruction update is performed only when the number of valid candidate removal objects reaches the minimum batch removal quantity; no new removal is triggered when the number of active 3D Gaussian points is lower than the lower limit. In one specific parameter setting, the upper limit can be set to 50,000 3D Gaussian points, the lower limit can be set to 40,000 3D Gaussian points, the minimum reconstruction interval can be set to 5 keyframes, and the minimum batch removal quantity can be set to 1,000 3D Gaussian points; the above values ​​are only used to illustrate the idea of ​​hysteresis control and batch update, and can be adjusted according to the platform memory capacity and scene scale.

[0079] In some embodiments, in S3, each keyframe records the 3D Gaussian points visible in the active submap through a bit set. The i-th bit of the bit set being 1 indicates that the i-th 3D Gaussian point in the active submap is located in front of the camera of the keyframe and falls within the preset view frustum of the keyframe. The co-view coefficient of keyframe A and keyframe B is represented by the Jaccard coefficient of the visible 3D Gaussian point set, that is, the co-view coefficient is the ratio of the number of intersection elements to the number of union elements of the visible 3D Gaussian point sets of the two keyframes.

[0080] Specifically, please see Figure 4 Each keyframe records the set of 3D Gaussian points visible in the active submap. To improve the efficiency of set intersection and union query operations, in one implementation, a bit set is used to record visibility. A value of 1 in the i-th bit of the bit set indicates that the i-th 3D Gaussian point in the active submap is located in front of the camera in that keyframe and falls within the preset view frustum of that keyframe. The degree of co-visibility between two keyframes A and B is represented by the Jaccard coefficient of the visible 3D Gaussian point set, which is calculated as follows:

[0081] C(A,B) = |G A ∩ G B | / |G A ∪ G B |

[0082] Among them, G A Let G represent the set of visible 3D Gaussian points in keyframe A. B Let G represent the set of visible 3D Gaussian points in keyframe B. A ∩ G B | represents the number of elements in the intersection of two sets, | G A ∪ G B | represents the number of elements in the union of two sets, and C(A, B) represents the co-visibility coefficient between two keyframes, with a value ranging from 0 to 1; the larger the co-visibility coefficient, the closer the sets of 3D Gaussian points observed by the two keyframes are.

[0083] In other embodiments, the management of candidate keyframes includes the following steps: when the co-view coefficient between a candidate keyframe and an existing keyframe is higher than a preset redundancy threshold, the candidate keyframe replaces the corresponding redundant keyframe; otherwise, when the positional distance and line-of-sight angle between a candidate keyframe and the nearest keyframe are both lower than the corresponding thresholds, insertion is skipped; in other cases, the candidate keyframe is inserted and the co-view relationship is updated incrementally; when the maximum co-view coefficient between the newly inserted keyframe and the existing keyframes in the window is lower than a preset scene switching threshold, the nearest preset number of keyframes are retained; when the keyframe window capacity exceeds the limit, keyframe marginalization is performed; when performing keyframe marginalization, the latest keyframe, the keyframe with the lowest average co-view coefficient, and the keyframe that has an exclusive observation contribution to the active sub-map are protected; for the remaining keyframes, a marginalization score is calculated based on redundancy, co-view coefficient with the latest keyframe, normalized dwell time, and coverage impact, and keyframes whose scores meet the preset marginalization conditions are removed.

[0084] Specifically, please see Figure 4Upon arrival of a candidate keyframe, the co-view coefficient between it and existing keyframes within the window is first calculated. If the co-view coefficient between the candidate keyframe and existing keyframes is higher than a preset redundancy threshold, it indicates that the sets of 3D Gaussian points observed by both are highly overlapping. In this case, the newer candidate keyframe replaces the corresponding redundant keyframe without increasing the number of keyframes in the window. Otherwise, if the positional distance and the angle between the candidate keyframe and the nearest keyframe are both less than the corresponding thresholds, it indicates that the pose of the candidate keyframe is too close to that of the nearest keyframe. In this case, the candidate keyframe is skipped without insertion. In other cases, the candidate keyframe is added to the window and the co-view matrix is ​​updated incrementally. If the maximum co-view coefficient between the newly inserted keyframe and existing keyframes in the window is lower than a preset scene switching threshold, a scene switch is determined to occur. In this case, several recent keyframes are retained to prevent older scene keyframes from continuing to occupy the current optimization budget.

[0085] In one specific parameter setting, the redundancy threshold can be set to 0.85, the scene switching threshold can be set to 0.10, the minimum insertion distance can be set to 0.3m, the minimum view angle can be set to 10 degrees, and the maximum number of keyframes within the window can be set to 10; the above values ​​are examples and can be adjusted according to the scene scale and motion speed.

[0086] When the keyframe window capacity exceeds the limit, a keyframe protection set is first constructed to protect the latest keyframe, keyframes with low average co-view coefficients, and keyframes that make exclusive observation contributions to the active submap, such as keyframes whose unique observable 3D Gaussian points account for a proportion of the active submap exceeding a preset proportion threshold. For the remaining keyframes, a marginalization score is calculated based on redundancy, co-view degree with the latest keyframe, normalized dwell time, and coverage impact. An example scoring function is calculated as follows:

[0087] S = 0.40R + 0.25C n + 0.20A + 0.15(1- I c )

[0088] Where S represents the marginalization score, R represents the average co-visibility between this keyframe and the other keyframes within the window, and C n This indicates the co-viewability between the current keyframe and the latest keyframe, where A represents the normalized dwell time, and I represents the co-viewability between the current keyframe and the latest keyframe. c This represents the coverage impact, specifically the proportion of 3D Gaussian points in the active submap that are no longer observable by other keyframes after the keyframe is removed. Unprotected keyframes that meet the preset marginalization criteria are selected for removal, and the keyframe set, common-view matrix, and visibility information are updated simultaneously.

[0089] In some embodiments, when selecting a training perspective from the set of keyframes, a basic sampling weight is determined based on the number of times each keyframe has been selected, with keyframes selected more frequently having a smaller basic sampling weight. Furthermore, the basic sampling weight is modified based on the co-visibility between candidate keyframes and recently selected keyframes, with candidate keyframes having a lower co-visibility with recently selected keyframes having a larger modified weight. The modified weights are then normalized to obtain the sampling probability of each keyframe, and the training perspective is selected according to the sampling probability.

[0090] Specifically, to avoid randomly selecting the training perspective from the global keyframe set all the time, this embodiment selects the training perspective within the current shared keyframe set. Let n be the number of times the i-th keyframe has been selected. i The maximum number of selections within the window is n. max The temperature parameter is T, and its basic sampling weight is calculated as follows:

[0091] w i = exp((n max -n i ) / T)

[0092] Among them, w i n represents the base sampling weight of the i-th keyframe. i n represents the number of times the i-th keyframe has been selected. max This represents the maximum number of times each keyframe within the window has been selected, where T represents the temperature parameter, and exp(·) represents an exponential function with the natural constant as its base. From the above formula, it can be seen that the more times a keyframe has been selected, the smaller its base sampling weight.

[0093] Normalizing the base sampling weights yields the sampling probability of each keyframe. Furthermore, the base sampling weights can be modified for diversity based on the maximum co-viewpoint between candidate keyframes and recently selected keyframes to reduce the probability of continuously selecting highly similar training perspectives. In one specific implementation, the temperature parameter T can be set to 2.0.

[0094] In some embodiments, in S4, after each forward differentiable rendering, a set to be updated is determined based on whether the 3D Gaussian point makes a valid synthetic contribution to at least one pixel. Backpropagation and optimizer parameter updates are performed only on the 3D Gaussian points in the set to be updated, while the parameters of the 3D Gaussian points that did not participate in the current rendering remain unchanged. The optimizer state includes the first-order moment state and the second-order moment state of the adaptive moment estimation optimizer. When a 3D Gaussian point is removed from the active submap, its optimizer state is persisted to the corresponding octagonal leaf node. When the 3D Gaussian point is reactivated, the optimizer state is restored from the corresponding octagonal leaf node to continue the optimization process of the 3D Gaussian point.

[0095] Specifically, after selecting the training viewpoint from the shared-view keyframe set formed in step S3 according to the aforementioned sampling probability, 3D Gaussian differentiable rendering is performed on the active sub-map on the graphics processor side, and a loss function is calculated. The loss function may include image brightness loss, structural similarity loss, and optional depth loss. After each forward differentiable rendering, a set to be updated is determined based on whether each 3D Gaussian point makes a valid synthetic contribution to at least one pixel. Backpropagation and parameter updates are performed only on the 3D Gaussian points in the set to be updated. The parameters of 3D Gaussian points not participating in the current rendering remain unchanged, thereby reducing invalid gradient calculations. Dormant 3D Gaussian points not loaded into the active sub-map do not participate in the rendering and parameter updates of the current round.

[0096] For the implementation using an adaptive moment estimation optimizer, the optimizer state for each 3D Gaussian point can include a first-order moment state and a second-order moment state. When a 3D Gaussian point transitions from an active state to a dormant state, the updated 3D Gaussian parameters and the corresponding optimizer state are written back to the corresponding octagonal leaf node via a leaf node reference. When the 3D Gaussian point is subsequently activated again, the previously saved parameters and optimizer state are restored from the corresponding leaf node to continue its optimization process, avoiding repeated initialization of the optimizer from zero when the 3D Gaussian point repeatedly enters and exits the active submap.

[0097] Please see Figure 5 , Figure 5 The overall system performance of this embodiment and the comparative method is shown. The comparative method uses an existing 3D Gaussian simultaneous localization and mapping method, which maintains the established global 3D Gaussian map on the graphics processor side and optimizes the Gaussian map by randomly selecting training perspectives from all keyframes according to its training strategy. This embodiment, on the other hand, uses a global hash octree on the main memory side to store the complete Gaussian map, constructs only the 3D Gaussian points corresponding to the currently visible area as the active sub-map on the graphics processor side, and selects training perspectives from the set of shared keyframes.

[0098] Under the same data sequence, rendering resolution, and loss function settings, the comparison method achieved a mapping frame rate of 1.53 frames per second, while the method in this embodiment achieved 9.84 frames per second. The device-side memory usage of the comparison method was 32.8 GB, while that of the method in this embodiment was 3.5 GB. These experimental results are used to illustrate the performance of the overall technical solution under these experimental conditions, and do not attribute the improvement solely to any single technical feature.

[0099] It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention. Those skilled in the art can adjust or replace the coarse voxel scale, octree depth, keyframe threshold, window size, loss function, optimizer type, and specific data structure implementation without departing from the core technical concept of the present invention, and such adjustments or replacements should all be included within the protection scope of the present invention.

[0100] Based on the same inventive concept, please see Figure 6 The present invention also provides a three-dimensional Gaussian simultaneous localization and mapping system based on a hash octree, comprising:

[0101] The data synchronization and pose estimation module is used to synchronize the lidar point cloud, inertial measurement data and camera images in time, and the pose of the current frame is estimated by the multi-sensor positioning front end.

[0102] The global map management module is used to determine the coordinates of bold voxels on the main memory side based on the center position of the three-dimensional Gaussian points, manage bold voxels with hash mapping, and set up fine-grained octrees in each bold voxel to perform hierarchical spatial indexing and persistent storage of global three-dimensional Gaussian points.

[0103] The visibility sub-map module is used to determine the currently participating bold voxels based on the current frame pose, camera frustum, and the observation range of the co-view keyframe. It loads the 3D Gaussian points within the corresponding bold voxels from the global hash octree map to the active sub-map on the graphics processor side, and removes the 3D Gaussian points that meet the dormancy conditions from the active sub-map so that they continue to remain in the global hash octree map.

[0104] The common-view keyframe management module is used to determine the set of visible 3D Gaussian points for candidate keyframes and existing keyframes in the window, calculate the common-view coefficient based on the intersection and union relationship between the sets of visible 3D Gaussian points, and perform keyframe insertion, replacement or marginalization based on the common-view coefficient and the pose difference between keyframes to form a set of keyframes for active submap optimization.

[0105] The 3D Gaussian optimization and write-back module is used to select a training viewpoint from the keyframe set, perform differentiable rendering on the 3D Gaussian points in the active sub-map on the graphics processor side and calculate the loss, determine the 3D Gaussian points to be updated based on the effective contribution of the 3D Gaussian points to the rendering results, perform parameter updates on the 3D Gaussian points to be updated, and write the updated 3D Gaussian parameters and their corresponding optimizer states back to the corresponding leaf nodes in the global hash octree map on the main memory side.

[0106] Based on the same inventive concept, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the three-dimensional Gaussian synchronous localization and mapping method based on a hash octree as described above.

[0107] The program product of the present invention for implementing the above-described method may employ a portable compact disk read-only memory and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In the present invention, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0108] It should be noted that a computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0109] Unless otherwise defined, the technical or scientific terms used in this invention shall have the general meaning understood by one of ordinary skill in the art to which this invention pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this invention do not indicate quantitative limitations and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this invention are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this invention are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “A plurality” used in this invention refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. In this invention, the terms "first," "second," "third," etc., are used only to distinguish similar objects and do not represent a specific order of objects.

[0110] Obviously, the accompanying drawings are merely some examples or embodiments of the present invention. Those skilled in the art can apply the present invention to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, certain design, manufacturing, or production modifications made according to the technical content disclosed in the present invention are merely conventional technical means and should not be considered as insufficient disclosure of the present invention.

[0111] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-described technical content to create equivalent embodiments without departing from the scope of the present invention. The implementation schemes in the above embodiments can also be further combined or replaced. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for simultaneous localization and mapping of three-dimensional Gaussian systems based on hash octrees, characterized in that, The time-synchronized lidar point cloud, inertial measurement data, and camera images are processed, and the current frame pose is estimated by the multi-sensor positioning front end. An anisotropic 3D Gaussian ellipsoid is used as the scene representation primitive. The process includes the following steps: S1. Construct a global hash octree map: On the main memory side, the coordinates of bold voxels are determined based on the center position of the three-dimensional Gaussian points. Bold voxels are managed by hash mapping, and fine-grained octrees are set up in each bold voxel to perform hierarchical spatial indexing and persistent storage of the global three-dimensional Gaussian points. S2. Construct a visibility-driven active submap: Based on the current frame pose, camera frustum, and the observation range of the co-view keyframe, determine the bold voxels currently participating in optimization, load the 3D Gaussian points within the corresponding bold voxels from the global hash octree map to the active sub-map on the graphics processor side, and remove the 3D Gaussian points that meet the dormancy conditions from the active sub-map so that they continue to remain in the global hash octree map. S3. Perform common-view keyframe management: For candidate keyframes and existing keyframes within the window, a set of visible 3D Gaussian points is determined. The common vision coefficient is calculated based on the intersection and union relationship between the sets of visible 3D Gaussian points. Based on the common vision coefficient and the pose difference between keyframes, keyframes are inserted, replaced, or marginalized to form a set of keyframes for active submap optimization. S4. Perform active submap optimization and write-back: A training viewpoint is selected from the set of keyframes. Differentiable rendering is performed on the 3D Gaussian points in the active sub-map on the graphics processor side, and the loss is calculated. The 3D Gaussian points to be updated are determined based on their effective contribution to the rendering results. The parameters of the 3D Gaussian points to be updated are updated. After optimization, the updated 3D Gaussian parameters and their corresponding optimizer states are written back to the corresponding leaf nodes in the global hash octree map on the main memory side.

2. The method according to claim 1, characterized in that, In S1, the global hash octree map includes a two-level spatial index: a coarse-grained hash grid and a fine-grained octree within a bold voxel. The center position of the three-dimensional Gaussian point is quantized into integer bold voxel coordinates according to a preset bold voxel side length, and the integer bold voxel coordinates are used as hash keys to locate the corresponding bold voxel. Within the corresponding bold voxel, the space is divided into eight sub-regions layer by layer according to the three-axis direction until a preset maximum depth is reached. Each leaf node of the octree stores at most one three-dimensional Gaussian point, and each bold voxel maintains a reference list of its occupied leaf nodes so as to access the three-dimensional Gaussian point within the bold voxel without recursively traversing the entire octree.

3. The method according to claim 1, characterized in that, The three-dimensional Gaussian point includes at least a center position, appearance parameters, scale parameters, rotation parameters, and opacity parameters, and includes a status identifier to characterize whether it is located in an active submap and its corresponding position in the active submap; when inserting a new three-dimensional Gaussian point into the global hash octree map, the following steps are performed on the candidate LiDAR points in sequence: coverage filtering based on the transmittance rendered by the current Gaussian model to remove candidate points in areas that have been fully modeled; Depth-based deduplication of image projection pixels retains the candidate point with the smallest depth when multiple candidate points correspond to the same pixel; and spatial deduplication based on the occupancy status of octagonal leaf nodes creates and inserts new 3D Gaussian points only when the target leaf node is not occupied.

4. The method according to claim 1, characterized in that, In S2, the active submap is synchronously maintained as follows: active bold voxel set, leaf node reference list corresponding one-to-one with active 3D Gaussian points, active 3D Gaussian parameter array, and optimizer state array; the leaf node reference list, 3D Gaussian parameter array, and optimizer state array correspond to the same 3D Gaussian point according to the same index.

5. The method according to claim 4, characterized in that, The update of the active submap includes three states: reconstruction update, incremental append, and no change. When there are invisible 3D Gaussian points that meet the preset conditions and need to be removed from the active submap, a reconstruction update is performed; when there are only newly activated 3D Gaussian points, an incremental append is performed; when the active set is consistent with the parameter set on the graphics processor side, no change is performed. The overhead caused by frequent reconstruction is reduced by using at least one of the following constraints: upper limit threshold, lower limit threshold, minimum update interval, and minimum number of removals.

6. The method according to claim 1, characterized in that, In S3, each keyframe records the 3D Gaussian points visible in the active submap through a bit set. The i-th bit of the bit set being 1 indicates that the i-th 3D Gaussian point in the active submap is located in front of the camera of the keyframe and falls within the preset view frustum of the keyframe. The co-view coefficient of keyframe A and keyframe B is represented by the Jaccard coefficient of the visible 3D Gaussian point set, that is, the co-view coefficient is the ratio of the number of intersection elements to the number of union elements of the visible 3D Gaussian point sets of the two keyframes.

7. The method according to claim 6, characterized in that, The management of candidate keyframes includes the following steps: when the co-view coefficient between a candidate keyframe and an existing keyframe is higher than a preset redundancy threshold, the candidate keyframe replaces the corresponding redundant keyframe; otherwise, when the positional distance and line-of-sight angle between a candidate keyframe and the nearest keyframe are both lower than the corresponding thresholds, insertion is skipped; in other cases, the candidate keyframe is inserted and the co-view relationship is updated incrementally; when the maximum co-view coefficient between the newly inserted keyframe and the existing keyframes in the window is lower than a preset scene switching threshold, the nearest preset number of keyframes are retained; when the keyframe window capacity exceeds the limit, keyframe marginalization is performed; when performing keyframe marginalization, the latest keyframe, the keyframe with the lowest average co-view coefficient, and the keyframe that has an exclusive observation contribution to the active sub-map are protected; for the remaining keyframes, a marginalization score is calculated based on redundancy, co-view coefficient with the latest keyframe, normalized dwell time, and coverage impact, and keyframes whose scores meet the preset marginalization conditions are removed.

8. The method according to claim 6, characterized in that, When selecting a training perspective from the set of keyframes, the basic sampling weight is determined based on the number of times each keyframe has been selected. The more times a keyframe has been selected, the smaller its basic sampling weight. Furthermore, the basic sampling weight is modified based on the co-visibility between the candidate keyframe and recently selected keyframes. The candidate keyframe with the lower co-visibility with recently selected keyframes has the larger modified weight. The modified weight is normalized to obtain the sampling probability of each keyframe, and the training perspective is selected according to the sampling probability.

9. The method according to claim 1, characterized in that, In S4, after each forward differentiable rendering, a set to be updated is determined based on whether the 3D Gaussian point makes a valid synthetic contribution to at least one pixel. Backpropagation and optimizer parameter updates are performed only on the 3D Gaussian points in the set to be updated, while the parameters of 3D Gaussian points that did not participate in the current rendering remain unchanged. The optimizer state includes the first-order moment state and the second-order moment state of the adaptive moment estimation optimizer. When a 3D Gaussian point is removed from the active submap, its optimizer state is persisted to the corresponding octagonal leaf node. When the 3D Gaussian point is reactivated, the optimizer state is restored from the corresponding octagonal leaf node to continue the optimization process of the 3D Gaussian point.

10. A 3D Gaussian simultaneous localization and mapping system based on a hash octree, characterized in that, include: The data synchronization and pose estimation module is used to synchronize the lidar point cloud, inertial measurement data and camera images in time, and the pose of the current frame is estimated by the multi-sensor positioning front end. The global map management module is used to construct a global hash octree map. On the main memory side, the coordinates of bold voxels are determined based on the center position of the three-dimensional Gaussian points. The bold voxels are managed by hash mapping, and fine-grained octrees are set up in each bold voxel to perform hierarchical spatial indexing and persistent storage of the global three-dimensional Gaussian points. The visibility sub-map module is used to determine the currently participating bold voxels based on the current frame pose, camera frustum, and the observation range of the co-view keyframe. It loads the 3D Gaussian points in the corresponding bold voxels from the global hash octree map to the active sub-map on the graphics processor side, and removes the 3D Gaussian points that meet the dormancy conditions from the active sub-map so that they continue to remain in the global hash octree map. The common-view keyframe management module is used to determine the set of visible 3D Gaussian points for candidate keyframes and existing keyframes in the window, calculate the common-view coefficient based on the intersection and union relationship between the sets of visible 3D Gaussian points, and perform keyframe insertion, replacement or marginalization based on the common-view coefficient and the pose difference between keyframes to form a set of keyframes for active submap optimization. The 3D Gaussian optimization and write-back module is used to select a training viewpoint from the keyframe set, perform differentiable rendering on the 3D Gaussian points in the active sub-map on the graphics processor side and calculate the loss, determine the 3D Gaussian points to be updated based on the effective contribution of the 3D Gaussian points to the rendering results, perform parameter updates on the 3D Gaussian points to be updated, and write the updated 3D Gaussian parameters and their corresponding optimizer states back to the corresponding leaf nodes in the global hash octree map on the main memory side.

Images