Lightweight SLAM construction method and system based on 3D Gaussian sputtering and adaptive clustering

Abstract

The invention relates to the technical field of SLAM construction, and provides a lightweight SLAM construction method and system based on 3D Gaussian sputtering and adaptive clustering, and the method comprises the steps: carrying out the feature matching and pose estimation of an image flow; selecting a key frame of the image stream according to a common-view condition, and inserting the key frame into a key frame window; optimizing a Gaussian map according to the key frame; and carrying out region clustering and updating optimization on the Gaussian map. The invention aims to solve the problems of high calculation complexity, insufficient real-time performance and large rendering overhead of the existing SLAM system in a large-scale dynamic scene, and the core technology comprises the following steps: (1) adopting plane and high-density feature point detection to realize fine processing of a key area; and (2) reducing data redundancy and optimizing resource management in combination with coarse and fine granularity adaptive clustering. The technology can be widely applied to the fields of automatic driving, robot navigation, augmented reality, smart cities and the like, and an innovative solution is provided for autonomous navigation, three-dimensional reconstruction and efficient rendering in a complex environment.

Description

Technical Field

[0001] The present invention relates to the technical field of SLAM construction, and in particular to a lightweight SLAM construction method and system based on 3D Gaussian sputtering and adaptive clustering. Background Art

[0002] With the rapid development of autonomous driving, robotic navigation, augmented reality (AR), and virtual reality (VR), 3D scene reconstruction and rendering technologies play a crucial role in Simultaneous Localization and Mapping (SLAM). SLAM systems construct a 3D map of the environment in real time, enabling devices or robots to accurately locate and navigate in dynamic environments. SLAM systems typically rely on a variety of scene representation methods, such as 3D point clouds, voxel grids, and mesh representations.

[0003] For SLAM systems based on 3D Gaussian sputtering, 3D Gaussian sputtering replaces the previous geometric grid and neural radiation field with a Gaussian distribution, reducing computational complexity while improving rendering efficiency. Although this method performs well in terms of rendering accuracy and efficiency, as scene complexity increases, existing 3D Gaussian sputtering frameworks still face the problem of high computational overhead when processing large-scale, unstructured point clouds. In particular, in application scenarios with real-time rendering and high precision requirements, the balance between system performance and computational efficiency remains a challenge. Summary of the Invention

[0004] Based on this, in order to solve the problems existing in the existing SLAM system based on 3D Gaussian sputtering, the present invention provides a lightweight SLAM construction method and system based on 3D Gaussian sputtering and adaptive clustering. It performs regional clustering and update optimization on the Gaussian map through key area detection and adaptive clustering modules, which can improve the efficiency and accuracy of the system. The specific technical solution is as follows:

[0005] A lightweight SLAM construction method based on 3D Gaussian sputtering and adaptive clustering includes the following steps:

[0006] Provide a tracking thread module, obtain the image stream, and perform feature matching and pose estimation on the image stream;

[0007] Providing a key frame management module to select key frames of the image stream according to the co-visibility condition and insert the key frames into the key frame window;

[0008] Provides a map-building thread module to optimize Gaussian maps based on key frames;

[0009] Provides a key area detection and adaptive clustering module to perform regional clustering and update optimization on Gaussian maps.

[0010] The lightweight SLAM construction method based on 3D Gaussian sputtering and adaptive clustering first obtains an image stream and performs feature matching and pose estimation on the image stream. Then, the key frames of the image stream are selected according to the common visibility condition and the key frames are inserted into the key frame window. The Gaussian map is optimized according to the key frames. Finally, the Gaussian map is regionally clustered and updated and optimized. This can improve the efficiency and accuracy of the system and solve the problem of the difficulty in balancing system performance and computational efficiency in the existing SLAM construction method based on 3D Gaussian sputtering.

[0011] Preferably, the specific method for performing regional clustering and updating optimization on the Gaussian map includes:

[0012] Plane detection is achieved through random sampling consistency algorithm, high-density feature point extraction is completed by combining KD tree search and density analysis, and key geometric areas containing planes and high-density feature points are constructed;

[0013] Plane and high-density features are regarded as key areas, and different clustering strategies are adopted between key areas and non-key areas according to the regional feature differences of point clouds;

[0014] After completing the clustering of key areas, the clustering results of the plane are fused with the clustering results of high-density feature points.

[0015] Preferably, the lightweight SLAM construction method further includes dividing the point cloud into key areas and non-key areas by analyzing the geometric features of the point cloud, and performing clustering optimization for different areas by using corresponding clustering methods.

[0016] Preferably, the method further comprises defining the color center point of each cluster as the average value of the colors of all points in the corresponding cluster.

[0017] Preferably, the specific method of selecting the key frame of the image stream according to the common visibility condition includes:

[0018] Obtain the degree of overlap of the Gaussian distribution between the two frames, and calculate the common visibility between the two frames based on the degree of overlap.

[0019] A lightweight SLAM construction system based on 3D Gaussian sputtering and adaptive clustering is used to implement the lightweight SLAM construction method, which includes:

[0020] The tracking thread module is used to obtain the image stream and perform feature matching and pose estimation on the image stream;

[0021] A key frame management module, configured to select key frames of an image stream according to a common visibility condition and insert the key frames into a key frame window;

[0022] Mapping thread module, used to optimize Gaussian maps based on key frames;

[0023] Key region detection and adaptive clustering module, used to perform regional clustering and update optimization on Gaussian maps.

[0024] Preferably, the key area detection and adaptive clustering module includes:

[0025] Feature point extraction unit, which is used to realize plane detection through random sampling consistency algorithm, extract high-density feature points by combining KD tree search and density analysis, and construct key geometric areas containing planes and high-density feature points;

[0026] A clustering strategy selection unit is used to regard plane and high-density features as key areas and adopt different clustering strategies between key areas and non-key areas according to the regional feature differences of the point cloud;

[0027] The clustering result fusion unit is used to fuse the clustering result of the plane with the clustering result of the high-density feature points after completing the clustering of the key areas.

[0028] Preferably, the lightweight SLAM construction system further includes a 3D Gaussian sputtering rendering framework optimization module, which includes:

[0029] Adaptive clustering unit, used to analyze the geometric features of the point cloud, divide the point cloud into key areas and non-key areas, and perform clustering optimization for different areas using corresponding clustering methods;

[0030] The cluster-based color update unit is used to define the color center point of each cluster as the average value of the colors of all points in the corresponding cluster.

[0031] Preferably, the key frame management module includes:

[0032] The common visibility acquisition unit is used to obtain the overlap degree of the Gaussian distribution between the two frames and calculate the common visibility between the two frames according to the overlap degree.

[0033] Preferably, the tracking thread module includes:

[0034] The adjustment unit is used to adjust the position and attitude of the camera by comparing the photometric information of the current image with the rendered image. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily drawn to scale, but rather the emphasis is placed on illustrating the principles of the embodiments. In different views, the same reference numerals designate corresponding parts.

[0036] Figure 1 This is a schematic diagram of the overall process of a lightweight SLAM construction method according to one embodiment of the present invention;

[0037] Figure 2 1 is a flow chart of a method for calculating the distance from a Gaussian ellipsoid to a plane in one embodiment of the present invention;

[0038] Figure 3 1 is a flow chart of a KD tree accelerated neighborhood search method according to an embodiment of the present invention;

[0039] Figure 4 is a schematic diagram of the fine-grained clustering principle in one embodiment of the present invention;

[0040] Figure 5 1 is a schematic diagram comparing the changes in mean and covariance matrices before and after Gaussian clustering in one embodiment of the present invention;

[0041] Figure 6 is a schematic diagram of a clustering-based Gaussian sputtering rendering model in one embodiment of the present invention;

[0042] Figure 7 1 is a schematic diagram of the overall structure of a lightweight SLAM construction system according to an embodiment of the present invention;

[0043] Figure 8 4 is a flow chart of a lightweight SLAM construction method in another embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with its embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0045] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly attached to the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only implementation methods.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0047] The "first" and "second" in the present invention do not represent specific quantities and orders, but are only used to distinguish names.

[0048] Before describing the specific embodiments of the present application, a brief introduction to the prior art is given.

[0049] The existing SLAM construction systems include the following three:

[0050] (1) Traditional SLAM systems: SLAM systems based on 3D point clouds and voxel grids have, to a certain extent, promoted the reconstruction of 3D scenes and environmental perception. However, these systems still have the following shortcomings: First, the representation of 3D point clouds usually produces high-density data, resulting in a huge consumption of computing and memory resources; second, the computational complexity of voxel grids and grid representations is high, especially when dealing with complex and dynamic scenes, making real-time performance a major challenge; third, fixed-resolution voxel grids lack sufficient flexibility and cannot cope with the efficient processing of complex geometric structures.

[0051] (2) Neural Radiance Field-Based SLAM Systems: 3D reconstruction methods based on neural radiance fields successfully improve the rendering accuracy of SLAM systems by learning the light and color distribution in the scene. While these methods excel in processing high-quality 3D reconstructions, they also face challenges with heavy computational burdens and insufficient real-time performance, especially in dynamic, unstructured scenes, where it is difficult to balance rendering speed and accuracy.

[0052] (3) SLAM system based on 3D Gaussian sputtering: 3D Gaussian sputtering uses Gaussian distribution to replace the previous geometric grid and neural radiation field, which reduces the computational complexity and improves the rendering efficiency.

[0053] SLAM systems based on 3D Gaussian sputtering use Gaussian distributions instead of traditional grids and voxel grids, significantly optimizing the processing performance of three-dimensional scenes by improving the rendering efficiency of point cloud data and reducing computational complexity. 3D Gaussian sputtering is widely used to represent three-dimensional point clouds in dynamic scenes, and to some extent, it solves the real-time issues of traditional point cloud representation and voxel grid methods. Although this method performs well in terms of rendering accuracy and efficiency, as scene complexity increases, the existing 3D Gaussian sputtering framework still faces the problem of high computational overhead when processing large-scale, unstructured point clouds. This is particularly challenging in applications where real-time rendering and high precision are required, and the balance between system performance and computational efficiency remains a challenge.

[0054] In addition, existing technologies still have the following obvious shortcomings when processing large-scale dynamic point cloud data under complex environments and real-time performance requirements:

[0055] ① High computational complexity: Many existing point cloud representation methods, such as voxel-based grids and traditional clustering-based techniques, are computationally intensive and lack sufficient dynamic adaptability. Especially in the case of large-scale point cloud data and dynamic environments, these methods require significant computational resources during rendering and processing, resulting in system response delays and an inability to meet real-time performance requirements.

[0056] ② Low clustering efficiency: Existing clustering methods are often inefficient, especially when dealing with complex or irregularly shaped data. Traditional clustering methods may perform excessive calculations on unimportant areas, resulting in an increased computational burden and an inability to effectively allocate computing resources and optimize performance.

[0057] ③ Lack of adaptability and flexibility: Existing methods typically use fixed clustering strategies and data representations, making it difficult to dynamically adjust to the characteristics of different regions of the point cloud. This is especially true in large-scale dynamic scenes, where the density and morphology of point cloud data change rapidly. Traditional methods cannot fully adapt to these changes, resulting in an imbalance between rendering quality and computational efficiency.

[0058] ④ High rendering overhead: Existing SLAM systems often face significant rendering overhead when rendering large-scale 3D scenes. This overhead not only affects the real-time performance of the system, but also affects the final rendering quality and map accuracy. Especially in scenes with high precision requirements, it is difficult to achieve both rendering efficiency and accuracy.

[0059] ⑤ Lack of effective resource management: Traditional SLAM methods lack effective resource management mechanisms in key frame management, map optimization, etc., especially when dealing with large-scale dynamic scenes. How to efficiently process large amounts of data and reduce computing resource consumption remains a difficulty.

[0060] In order to solve the problem of the existing SLAM construction method based on 3D Gaussian sputtering that it is difficult to balance system performance and computational efficiency, an embodiment of the present invention provides a lightweight SLAM construction method based on 3D Gaussian sputtering and adaptive clustering, such as Figure 1 As shown, the following steps are included:

[0061] S1 provides a tracking thread module that obtains image streams and performs feature matching and pose estimation on the image streams.

[0062] Specifically, in the tracking thread, the core goal of camera pose optimization is to adjust the camera's position and pose by comparing the photometric information of the current image with the rendered image. In this lightweight SLAM framework, we introduce a region weighting strategy to adjust the weights of different regions, giving more attention to key areas during the optimization process.

[0063] Photometric residual E photo It can be expressed in a weighted way as: Among them, the photometric residual E photo Represents the difference between the photometric information of the current image and the rendered image. r The weight of region r is used to adjust the importance of different regions in the optimization process through the regional weighting strategy. r (G,T CW ) through the current camera pose T CW The luminosity value of the rendered image in region r. r Represents the luminosity value of the actual observed image in region r.

[0064] Geometric residual E geo It is used to measure the error between the depth map rendered from the current camera pose and the actual depth map. In order to incorporate the importance of the region, we define the weighted geometric residual as:

[0065] Among them, the geometric residual E geo It is used to measure the error between the depth map rendered from the current camera pose and the actual depth map, D r (G,T CW ) is obtained by the current posture T CW The rendered depth map, It is an actual deep observation.

[0066] The final optimization objective function combines the photometric and geometric residuals:

[0067] Among them, λ photo is a hyperparameter that controls the balance between photometric and geometric residuals, and They represent the photometric residual and geometric residual of region r respectively. photo The hyperparameter that controls the balance between photometric and geometric residuals is used to adjust the relative importance of the two in the optimization process, E total Represents the final optimization objective function, which combines the photometric residual and the geometric residual.

[0068] S2, providing a key frame management module, selecting key frames of the image stream according to the co-visibility condition, and inserting the key frames into the key frame window.

[0069] Preferably, the specific method of selecting the key frame of the image stream according to the common visibility condition includes: obtaining the overlap degree of Gaussian distribution between two frames, and calculating the common visibility between the two frames according to the overlap degree.

[0070] Specifically, in order to reduce redundant keyframes and improve the real-time performance of the system, this lightweight SLAM framework adopts a dynamic keyframe management method based on key area detection. The insertion of keyframes is achieved by calculating the common visibility Cov between two frames. ijTo judge, it is defined as the degree of overlap of the Gaussian distribution between two frames: Among them, Cov ij Indicates the co-visibility between two frames (assuming they are the i-th frame and the j-th frame), which is used to measure the degree of overlap of the Gaussian distribution of the two frames and reflects the correlation between the two frames in scene observation. i , α j are the weight coefficients of Gaussian distribution in the i-th frame and the j-th frame, G i ∩G j It represents the intersection area of the Gaussian distribution in the i-th frame and the j-th frame, that is, the part of the scene observed in both frames, and the summation operation is for the Gaussian distribution component in the intersection.

[0071] To improve system stability, the framework prunes the newly inserted Gaussian distribution, especially in the monocular case when the Gaussian position is inaccurate. The pruning condition can be mathematically expressed as: Visible(G k )=|{i|G k is visible in F i}|≥M. Among them, G k represents a Gaussian point, F i Represents the i-th frame image, |{i|G k is visible in F i}| represents the statistical Gaussian point G k In all frames {F i The number of frames visible in} is represented by the set count. M is the preset threshold, which represents the Gaussian point G k The minimum number of visible frames that must be met. If the number of visible frames of a Gaussian point reaches or exceeds M, it is retained; otherwise, it is removed to prevent invalid Gaussian points from interfering with the optimization process.

[0072] If this condition is met, the Gaussian point will be retained in the keyframe window; otherwise, the Gaussian point will be removed. This pruning operation effectively avoids the interference of invalid Gaussian points on the optimization process.

[0073] S3 provides a mapping thread module that optimizes Gaussian maps based on key frames.

[0074] Preferably, in the map optimization stage, the Gaussian distribution of the point cloud is adjusted by global optimization to ensure the global consistency of the map. To prevent excessive stretching in the depth direction, this lightweight SLAM framework introduces an isotropic regularization term E iso , which suppresses depth distortion in the scene by constraining the scale consistency of each Gaussian. The isotropic regularization term is calculated as It is used to penalize anisotropy (i.e., scale differences in different directions) and encourage the scale s of each Gaussian distribution to be i As close to its mean as possible This enhances the isotropic properties and improves the stability and accuracy of 3D reconstruction. Where G represents a set of Gaussian distributions, which is mainly used to optimize the map by adjusting these Gaussian distributions during the map optimization phase. i Represents the scale vector of the i-th Gaussian distribution. Its main function is to measure the scale characteristics of each Gaussian in the calculation of the isotropic regularization term to ensure its consistency. i represents the index of traversing each Gaussian distribution. is the average scale of the Gaussian distribution, which is calculated by taking s i The mean of the components in each direction ensures uniform scaling of the Gaussian distribution. 1 provides a baseline for isotropic scaling, where all components are equal to 1, resulting in an isotropic reference scale vector. ||1 is the L1 norm, which calculates the sum of the absolute values of the original scale and the uniform scale. Using the L1 norm promotes sparse scaling, prioritizing the reduction of maximum scale differences and making the Gaussian distribution more spherical, further improving rendering stability and consistency.

[0075] The objective function of map optimization combines the photometric residual, geometric residual and isotropic regularization term, which is expressed as

[0076] Where W represents the current key frame window, λ geo and λ iso is a hyperparameter that balances the geometric error and the regularization term, specifically the hyperparameter of the geometric residual and the isotropic regularization term. Photometric and geometric errors are handled by the same weighting strategy, where and are the photometric and geometric residuals of key frame k, respectively. Represents the transformation T of the keyframe window CW And the Gaussian distribution G is minimized.

[0077] The variables in cooperate with each other during the map optimization process and can adjust parameters such as Gaussian distribution and camera pose by minimizing the objective function, thereby improving the global consistency and accuracy of the map.

[0078] S4 provides a key area detection and adaptive clustering module to perform regional clustering and update optimization on the Gaussian map.

[0079] Preferably, in step S4, as Figure 8 As shown in Figure 2, the specific methods for regional clustering and updating optimization of Gaussian maps include:

[0080] S41, plane detection is achieved through random sampling consistency algorithm, high-density feature point extraction is completed by combining KD tree search and density analysis, and key geometric areas containing planes and high-density feature points are constructed.

[0081] Specifically, in a Gaussian ellipsoid cloud, the random sampling consensus algorithm identifies the maximum point set that conforms to the plane model through random sampling, and its mathematical model is: Ax+By+Cz+D=0.

[0082] Where (x, y, z) are the mean coordinates of the three-dimensional Gaussian ellipsoid, and (A, B, C, D) are the plane parameters to be estimated.

[0083] The specific process includes: randomly selecting three non-collinear mean points from the Gaussian ellipsoid cloud to construct a candidate plane, and calculating the distance from each point to the plane

[0084] If d is less than a preset threshold, the point is considered an inlier point in the plane. After multiple iterations, the plane model containing the largest number of inliers is finally selected as the optimal solution.

[0085] Figure 2 This section shows the calculation of the distance from a plane to a Gaussian ellipsoid. By calculating the plane formula and the point-to-plane distance formula, the distance value of each Gaussian ellipsoid can be obtained.

[0086] High-density feature points are detected by combining KD tree (K-Dimension tree) to accelerate neighborhood search and density analysis. KD tree is an efficient space partitioning data structure used to quickly find the query point p i Neighborhood point set within radius r Among them, p j Represents any point in the data set, used to compare with the query point p i Perform distance comparison to determine whether it belongs to p i Neighborhood points, d(p i ,p j ) represents point p i and point p j The distance between them.

[0087] Figure 3 The left side shows the application of KD tree in high-density feature point detection. The figure effectively screens high-density points through spatial division of the point cloud. Figure 3 The right side shows the binary tree construction method of the KD tree, which constructs a binary tree by dividing the point cloud space layer by layer, thus providing theoretical support for neighborhood search.

[0088] By utilizing the idea based on random sampling consistency, planar features are accurately extracted from Gaussian point clouds, and the density analysis method accelerated by spatial division is combined to identify high-density areas. While improving detection accuracy, the device significantly improves computational efficiency and provides effective region segmentation for subsequent point cloud clustering and feature matching.

[0089] S42 regards plane and high-density features as key areas, and adopts different clustering strategies between key areas and non-key areas according to the regional feature differences of point clouds.

[0090] Specifically, this embodiment proposes an adaptive clustering optimization method based on key regions for the detection results of planes and high-density feature points. This adaptive clustering optimization method considers the detected planes and high-density features as key regions and adopts different clustering strategies between key and non-key regions based on the regional feature differences of the point cloud to improve clustering accuracy and computational efficiency.

[0091] For key areas, this embodiment adopts a density-based fine-grained clustering algorithm. The algorithm determines the clustering affiliation of three-dimensional Gaussian points by density, and classifies neighboring points with sufficiently high density into the same category, which can effectively meet the clustering needs of irregularly shaped data.

[0092] Figure 4 A diagram showing the principle of fine-grained clustering. Figure 4 A schematic diagram of the fine-grained clustering principle is shown in Figure 1. In the figure, the core point (red) points to the points in its neighborhood through arrows, the boundary points (yellow) are in the neighborhood of the core point but do not meet the conditions to become a core point, and the noise points (blue) are outside the point cloud.

[0093] For non-critical areas, this embodiment uses a coarse-grained clustering method with low computational complexity. The coarse-grained clustering algorithm performs clustering by minimizing the square distance between each point in a cluster and the cluster center, thereby effectively allocating Gaussian points to each cluster.

[0094] S43, after completing the clustering of the key areas, the clustering result of the plane is fused with the clustering result of the high-density feature points.

[0095] Specifically, after completing the clustering of the key areas, this embodiment fuses the clustering results of the plane with the clustering results of the high-density feature points to improve the integrity and effectiveness of the scene geometric features. The fusion strategy is expressed as: R = C plane ∪C corner Among them, R represents the intermediate result after fusion, which is obtained by clustering the plane result C plane And high-density feature point clustering result C corner It is obtained by performing a certain fusion operation (indicated by the symbol “∪” in the figure).

[0096] C plane Represents the clustering result of the plane area, that is, the result obtained by clustering analysis of the plane area. corner Represents the clustering result of high-density feature points, that is, the result obtained by clustering analysis of high-density feature points.

[0097] Finally, the clustering results of key areas and non-key areas are merged to obtain the complete clustering result: C final =R∪C non-key .

[0098] Among them, C non-key represents the clustering result of non-critical areas, and C final This is the final clustering result after fusion.

[0099] Fine-grained and coarse-grained clustering strategies are adopted for different regions, and dynamic adjustments are made according to the regional characteristics of the point cloud (such as planes and high-density feature points). This method uses a fine clustering strategy when processing key areas to ensure high-fidelity representation of key areas, while coarse-grained clustering is used for non-critical areas to reduce computational overhead.

[0100] In summary, the lightweight SLAM construction method based on 3D Gaussian sputtering and adaptive clustering first obtains an image stream and performs feature matching and pose estimation on the image stream, then selects the key frame of the image stream according to the common visibility condition, and inserts the key frame into the key frame window, then optimizes the Gaussian map according to the key frame, and finally performs regional clustering and update optimization on the Gaussian map, which can improve the efficiency and accuracy of the system and solve the problem of being difficult to balance system performance and computational efficiency in the existing SLAM construction method based on 3D Gaussian sputtering. In addition, the lightweight SLAM construction method comprehensively considers multiple factors such as regional weight, key frame management, map optimization and global consistency, while ensuring high precision, significantly improving the real-time and robustness of the system, and is particularly suitable for real-time SLAM applications in large-scale dynamic scenes.

[0101] As a preferred technical solution, the lightweight SLAM construction method also includes dividing the point cloud into key areas and non-key areas by analyzing the geometric features of the point cloud, and performing clustering optimization for different areas using corresponding clustering methods.

[0102] Specifically, in the GS model, the calculation formula for Gaussian distribution is: It represents the geometric position, shape and color information of the point.

[0103] Among them, (x, y, z) represents the geometric position coordinates of a point in three-dimensional space, which is used to describe the specific position information of the point in space, μ i Is a three-dimensional mean vector, representing the center position of the i-th Gaussian distribution, that is, the average position of the local area of the point cloud described by the Gaussian distribution. iIt is a 3×3 covariance matrix used to describe the shape of the i-th Gaussian distribution and the correlation between the dimensions. It describes the distribution discreteness and directional characteristics of the point cloud in the local area. i ,Σ i ) indicates that at a given position (x, y, z), the probability density value based on the i-th Gaussian distribution reflects that the point belongs to the μ i Center, Σ i is the probability of the Gaussian distribution of the covariance characteristic. x is a three-dimensional vector, that is, x = [x, y, z] T , which represents a point in space. It is essentially the same concept as the previous (x, y, z), but is expressed in vector form to facilitate matrix operations.

[0104] In order to further improve rendering efficiency and reduce redundant data, this embodiment proposes an adaptive clustering mechanism to optimize the expressiveness of the GS model. This mechanism first divides the point cloud into key areas and non-key areas by analyzing the geometric features of the point cloud, and applies corresponding clustering methods to different areas. The calculation formula of the mean and covariance matrix of each cluster is as well as

[0105] Among them, μ′ i Represents the mean vector of the i-th cluster, which is the center point of the cluster and integrates the position information of all Gaussian points in the cluster. i is the number of points contained in the cluster, which is used to normalize the mean vector within the cluster.

[0106] C i Represents the set of Gaussian points in the i-th cluster, that is, the set of all Gaussian points belonging to the cluster. The calculated mean μ′ i As the center point of the cluster, the covariance matrix Σ′ i It integrates the geometric characteristics of the region, reflects the discrete distribution of points within the cluster and the correlation between dimensions, thereby simplifying the overall representation of the point cloud. j Represents the set C i The mean vector of the j-th Gaussian point is the center position of a single Gaussian distribution.

[0107] Σ j Represents the set C i The covariance matrix of the jth Gaussian point in describes the shape and correlation characteristics of a single Gaussian distribution. Cov({μj|j∈Ci}) represents the set C iIt takes into account the changes and correlations between the centers of different Gaussian distributions within the cluster, and further improves the calculation of the cluster covariance matrix to better represent the geometric characteristics of the entire cluster area.

[0108] Figure 5 The figure shows the changes in the mean and covariance matrix of the Gaussian point cloud before and after clustering. Before clustering, the mean and covariance matrix of the point cloud are calculated independently, and each Gaussian point uses its own mean and covariance matrix ( Figure 5 Left). After clustering, all Gaussian point clouds are aggregated into clusters, and new mean and covariance matrices are calculated, making the representation of each cluster more simplified and compact ( Figure 5 (right). This clustering mechanism not only reduces point cloud redundancy but also improves rendering efficiency, thereby optimizing the overall rendering process.

[0109] The lightweight SLAM construction method further includes defining the color center point of each cluster as the average value of the colors of all points in the corresponding cluster.

[0110] Specifically, in addition to clustering and optimizing the geometric features of the Gaussian point cloud, this embodiment also proposes a color update mechanism based on clustering to further improve the efficiency and quality of the rendering process. i The color center point It is defined as the average value of the colors of all points in the cluster, and the calculation formula is:

[0111] Among them, N i Represents cluster C i The number of midpoints, c j is the color of each point in the cluster, that is, cluster C i The color of the jth point in the cluster C. i Point in, j∈C i Indicates that j is cluster C i A point in the.

[0112] Based on this mechanism, the final color accumulation formula can be simplified by using the color of the cluster center point as follows: Where C represents the final color accumulation result, and N represents the total number of clusters. i is used to index different clusters, and the value range of i is from 1 to N. i represents the transmission parameter associated with the i-th cluster. i Denotes the density parameter associated with the i-th cluster. i Represents the distance parameter associated with the i-th cluster.

[0113] like Figure 6As shown in Figure 2, this optimization method makes color data updates more efficient during the rendering process, reducing computational complexity and improving rendering efficiency. Furthermore, the clustering-based color update mechanism significantly reduces the impact of redundant data on system performance while preserving the geometric and lighting details of the Gaussian point cloud.

[0114] An embodiment of the present invention further provides a lightweight SLAM construction system based on 3D Gaussian sputtering and adaptive clustering, which is used to implement the lightweight SLAM construction method. Figure 7 As shown in the figure, it includes a tracking thread module, a key frame management module, a mapping thread module, and a key area detection and adaptive clustering module.

[0115] The tracking thread module is used to obtain the image stream and perform feature matching and pose estimation on the image stream; preferably, the tracking thread module includes: an adjustment unit, which is used to adjust the position and pose of the camera by comparing the photometric information of the current image with the rendered image.

[0116] In the tracking thread, the core goal of camera pose optimization is to adjust the camera's position and pose by comparing the photometric information of the current image with the rendered image. The lightweight SLAM construction system introduces a region weighting strategy by building a lightweight SLAM framework, adjusting the weights of different regions to give more attention to key areas during the optimization process.

[0117] The key frame management module is used to select key frames of the image stream according to the common visibility condition and insert the key frames into the key frame window. Preferably, the key frame management module includes: a common visibility acquisition unit, which is used to obtain the overlap degree of the Gaussian distribution between two frames and calculate the common visibility between the two frames based on the overlap degree.

[0118] The mapping thread module is used to optimize the Gaussian map based on key frames; the key area detection and adaptive clustering module is used to perform regional clustering and update optimization on the Gaussian map.

[0119] Preferably, the key area detection and adaptive clustering module includes a feature point extraction unit, a clustering strategy selection unit and a clustering result fusion unit.

[0120] The feature point extraction unit is used to realize plane detection through random sampling consistency algorithm, complete high-density feature point extraction by combining KD tree search and density analysis, and construct key geometric areas containing planes and high-density feature points; the clustering strategy selection unit is used to regard planes and high-density features as key areas, and adopt different clustering strategies between key areas and non-key areas according to the regional feature differences of the point cloud; the clustering result fusion unit is used to fuse the clustering results of the plane with the clustering results of the high-density feature points after completing the clustering of the key areas.

[0121] Specifically, for key areas, this embodiment uses a density-based fine-grained clustering algorithm. The algorithm determines the clustering of three-dimensional Gaussian points by density, classifying neighboring points with sufficiently high density into the same category. This effectively addresses the clustering needs of irregularly shaped data. For non-key areas, this embodiment uses a coarse-grained clustering method with lower computational complexity. The coarse-grained clustering algorithm clusters by minimizing the squared distance from each point within a cluster to the cluster center, effectively assigning Gaussian points to clusters.

[0122] The key area detection and adaptive clustering module can reduce the computational burden and improve clustering efficiency by performing fine-grained processing on key areas (such as planes and high-density feature points) and combining coarse-grained and fine-grained adaptive clustering methods, thereby ensuring high-fidelity representation of key parts in the scene, while reducing computational resource consumption through coarse-grained clustering of non-critical parts.

[0123] Preferably, the lightweight SLAM construction system further includes a 3D Gaussian sputtering rendering framework optimization module, and the 3D Gaussian sputtering rendering framework optimization module includes an adaptive clustering unit and a clustering-based color update unit.

[0124] The adaptive clustering unit is used to analyze the geometric features of the point cloud, divide the point cloud into key areas and non-key areas, and perform clustering optimization for different areas using corresponding clustering methods; the clustering-based color update unit is used to define the color center point of each cluster as the average color of all points in the corresponding cluster.

[0125] Specifically, the 3D Gaussian sputtering rendering framework optimization module can be understood as a 3D Gaussian sputtering rendering framework optimization module based on dual geometry-color clustering. Building on traditional clustering methods, the module combines geometry and color information, updates the Gaussian model's geometric features, such as the mean and covariance, using clustering results, and optimizes the color representation of the Gaussian point cloud. This dual clustering optimization of the rendering framework effectively reduces data redundancy, improves rendering efficiency and accuracy, and reduces data redundancy, enabling the system to more efficiently handle large-scale dynamic scenes.

[0126] For the above point cloud clustering, it can also be achieved by the following methods:

[0127] 1. Deep Learning-Based Point Cloud Clustering: This method automatically clusters point clouds by training a neural network model, particularly for point cloud data with diverse features in complex scenes. By automatically identifying key areas and clustering them, the trained model can reduce the computational burden of traditional clustering algorithms. This method does not rely on explicit geometric feature extraction and exhibits strong adaptability.

[0128] 2. Point cloud clustering based on graph neural networks (GNNs): Graph neural networks can fuse local and global information by building a graph structure for point clouds, taking into account the relationships between point clouds during clustering. This approach can improve the accuracy of point cloud clustering, especially when dealing with irregular shapes and complex structures, and can propagate global information over a wider range.

[0129] 3. Point cloud clustering based on the traditional multi-scale voxel grid method: By processing point cloud data using voxel grids of different scales, point clouds can be clustered and reconstructed at different resolutions, thereby reducing the computational burden and improving the real-time performance of the system. Its implementation is relatively simple and suitable for certain static scenes.

[0130] In summary, the present invention has the following beneficial effects:

[0131] 1. An optimization method combining high-density region detection and adaptive clustering: This method significantly improves point cloud processing efficiency by clustering key regions with fine granularity and non-critical regions with coarse granularity. This method not only reduces unnecessary computational burden while maintaining high rendering accuracy, but also effectively improves the real-time processing capabilities of large-scale dynamic scenes.

[0132] 2. Based on 3D Gaussian sputtering, a dual clustering optimization scheme combining geometry and color is used to further reduce data redundancy and improve rendering performance, especially in complex and dynamic scenes, where rendering efficiency is significantly improved.

[0133] 3. Lightweight SLAM system based on regional weight and keyframe management: It comprehensively considers multiple factors such as regional weight, keyframe management, map optimization and global consistency, and adjusts regional weight and keyframe management strategies through optimization functions to ensure high-precision three-dimensional scene reconstruction while improving the real-time performance of the system. It can achieve more efficient and robust real-time positioning and map construction, and adapt to real-time application requirements in large-scale environments.

[0134] 4. Balance between high precision and computational efficiency: Through improved clustering strategies, rendering optimization methods, and lightweight design, the system significantly improves computational efficiency and real-time performance while ensuring high rendering accuracy. This makes it particularly suitable for real-time SLAM tasks in large-scale dynamic scenes.

[0135] 5. The present invention can be applied to the field of autonomous driving, supporting environmental perception and map construction, and improving autonomous navigation capabilities in complex environments; applied to the robotics industry, this technology is suitable for service, industrial, and exploratory robots, especially for task execution in dynamic and unknown environments, such as logistics and distribution; applied to virtual reality (VR) and augmented reality (AR), this technology can be used for three-dimensional scene reconstruction and real-time rendering, enhancing virtual experience and remote collaboration; applied to the field of smart cities and infrastructure, this technology can be used for urban modeling and infrastructure monitoring, optimizing urban planning and management; or applied to scientific research institutions and academic units, especially in research fields such as SLAM (simultaneous localization and mapping), three-dimensional reconstruction, and computer vision, and can promote technological innovation and algorithm optimization through this technology.

[0136] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A lightweight SLAM construction method based on 3D Gaussian sputtering and adaptive clustering, characterized in that: The method comprises the following steps: Provide a tracking thread module, obtain the image stream, and perform feature matching and pose estimation on the image stream; Providing a key frame management module to select key frames of the image stream according to the co-visibility condition and insert the key frames into the key frame window; Provides a map-building thread module to optimize Gaussian maps based on key frames; Provides a key area detection and adaptive clustering module to perform regional clustering and update optimization on Gaussian maps.

2. The lightweight SLAM construction method based on 3D Gaussian sputtering and adaptive clustering as claimed in claim 1, characterized in that The specific methods for regional clustering and updating optimization of Gaussian maps include: Plane detection is achieved through random sampling consistency algorithm, high-density feature point extraction is completed by combining KD tree search and density analysis, and key geometric areas containing planes and high-density feature points are constructed; Plane and high-density features are regarded as key areas, and different clustering strategies are adopted between key areas and non-key areas according to the regional feature differences of point clouds; After completing the clustering of key areas, the clustering results of the plane are fused with the clustering results of high-density feature points.

3. The lightweight SLAM construction method based on 3D Gaussian sputtering and adaptive clustering as claimed in claim 2, characterized in that The lightweight SLAM construction method also includes dividing the point cloud into key areas and non-key areas by analyzing the geometric features of the point cloud, and performing clustering optimization for different areas using corresponding clustering methods.

4. The lightweight SLAM construction method based on 3D Gaussian sputtering and adaptive clustering as claimed in claim 3, characterized in that The lightweight SLAM construction method further includes defining the color center point of each cluster as the average value of the colors of all points in the corresponding cluster.

5. The lightweight SLAM construction method based on 3D Gaussian sputtering and adaptive clustering as claimed in claim 4, characterized in that The specific method of selecting the key frame of the image stream according to the common visibility condition includes: Obtain the degree of overlap of the Gaussian distribution between the two frames, and calculate the common visibility between the two frames based on the degree of overlap.

6. A lightweight SLAM construction system based on 3D Gaussian sputtering and adaptive clustering, for implementing the lightweight SLAM construction method according to any one of claims 1 to 5, characterized in that: The lightweight SLAM construction system includes: The tracking thread module is used to obtain the image stream and perform feature matching and pose estimation on the image stream; A key frame management module, configured to select key frames of an image stream according to a common visibility condition and insert the key frames into a key frame window; Mapping thread module, used to optimize Gaussian maps based on key frames; Key region detection and adaptive clustering module, used to perform regional clustering and update optimization on Gaussian maps.

7. The lightweight SLAM construction system based on 3D Gaussian sputtering and adaptive clustering as claimed in claim 6, characterized in that The key area detection and adaptive clustering module includes: Feature point extraction unit, which is used to realize plane detection through random sampling consistency algorithm, extract high-density feature points by combining KD tree search and density analysis, and construct key geometric areas containing planes and high-density feature points; A clustering strategy selection unit is used to regard plane and high-density features as key areas and adopt different clustering strategies between key areas and non-key areas according to the regional feature differences of the point cloud; The clustering result fusion unit is used to fuse the clustering result of the plane with the clustering result of the high-density feature points after completing the clustering of the key areas.

8. The lightweight SLAM construction system based on 3D Gaussian sputtering and adaptive clustering as claimed in claim 7, characterized in that The lightweight SLAM construction system further includes a 3D Gaussian sputtering rendering framework optimization module, which includes: Adaptive clustering unit, used to analyze the geometric features of the point cloud, divide the point cloud into key areas and non-key areas, and perform clustering optimization for different areas using corresponding clustering methods; The cluster-based color update unit is used to define the color center point of each cluster as the average value of the colors of all points in the corresponding cluster.

9. The lightweight SLAM construction system based on 3D Gaussian sputtering and adaptive clustering as claimed in claim 8, characterized in that The keyframe management module includes: The common visibility acquisition unit is used to obtain the overlap degree of the Gaussian distribution between the two frames and calculate the common visibility between the two frames according to the overlap degree.

10. The lightweight SLAM construction system based on 3D Gaussian sputtering and adaptive clustering according to claim 9, characterized in that: The trace thread module includes: The adjustment unit is used to adjust the position and attitude of the camera by comparing the photometric information of the current image with the rendered image.

Images