3D Gaussian sputtering compression method and system based on residual quantization and dynamic pruning

Abstract

The invention belongs to the technical field of computer graphics and real-time 3D scene reconstruction, and relates to a real-time compression and optimization method based on a 3D Gaussian sputtering (3DGS) model, which realizes efficient storage, transmission and rendering of a three-dimensional scene through residual vector quantization (RVQ) and a dynamic pruning technology, and comprises the following steps: S1, residual quantization vector quantization; s2, enhancing a dynamic pruning strategy; and S3, index compression and coding optimization. The invention provides a real-time 3D Gaussian sputtering (3D Gaussian sputtering) model compression system combining residual vector quantization and dynamic pruning, which can realize high-quality real-time rendering and extremely high storage efficiency. Through three core technologies of a vector quantization strategy, dynamic pruning strategy enhancement and index compression and coding optimization, on the premise that the rendering quality is guaranteed, the purposes that the model storage amount is reduced by more than 45 times, and the rendering speed is increased by 3 times are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer graphics and real-time 3D scene reconstruction, and relates to a real-time compression and optimization method based on the 3D Gaussian Splatting (3DGS) model. Through residual vector quantization (RVQ) and dynamic pruning techniques, efficient storage, transmission, and rendering of three-dimensional scenes are achieved. Background Art

[0002] Although the original 3D Gaussian Splatting (3DGS) technology has shown significant advantages in the field of real-time rendering, it faces multiple technical bottlenecks in practical applications: In terms of storage efficiency, the parameter scale of a single scene is relatively high (for example, the storage of the truck scene in the Tanks & Temples dataset reaches 4.2GB), far exceeding the carrying capacity of low-end devices. As a result, when actually deployed, the resolution needs to be compressed from 4K to 720p to adapt to the hardware, causing serious waste of storage space and network transmission bandwidth; At the quantization method level, existing solutions use 32-bit floating-point storage with full precision. Experiments show that the opacity parameter only requires 4-bit quantization to meet the perceptual requirements (PSNR loss < 0.3dB), while forced 8-bit uniform quantization will lead to the loss of high-frequency geometric features (such as the SSIM value of building edges dropping to 0.82). Practical applications are limited. In the cloud rendering scenario, the loading delay of a 1GB model exceeds 8 seconds (100Mbps bandwidth), unable to meet the real-time interaction requirements such as AR navigation, and there is a lack of a compressive sensing mechanism for parameter updates, with the dynamic editing delay ratio exceeding 40%. These defects severely restrict the implementation and application of 3DGS technology in scenarios such as mobile devices, XR devices, and real-time streaming media.

[0003] In view of the above analysis, the technical problems that urgently need to be solved in the existing technology are as follows:

[0004] (1) In terms of storage efficiency, the parameter scale of a single scene is relatively high (for example, the storage of the truck scene in the Tanks & Temples dataset reaches 4.2GB), far exceeding the carrying capacity of low-end devices. As a result, when actually deployed, the resolution needs to be compressed from 4K to 720p to adapt to the hardware, causing serious waste of storage space and network transmission bandwidth;

[0005] (2) At the quantization method level, existing solutions use 32-bit floating-point storage with full precision. Experiments show that the opacity parameter only requires 4-bit quantization to meet the perceptual requirements (PSNR loss < 0.3dB), while forced 8-bit uniform quantization will lead to the loss of high-frequency geometric features (such as the SSIM value of building edges dropping to 0.82). Practical applications are limited. In the cloud rendering scenario, the loading delay of a 1GB model exceeds 8 seconds (100Mbps bandwidth), unable to meet the real-time interaction requirements such as AR navigation, and there is a lack of a compressive sensing mechanism for parameter updates, with the dynamic editing delay ratio exceeding 40%. Summary of the Invention

[0006] In response to the problems existing in the prior art, the present invention provides a 3D Gaussian sputtering compression method and system based on residual quantization and dynamic pruning.

[0007] The present invention is implemented as follows: a 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning, comprising:

[0008] S1, residual quantization vector quantization;

[0009] S2, dynamic pruning strategy enhancement;

[0010] S3, index compression and encoding optimization.

[0011] S1 specifically includes:

[0012] 1. Parameter grouping and dynamic codebook allocation:

[0013] Input: original Gaussian parameters (position, color SH coefficient, covariance rotation / scale, opacity)

[0014] Parameter grouping: The color group is split into DC component (1 dimension) and SH coefficient (15 dimensions). The SH coefficient is further divided into low frequency (1-3 orders) and high frequency (4-15 orders). The covariance group is split into rotation quaternion (4 dimensions) and logarithmic scale (3 dimensions).

[0015] 2. Information entropy calculation: Calculate information entropy for each set of parameters:

[0016]

[0017] Parameter meaning:

[0018] x: Parameters to be compressed (such as color SH coefficients, covariance components, etc.).

[0019] p: The probability of a parameter value appearing in the dataset.

[0020] 3. Dynamic allocation of codebook:

[0021]

[0022] K total is the total codebook capacity (e.g. 32K), K g is the codebook size of the gth group

[0023] 4. Residual Quantization (RVQ) and Mixed Precision Training:

[0024] For each set of parameters, a low-bit codebook (4 bits, 16 centers) is used for initial quantization:

[0025]

[0026] Residual calculation and secondary quantization:

[0027]

[0028] Where r (residual) is an 8-bit codebook (256 centers)

[0029]

[0030] Core idea: During the training process, quantizers of different precisions are jointly optimized, and the soft and hard quantization weights are controlled through temperature annealing to gradually converge to the optimal quantization result.

[0031] Temperature annealing weight formula

[0032]

[0033] E hard : Hard quantization (direct rounding) error

[0034] E soft : Error of soft quantization (retaining floating point residual)

[0035] T: Temperature parameter, with an initial high value (such as 1.0) and gradually decreasing to close to 0 (such as 0.1).

[0036] 5. Output:

[0037] Quantization parameters:

[0038] Codebook data: {C coarse ,C residual}

[0039] S2 specifically includes:

[0040] 1. Energy function calculation and threshold setting:

[0041] Input: Gaussian parameters in training and its multi-view visibility records.

[0042] Multi-factor energy calculation:

[0043]

[0044] σ: opacity;

[0045] x i : distance from Gaussian to scene origin;

[0046] N visible : The number of times the Gaussian is visible in the training view

[0047] Dynamic threshold adjustment: τ = μ E -k·σ E

[0048] μ E is the average energy of all current Gaussians, and σ E is the standard deviation.

[0049] 2. Soft Pruning and Recovery Mechanism:

[0050] Dormancy Flag: If E i < τ, mark Gaussian i as dormant (set σ i = 0), retain the parameters but do not participate in rendering.

[0051] Region Loss Monitoring: Divide the scene into a 16×16×16 voxel grid, and count the rendering loss L_region of dormant Gaussians in each voxel. If the rendering loss L_region > η in M consecutive iterations, then wake up the dormant Gaussians in this region; trigger the recovery mechanism. By default, M = 10 and η = 0.05 (adjustable).

[0052] Gaussian Recovery: Wake up the dormant Gaussians in this voxel, reset σi to the initial value, and increase its energy weight α to 0.8 (to prevent being pruned again)

[0053] 3. Output: Pruned Gaussian Set: Only retain active (σ i > 0) and recovered Gaussians.

[0054] Dormancy Pool: Store the parameters of dormant Gaussians for subsequent recovery.

[0055] S3 specifically includes:

[0056] 1. Morton Code Sorting and Differential Encoding

[0057] Input: Quantized Gaussian indices (integer sequence).

[0058] Coordinate Normalization and Morton Code Calculation:

[0059] Normalize the Gaussian center coordinates [x, y, z] to the integer range [, 2^16 - 1], and calculate the three-dimensional Morton code.

[0060] Index Sorting: Rearrange the indices in ascending order according to the Morton code to generate an ordered sequence {I1, I2,..., IN}.

[0061] Differential Calculation:

[0062]

[0063] 2. Entropy Encoding and Compression

[0064] Huffman Code Table Generation:

[0065] Statistical frequency distribution of δ i , and assign short codewords to high-frequency values (such as δ ≤ 5)

[0066] Binary compression:

[0067] Convert the difference sequence into a Huffman - coded bitstream and store it in big - endian order.

[0068] 3. Output: Compressed data: Huffman coding table + differential - coded bitstream.

[0069] Compression ratio: Reduced from 4 bytes / ID of the original index to an average of 0.5 bytes / ID (compression ratio of 85%).

[0070] A 3D Gaussian sputtering compression system based on residual quantization and dynamic pruning for implementing the 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning, comprising:

[0071] A quantization module for performing residual quantization vector quantization;

[0072] A strategy enhancement module for performing dynamic pruning strategy enhancement;

[0073] A compression and optimization module for performing index compression and coding optimization.

[0074] A computer device, which includes a memory and a processor. When a computer program stored in the memory is executed by the processor, the processor executes the steps of the 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning.

[0075] A computer - readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning.

[0076] An information data processing terminal includes the 3D Gaussian sputtering compression system based on residual quantization and dynamic pruning.

[0077] Combined with the above - mentioned technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by the present invention are as follows:

[0078] First, the correspondence between the core problems of the prior art and the solutions of the present invention:

[0079] 1. Storage efficiency bottleneck

[0080] Root cause of the problem: The scale of the original 3DGS model parameters is huge, and low - end devices cannot bear it, resulting in resolution compression and bandwidth waste.

[0081] Solutions of the present invention: Residual Vector Quantization (RVQ): Through two-stage quantization (4-bit coarse quantization + 8-bit residual quantization), the PSNR is increased by 0.2 - 0.3 dB at the same bit rate, the quantization error is reduced by 50%, and the storage amount is significantly reduced.

[0082] Dynamic codebook allocation: Adaptively allocate the codebook capacity according to the parameter entropy value (such as allocating a 16K codebook for high-frequency SH coefficients and only 4K for low-frequency ones), reducing redundant codewords.

[0083] Index compression optimization: Morton code sorting + differential Huffman coding reduce the index volume by 85% and the compression rate reaches 85%.

[0084] 2. Contradiction between quantization accuracy and detail loss

[0085] Root cause of the problem: Existing 8-bit uniform quantization causes the loss of high-frequency geometric features (such as the SSIM value of building edges drops to 0.82).

[0086] Solutions of the present invention:

[0087] Residual quantization compensation: Perform secondary quantization on the high-frequency components of color SH to retain high-frequency details, reduce the quantization error by 50%, and only cause the PSNR to drop by 0.07 dB (from the original 27.42 dB to 27.35 dB of the present invention on the Tanks&Temples dataset).

[0088] Mixed-precision codebook: Dynamically balance accuracy and efficiency, increase the backpropagation speed by 15%, and at the same time support the adaptation of low-end devices.

[0089] 3. Insufficient rendering speed and real-time performance

[0090] Root cause of the problem: The original 3DGS rendering speed is 134 FPS (1080p), which cannot meet the real-time interaction requirements such as AR navigation.

[0091] Solutions of the present invention:

[0092] Dynamic pruning strategy: Prune 70% of the low-contribution Gaussians through the energy function, reduce the rasterization calculation amount, and increase the rendering speed to 412 FPS (a 3-fold increase).

[0093] Hardware-friendly encoding: Store the index continuously after Morton code sorting, improve the GPU cache hit rate, and increase the decoding speed by 40%.

[0094] 4. Excessively high dynamic editing latency

[0095] Root cause of the problem: The lack of a compressive sensing mechanism for parameter updates, and the proportion of dynamic editing latency exceeds 40%.

[0096] Solutions of the present invention:

[0097] Soft Pruning and Recovery Mechanism: The probability of accidentally deleting key Gaussians drops by 80%. Details are dynamically recovered through regional loss monitoring to ensure real-time performance during editing.

[0098] Progressive Quantization Training: Avoid quantization interference with parameter convergence in the initial stage, and gradually strengthen the quantization intensity in the later stage. The number of codebook updates is reduced by 30%.

[0099] Second, the present invention adopts Residual Vector Quantization (RVQ), dynamic codebook allocation, and index optimization to achieve a compression ratio of 45 times, effectively reducing the storage requirements of the 3DGS (3D Gaussian Splatting) model. For example, in the kitchen scene of the Mip-NeRF 360 dataset, the volume of the original model is compressed from 440MB to 18.9MB, greatly alleviating the storage bottleneck of low-end devices. In addition, dynamic codebook allocation avoids overfitting / underfitting problems that may be caused by fixed allocation, enabling the compression method to maintain stable compression ratios and high fidelity in both urban scenes (high geometric complexity) and indoor scenes (high material details).

[0100] Compared with existing methods, the present invention can still maintain the clarity of high-frequency textures and edges to the greatest extent during the compression process. On the Tanks&Temples dataset, the PSNR (Peak Signal-to-Noise Ratio) of the present invention is 27.35dB, superior to 26.97dB of CompGS, with a PSNR drop of only 0.07dB. At the same time, the SSIM and LPIPS metrics are close to those of the original model, ensuring that the visual quality is nearly lossless. In addition, the soft pruning and recovery mechanism introduced by the present invention can effectively reduce the probability of accidentally deleting key Gaussians, guaranteeing the rendering quality in complex scenes.

[0101] By optimizing the calculation process, the rendering speed of the present invention is increased to 412FPS (1080p), far exceeding 134FPS of the original 3DGS, meeting the real-time rendering requirements in AR / VR scenarios. In addition, this method can achieve 1080p@62FPS on low-end graphics cards (such as NVIDIA GTX1050Ti, 4GB), with only a 3.2GB video memory occupancy. Compared with traditional methods that rely on high-end GPUs, the present invention achieves stronger hardware compatibility through video memory hierarchical management, calculation pipeline decomposition, and dynamic power consumption control, enabling low-end devices to also run high-quality 3DGS rendering.

[0102] The present invention reduces the number of codebook updates through progressive quantization training, shortening the training time by 20%. At the same time, the combination of mixed-precision codebooks improves the backpropagation speed. In addition, the proposed multi-factor energy function pruning strategy can adapt to optimization requirements in different scenarios, ensuring stable rendering performance in complex scenes.

[0103] The present invention is superior to CompGS in multiple core technology dimensions. In terms of the quantization strategy, the present invention adopts RVQ + dynamic codebook allocation, while CompGS adopts K-means clustering + fixed codebook, resulting in insufficient generalization ability; in terms of the pruning mechanism, the energy function-driven soft pruning combined with spatial awareness recovery of the present invention can more accurately retain key Gaussians compared to the hard threshold pruning of CompGS; in terms of index compression, the present invention uses Morton code sorting + Huffman coding to achieve a compression ratio of 85%, which is superior to the run-length encoding (RLE) of CompGS; finally, it realizes 45-fold storage compression and 3-fold speed improvement, while CompGS can only achieve 40-50-fold storage compression and 2-3-fold speed improvement.

[0104] While compressing storage, the present invention minimizes quality loss to the greatest extent. Compared with CompGS where the PSNR drops by about 0.3dB, the PSNR of the present invention only drops by 0.07dB, ensuring the integrity of high-frequency details. In addition, the adaptability of this method to low-end devices is better, with the CUDA utilization rate reaching 90%, and it can operate efficiently on devices with video memory ≤ 4GB, while CompGS relies on high-end GPUs, which limits its application on ordinary hardware. The present invention is superior to the prior art in terms of storage efficiency, rendering quality, real-time performance, hardware compatibility, and training optimization, providing a better solution for the application of 3DGS in actual scenarios.

[0105] Third, as the creative auxiliary evidence of the claims of the present invention, it is also reflected in the following important aspects:

[0106] 1. The expected benefits and commercial value after the transformation of the technical solution of the present invention are:

[0107] (1) Expected benefits: The technical solution of the present invention will directly create significant economic benefits through three core advantages: 45-fold storage compression, 3-fold rendering speed improvement, and high-fidelity rendering quality:

[0108] Productization benefits: Develop independent compression tools or SDKs, and sell them to content creators such as games and movies. The pricing is based on the model scale or functional modules, and it is expected to cover more than 30% of the market share.

[0109] Optimization of operating costs: After cloud rendering service providers adopt this technology, the server costs are reduced by 60%. They can seize the market through low-price strategies (such as reducing the rendering cost from 0.5 / frame to 0.2 / frame), and at the same time support millions of concurrent users, with the revenue increased by 40%.

[0110] (2) Commercial value: The commercial value of the technical solution is reflected in the trillion-level market coverage and ecological construction:

[0111] Scene Penetration: In fields such as AR / VR (expected market of $108 billion in 2025), gaming ($196 billion in 2024), and the metaverse ($800 billion in 2025), lightweight deployment of 3D models is achieved to promote the popularization of applications such as virtual fitting and real-time navigation.

[0112] Hardware Adaptation and Ecological Barriers: Support low-end graphics cards with video memory ≤ 4GB (such as GTX 1050Ti), adapt to mainstream consumer-grade hardware, cooperate with manufacturers such as Intel and AMD to form a closed-loop of "hardware algorithm application", and build technical standards.

[0113] Social Benefits and Long-term Impact: The energy consumption of model storage and transmission is reduced by 80%, contributing to the carbon neutrality goal; promoting the sinking of 3D technology into fields such as education and medical care, deriving new scenarios such as autonomous driving environment modeling and industrial simulation, and creating indirect economic value exceeding 50 billion yuan.

[0114] 2. The technical solution of the present invention overcomes technical biases:

[0115] The technical solution of the present invention successfully overcomes the long-existing technical biases in the field of 3D model compression through innovative residual vector quantization, dynamic pruning strategies, and index compression optimization: Traditional technologies generally believe that high-bit quantization is the only way to ensure rendering quality, while the 4-bit coarse quantization combined with 8-bit residual quantization strategy proposed by the present invention realizes a 0.2 - 0.3dB increase in PSNR and a 50% reduction in quantization error under low-bit quantization through a residual compensation mechanism, demonstrating the non-linear balance between compression and quality; aiming at the bias that pruning inevitably leads to irrecoverable quality loss, the present invention constructs a closed-loop of dynamic pruning driven by a multi-factor energy function and spatial perception recovery, dynamically wakes up dormant Gaussians through voxel-level loss monitoring, with the misdeletion rate decreasing by 80% and the PSNR only losing 0.07dB; in terms of hardware adaptation, it breaks the inherent perception that "real-time rendering depends on high-end GPUs", and realizes 1080p rendering on low-end graphics cards with video memory ≤ 4GB through video memory hierarchical management, calculation pipeline decomposition, and Morton code optimization, with a CUDA utilization rate of 90%; in addition, it subverts the limitation that the traditional index compression rate is difficult to exceed 65%, adopts Morton code sorting + differential Huffman coding, and compresses the index volume by 85%, laying a foundation for the large-scale application of 3D Gaussian sputtering technology in mobile devices, XR devices, and real-time streaming media. Brief Description of the Drawings

[0116] Figure 1 It is a flowchart of a 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning provided by an embodiment of the present invention;

[0117] Figure 2 It is a structural diagram of a 3D Gaussian sputtering compression system based on residual quantization and dynamic pruning provided by an embodiment of the present invention;

[0118] Figure 3 It is the flowchart of dynamic pruning update provided by the embodiment of the present invention;

[0119] Figure 4 It is the flowchart of the operation of the embodiment provided by the embodiment of the present invention;

[0120] Figure 5 It is the comparison chart of the effects of various methods provided by the embodiment of the present invention;

[0121] Figure 6 It is the comparison chart before and after Gaussian scene compression provided by the embodiment of the present invention. Detailed implementation manners

[0122] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0123] The working principle of the 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning.

[0124] 1. Overview and working principle overview

[0125] The purpose of this method is to efficiently compress the 3D Gaussian sputtering representation to reduce storage and computing requirements while maintaining high-quality rendering. The core idea is to combine strategies such as residual vector quantization (RVQ), dynamic pruning, and index optimization to improve data compactness, reduce storage bandwidth, and optimize rendering calculations. It is mainly divided into three major stages: S1 (residual vector quantization), S2 (dynamic pruning strategy), and S3 (index compression and coding optimization). Throughout the process in the 3DGS rendering task, redundant Gaussian data can be greatly reduced, and storage and rendering efficiency can be improved.

[0126] 2. Residual quantization and mixed-precision training (S1)

[0127] (1) Parameter grouping and dynamic codebook allocation

[0128] Before quantization, the Gaussian parameters are first grouped, such as color, covariance, spatial coordinates, etc., to optimize the compression efficiency. For example:

[0129] Color is split into the DC component and SH coefficients (separation of low frequency and high frequency).

[0130] The covariance matrix is decomposed into a rotation quaternion and a logarithmic scale.

[0131] Calculate the information entropy of each group of parameters, and allocate the codebook capacity accordingly to ensure that more important information is subjected to high-precision quantization processing.

[0132] (2) Residual vector quantization (RVQ)

[0133] First, a low bit - rate codebook (e.g., 4 - bit, 16 centers) is used for coarse quantization, and the residual is calculated. Subsequently, the residual is second - quantized with higher precision (e.g., 8 - bit, 256 centers) to further reduce the error.

[0134] (3) Mixed - precision training

[0135] To dynamically adjust the quantization strategy during training, a temperature annealing mechanism is introduced:

[0136] In the initial stage (T = 1.0), soft quantization is adopted, and the floating - point residual is retained to avoid premature convergence.

[0137] In the later stage of training (T→0.1), it gradually turns to hard quantization to improve the storage and computational efficiency of the model.

[0138] Finally, the output of this stage is the quantized parameters and the corresponding codebook data for subsequent storage and index optimization.

[0139] 3. Dynamic pruning strategy enhancement (S2)

[0140] (1) Energy function calculation and dynamic pruning

[0141] Calculate the energy of each Gaussian, including opacity (σ), the distance of the Gaussian to the origin, and visibility under multiple perspectives.

[0142] Adopt a dynamic threshold adjustment mechanism. Based on the mean and standard deviation of the energy of all current Gaussians, adaptively determine the pruning threshold to ensure that only low - contribution Gaussian points are removed.

[0143] (2) Soft pruning and recovery mechanism

[0144] Dormancy marking: If the energy of a certain Gaussian is less than the set threshold, it is temporarily dormant and does not participate in the rendering calculation, but its parameters are still retained.

[0145] Region loss monitoring: Divide the scene into a 16×16×16 voxel grid and monitor the rendering loss (L_region) of each region. If the loss exceeds a certain threshold for M consecutive iterations (default 10 times), the dormant Gaussians in this region are awakened.

[0146] Gaussian recovery: For the awakened Gaussians, readjust their opacity (σ) and give them a higher energy weight α (default 0.8) to avoid being pruned repeatedly.

[0147] Finally, an optimized Gaussian set is output, effectively reducing unnecessary calculations while retaining key visual information.

[0148] 4. Index compression and coding optimization (S3)

[0149] (1) Morton Code Sorting and Differential Encoding

[0150] To optimize data storage and indexing:

[0151] First, normalize the three-dimensional coordinates (x, y, z) of the Gaussian to the integer range of [0, 2^16 - 1], and calculate its three-dimensional Morton Code for spatial sorting.

[0152] Sort the Gaussian index by the Morton Code to make adjacent data storage more compact and improve local access efficiency.

[0153] Calculate the differential encoding of the index to further reduce storage redundancy.

[0154] (2) Entropy Encoding and Compression

[0155] Statistically analyze the frequency distribution of the differential sequence, and use Huffman Encoding to assign short codewords to high-frequency values, thereby further reducing the data storage cost.

[0156] Convert the differential encoding bitstream to the big-endian storage format, ultimately achieving an increase in the data compression ratio, from 4 bytes / ID of the original index to an average of 0.5 bytes / ID.

[0157] 5. Summary of Key Optimization Strategies

[0158] This method improves the storage and calculation efficiency of 3D Gaussian sputtering data through three core optimization strategies:

[0159] (1) Residual Vector Quantization (RVQ): Adopt a step-by-step quantization strategy to maintain high precision while reducing the amount of data.

[0160] (2) Dynamic Pruning: Based on an adaptive energy evaluation mechanism, effectively reduce redundant Gaussians and improve the rendering speed.

[0161] (3) Index Compression: Utilize Morton Code sorting, differential encoding, and entropy encoding to greatly reduce the storage requirements.

[0162] 6. Expected Technical Effects and Application Scenarios

[0163] (1) Storage Optimization

[0164] After adopting this method, the Gaussian data storage volume is reduced by more than 60%, which is suitable for large-scale 3D scene storage and rendering.

[0165] (2) Improvement in Computational Efficiency

[0166] Due to pruning and index optimization, the rendering computation is reduced by 30%-50%, which can accelerate tasks such as NeRF scene rendering and 3D point cloud reconstruction.

[0167] (3) Adaptive pruning mechanism

[0168] This method can adaptively adjust the pruning strategy from different perspectives to achieve higher-quality visual fidelity, and is applicable to fields such as AR / VR, autonomous driving, and game rendering.

[0169] This method innovatively combines strategies such as residual quantization, dynamic pruning, and index compression, greatly reducing the storage requirements of 3D Gaussian splatter data and optimizing the rendering computation efficiency. Compared with traditional methods, this method reduces the storage occupancy by 60% and the rendering computation by 30-50%, and is particularly suitable for application scenarios such as efficient 3D scene rendering, UAV image reconstruction, augmented reality (AR), and virtual reality (VR).

[0170] Such as Figure 1 shown, the 3D Gaussian splatter compression method based on residual quantization and dynamic pruning includes:

[0171] S1: Residual quantization vector quantization, grouping 3D Gaussian parameters and dynamically allocating codebook capacity, reducing data storage occupancy through step-by-step quantization;

[0172] S2: Enhancement of dynamic pruning strategy, adaptively pruning redundant Gaussian points based on energy function calculation and visibility analysis;

[0173] S3: Index compression and coding optimization, using Morton code sorting, differential coding, and entropy coding to improve data storage efficiency.

[0174] Such as Figure 2 shown, the 3D Gaussian splatter compression system based on residual quantization and dynamic pruning includes:

[0175] Quantization module, used for residual quantization vector quantization;

[0176] Strategy enhancement module, used for enhancing the dynamic pruning strategy;

[0177] Compression and optimization module, used for index compression and coding optimization.

[0178] 7. Complete process of residual quantization vector quantization strategy

[0179] 1) Parameter grouping and dynamic codebook allocation

[0180] Input: Original Gaussian parameters (position, color SH coefficients, covariance rotation / scale, opacity)

[0181] Parameter grouping: The color group is split into DC component (1 dimension) and SH coefficient (15 dimensions). The SH coefficient is further divided into low frequency (1-3 orders) and high frequency (4-15 orders). The covariance group is split into rotation quaternion (4 dimensions) and logarithmic scale (3 dimensions).

[0182] Information entropy calculation: Calculate information entropy for each set of parameters:

[0183]

[0184] Parameter meaning:

[0185] x: Parameters to be compressed (such as color SH coefficients, covariance components, etc.).

[0186] p: The probability of a parameter value appearing in the dataset.

[0187] 2) Dynamic allocation of codebook:

[0188]

[0189] K total is the total codebook capacity (e.g. 32K), K g is the codebook size of the gth group

[0190] Residual Quantization (RVQ) and Mixed Precision Training

[0191] For each set of parameters, a low-bit codebook (4 bits, 16 centers) is used for initial quantization:

[0192]

[0193] Residual calculation and secondary quantization:

[0194]

[0195] Where r (Residual) is an 8-bit codebook (256 centers)

[0196]

[0197] Core idea: During the training process, quantizers of different precisions are jointly optimized, and the soft and hard quantization weights are controlled through temperature annealing to gradually converge to the optimal quantization result.

[0198] Temperature annealing weight formula:

[0199]

[0200] E hard : Hard quantization (direct rounding) error

[0201] E soft: Error of soft quantization (retaining floating point residual)

[0202] T: Temperature parameter, with an initial high value (such as 1.0) and gradually decreasing to close to 0 (such as 0.1).

[0203] 3) Output:

[0204] Quantization parameters:

[0205] Codebook data: {C coarse ,C residual}

[0206] The coarse quantization codebook C coarse retains low-frequency features, and the residual codebook C residual compensates for high-frequency details

[0207] 8. Enhanced dynamic pruning strategy

[0208] 1) Input: Gaussian parameters in training and their multi-view visibility records.

[0209] Multi-factor energy calculation:

[0210]

[0211] σ: opacity;

[0212] x i : distance from Gaussian to scene origin;

[0213] N visible : The number of times the Gaussian is visible in the training view

[0214] α=0.6,β=0.3,γ=0.1 (weights are adjustable)

[0215] 2) Soft pruning and recovery mechanism

[0216] Dynamic threshold adjustment: τ = μ E -k·σ E

[0217] μ E is the energy mean of all current Gaussians, σ E is the standard deviation.

[0218] Sleep flag: If Ei<τ, mark Gaussian i as dormant (set σi=0), retain the parameters but do not participate in rendering. The dormant Gaussian does not participate in forward rendering, but participates in backpropagation updates.

[0219] Region loss monitoring: Divide the scene into a voxel grid of 16×16×16, and count the rendering loss L region of the dormant Gaussians in each voxel. If the rendering loss L region in M consecutive iterations > η, then wake up the dormant Gaussians in this region; trigger the recovery mechanism. By default, M = 10 and η = 0.05 (adjustable).

[0220] Gaussian recovery: Wake up the dormant Gaussian in this voxel, reset σi to the initial value, and increase its energy weight α to 0.8 (to prevent it from being pruned again).

[0221] 3) Output: Pruned Gaussian set: Only keep the active (σi > 0) and recovered Gaussians.

[0222] Dormant pool: Store the parameters of the dormant Gaussians for subsequent recovery.

[0223] The specific pruning update and codebook update operations are as Figure 3 shown

[0224] 9. Index Compression and Coding Optimization

[0225] 1) Morton Code Sorting and Differential Coding

[0226] Input: Quantized Gaussian indices (integer sequence).

[0227] Coordinate normalization and Morton code calculation:

[0228] Normalize the Gaussian center coordinates [x, y, z] to the integer range of [0, 2^16 - 1], and calculate the three-dimensional Morton code.

[0229] Index sorting: Rearrange the indices in ascending order according to the Morton code to generate an ordered sequence {I 1, I 2,..., I N}.

[0230] Differential calculation:

[0231]

[0232] 2) Entropy Coding and Compression

[0233] Huffman code table generation:

[0234] Statistical frequency distribution of δi, and assign short codewords to high-frequency values (such as δ ≤ 5).

[0235] Binary compression:

[0236] Convert the differential sequence into a Huffman-coded bit stream and store it in big-endian order.

[0237] 3) Output: Compressed data: Huffman coding table + differential coding bit stream.

[0238] Compression ratio: Reduced from 4 bytes / ID in the original index to an average of 0.5 bytes / ID (compression ratio of 85%).

[0239] As can be seen from the above description, the beneficial effects of the technical solution of the present invention are as follows:

[0240] 1) Significantly reduce storage overhead and improve deployment efficiency

[0241] Effect: The model volume is compressed to 1 / 45 of the original 3DGS, breaking through the storage bottleneck of low-end devices.

[0242] Technical support:

[0243] Dynamic codebook allocation: Allocate the codebook capacity according to the parameter entropy value to reduce redundant codewords (e.g., the codebook for high-frequency SH coefficients is increased to 16K, and only 4K for low-frequency).

[0244] Residual quantization (RVQ): Two-stage quantization retains high-frequency details, and the PSNR is increased by 0.2 - 0.3 dB at the same code rate.

[0245] Index compression optimization: Morton code sorting + differential Huffman coding, reducing the index volume by 85%.

[0246] 2) Improve rendering speed and meet real-time requirements

[0247] Effect: The rendering speed reaches 412 FPS (at 1080p resolution), which is 3 times higher than the original 3DGS (134 FPS).

[0248] Technical support:

[0249] Dynamic pruning: Prune 70% of the low-contribution Gaussians through the energy function, reducing the rasterization calculation amount.

[0250] Hardware-friendly coding: The index after Morton code sorting is stored continuously, improving the GPU cache hit rate and increasing the decoding speed by 40%.

[0251] 3) High-fidelity rendering quality and controllable detail loss

[0252] Effect: The PSNR only drops by 0.07 dB (on the Tanks&Temples dataset, from 27.42 dB of the original 3DGS to 27.35 dB of the present invention), and the high-frequency texture and edge sharpness are better than CompGS.

[0253] Technical support:

[0254] Residual quantization compensation: Secondary quantization of the high-frequency components of color SH reduces the quantization error by 50%.

[0255] Soft pruning recovery mechanism: The probability of erroneously deleting key Gaussians drops by 80%, and details are dynamically recovered through regional loss monitoring.

[0256] 4) Adaptive scene complexity with strong generalization ability

[0257] Effect: Maintain stable compression ratio and quality in both urban scenes (high geometric complexity) and indoor scenes (high material details).

[0258] Technical support:

[0259] Dynamic codebook allocation: The codebook capacity is adaptively adjusted according to the parameter entropy, avoiding overfitting / underfitting caused by fixed allocation.

[0260] Multi-factor energy function: Integrate indicators such as spatial density and view coverage to meet the pruning requirements of different scenes.

[0261] 5) Optimization of training and inference efficiency

[0262] Effect: The training time is shortened by 20%, and the convergence stability of the codebook is improved.

[0263] Technical support:

[0264] Progressive quantization training: Avoid quantization interference with parameter convergence in the initial stage, and gradually strengthen the quantization intensity in the later stage. The number of codebook update times is reduced by 30%.

[0265] Mixed-precision codebook: Dynamically balance precision and efficiency through soft selection weights, and the backpropagation speed is increased by 15%.

[0266] Example:

[0267] This example elaborates on the specific implementation of a real-time 3D Gaussian Splatting model compression system that combines residual vector quantization and dynamic pruning. Figure 4 It is the project workflow diagram in the embodiments of the present invention.

[0268] Specifically, the present invention uses the kitchen scene in the Mip-NeRF 360 dataset for experimental verification.

[0269] Step A: Use the SFM technology to generate sparse point clouds from the kitchen scene pictures in Mip-NeRF 360, and then input them into the 3DGS original model for 3D modeling. The established 3D Gaussian scene is constructed by millions of Gaussians. Each Gaussian sphere contains position coordinates, rotation quaternions, normal vectors, opacity, colors, and up to 44 spherical harmonic function bands (SH1 - SH44) to achieve view-dependent color information expression. Measured psnr = 32.06, ssim = 0.930, lpips = 0.120, and the storage size is 440MB

[0270] Step B: Input the established three-dimensional Gaussian scene into the three-dimensional model compression system designed by the present invention. This system compresses the three-dimensional Gaussian scene through three modules, including vector quantization strategy, dynamic pruning strategy enhancement, and index compression and coding optimization.

[0271] Module 1 - Vector Quantization Strategy: This module uses hierarchical residual quantization and entropy-driven codebook allocation to achieve efficient compression. First, the parameters are divided into three groups according to physical characteristics: low-frequency (position / covariance), medium-frequency (SH low-frequency components), and high-frequency (SH high-frequency components). The codebook capacity is dynamically allocated based on information entropy (where the high-frequency group is allocated a 16K codebook (12-bit index)). The error is reduced through two-level residual quantization (RVQ): the first-level K-means generates the basic codebook, and the second-level further quantizes the residuals (the codebook capacity is reduced to 1 / 4). The final parameter recovery formula is X = Q1(X) + Q2(R). This strategy achieves a compression ratio of 10:1, with a PSNR loss ≤ 0.1 dB, and at the same time supports FP16 / FP32 mixed-precision codebook adaptation for low-end devices.

[0272] Module 2 - Dynamic Pruning Strategy Enhancement: This module restores the balance between rendering efficiency and quality through an energy function and spatial awareness. The energy function combines density (weight 0.6), view coverage rate (0.3), and gradient magnitude (0.1), retains the top 30% of high-energy Gaussian points, and the rest are stored in the sleep pool according to spatial hashing (0.1m 3 voxel granularity). During real-time rendering, the local image quality is monitored, and the sleep points are smoothly activated through neighboring interpolation, with a restoration delay ≤ 5 ms. Combining GPU batch rasterization (8x8 pixels / thread block) and Warp-level transparent pixel merging, the rendering frame rate is increased to over 1080p@400 FPS, and the computational load is reduced by 22%.

[0273] Module 3 - Index Compression and Coding Optimization: This module reduces storage and transmission overhead through spatial indexing and entropy coding. The spatial position of Gaussian points is encoded using 32-bit Morton codes (Z-order curve sorting), which improves data locality and generates 128-byte aligned memory blocks, adapting to the GPU SIMD instruction set. Index differential coding converts the difference between adjacent Morton codes into a Huffman code stream. Short differences (≤ 8) are represented by short codes, and long differences trigger block reorganization, with the index volume compressed by 70%.

[0274] Step C: In view of the hardware bottleneck of low-end graphics cards (video memory ≤ 4GB, limited computing power) in this embodiment, efficient deployment is achieved through video memory hierarchical management, computing pipeline decomposition, and dynamic power consumption control. The video memory hierarchical strategy is adopted, and high-frequency parameters (position, SH low-frequency components) are stored in the resident video memory area (≤ 1.5GB) with FP16 precision, and low-frequency parameters (covariance matrix, SH high-frequency components) are stored according to 0.5m 3Spatial block dynamic loading, only load the currently visible 3×3 blocks (27 blocks) through frustum culling and transfer them asynchronously (the latency ≤ 3ms is achieved by CUDA streams). The computing tasks are decomposed into a three-level pipeline: the first-level pipeline (50% CUDA Core) processes Gaussian projection and sorting (8 points / thread, batch size 256), the second-level pipeline (30% CUDA Core) performs rasterization and blending (8x8 pixel blocks / thread block, Warp-level atomic operations), and the third-level pipeline (20% CUDA Core) is responsible for residual compensation and super-resolution reconstruction. In terms of power consumption control, the GPU frequency is dynamically adjusted according to the core temperature (full speed of 1455MHz / 75W below 70°C, frequency reduction to 1083MHz / 45W above 85°C), and BC7 compressed textures are enabled to reduce the bandwidth by 40%. When the temperature is high, the number of active points per frame is limited to ≤ 30k. It has been verified that on NVIDIA GTX 1050Ti (4GB), 1080p@62FPS is achieved, with a video memory occupancy of 3.2GB and a peak power consumption of 73W (temperature 78°C); on the AMD RX 560 platform, the video memory occupancy is 3.5GB and the frame rate is 58FPS, and on the Intel Arc A380, the frame rate is increased to 72FPS (temperature 71°C), supporting the rendering of 100 million Gaussian points (only 30 million by conventional methods), with a CUDA core utilization rate of 90%, and no frequency reduction during 72 hours of continuous load at 70°C (error rate < 0.1%). The measured psnr = 31.86, ssim = 0.921, lpips = 0.129 for the compressed data, and the storage size is only 18.9MB.

[0275] An application embodiment of the present invention provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor is caused to execute the steps of the 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning.

[0276] An application embodiment of the present invention provides a computer-readable storage medium, storing a computer program. When the computer program is executed by the processor, the processor is caused to execute the steps of the 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning.

[0277] An application embodiment of the present invention provides an information data processing terminal, which includes a 3D Gaussian sputtering compression system based on residual quantization and dynamic pruning.

[0278] The present invention can be widely applied to the following fields and products: In real-time 3D modeling and rendering engines (such as Unreal Engine, 3D Gaussian splatter plugins for Unity), through dynamic codebook allocation and residual quantization techniques, the storage volume of tens of millions of Gaussian parameters is compressed from 12.4 GB to 3.8 GB, and real-time rendering at 55 FPS is achieved, supporting VR / AR headsets (such as Meta Quest Pro) to smoothly load large virtual scenes; in autonomous driving lidar point cloud processing systems (such as embedded processors for Velodyne VLS-128), combined with dynamic pruning and mixed-precision quantization, single-frame point cloud data is compressed from 50 MB to 8.2 MB, and the decoding delay is less than 9 ms, enabling L4-level autonomous driving vehicles (such as Waymo's fifth-generation system) to perceive obstacles in real time under 5G networks; in edge AI inference devices (such as the on-board target detection module of the DJI Matrice 300 RTK drone), using a hierarchical vector quantization strategy, the volume of the YOLOv5 model is compressed from 7.1 MB to 1.2 MB, and the inference speed is increased to 38 FPS, achieving the recognition of people and vehicles in 4K video streams; in medical imaging cloud platforms (such as the United Imaging uCloud precise diagnosis system), through ROI region adaptive compression and residual secondary quantization, the compression ratio of MRI images is increased from 10:1 to 15:1, and the transmission time is shortened from 3.2 s to 1.8 s, supporting primary hospitals to access high-fidelity images of 45 dB from tertiary hospitals in real time; in industrial digital twin systems (such as the Siemens MindSphere platform), based on entropy-driven codebook allocation technology, the point cloud model of production line equipment is compressed by 60%, enabling the full loading of the factory model of 2000 devices in a 16 GB memory workstation, facilitating real-time fault prediction. These product-level applications verify the technical universality of the present invention in the field of high-dimensional data compression, and reconstruct the technical benchmarks of the 3D graphics, autonomous driving, edge intelligence, and medical imaging industries with "lower storage, faster computing, and smaller quality loss".

[0279] As Figure 5 shown, in the experiment, we used three common parameters, PSNR, L-PIPS, and SSIM, as the measurement indicators for the quality of the generated images. The storage size of the model is represented by size, and the unit is MB. To better evaluate the performance of this method, we also compared it with several model compression directions similar to our work. 3DGS is used as the baseline, and the methods in the middle part are all the mainstream compression methods currently designed for 3DGS. Red is the best method, followed by orange, and then yellow. It can be seen from the results that our method can still achieve or even exceed the performance of existing methods in terms of reconstruction quality while maintaining a small model size. To better highlight the performance improvement of the model, the present invention adopted the same parameter configuration as the original 3DGS method in the experiment. All experiments were carried out on an NVIDIA 3090 GPU.

[0280] The comparison diagram of the effects is as Figure 6 shown. For the Gaussian scenario with high storage requirements, after being processed by this invention, without sacrificing the fidelity, the storage requirements are greatly reduced.

[0281] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated designed hardware. Those of ordinary skill in the art can understand that the above devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code is provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and their modules of the present invention can be implemented by hardware circuits of programmable hardware devices such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above hardware circuits and software such as firmware.

[0282] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning, characterized in that Including: S1: Residual quantization vector quantization, grouping 3D Gaussian parameters and dynamically allocating codebook capacity, reducing data storage occupancy through step-by-step quantization; S2: Enhancement of dynamic pruning strategy, adaptively pruning redundant Gaussian points based on energy function calculation and visibility analysis; S3: Index compression and coding optimization, using Morton code sorting, differential coding, and entropy coding to improve data storage efficiency.

2. The 3D Gaussian sputtering compression method according to claim 1, characterized in that, S1 includes: Grouping Gaussian parameters, including color (DC component + SH coefficients), covariance (rotation quaternion + logarithmic scale); Calculating the information entropy of each group of parameters and allocating the quantization codebook capacity according to the entropy ratio; Adopting step-by-step residual quantization (RVQ), first performing low-bitrate coarse quantization and then performing higher-precision secondary quantization on the residual part; During the training process, dynamically adjusting the weights of soft quantization and hard quantization in combination with the temperature annealing mechanism to make the final quantization result tend to be optimal.

3. The 3D Gaussian sputtering compression method according to claim 1, wherein S2 Including: Calculating the energy of each Gaussian, including factors such as opacity, distance, visibility, etc., and constructing an energy function; Setting a dynamic threshold, adjusting the pruning criterion based on the Gaussian energy mean and standard deviation, and putting low-contribution Gaussians into dormancy; Adopting a regional loss monitoring method, counting the rendering loss in the voxel grid. If the loss exceeds the set threshold for M consecutive iterations, reactivate the dormant Gaussians.

4. The 3D Gaussian sputtering compression method according to claim 1, characterized in that S3 Including: Normalizing the Gaussian center coordinates (x, y, z) using Morton code and calculating the three-dimensional Morton code; Sorting the indexes according to the Morton code to improve the locality of data access; Adopting differential coding, storing the increment between adjacent indexes to reduce storage redundancy.

5. The 3D Gaussian sputtering compression method according to claim 1, wherein S3 also includes: Performing entropy coding on the differentially encoded index data, and optimizing the storage structure using Huffman coding; Statistically analyzing the distribution of index increments, assigning shorter codewords to high-frequency values to reduce storage occupancy; Storing the Huffman coding table and bitstream in a binary compression format, reducing the index storage requirement from 4 bytes / ID to 0.5 bytes / ID.

6. The 3D Gaussian sputtering compression method according to claim 1, wherein Also including: Combining the multi-view information of Gaussian parameters to optimize the pruning strategy, making the Gaussian points have a higher survival probability under key views and ensuring the rendering quality; Adopting a multi-level index structure, enabling rapid retrieval of the area to be activated when dynamically restoring dormant Gaussian points and improving the calculation efficiency; Through joint optimization of the three modules of quantization, pruning, and index storage, the overall compression ratio is improved while ensuring the balance of rendering quality and calculation efficiency.

7. A 3D Gaussian sputtering compression system based on residual quantization and dynamic pruning for implementing the 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning as described in any one of claims 1 to 6, characterized in that, Including: A quantization module for performing residual quantization vector quantization; A strategy enhancement module for enhancing the dynamic pruning strategy; A compression and optimization module for performing index compression and coding optimization.

8. A computer device, the computer device includes a memory and a processor. When a computer program stored in the memory is executed by the processor, the processor executes the steps of the 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning as described in any one of claims 1 to 6.

9. A computer-readable storage medium, storing a computer program. When the computer program is executed by the processor, the processor executes the steps of the 3D Gaussian sputtering compression method based on residual quantization and dynamic pruning as described in any one of claims 1 to 6.

10. An information data processing terminal, the information data processing terminal includes a 3D Gaussian sputtering compression system based on residual quantization and dynamic pruning as described in claim 7.

Images