3D Gaussian splash compression method based on Gaussian body parameter layer frequency domain signature

Abstract

The invention discloses a 3D Gaussian splash compression method based on Gaussian body parameter layer frequency domain signature. According to the method, firstly, a Gaussian body set is obtained from a three-dimensional scene reconstruction or training result to serve as input, then the Gaussian body set sequentially passes through a Gaussian parameter projection module, a spectrum signature calculation module, a signature quantization and coding module, a spectrum driven pruning and merging module and a verification and rollback module, and finally, a compressed three-dimensional Gaussian model is output. The signature quantization and coding module performs frequency band division, quantization and bit depth allocation on a spectrum signature to realize first compression. And the spectrum-driven pruning and merging module calculates the importance score of each Gaussian based on the signature, performs pruning on the low-importance Gaussian, and merges and substitutes the gausses with similar spectrums to realize secondary compression. The method solves the problems that the number of Gaussian bodies in an existing method is large, the occupied space is large, and the rendering efficiency is low, and is suitable for the technical fields of three-dimensional reconstruction, virtual reality, augmented reality, digital twinning, graphic rendering and the like.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, especially the field of 3D scene reconstruction and rendering. It relates to a 3D Gaussian splash compression method based on Gaussian volume parameter layer frequency domain signature, which is a compression method using Gaussian volume parameter layer frequency domain transformation. Background Technology

[0002] 3D Gaussian Splatting (3DGS) is a real-time rendering technique based on Gaussian distribution. It generates a 3D representation of a scene through machine learning training and utilizes GPU acceleration for efficient rendering. Recent advancements in 3DGS-based 3D scene reconstruction have yielded some results and breakthroughs in new view compositing techniques. It represents a scene as a cluster of 3DGs, each defined by its own 3D coordinates, covariance matrix, color, transparency, and other parameters. During rendering, each Gaussian volume is projected onto the pixel plane, and a weighted sum is obtained to obtain the rendered image. However, complex real-world scenes often require hundreds of thousands or even millions of Gaussian volumes to fully represent them. As the number of Gaussian volumes increases, the model storage and rendering computational load rise sharply, leading to problems such as high GPU memory usage, high loading latency, and decreased rendering frame rate.

[0003] Existing compression and optimization methods mainly include: (1) Spatial domain compression method: reduce the number of Gaussians by means of Gaussian clustering, voxel sampling, scale optimization, etc., but these methods will sacrifice high-frequency texture information, resulting in blurred edges and loss of details.

[0004] (2) Image-level frequency domain method: By introducing frequency domain regularization into the rendered image layer to control the learning process of different frequencies, the model convergence is optimized. However, this type of method only operates at the pixel level and cannot achieve direct compression of Gaussian parameters.

[0005] Therefore, there is an urgent need for a new method that can directly utilize frequency domain information for compression and pruning in the Gaussian parameter layer, so as to achieve efficient compression and fast rendering without sacrificing details. Summary of the Invention

[0006] The purpose of this invention is to provide a 3D Gaussian splash compression method based on Gaussian volume parameter layer frequency domain signature.

[0007] The method of this invention first obtains a set of Gaussian volumes from the 3D scene reconstruction or training results as input. The Gaussian volumes together constitute the 3D representation of the scene, which is used to fully describe the distribution characteristics of each point in the 3D space. Then, it passes through the Gaussian parameter projection module, the spectrum signature calculation module, the signature quantization and encoding module, the spectrum-driven pruning and merging module, and the verification and rollback module in sequence, and finally outputs the compressed 3D Gaussian model.

[0008] The Gaussian parameter projection module maps the input three-dimensional Gaussian volume onto a two-dimensional imaging plane through camera projection relationships, obtaining equivalent two-dimensional Gaussian parameters, and outputting a set of two-dimensional Gaussian parameters suitable for frequency domain analysis.

[0009] The aforementioned spectrum signature calculation module is based on the analytical Fourier transform of 2D Gaussian to calculate the energy distribution and phase characteristics of each Gaussian body in different frequency ranges. According to the characteristics of the Gaussian function in the frequency domain, the entire frequency range is divided into three intervals: low frequency, mid frequency, and high frequency. By analyzing the energy proportion and phase distribution of each frequency band, a fixed-length spectrum signature is generated to represent the characteristics of the Gaussian body in the frequency domain.

[0010] The signature quantization and encoding module compresses the obtained spectral signature to achieve the first compression of the parameter layer. First, the energy value of each frequency band is quantized. Then, different bit widths are allocated according to the importance of different frequency bands. Finally, the quantized signature is entropy encoded or vector quantization encoded to form a compact data representation structure.

[0011] The spectrum-driven pruning and merging module calculates the importance score of each Gaussian based on its signature, prunes low-importance Gaussians, and merges and replaces multiple Gaussians with similar spectra to achieve a second compression, namely structured compression. Based on the spectral signature of each Gaussian, its importance score in the reconstruction process is calculated. When the importance score is lower than a set threshold, the corresponding Gaussian is deleted. For multiple Gaussians with high spectral feature similarity, a merging operation is performed.

[0012] The verification and rollback module calculates the differences in brightness, structure, and detail of the model before and after compression. When the detection results show that the overall error or local error exceeds the preset threshold, the rollback mechanism is activated. The verification process includes frequency domain error detection and image-level rendering quality assessment.

[0013] This invention shifts frequency domain analysis down to the Gaussian parameter layer, where each Gaussian has its own individual frequency domain contribution that can be estimated and used as input for compression decisions. Compared to methods that only perform frequency domain regularization on the image layer, this invention can directly achieve efficient structural compression at the representation layer and supports layered coding, progressive decoding, and viewpoint-adaptive compression strategies. This invention addresses the problems of a large number of Gaussian volumes, large space occupation, and low rendering efficiency in existing 3D scene representations, and is applicable to fields such as 3D reconstruction, Virtual Reality (VR), Augmented Reality (AR), digital twins, and graphics rendering. Attached Figure Description

[0014] Figure 1 This is a general framework diagram of the method of the present invention; Figure 2This is a schematic diagram of the spatial projection of a Gaussian volume and the mapping of two-dimensional parameters; Figure 3 It is a spectrum signature calculation and energy distribution structure diagram; Figure 4 This is a flowchart of signature quantization and encoding; Figure 5 This is a schematic diagram of pruning and merging based on spectral importance; Figure 6 This is a comparison chart of the Gaussian volume distribution and rendering effect before and after compression. Detailed Implementation

[0015] A 3D Gaussian splash compression method based on Gaussian volume parameter layer frequency domain signature first obtains a set of Gaussian volumes from the 3D scene reconstruction or training results as input. These Gaussian volumes collectively constitute the 3D representation of the scene, used to fully describe the distribution characteristics of each point in 3D space. Each Gaussian volume contains parameter information such as its spatial location, shape, orientation, color, and transparency. Then, the model sequentially passes through a Gaussian parameter projection module, a spectrum signature calculation module, a signature quantization and encoding module, a spectrum-driven pruning and merging module, and a verification and backoff module. Finally, a compressed 3D Gaussian model is output. This model significantly reduces storage and computational burden while preserving the original scene details and texture features, thus improving rendering efficiency. The final result can be directly applied to 3D reconstruction, virtual reality, augmented reality, and digital twin scenarios, achieving efficient real-time rendering and scalable 3D representation. Its main structure is as follows: Figure 1 As shown.

[0016] The Gaussian parameter projection module maps the input 3D Gaussian volume to a 2D imaging plane using camera projection relationships, obtaining equivalent 2D Gaussian parameters. This process maintains the consistency of the Gaussian volume's shape and scale across different viewpoints and ensures that spatial geometric features are correctly projected. Simultaneously, it preserves the color and transparency information corresponding to each Gaussian volume, allowing the 2D parameters to accurately reflect its visual attributes. The module outputs a set of 2D Gaussian parameters suitable for frequency domain analysis.

[0017] like Figure 2 As shown, the Gaussian parametric projection module takes a set of three-dimensional Gaussian volumes as input. Each Gaussian body is determined by its central position vector. Covariance Matrix Represented by the camera projection matrix. And Jacobi matrix Calculate the equivalent two-dimensional parameters: , .in, It consists of the partial derivatives of the projection transformation. This process ensures the consistency of the shape, scale, and orientation of the Gaussian body on the two-dimensional imaging plane, and preserves its spatial structural features.

[0018] The Gaussian parametric projection module uses a matrix batch calculation method to accelerate parameter mapping and stores the results in a compact structure. , Indicates color, Indicates transparency.

[0019] The spectral signature calculation module uses the analytic Fourier transform of a 2D Gaussian to calculate the energy distribution and phase characteristics of each Gaussian volume across different frequency ranges. Based on the characteristics of the Gaussian function in the frequency domain, the entire frequency range is divided into low-frequency, mid-frequency, and high-frequency intervals to describe the overall structure, local texture, and detail variations of the scene. By analyzing the energy proportion and phase distribution of each frequency band, a fixed-length spectral signature is generated to represent the characteristics of the Gaussian volume in the frequency domain. This spectral signature, as a compact description at the parameter level, reflects visual information and provides a basis for subsequent compression.

[0020] like Figure 3 As shown, the spectrum signature calculation module is used to perform an analytical Fourier transform on the two-dimensional Gaussian function to obtain the frequency domain energy distribution and phase information of each Gaussian. Based on the Fourier properties of the Gaussian function, the spectrum analytical expression is: .in, This represents the Fourier transform of the Gaussian function in the frequency domain, i.e., the spectrum function; A frequency vector, describing the frequency domain coordinates; The covariance matrix is ​​used to determine the attenuation rate of the spectrum in each direction. The Gaussian center position vector controls the phase distribution of the spectrum; For constant terms, superscript This indicates the transpose operation.

[0021] This formula shows that the Gaussian function still maintains a Gaussian distribution in the frequency domain, its decay rate is determined by the covariance, and the phase part of the spectrum is controlled by the position term.

[0022] Frequency distribution range Divided into low-frequency bands Mid-frequency band and high frequency band , This is the upper limit of frequency. The boundary frequency between low and mid frequencies. These are the boundary frequencies between the mid-frequency and high-frequency ranges. The boundary values ​​for each frequency band are determined based on the covariance spectrum energy distribution of the Gaussian body in space and its frequency domain attenuation characteristics. Specifically, the Gaussian body at the [missing information]th [missing information] frequency... Each frequency band Energy components within This is used to measure its energy intensity in that frequency band. Indicates the first The Fourier transform function of a two-dimensional Gaussian in the frequency domain, that is, the Gaussian in the frequency domain. Spectral distribution at that location.

[0023] In most natural scenarios, approximately 70% of the Gaussian volume spectrum energy is distributed in the low-frequency region. 20% are distributed in the medium frequency range ( The remaining approximately 10% is distributed in the high-frequency region ( Therefore, this embodiment uses the following proportions for adaptive partitioning: , This ratio can be dynamically adjusted according to the energy spectrum of the actual scene to adapt to the spectral distribution of different scenes. This yields the spectral energy vector. ,in, Indicates the first The energy components of a Gaussian in the low-frequency range correspond to the overall outline and brightness of the scene. It represents the energy components in the mid-frequency region, reflecting structural and texture information; It represents its high-frequency energy components, characterizing details and edge features; phase term This reflects the influence of the Gaussian center position on the spectral phase. This four-dimensional eigenvector... This is the spectral signature for each Gaussian body, used for subsequent quantization and compression analysis.

[0024] The signature quantization and encoding module performs frequency band division, quantization, and bit depth allocation on the spectral signature to form a compact encoded representation, achieving the first compression, namely parameter representation layer compression. This step compresses the spectral signature obtained in the previous step, achieving the first compression at the parameter layer. First, the system quantizes the energy values ​​of each frequency band to reduce data precision redundancy. Then, different bit widths are allocated according to the importance of different frequency bands to improve the overall compression ratio while ensuring the accuracy of high-frequency information. Finally, the quantized signature is entropy encoded or vector quantization encoded to form a compact data representation structure. After this step, the original high-dimensional Gaussian parameters are converted into low-storage, highly decodable encoded data.

[0025] Signature quantization and encoding module, such as Figure 4 As shown, after the spectral signature is generated, it needs to be quantized and encoded to achieve the first compression. The quantization process includes two steps: (1) Band quantization: Log-uniform quantization is adopted according to the energy range to reduce the accuracy redundancy caused by high dynamic range.

[0026] (2) Bit depth allocation: Different bit widths (e.g., 4 bits, 6 bits, 8 bits) are allocated to high frequency, medium frequency and low frequency components respectively, so as to balance high frequency sensitivity and compression ratio.

[0027] During the encoding stage, Huffman or Vector Quantization (VQ) techniques are used to further compress the data volume. The quantized signature is stored in a 16-24 byte structure, achieving a compression ratio of 3-4 times.

[0028] The spectrum-driven pruning and merging module calculates the importance score of each Gaussian based on its signature. It prunes low-importance Gaussians and merges and replaces multiple Gaussians with similar spectra, achieving a second compression, i.e., structured compression. This step is the core of the structured layer compression. Based on the spectral signature of each Gaussian, its importance score in the reconstruction process is calculated. This score comprehensively considers high-frequency and mid-frequency energy components as well as reconstruction error estimation to evaluate the contribution of the Gaussian to the overall rendering quality. When the importance score is below a set threshold, the system deletes the corresponding Gaussian, thus pruning redundant data. For multiple Gaussians with high spectral feature similarity, the system performs a merging operation, generating new representative Gaussians in an energy-weighted manner. Through this module's processing, the system effectively reduces the number of Gaussians, achieving structured compression.

[0029] The spectrum-driven pruning and merging module is the core of the second-stage compression, determining whether to retain or delete Gaussian bodies based on spectral importance. For example... Figure 5 As shown, calculate the Gaussian importance index: .in, , These represent the energy components of the Gaussian in the high-frequency and mid-frequency ranges, respectively. This is the reconstruction error estimation term, used to measure the impact of the Gaussian on the overall reconstruction quality after compression; , , This is the weighting parameter. Based on the sensitivity analysis results of different frequency bands in the reconstruction error, it is set to... , , .

[0030] The design of this indicator is based on the following: the high-frequency part corresponds to scene details and contributes the most to perceived quality; the mid-frequency part is important for maintaining structure; and the low-frequency part mainly affects overall brightness and shape, so it has a smaller weight in the compression stage.

[0031] When Gaussian importance index ( When setting a threshold, if a Gaussian is deemed to have low importance, it is pruned. If the cosine similarity of the spectral signatures of multiple Gaussians is higher than 0.95, they are merged to generate a new representative Gaussian. .in, Represents Gaussian body Total energy across the entire frequency domain, These are energy normalization weighting coefficients used to ensure that the merged Gaussians maintain consistency in energy distribution. This mechanism achieves adaptive structured compression based on spectral distribution, making the compression process physically interpretable.

[0032] The verification and rollback module performs rapid rendering or frequency domain error estimation on the compressed model. If the verification finds that the reconstruction error exceeds the limit, the deleted Gaussian volume or local fine-tuning parameters are restored according to preset rules to correct the error. After pruning and merging, the system performs rapid verification of the compressed model. The verification process includes frequency domain error detection and image-level rendering quality assessment. The system calculates the differences in brightness, structure, and detail of the model before and after compression and analyzes the impact of compression. When the detection results show that the overall error or local error exceeds the preset threshold, the system activates the rollback mechanism. This mechanism can restore the deleted Gaussian volume or correct the model through local parameter fine-tuning, thereby ensuring that the compressed model still has high fidelity.

[0033] This step assesses and processes the validation results from the previous stage. When the reconstruction error exceeds the limit, the system selects a correction strategy based on the error distribution. For regions with concentrated errors, the system prioritizes restoring the pruned Gaussian volume; for parts with smaller errors that still affect visual quality, the system corrects them by fine-tuning parameters. This correction process is performed in both the frequency and spatial domains, enabling the model to maintain visual consistency while preserving its compression ratio.

[0034] After pruning, the model enters the verification phase, where frequency domain errors and rendering errors are detected. For example... Figure 6 As shown: 1. Perform fast rendering and calculate the PSNR and SSIM of the images before and after compression; 2. If the error in any local area exceeds the set threshold (e.g., PSNR decrease > 1.5dB), then the Gaussian body that was pruned in the corresponding area is restored; 3. Simultaneously, update the local frequency band energy distribution to correct subsequent compression strategies.

[0035] The verification and rollback modules function like a "loop self-healing mechanism" at the algorithm level, effectively avoiding sudden changes in image quality caused by excessive pruning. This invention's method, by introducing frequency domain feature description and pruning mechanisms into the Gaussian parameter layer, achieves efficient compression and fast rendering of 3D Gaussian Splatting representations, combining high fidelity, low storage overhead, and good scalability, making it suitable for various 3D vision tasks and real-time rendering systems.

[0036] After verification and rework are completed and the error meets the set threshold, the system outputs the final compressed 3D Gaussian model. This model significantly reduces storage and computational burden while preserving the original scene details and texture features, thus improving rendering efficiency. The final result can be directly applied to scenarios such as 3D reconstruction, virtual reality, augmented reality, and digital twins, achieving efficient real-time rendering and scalable 3D representation.

Claims

1. A 3D Gaussian sputtering compression method based on Gaussian volume parameter layer frequency domain signature, characterized in that: First, a set of Gaussian volumes is obtained from the 3D scene reconstruction or training results as input. The Gaussian volumes together constitute the 3D representation of the scene, which is used to fully describe the distribution characteristics of each point in the 3D space. Then, it sequentially passes through the Gaussian parameter projection module, the spectrum signature calculation module, the signature quantization and encoding module, the spectrum-driven pruning and merging module, and the verification and rollback module, finally outputting a compressed 3D Gaussian model. The Gaussian parameter projection module maps the input three-dimensional Gaussian volume to a two-dimensional imaging plane through camera projection relationship, obtains equivalent two-dimensional Gaussian parameters, and outputs a set of two-dimensional Gaussian parameters suitable for frequency domain analysis. The aforementioned spectrum signature calculation module is based on the analytical Fourier transform of a 2D Gaussian to calculate the energy distribution and phase characteristics of each Gaussian body in different frequency ranges. Based on the characteristics of the Gaussian function in the frequency domain, the entire frequency range is divided into three intervals: low frequency, mid frequency, and high frequency. By analyzing the energy proportion and phase distribution of each frequency band, a fixed-length spectral signature is generated to represent the characteristics of the Gaussian body in the frequency domain. The signature quantization and encoding module compresses the obtained spectral signature to achieve the first compression of the parameter layer. First, the energy value of each frequency band is quantized. Then, different bit widths are allocated according to the importance of different frequency bands. Finally, the quantized signature is entropy encoded or vector quantization encoded to form a compact data representation structure. The spectrum-driven pruning and merging module calculates the importance score of each Gaussian based on its signature, prunes low-importance Gaussians, and merges and replaces multiple Gaussians with similar spectra to achieve a second compression, namely structured compression. It calculates the importance score of each Gaussian in the reconstruction process based on its spectral signature. When the importance score is below a set threshold, the corresponding Gaussian is deleted. For multiple Gaussians with high spectral feature similarity, a merging operation is performed. The verification and rollback module calculates the differences in brightness, structure, and detail of the model before and after compression. When the detection results show that the overall error or local error exceeds the preset threshold, the rollback mechanism is activated.

2. The 3D Gaussian sputtering compression method based on Gaussian volume parameter layer frequency domain signature as described in claim 1, characterized in that: The Gaussian parameter projection module takes as input a set of three-dimensional Gaussian volumes. Each Gaussian body is determined by its central position vector. Covariance Matrix Representation; via camera projection matrix And Jacobi matrix Calculate the equivalent two-dimensional parameters: , The Gaussian parameter projection module employs a matrix batch calculation method, storing the results uniformly as a compact structure. , Indicates color, Indicates transparency.

3. The 3D Gaussian sputtering compression method based on Gaussian volume parameter layer frequency domain signature as described in claim 1, characterized in that: The aforementioned spectrum signature calculation module is used to perform an analytical Fourier transform on a two-dimensional Gaussian, and the spectrum function... , This represents the Fourier transform result of the Gaussian function in the frequency domain. A frequency vector, describing the frequency domain coordinates; The covariance matrix is ​​used to determine the attenuation rate of the spectrum in each direction. The Gaussian center position vector controls the phase distribution of the spectrum; For constant terms, superscript Indicates the transpose operation; Frequency distribution range Divided into low frequency bands Mid-frequency band and high frequency band , This is the upper limit of frequency. The boundary frequency between low and mid frequencies. The boundary frequency between the intermediate and high frequencies; Gaussian frequency at the 1st... Each frequency band Energy components within , Indicates the first The Fourier transform function of a two-dimensional Gaussian in the frequency domain, that is, the Gaussian in the frequency domain. Spectral distribution at; This yields the spectral energy vector. , Indicates the first The energy components of a Gaussian in the low-frequency range correspond to the overall outline and brightness of the scene. It represents the energy components in the mid-frequency region, reflecting structural and texture information; It represents its high-frequency energy components, characterizing details and edge features; Phase Term This four-dimensional eigenvector reflects the influence of the Gaussian center position on the spectral phase. That is, the spectral signature of each Gaussian body.

4. The 3D Gaussian sputtering compression method based on Gaussian volume parameter layer frequency domain signature as described in claim 3, characterized in that: ， 。 5. The 3D Gaussian sputtering compression method based on Gaussian volume parameter layer frequency domain signature as described in claim 1, characterized in that: The signature quantization and encoding module includes a quantization process that includes band quantization and bit depth allocation. Band quantization is performed using logarithmic uniform quantization based on the energy range. Bit depth allocation assigns different bit widths to high-frequency, mid-frequency, and low-frequency components. The encoding stage uses Huffman or vector quantization techniques to further compress the data volume. The quantized signature is stored in a 16-24 byte structure, achieving a compression rate of 3-4 times.

6. The 3D Gaussian sputtering compression method based on Gaussian volume parameter layer frequency domain signature as described in claim 1, characterized in that: The spectrum-driven pruning and merging module determines whether to retain or delete Gaussian bodies based on their spectral importance, first calculating the Gaussian importance index: , , These represent the energy components of the Gaussian in the high-frequency and mid-frequency ranges, respectively. To reconstruct the error estimation term, , , For the weighting parameters; when the Gaussian importance index Less than the set threshold If the spectral signature cosine similarity of multiple Gaussians is higher than 0.95, then they are merged to generate a new representative Gaussian. , Represents Gaussian body Total energy across the entire frequency domain, This is the energy normalization weighting coefficient.

Images