3D Gaussian Splatting Compression With Graph Fourier Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D Gaussian Splatting (3DGS) data requires significant memory due to millions of Gaussian primitives, and current compression methods focus on training optimization rather than post-training compression, which is inefficient for storage and transmission.
Innovation Solution
Implementing lossless and lossy compression techniques using geometry-based point cloud compression (G-PCC) frameworks, occupancy tree encoding, transform coding, and block-based graph Fourier transforms to compress 3DGS geometry and attributes, with edge weights computed via KL-divergence, and pre-processing for effective signal compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3DGS uses millions of Gaussian primitives for high-quality rendering, then rendering quality is improved, but memory requirements increase significantly
Solution Approach 1:
The patent segments the 3DGS data into geometry components (positions, scales, rotations) and attribute components (base colors, spherical harmonics coefficients, opacities), allowing independent compression of each part. This segmentation enables selective compression strategies that maintain rendering quality while reducing memory usage.
Solution Approach 2:
The patent applies parameter changes by transforming spherical harmonics coefficients through graph Fourier transform and adjusting the representation parameters of Gaussian primitives. This changes the data representation to achieve better compression ratios while preserving the essential information needed for high-quality rendering.
2Quantity of substance
If existing compression methods prune redundant Gaussians during training, then model size is reduced, but compression efficiency is limited
Solution Approach 1:
The patent performs preliminary actions by pre-processing the 3DGS data before compression, including color space conversion and spherical harmonics coefficients reduction. This preparation step optimizes the data structure to enable more efficient compression algorithms to work effectively.
Solution Approach 2:
The patent introduces an intermediary compression framework that bridges the gap between raw 3DGS data and final compressed representation. This intermediary layer includes transform coding and graph-based compression operations that efficiently reduce data size without losing essential rendering information.
3Quantity of substance
If traditional signal compression is applied to 3DGS, then compression is achieved, but rendering quality loss occurs
Solution Approach 1:
The patent applies local quality by treating different components of 3DGS data with different compression strategies based on their importance. Geometry components receive one type of compression while attribute components receive another, with the graph Fourier transform applied selectively to preserve local characteristics that are critical for rendering quality.
Solution Approach 2:
The patent uses composite compression techniques that combine multiple compression methods (occupancy tree coding for positions, transform coding for attributes, graph Fourier transform for spherical harmonics) to create a hybrid compression system that maintains rendering quality while achieving efficient compression.
Data Source
AI summary
Post training compression of 3DGS data is agnostic to training in a traditional signal compression perspective. Gaussian parameters are treated as signals. Pre-processing and transform coding techniques are used to compress the signals effectively. Firstly, lossless/lossy compression is performed on 3DGS geometry (positions, scales, rotations) using a point cloud coding-based (e.g., G-PCC, GeS) framework. Positions are compressed using occupancy tree coding. Scales and rotations are encoded as attributes using transform coding. The widely used block-based graph Fourier transform (GFT) is used to compress the attributes (base colors, spherical harmonic coefficients and opacities). In addition, a graph construction strategy is used for 3DGS data that computes the edge weights based on similarity (or dissimilarity) between the 3D Gaussian distributions using KL-divergence. Alternatively, positions can be encoded using occupancy tree (e.g., G-PCC, GeS) or AI-based PCC methods, and any subset of Gaussian parameters or the transformed coefficients of Gaussian parameters can be mapped into 2D frames and encoded by video coders.