3D Gaussian Splatting Inpainting for Point Cloud Compression Artifacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D data compression methods, such as the trisoup geometry encoder, result in rendering artifacts and memory inefficiencies due to the approximation of point clouds, particularly in applications like autonomous driving and virtual/augmented reality, where geometry continuity is not ensured and memory consumption is high.
Innovation Solution
Incorporating a 3D Gaussian splatting (3DGS) representation to inpaint missing data at the decoder side, using a selective generation and transmission of 3D Gaussian models based on size and opacity, combined with traditional trisoup-based encoded point cloud geometry to overcome rendering artifacts and reduce memory requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If traditional trisoup geometry encoder is used for 3D point cloud compression, then data compression is achieved, but rendering artifacts and geometry discontinuities occur
Solution Approach 1:
The patent introduces 3D Gaussian splatting as an intermediary representation between the compressed point cloud data and the final rendering output. The 3DGS model acts as a mediator that fills geometric gaps and smooths artifacts introduced by traditional compression methods, thereby maintaining both compression efficiency and rendering quality without requiring complete geometric fidelity in the compressed data
Solution Approach 2:
The patent combines two different representation methods - traditional trisoup geometry encoding and 3D Gaussian splatting - into a composite approach. The trisoup provides the base geometric structure while 3DGS adds complementary information to fill gaps and smooth artifacts, creating a hybrid representation that leverages the strengths of both methods to overcome their individual limitations
2Device complexity
If traditional point cloud representation is used, then data structure is simple, but memory consumption is high
Solution Approach 1:
The patent transforms the representation parameters of 3D points from traditional coordinate-based storage to 3D Gaussian model parameters (mean position, covariance, opacity). This parameter transformation enables more efficient memory utilization by representing groups of points with compact Gaussian parameters, reducing the overall data volume while maintaining or improving rendering quality
3Manufacturing precision
If 3D Gaussian splatting is used to inpaint missing data, then rendering quality improves, but data transmission complexity increases
Solution Approach 1:
The patent applies 3D Gaussian splatting selectively rather than to all points in the point cloud. By identifying and applying 3DGS only to regions with significant artifacts or geometric gaps, the method achieves improved rendering quality where needed while avoiding the overhead of processing entire datasets with the more complex representation, thus balancing quality improvement with transmission efficiency
Data Source
Figure 1
Figure 2A~2D
Figure 3
AI summary
In one implementation, a method of decoding point cloud data is provided. The point cloud data is decoded with a 3D data decompression method to form reconstructed point cloud data, and data indicative of a 3D Gaussian splatting representation is decoded and used to inpaint the reconstructed point cloud. On the encoder side, the point cloud data is encoded with a 3D data compression method. A 3D Gaussian splatting representation can be generated based on the point cloud data. The encoder then encodes data indicative of the 3D Gaussian splatting representation.