Point Cloud Attribute Coding With Frequency-Domain Motion Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video-based compression techniques for point cloud attribute data are inefficient due to non-uniform sampling, leading to ineffective motion compensation and high computing costs, unsuitable for real-time processing in metaverse-based applications.
Innovation Solution
A two-stage motion compensation scheme combining spatial-domain motion prediction and frequency-domain motion compensation using graph signal processing techniques, including block tree partitioning, iterative closest point search, and graph Fourier transform to efficiently encode and reconstruct 3D virtual objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional video-based compression techniques are applied to point cloud attribute data, then encoding simplicity is improved, but motion compensation effectiveness deteriorates due to non-uniform sampling
Solution Approach 1:
The patent transforms point cloud attribute data from spatial domain to frequency domain using Graph Fourier Transform, changing the representation parameters from spatial coordinates to frequency coefficients. This allows motion compensation to operate on frequency-domain residuals that better capture the correlated variations in non-uniformly sampled data, resolving the contradiction between encoding simplicity and motion compensation effectiveness
Solution Approach 2:
The patent introduces frequency-domain residuals as an intermediary representation between the reference frame and current frame. Instead of directly comparing non-uniformly sampled spatial data, the system computes residuals in the frequency domain after motion estimation, which serves as a mediator that preserves temporal coherence while accommodating non-uniform sampling patterns
2Stability of the object's composition
If direct 3D motion compensation in temporal domain is applied, then temporal coherence is improved, but computing cost increases prohibitively
Solution Approach 1:
The patent replaces direct spatial-domain pointwise comparison with a frequency-domain approach using Graph Fourier Transform. Instead of mechanically comparing each point in 3D space between frames (which is computationally expensive), the system transforms the data to frequency domain where motion compensation can be performed more efficiently on spectral coefficients, reducing computing cost while preserving temporal coherence
Solution Approach 2:
The patent moves the motion compensation operation from the spatial dimension to the frequency dimension. By applying Graph Fourier Transform, the system represents point cloud data in terms of frequency components rather than spatial positions, enabling temporal coherence to be maintained through frequency-domain residual coding without the prohibitive computational cost of direct 3D pointwise comparison
3Device complexity
If 2D projected patches are used for motion compensation, then processing complexity is reduced, but temporal coherence is lost resulting in ineffective motion prediction
Solution Approach 1:
The patent maintains 3D spatial relationships by performing motion compensation in the frequency domain derived from 3D point cloud data, rather than projecting to 2D. The Graph Fourier Transform operates on the 3D graph structure of the point cloud, preserving spatial topology and temporal coherence while enabling efficient processing through spectral analysis, thus avoiding the loss of temporal coherence inherent in 2D projection methods
Data Source
AI summary
Systems and methods are disclosed herein for temporally predictive coding of three-dimensional (3D) dynamic point cloud attributes. A first frame and a second frame of point cloud data are accessed. The point cloud data points include 3D spatial coordinates and one or more graphic attributes. A block tree data structure comprising a plurality of blocks is generated based on a tree partitioning of the second frame of point cloud data. Matching block pairs between the first frame and the second frame are identified from the plurality of blocks based on block-wise searching. Frequency-domain projections are generated for each matching block pair via a graph Fourier transform (GFT) algorithm. A bitstream of motion-compensated residuals is generated based on differences in the frequency-domain projections for each matching block pair.