Point Cloud Attribute Coding With Graph-Frequency Motion Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video-based compression techniques for dynamic point cloud attribute coding are inefficient due to non-uniform sampling, leading to ineffective motion compensation and high computing costs, unsuitable for real-time processing in metaverse-based applications.
Innovation Solution
A two-stage motion compensation scheme combining spatial-domain motion prediction and frequency-domain motion compensation using graph signal processing techniques, including block tree partitioning, iterative closest point search, and graph Fourier transform to efficiently encode and reconstruct 3D virtual objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If direct 3D motion compensation in the temporal domain is applied to point cloud data, then motion compensation can be performed, but the non-uniform sampling requires the same number of points between matching partitions which is prohibitively costly in computing power and time
Solution Approach 1:
The patent transforms the point cloud attribute data from the spatial domain to the frequency domain using Graph Fourier Transform. This parameter transformation allows motion compensation to be performed on frequency coefficients rather than original point attributes, reducing the computational burden while maintaining compensation accuracy. The frequency domain representation enables efficient processing of non-uniformly sampled data without requiring equal point counts between partitions.
Solution Approach 2:
The patent replaces the direct spatial-domain pointwise comparison mechanism with a frequency-domain coefficient comparison mechanism. Instead of performing computationally intensive point-by-point matching in the spatial temporal domain, the system uses Graph Fourier Transform to convert data to frequency domain where motion compensation can be performed more efficiently on transformed coefficients, significantly reducing computing power and time requirements.
2Ease of manufacture
If 2D projected patches are used for motion compensation, then existing video-based compression techniques can be applied, but temporal coherence is lost resulting in most patches being coded in intraframe mode
Solution Approach 1:
The patent moves the processing from 2D projected patches back to 3D point cloud data by utilizing the spatial relationships in three dimensions. The Graph Fourier Transform operates on the 3D point cloud structure, preserving spatial and temporal coherence that would be lost in 2D projection. This dimensional approach allows interframe coding to be effective while maintaining the benefits of structured processing.
Solution Approach 2:
The patent introduces Graph Fourier Transform as an intermediary mechanism between the point cloud data and motion compensation processes. This transform acts as a bridge that converts spatial domain data to frequency domain coefficients, enabling efficient motion compensation while preserving temporal coherence. The frequency domain representation serves as an intermediate form that maintains relationships lost in direct 2D projection methods.
3Reliability
If pointwise comparison between partitions is applied, then motion compensation can be performed, but it is prohibitively costly in computing power and time
Solution Approach 1:
The patent changes the parameters being compared from original point attributes in spatial domain to frequency domain coefficients. This parameter transformation reduces the complexity of motion compensation by working with transformed data that captures essential motion information in a more compact and computationally efficient form, significantly reducing computing cost while maintaining accuracy.
Solution Approach 2:
The patent substitutes the direct pointwise comparison mechanism with a frequency-domain coefficient comparison mechanism. Instead of performing computationally expensive spatial domain matching, the system uses Graph Fourier Transform to convert data to frequency domain where motion compensation operates on coefficients rather than individual point attributes, dramatically reducing computing power and time requirements.
Data Source
AI summary
Systems and methods are disclosed herein for temporally predictive coding of three-dimensional (3D) dynamic point cloud attributes. A first frame and a second frame of point cloud data are accessed. The point cloud data points include 3D spatial coordinates and one or more graphic attributes. A block tree data structure comprising a plurality of blocks is generated based on a tree partitioning of the second frame of point cloud data. Matching block pairs between the first frame and the second frame are identified from the plurality of blocks based on block-wise searching. Frequency-domain projections are generated for each matching block pair via a graph Fourier transform (GFT) algorithm. A bitstream of motion-compensated residuals is generated based on differences in the frequency-domain projections for each matching block pair.