Point Cloud Viewpoint Scalable Compression via 3D Motion Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Six degree of freedom (6DoF) video rendering is computationally expensive and requires large storage and transmission capacities due to the complexity of point cloud data, making it challenging to render high-quality immersive video experiences efficiently.
Innovation Solution
A processing system with a graphics processor that employs multiresolution streaming, hierarchical compression, and point cloud data decomposition to reduce computational load and storage requirements, enabling efficient rendering and transmission of 6DoF video by using techniques such as 3D motion vectors and spatial transforms for compression and prioritizing data based on viewer attention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If point cloud data is used to represent 6DoF video, then immersive video experience with viewpoint control is achieved, but computational cost for rendering increases significantly
Solution Approach 1:
The point cloud data is divided into multiple resolutions or levels of detail. Coarser resolutions are used for background or less important regions, while finer resolutions are applied to foreground or viewer attention regions. This segmentation allows the system to maintain viewpoint control adaptability while reducing the overall computational burden by not processing all points at maximum detail levels.
Solution Approach 2:
Different quality levels or point densities are applied to different spatial regions within the point cloud. Regions that are more likely to be in the viewer's field of view or contain important objects are rendered with higher point density and quality, while peripheral or less important regions use lower point density. This local quality approach maintains immersive experience where needed while reducing computational cost in other areas.
2Reliability
If point cloud data is used to represent 6DoF video, then complete scene representation is achieved, but storage capacity and transmission bandwidth requirements increase
Solution Approach 1:
The system dynamically adjusts the point cloud data rate and quality based on viewer position, viewing angle, and attention focus. As the viewer moves or changes viewpoint, the system adaptively allocates bandwidth to transmit only the necessary portions of the point cloud at appropriate resolutions. This dynamic adaptation maintains complete scene representation reliability while optimizing data rate to match actual viewing requirements rather than transmitting all data at maximum quality.
Solution Approach 2:
Instead of transmitting or storing the complete high-resolution point cloud for all possible viewpoints, the system transmits partial point cloud data that is sufficient for the current or predicted viewer position. The transmitted data includes only the necessary portions at appropriate resolutions, using partial action to reduce data rate while maintaining adequate scene representation for the actual viewing experience.
3Productivity
If high frame rates are targeted for 6DoF video rendering, then smooth immersive experience is achieved, but rendering speed requirements become more demanding
Solution Approach 1:
The system performs preliminary processing of the point cloud data during encoding or in advance of rendering, such as pre-computing geometric structures, organizing points into efficient data structures, or pre-identifying important regions. This preliminary action reduces the computational workload during actual real-time rendering, enabling the system to achieve high frame rates by having critical processing completed beforehand rather than during the time-critical rendering phase.
Data Source
AI summary
An apparatus comprises a processor to divide a first point cloud data set frame representing a three dimensional space at a first point in time into a matrix of blocks, determine at least one three dimensional (3D) motion vector for at least a subset of blocks in the matrix of blocks, generate a predicted second point cloud data set frame representing a prediction of the three dimensional space at a second point in time by applying the at least one 3D motion vector to the subset of blocks in the matrix of blocks, compare the predicted second point cloud data set frame to a second point cloud data set frame representing a prediction of the three dimensional space at a second point in time to generate a prediction error parameter, and encode the second point cloud data set frame as a function of the first point cloud data set frame and the at least one three dimensional (3D) motion vector when the prediction error factor is beneath an error threshold to produce an encoded second point cloud data set frame.