Point Cloud Viewpoint Prediction for Bandwidth-Efficient 6DoF Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The rendering of point cloud data for immersive video is computationally expensive and requires large data rates, with conventional techniques transmitting unnecessary data without considering the user's perspective or relevance.
Innovation Solution
Adaptive resolution of point cloud and viewpoint prediction to stream only relevant data based on the user's perspective, predicting viewer motion for subsequent frames and ensuring viewport-dependent 6DoF video streaming.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If point cloud data is transmitted with high resolution and full detail, then video quality and fidelity are improved, but bandwidth consumption and data transmission requirements increase significantly
Solution Approach 1:
The patent applies local quality by transmitting point cloud data at different resolutions based on spatial location and viewer perspective. Points within the viewer's current viewport are transmitted at high resolution, while points outside the viewport are transmitted at lower resolution or omitted entirely. This selective quality adjustment maintains video fidelity where needed while reducing overall bandwidth consumption.
Solution Approach 2:
The patent implements partial action by transmitting only the portion of point cloud data that is relevant to the current viewer perspective rather than the complete dataset. Viewpoint prediction algorithms identify which points will be visible in upcoming frames, and only those points are transmitted at full detail, leaving out unnecessary data that would not be perceived by the viewer.
2Loss of information
If all point cloud data is transmitted without selection, then complete information is provided, but irrelevant and unnecessary data increases transmission load
Solution Approach 1:
The patent applies preliminary action through viewpoint prediction that occurs before data transmission. The system predicts the viewer's future viewport position based on current motion trends, then pre-identifies which points will be needed in subsequent frames. This allows the system to transmit only the relevant subset of points in advance, avoiding the transmission of information that will not be used.
Solution Approach 2:
The patent implements feedback mechanisms where the system continuously monitors viewer position, motion patterns, and viewport changes. This feedback information is used to dynamically adjust which points are transmitted and at what resolution, ensuring that information completeness is maintained for visible regions while eliminating transmission of information that the viewer will not perceive.
3Manufacturing precision
If point cloud rendering is performed with high detail, then immersive video quality is improved, but computational cost and processing requirements increase
Solution Approach 1:
The patent applies local quality in rendering by allocating computational resources selectively. Points within the current and predicted viewports are rendered with high detail using full point cloud processing, while points outside these regions are rendered at lower detail or skipped entirely. This spatially-variable rendering approach maintains immersive video quality in visible areas while dramatically reducing overall computational cost.
Data Source
AI summary
A mechanism is described for facilitating adaptive resolution and viewpoint-prediction for immersive media in computing environments. An apparatus of embodiments, as described herein, includes one or more processors to receive viewing positions associated with a user with respect to a display, and analyze relevance of media contents based on the viewing positions, where the media content includes immersive videos of scenes captured by one or more cameras. The one or more processors are further to predict portions of the media contents as relevant portions based on the viewing positions and transmit the relevant portions to be rendered and displayed.