6DoF Point Cloud Streaming With Adaptive Resolution and Viewpoint Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The rendering of point cloud data for 6DoF video is computationally expensive and requires large data rates, leading to inefficiencies in storage and transmission, with conventional techniques often transmitting irrelevant data without considering the user's perspective.
Innovation Solution
Adaptive resolution of point cloud and viewpoint prediction techniques that predict the user's motion for subsequent frames, ensuring only relevant data is streamed, thereby reducing bandwidth usage and sending content with the highest possible fidelity based on throughput restrictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If point cloud data is transmitted with high resolution for 6DoF video, then video quality is improved, but data transmission bandwidth and storage capacity are excessively consumed
Solution Approach 1:
The patent applies local quality by transmitting point cloud data at different resolutions for different spatial regions. The viewport region is transmitted at high resolution to maintain video quality where the user is looking, while peripheral regions are transmitted at lower resolution to reduce bandwidth consumption. This selective resolution approach resolves the contradiction between video quality and bandwidth usage.
Solution Approach 2:
The system performs preliminary action by predicting the user's future viewpoint and motion trajectory before actually rendering or transmitting the corresponding point cloud data. By anticipating where the user will look next, the system can pre-process and prioritize transmission of relevant data regions, improving both video quality in the viewport and bandwidth efficiency by avoiding transmission of unnecessary peripheral data.
2Reliability
If conventional techniques transmit all point cloud data without perspective consideration, then complete scene data is provided, but irrelevant and unnecessary data is wasted
Solution Approach 1:
The patent extracts only the relevant portion of point cloud data that corresponds to the user's current and predicted future viewport. By separating and transmitting only this extracted subset rather than the complete scene data, the system maintains reliability for the visible portion while dramatically improving transmission efficiency by eliminating irrelevant data transmission.
Solution Approach 2:
The system applies dynamics by continuously adapting the transmitted point cloud data based on real-time user head position, orientation, and predicted motion. The data transmission is dynamic rather than static, adjusting the viewport region and resolution based on user perspective changes, which maintains scene completeness for the relevant area while optimizing transmission efficiency throughout the viewing experience.
3Speed
If high frame rates are achieved with point cloud rendering, then video smoothness is improved, but computational cost increases significantly
Solution Approach 1:
The patent applies partial action by rendering and transmitting only the point cloud data within the current and predicted future viewport regions rather than processing the entire scene. This partial processing approach enables achieving high frame rates for the visible content while significantly reducing computational cost by excluding peripheral and irrelevant regions from the rendering pipeline.
Data Source
AI summary
A mechanism is described for facilitating adaptive resolution and viewpoint-prediction for immersive media in computing environments. An apparatus of embodiments, as described herein, includes one or more processors to receive viewing positions associated with a user with respect to a display, and analyze relevance of media contents based on the viewing positions, where the media content includes immersive videos of scenes captured by one or more cameras. The one or more processors are further to predict portions of the media contents as relevant portions based on the viewing positions and transmit the relevant portions to be rendered and displayed.