Immersive Media Streaming via Adaptive Tile Density
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for streaming mixed reality videos face challenges in efficiently transmitting high-immersive-quality content, particularly in handling user head position changes and objects with six Degrees of Freedom (DOF), leading to latency issues and unsuitable Region of Interest (ROI) calculations, which result in low-quality content delivery.
Innovation Solution
A method and system for adaptive streaming of immersive media that uses a dominance calculation module to determine the Region of Interest (ROI) based on user motion parameters, dynamically adjusting pixel density and tile selection to optimize content delivery, incorporating a server-client interaction for efficient bandwidth utilization and multidimensional media buffering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high bandwidth is used to maintain immersive quality throughout the frame, then image quality is improved, but bandwidth consumption increases
Solution Approach 1:
The patent applies local quality by differentiating image quality across different spatial regions. The viewport region (where the user is looking) receives high-quality rendering with full pixel density, while peripheral regions use lower quality with reduced pixel density. This resolves the contradiction by maintaining high image quality where needed while reducing overall bandwidth consumption through selective quality adjustment in different frame regions.
Solution Approach 2:
The patent segments the omnidirectional video frame into multiple tiles or regions, allowing independent quality control for each segment. The viewport-containing tiles are transmitted at high quality, while other tiles use lower quality settings. This segmentation approach enables targeted bandwidth allocation that improves perceived image quality without proportionally increasing total bandwidth consumption.
2Adaptability or versatility
If the user's head position changes frequently, then immersive experience is improved, but download latency increases
Solution Approach 1:
The patent implements preliminary action by pre-calculating and buffering video tiles for anticipated viewport positions before the user actually looks in those directions. The system predicts future head movements and prepares corresponding high-quality tiles in advance, so when the user turns their head, the required content is already available or nearly available, significantly reducing perceived download latency while maintaining adaptability to head position changes.
Solution Approach 2:
The patent applies dynamics by making the video delivery system adaptive to real-time head position changes. The system dynamically adjusts which tiles are prioritized for download and rendering based on current and predicted viewport positions. This dynamic adaptation allows the system to respond to frequent head movements without being constrained by fixed download schedules, reducing latency while preserving immersive experience.
3Measurement precision
If Region of Interest (ROI) calculations are performed for objects with six Degrees of Freedom (DOF), then accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent segments the complex 6DOF ROI calculation problem into manageable components by processing tiles independently. Instead of calculating ROI for the entire omnidirectional frame at once, the system divides the frame into multiple tiles and performs ROI calculations for each tile based on its relationship to the viewport. This segmentation reduces per-tile computational complexity while maintaining overall accuracy through coordinated processing of all tiles.
Solution Approach 2:
The patent applies partial action by focusing computational resources only on tiles that contain or are near the viewport region. Rather than performing exhaustive 6DOF ROI calculations for all tiles in the omnidirectional frame, the system identifies and processes only the relevant subset of tiles that contribute to the current view. This partial processing approach maintains ROI calculation accuracy for visible regions while significantly reducing overall computational complexity.
Data Source
AI summary
An immersive device and method for streaming of immersive media are provided. The method includes receiving body motion coordinates of a user in each of three dimensions and determining deviation in the body motion by a first immersive device. A dominant direction of motion based on the deviation calculated is then determined. A request by the first immersive device for at least one media frame to be displayed is received by a second immersive device. The received request is processed by the second immersive device and the media frame to be displayed is segmented into a plurality of tiles by the second immersive device. One tile with the first pixel density and a remainder of the tiles of the plurality of tiles with a second pixel density are stitched by the second immersive device to render one immersive media frame which is displayed to the user.


