Omnidirectional Media Tile Segmentation for Bandwidth Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current omnidirectional media streaming technologies face challenges in bandwidth management due to lack of fine-grained calculation of required and available bandwidth, leading to bandwidth spikes and throttling, especially in viewport-dependent streaming where tile segment qualities vary significantly, and there is no mechanism to signal segment priority or sizes.
Innovation Solution
A method to signal and store segment properties such as sizes and priority ranks in omnidirectional media, allowing media players to access and encode this information in metadata tracks, enabling better bandwidth allocation and quality prioritization during streaming.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If viewport-dependent streaming is used to enable multiple 3DoF and 3DoF+ content consumption, then adaptability and user experience are improved, but bandwidth management becomes more complex and bandwidth spikes occur due to lack of fine-grained calculation
Solution Approach 1:
The patent segments the omnidirectional content into multiple independent tiles, where each tile can be encoded and streamed separately with different quality levels. This segmentation enables fine-grained bandwidth management by allowing the client to selectively download only the necessary tiles based on available bandwidth, preventing bandwidth spikes while maintaining adaptability.
Solution Approach 2:
The patent applies preliminary action by signaling the sizes of all tile segments in advance through metadata tracks before the actual streaming occurs. This allows the client to perform look-ahead measurements and calculate required bandwidth beforehand, enabling proactive bandwidth management decisions rather than reactive responses to bandwidth spikes.
2Manufacturing precision
If tile segment sizes are allowed to vary by large extent to optimize quality, then manufacturing precision and quality are improved, but bandwidth management deteriorates due to inability to calculate required bandwidth
Solution Approach 1:
The patent divides the omnidirectional content into multiple independent tiles that can be encoded at different quality levels and sizes. Each tile is treated as a separate segment with its own size information signaled in metadata, enabling both high quality precision for important tiles and efficient bandwidth management through selective downloading.
Solution Approach 2:
The patent implements feedback by providing the client with size information for each tile segment through metadata tracks. This feedback mechanism enables the client to calculate the total bandwidth required to download all necessary tiles and make informed decisions about quality selection and download prioritization based on available bandwidth.
3Reliability
If all tiles are downloaded at high quality to maximize user experience, then quality is improved, but bandwidth consumption increases and bandwidth throttling occurs
Solution Approach 1:
The patent applies local quality by allowing different tiles to be encoded and downloaded at different quality levels based on their importance and the user's viewing needs. Tiles in the current viewport can be downloaded at high quality while tiles outside the viewport are downloaded at lower quality or not at all, optimizing the balance between user experience and bandwidth consumption.
4Device complexity
If segment size information is not signaled in advance, then device complexity is reduced, but measurement precision deteriorates leading to inability to perform look-ahead measurements
Solution Approach 1:
The patent implements preliminary action by signaling the sizes of all tile segments in advance through metadata tracks before the actual streaming occurs. This allows the client to perform look-ahead measurements and calculate the total bandwidth required to download all necessary tiles, enabling precise bandwidth management decisions to be made before downloading begins.
Data Source
AI summary
A method, apparatus and computer program product to signal and store various segment properties in omnidirectional media, including sizes, priority ranks, and the like. The method, apparatus and computer program product access one or more media segments (30). The one or more media segments includes media data configured for streaming playback. The method, apparatus and computer program product access size information (32) or priority rank information of one or more tiles for each of the one or more media segments. The method, apparatus and computer program product encode the size information (34) or priority rank information for each of the one or more media segments in a media file within one or more metadata tracks associated with the one or more media segments or one or more sub-segments associated with the one or more media segments.