ISOBMFF VR Metadata Signaling for Distortion-Free Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current file formats like ISOBMFF do not include specifications for storing virtual reality (VR) video, leading to VR content being treated as conventional planar video, resulting in distorted rendering when played by devices not capable of handling VR content.
Innovation Solution
Modifications and additions to ISOBMFF are introduced to indicate the presence of VR content, using new boxes and metadata to signal VR-specific parameters, such as the 'mvvp' sample entry for timed metadata tracks, to differentiate VR content and ensure proper rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If VR content is stored using conventional file formats like ISOBMFF without VR-specific specifications, then the file format remains compatible with existing devices and simple to implement, but the VR content is rendered as conventional planar video resulting in distorted rendering
Solution Approach 1:
The patent embeds VR-specific metadata boxes (such as 'vr' box with viewport information, 'omnidirectional' box with projection parameters) within the existing ISOBMFF file structure. These nested boxes contain VR-specific parameters like field of view, projection type, and viewport coordinates, allowing VR content to be properly identified and rendered while maintaining the overall ISOBMFF framework that existing devices can recognize
Solution Approach 2:
The patent pre-encapsulates VR-specific parameters and metadata within the file structure during the encoding phase. By embedding viewport information, projection parameters, and VR content identification markers in advance within the file boxes, the content is prepared for correct VR rendering before playback, eliminating the need for real-time conversion or complex runtime processing
2Reliability
If VR-specific metadata and boxes are added to ISOBMFF to enable proper VR content identification, then VR content can be rendered correctly, but the file format complexity increases
Solution Approach 1:
The patent divides VR-specific information into separate, modular metadata boxes (such as individual boxes for viewport parameters, projection information, and content identification) that can be independently processed. This segmentation allows player devices to selectively parse only the VR-related boxes when VR content is detected, rather than processing the entire file structure, thus managing complexity through modularity
3Adaptability or versatility
If new boxes and metadata are introduced to signal VR-specific parameters, then VR content differentiation is enabled, but the ease of operation decreases due to increased processing requirements
Solution Approach 1:
The patent designs the VR metadata boxes to use standardized ISOBMFF box structures and syntax that are already recognized by conventional players. The VR-specific boxes follow the same hierarchical organization and data formatting as existing ISOBMFF elements, allowing a single file format implementation to serve both conventional and VR content playback without requiring separate processing paths or complex device-specific extensions
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
In various implementations, modifications and/or additions to the ISOBMFF can indicate that a file that has been formatted according to the ISOBMFF, or a format derived from the ISOBMFF, includes virtual reality content. The file can include a restricted scheme information box, written into a track box in the file. The restricted scheme information box can indicate a virtual reality scheme for the contents of the track. For example, a signaling mechanism may indicate a most-viewed viewport associated with the virtual reality data.