Temporal Track Derivations for 3D Video Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies for processing 3D spherical content, such as omnidirectional video, are computationally intensive and bandwidth-consuming, leading to difficulties in preserving user experience when resources are limited, as they often require processing and delivering the entire spherical content regardless of the user's viewport.
Innovation Solution
The implementation of temporal track derivations using edit operations within the ISO Base Media File Format (ISOBMFF) allows for the specification and application of operations on multimedia data to generate derived tracks that only include necessary multimedia data, reducing bandwidth and processing requirements by selectively removing or modifying media units based on user viewport changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the entire spherical content is processed and delivered, then the user can view content at any viewport, but the bandwidth consumption and processing burden increase significantly
Solution Approach 1:
The spherical content is divided into multiple viewport-specific segments or regions. Instead of delivering the entire spherical content, the system segments the content according to different possible viewports and delivers only the relevant segments based on the user's actual viewing direction, thereby reducing bandwidth consumption while maintaining viewport flexibility.
Solution Approach 2:
The system applies different quality levels or processing depths to different regions of the spherical content based on the user's viewport. The region corresponding to the user's current viewing direction is processed with high quality, while other regions are either not processed or processed with lower quality, optimizing bandwidth usage while preserving the user experience.
2Reliability
If the entire spherical content is processed and delivered, then complete content coverage is achieved, but the processing time and computational resources increase
Solution Approach 1:
The system extracts only the necessary portions of the spherical content that are relevant to the user's current viewport, rather than processing the entire content. This extraction approach maintains content completeness for the user's perspective while dramatically improving processing efficiency by avoiding unnecessary computation on irrelevant regions.
Solution Approach 2:
The system performs partial processing on the spherical content, focusing computational resources only on the regions that will be viewed by the user. This partial action approach achieves sufficient content completeness for the user's viewport while significantly reducing processing time and computational resource requirements.
3Loss of energy
If viewport-dependent processing is implemented, then bandwidth consumption is reduced, but the system complexity increases
Solution Approach 1:
The system performs preliminary actions by pre-organizing the spherical content into viewport-specific segments or regions before delivery. This preliminary organization enables the client to efficiently select and process only the relevant content based on the user's viewport, reducing bandwidth consumption while managing system complexity through structured data organization rather than complex real-time processing.
Data Source
AI summary
The techniques described herein relate to methods, apparatus, and computer readable media configured to access multimedia data comprising a hierarchical track structure with a first track of a first sequence of temporally-related media units at a first level, and a second track at a second level comprising metadata specifying a temporal track derivation operation. The metadata includes a set of one or more operations to perform on the first track, each operation including a unit duration of the first sequence and a start unit in the first sequence. The temporal track derivation operation is performed on a set of media units comprising at least the first sequence, and includes applying the set of one or more operations to temporally modify the first sequence to generate second media data for the second track that includes a second sequence of temporally-related media units from the set of media units.