Video Track Re-timing via Composition Timeline Skip and Shift
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional track re-timing approaches in ISOBMFF, such as using edit lists, are limited as they operate on the presentation timeline and cannot be further manipulated on the composition timeline, restricting their use in building hierarchies of derived tracks and performing subsequent transformations.
Innovation Solution
The implementation of re-timing derivation operations that allow for skipping and shifting of video media units on the composition timeline, enabling derived tracks to be used as inputs for further processing and supporting complex re-timing operations without relying on edit lists.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If edit lists are used for track re-timing in ISOBMFF, then re-timing operation can be performed on presentation timeline, but the derived track cannot be further manipulated on composition timeline and cannot be used in building hierarchies of derived tracks
Solution Approach 1:
The patent segments the re-timing operation into two independent parts: a skip operation that removes unwanted media units and a shift operation that adjusts timing. This segmentation allows each operation to work independently on the composition timeline, enabling derived tracks to be further manipulated without the limitations of edit lists.
Solution Approach 2:
The patent transitions from operating solely on the presentation timeline (edit list approach) to operating on the composition timeline (skip/shift operations). This dimensional change enables derived tracks to maintain composition timeline references, allowing subsequent transformations and hierarchy building that were previously impossible.
2Manufacturing precision
If edit lists are used for re-timing, then timing adjustment can be achieved, but subsequent transformations and hierarchy building are restricted
Solution Approach 1:
The patent extracts the re-timing functionality from the edit list mechanism and implements it through skip and shift operations that work directly on the composition timeline. This extraction eliminates the dependency on edit lists, simplifying the overall track processing complexity while maintaining timing adjustment accuracy.
Solution Approach 2:
The patent changes the fundamental parameters of track manipulation by introducing skip (remove media units) and shift (adjust timing) operations that operate on composition timeline references. These parameter changes enable subsequent transformations and hierarchy building without increasing processing complexity.
3Adaptability or versatility
If all spherical content is processed for omnidirectional video, then complete content is available for any viewport, but compute intensity and bandwidth consumption increase significantly
Solution Approach 1:
The patent applies partial action by processing only the necessary portions of spherical content based on viewport requirements. Instead of processing all content, the system processes only the relevant segments, reducing compute intensity and bandwidth consumption while maintaining adequate viewport coverage.
Solution Approach 2:
The patent introduces dynamic content selection where the processing scope adapts to the user's viewport position. The system dynamically determines which content segments need processing based on current viewing conditions, optimizing the balance between viewport coverage and computational resources.
Data Source
AI summary
The techniques described herein relate to methods, apparatus, and computer readable media configured to access multimedia data comprising a hierarchical track structure comprising at least a first track at a first level of the hierarchical track structure comprising first media data, wherein the first media data comprises a first sequence of video media units, and a second track at a second level in the hierarchical track structure different than the first level of the first track, the second track comprising metadata specifying a re-timing derivation operation. Output video media units are generated according to the second track, comprising performing the re-timing derivation operation on the first sequence of video media units to modify a timing of the first sequence of video media units by removing one or more video media units associated with the re-timing derivation operation and/or shifting timing information of the first sequence of video media units.