Video Frame Synchronization with Auxiliary Media
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video streaming technologies, such as DASH, struggle with synchronizing auxiliary media with video content during seek operations, leading to desynchronization issues when media clients jump forward or backward in the playback timeline.
Innovation Solution
The method involves tagging media file segments with control information that allows the media client to determine the progress of auxiliary media rendering, enabling synchronized rendering of video frames and auxiliary media by adjusting the rendering start time or position based on the control information embedded in each media file segment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If auxiliary media is distributed separately from video content using events in DASH, then encoding overhead is reduced and flexibility is improved, but synchronization during seek operations deteriorates
Solution Approach 1:
The patent applies preliminary action by embedding control information (such as rendering start time offsets and duration indicators) into media file segments during the encoding phase. This allows the media client to pre-calculate and adjust the rendering timing of auxiliary media before actual playback, ensuring synchronization is maintained even when seek operations occur. The control information is prepared in advance within each segment, enabling the client to correctly position auxiliary media rendering relative to video content without requiring real-time coordination.
2Ease of operation
If auxiliary media rendering starts from the beginning upon receiving an event message, then simplicity of implementation is improved, but synchronization with video content during seek operations deteriorates
Solution Approach 1:
The patent applies local quality by making each media file segment self-contained with specific control information tailored to that segment's context. Each segment includes rendering start time offsets and duration indicators that are locally valid for that particular segment, allowing the media client to independently determine the correct rendering timing for auxiliary media associated with each segment. This localized approach maintains synchronization without requiring complex global coordination or restarting rendering from the beginning.
3Adaptability or versatility
If HTML-based auxiliary media is used with DASH events, then adaptability and customization are improved, but control over rendering progress deteriorates
Solution Approach 1:
The patent introduces control information as an intermediary layer between the DASH event messages and the HTML-based auxiliary media rendering. This control information, embedded in media file segments, acts as a mediator that translates the flexible HTML rendering requirements into precise timing instructions. The intermediary control data includes rendering start time offsets and duration indicators that bridge the gap between the flexible HTML/CSS/JavaScript auxiliary media and the strict timing requirements of video synchronization, allowing both adaptability and control to coexist.
Data Source
AI summary
The present disclosure relates to synchronizing the rendering of video frames included in media file segments with the rendering of auxiliary media. A method implementation of this technique comprises the step of determining, from a sequence of media file segments, a sub-sequence of media file segments, wherein rendering of the sub-sequence is to be overlaid by rendering of auxiliary media starting in a first media file segment of the sub-sequence. The method further comprises determining control information enabling a media client to determine how much rendering of the auxiliary media has progressed upon rendering of each media file segment of the sub-sequence, and initiating a tagging of the media file segments of the sub-sequence with the control information.