Video Segment Alignment via Closed Caption Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In adaptive bitrate (ABR) streaming, partial transcoding of video segments leads to misalignment of frames between renditions, resulting in a poor playback experience due to incorrect frame transitions during rendition switching.
Innovation Solution
Utilize embedded closed captions, such as CEA-608/708 formatted captions within SEI messages, to detect and correct segment misalignments by comparing caption data across renditions, and optionally re-transcoding misaligned segments to ensure frame alignment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If partial transcoding is used to update ABR ladder renditions, then processing and storage efficiency is improved, but frame alignment between segments deteriorates
Solution Approach 1:
The patent applies preliminary action by detecting frame alignment issues between renditions before they cause playback problems. The system pre-analyzes frame timestamps and identifies misalignments in advance, allowing corrective actions to be taken before actual rendition switching occurs during playback, thus maintaining both processing efficiency and frame alignment.
2Adaptability or versatility
If rendition switching is enabled for adaptive bitrate streaming, then adaptability to network conditions is improved, but frame synchronization between renditions deteriorates
Solution Approach 1:
The patent implements feedback by continuously monitoring frame timestamps and alignment status during rendition switching. When misalignment is detected, the system provides feedback signals to adjust the switching timing or re-synchronize frames, ensuring that adaptive bitrate switching maintains frame synchronization reliability through closed-loop control.
Solution Approach 2:
The patent uses frame timestamps and synchronization metadata as intermediary elements between different renditions. These intermediaries carry timing information that mediates the switching process, allowing the player to coordinate transitions between renditions while maintaining frame alignment, thus enabling network adaptability without sacrificing synchronization reliability.
3Speed
If segments are split for quick switching, then switching speed is improved, but frame alignment between segments deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing frame timestamp information for each segment during the transcoding phase. This preliminary preparation allows the playback system to quickly switch between segments while maintaining alignment, as the synchronization data is already available without requiring real-time analysis, thus preserving both switching speed and segment alignment.
Data Source
AI summary
An aspect of the disclosure is related to methods and systems configured to detect and correct misalignments of segments in corresponding renditions of video content. A video is transcoded to generate renditions at respective different bit rates and/or resolutions, wherein a given rendition comprises a plurality of segments, the segments comprising a plurality of frames. First closed caption text is accessed for a first frame of a first segment of a first rendition. Second closed caption text for a first frame of a first segment of a second rendition is accessed. A determination is made as to whether the first closed caption text matches the second closed caption text. If they fail to match, the first segment of the second rendition is re-rendered. The re-rendered first segment of the second rendition is streamed over a network to a client device.