Multimedia Playback Audio Video Synchronization via PTS Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Achieving accurate audio/video synchronization timing within the 10ms requirement of the HBBTV 2.0.1 standard is challenging for multimedia playback apparatuses.
Innovation Solution
A multimedia playback apparatus that includes a processor and non-volatile memory, which executes analyzing, video, and audio drivers to obtain and convert presentation timestamps (PTS) from video and audio streams, synchronizing audio content with video content by modifying audio packets with converted PTS values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional PTS conversion methods are used, then the system is simple to implement, but the audio/video synchronization accuracy cannot meet the 10ms requirement
Solution Approach 1:
The patent divides the PTS conversion process into three separate conversion stages: video PTS to video TEMI, video TEMI to audio TEMI, and audio TEMI to audio PTS. Each stage is handled by a dedicated conversion module, allowing precise control and adjustment at each step to achieve the required 10ms synchronization accuracy while maintaining manageable system complexity through modular design.
Solution Approach 2:
The patent introduces TEMI (Timeline External Media Information) as an intermediary format between video PTS and audio PTS. This intermediary representation allows for precise timing correlation between video and audio streams through defined reference relationships, enabling accurate synchronization without directly converting between incompatible timestamp systems.
2Reliability
If separate processing is used for broadcast and broadband streams, then the system can handle different sources, but achieving synchronized playback within 10ms is difficult
Solution Approach 1:
The patent implements separate processing paths for broadcast video streams and broadband audio streams, with dedicated parsing, conversion, and decoding modules for each. This segmentation allows independent optimization of each stream's processing while maintaining synchronized output through the TEMI-based timing correlation mechanism.
Solution Approach 2:
The patent transforms timestamp parameters between different reference systems (video PTS timeline to audio PTS timeline) through the TEMI intermediary. By changing the temporal reference parameters and using defined correlation points, the system reconciles timing differences between broadcast and broadband sources to achieve reliable synchronization.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A multimedia playback apparatus is provided, which includes a non-volatile memory and a processor. The processor executes an analyzing program, a first driver, and a second driver to perform the following steps: parsing a first packet in a first media stream and a second packet in a second media to obtain a first presentation timestamp (PTS) and a second PTS; converting the second PTS into a third PTS with reference to a first timeline on which the first PTS is located; modifying the second packet by replacing the second PTS with the third PTS in the second packet; decoding the first packet and the modified second packet by the first driver and the second driver to generate first media content and second media content, respectively; and synchronizing the second media content with the first media content by the second driver using the third PTS and the first PTS.