Multimedia Flow Synchronization via Auxiliary Timing Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Synchronization of multimedia flows across different networks and transport protocols is challenging, especially when audio and video sources are not connected by a reliable network, requiring an independent synchronization mechanism to ensure synchronized rendering.
Innovation Solution
A method for processing multimedia data flows that involves receiving multimedia data from two networks using different transport protocols, synchronizing the flows using synchronization information in the data field, and rendering them synchronously, with the option to delay content until it can be rendered in sync, using auxiliary data packets to adjust timing references between protocols like MPEG-2 TS and RTP.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multimedia flows are transmitted through different networks using different transport protocols (MPEG-2 TS, RTP), then the system gains flexibility and adaptability for various application scenarios, but synchronization between audio and video flows becomes difficult when networks are not reliably connected
Solution Approach 1:
The patent introduces an intermediary synchronization mechanism that acts as a mediator between different transport protocols. Auxiliary data packets are inserted into the MPEG-2 TS stream containing timing information from the RTP stream, enabling the rendering device to synchronize flows from unreliable networks without requiring direct network connectivity between sources.
Solution Approach 2:
The patent applies preliminary action by pre-inserting synchronization information into auxiliary data packets before the rendering process. The timing references and timestamps are prepared in advance within the auxiliary data structure, allowing the rendering device to perform synchronization without real-time network dependency.
2Reliability
If a common clock (wall clock) is used to synchronize audio and video flows from different sources, then synchronization can be achieved when sources share the same time base, but this approach fails when sources are not connected by a reliable network
Solution Approach 1:
The auxiliary data packets serve as an intermediary that carries timing information between independently clocked sources. Instead of requiring sources to share a common clock, the patent uses auxiliary data to transfer reference timing information from one stream to another, enabling synchronization without direct network connectivity between sources.
3Ease of operation
If timestamps from different transport protocols are directly used for synchronization, then the rendering device can regulate flow rendering, but different time bases and clock frequencies between protocols cause synchronization failures
Solution Approach 1:
The patent changes the parameters of timing information by converting and normalizing timestamps from different protocols into a unified time base. The auxiliary data structure contains fields that map RTP timestamps to MPEG-2 TS presentation timestamps, adjusting clock frequency differences and time base variations to achieve precise synchronization.
Data Source
AI summary
The present invention relates to a reception device and a method for the processing of multimedia data flows in said device comprising an interface to a first network and an interface to a second network, said method comprising the following steps: receive, from the first network, a first flow comprising multimedia data through a first transport protocol fitted with a first data synchronization system; receive, from the second network, a second flow comprising multimedia data through a second transport protocol fitted with a second data synchronization system, said second synchronization system being based on timing references different from those of the first data synchronization system; characterized in that said first and second flows transport the same synchronization information in the payload field of the first and of the second transport protocol, said synchronization information comprising data indicating the moment from which said multimedia data must be rendered; synchronize said first and second flows by using said synchronization information; and perform a step for rendering said first and second multimedia flows.