Multi-Device Lip Sync via RTCP Clock Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for wireless audio and video synchronization, such as those using Bluetooth and Wi-Fi speakers, fail to achieve lip synchronization due to delays and clock frequency discrepancies, leading to asynchronous audio and video outputs.
Innovation Solution
A multi-device lip synchronization method that uses Real-Time Control Protocol (RTCP) packets to synchronize the system clock of secondary devices with the program clock reference (PCR) and presentation time stamp (PTS) from a primary device, ensuring accurate timing for audio and video playback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If Bluetooth or Wi-Fi protocol is used for wireless audio transmission, then cross-device audio output is achieved, but lip synchronization between devices cannot be implemented
Solution Approach 1:
The patent introduces a timing synchronization mechanism as an intermediary between the audio transmission protocol and the playback device. This mechanism uses timing information embedded in the audio data stream to coordinate playback timing across multiple devices, enabling lip synchronization while maintaining wireless audio transmission capability.
Solution Approach 2:
The patent implements a feedback mechanism where the primary device monitors playback timing and sends timing adjustment instructions to secondary devices. This closed-loop control ensures that audio playback remains synchronized with video across multiple devices, resolving the synchronization issue without compromising wireless audio output.
2Ease of operation
If Wi-Fi router forwarding is used for audio transmission, then wireless connectivity is achieved, but delay increases and asynchronization between audio and video output is exacerbated
Solution Approach 1:
The patent applies preliminary action by pre-calculating and embedding timing information into the audio data stream before transmission. This allows secondary devices to anticipate playback timing and adjust accordingly, compensating for router-induced delays without requiring real-time feedback loops that would increase latency further.
3Adaptability or versatility
If system clock of secondary device is not synchronized with primary device, then device independence is maintained, but audio-video synchronization cannot be achieved
Solution Approach 1:
The patent makes the system clock synchronization dynamic by allowing secondary devices to adjust their clock timing based on received timing information from the primary device. This dynamic adjustment mechanism enables precise audio-video synchronization while preserving device independence, as each device autonomously adjusts its timing based on synchronization protocols rather than hard-coded fixed timing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments of the present invention provide a multi-device lip synchronization method and a device. The method includes: receiving, by a secondary device, an RTCP packet sent by a primary device; correcting an STC of the secondary device based on a PCR in the RTCP packet, a program clock frequency of the primary device, a program clock frequency of the secondary device, and an RTCP delay; then receiving RTPs published by the primary device, splicing the RTPs into a complete audio data frame, and putting the audio data frame into a PCM buffer of the secondary device; and outputting the audio data frame in the PCM buffer.