Videoconference Server Audio Video Synchronization Buffering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In videoconference systems, the inefficiency of packet retransmission mechanisms leads to excessive transmission delays, particularly affecting lip synchronization when video packets are lost during transmission, as the processing delay applied to audio packets is also applied to video packets, causing inefficiencies in synchronization.
Innovation Solution
Synchronizing audio and video packets at the client devices rather than the videoconference server, allowing for flexible transmission delays and early detection and recovery of lost video packets by utilizing client-side video delay modules and buffers to minimize recovery latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If packet retransmission mechanism is used to recover lost video packets, then video packet loss is recovered, but transmission delay exceeds maximum lip synchronization delay
Solution Approach 1:
The server performs preliminary actions by buffering video packets and calculating audio processing delays before transmission. When packet loss occurs, the client can request retransmission and the server can quickly retrieve the buffered packet without requiring full reprocessing, thus reducing recovery delay while maintaining lip synchronization
Solution Approach 2:
The system implements feedback mechanisms where the receiving client detects lost video packets and sends recovery requests back to the emitting client or server. The server uses this feedback to retransmit specific lost packets efficiently, balancing reliability improvement with minimal delay addition
2Stability of the object's composition
If processing delay is applied to video packets to synchronize with audio packets, then lip synchronization is maintained, but flexibility in transmission delay is reduced
Solution Approach 1:
The system segments the delay compensation function: the server applies a base processing delay to video packets to match audio processing time, while the client device has the capability to adjust additional delay parameters. This segmentation allows the server to maintain stable synchronization while the client adapts delay flexibility based on network conditions and device performance
Solution Approach 2:
The video delay module at the client side provides dynamic adjustment of video packet timing. The system transitions from static delay application at the server to dynamic delay adjustment at the client, allowing flexible adaptation of transmission delay while maintaining lip synchronization through real-time parameter adjustment
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
The invention comprises a videoconference server (2) comprising : - an audio processing module (9) configured to, o calculating an average audio processing delay of processing audio packets intended to the second client device, in order that the second client device may synchronize a received video packet with the associated received audio packet, - a video router (10) configured to: o duplicate the video packet and temporarily store the duplicated video packet, o receive a retrieving request from the second client device (4) for retrieving a lost video packet, o retrieving the duplicated video packet associated with the identification information in the video buffer (17).