Lip synchronous method for multimedia real-time transmission in packet network and apparatus thereof

Abstract

The invention relates to multimedia real-time transmission in packet network, disclosing a lip synchronizing method and device for multimedia real-time transmission in packet network, able to simply and effectively regulate to overcome network dithering and variation and implementing lip synchronization of audio and video streams. And it sets buffers which can flexibly regulate buffering time length according to network time delay information and implement synchronization of audio and video streams by regulating their buffering time lengths; it sets a time delay detection module to detect transmission time delays of audio and video data in packet network and then uses synchronizing module to refer to their average transmission time delay and transmission time delay so as to synchronize audio and video modules; it uses the synchronizing module to calculate buffer-storing time lengths of audio and video buffers and flexibly sets them to resist network transmission dithering; it makes one of the audio and video parties regularly transmit synchronous time reference point to the other one so as to coordinate their synchronization.

Description

technical field [0001] The invention relates to multimedia real-time transmission in packet network, in particular to a lip synchronization method and device for multimedia real-time transmission in packet network. Background technique [0002] With the rapid development of the Internet (Internet), the application of streaming media technology is becoming more and more extensive, from online broadcasting, movie playing to distance learning and online news sites, etc. all use streaming media technology. Currently, there are mainly two ways to transmit video and audio on the Internet: Download and Streaming. Streaming transmission is the continuous transmission of video / audio signals, and when the streaming media is played on the client computer, the rest continues to download in the background. There are two ways of streaming transmission (Progressive Streaming) and real-time streaming (Realtime Streaming). Live streaming is real-time delivery and is especially suitable for...

AI Technical Summary

Problems solved by technology

In the case of packet network transmission, two independent streams have different delays in network transmission due to different paths and different quality of service registrations, and the delay of network transmission due to jitter It also varies greatly and is unstable, making the fixed delay in the Jitter Buffer unable to compensate for the synchronization deviation, which eventually leads to audio streams, video out-of-sync and lip synchronization failure

[0012] In practical application, the above scheme has the following problems: First, the delay of audio stream and video stream in JitterBuffer is fixed and cannot be adjusted dynamically, which cannot adapt to changes in network conditions

For example, when the network conditions are good, the multimedia transmission itself is fast, and the buffering delay is too large, which will lead to a waste of system delay; and when the network conditions are poor, the jitter is strong and cannot be eliminated, causing the two streams to lose synchronization and cannot achieve synchronization. Effect

[0013] Secondly, after a fixed delay, the synchronization deviation between the two streams is also fixed. When the network conditions change, such as getting better or worse, the synchronization deviation changes, but after the synchronization process, the two streams are increased. synchronization deviation

[0014] Again, the audio stream and the video stream are processed independently and there is no reference to synchronize with each other during synchronization. The two streams are simply synchronized by adding a fixed delay, and cannot be adjusted according to the synchronization results of the two streams.

[0015] The main reason for this situation is that there is no adjustment mechanism to set a fixed delay when buffering Jitter Buffer processing for different streams of multimedia streams, namely audio streams and video streams.

Images