An audio and video automatic synchronization method and synchronization system

Abstract

The invention relates to an audio and video automatic synchronization method and synchronization system, which are used for network live broadcast. The method comprises the following steps of: receiving an audio stream and a video stream, wherein the audio stream comprises a plurality of audio frames with time stamps, the video stream comprises a plurality of video frames with time stamps, and thevideo frames comprise key frames and non-key frames; Extracting a key frame from the video stream; Encoding the key frame, judging whether the key frame is successfully encoded or not, and if yes, selecting the key frame as an initial frame; Taking the initial frame as an audio and video synchronization starting point, and enabling the video frames to automatically correspond to the audio framesone by one according to the timestamps; And encoding the video frames and the audio frames which are in one-to-one correspondence so as to synchronously push streams to the video frames and the audioframes. According to the audio and video automatic synchronization method and system, the initial frame is selected and serves as the audio and video synchronization starting point, the video frames and the audio frames are automatically in one-to-one correspondence according to the timestamps, audio and video synchronization is conducted through the initial frame, and simpleness and rapidness areachieved.

Description

technical field [0001] The invention relates to the technical field of webcasting, in particular to an audio and video automatic synchronization method and synchronization system. Background technique [0002] Each frame of audio or video has a duration, and the sampling frequency refers to the number of times the amplitude of the sound wave is sampled per second when digitizing an analog sound waveform. The frequency range of normal human hearing is between 20Hz and 20kHz. According to the Nyquist sampling theory, in order to ensure that the sound is not distorted, the sampling frequency should be around 40kHz. Commonly used audio sampling frequencies are 8kHz, 11.025kHz, 22.05kHz, 16kHz, 37.8kHz, 44.1kHz, 48kHz, etc. If a higher sampling frequency is used, the sound quality of DVD can also be achieved. Coding (Advanced Audio Coding) When decoding audio, the decoding time of one frame must be controlled within 23.22 milliseconds. An AAC raw frame contains 1024 samples and...

AI Technical Summary

Problems solved by technology

[0020] In addition, the video image is too dependent on the key frame, and the key frame is generally relatively small. It takes a time difference for the camera to start transmitting the image to the device, which will cause the loss of the first key frame, so all data before the second key frame is invalid. , but the audio data is indeed all valid. If it is not handled properly here, the audio will be ahead of the video in the whole process.

[0021] In the process of traditional audio and video transmission, the possibility of audio and video being out of sync is relatively small, and even if there is a difference, it is relatively small. However, after the rise of webcasting, various special effects processing in video, audio voice changing processing, and even the use of robot voice packets In situations such as recognizing text and generating sound, the time gap between audio and video is very large, and there is great uncertainty. Sometimes video processing is fast, and sometimes audio processing is fast. Manual adjustments cannot synchronize audio and video, and the above audio and video synchronization methods are not applicable. Happening

Images