Audio and video synchronization method for video conference system

Abstract

The invention provides an audio and video synchronization method for a video conference system, which comprises the following steps that: a signaling server receives a conference creating / joining request of conference terminals, and distributes a unique identifier SSRC of audio and video data and media node servers to a plurality of conference terminals participating in a conference; the plurality of conference terminals participating in the conference send audio and video data to the corresponding media node servers, and subscribe video data and conference audio data output of the M conference terminals respectively; a conference audio processing node server obtains audio data uploaded by the corresponding conference terminals from the multiple media node servers, and N paths of audio data with the maximum audio energy value are screened out to serve as conference audio data to be output; and the conference terminal outputs and distributes the conference audio data to the corresponding video data channel according to the unique identifier SSRC, and performs audio and video data synchronization based on the timestamp of the video data, so that the occupation of playing resources of the client and the server is small, and the effective synchronization of the audio and the video is ensured.

Description

technical field [0001] The invention relates to the technical field of video conferencing, in particular to an audio and video synchronization method for a video conferencing system. Background technique [0002] The current online office market is very hot. For some large enterprises and organizations, with branches all over the country and a large number of employees participating in conferences, large-scale audio and video conferences have become a rigid demand. [0003] However, the current large-scale video conferencing solution based on the MCU (Multipoint Conferencing Unit, multipoint control unit) architecture is not suitable for large-scale audio and video conferencing scenarios, because the server needs to decode, mix streams, and re-encode, which consumes a lot resources; the large-scale video conferencing solution based on the SFU (Selective Forwarding Unit, selective forwarding unit) architecture is relatively flexible, and you can subscribe to the required audi...

AI Technical Summary

Problems solved by technology

[0003] However, the current large-scale video conferencing solution based on the MCU (Multipoint Conferencing Unit, multipoint control unit) architecture is not suitable for large-scale audio and video conferencing scenarios, because the server needs to decode, mix streams, and re-encode, which consumes a lot resources; the large-scale video conferencing solution based on the SFU (Selective Forwarding Unit, selective forwarding unit) architecture is relatively flexible, and you can subscribe to the required audio and video streams on demand, but you can selectively watch certain channels of video during the conference Video and audio need to be fully subscribed. For large-scale audio and video conferences, if all audio is subscribed and forwarded to the client for processing, it will definitely bring a lot of pressure to the client. At the same time, the pressure on the signaling channel of the server is also very high. In addition, the response speed of the client will be slowed down and the delay will be increased.

[0004] The current existing solution is to use the SFU architecture and the MCU architecture in combination, the audio is uniformly mixed by the server, and then the mixed data is sent to the client, which effectively solves the problems existing in the pure SFU architecture, but introduces There is a new technical problem, that is, the audio and video cannot be effectively synchronized

Images