Unlock instant, AI-driven research and patent intelligence for your innovation.

Sound mixing method, device, equipment, system and readable storage medium

A sound mixing and audio technology, applied in the field of signal processing, can solve the problems of noisy sound, the platform or terminal performance cannot meet the audio codec and audio stream forwarding, affecting the sound effect, etc.

Active Publication Date: 2021-08-20
SUZHOU KEDA TECH
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] For the two mixing methods mentioned above, when the number of parties joining the conference is small, the performance of the platform or terminal can meet the requirements of decoding and playback; The performance of the terminal cannot meet the needs of a large number of audio codecs and audio stream forwarding
Moreover, in most conferences, it is difficult to ensure that all terminals can be muted, which will cause a lot of noise to enter the mix, causing the mixed sound to be noisy, seriously affecting the sound effect

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Sound mixing method, device, equipment, system and readable storage medium
  • Sound mixing method, device, equipment, system and readable storage medium
  • Sound mixing method, device, equipment, system and readable storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0070] This embodiment describes the first audio mixing method in detail. The method can be applied to a multipoint conference controller. Among them, the multipoint conference controller, that is, the multipoint control unit (MCU, Multipoint Control Unit) is the abbreviation of the central control device of the video conference system, and is a key device of the multipoint video conference system.

[0071] Please refer to figure 1 , figure 1 It is a flowchart of the first sound mixing method in the embodiment of the present invention, and the method includes the following steps:

[0072] S101. Receive audio RTP packets sent by each conference terminal, and read extension header information of each audio RTP packet.

[0073] Wherein, the extended header information includes sound feature information or mute flag information.

[0074] Wherein, RTP refers to a Real-time Transport Protocol (Real-time Transport Protocol), that is, an audio RTP packet is a data packet with an a...

Embodiment 2

[0116] This embodiment describes the second sound mixing method in detail. This method can be applied in SFU. Among them, SFU (Selective Forwarding Unit, selective forwarding unit).

[0117] Please refer to figure 2 , image 3 It is a flowchart of the first sound mixing method in the embodiment of the present invention, and the method includes the following steps:

[0118] S201. Receive audio RTP packets sent by each conference terminal, and read extended header information of each audio RTP packet.

[0119] The extended header information includes sound feature information or mute flag information.

[0120] Wherein, receiving the audio RTP packet sent by each conference terminal may specifically be that the SFU receives an audio RTP packet sent by each conference terminal with extended header information that matches the sound recognition result; wherein, the sound recognition result is that the conference terminal uses sound feature recognition The model is obtained af...

Embodiment 3

[0135] Corresponding to the above method embodiment, the embodiment of the present invention also provides a first sound mixing device, which can be applied to a multipoint conference controller. The first sound mixing device described below and the sound mixing method described above can be used refer to each other.

[0136] see image 3 As shown, the device includes the following modules:

[0137] The extended header information reading module 101 is used to receive the audio RTP packet sent by each conference terminal, and read the extended header information of each audio RTP packet; the extended header information includes sound feature information or mute flag information;

[0138] The mute judging module 102 is used to judge whether the current mixing time is in a mute state by using the mute flag information corresponding to the current mixing time;

[0139] Static sound mixing module 103, used for generating audio frames containing mute marks when in a mute state; ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses two sound mixing methods, devices, equipment, systems and readable storage media that can be applied to multipoint conference controllers and SFUs respectively. When the multipoint conference controller is used for audio mixing processing, the multipoint conference controller does not perform audio mixing processing in the mute state, and performs audio mixing after screening the audio RTP packets in the non-mute state, which can greatly reduce the multipoint conference control. The volume of the audio mixing task of the controller, in the scenario where there are many participants, the multi-point conference controller can also meet the audio mixing requirements. When the terminal test is used for audio mixing processing, the SFU does not forward audio data packets in the mute state, and the screening of audio RTP packets in the non-mute state can greatly reduce the number of audio RTP packets forwarded by the SFU to each conference terminal, that is, the conference The mixing task of the terminal can be reduced. In this way, in a scenario where there are many participants, the terminal side can also meet the audio mixing requirements.

Description

technical field [0001] The present invention relates to the technical field of signal processing, in particular to two sound mixing methods, devices, equipment, systems and readable storage media. Background technique [0002] In a multi-party video conference, each receiving end must be able to hear the voices of other terminals, but cannot hear its own voice, so it needs to have a sound mixing function; the sound mixing can be done on the platform or in the done on the terminal side. When mixing on the platform, the platform needs to receive and decode the audio stream sent by multiple terminals, and then mix the audio for different terminals so that it does not include the sound sent by the receiving terminal, and finally encode the mixed audio sent to the receiving end. If the terminal performs audio mixing, the platform receives the audio code stream of the terminal and forwards it to other terminals, and the terminal decodes multiple audio code streams after receivin...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04M3/56G10L19/012H04L29/06
CPCH04M3/568H04L65/403H04L69/22G10L19/012H04L65/65H04L65/70
Inventor 王展胡小鹏顾振华
Owner SUZHOU KEDA TECH