Multi-channel speech enhancement method for teleconference

Abstract

The invention provides a multi-channel speech enhancement method for a teleconference, which comprises the following steps of: firstly, scanning the amplitudes of plane waves in different directions, finding the direction of a speaker, collecting a sound signal, then adjusting the phase and gain of the sound signal based on a speech enhancement algorithm, enhancing the signal, and finally playing at a remote end. Compared with the prior art, the method has the beneficial effects that sound signals in the direction of a speaker are enhanced, noise signals in other directions are inhibited, the influence of reverberation is reduced, and the call quality is improved; a frequency-independent beam former and a traditional delay summation beam former are combined, the respective defects of the two beam formers are overcome, and the beam formers have high signal-to-noise ratios no matter low-frequency signals or high-frequency signals.

Description

Technical Field

[0001] The invention relates to the field of speech enhancement, and in particular to a multi-channel speech enhancement method for remote conferences. Background Art

[0002] With the rapid development of information technology, remote conferencing has become an indispensable part of modern work and life, which has greatly promoted communication and collaboration between teams across regions. However, remote conferencing still faces many challenges in practical applications, one of the most significant pain points being the unstable and poor voice quality. In complex acoustic environments, such as home offices, cafes or public places, background noise, echo, reverberation, and multiple speakers speaking at the same time often make it difficult for meeting participants to understand and communicate clearly, seriously affecting meeting efficiency and experience.

[0003] In order to improve the voice quality of remote conferences, the industry has developed a variety of technical means, including but not limited to single-channel voice enhancement algorithms, noise suppression algorithms, and echo cancellation technologies. These technologies have alleviated the background noise and echo problems to a certain extent, but they still have obvious limitations. For example, single-channel voice enhancement algorithms are often unable to cope with complex environmental noise and are difficult to effectively distinguish and suppress background noise from useful voices; while traditional echo cancellation technology relies on accurate acoustic models and environmental parameter estimation, has poor adaptability to environmental changes, and is less effective under high-latency network conditions. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes a multi-channel speech enhancement method for remote conferencing, the steps of which include: S1: collecting sound based on a microphone array and establishing a spatial coordinate system, wherein the microphone array includes a plurality of microphones; S2: The sound field model of the speaker's voice in the conference room reaching the microphone array is approximated as a far field, the sound wave emitted by the speaker is equivalent to a plane wave, the sound field at the location of any microphone of the microphone array is equivalent to the superposition of plane waves in all directions, and the direction of the speaker is determined according to the amplitude of the plane wave; S3: According to the angle and sound signal of the speaker, the phase and gain of the sound signal are adjusted by a speech enhancement algorithm to enhance the sound signal in the direction of the speaker and suppress noise in other directions; S4: Play the enhanced sound signal at the remote receiving end.

[0005] Preferably, the microphone array is composed of a plurality of microphones arranged linearly, and collects sound signals through a plurality of channels. The spatial coordinate system takes the left end of the microphone array as the coordinate center, and extends toward the right end as the positive direction of the z-axis.

[0006] Preferably, in the step S2, the plane wave amplitude in a certain direction is calculated at any microphone, and then traverses from 0 degrees to 180 degrees, each angle corresponds to a plane wave amplitude, and the angle corresponding to the maximum value is the incident angle of the sound signal of the target object.

[0007] Preferably, in the step S3, the speech enhancement algorithm uses a super-directional beamforming algorithm, mainly by calculating high-order derivatives in the wave number domain to obtain a weight vector, bringing the weight vector into a beamformer, and outputting a processed sound signal. The specific steps include: S31: approximating the sound field to a finite superposition of plane waves, associating the sound signals received by the multiple microphones through a wave equation, reconstructing the sound field, and differentiating the reconstructed sound field to obtain a high-order derivative; S32: Calculate the high-order derivative by multiplication to obtain a weight vector of the sound signal, bring it into the beamformer, and output an enhanced signal.

[0008] Preferably, the beamformer is obtained by mixing a frequency-independent beamformer and a traditional delay-sum beamformer, which still has a high signal-to-noise ratio for high-frequency signals.

[0009] The present invention proposes a multi-channel speech enhancement method for remote conferences. First, the plane wave amplitude in different directions is scanned to find the direction of the speaker and collect the sound signal. Then, based on the speech enhancement algorithm, the phase and gain of the sound signal are adjusted to enhance the signal, and finally the signal is played at the remote end. Compared with the prior art, the present invention has the following beneficial effects: enhancing the sound signal in the direction of the speaker, suppressing the noise signal in other directions, and improving the call quality; combining the frequency-independent beamformer with the traditional delay-sum beamformer to overcome the shortcomings of the two beamformers. Whether it is a low-frequency or high-frequency signal, the beamformer has a higher signal-to-noise ratio. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 The following is the overall flow chart; Figure 2 It is the flow chart of speech enhancement algorithm; DETAILED DESCRIPTION

[0011] According to the following Figure 1 and Figure 2 , giving the preferred embodiments of the present invention and describing them in detail, so as to better understand the functions and features of the present invention.

[0012] according to Figure 1 , a multi-channel speech enhancement method for remote conference according to an embodiment of the present invention comprises the following steps: S1: Collect sound based on microphone array and establish a spatial coordinate system. The microphone array consists of M microphones arranged linearly. The distance between adjacent microphones is d. It is used to collect sound signals. The spatial coordinate system takes the left end of the microphone array as the coordinate center and extends to the right end as the positive direction of the z axis. The angle between the speaker direction and the z axis is The sound signal received by a microphone can be expressed as: (1) in is the wave number, is the frequency is the speed of sound,

[0013] S2: The sound field model of the speaker's voice in the conference room reaching the microphone array is approximated as the far field, and the sound wave emitted by the speaker is equivalent to a plane wave. The sound field at the location of any microphone in the microphone array can be equivalent to the superposition of plane waves from all directions. In the Cartesian coordinate system, any point The sound field is expressed as the superposition of multiple plane waves. The formula is as follows: (2) In the z-axis direction, formula (2) can be further expressed as: (3) Among them, the angle The corresponding plane wave amplitude is Traversing from 0 to 180 degrees, when hour, There is a maximum value, that is, the plane wave amplitude in the speaker's direction is the largest, The direction of the speaker.

[0014] S3: According to the angle of the speaker and the sound signal, the phase and gain of the sound signal are adjusted through the speech enhancement algorithm to enhance the sound signal of the target object and suppress the noise in other directions. The speech enhancement algorithm is based on the super-directional beamforming algorithm. The mathematical model of the beamformer is constructed as follows: In practice, the microphone receives not only the speaker's voice signal, but also a noise signal. By introducing the noise vector Signals received by multiple microphones It can be expressed as: (4) in, T stands for transpose, represents the position of the microphone, By introducing the weight vector To improve the signal-to-noise ratio, the output of the beamforming algorithm is: (5) in The associated beamformer can be defined as: (6) in stands for complex conjugate transpose; The speech enhancement algorithm is as follows: Figure 2 As shown, the steps are as follows: S31: The sound field is approximated as a finite superposition of plane waves, the sound signals received by multiple microphones are associated together through wave equations, the sound field is reconstructed, and the reconstructed sound field is differentiated to obtain high-order derivatives; according to formula (3), the sound field of finite plane wave superposition at any microphone position can be obtained: (7) in L is the number of angular discrete points,

[0015] For the M microphones on the z-axis, the sound field can be expressed by a vector: (8) in

[0016] and

[0017] A is a full rank matrix, so formula (8) can be equivalent to: (9)

[0018] The matrix A can be decomposed into: (10) in is the identity matrix, It can be expressed as: (11) in , are singular values, is a matrix with all elements equal to zero. By using formulas (10) and (11), we can obtain: (12) According to formulas (10) and (12), we can get: (13) To increase the robustness of x, in the matrix Introducing filter factors ,get: (14) in is the cutoff parameter, which can be obtained through experiments. Substituting formula (14) into (13), we can get: (15) Among them

[0019] By differentiating formula (3), we can obtain the nth-order derivative of the sound field at any microphone: (16) S33: Calculate the high-order derivative by multiplication to obtain a weight vector of the sound signal, substitute it into the beamformer, and output an enhanced signal.

[0020] By calculating the derivative in the spatial domain through multiplication in the wavenumber domain, similar to the discretization of formula (3), formula (16) can be expressed as: (17) in and ,for Substituting formula (15) into (17), we can obtain: (18) In formula (18), the left side is the real high-order derivative of the sound field, and the right side is the high-order derivative of the reconstructed sound field. , whose derivative is: (19) The present invention proposes a hybrid beamformer, which is obtained by mixing a frequency-dependent beamformer and a delay-sum beamformer. The frequency-dependent beamformer is: (20) The delay-sum beamformer is: (twenty one) Formulas (20) and (21) can be unified into a beamformer: (twenty two) in is the steering vector, Table Hadamard product, is a column vector, when the vector Only the When elements are 1 and the rest are 0, , Represents the integer part, when the vector When all elements are 1,

[0021] According to Euler's formula and binomial theorem, It can be expressed as: (twenty three) Substituting formula (23) into (22) we can obtain: (twenty four) According to formula (24), the beamformer can finally be expressed as: (25) in ,for ,

[0022] Substituting formula (19) into (25), we can obtain: (26) Substituting formula (26) into (5), we obtain Directional enhancement signal.

[0023] S4: Play the enhanced sound signal at the remote receiving end.

Claims

1. A multi-channel speech enhancement method for remote conferencing, the steps comprising: S1: collecting sound based on a microphone array and establishing a spatial coordinate system, wherein the microphone array includes a plurality of microphones; S2: The sound field model of the speaker's voice in the conference room reaching the microphone array is approximated as a far field, the sound wave emitted by the speaker is equivalent to a plane wave, the sound field at the location of any microphone of the microphone array is equivalent to the superposition of plane waves in all directions, and the direction of the speaker is determined according to the amplitude of the plane wave; S3: According to the angle and sound signal of the speaker, the phase and gain of the sound signal are adjusted by a speech enhancement algorithm to enhance the sound signal in the direction of the speaker and suppress noise in other directions; S4: Play the enhanced sound signal at the receiving end.

2. A multi-channel speech enhancement method for remote conferencing according to claim 1, characterized in that: The microphone array is composed of a plurality of microphones arranged linearly, and collects sound signals through a plurality of channels. The spatial coordinate system takes the left end of the microphone array as the coordinate center, and extends toward the right end as the positive direction of the z-axis.

3. A multi-channel speech enhancement method for remote conferencing according to claim 1, characterized in that: In the step S2, the plane wave amplitude in a certain direction is calculated at any microphone, and then traverses from 0 degrees to 180 degrees, each angle corresponds to a plane wave amplitude, and the angle corresponding to the maximum value is the incident angle of the sound signal of the target object.

4. A multi-channel speech enhancement method for remote conferencing according to claim 1, characterized in that: In the step S3, the speech enhancement algorithm uses a super-directional beamforming algorithm, which mainly calculates high-order derivatives in the wave number domain to obtain a weight vector, brings the weight vector into a beamformer, and outputs a processed sound signal. The specific steps include: S31: approximating the sound field to a finite superposition of plane waves, associating the sound signals received by the multiple microphones through a wave equation, reconstructing the sound field, and differentiating the reconstructed sound field to obtain a high-order derivative; S32: Calculate the high-order derivative by multiplication to obtain a weight vector of the sound signal, bring it into the beamformer, and output an enhanced signal.

5. A multi-channel speech enhancement method for remote conferencing according to claim 4, characterized in that: The beamformer is obtained by mixing a frequency-independent beamformer with a traditional delay-sum beamformer, and still has a high signal-to-noise ratio for high-frequency signals.

Images