A microphone array speech enhancement system and method for audio and video information fusion

Abstract

The invention discloses a microphone array voice enhancement system and method for audio and video information fusion. The system includes a video acquisition module, a microphone array receiving module, an audio and video incoming direction joint estimation module, a microphone array speech enhancement module and an audio and video joint speech enhancement module, and the video acquisition module is used to collect the video signal of the speaker in the application scene; the microphone The array receiving module is used to receive the speaker's audio signal; the audio and video direction of arrival joint estimation module uses audio and video information to jointly estimate the direction of arrival of the speaker's audio; the microphone array voice enhancement module uses the array voice signal received by the microphone array receiving module To enhance the voice signal; the audio-video joint voice enhancement module uses voice and video signals to jointly enhance the voice twice. The invention can significantly improve the performance of the microphone array voice enhancement system, and can be widely used in video conferences, car phones, mobile video call terminals and other occasions.

Description

technical field [0001] The invention relates to the field of speech signal processing, in particular to a microphone array speech enhancement system for fusion of audio and video information. Background technique [0002] In the actual use environment, the communication equipment is susceptible to interference such as background noise and reverberation, which affects the quality and intelligibility of the voice signal. Therefore, in many communication applications, effective voice enhancement processing is required to suppress noise and improve voice. Clarity, intelligibility and comfort. [0003] At present, the commonly used speech enhancement methods mainly include two categories. One is the speech enhancement method based on a single microphone, including spectral subtraction, Wiener filter, MMSE, Kalman filter, wavelet transform, etc. This type of method uses a single microphone to receive speech signals. The noise is suppressed by filtering and processing in the time ...

AI Technical Summary

Problems solved by technology

However, the existing microphone array speech enhancement technologies are all based on air-conduction speech sensors, which have the following deficiencies in practical applications: (1) when the environmental noise is strong, the output speech quality is not high; (2) when the environment is used When there are multiple sound sources in the environment, the estimation of the direction of arrival of the microphone array is prone to misjudgment; (3) When there are multiple sound sources in the use environment, the traditional direction of arrival estimation usually selects the sound source signal with the strongest sound. Enhanced, it is inconvenient for users to specify a certain sound source for monitoring

Images