Joint Audio Masking for Residual Echo and Reverberation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio systems face challenges in effectively mitigating additive noise, reverberation, and residual echo during voice communication, which degrade audio quality and intelligibility.
Innovation Solution
A deep neural network (DNN) with convolutional recurrent connectivity is employed to process audio signals, performing noise reduction, residual echo suppression, and dereverberation, using complex-valued spectrograms and additional input data to generate speech and noise masks, thereby isolating target speech and background noise.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-generated harmful factors
If acoustic echo cancellation (AEC) processing is performed to remove echo signals, then residual echo is reduced, but additive noise and reverberation remain in the output signal
Solution Approach 1:
The patent combines multiple audio processing functions (noise reduction, residual echo suppression, and dereverberation) into a single deep neural network model. The DNN processes the microphone signal, AEC output signal, and echo estimate signal simultaneously to generate speech and noise masks, achieving joint mitigation of additive noise, reverberation, and residual echo in one unified system rather than separate processing stages.
2Reliability
If deep neural network processing is applied to reduce noise, reverberation, and residual echo, then audio quality is enhanced, but computational complexity increases
Solution Approach 1:
The deep neural network is segmented into distinct functional components: an encoder that processes input signals, a decoder that generates output masks, and a mask application stage. The network processes complex-valued spectrograms by separating magnitude and phase information, and generates speech and noise masks that are applied to the AEC output signal. This segmentation allows for optimized computation at each stage while maintaining overall audio quality enhancement.
3Measurement precision
If speech and noise masks are generated to isolate target speech, then speech clarity is improved, but background noise may be excessively suppressed
Solution Approach 1:
The deep neural network dynamically adjusts the speech and noise masks based on the input signal characteristics. The network learns to differentiate between harmful background noise and useful ambient sounds, applying variable suppression levels across different time-frequency regions. This dynamic masking allows preservation of meaningful background noise while suppressing detrimental noise, reverberation, and residual echo, maintaining natural audio quality.
Data Source
AI summary
A system configured to improve audio processing by performing dereverberation, noise reduction, and residual echo suppression during a communication session. The system may include a deep neural network (DNN) configured to jointly mitigate additive noise, reverberation, and residual echo. The DNN may be a convolutional recurrent network with dense connectivity (CRN-DC) and may be configured to process complex-valued spectrograms corresponding to the isolated audio data and/or estimated echo data generated by during echo cancellation. The DNN may generate a speech mask and/or an ambient noise mask, enabling the device to generate output audio data representing target speech and a variable amount of ambient noise. For example, the device may separately reconstruct the target speech using the speech mask and the background noise using the ambient noise mask, which enables the device to control the amount of ambient noise represented in the output audio data.