Speech Enhancement Using Acoustic Environment Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video conferencing systems face challenges in managing heterogeneous and dynamic audio environments, leading to noise interference and degraded speech quality, with existing client-side solutions suffering from latency and resource inefficiencies.
Innovation Solution
A server-side speech detection and enhancement system for Multipoint Conferencing Nodes (MCNs) is trained on acoustic environments, using Time-Frequency domain features and statistical generative models to differentiate speech from noise, applying noise reduction masks tailored to specific environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If client-side speech detection and enhancement solutions are used, then speech quality can be improved, but computational requirements increase and latency issues occur
Solution Approach 1:
The patent introduces an acoustic environment classifier as an intermediary component that categorizes the audio environment first, then selects appropriate speech enhancement models. This mediator reduces the computational burden by avoiding the need to run all possible enhancement algorithms, thereby reducing energy consumption while maintaining speech quality improvement.
Solution Approach 2:
The system dynamically changes parameters by selecting different speech enhancement models based on the classified acoustic environment. Instead of using a fixed high-computation model always, the system adapts the processing intensity to match the environment type, reducing computational requirements in simple environments while maintaining quality when needed.
2Manufacturing precision
If deep learning models are used for speech enhancement, then speech intelligibility improves, but device complexity increases
Solution Approach 1:
The patent segments the speech enhancement task into multiple stages: acoustic environment classification, model selection, and speech enhancement. This segmentation allows the system to use simpler models for specific environments rather than deploying complex universal models, thereby reducing overall device complexity while maintaining speech intelligibility through specialized processing.
Solution Approach 2:
The system dynamically selects speech enhancement models based on the classified acoustic environment. This dynamic approach allows the system to use lighter models for simple environments and more sophisticated models only when necessary, reducing the average device complexity while preserving speech intelligibility in challenging conditions.
3Object-affected harmful factors
If noise reduction is applied aggressively, then noise is reduced, but the underlying acoustic background is lost
Solution Approach 1:
The patent applies different speech enhancement strengths based on the local characteristics of the acoustic environment. For environments where acoustic background is important (like cafes or offices), the system applies milder enhancement to preserve the background ambiance, while applying stronger reduction only in environments where noise is the primary concern. This local quality approach ensures noise reduction without universal loss of acoustic background.
Data Source
AI summary
A method of speech detection, speech enhancement, and training in a speech detection and speech enhancement unit. The method comprising receiving input audio segments, and determining an acoustic environment based on input audio auxiliary information, extracting T-F-domain features from the received input audio segments, determining if each of the received input audio segments is speech by inputting the T-F domain features into a speech detection classifier trained for the determined acoustic environment, determining, when one of the received input audio segments is speech, if the received audio segment is noisy speech by inputting the T-F domain features into a noise classifier using a statistical generative model representing the probability distributions of the T-F domain features of noisy speech trained for the determined acoustic environment, and applying a noise reduction mask on the received input audio segments according to the determination of the received audio segment is noisy speech.