Real-Time Speech Enhancement Using NMF Noise Suppression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hearing impaired individuals face difficulties in distinguishing speech from background noise in noisy environments, as existing hearing aids and assistive technologies often amplify noise indiscriminately, and current software solutions are insufficient for real-time speech isolation in live conversations.
Innovation Solution
The development of systems and methods using deep learning and Non-negative Matrix Factorization (NMF) to identify and enhance speech in real-time by distinguishing speech from background noise, employing a trained NMF dictionary to suppress noise and amplify speech, implemented in lightweight, on-ear or in-ear modules with wireless communication and signal processing capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If traditional hearing aids amplify sound indiscriminately, then the volume of speech is increased, but the volume of background noise is also increased making it difficult to distinguish speech from noise
Solution Approach 1:
The patent segments the mixed audio signal into distinct speech and noise components using Non-negative Matrix Factorization (NMF). The NMF algorithm decomposes the audio signal matrix into two non-negative matrices: one representing speech sources and another representing noise sources. This segmentation allows selective amplification of speech while suppressing noise, resolving the contradiction between increasing speech volume and avoiding noise amplification.
Solution Approach 2:
The patent applies different processing qualities to different components of the audio signal. Speech components are amplified with high fidelity and minimal distortion, while noise components are suppressed or filtered out. This local differentiation of quality ensures that the useful speech signal is enhanced while the harmful noise is eliminated, rather than treating all audio components uniformly.
2Measurement precision
If offline speech separation algorithms are used, then speech can be isolated from mixed audio signals, but the processing cannot be performed in real-time for live conversations
Solution Approach 1:
The patent performs preliminary action by pre-training the NMF algorithm with large amounts of audio data to learn speech and noise patterns. This offline training phase prepares the system in advance, storing learned speech and noise dictionaries that can be rapidly applied during real-time processing. The computationally intensive learning phase is separated from the real-time application phase, allowing high accuracy without real-time training overhead.
Solution Approach 2:
The patent implements dynamic adaptation by continuously updating the NMF decomposition as new audio data arrives. The system dynamically adjusts the speech and noise components based on changing acoustic conditions, speaker characteristics, and environmental factors. This dynamic approach maintains high speech isolation accuracy while adapting to real-time conditions without requiring complete reprocessing of historical data.
3Measurement precision
If complex deep learning models are used for speech enhancement, then speech isolation accuracy is improved, but device complexity and computational requirements increase
Solution Approach 1:
The patent extracts the essential speech and noise characteristics into separate NMF component matrices, isolating the core features needed for speech enhancement. By extracting only the relevant speech and noise components rather than processing the entire complex audio signal, the system achieves high distinction accuracy with reduced computational complexity. The NMF decomposition extracts the essential structure of the audio signal into manageable components.
Solution Approach 2:
The patent changes the parameter representation of the audio signal by transforming it from the time domain to a non-negative matrix factorization domain. This parameter transformation allows the system to work with simplified non-negative constraints and multiplicative updates rather than complex iterative optimization, reducing computational complexity while maintaining or improving speech distinction accuracy through the structured parameter space of NMF.
Data Source
AI summary
A system may perform speech enhancement of audio data in real-time by suppressing noise components that are present in the audio data while preserving speech components. The system may include an in-ear module and a separate signal processing module that is wirelessly communicatively coupled to the in-ear module. The system may include non-negative matrix factorization (NMF) dictionaries capable of identifying frequency band components associated with speech and frequency band components associated with noise. The NMF dictionaries may be trained using voice samples and noise samples. The NMF dictionaries may be applied to noisy speech data to produce an NMF representation of the speech data which may then be applied using a dynamic mask to the noisy speech data in order to suppress the noise components of the noisy speech data and produce speech enhanced data.