Two-Stage NMF Speech Modeling for Residual Noise Suppression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech enhancement systems often suffer from residual noise due to under-estimation of noise references and corrupted speech envelopes in sub-bands, leading to suboptimal quality of noise-suppressed speech outputs.
Innovation Solution
The method employs a two-stage dictionary-based approach using Non-negative Matrix Factorization (NMF) for speech modeling, where a first stage reduces residual noise using a Wiener filter and a second stage restores the speech envelope, utilizing speaker-specific dictionaries and real-time noise dictionaries to differentiate noise and speech harmonics effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single-stage noise suppression method is used, then the processing complexity is low, but residual noise remains due to under-estimation of noise references
Solution Approach 1:
The noise suppression process is divided into two distinct stages: a first stage that performs initial noise suppression to reduce computational complexity, and a second stage that performs residual noise suppression to improve accuracy. This segmentation allows the system to balance between processing complexity and noise suppression accuracy by handling different noise components in separate stages with appropriate algorithms for each.
2Reliability
If aggressive noise suppression is applied, then residual noise is reduced, but speech envelope corruption occurs in sub-bands
Solution Approach 1:
The system applies different processing strategies to different frequency sub-bands based on their local characteristics. In sub-bands dominated by speech, the system preserves the speech envelope structure, while in noise-dominated sub-bands, more aggressive suppression is applied. This local quality approach ensures that speech information is preserved where present while effectively suppressing noise where dominant.
Solution Approach 2:
The noise suppression algorithm dynamically adjusts its behavior based on the local signal-to-noise ratio and speech activity detection in each sub-band. The system adapts the suppression strength and envelope preservation parameters according to the instantaneous characteristics of each frequency band, allowing optimal performance across varying noise conditions without corrupting speech envelopes.
3Measurement precision
If real-time noise dictionary is obtained based on noisy speech spectrum, then noise modeling accuracy is improved, but computational load increases
Solution Approach 1:
The system performs preliminary noise dictionary construction during periods when noise is dominant or speech activity is low, preparing noise models in advance before they are needed for suppression. This preliminary action allows the computationally intensive dictionary construction to be performed when it has the greatest impact, reducing the real-time computational load during active speech periods while maintaining high noise modeling accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for speech modeling by an electronic device is described. The method includes obtaining a real-time noise reference based on a noisy speech signal. The method also includes obtaining a real-time noise dictionary based on the real-time noise reference. The method further includes obtaining a first speech dictionary and a second speech dictionary. The method additionally includes reducing residual noise based on the real-time noise dictionary and the first speech dictionary to produce a residual noise-suppressed speech signal at a first modeling stage. The method also includes generating a reconstructed speech signal based on the residual noise-suppressed speech signal and the second speech dictionary at a second modeling stage.