Low-Order NMF Speech Enhancement With Multiple Dictionaries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech enhancement techniques, such as non-negative matrix factorization (NMF), are sensitive to initialization strategies and computationally costly, especially with higher-order factorizations, leading to inconsistent results and high computational overhead.
Innovation Solution
A semi-supervised speech enhancement system using low-order NMF with multiple dictionaries derived from clean speech samples, employing a statistical approach to generate enhanced speech representations by combining frequency-domain clean speech samples, and leveraging spatio-temporal continuity properties for robust prior modeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If higher-order NMF factorizations are used to improve speech enhancement quality, then the number of dictionary atoms increases, but computational resources (memory and floating point operations) increase proportionally to O(K)
Solution Approach 1:
The patent segments the speech enhancement task by using multiple low-order dictionaries (each with a small number of atoms K) instead of a single high-order dictionary. Each dictionary captures a specific aspect or mode of speech representation, and the final enhancement is achieved by combining results from multiple dictionaries. This segmentation reduces the computational complexity from O(K_high) to O(K_low × number of dictionaries), where K_low is much smaller than K High.
Solution Approach 2:
The patent introduces a new dimension by using multiple dictionaries with different numbers of atoms (under-complete and over-complete representations). Instead of relying on a single high-dimensional dictionary, the system combines results from multiple lower-dimensional dictionaries, each providing a different perspective on the speech signal. This dimensional approach allows the system to achieve high enhancement quality while maintaining low computational complexity for each individual dictionary.
2Ease of operation
If random initialization strategy is used for NMF to handle non-uniqueness of solutions, then implementation simplicity is improved, but factorization quality becomes inconsistent and depends heavily on initialization
Solution Approach 1:
The patent applies preliminary action by pre-computing multiple dictionaries with different numbers of atoms using a systematic approach rather than random initialization. The dictionaries are generated in advance with controlled numbers of atoms (under-complete and over-complete), and their representations are stored for later use in enhancing speech. This preliminary preparation eliminates the need for random initialization during the actual speech enhancement task, ensuring consistent and reliable factorization quality.
Solution Approach 2:
The patent changes the parameter of dictionary atom count to create multiple dictionaries with different dimensions (under-complete and over-complete representations). By varying this parameter across multiple dictionaries, the system captures different aspects of speech structure and achieves more robust factorization results. This parameter change approach replaces random initialization with a systematic variation of dictionary size, improving both consistency and quality.
3Productivity
If iterative updates are used to solve the non-convex NMF problem, then local minima can be reached, but globally optimal solutions cannot be guaranteed and quality depends heavily on initialization
Solution Approach 1:
The patent uses preliminary action by pre-computing multiple dictionaries with different numbers of atoms using a systematic approach rather than random initialization. The dictionaries are generated in advance with controlled numbers of atoms (under-complete and over-complete), and their representations are stored for later use in enhancing speech. This preliminary preparation eliminates the need for random initialization during the actual speech enhancement task, ensuring consistent and reliable factorization quality.
Solution Approach 2:
The patent segments the speech enhancement task by using multiple low-order dictionaries (each with a small number of atoms K) instead of a single high-order dictionary. Each dictionary captures a specific aspect or mode of speech representation, and the final enhancement is achieved by combining results from multiple dictionaries. This segmentation reduces the computational complexity from O(K High) to O(K Low × number of dictionaries), where K Low is much smaller than K High.
Data Source
AI summary
A system is provided that employs a statistical approach to semi-supervised speech enhancement with a low-order non-negative matrix factorization ("NMF"). The system enhances noisy speech based on multiple dictionaries with dictionary atoms derived from the same clean speech samples and generates an enhanced speech representation of the noisy speech by combining, for each dictionary, a clean speech representation of the noisy speech generated based on a NMF using the dictionary atoms of the dictionary. The system generates frequency-domain ("FD") clean speech sample representations of the clean speech samples, for example, using a Fourier transform. To generate each dictionary, the system generates a dictionary-unique initialization of the dictionary atoms and the activations and performs a NMF of the FD clean speech samples.