Convolutional Non-Negative Matrix Factorization for Audio Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current audio signal processing techniques, such as non-negative matrix factorization (NMF), face challenges in effectively separating desired audio content from undesired content due to limitations in time and frequency resolution and the introduction of redundancy when attempting to isolate specific audio components.
Innovation Solution
The implementation of convolutional non-negative matrix factorization (CNMF) in multiple layers, where desired and undesired content are represented by separate dictionaries with controlled sparsity, allowing for the decomposition of multi-dimensional audio signals into their respective components through the use of trained dictionaries and coefficient matrices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a longer window is used in the analysis procedure, then frequency resolution is improved, but time resolution deteriorates
Solution Approach 1:
The patent applies segmentation by dividing the audio signal into multiple frames and processing each frame separately through the analysis procedure. This allows the system to use longer windows for each frame to achieve better frequency resolution while maintaining overall time resolution through the frame-by-frame processing approach.
2Manufacturing precision
If the rank I of the factorization is increased, then reconstruction quality is improved, but redundancy is introduced in the basis
Solution Approach 1:
The patent extracts only the most relevant basis vectors from the factorization process by selecting a subset of the I basis vectors that contribute most significantly to the reconstruction. This allows the system to achieve good reconstruction quality while avoiding the redundancy that would result from using all I basis vectors, thereby resolving the contradiction between reconstruction quality and redundancy.
3Measurement precision
If sparsity is imposed on the coefficient matrix to select relevant elements, then separation accuracy is improved, but complexity of the factorization process increases
Solution Approach 1:
The patent applies preliminary action by pre-training the dictionary W before performing the factorization process. This pre-training step prepares the basis vectors in advance, allowing the subsequent factorization with sparsity constraints to focus only on selecting the relevant pre-trained basis vectors rather than learning them from scratch. This reduces the overall complexity of the factorization process while maintaining separation accuracy.
Data Source
AI summary
The present disclosure provides new variants of non-negative matrix factorization suitable for separating desired audio content from undesired audio content. In certain embodiments, a multi-dimensional non-negative representation of an audio signal is decomposed into desired content and undesired content by performing convolutional non-negative matrix factorization (CNMF) on multiple layers, each layer having a respective non-negative matrix representation. In certain embodiments, the desired content is represented by a first dictionary and the undesired content is represented by a second dictionary, and sparsity is imposed on activations of basic elements of the first or the second dictionary, wherein a degree of sparsity is controlled by setting a minimum number of components with significant activations of the first or second dictionary, respectively.