Iterative Neural Network Mask Estimation for Sound Source Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sound source separation technologies using a single microphone struggle to handle an unknown number of sound sources, as they require pre-defined clusters for k-means clustering and are limited by the number of nodes in the neural network's output layer.
Innovation Solution
An estimation device and method that iteratively processes the input audio signal using a neural network, updating mask information by changing supplemental information in each iteration, allowing for the estimation of mask information for an arbitrary number of sound sources without being constrained by the number of nodes in the neural network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If k-means clustering is used for sound source separation, then the separation can be performed without sound source position information, but the number of sound sources must be known in advance which limits handling of unknown number of sound sources
Solution Approach 1:
The patent employs iterative processing where the neural network repeatedly processes the audio signal with updated mask information. In each iteration, the system estimates masks for sound sources based on current cluster assignments, then uses these masks to refine the clustering in the next iteration. This dynamic iterative approach allows the system to adapt to any number of sound sources without requiring pre-defined cluster counts, resolving the contradiction between adaptability and complexity.
2Adaptability or versatility
If the number of nodes in the neural network output layer is increased to handle more sound sources, then the separable number of sound sources increases, but the device complexity and training difficulty increase
Solution Approach 1:
The patent segments the sound source separation task into multiple iterations, where in each iteration the neural network processes the audio signal to estimate masks for a subset of sound sources. Instead of requiring the network to simultaneously handle all possible sound sources with a large output layer, the system divides the problem across time steps, with each iteration focusing on a specific subset. This segmentation allows the network to maintain a fixed, manageable number of nodes while effectively handling arbitrary numbers of sound sources through iterative processing.
Solution Approach 2:
The system dynamically adjusts the effective number of active nodes in the output layer across iterations. By using iterative mask estimation and updating which sound sources are actively processed in each iteration, the system can adapt the effective capacity to match the actual number of sound sources present, rather than requiring a statically configured large output layer. This dynamic approach resolves the contradiction by allowing flexible adaptation without increasing structural complexity.
3Ease of operation
If directly outputting mask estimations from the neural network is used, then the process is simplified, but the number of nodes in the output layer limits the separable number of sound sources
Solution Approach 1:
The patent segments the mask estimation process into multiple iterations, where the neural network outputs mask information for a subset of sound sources in each iteration. This segmentation allows the network to maintain a simple, fixed architecture with a limited number of output nodes while still being able to handle arbitrary numbers of sound sources by distributing the estimation task across multiple time steps. The simplicity of direct output is preserved in each iteration, while the overall system achieves high adaptability through the iterative structure.
Data Source
AI summary
An estimation device includes a memory, and processing circuitry coupled to the memory and configured to receive an input of an input audio signal that is an audio signal in which sounds from a plurality of sound sources are mixed, and an input of supplemental information, and output an estimation result of mask information that identifies a mask for extracting a sound of any one of the sound sources included in an entire or a part of a signal included in the input audio signal, the signal being identified by the supplemental information, cause a neural network to iterate a process of outputting the estimation result of the mask information, and cause the neural network to output an estimation result of the mask information for a different sound source, by inputting a different piece of the supplemental information to the neural network at each iteration.