Microphone Source Separation Using Learned Acoustic Paths
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Source separation in reverberant environments is challenging due to the spatial spread of echoes, making it difficult to effectively separate audio sources without prior knowledge of room characteristics or microphone setups.
Innovation Solution
A semi-supervised method is proposed for source separation, involving a training phase where parameters are learned for each individual source, and a testing phase where a reconstruction model is estimated using these learned parameters to achieve accurate source separation without interference from other sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If source separation is performed in reverberant environments using conventional methods, then separation can be achieved with prior knowledge of room characteristics, but the device complexity and difficulty of operation increase significantly
Solution Approach 1:
The system performs self-calibration by automatically estimating room impulse response and acoustic characteristics from the mixed audio signals themselves, without requiring external calibration equipment or prior knowledge of the recording environment. The calibration process is integrated into the source separation workflow, allowing the system to adapt to any reverberant environment autonomously.
Solution Approach 2:
The patent introduces intermediate processing steps including room impulse response estimation, spatial covariance matrix computation, and spectral basis function derivation that act as mediators between the raw mixed signals and the final separated sources. These intermediaries transform the complex reverberant environment into a manageable form for separation.
2Ease of operation
If source separation is performed without prior knowledge of room characteristics, then ease of operation improves, but measurement precision and reliability deteriorate
Solution Approach 1:
The system performs preliminary calibration by estimating room impulse response and acoustic characteristics from the mixed signals before executing the source separation. This preliminary action prepares the separation algorithms with environment-specific parameters, ensuring accurate separation without requiring user input about the recording conditions.
Solution Approach 2:
The system uses feedback mechanisms where the estimated spatial covariance matrices and room characteristics are continuously refined based on the separation results. The calibration parameters are adjusted iteratively to optimize separation performance, allowing the system to adapt to the specific acoustic environment automatically.
3Productivity
If conventional source separation methods are used in reverberant environments, then processing can be performed with available information, but loss of information increases due to spatial spread of echoes
Solution Approach 1:
The patent transforms the problem from the time domain to the frequency domain using short-time Fourier transform, and further to the spatial domain using spectral basis functions and spatial covariance matrices. This dimensional transformation allows the system to separate sources by exploiting spatial distribution patterns of reverberation, recovering information that would be lost in conventional time-domain approaches.
Solution Approach 2:
The system changes parameters by estimating frequency-dependent spatial covariance matrices and room impulse responses that capture the reverberant characteristics. By adapting these parameters to the specific acoustic environment, the system preserves source information that would otherwise be degraded by reverberation, maintaining high processing efficiency.
Data Source
Figure 1~2
Figure 3
Figure 4~8
AI summary
Embodiments of source separation for reverberant environment are disclosed. According to a method, first microphone signals for each individual one of at least one source are captured respectively by at least two microphones for a period during which only the individual one produces sounds. Mixing parameters for modeling acoustic paths between the at least one source and the at least two microphones are learned by a processor based on the first microphone signals. Second microphone signals are captured respectively by the at least two microphones for a period during which all of the at least one source produce sounds. The reconstruction model is estimated by the processor based on the mixing parameters and second microphone signals. The processor performs the source separation by applying the reconstruction model.