CNN Sound Source Extraction from Multi-Channel Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional sound source extraction methods, such as center-channel extraction, often fail to accurately isolate sound sources due to reliance on volume and phase differences, leading to incomplete extraction of center sound sources or inclusion of ambient sounds.
Innovation Solution
A method and apparatus using a convolutional neural network (CNN) to transform multi-channel audio signals into 2D data, extracting feature maps from these signals to predict and separate both common and independent sound sources through operations between feature maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional center-channel extraction method using volume and phase difference is used, then the extraction process is simple, but the sound source extraction accuracy is poor and ambient sounds are included
Solution Approach 1:
The patent transforms the audio signal from traditional time-domain or frequency-domain representation to a time-frequency domain representation using short-time Fourier transform (STFT). This dimensional transformation allows the system to analyze both temporal and spectral characteristics simultaneously, enabling more accurate sound source extraction by identifying components that are consistent across multiple channels in the time-frequency domain, rather than relying solely on volume and phase differences in the time domain.
Solution Approach 2:
The patent introduces an intermediate processing stage that computes the magnitude and phase of the STFT for each channel, then uses these intermediate representations to identify center channel components. By computing the magnitude spectrum and phase information as intermediaries, the system can separate sound sources more accurately before final reconstruction, avoiding direct reliance on simple volume and phase difference comparisons.
2Ease of operation
If conventional reference value-based extraction is used, then the method is straightforward to implement, but center sound source components are not completely extracted and ambient sounds remain
Solution Approach 1:
The patent applies STFT to transform the audio signal into the time-frequency domain, adding a frequency dimension to the analysis. This allows the system to identify center channel components by examining their spectral characteristics across multiple channels simultaneously. The time-frequency representation enables more reliable detection of sound source components that are present in all channels, distinguishing them from ambient sounds that may not exhibit the same consistency pattern.
Solution Approach 2:
The patent replaces the conventional mechanical approach of direct volume and phase difference comparison with a spectral analysis-based method. Instead of mechanically comparing raw signal properties, the system substitutes this with Fourier transform-based spectral analysis, which provides a more sophisticated and reliable means of identifying sound source components by examining their frequency-domain characteristics across multiple channels.
Data Source
AI summary
Disclosed is an apparatus and method for extracting a sound source from a multi-channel audio signal. A sound source extracting method includes transforming a multi-channel audio signal into two-dimensional (2D) data, extracting a plurality of feature maps by inputting the 2D data into a convolutional neural network (CNN) including at least one layer, and extracting a sound source from the multi-channel audio signal using the feature maps.