Multichannel Speech Enhancement Using Phase-Aware Spectral Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Speech processing systems suffer from recognition accuracy degradation due to far-field speech signal corruption by reverberation and background noise, which is not effectively addressed by existing single-microphone systems.
Innovation Solution
Employing a microphone array to capture multichannel audio signals, utilizing spatial information through modified discrete cosine transform (MDCT) to generate a time-frequency representation, and applying deep neural networks (DNNs) to enhance speech recognition by preserving phase information and determining direction of arrival (DOA) for improved signal processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single microphone is used to capture audio signals, then the device complexity is low, but the speech recognition accuracy degrades due to signal corruption from reverberation and background noise
Solution Approach 1:
The audio signal processing is segmented into multiple stages: multichannel capture, spectral representation generation, magnitude/phase weight computation, and enhanced signal reconstruction. Each stage processes specific aspects of the signal independently, allowing the system to handle complex acoustic environments through modular processing steps rather than a single complex filter
Solution Approach 2:
A speech enhancement frontend component is introduced as an intermediary between the multichannel microphone array and the single-channel ASR backend. This frontend generates spectral representations and computes magnitude- and phase-dependent weights to transform multichannel signals into enhanced single-channel signals, bridging the gap between the two systems while preserving spatial information
2Measurement precision
If a microphone array is used to capture multichannel audio signals, then the speech enhancement accuracy is improved, but the processing complexity increases due to the need for spectral transformation and spatial information extraction
Solution Approach 1:
The patent replaces traditional mechanical signal processing methods with neural network-based computation. A neural network is trained to directly predict magnitude- and phase-dependent weights from spectral representations, substituting complex mathematical transformations and spatial algorithms with a learned mapping that achieves the same enhancement goals with simpler real-time processing
Solution Approach 2:
The system transforms the audio signal into the spectral domain using Short-Time Discrete Cosine Transform (ST-DCT), changing the representation parameters from time-domain samples to frequency-domain coefficients. This parameter transformation enables more effective computation of magnitude and phase weights and facilitates better speech enhancement in the spectral representation
3Object-affected harmful factors
If traditional single-channel audio processing is used, then the processing speed is high, but the ability to mitigate reverberation and background noise is insufficient
Solution Approach 1:
The patent converts the harmful effects of reverberation and background noise into useful spatial information. By processing multichannel signals and computing magnitude- and phase-dependent weights, the system extracts directional cues and spatial characteristics that were originally part of the noise problem, then uses this information to enhance the speech signal and suppress reverberation and noise
Data Source
AI summary
A method, computer program product, and computing system for enhancement of audio signals received from a plurality of microphones. A multichannel audio signal is received from a plurality of microphones and is processed with a short-time discrete cosine transform (STDCT) to generate a real-valued spectral representation of the multichannel signal encoding both magnitude and phase information. Magnitude- and phase-dependent weights are generated, and an enhanced single-channel signal is produced based upon, at least in part, the spectral representation of the multichannel signal and the magnitude- and phase-dependent weights.