Dual-Domain Speech Enhancement Using Time-Frequency Correlations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech enhancement methods fail to fully integrate frequency-domain and time-domain correlations, leading to suboptimal performance in separating speech from noise components.
Innovation Solution
A system that utilizes a classification network with an attention mechanism to generate a target feature representation based on both frequency and time correlations, enhancing speech components relative to noise components in the output audio signal.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing speech enhancement methods use only time-frequency domain or time-domain approaches, then the processing is simpler, but the speech separation performance is suboptimal
Solution Approach 1:
The patent merges time-frequency domain processing with time-domain processing by integrating a time-frequency network and a time-domain network into a unified dual-domain speech enhancement system. The time-frequency network processes spectral features while the time-domain network processes waveform features, and their outputs are combined to achieve superior speech separation performance that leverages the strengths of both domains simultaneously.
Solution Approach 2:
The patent transitions from single-domain processing to dual-domain processing by adding the time-frequency dimension to the conventional time-domain approach. This dimensional expansion allows the system to capture both spectral characteristics (frequency vs. time) and temporal characteristics (waveform evolution), providing a more comprehensive representation for speech enhancement.
2Reliability
If frequency and time correlations are not fully integrated, then the computational load is lower, but the speech enhancement performance deteriorates
Solution Approach 1:
The patent segments the correlation analysis into distinct frequency-domain correlation processing and time-domain correlation processing modules. The time-frequency network handles frequency correlation by analyzing spectral patterns across frequency bins, while the time-domain network handles temporal correlation by analyzing waveform patterns over time. This segmentation allows for specialized optimization of each correlation type while maintaining overall system efficiency.
Solution Approach 2:
The patent introduces an intermediary feature fusion module that combines the outputs of the time-frequency network and time-domain network. This intermediary component integrates the frequency correlation information and time correlation information in a computationally efficient manner, allowing the system to leverage both types of correlations without simply summing their computational costs, thus achieving better performance with optimized resource usage.
Data Source
AI summary
In accordance with implementations of the subject matter described herein, a solution for speech enhancement is proposed. In this solution, a target time-frequency representation at least indicating intensities of an input audio signal at different frequencies over time is obtained. The input audio signal comprises a speech component and a noise component. Frequency correlation information and time correlation information of the input audio signal is determined based on the target time-frequency representation. A target feature representation is generated based on the frequency correlation information, the time correlation information, and the target time-frequency representation. The target feature representation is for distinguishing the speech component and the noise component. An output audio signal is generated based on the target feature representation and the target time-frequency representation. The speech component is enhanced relative to the noise component in the output audio signal. In this way, the performance of speech enhancement can be improved.