Speech Separation Algorithm Preserves Speaker Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech separation networks suffer from long-term forgetting issues when a speaker remains silent for an extended period, leading to separation errors due to the gradual loss of speech features and increased number of iterations, which reduces network performance.
Innovation Solution
A new speech separation algorithm that adaptively connects high-quality audio segments to preserve speaker features, using a high-quality audio segment search module and an adaptive path separation module to fuse hidden layer state information, thereby reducing distortion and improving signal-to-noise ratio.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If recurrent neural networks are used for speech separation, then sequence relationships and timing-dependent nature are captured well, but speaker features are lost when speakers remain silent for extended periods
Solution Approach 1:
The patent applies preliminary action by determining target audio segments in advance based on speech quality assessment before performing speech separation. The system identifies and selects audio segments with high speech quality metrics (such as signal-to-noise ratio, spectral flatness) ahead of time, creating a pool of reference segments that preserve speaker features. This preliminary selection ensures that even when speakers are silent, the system has pre-identified segments that can be used to maintain speaker characteristics, thereby preventing feature loss without requiring continuous speech input during the separation process.
2Measurement precision
If deep learning-based algorithms are used for speech separation, then nonlinear modeling capabilities are enhanced, but computational complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the audio signal into discrete audio segments and further classifying them into target audio segments based on speech quality. Instead of processing the entire audio stream as a single continuous signal, the system segments the audio into manageable units, evaluates their quality metrics, and selectively processes only the most informative segments. This segmentation approach reduces the computational burden on deep learning models by focusing calculations on relevant segments rather than the entire audio signal, thereby lowering overall model complexity while maintaining high separation accuracy.
3Productivity
If speech separation is performed continuously, then real-time processing is maintained, but speaker features deteriorate over time due to long-term forgetting
Solution Approach 1:
The patent implements feedback by continuously monitoring speech quality metrics of audio segments and using this information to adjust the speech separation process. The system evaluates speech quality parameters (such as signal-to-noise ratio, spectral characteristics) in real-time and feeds this information back to the separation algorithm. Based on this feedback, the system dynamically identifies target audio segments that best represent speaker features and uses these as references to guide the separation process. This feedback mechanism ensures that speaker features are continuously reinforced and updated, preventing the deterioration that would occur in continuous processing without quality-based adaptation.
Data Source
AI summary
The present disclosure provides a method performed by an electronic device and an apparatus. A method performed by an electronic device may include: obtaining an audio signal comprising a speech signal uttered by at least one sound source; determining a target audio segment of the audio signal, wherein the target audio segment is determined based on a speech quality of at least one audio segment, wherein the at least audio segment is divided from the audio signal; and performing speech separation on the audio signal based on the target audio segment to obtain at least one separated speech signal corresponding to the at least one sound source.