Binaural Speech Enhancement with Self-External Speech Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech enhancement technologies fail to account for spatial aspects in binaural recordings, leading to significant differences in loudness and tonality between self- and external speech, and do not provide optimal processing for each type of speech.
Innovation Solution
A method for binaural recordings that identifies self- and external speech using time-frequency transforms, spectral features, and segmentation techniques, applying independent enhancement settings to each segment based on classification results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If speech enhancement is applied to binaural recordings without speaker segmentation, then processing simplicity is maintained, but tonality and loudness balance between self- and external speech deteriorates
Solution Approach 1:
The patent applies segmentation by dividing the binaural recording into separate self-speech and external-speech segments using speaker segmentation and diarization techniques. This allows independent enhancement processing for each segment type, optimizing tonality and loudness balance for each while maintaining overall system manageability through automated classification.
2Manufacturing precision
If speaker segmentation and classification are applied to differentiate self- and external speech, then tonality and loudness balance is improved, but processing complexity increases
Solution Approach 1:
The patent implements preliminary action by performing speaker segmentation and classification before the speech enhancement stage. This pre-processing step categorizes speech segments into self- or external-speech types, enabling subsequent optimized enhancement processing. The classification results guide the enhancement parameters applied to each segment, achieving balanced tonality and loudness efficiently.
3Device complexity
If spatial aspects in binaural recordings are ignored, then processing simplicity is maintained, but speech enhancement effectiveness deteriorates
Solution Approach 1:
The patent applies local quality by treating self-speech and external-speech segments with different enhancement settings and parameters. Self-speech segments receive processing optimized for near-field characteristics, while external-speech segments receive processing suited for far-field conditions. This localized approach to enhancement based on spatial classification improves overall effectiveness while maintaining manageable complexity through automated segmentation.
Data Source
AI summary
Disclosed herein are method, systems, and computer-program products for segmenting a binaural recording of speech into parts containing self-speech and parts containing external speech, and processing each category with different settings, to obtain an enhanced overall presentation. The segmentation is based on a combination of: i) feature-based frame-by-frame classification, and ii) detecting dissimilarity by statistical methods. The segmentation information is then used by a speech enhancement chain, where independent settings are used to process the self- and external speech parts.