Voice Filter Model for Target Speaker Isolation in ASR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automatic speech recognition (ASR) systems face challenges in isolating the voice of a target speaker from overlapping speech and background noise, leading to reduced accuracy and increased error rates due to over-suppression of the target speaker's voice.
Innovation Solution
A voice filter model is integrated with the ASR system, utilizing a trained voice filter model to generate a predicted mask that isolates the frequency representation corresponding to the target speaker's voice, processed using an asymmetric loss function and adaptive suppression strength to mitigate over-suppression and enhance speech recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional ASR systems process audio data without voice separation, then the system is simpler and faster, but the accuracy is reduced due to overlapping speech and background noise
Solution Approach 1:
The patent segments the audio signal by speaker using voice activity detection and speaker diarization techniques. The system divides the audio stream into distinct speaker segments, allowing the ASR system to process each speaker's utterance separately, thereby improving accuracy without requiring complete audio reconstruction.
Solution Approach 2:
The patent introduces an intermediary processing layer between audio capture and ASR recognition. This intermediary layer includes voice activity detection, speaker diarization, and selective audio reconstruction components that prepare the audio data for more accurate recognition by isolating target speaker voices from background noise and other speakers.
2Measurement precision
If voice filter model is applied to isolate target speaker voice, then speech recognition accuracy is improved, but processing time and computational resources increase
Solution Approach 1:
The patent applies partial action by selectively reconstructing audio signals only for time segments and frequency bands where the target speaker is detected to be active. Instead of processing the entire audio stream through the voice filter model, the system identifies relevant segments using voice activity detection and applies the filter only to those segments, reducing computational overhead while maintaining accuracy.
Solution Approach 2:
The patent performs preliminary voice activity detection and speaker diarization before applying the voice filter model. This preliminary processing identifies which segments contain the target speaker's voice, allowing the computationally intensive audio reconstruction and filtering to be performed only on those specific segments rather than the complete audio stream.
3Object-affected harmful factors
If voice filter model processes audio data, then background noise is reduced, but the target speaker's voice may be over-suppressed
Solution Approach 1:
The patent implements feedback mechanisms where the voice filter model continuously monitors the processed audio output and compares it against the original audio and speaker embeddings. When over-suppression of the target speaker's voice is detected, the system adjusts the filter parameters in real-time to maintain adequate voice levels while continuing to suppress background noise and other speakers.
Solution Approach 2:
The patent dynamically changes the filter parameters based on the detected speaker embedding and voice activity. The voice filter model adjusts suppression strength, frequency response, and temporal characteristics of the filter in real-time based on the specific speaker's voice characteristics and the current audio context, preventing over-suppression while maintaining noise reduction.
Data Source
AI summary
Processing of acoustic features of audio data to generate one or more revised versions of the acoustic features, where each of the revised versions of the acoustic features isolates one or more utterances of a single respective human speaker. Various implementations generate the acoustic features by processing audio data using portion(s) of an automatic speech recognition system. Various implementations generate the revised acoustic features by processing the acoustic features using a mask generated by processing the acoustic features and a speaker embedding for the single human speaker using a trained voice filter model. Output generated over the trained voice filter model is processed using the automatic speech recognition system to generate a predicted text representation of the utterance(s) of the single human speaker without reconstructing the audio data.