Target Speaker Extraction Using Multi-Model Audio Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio processing systems struggle to effectively extract the speech of a target speaker from a mixture of audio signals, especially in environments with multiple speakers.
Innovation Solution
A system comprising a multi-speaker detection model, a target speaker voice activity detection (VAD) model, and a speech separation model, which analyzes audio frames to determine the presence of a single target speaker or multiple speakers, and accordingly suppresses or separates the target speaker's voice using voiceprints and machine learning algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional audio processing methods are used to extract target speaker speech, then the system simplicity is maintained, but the extraction effectiveness and audio clarity deteriorate in multi-speaker environments
Solution Approach 1:
The system segments the audio processing task into three distinct modules: multi-speaker detection model that identifies the number of speakers, target speaker VAD model that detects target speaker presence, and speech separation model that separates mixed speech signals. This segmentation allows each model to specialize in a specific function, improving overall extraction effectiveness while managing system complexity through modular architecture
Solution Approach 2:
The patent introduces voiceprint templates as an intermediary element that mediates between the audio signal and the detection models. The voiceprint template serves as a reference for comparing against detected speech segments, enabling accurate target speaker identification without requiring direct complex processing between the audio signal and the final output
2Object-affected harmful factors
If simple audio processing is used, then the system ease of operation is maintained, but the ability to suppress non-target speaker speech deteriorates
Solution Approach 1:
The system implements feedback mechanisms where the multi-speaker detection model continuously monitors the audio input and dynamically adjusts the processing pipeline. When multiple speakers are detected, the system activates the speech separation model with feedback from the voiceprint comparison results, enabling adaptive noise suppression that responds to changing speech conditions in real-time
Solution Approach 2:
The patent changes key parameters dynamically: the system adjusts the processing mode (single-speaker vs. multi-speaker detection) based on detected conditions, modifies voiceprint template matching thresholds adaptively, and switches between different model outputs based on the number of detected speakers. These parameter changes enable effective noise suppression without requiring permanently complex system architecture
3Reliability
If advanced speech separation models are deployed, then the audio clarity improvement is achieved, but the processing time and computational resources increase
Solution Approach 1:
The system dynamically selects processing paths based on real-time conditions: when only one speaker is detected, it uses the faster VAD-based approach; when multiple speakers are detected, it activates the more computationally intensive speech separation model. This dynamic adaptation ensures audio clarity when needed while minimizing processing time in simpler scenarios
Solution Approach 2:
The system performs preliminary voiceprint template creation and multi-speaker detection before committing to full speech separation processing. This preliminary action allows the system to prepare necessary components in advance and only invoke the computationally heavy speech separation model when actually needed, reducing overall processing time while maintaining audio clarity
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media relate to a method for target speaker extraction. A target speaker extraction system receives an audio frame of an audio signal. A multi-speaker detection model analyzes the audio frame to determine whether the audio frame includes only a single-speaker or multiple speakers. When the audio frame includes only a single-speaker, the system inputs the audio frame to a target speaker VAD model to suppress speech in the audio frame from a non-target speaker based on comparing the audio frame to a voiceprint of a target speaker. When the audio frame includes multiple speakers, the system inputs the audio frame to a speech separation model to separate the voice of the target speaker from a voice mixture in the audio frame.