Dual-Path Speech Processing for Diarization and Target Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech processing technologies face high design costs due to the lack of a unified approach to address the challenges of speaker diarization and target speaker speech extraction, which are specific neural network models designed for different tasks in a related technology.
Innovation Solution
A unified approach to speaker diarization and target speaker extraction is achieved by using a neural network model that processes mixed speech and reference speech through a dual-path recurrent neural network and bidirectional long short-term memory network, enabling simultaneous support for speaker diarization and target speaker speech extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate neural network models are designed for speaker diarization and target speaker speech extraction, then task-specific performance is improved, but design costs and model complexity increase
Solution Approach 1:
The patent applies multi-functionality by designing a unified neural network model that can perform both speaker diarization and target speaker speech extraction tasks. The model uses a dual-path architecture where one path processes mixed speech for diarization while the other path extracts target speaker speech, allowing a single model to serve multiple purposes and reducing overall system complexity
Solution Approach 2:
The patent segments the neural network into two parallel processing paths: a first path for speaker diarization and a second path for target speaker speech extraction. Both paths share common input layers but diverge into separate processing branches that converge again, enabling independent optimization of each task while maintaining a unified model structure
2Device complexity
If a unified neural network model is used for both speaker diarization and target speaker speech extraction, then design costs are reduced, but task-specific accuracy may deteriorate
Solution Approach 1:
The unified model is segmented into distinct processing paths with specialized layers. The first path includes layers optimized for diarization tasks while the second path includes layers optimized for speech extraction, allowing each path to be tuned for its specific task while sharing the overall model framework
Solution Approach 2:
Different portions of the neural network are given different qualities and characteristics. The dual-path architecture allows each path to have its own specific activation functions, loss weights, and hyperparameters optimized for its respective task, while both paths benefit from the shared computational infrastructure
Data Source
AI summary
A speech processing method and a related device thereof are described. The method includes obtaining a mixed speech and a reference speech of a target object, where the mixed speech includes a speech of the target object and a speech of another object other than the target object. The method also includes processing the mixed speech, the reference speech, and an intermediate output of a second model by using a first model, to obtain an intermediate output of the first model and a final output of the first model, where the final output of the first model is used to obtain the speech of the target object. Furthermore, the method includes processing the mixed speech and the intermediate output of the first model by using the second model, to obtain the intermediate output of the second model and a final output of the second model.