Target Speaker Speech Recognition With Joint Feature Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition technologies struggle to accurately identify the speech content of a target speaker from mixed speech, often due to insufficient voiceprint information in d-vectors and independent training of feature extraction and speech recognition models, leading to cascading errors and unsatisfactory recognition results.
Innovation Solution
A method involving joint training of a feature extraction model and a speech recognition model, utilizing multi-scale voiceprint features and a pre-established model to extract the target speaker's speech feature from mixed speech, followed by encoding and decoding to obtain accurate recognition results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If feature extraction and speech recognition models are trained independently, then training complexity is reduced, but recognition accuracy deteriorates due to cascading errors
Solution Approach 1:
The patent merges the feature extraction model and speech recognition model into a joint training framework. The two models are trained simultaneously with shared optimization objectives, allowing gradients to flow through both models together. This eliminates the cascading errors that occur when models are trained independently, as the feature extraction model learns features that are directly optimized for the speech recognition task while the recognition model adapts to the specific feature representation. The joint training process unifies the training complexity management while improving recognition accuracy through coordinated optimization.
2Speed
If only short-term voiceprint features are used, then processing speed is improved, but feature information completeness deteriorates
Solution Approach 1:
The patent segments the voiceprint feature extraction into multiple temporal scales: short-term features capture immediate acoustic characteristics for rapid processing, while long-term features capture broader speaker identity information for completeness. These segmented features are then fused in the joint training framework, allowing the system to leverage both the speed advantages of short-term features and the information completeness of long-term features. The segmentation approach enables hierarchical processing where different temporal granularities serve different functional needs.
Solution Approach 2:
The patent adds a temporal dimension to the feature extraction by incorporating both short-term and long-term voiceprint features. This dimensional expansion allows the system to process speech at multiple time scales simultaneously, enriching the feature space without sacrificing processing efficiency. The multi-scale temporal features provide complementary information that improves recognition accuracy while maintaining acceptable processing speeds through the joint optimization framework.
3Loss of information
If multi-scale voiceprint features are extracted, then voiceprint information richness is improved, but computational complexity increases
Solution Approach 1:
The patent combines multiple voiceprint feature extraction tasks (short-term and long-term) into a unified joint training framework with the speech recognition model. By merging these operations, the system shares computational resources and optimization processes across all feature extraction and recognition tasks. The joint training allows gradients from the recognition loss to propagate through both short-term and long-term feature extraction pathways simultaneously, efficiently utilizing computational resources while achieving rich multi-scale feature representation.
Data Source
AI summary
Provided in the present application are a speech recognition method, apparatus and device, and a storage medium. The method comprises: acquiring a speech feature of target mixed speech and a speaker feature of a specified speaker; taking the direction of tending to a target speech feature as an extraction direction, and according to the speech feature of the target mixed speech and a speaker feature of a target speaker, extracting a speech feature of the target speaker from the speech feature of the target mixed speech, so as to obtain an extracted speech feature of the target speaker; and acquiring a speech recognition result of the specified speaker according to an extracted speech feature of the specified speaker.