Voice Morphing for Personalized ASR Across Diverse Voices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic speech recognition (ASR) and natural language understanding (NLU) systems suffer from inaccuracies due to variations in voice and environmental noise, leading to errors in phoneme inference and transcription.
Innovation Solution
The integration of a voice morphing model and an acoustic model, trained jointly or separately, to transform and condition speech audio to a target voice, incorporating voiceprint-based loss functions to minimize noise and distortion, ensuring intelligible and accurate phoneme inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a generic acoustic model is used for speech recognition, then the system can handle multiple voices and environments, but accuracy decreases due to voice variations and environmental noise
Solution Approach 1:
The system segments the speech recognition task into two distinct models: a voice morphing model that handles voice-specific transformations and an acoustic model that performs phoneme inference. This segmentation allows each model to specialize, with the voice morphing model adapting to different voices and the acoustic model maintaining high accuracy for phoneme recognition.
Solution Approach 2:
The voice morphing model acts as an intermediary between the input speech audio and the acoustic model. It transforms the input audio into a morphed version that is optimized for the acoustic model's phoneme inference, thereby improving overall accuracy while maintaining versatility across different voices.
2Measurement precision
If the system is personalized to a target voice using voice morphing, then phoneme inference accuracy improves, but the system becomes less adaptable to changing users
Solution Approach 1:
The voice morphing model is designed to be dynamic and retrainable. It can be trained on new voice data whenever a user changes, allowing the system to adapt to new users while maintaining the accuracy benefits of voice-specific personalization. The model's parameters can be updated without requiring complete system reconfiguration.
3Ease of manufacture
If voice morphing and acoustic models are trained separately, then training flexibility increases, but error accumulation occurs across models
Solution Approach 1:
The training process incorporates feedback mechanisms where the performance of the combined voice morphing and acoustic models is evaluated, and this feedback is used to adjust training parameters and strategies. This ensures that while training remains flexible and separate, the models are coordinated to minimize error accumulation.
4Measurement precision
If the acoustic model is trained on a single voice, then phoneme inference accuracy for that voice improves, but the system cannot handle multiple voices effectively
Solution Approach 1:
The system separates the voice-adaptation function (voice morphing model) from the phoneme-recognition function (acoustic model). This allows the acoustic model to be trained on a single target voice for high accuracy, while the voice morphing model handles the adaptation to different voices by transforming their speech into the target voice characteristics.
Data Source
AI summary
A voice morphing model can transform diverse voices to one or a small number of target voices. Speech recognition on diverse voices can be performed by morphing it to a target voice and then performing recognition on audio with the target voice. A source of requests for speech recognition can pass audio and a voiceprint with requests. Speech recognition can run with improved accuracy by biasing an acoustic model for the voice in the audio using the voiceprint. The audio can be used to calculate a new voiceprint, which can be used to update the voiceprint included with the audio. The updated voiceprint can be sent back to the source and then used with future speech recognition requests.