Retrieval-Based Voice Conversion for Non-Native ASR Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Automatic Speech Recognition (ASR) systems struggle to accurately recognize and transcribe speech from non-native language speakers due to phonetic and prosodic variations, leading to high word error rates and inclusivity issues.
Innovation Solution
A method and system that converts non-native speech into a native speaker's voice before transcription using a Retrieval-Based Voice Conversion (RVC) technique, employing a phoneme encoder and a Mel-Spectrogram Generative Model to maintain linguistic content while altering acoustic properties to match native pronunciation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional ASR systems are used for non-native speakers, then the system is simple and easy to operate, but the recognition accuracy and transcription quality deteriorate due to phonetic and prosodic variations
Solution Approach 1:
The patent introduces voice conversion technology as an intermediary component between the speech input and ASR system. The voice conversion module transforms non-native speech into native-like speech patterns, serving as a mediator that bridges the gap between diverse accents and the ASR system's expectations, thereby improving recognition accuracy without requiring changes to the core ASR architecture
Solution Approach 2:
The system performs voice conversion before the speech recognition process. By pre-processing the speech signal to convert non-native phonetic and prosodic patterns into native-like patterns beforehand, the ASR system can process the converted speech with its existing capabilities, achieving high accuracy without needing to be retrained or modified
2Measurement precision
If voice conversion is applied to improve ASR performance for non-native speakers, then transcription accuracy improves, but processing time and computational resources increase
Solution Approach 1:
The system applies voice conversion selectively to only the phonetic and prosodic features that most impact ASR performance, rather than attempting to transform all aspects of speech. This partial application of voice conversion achieves sufficient accuracy improvement while limiting the additional processing time and computational overhead
3Adaptability or versatility
If traditional ASR systems are used without adaptation, then the device complexity remains low, but the system fails to accommodate phonetic and prosodic variations of different accents
Solution Approach 1:
The voice conversion module serves as an adaptable intermediary layer that can be integrated with existing ASR systems. This modular approach enables the system to accommodate various accents and languages without redesigning the core ASR architecture, maintaining relative simplicity while achieving high adaptability
Solution Approach 2:
The voice conversion system is designed to handle multiple languages and accents universally. By creating a language-agnostic conversion layer that can process diverse speech patterns, the system achieves broad adaptability across different linguistic contexts while using a consistent architectural framework
Data Source
AI summary
A computer-implemented method of generating speech from a source voice of a speaker in a language for a speech device connected to the computer, where the speech device language is a non-native language of the speaker. The method includes inputting speech in the source voice, encoding the speech in the source voice into a content vector, retrieving top k similar content vectors from a vector representation database that substantially match phonetic patterns of the encoded source voice, generating a new voice waveform based on the similar content vectors that contains linguistic content of the source voice, but with the phonetic patterns of a native speaker's voice of the speech device language, and outputting the new voice waveform.