Real-Time Accent Conversion Using Linguistic Intermediary
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing software solutions for accent conversion between speakers of the same language fail to account for different pronunciations inherent to accents and introduce significant latency, making real-time communication impractical.
Innovation Solution
A computing device employs machine-learning models to receive input speech in a first accent, derive a linguistic representation, and synthesize it in a second accent with low latency using automatic speech recognition, voice conversion, and output speech generation engines, enabling real-time accent conversion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If voice conversion methods adjust audio characteristics to resemble target speaker's voice, then audio characteristics are improved, but pronunciation differences inherent to accents remain unaddressed
Solution Approach 1:
The patent introduces an intermediary representation (linguistic features or phoneme-level representation) between the source speech and target speech. This intermediary layer allows the system to separate pronunciation information from other speech characteristics, enabling selective conversion of audio characteristics while preserving or appropriately transforming pronunciation differences inherent to the source accent.
2Measurement precision
If STT-TTS conversion is used for accent conversion, then pronunciation accuracy is improved, but latency increases to several seconds
Solution Approach 1:
The patent extracts and removes the intermediate text conversion step from the traditional STT-TTS pipeline. By directly mapping source speech to target speech through learned transformations in latent space, the system eliminates the time-consuming text generation and re-synthesis process, reducing latency from several seconds to near real-time while maintaining pronunciation accuracy.
Solution Approach 2:
The system performs preliminary training to learn accent-specific transformations and pronunciation mappings before actual conversion. This pre-learning phase captures pronunciation patterns and relationships, enabling the model to perform rapid real-time conversion without needing to process through full text representations during actual use.
3Speed
If traditional voice conversion is used, then processing speed is improved, but accent nuances including pronunciation differences are lost
Solution Approach 1:
The patent applies local quality by treating different aspects of speech (pronunciation, pitch, timbre, rhythm) with different levels of transformation. Rather than uniformly converting all speech characteristics, the system selectively transforms audio characteristics while preserving or appropriately adapting pronunciation features that are inherent to the source accent, thereby maintaining accent nuances at the phoneme level.
Data Source
AI summary
Techniques for real-time accent conversion are described herein. An example computing device receives an indication of a first accent and a second accent. The computing device further receives, via at least one microphone, speech content having the first accent. The computing device is configured to derive, using a first machine-learning algorithm trained with audio data including the first accent, a linguistic representation of the received speech content having the first accent. The computing device is configured to, based on the derived linguistic representation of the received speech content having the first accent, synthesize, using a second machine learning-algorithm trained with (i) audio data comprising the first accent and (ii) audio data including the second accent, audio data representative of the received speech content having the second accent. The computing device is configured to convert the synthesized audio data into a synthesized version of the received speech content having the second accent.