Polyglot TTS Phoneme Mapping for Foreign Language Pronunciation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Text-to-Speech (TTS) systems struggle to accurately pronounce foreign language words and phonemes, leading to simplistic output and increased memory requirements, as they are typically designed for mono-lingual input and lack the ability to produce phonemes unique to other languages, resulting in inconsistent voice identity and unnatural pronunciation.
Innovation Solution
A method that generates a target sequence of language-dependent phoneme symbols by identifying specific foreign languages, applying monolingual phoneme resources, replacing foreign phoneme symbols with language-independent substitutes, and deriving a target unit sequence for speech waveform generation, ensuring consistent and proficient foreign pronunciation while maintaining a unique speaker identity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If foreign language words are transcribed using the phoneme set of the native language, then the TTS system can pronounce foreign words, but the pronunciation becomes inappropriate and loses foreign language characteristics
Solution Approach 1:
The patent introduces an intermediary phoneme mapping layer between the foreign language input and the native language phoneme database. Instead of directly transcribing foreign words using native phonemes, the system first converts foreign words to a language-independent phoneme representation (such as IPA), then maps these to the nearest native phonemes. This intermediary step preserves foreign language characteristics while enabling pronunciation through the native phoneme set.
Solution Approach 2:
The patent changes the parameter representation of phonemes by introducing language-specific phoneme sets and mapping parameters. Each language has its own phoneme set with specific acoustic and articulatory parameters. The system dynamically adjusts which phoneme set to use based on the detected language, and uses parameter-based mapping (such as phonetic feature vectors) to translate between different phoneme representations, thereby maintaining pronunciation accuracy across languages.
2Adaptability or versatility
If a list of foreign words is added to the pronunciation lexicon, then foreign words can be pronounced, but memory space requirements increase significantly
Solution Approach 1:
The patent creates a universal phoneme mapping mechanism that works across multiple languages without requiring separate word lists for each language combination. The language-independent phoneme representation (e.g., IPA) serves as a universal intermediary that can map foreign words from any language to the native phoneme set. This multi-functional approach allows the same mapping infrastructure to handle any foreign language, eliminating the need for extensive language-specific lexicon expansions.
Solution Approach 2:
Instead of storing actual foreign language words and their pronunciations in the lexicon, the system uses phoneme mapping rules that generate pronunciations on-the-fly. The phoneme mapping layer effectively creates a virtual copy of the foreign word pronunciation by translating the phoneme sequence rather than storing pre-recorded pronunciations. This reduces memory requirements while maintaining pronunciation quality.
3Manufacturing precision
If the TTS system switches to a different voice for foreign language words, then foreign phonemes can be produced, but the speaker identity becomes inconsistent
Solution Approach 1:
The patent applies local quality by allowing different phoneme representations for different language segments within the same speech output. Each foreign language word is identified and processed with its specific phoneme set, while the rest of the speech uses the native phoneme set. This localized application of language-specific phonemes within a unified speaker voice maintains both foreign phoneme accuracy and speaker identity consistency.
Solution Approach 2:
The patent segments the input text into native language and foreign language portions, applying different phoneme mapping rules to each segment. The language detection and segmentation module identifies foreign language words, and the system processes these segments separately through appropriate phoneme mappings before synthesizing the final speech. This segmentation allows the TTS to produce foreign phonemes accurately while maintaining a consistent speaker identity throughout the entire utterance.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
A method is disclosed to generate a speech output from a text input written in a first language and containing inclusions in a second language. The speech output generated by the disclosed method is characterized by a consistent, unique speaker identity. Words in the native language are pronounced with a native pronunciation and words in the foreign language are pronounced with a proficient foreign pronunciation. Language dependent phoneme symbols generated for words of the second language are replaced with language dependent phoneme symbols of the first language, where said replacing includes the steps of assigning to each language dependent phoneme symbol of the second language a language independent target phoneme symbol, mapping to each one language independent target phoneme symbol a language independent substitute phoneme symbol assignable to a language dependent substitute phoneme symbol of the first language, substituting the language dependent phoneme symbols of the second language by the language dependent substitute phoneme symbols of the first language. This results in a target unit sequence of phoneme symbols of the first language. From a waveform unit database of the first language a waveform unit sequence approximating the at least one target unit sequence is selected and concatenated to the speech waveform.