Speech Synthesis with Nativized Phoneme Mapping for Foreign Fragments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional text-to-speech systems struggle to synthesize speech naturally when encountering foreign language fragments within native language text, often resulting in unnatural pronunciation shifts or omissions.
Innovation Solution
A method and system that processes textual inputs with both native and foreign language words by determining native and nativized phonetic representations, using an augmented phoneme set for foreign words to mimic a native speaker's pronunciation, incorporating contextual information and training data to enhance naturalness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional TTS systems use native language synthesis rules for the entire input including foreign fragments, then the synthesis process is simple and consistent, but the foreign fragments are pronounced unnaturally without reflecting how a native speaker of that foreign language would pronounce them
Solution Approach 1:
The system segments the input text into native language portions and foreign language fragments, then applies different synthesis rules to each segment. Foreign fragments are identified and processed separately using language-specific phoneme sets and pronunciation rules, while native portions use standard synthesis. This segmentation allows the system to maintain simplicity for the majority of text while achieving high pronunciation accuracy for foreign fragments.
2Manufacturing precision
If TTS systems switch between native and foreign synthesis rules for different language segments, then pronunciation accuracy of foreign fragments improves, but the synthesized speech sounds unnatural due to abrupt transitions between different speaker characteristics
Solution Approach 1:
The system applies local quality by using different phoneme sets and synthesis rules only in the specific local context of foreign language fragments, while the rest of the speech maintains the native speaker's characteristic pronunciation. This allows foreign fragments to be pronounced accurately according to their native language rules without forcing a complete switch to a foreign speaker's voice characteristics, thereby maintaining overall naturalness.
3Device complexity
If TTS systems replace foreign phonemes with closest native equivalents or use native grapheme-to-phoneme rules for foreign fragments, then the synthesis process remains simple, but aspects of the foreign language pronunciation are unnaturally omitted
Solution Approach 1:
The system introduces an intermediary phoneme set that contains both native language phonemes and additional phonemes specific to the foreign language. This intermediary phoneme set acts as a bridge, allowing the system to maintain a unified synthesis framework while preserving foreign language pronunciation characteristics that would otherwise be lost. The intermediary phonemes enable accurate representation of foreign sounds without requiring complete separation of synthesis pipelines.
Data Source
AI summary
A method for synthesizing speech from a textual input includes receiving the textual input, the textual input including native words in a native language and foreign words in a foreign language, and processing the textual input to determine a phonetic representation of the textual input. The processing includes determining a native phonetic representation of the of the native words, and determining a nativized phonetic representation of the foreign words. Determining the nativized phonetic representation includes forming a foreign phonetic representation of the foreign words using a foreign phoneme set, and mapping the foreign phonetic representation to the nativized phonetic representation according to a model of a native speaker's pronunciation of foreign words.