Crosslingual Phoneme Mapping for Speech Synthesis

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Text-to-speech systems face challenges in rendering cross-lingual natural language text effectively, as they often struggle to convey unfamiliar language terms in a way that is comprehensible to users, leading to repeated requests for clarification and resource inefficiencies.

Innovation Solution

The implementation of a universal phoneme set that maps unfamiliar phonemes from secondary languages to familiar phonemes in a primary language, allowing for the generation of computer-generated speech that utilizes phonemes recognizable to the user, or using universal phonemes if the secondary language is familiar, thereby improving comprehension and reducing resource usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If universal phonemes from secondary language are used directly in speech generation, then native-like pronunciation is achieved, but user comprehension deteriorates when user is unfamiliar with the secondary language

Engineering Contradiction:

Improvepronunciation accuracyVSAvoiduser comprehension

Core Design Contradiction:

Measurement precisionVSLoss of information

Solution Approach 1:

The patent introduces a mapping mechanism that acts as an intermediary between universal phonemes and language-specific phonemes. When a user encounters an unfamiliar language, the system maps the universal phonemes to the user's familiar language phonemes through defined mappings, enabling comprehension while preserving the ability to use native pronunciation when appropriate.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Loss of information

If phoneme mapping to primary language is performed for all secondary language terms, then user comprehension improves, but native-like pronunciation is lost

Engineering Contradiction:

Improveuser comprehensionVSAvoidpronunciation accuracy

Core Design Contradiction:

Loss of informationVSMeasurement precision

Solution Approach 1:

The system dynamically adjusts phoneme mapping based on user familiarity with the secondary language. When the user is familiar with the secondary language, no mapping is performed and native phonemes are used directly. When the user is unfamiliar, mapping to the primary language is applied. This dynamic adaptation resolves the contradiction by selecting the appropriate phoneme representation contextually.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the phoneme representation parameter based on user language familiarity. The system detects whether the user understands the secondary language and accordingly switches between using universal phonemes directly (for familiar users) and mapped phonemes (for unfamiliar users), optimizing both comprehension and pronunciation accuracy.

Inventive Principle:

Principle #35Parameter changes

3Loss of information

If speech rendering is repeated due to poor comprehension, then user understanding may improve, but device resource consumption increases

Engineering Contradiction:

Improveuser comprehensionVSAvoiddevice energy consumption

Core Design Contradiction:

Loss of informationVSUse of energy by moving object

Solution Approach 1:

The system performs preliminary phoneme mapping before speech generation when the user is unfamiliar with the secondary language. By pre-processing the phonemes to match the user's familiar language, the system ensures comprehension on the first rendering, preventing the need for repeated speech output and thereby conserving device energy and resources.

Inventive Principle:

Principle #10Preliminary action

4Measurement precision

If language-specific phoneme sets are maintained for all languages, then native pronunciation is preserved, but system complexity increases

Engineering Contradiction:

Improvepronunciation accuracyVSAvoidphoneme set management

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent employs a universal phoneme set that can represent sounds from multiple languages. Instead of maintaining separate phoneme sets for each language, the universal set serves multiple functions across different languages. Language-specific phonemes are only invoked when needed through mapping, reducing overall system complexity while preserving native pronunciation capabilities.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12080271B2Speech generation using crosslingual phoneme mapping

Publication Date: 2024.09.03 GOOGLE LLC

AI summary

Computer generated speech can be generated for cross-lingual natural language textual data streams by utilizing a universal phoneme set. In a variety of implementations, the natural language textual data stream includes a primary language portion in a primary language and a secondary language portion that is not in the primary language. Phonemes corresponding to the secondary language portion can be determined from a set of phonemes in a universal data set. These phonemes can be mapped back to a set of phonemes for the primary language. Audio data can be generated for these phonemes to pronounce the secondary language portion of the natural language textual data stream utilizing phonemes associated with the primary language.