Digital Human Phoneme Personalization for Accurate TTS Responses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital humans lack the ability to predict user questions through observation and provide phonetically accurate responses, leading to reduced user engagement due to diminished physical interactions and difficulty in assessing user interests remotely.
Innovation Solution
Implement phoneme-based pronunciations using text-to-speech models, integrating user-provided pronunciation data and machine learning algorithms to enhance digital human responses, and adapt interactions based on user feedback and context.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If digital humans use conventional text-to-speech methods, then the system complexity is low, but the pronunciation accuracy and user engagement are reduced
Solution Approach 1:
The system performs preliminary actions by collecting and storing phoneme-based pronunciation data from multiple users before actual interaction. This pre-collected pronunciation data is stored in a database and retrieved during text-to-speech conversion, enabling accurate pronunciation without real-time complexity
Solution Approach 2:
The patent introduces phoneme-based pronunciation data as an intermediary layer between the text input and speech output. This intermediary layer contains pre-processed pronunciation information that mediates the conversion process, improving accuracy while reducing the computational complexity of real-time phonetic analysis
2Adaptability or versatility
If digital humans adapt to user preferences through learning, then the user engagement improves, but the processing time and computational resources increase
Solution Approach 1:
The system performs preliminary learning by collecting pronunciation preferences from multiple users during setup phases, storing these preferences in advance. During actual interaction, the system retrieves pre-processed pronunciation data rather than analyzing user preferences in real-time, reducing processing time while maintaining adaptability
Solution Approach 2:
The system implements self-service by automatically collecting, processing, and storing user pronunciation preferences without requiring manual configuration. The digital human autonomously adapts to user preferences by retrieving stored pronunciation data, eliminating the need for time-consuming manual setup while maintaining high adaptability
Data Source
AI summary
Techniques for providing phoneme-based pronunciations for digital humans are provided. One method comprises obtaining a response, generated by a language model, to be delivered by a digital human in a spoken format, wherein the response comprises multiple words; obtaining a phoneme-based pronunciation for one or more of the words, wherein the phoneme-based pronunciation is based on a user-provided pronunciation obtained from a user prior to the obtaining the response; and providing the phoneme-based pronunciation to the digital human in a processor-readable format, wherein the digital human transforms the processor-readable format into a spoken format using a text-to-speech model. The user-provided pronunciation may be provided by the user in a feedback manner to update a pronunciation employed by the digital human. The phoneme-based pronunciation may be obtained from a hierarchical phoneme repository that employs inheritance across hierarchical levels.