Dialog TTS Pronunciation Selection Using Confidence Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
During dialog interactions with voice-enabled devices, there is often a discrepancy between user pronunciation of words in spoken queries and the text-to-speech pronunciation of corresponding responses, leading to inconsistent and potentially incorrect synthesized speech output.
Innovation Solution
A pronunciation decision model is employed to determine which between user and text-to-speech pronunciations is more reliable by analyzing user and TTS pronunciation-related features, and updates based on user feedback to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If the TTS system uses its own pronunciation for words in responses, then the TTS output is consistent with its internal pronunciation rules, but it may conflict with the user's expected pronunciation based on their spoken query
Solution Approach 1:
The system dynamically adjusts the pronunciation selection between TTS pronunciation and user pronunciation based on real-time dialog context. The pronunciation decision model evaluates multiple factors including whether the word appears in the user's spoken query and compares pronunciations to determine the most appropriate selection, making the system adaptable rather than static
Solution Approach 2:
The system uses feedback from the user's spoken queries to inform pronunciation decisions in subsequent TTS responses. By analyzing the user's pronunciation of words in their queries and comparing it with the TTS pronunciation, the system learns and adapts to user preferences, creating a closed-loop feedback mechanism that improves pronunciation accuracy over time
2Adaptability or versatility
If the system always uses user pronunciation from spoken queries, then it adapts to user preferences, but it may introduce pronunciation errors when user pronunciation is incorrect or ambiguous
Solution Approach 1:
The pronunciation decision model acts as an intermediary between the user pronunciation and TTS pronunciation. It evaluates multiple factors including ASR confidence scores, whether the word is a proper noun, and the clarity of user pronunciation to mediate the selection process, preventing direct adoption of potentially erroneous user pronunciations
Solution Approach 2:
The system changes the selection parameter between TTS pronunciation and user pronunciation based on multiple conditional factors. When ASR confidence is low or the word is a proper noun with established pronunciation, the system parameters shift to favor TTS pronunciation. When user pronunciation is clear and confidence is high, parameters shift to favor user pronunciation, dynamically adjusting reliability based on contextual parameters
Data Source
AI summary
A method for instantaneous learning in text-to-speech (TTS) during dialog includes receiving a user pronunciation of a particular word present in a query spoken by a user. The method also includes receiving a TTS pronunciation of the same particular word that is present in a TTS input where the TTS pronunciation of the particular word is different than the user pronunciation of the particular word. The method also includes obtaining user pronunciation-related features and TTS pronunciation related features associated with the particular word. The method also includes generating a pronunciation decision selecting one of the user pronunciation or the TTS pronunciation of the particular word that is associated with a highest confidence. The method also include providing the TTS audio that includes a synthesized speech representation of the response to the query using the user pronunciation or the TTS pronunciation for the particular word.