Personalized Text-to-Speech Voice Data Collection by Conversational Prompting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for personalized text-to-speech (P-TTS) systems rely on users reading text to gather sound sources, which often results in unnatural speech and imbalances in sound source data, affecting the quality of personalized voice synthesis.
Innovation Solution
An electronic device uses a chat-bot to engage users in conversation, identifying pronunciation variants with insufficient data and prompting relevant questions to gather natural utterances, thereby enhancing the quality and balance of sound sources for P-TTS.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a separate speech synthesis process is used for each language, then speech synthesis accuracy for each language is improved, but device complexity and processing overhead increase
Solution Approach 1:
The patent implements a single speech synthesis model that can handle multiple languages through language identification and code switching mechanisms. The NLP module detects the input language and routes it to the appropriate synthesis path, allowing one universal model to perform multiple language-specific functions without requiring separate dedicated models for each language.
Solution Approach 2:
The patent introduces an NLP module as an intermediary between text input and speech synthesis. This mediator identifies the language type and determines whether to apply code switching or direct synthesis, thereby managing the complexity of multi-language support without requiring the core synthesis engine to be restructured for each language.
2Speed
If code switching is performed without accurate language identification, then speech synthesis speed is improved, but synthesis accuracy deteriorates
Solution Approach 1:
The patent performs language identification through the NLP module before initiating the speech synthesis process. By determining the input language in advance, the system can pre-select the appropriate synthesis path (code switching or direct synthesis), avoiding the need for post-processing corrections and ensuring accurate synthesis from the start.
Solution Approach 2:
The system incorporates feedback mechanisms where the NLP module continuously monitors input text characteristics and adjusts language identification accordingly. This feedback loop ensures that even in mixed-language inputs, the system accurately determines which portions require code switching and which can be synthesized directly, maintaining both speed and accuracy.
3Device complexity
If mixed-language text is synthesized without code switching, then processing simplicity is maintained, but speech naturalness and accuracy deteriorate
Solution Approach 1:
The patent segments mixed-language text into distinct language portions through NLP analysis. Each segment is identified by its language type and processed accordingly - some segments undergo code switching while others are synthesized directly. This segmentation allows the system to handle complexity only where necessary rather than applying uniform complex processing to all text.
Solution Approach 2:
The system dynamically adjusts its synthesis approach based on the detected language composition of the input text. Rather than using a fixed processing method, the system flexibly switches between code switching and direct synthesis modes for different segments, optimizing both accuracy and processing efficiency for each specific input scenario.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
According to an embodiment, an electronic device may comprise a memory and at least one processor operatively connected to the memory, wherein the at least one processor: when a speech assistant application is executed, identifies a pronunciation string, in which the quantity of sound source data stored in the memory has a value less than a specific value, among a plurality of pronunciation strings; identifies a topic on the basis of the identified pronunciation string; obtains a question text corresponding to a word including the identified pronunciation string among a plurality of words included in the topic; outputs a question speech corresponding to the question text; and receives an utterance after outputting the question speech..