Intonation Template Control for Intent-Aware Voice Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional voice synthesis technologies struggle to naturally reflect the intonation matching the intention of an utterance, particularly in virtual assistant services and robot technology, and require significant data and time for neural network training to adjust voice intonation.
Innovation Solution
An electronic device with a memory and processor that utilizes an intonation template generator trained through unsupervised learning to classify intonation templates based on voice signals, allowing it to synthesize voice that matches the user's intended intonation by identifying appropriate templates and encoding intonation information for voice synthesis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional voice synthesis technology is used, then text can be recited in a certain style, but it cannot naturally reflect intonation matching the intention of an utterance
Solution Approach 1:
The patent segments intonation control from overall voice style by introducing intonation templates that specifically target pitch contour patterns. This allows independent adjustment of intonation characteristics without modifying the entire voice synthesis system, enabling precise control over intonation matching user intention while maintaining ease of operation.
Solution Approach 2:
The patent changes the parameter representation by using pitch contour patterns as explicit parameters in intonation templates. These templates define specific pitch trajectories (rising, falling, rising-falling patterns) that directly control the intonation of synthesized speech, transforming the abstract concept of intonation into controllable parameters that match user intention.
2Reliability
If neural network training is performed to adjust voice synthesis style, then intonation can be modified, but it requires a lot of data and time
Solution Approach 1:
The patent applies preliminary action by pre-defining intonation templates with standardized pitch contour patterns before actual voice synthesis. These templates are prepared in advance and can be directly selected or combined based on user intention, eliminating the need for time-consuming neural network training while maintaining reliable intonation control.
Solution Approach 2:
The patent uses copying by creating reusable intonation templates that can be applied across different speech contexts. Instead of training a new neural network for each intonation requirement, the system copies and combines predefined template patterns to achieve the desired intonation, significantly reducing data and time requirements.
3Adaptability or versatility
If overall voice style is modified, then voice synthesis can be adjusted, but it is difficult to adjust only the intonation specifically
Solution Approach 1:
The patent segments the voice synthesis control into independent components: overall voice style and specific intonation. By introducing separate intonation templates that only affect pitch contour, the system enables granular control where users can adjust intonation without changing other voice characteristics, improving both adaptability and ease of operation.
Solution Approach 2:
The patent applies local quality by allowing different parts of the voice synthesis process to have different control characteristics. The intonation templates specifically target the pitch dimension while leaving other voice qualities (timbre, volume, rhythm) unchanged, enabling precise local adjustment of intonation within the overall voice style framework.
Data Source
AI summary
An example electronic device and an example control method thereof are provided. The example electronic device may include: a memory configured to store information about a plurality of intonation templates, which are obtained by training an intonation template generator based on a plurality of voice signals and represent intonations of the plurality of voice signals; and a processor configured to, when a voice signal is received, obtain information about a user's intention corresponding to the voice signal, obtain first text to respond to the voice signal, based on the voice signal and the information about the user's intention, identify an intonation template corresponding to the first text based on the information about the plurality of intonation templates by inputting information about the first text to an intonation classifier, obtain intonation information corresponding to the identified intonation template by inputting information about the identified intonation template to the intonation encoder, and obtain an output voice signal corresponding to text by inputting the first text and the intonation information to a voice synthesis module.