Text-to-Speech Emotional Content via Neutral Model Adjustments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current text-to-speech technologies fail to efficiently generate speech with emotional content, often resulting in neutral renditions or requiring separate voice models for each emotion type, leading to high storage costs and limited emotional expression.
Innovation Solution
The approach involves applying emotion-specific adjustments on a per-phoneme, per-state, or per-frame basis, using a decision tree clustering scheme to adapt neutral voice models, allowing for efficient generation of speech with specific emotional characteristics without the need for separate models for each emotion type.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If separate voice models are used for each emotion type, then emotional expression capability is improved, but memory storage cost increases
Solution Approach 1:
The patent segments the emotional content generation into two independent components: a shared neutral voice model and separate emotion-specific adjustment parameters. The voice model processes the core speech content while emotion parameters (stored in decision trees) apply targeted modifications for rhythm, prosody, and temporal characteristics. This segmentation allows the system to support multiple emotions without storing complete separate models for each emotion type, thereby reducing memory storage requirements while maintaining emotional expression capability.
Solution Approach 2:
The neutral voice model serves as a universal foundation that can generate speech for any emotion type when combined with appropriate emotion-specific adjustment parameters. The decision tree structures store compact emotion representations that can be applied to the same voice model to produce different emotional expressions. This multi-functionality approach allows one voice model to handle multiple emotion types, eliminating the need for separate dedicated models for each emotion and thus reducing overall storage costs.
2Manufacturing precision
If emotion-specific adjustments are applied on a per-phoneme, per-state, or per-frame basis, then emotional expression precision is improved, but computational complexity increases
Solution Approach 1:
The patent applies emotion-specific adjustments at different granularities (per-phoneme, per-state, or per-frame basis) depending on the specific emotional parameter being modified. This local quality approach allows the system to apply computational resources selectively - using fine-grained per-frame adjustments for critical emotional characteristics like prosody while using coarser per-phoneme adjustments for less critical parameters. The decision tree structures enable this selective application by organizing adjustment parameters at appropriate levels of granularity, thereby achieving high emotional expression precision without uniformly increasing computational complexity across all parameters.
3Quantity of substance
If a single neutral voice model is used for all emotions, then memory efficiency is improved, but emotional versatility deteriorates
Solution Approach 1:
The patent introduces emotion-specific adjustment parameters stored in decision tree structures as intermediary elements between the neutral voice model and the final emotional speech output. These decision trees act as mediators that translate the emotion type into specific parameter adjustments (rhythm, prosody, temporal characteristics) that are then applied to the neutral model's output. This intermediary mechanism enables a single neutral voice model to achieve multiple emotional expressions by combining it with different decision tree configurations, thereby maintaining memory efficiency while achieving emotional versatility.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques for converting text to speech having emotional content. In an aspect, an emotionally neutral acoustic trajectory is predicted for a script using a neutral model, and an emotion-specific acoustic trajectory adjustment is independently predicted using an emotion-specific model. The neutral trajectory and emotion-specific adjustments are combined to generate a transformed speech output having emotional content. In another aspect, state parameters of a statistical parametric model for neutral voice are transformed by emotion-specific factors that vary across contexts and states. The emotion-dependent adjustment factors may be clustered and stored using an emotion-specific decision tree or other clustering scheme distinct from a decision tree used for the neutral voice model.