Emotional Classifier Model for Text-to-Speech Synthesis

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional text-to-speech (TTS) computing systems require manually labeled data for model training and emotional labels at runtime, which is cumbersome and inefficient. Additionally, they lack the ability to specify emotion levels and domain-specific prosody features.

Innovation Solution

The proposed computing system synthesizes expressive speech from text without requiring emotional labels during training or at runtime. It enables users to specify emotion levels and generates domain-specific speech by using an emotional classifier model and a TTS model that processes textual and phoneme embeddings to produce spectrograms reflecting the intended emotions and domains.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional TTS systems use manually labeled data for model training, then the model can generate expressive speech with matched emotions, but the training process becomes cumbersome and time-consuming

Engineering Contradiction:

Improveemotion accuracyVSAvoidtraining time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs self-labeling by using an emotional classifier model to automatically extract emotional information from text content without requiring manual annotation. The classifier analyzes text features and generates emotional labels autonomously, eliminating the need for human annotators while maintaining reliable emotion recognition for speech synthesis

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The emotional classifier model is pre-trained on a large corpus of text data to learn emotional patterns and expressions. This preliminary training enables the system to automatically classify emotions in new text inputs during inference, avoiding the need for manual labeling during both training and runtime operations

Inventive Principle:

Principle #10Preliminary action

2Reliability

If conventional TTS systems require emotional labels at runtime, then expressive speech can be generated, but the user experience becomes cumbersome

Engineering Contradiction:

Improveemotion expressionVSAvoiduser input requirement

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The system automatically determines emotional labels by analyzing the text content itself using the pre-trained emotional classifier. This self-service approach extracts emotional information directly from the text without requiring users to manually specify emotions, making the system easy to operate while maintaining reliable emotion expression in the synthesized speech

Inventive Principle:

Principle #25Self-service

3Adaptability or versatility

If conventional TTS systems collect audio data from different speakers and locales, then the model can handle diverse speech patterns, but data collection and storage becomes burdensome

Engineering Contradiction:

Improvespeaker diversityVSAvoiddata storage

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The system extracts emotional and prosodic information directly from text features using the emotional classifier model, rather than storing and processing large quantities of audio data from multiple speakers. This extraction approach captures essential speech characteristics without the storage burden of maintaining extensive audio datasets

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent replaces the mechanical approach of collecting and storing physical audio data with a computational approach that extracts emotional features from text representations. This substitution eliminates the need for physical data storage infrastructure while maintaining the ability to handle diverse speech patterns through learned text features

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Ease of manufacture

If conventional TTS systems provide fixed emotion categories, then simple emotion synthesis is achieved, but nuanced emotion levels cannot be expressed

Engineering Contradiction:

Improvesystem simplicityVSAvoidemotion level control

Core Design Contradiction:

Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The system dynamically adjusts emotion intensity levels by scaling the output of the emotional classifier model. Instead of fixed emotion categories, the classifier produces continuous or graded emotional representations that can be modulated to express nuanced levels of emotion, allowing the synthesized speech to convey subtle emotional variations while maintaining system simplicity

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS12293756B2Computing system for domain expressive text to speech

Publication Date: 2025.05.06 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

A computing system obtains text that includes words and provides the text as input to an emotional classifier model that has been trained based upon emotional classification. The computing system obtains a textual embedding of the computer-readable text as output of the emotional classifier model. The computing system generates a phoneme sequence based upon the words of the text. The computing system, generates, by way of an encoder of a text to speech (TTS) model, a phoneme encoding based upon the phoneme sequence. The computing system provides the textual embedding and the phoneme encoding as input to a decoder of the TTS model. The computing system causes speech that includes the words to be played over a speaker based upon output of the decoder of the TTS model, where the speech reflects an emotion underlying the text due to the textual embedding provided to the encoder.