Flow-Matched Text-to-Speech for Emotional Voice Cloning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional text-to-speech synthesis systems are slow, require extensive hardware resources, lack fine-grained emotional control, and struggle with voice cloning without extensive re-training, and often lack conversational context.

Innovation Solution

A system utilizing a speaker encoder, dialog context encoder, and speech generator, trained with rectified flow matching, enables fast generation of natural-sounding speech with emotional content and voice cloning capabilities without retraining, using precomputed speaker embeddings and cached information.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If conventional autoregressive TTS systems are used, then speech quality and expressiveness are improved, but generation speed deteriorates

Engineering Contradiction:

Improvespeech qualityVSAvoidgeneration speed

Core Design Contradiction:

Manufacturing precisionVSSpeed

Solution Approach 1:

The system segments the speech generation process into discrete flow matching steps that can be processed independently and in parallel, breaking down the sequential autoregressive generation into concurrent computational units that maintain quality while accelerating output

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent replaces the traditional autoregressive mechanical generation mechanism with a flow matching approach that uses continuous probability flow transformation, substituting the sequential token-by-token generation with a parallel continuous transformation process

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If separate models are created for each speaker, then voice cloning accuracy is improved, but memory requirements and device complexity deteriorate

Engineering Contradiction:

Improvevoice cloning accuracyVSAvoidmodel storage requirements

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system creates a universal base model that can generate speech for multiple speakers, making the model multi-functional by incorporating speaker-specific characteristics through conditional inputs rather than requiring separate dedicated models for each speaker

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent changes the approach from creating separate models to adjusting parameters within a single model, using speaker embeddings and style tokens as configurable parameters that allow one model to adapt to multiple speakers without retraining

Inventive Principle:

Principle #35Parameter changes

3Speed

If flow matching systems are used for high speed generation, then generation speed is improved, but conversational context capability deteriorates

Engineering Contradiction:

Improvegeneration speedVSAvoidconversational context

Core Design Contradiction:

SpeedVSLoss of information

Solution Approach 1:

The system merges flow matching with transformer-based contextual encoding, combining the speed advantages of flow matching with the contextual understanding capabilities of transformers to maintain both high speed and conversational awareness

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS12462787B1Text-to-speech synthesis using generative artificial intelligence models

Publication Date: 2025.11.04 PHEON INC

AI summary

A method and a system for generating human speech audio in a conversation using a trained generative AI model are provided. The method includes receiving a text input representing a portion of the conversation, receiving dialog context associated with the conversation, receiving information representing at least one voice and speaking style of at least one speaker in the conversation, generating the at least one voice and speaking style based on the received information, and generating at least one emotional audio response for the at least one speaker using the at least one voice and speaking style and without retraining the trained generative AI model.