Voice Cloning Speech Synthesis From Short Multilingual Recordings

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing Text-To-Speech systems are limited by the number of voices they can support, require studio recordings for training, lack multilingual capabilities, and struggle to reproduce a wide emotional/expressive range due to insufficient audio recordings and quality, leading to reduced scalability and audio quality.

Innovation Solution

A method and system that utilize deep learning algorithms to create optimized acoustic and linguistic features, allowing for voice cloning from short audio recordings, enabling multilingual capabilities and enhanced expressiveness, while being efficient and cost-effective.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If studio recordings are used for training, then audio quality is improved, but cost and time consumption increase

Engineering Contradiction:

Improveaudio qualityVSAvoidtraining time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The patent uses short audio recordings (a few seconds to minutes) instead of requiring extensive studio recordings. These brief recordings are sufficient to extract voice features and train the model, replacing the need for hours of high-quality studio audio data.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Solution Approach 2:

The system performs self-supervised learning by automatically extracting acoustic features and voice characteristics from the provided audio recordings without requiring manual annotation or extensive preprocessing. The model learns directly from the raw audio data through self-attention mechanisms and position-wise feed-forward networks.

Inventive Principle:

Principle #25Self-service

2Adaptability or versatility

If more voices are supported, then versatility is improved, but system complexity increases

Engineering Contradiction:

Improvenumber of supported voicesVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements a universal voice synthesis system that can handle multiple voices and languages through a single model architecture. The system uses language-agnostic acoustic features and position-wise encodings that work across different languages and voice characteristics, allowing one system to perform multiple voice cloning tasks.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system controls different voices by adjusting parameters in the continuous voice space rather than requiring separate models for each voice. By modifying voice embedding vectors and using position-wise feed-forward networks, the system can dynamically switch between different voice characteristics while maintaining the same underlying model structure.

Inventive Principle:

Principle #35Parameter changes

3Adaptability or versatility

If more audio recordings are collected, then emotional range is improved, but data processing complexity increases

Engineering Contradiction:

Improveemotional rangeVSAvoiddata processing complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent extracts only the essential emotional and acoustic features from audio recordings rather than processing the entire audio dataset. The system uses self-attention mechanisms to identify and extract key prosodic features, pitch contours, and emotional characteristics, discarding redundant information while preserving expressive content.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system performs preliminary feature extraction and acoustic modeling before the main synthesis process. By pre-processing audio data to extract acoustic features, pitch, and emotional characteristics in advance, the system reduces the complexity of the main generation task and enables more efficient processing during inference.

Inventive Principle:

Principle #10Preliminary action

4Ease of operation

If existing Text-To-Speech systems are used, then basic speech synthesis is achieved, but expressiveness and naturalness are limited

Engineering Contradiction:

Improvebasic speech synthesis capabilityVSAvoidexpressiveness and naturalness

Core Design Contradiction:

Ease of operationVSManufacturing precision

Solution Approach 1:

The patent replaces traditional rule-based and statistical speech synthesis methods with deep learning-based neural networks. The system uses self-attention mechanisms, position-wise feed-forward networks, and continuous voice spaces to model speech production, replacing mechanical and statistical approaches with learned representations that capture natural speech patterns and emotional expression.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS20260100183A1Method and system for producing synthesized speech digital audio content

Publication Date: 2026.04.09 VOISEED SRL

AI summary

A method for producing synthesized speech digital audio content, wherein:a feature extractor module receives an audio recording of a speaker's voice, extracts a plurality of acoustic features and converts them to an audio latent representation matrix;a phonemizing module receives as input a target text and converts the target text to a sequence of phonemes;a tokenizing module receives as input the sequence of phonemes of the target text;a linguistic encoder module receives as input the sequence of phoneme vectors and converts the sequence of phoneme vectors to a sequence of respective linguistic latent vectors;an acoustic model module produces a predicted audio latent representation matrix; anda vocoder module decodes the predicted audio latent representation matrix into the corresponding audio signal of the speech of the synthesized virtual voice.