Speech Synthesis Dataset Fusion for Multi-Speaker Emotion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech synthesis technologies face challenges in generating high-quality multi-speaker, multi-language, and multi-emotion speech due to the difficulty in acquiring and labeling datasets that include diverse voices, languages, and emotional expressions.
Innovation Solution
A method for constructing a training dataset for speech synthesis models by fusing language, speaker, and emotion information within a single utterance, allowing for the generation of a high-capacity training dataset that can synthesize various speech scenarios with minimal additional data and cost.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If speech data from multiple speakers, languages, and emotions are collected separately, then the diversity of speech synthesis is improved, but the cost and complexity of data collection increase significantly
Solution Approach 1:
The patent merges multiple speech data dimensions (speaker identity, language, emotion) into a single integrated dataset structure. By fusing these different types of information within one utterance record, the system achieves multi-speaker, multi-language, and multi-emotion synthesis capabilities without requiring separate data collection processes for each dimension, thus reducing overall complexity while maintaining versatility.
Solution Approach 2:
The patent creates a universal speech dataset that serves multiple functions simultaneously. The same dataset enables the synthesis model to handle different speakers, languages, and emotions through a unified structure with embedded metadata labels, eliminating the need for separate specialized datasets for each synthesis type and reducing data collection complexity.
2Manufacturing precision
If voice actors are used to collect speech data for multiple languages and emotions, then the quality of speech synthesis is improved, but the cost increases significantly
Solution Approach 1:
The patent uses automatic speech recognition and text-to-speech synthesis to generate synthetic speech data that copies and transforms existing speech patterns. Instead of hiring voice actors to produce all variations, the system automatically generates diverse speech samples by manipulating existing data with learned speaker characteristics, language patterns, and emotional expressions, significantly reducing costs while maintaining quality.
Solution Approach 2:
The patent applies parameter changes to existing speech data to create diverse variations. By modifying acoustic parameters, pitch, timbre, and temporal characteristics through signal processing and deep learning transformations, the system generates speech that mimics different speakers, languages, and emotions from a limited set of original recordings, reducing the need for extensive expensive data collection.
3Reliability
If comprehensive label information for speakers, emotions, and languages is provided for each speech data, then the training effectiveness is improved, but the data processing complexity increases
Solution Approach 1:
The patent segments label information into distinct, organized categories (speaker identity, language type, emotion state) that can be independently processed and applied. This segmentation allows the training system to efficiently handle each label type separately while maintaining their relationships within the unified dataset structure, improving training effectiveness without overwhelming processing complexity.
Solution Approach 2:
The patent introduces an intermediary processing layer that automatically generates and validates label information using speech analysis algorithms and metadata extraction. This intermediary system bridges the gap between raw speech data and the comprehensive labeling required for effective training, automatically creating speaker, language, and emotion labels without manual annotation, thus reducing processing complexity while maintaining reliability.
Data Source
AI summary
There is provided a training dataset construction method for speech synthesis through fusion of language, speaker, emotion within an utterance. A training dataset construction method of a speech synthesis model according to an embodiment collects speech data having different speech utterance information, increases the speech data by fusing the collected speech data within one utterance, and generates a training dataset by using the increased speech data. Accordingly, a training dataset for speech synthesis is constructed through fusion of language, speaker, emotion within one utterance, so that quality of speech synthesis of multi-speaker/multi-language/emotion can be enhanced.