Speech Unit Clustering for Text-to-Speech Corpus Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Text-to-speech systems face challenges in efficiently concatenating speech sounds to minimize target and join costs, leading to discontinuity in synthesized audio, especially when resources are limited, such as in devices with restricted memory.
Innovation Solution
A method that determines concatenation features of speech sounds and clusters them based on these features to select a representative sound for concatenation, reducing the size of the speech corpus while maintaining sparsity and optimizing prosody.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large corpus of speech sounds is stored to improve synthesis quality, then the quality of synthesized audio is improved, but the memory resource consumption increases
Solution Approach 1:
The speech corpus is segmented into multiple clusters based on concatenation features (such as phonetic context, pitch, and duration characteristics). Each cluster represents a group of speech sounds with similar concatenation properties, allowing the system to organize and manage speech units in a structured manner that reduces memory requirements while maintaining synthesis quality.
Solution Approach 2:
The system changes the parameter representation by selecting speech sounds based on concatenation features rather than storing all possible speech variants. By identifying and utilizing key concatenation parameters (acoustic transition characteristics), the system can select representative speech sounds from each cluster, reducing the overall corpus size while preserving the ability to generate high-quality synthesized speech through feature-based matching.
2Quantity of substance
If speech sounds are selected based on concatenation features to reduce corpus size, then memory usage is reduced, but the complexity of the selection process increases
Solution Approach 1:
The system performs preliminary clustering of speech sounds based on concatenation features during the corpus preparation phase. This pre-processing step organizes speech units into clusters before actual synthesis occurs, so that during runtime, the selection process only needs to identify the appropriate cluster and select a representative sound, significantly reducing the complexity of real-time selection while maintaining corpus size reduction benefits.
3Reliability
If representative speech sounds are selected from clusters to minimize join cost, then discontinuity in synthesized audio is reduced, but the time required for speech sound selection increases
Solution Approach 1:
The system creates and stores representative speech sounds for each cluster that exemplify the concatenation features of that cluster. During synthesis, instead of analyzing multiple speech sounds to find the best match, the system can directly select the pre-identified representative sound from the appropriate cluster, dramatically reducing selection time while maintaining audio continuity through the preserved concatenation characteristics of the representative samples.
Data Source
AI summary
A device may receive a plurality of speech sounds that are indicative of pronunciations of a first linguistic term. The device may determine concatenation features of the plurality of speech sounds. The concatenation features may be indicative of an acoustic transition between a first speech sound and a second speech sound when the first speech sound and the second speech sound are concatenated. The first speech sound may be included in the plurality of speech sounds and the second speech sound may be indicative of a pronunciation of a second linguistic term. The device may cluster the plurality of speech sounds into one or more clusters based on the concatenation features. The device may provide a representative speech sound of the given cluster as the first speech sound when the first speech sound and the second speech sound are concatenated.