Prosody Prediction in Speech Synthesis via Text Fragment Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech synthesis technologies face challenges in predicting prosody effectively, with rule-based methods requiring complex rules and example-based methods needing exact audio matches, making them impractical for producing natural-sounding speech across various domains.
Innovation Solution
A method that compares input text to a dataset of text fragments with aligned spoken audio, extracts prosody from matching fragments, and applies it to synthesize speech, allowing for natural prosody prediction without requiring exact matches or extensive rule programming, using domain-specific data sets and chunking techniques to match structural and semantic features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If rule-based methods are used for prosody prediction, then prosody can be predicted without audio examples, but the system requires complex rules and extensive programming
Solution Approach 1:
The patent copies prosodic patterns from pre-recorded audio examples instead of using complex rules. The system stores multiple audio examples with their corresponding text and prosodic features, then retrieves and applies appropriate prosody from these copied examples during synthesis, eliminating the need for extensive rule programming
Solution Approach 2:
The patent performs preliminary action by pre-recording and storing audio examples with their prosodic characteristics before actual speech synthesis. This advance preparation creates a reusable database of prosodic patterns that can be directly applied without real-time rule processing
2Reliability
If example-based methods are used for prosody prediction, then natural prosody can be obtained, but exact audio matches are required which limits applicability
Solution Approach 1:
The patent segments text into chunks and matches them against stored examples at the chunk level rather than requiring exact full-text matches. This segmentation allows the system to find partial matches and apply appropriate prosody from stored examples even when the input text differs from stored examples, thereby improving domain applicability while maintaining natural prosody
Solution Approach 2:
The patent changes the matching parameter from exact text equality to similarity-based matching with flexibility for variations. The system allows mismatches in certain parameters (such as specific words or phrases) while still finding appropriate prosodic patterns from stored examples, enabling broader domain applicability
3Measurement precision
If domain-specific data sets are used, then prosody accuracy improves for that domain, but the system requires separate data sets for different domains
Solution Approach 1:
The patent creates a universal prosody prediction system that can handle multiple domains through a single integrated architecture. The system uses a common framework for storing, retrieving, and applying prosodic patterns that works across different domains (news, weather, sports, etc.), eliminating the need for separate processing systems for each domain while maintaining domain-specific accuracy through domain-specific training data
Data Source
AI summary
Techniques for predicting prosody in speech synthesis may make use of a data set of example text fragments with corresponding aligned spoken audio. To predict prosody for synthesizing an input text, the input text may be compared with the data set of example text fragments to select a best matching sequence of one or more example text fragments, each example text fragment in the sequence being paired with a portion of the input text. The selected example text fragment sequence may be aligned with the input text, e.g., at the word level, such that prosody may be extracted from the audio aligned with the example text fragments, and the extracted prosody may be applied to the synthesis of the input text using the alignment between the input text and the example text fragments.