Character-Level Speech Synthesis With Prosody and Duration Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech synthesis technologies require extensive preprocessing of words, phonemes, and pause information, leading to high computational demands and difficulty in processing, especially on devices with limited computing power.
Innovation Solution
A character-level speech synthesis method that generates hidden layer and prosodic features to predict pronunciation duration, allowing direct synthesis without extensive preprocessing, using an acoustic model to generate acoustic features based on these features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If phoneme-based speech synthesis is used with extensive preprocessing of words, phonemes, and pause information, then speech synthesis quality is improved, but device complexity and processing difficulty increase significantly
Solution Approach 1:
The patent segments the speech synthesis process into distinct functional modules: text preprocessing module, acoustic feature extraction module, prosodic feature extraction module, and speech synthesis module. Each module handles specific tasks independently, reducing overall processing complexity while maintaining synthesis quality.
Solution Approach 2:
The patent extracts essential features (acoustic features and prosodic features) from the input text, separating them from the raw text data. This extraction approach reduces the complexity of subsequent synthesis operations by working with condensed feature representations rather than raw text.
2Measurement precision
If extensive preprocessing of words, phonemes, and pause information is performed, then speech synthesis accuracy is improved, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary extraction of acoustic and prosodic features from the text before the actual synthesis process. These pre-extracted features are stored and reused during synthesis, avoiding redundant processing and reducing overall processing time while maintaining accuracy.
Solution Approach 2:
The patent creates feature representations (acoustic features and prosodic features) that copy the essential characteristics of the original text in a compressed form. These feature copies are then used for synthesis operations, reducing the computational burden compared to processing the full original text repeatedly.
3Productivity
If character-level speech synthesis is implemented without extensive preprocessing, then processing efficiency is improved, but speech synthesis quality may be compromised
Solution Approach 1:
The patent changes the representation parameters of the text from raw character sequences to extracted acoustic and prosodic feature vectors. This parameter transformation enables efficient character-level processing while preserving the quality information needed for high-fidelity speech synthesis.
Solution Approach 2:
The patent applies different processing strategies to different parts of the text: acoustic features capture local phonetic characteristics while prosodic features capture local rhythmic and intonation patterns. This localized feature extraction maintains synthesis quality without requiring extensive global preprocessing.
Data Source
AI summary
Embodiments of this application provide a speech synthesis method performed by an electronic device. The method includes: acquiring a target text to be synthesized into a speech; generating hidden layer features and prosodic features of the target text, and predicting pronunciation duration of characters in the target text using an acoustic model corresponding to the target text; generating acoustic features corresponding to the target text based on the hidden layer features, the prosodic features and the pronunciation duration; and synthesizing a target speech corresponding to the target text according to the acoustic features. Using the solution provided by the embodiments of this application is beneficial to reducing the difficulty of speech synthesis.