Text-Based Speech Synthesis Using Discrete Character Feature Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech synthesis methods, whether rule-based or statistical machine learning-based, face challenges in efficiently handling multi-tone characters due to the need for extensive manual rule-setting and phonetic annotations, which increases workload and is limited by uneven sample distributions.
Innovation Solution
A text-based speech synthesis method that discretely characterizes each character in a target text using One-Hot coding to generate feature vectors, which are then input into a pre-trained spectrum conversion model to produce Mel-spectrums, subsequently converted into speech, reducing the need for pinyin markings and alleviating pronunciation issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If rule-based methods are used for speech synthesis, then pronunciation accuracy can be improved, but the device complexity and workload increase due to extensive manual rule-setting
Solution Approach 1:
The patent replaces the mechanical rule-based system with a deep learning neural network model (Multi-Perceptual Attention Model). Instead of manually setting phonetic rules and pinyin markings, the system uses automated neural networks to predict pronunciation directly from text, eliminating the need for complex manual rule configuration while maintaining or improving pronunciation accuracy.
Solution Approach 2:
The system enables self-service by allowing the neural network to automatically learn and adapt pronunciation patterns from training data without requiring manual annotation of pinyin or phonetic rules. The model performs self-training and self-optimization through automated loss calculation and parameter updates, reducing human intervention to minimum.
2Extent of automation
If statistical machine learning methods are used for speech synthesis, then automation can be improved, but the reliability deteriorates due to uneven distribution of samples
Solution Approach 1:
The patent segments the pronunciation prediction task into multiple independent prediction heads within the neural network, each handling different aspects of pronunciation (e.g., tone, stress, phoneme selection). This segmentation allows the model to specialize in different pronunciation dimensions and handle uneven sample distributions more effectively by distributing the learning burden across multiple specialized components.
Solution Approach 2:
The system performs preliminary action by pre-training the neural network on large-scale text corpora to learn general pronunciation patterns before fine-tuning on specific datasets. This pre-training phase allows the model to acquire robust pronunciation knowledge that can generalize to unseen data, improving reliability even when training samples are unevenly distributed.
3Measurement precision
If phonetic annotations are added to training text, then pronunciation accuracy can be improved, but the loss of time increases due to increased workload
Solution Approach 1:
The patent replaces the manual phonetic annotation process with automated neural network prediction. Instead of requiring human experts to manually add pinyin and phonetic markings to training text, the system uses trained neural networks to automatically generate these annotations, reducing the time and labor required from hours or days to seconds or minutes.
Solution Approach 2:
The system merges the text processing and phonetic annotation tasks into a single integrated neural network pipeline. The same model that processes the input text also generates the phonetic predictions, eliminating the need for separate annotation steps and reducing overall processing time while maintaining accuracy.
Data Source
AI summary
A text-based speech synthesis method, a computer device, and a non-transitory computer-readable storage medium are provided. The text-based speech synthesis method includes: a target text to be recognized is obtained; each character in the target text is discretely characterized to generate a feature vector corresponding to each character; the feature vector is input into a pre-trained spectrum conversion model, to obtain a Mel-spectrum corresponding to each character in the target text output by the spectrum conversion model; and the Mel-spectrum is converted to speech to obtain speech corresponding to the target text.