Speech Synthesis Harmonic Modeling for Real-Time Vocoder Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vocoder technologies struggle to produce synthetic speech with good sound quality, high timbre restoration, and stable pronunciation while maintaining a small calculation amount, especially in applications requiring real-time processing.
Innovation Solution
A method and apparatus utilizing a speech synthesis model comprising a first generation sub-model and a second generation sub-model to process a spectrogram, where the first generation sub-model generates a harmonic structure and the second generation sub-model processes this structure and spectrogram to produce a target speech, employing convolutional networks and residual networks to optimize the synthesis process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional vocoder technologies are used, then computational requirements are low, but sound quality and timbre restoration are insufficient
Solution Approach 1:
The speech synthesis model is divided into two distinct sub-models: a first generation sub-model that extracts harmonic structure from spectrograms, and a second generation sub-model that synthesizes final speech. This segmentation allows each sub-model to specialize in specific tasks, improving overall sound quality while managing computational complexity through divided responsibilities.
Solution Approach 2:
The patent introduces an intermediate harmonic structure as a bridge between the input spectrogram and the final synthesized speech. This harmonic structure serves as a mediator that captures essential pitch and timbre information, enabling the second generation sub-model to produce higher quality speech without directly processing the full complexity of the original spectrogram.
2Manufacturing precision
If complex speech synthesis models are used, then timbre restoration improves, but real-time processing capability deteriorates
Solution Approach 1:
The first generation sub-model performs preliminary extraction of harmonic structure from the spectrogram before the main synthesis process. By pre-processing and isolating critical harmonic information, the system prepares optimized input for the second generation sub-model, enabling faster real-time processing while maintaining high timbre restoration quality.
3Manufacturing precision
If neural vocoder technologies are used, then sound quality improves, but computational resources increase
Solution Approach 1:
The patent extracts and isolates the harmonic structure component from the full spectrogram processing pipeline. By taking out only the essential harmonic information needed for high-quality speech synthesis and processing this extracted feature separately, the system achieves neural vocoder quality with reduced computational resource requirements.
Data Source
AI summary
A method for processing a speech includes: acquiring an original speech; extracting a spectrogram from the original speech; acquiring a speech synthesis model, where the speech synthesis model comprises a first generation sub-model and a second generation sub-model; generating a harmonic structure of the spectrogram, by invoking the first generation sub-model to process the spectrogram; and generating a target speech, by invoking the second generation sub-model to process the harmonic structure and the spectrogram.