Concurrent Phoneme Processing for Faster RNN Speech Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text-to-speech (TTS) systems using recurrent neural networks (RNNs) for autoregressive processing of phoneme audio features are inefficient due to iterative output processing, leading to slow audio synthesis speeds.
Innovation Solution
A concurrent processing method for adjacent phonemes using a target RNN that decouples the processing of audio features, allowing simultaneous calculation of speech data for adjacent phonemes without relying on previous frame outputs, reducing computational overhead and processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If autoregressive processing is used for phoneme audio features in TTS systems, then speech quality is maintained, but audio synthesis speed decreases
Solution Approach 1:
The patent segments the processing of adjacent phonemes into independent parallel tasks. Specifically, it divides the audio feature processing into separate encoders for different phoneme positions, allowing simultaneous computation of speech data for adjacent phonemes without sequential dependency, thus resolving the contradiction between maintaining quality through autoregressive processing and achieving speed through parallelization
Solution Approach 2:
The patent applies preliminary action by pre-computing and caching audio features for adjacent phonemes before they are needed for speech synthesis. The encoders process and store audio features for multiple phonemes in advance, enabling the decoder to retrieve and combine them without performing sequential autoregressive computations during the actual synthesis process, thereby improving speed while maintaining quality
2Measurement precision
If iterative output processing is used in RNN-based TTS, then accurate speech prediction is achieved, but processing time increases
Solution Approach 1:
The patent segments the iterative processing into independent encoder-decoder units that process different phonemes simultaneously. Each unit performs a simplified computation that does not require waiting for the previous iteration to complete, thereby reducing processing time while maintaining prediction accuracy through the collaborative output of multiple segmented units
Solution Approach 2:
The patent applies partial action by performing incomplete autoregressive steps in parallel for adjacent phonemes. Instead of completing the full iterative process sequentially for each phoneme, it performs partial computations that can be executed concurrently and combines the results, achieving acceptable prediction accuracy with significantly reduced processing time
Data Source
Figure 1
Figure 2~3a
Figure 3b
AI summary
A text data processing method is disclosed, and is applied to the field of artificial intelligence. The method includes: obtaining target text, where a phoneme of the target text includes a first phoneme and a second phoneme that are adjacent to each other (401); performing feature extraction on the first phoneme and the second phoneme, to obtain a first audio feature of the first phoneme and a second audio feature of the second phoneme (402); obtaining, by using a target recurrent neural network RNN and based on the first audio feature, first speech data corresponding to the first phoneme, and obtaining, by using the target RNN and based on the second audio feature, second speech data corresponding to the second phoneme, where a step of obtaining the first speech data corresponding to the first phoneme and a step of obtaining the second speech data corresponding to the second phoneme are concurrently performed (403); and obtaining, by using a vocoder and based on the first speech data and the second speech data, audio corresponding to the first phoneme and audio corresponding to the second phoneme (404). The target RNN can concurrently process the first audio feature and the second audio feature. This decouples a processing process of the first audio feature and a processing process of the second audio feature, and reduces duration for processing the audio features by the target RNN.