Speech Synthesis Device Using Segmented Neural Network Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional DNN speech synthesis devices based on an encoder/decoder structure with an attention mechanism require prolonged processing times, making it difficult to perform detailed editing of metrical features like phoneme duration and voice pitch until the entire input is processed.
Innovation Solution
A speech synthesis device that outputs intermediate expressions and metrical feature quantities in advance, allowing for successive generation and editing of speech waveforms, using an analyzing unit, a first processing unit with an encoder and metrical feature quantity decoder, and a second processing unit with a speech waveform decoder to improve response time and enable editing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If an encoder/decoder structure with attention mechanism is used for DNN speech synthesis, then speech quality is improved, but processing time is prolonged
Solution Approach 1:
The speech synthesis process is segmented into two independent stages: (1) encoding the entire input text to generate intermediate expressions and metrical feature quantities, and (2) successively decoding speech waveframes. This segmentation allows the time-consuming encoding stage to be performed once for the entire input, while the decoding stage can proceed in parallel for multiple output frames, thereby reducing overall processing time while maintaining speech quality.
Solution Approach 2:
The encoder performs preliminary processing of the entire input text sequence to generate intermediate expressions and metrical feature quantities before the decoding stage begins. This preliminary action allows subsequent decoding operations to proceed more efficiently without repeatedly processing the entire input sequence, thus reducing total processing time while preserving the attention mechanism's ability to maintain speech quality.
2Reliability
If the entire input text is processed before outputting speech, then complete speech synthesis is achieved, but editing of metrical features becomes difficult
Solution Approach 1:
The synthesis process is divided into an encoding stage that processes the entire input to generate intermediate expressions and metrical feature quantities, and a decoding stage that successively generates speech waveframes. This segmentation allows users to access and edit the metrical feature quantities after encoding but before or during decoding, enabling easy modification of pitch, duration, and other metrical features without reprocessing the entire input text.
Solution Approach 2:
The encoder performs preliminary processing to generate intermediate expressions and metrical feature quantities that are stored and made available for editing before the decoding stage completes. This preliminary generation of structured features allows users to intervene and modify metrical properties without compromising the completeness of the final speech synthesis output.
3Quantity of substance
If sequential processing of input text is used, then memory usage is reduced, but response time increases
Solution Approach 1:
The processing is segmented such that the encoder processes the entire input text sequentially to generate intermediate expressions and metrical feature quantities with moderate memory usage, while the decoder then successively generates speech waveframes in parallel operations. This segmentation allows the system to manage memory efficiently during encoding while achieving faster overall response time through parallel decoding operations.
Solution Approach 2:
The encoder performs preliminary processing of the input text to create compact intermediate representations and metrical feature quantities. This preliminary action reduces the memory burden for subsequent decoding operations while enabling faster generation of speech waveframes, thus improving response time without excessive memory consumption.
Data Source
AI summary
The response time till generation of waveforms is improved, and detailed editing for a metrical-feature quantity based on the entire input is performable before generation of waveforms. A speech synthesis device includes an analyzing unit, a first-processing unit, and a second-processing unit. The analyzing unit analyzes an input text and generates a language feature quantity sequence including one or more vectors indicating a language feature quantity. The first-processing unit includes an encoder that converts the language feature quantity sequence into an intermediate expression sequence including one or more vectors indicating a latent variable, using a first neural network; and includes a metrical-feature quantity decoder that generates a metrical-feature quantity from the intermediate expression sequence using a second neural network. The second-processing unit includes a speech waveform decoder that successively generates speech waveforms from the intermediate expression sequence and the metrical-feature quantity using a third neural network.