Speech Synthesis Feedback Loop for Computational Load Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech synthesis methods, particularly those using artificial neural networks, require significant computational resources due to the need for processing large amounts of audio features for each frame, which can be inefficient and resource-intensive.
Innovation Solution
The proposed method employs a feedback loop mechanism that uses energy information from previous audio frames to generate audio features for subsequent frames, reducing the computational load by processing audio in subsets and adjusting the feedback information generation period based on text type and speech rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If speech synthesis processes each audio frame independently using full audio features, then speech quality and accuracy are improved, but computational load and resource consumption increase significantly
Solution Approach 1:
The patent segments the audio feature processing into two parts: (1) full audio features are processed only for the first frame of each text segment, and (2) for subsequent frames, only energy information (a simplified feature) is processed using feedback from previously decoded features. This segmentation allows high-precision processing where needed while reducing computational load in routine operations.
Solution Approach 2:
The patent changes the parameter set used for processing different frames. For the first frame of each segment, the full audio feature vector is used. For subsequent frames, the parameter set is reduced to only energy information, which is computationally cheaper to process. This dynamic parameter change maintains accuracy for critical frames while reducing overall computational burden.
2Manufacturing precision
If feedback information generation period is shortened to improve speech rhythm accuracy, then naturalness of speech is improved, but computational frequency and energy consumption increase
Solution Approach 1:
The patent implements periodic feedback information generation based on text segment boundaries rather than at every audio frame. The feedback loop is activated periodically at the start of each text segment (every N frames), where N is determined based on speech rate and text type. This periodic approach maintains natural speech rhythm at segment boundaries while avoiding excessive computational frequency between segments.
Solution Approach 2:
The patent makes the feedback information generation period dynamic by adjusting it based on speech rate and text type. For faster speech rates or certain text types, the period N is reduced to maintain rhythm accuracy. For slower speech or other text types, N is increased to reduce computational load. This dynamic adjustment optimizes the balance between rhythm accuracy and computational efficiency.
Data Source
Figure 1A~1B
Figure 2
Figure 3~4
AI summary
A speech synthesis method performed by an electronic apparatus to synthesize speech from text and includes: obtaining text input to the electronic apparatus; obtaining a text representation by encoding the text using a text encoder of the electronic apparatus; obtaining an audio representation of a first audio frame set from an audio encoder of the electronic apparatus, based on the text representation; obtaining an audio representation of a second audio frame set based on the text representation and the audio representation of the first audio frame set; obtaining an audio feature of the second audio frame set by decoding the audio representation of the second audio frame set; and synthesizing speech based on an audio feature of the first audio frame set and the audio feature of the second audio frame set.