Vocoder Subframe Prediction for Faster Audio Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vocoders require multiple loops based on multiple sampling time points for speech prediction, leading to low speed and efficiency in audio synthesis processing.
Innovation Solution
The method involves dividing acoustic feature signals into multiple subframes in the frequency domain, performing time-domain down-sampling, and synchronously predicting sample values across adjacent sampling points using a sampling prediction network to reduce the number of loops required for audio synthesis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple loops based on multiple sampling time points are used for speech prediction, then prediction accuracy is improved, but processing speed and efficiency deteriorate
Solution Approach 1:
The acoustic feature signal is divided into multiple subframes in the frequency domain, and each subframe is down-sampled in the time domain. This segmentation reduces the number of sampling points that need to be processed in each loop, allowing multiple loops to maintain prediction accuracy while reducing the computational burden in each iteration, thereby improving overall processing speed.
Solution Approach 2:
The patent introduces frequency domain division as an additional dimension for processing. By dividing the signal into frequency subbands and applying time-domain down-sampling to each, the system processes fewer time points per loop while maintaining comprehensive coverage through frequency domain decomposition, thus resolving the contradiction between loop count and processing speed.
2Measurement precision
If multiple loops based on multiple sampling time points are used for speech prediction, then prediction accuracy is improved, but processing efficiency deteriorates
Solution Approach 1:
The acoustic feature signal is divided into multiple subframes in the frequency domain, and each subframe is down-sampled in the time domain. This segmentation reduces the number of sampling points that need to be processed in each loop, allowing multiple loops to maintain prediction accuracy while reducing the computational burden in each iteration, thereby improving overall processing speed.
Solution Approach 2:
The patent introduces frequency domain division as an additional dimension for processing. By dividing the signal into frequency subbands and applying time-domain down-sampling to each, the system processes fewer time points per loop while maintaining comprehensive coverage through frequency domain decomposition, thus resolving the contradiction between loop count and processing speed.
Data Source
AI summary
Embodiments of this application provide an audio processing method and apparatus, a vocoder, an electronic device, and a computer-readable storage medium. The audio processing method includes performing speech feature conversion on a text to obtain at least one acoustic feature frame; extracting a conditional feature corresponding to each acoustic feature frame from each acoustic feature frame of the at least one acoustic feature frame by a frame rate network; performing frequency division and time-domain down-sampling on the current frame of each acoustic feature frame to obtain n subframes corresponding to the current frame; synchronously predicting sample values corresponding to the current m adjacent sampling points on the n subframes to obtain m×n sub-prediction values; obtaining an audio prediction signal corresponding to the current frame; and performing audio synthesis on the audio prediction signal corresponding to each acoustic feature frame to obtain a target audio corresponding to the text.