Generative Model Speech Coding with Predictive Variance Regularization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio codecs struggle to compress speech effectively for minimal bandwidth usage, and machine-learning based generative models face challenges in handling diverse speech from various speakers and environments due to sensitivity to data quality and conditioning sequences.
Innovation Solution
The implementation of predictive-variance regularization to reduce the sensitivity of maximum likelihood criteria to outliers, combined with noise reduction techniques during model training, enhances the performance of generative models as audio codecs, achieving better representation of original speech at low bit rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If machine-learning based generative models are used as audio codecs, then speech compression is improved resulting in reduced bandwidth usage, but the models become sensitive to data quality and conditioning sequences making them difficult to train for diverse speech
Solution Approach 1:
The patent applies preliminary noise reduction to the training data before training the generative model. This preprocessing step removes unwanted signals and improves data quality, enabling the model to be trained effectively on diverse speech data while maintaining consistent performance across different speakers and environments.
Solution Approach 2:
The patent modifies the training objective function by incorporating predictive variance regularization. This changes the optimization parameters to penalize high variance in predictions, making the model more robust to variations in input data quality and conditioning sequences while maintaining good compression performance.
2Reliability
If predictive-variance regularization is applied to reduce sensitivity to outliers, then model robustness is improved, but training complexity increases
Solution Approach 1:
The patent incorporates predictive variance regularization into the existing training objective function. This modifies the loss function parameters to include a variance penalty term, which guides the model to produce more consistent predictions without requiring fundamental changes to the training architecture or process.
3Manufacturing precision
If noise reduction techniques are used during model training, then speech synthesis quality is improved, but processing time and computational resources increase
Solution Approach 1:
The patent applies noise reduction as a preliminary preprocessing step to the training data before the main model training begins. By removing unwanted signals in advance, the model trains more efficiently on cleaner data, achieving better synthesis quality without excessive computational overhead during the main training process.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A method includes receiving sampled audio data corresponding to utterances and training a machine learning (ML) model, using the sampled audio data, to generate a high-fidelity audio stream from a low bitrate input bitstream. The training of the ML model includes de-emphasizing the influence of low-probability distortion events in the sampled audio data on the trained ML model, where the de-emphasizing of the distortion events is achieved by the inclusion of a term in an objective function of the ML model, which term encourages low-variance predictive distributions of a next sample in the sampled audio data, based on previous samples of the audio data.