Deep Generative Speech Enhancement for Coded Audio Distortion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech coding technologies suffer from quality issues due to decreasing bitrates, leading to distorted and contaminated speech signals that are difficult to restore effectively.
Innovation Solution
A system utilizing self-supervised deep learning models to generate feature vectors from contaminated, coded speech, followed by a generative deep learning model to produce high-quality clean speech signals, leveraging robust features from low-level audio properties to high-level speaker characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If speech coding bitrate is decreased to enable low-cost mobile and internet communication, then communication cost is reduced, but speech quality deteriorates with various quality issues
Solution Approach 1:
The patent introduces a denoising autoencoder with linear regression decoder as an intermediary component between the coded speech and the final output. This intermediary model processes the noisy coded speech through learned representations to reconstruct clean speech, effectively mediating the quality degradation caused by low bitrate coding while maintaining communication cost efficiency.
Solution Approach 2:
The patent changes the parameters of the speech signal by transforming it through a deep learning model that learns optimal representations. The model adjusts internal parameters (weights and biases) during training to map coded speech to clean speech, effectively changing the signal characteristics to improve quality without increasing bitrate.
2Device complexity
If conventional audio processing techniques are used to restore clean speech from coded audio data, then processing simplicity is maintained, but speech restoration quality is insufficient
Solution Approach 1:
The patent replaces conventional mechanical audio processing techniques with a data-driven deep learning approach. Instead of using traditional signal processing algorithms, the system employs a trained denoising autoencoder model that automatically learns the transformation from coded to clean speech, achieving superior restoration quality.
Solution Approach 2:
The patent performs preliminary training of the denoising autoencoder model using training data before actual speech restoration. This preliminary action prepares the model with learned representations and parameters, enabling it to effectively restore speech quality when processing actual coded audio data in application scenarios.
3Manufacturing precision
If deep learning models are used to generate improved speech data, then speech quality is enhanced, but computational complexity increases
Solution Approach 1:
The patent segments the deep learning model into distinct functional components: an encoder that extracts features from coded speech, a denoising autoencoder that processes hidden representations, and a linear regression decoder that reconstructs clean speech. This segmentation allows for optimized computation at each stage and enables real-time processing through efficient inference.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A system for generating enhanced speech data using robust audio features is disclosed. In some embodiments, a system is programmed to use a self-supervised deep learning model to generate a set of feature vectors from given audio data that contains contaminated speech and is coded. The system is further programmed to use a generative deep learning model to create improved audio data corresponding to clean speech from the set of feature vectors.