GAN Audio Super Resolution Model for High Fidelity Signal Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio super-resolution techniques are unable to effectively convert low sample rate audio to high fidelity sample rate audio, resulting in compromised sound quality when using audio captured by common devices for professional productions.
Innovation Solution
An audio super-resolution model trained as a generator network in a generative adversarial network (GAN) is used to predict plausible high-frequency data for narrowband input audio, generating full-band audio that is indistinguishable from real full-band recordings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional spectral methods with MSE/MAE objective functions are used to estimate missing high-frequency content, then energy compensation for missing frequencies is achieved, but the estimated spectrogram lacks details and introduces smoothing effects
Solution Approach 1:
The patent changes the objective function parameter from traditional MSE/MAE to perceptual loss functions that operate in the time domain rather than frequency domain. This allows the model to preserve temporal details and avoid excessive smoothing while still compensating for missing high-frequency energy content.
Solution Approach 2:
The patent replaces the traditional spectral estimation approach with a waveform-based deep learning model that directly predicts time-domain audio signals. This substitution enables the model to capture fine temporal details that are lost in spectral methods while still recovering missing high-frequency content.
2Ease of manufacture
If deep learning methods trained at 16 kHz or 22 kHz sample rates are used, then audio processing capability is provided, but high fidelity applications requiring 44 kHz or higher sample rates cannot be achieved
Solution Approach 1:
The patent performs preliminary upsampling of the narrowband audio signal to the target high sample rate (44 kHz or higher) before processing. This preliminary action allows the model to work directly at the desired output sample rate, enabling high fidelity applications while maintaining ease of implementation through standard upsampling techniques.
Solution Approach 2:
The patent changes the training sample rate parameter from traditional 16 kHz or 22 kHz to high sample rates of 44 kHz or higher. This parameter change enables the model to learn features at the target fidelity level, allowing direct generation of high-quality audio suitable for professional applications without requiring post-processing resampling.
3Ease of operation
If narrowband audio from common devices is used for professional productions, then accessibility and ease of recording is improved, but audio quality is compromised
Solution Approach 1:
The patent replaces traditional spectral processing methods with a waveform-based generative adversarial network that directly synthesizes high-fidelity audio waveforms. This substitution enables the model to generate realistic high-frequency content that preserves temporal details, transforming low-quality narrowband audio into professional-grade wideband audio suitable for production use.
Solution Approach 2:
The patent introduces a generative adversarial network as an intermediary between narrowband input audio and the final high-fidelity output. This intermediary model learns to map low-sample-rate audio to high-sample-rate audio by training on paired datasets, effectively bridging the quality gap while maintaining accessibility of common recording devices.
Data Source
AI summary
Embodiments are disclosed for generating full-band audio from narrowband audio using a GAN-based audio super resolution model. A method of generating full-band audio may include receiving narrow-band input audio data, upsampling the narrow-band input audio data to generate upsampled audio data, providing the upsampled audio data to an audio super resolution model, the audio super resolution model trained to perform bandwidth expansion from narrow-band to wide-band, and returning wide-band output audio data corresponding to the narrow-band input audio data.