Generative Neural Network Pruning for Low-Bitrate Audio Artifact Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing generative neural networks for audio processing face high computational complexity and memory usage, limiting their deployment on devices with limited resources, while low-bitrate audio coding introduces pre-echo artifacts and quantization noise that degrade audio quality.
Innovation Solution
A method for determining parameters of a generative neural network, specifically a Generator in a Generative Adversarial Network (GAN) setup, involving pruning control information to optimize the trade-off between model complexity and perceptual quality, with sensitivity parameters adjusting the pruning severity across layers to maintain audio quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a fully trained generative neural network is used for audio processing, then audio quality enhancement and artifact reduction are improved, but memory usage and computational complexity increase significantly
Solution Approach 1:
The patent extracts and removes redundant or less important parameters from the generative neural network while retaining the essential parameters needed for audio quality enhancement. This parameter selection process reduces the overall model complexity and memory requirements while maintaining the network's ability to effectively reduce artifacts and enhance audio quality.
Solution Approach 2:
The patent applies different levels of parameter retention to different parts of the neural network based on their importance. Critical parameters that contribute most to audio enhancement are preserved with higher fidelity, while less critical parameters are reduced or removed. This localized optimization allows the system to maintain high audio quality enhancement capability while reducing overall computational complexity.
2Reliability
If a fully trained generative neural network is used for audio processing, then audio quality enhancement and artifact reduction are improved, but memory usage increases significantly
Solution Approach 1:
The patent extracts and removes redundant or less important parameters from the generative neural network while retaining the essential parameters needed for audio quality enhancement. This parameter selection process reduces the overall model complexity and memory requirements while maintaining the network's ability to effectively reduce artifacts and enhance audio quality.
Solution Approach 2:
Instead of starting with a full model and adding features, the patent inverts the approach by starting with a comprehensive parameter set and systematically removing unnecessary parameters. This inversion strategy enables more efficient memory usage while preserving the essential audio enhancement functionality.
3Quantity of substance
If low-bitrate audio coding is used to reduce bandwidth and storage, then information capacity requirements are reduced, but pre-echo artifacts and quantization noise increase
Solution Approach 1:
The patent converts the harmful pre-echo artifacts and quantization noise into beneficial training data for the neural network. By training the generative network on low-bitrate coded audio with these artifacts, the network learns to recognize and suppress these specific types of distortions, effectively transforming the harmful effects of low-bitrate coding into an opportunity for targeted artifact reduction.
Data Source
Figure 1
Figure 2~3A
Figure 3B
AI summary
Described herein is a method of determining parameters for a generative neural network for processing an audio signal, wherein the generative neural network includes an encoder stage mapping to a coded feature space and a decoder stage, each stage including a plurality of convolutional layers with one or more weight coefficients, the method comprising a plurality of cycles with sequential processes of: pruning the weight coefficients of either or both stages based on pruning control information, the pruning control information determining the number of weight coefficients that are pruned for respective convolutional layers; training the pruned generative neural network based on a set of training data; determining a loss for the trained and pruned generative neural network based on a loss function; and determining updated pruning control information based on the determined loss and a target loss. Further described are corresponding apparatus, programs, and computer-readable storage media.