Generative Neural Network Pruning for Low-Resource Audio Enhancement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based audio enhancement using generative neural networks is limited by high computational requirements and memory usage, particularly on devices with limited resources, leading to inefficiencies in processing low-bitrate coded audio.
Innovation Solution
A method for pruning the encoder and decoder stages of a Generative Adversarial Network (GAN) based on sensitivity parameters to reduce memory usage and computational complexity while maintaining performance, involving sensitivity analysis and alternating pruning and training passes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a fully trained generative neural network is used for audio enhancement, then audio quality improvement is achieved, but computational requirements and memory usage increase significantly
Solution Approach 1:
The patent extracts and removes unnecessary weights from the neural network through pruning techniques. Specifically, it prunes weights in convolutional layers that have minimal impact on audio quality, thereby reducing computational requirements and memory usage while maintaining the network's audio enhancement capabilities
Solution Approach 2:
The patent applies different pruning strategies to different parts of the neural network based on their importance. Critical layers for audio quality preservation are protected from aggressive pruning, while less critical layers undergo more extensive weight removal, achieving a balanced reduction in computational complexity
2Reliability
If a fully trained generative neural network is deployed on mobile devices, then audio enhancement performance is achieved, but memory capacity and computing resources are exceeded
Solution Approach 1:
The patent extracts and removes redundant weights from the neural network model, reducing the total memory footprint. By identifying and eliminating less important connections through pruning, the model achieves smaller memory requirements suitable for mobile devices while preserving essential audio enhancement functionality
Solution Approach 2:
The patent applies partial pruning rather than complete weight removal. By selectively pruning only the most redundant weights and protecting critical pathways, it achieves sufficient memory reduction for mobile deployment without excessively degrading model performance
Data Source
AI summary
Described herein is a method of determining parameters for a generative neural network for processing an audio signal, wherein the generative neural network includes an encoder stage mapping to a coded feature space and a decoder stage, each stage including a plurality of convolutional layers with one or more weight coefficients, the method comprising a plurality of cycles with sequential processes of: pruning the weight coefficients of either or both stages based on pruning control information, the pruning control information determining the number of weight coefficients that are pruned for respective convolutional layers; training the pruned generative neural network based on a set of training data; determining a loss for the trained and pruned generative neural network based on a loss function; and determining updated pruning control information based on the determined loss and a target loss. Further described are corresponding apparatus, programs, and computer-readable storage media.