Student Neural Network Optimization via Pruning and Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks require significant processing resources for training, which is computationally intensive and often centralized, and there is a need for optimized neural networks that can be deployed on devices with lesser processing capabilities while maintaining accuracy.
Innovation Solution
A computer-implemented method to optimize a student neural network by adjusting its parameters using a second processing system to match the reference output of a previously-trained neural network, involving techniques like pruning, weight reduction, and compression, ensuring the difference between reference and student neural network outputs satisfies a stopping criterion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a neural network is trained to maintain high predictive accuracy, then the model performance is improved, but the processing resources and computational power required increase significantly
Solution Approach 1:
The patent extracts and removes redundant connections and neurons from the neural network through pruning techniques. By identifying and eliminating unnecessary components that do not contribute significantly to predictive accuracy, the network is simplified while maintaining its core functionality and accuracy on edge devices with limited processing resources
Solution Approach 2:
The patent applies quantization to change the precision parameters of weights and activations from high-precision formats (e.g., 32-bit floating point) to lower-precision formats (e.g., 8-bit integers). This parameter transformation reduces computational complexity and memory requirements while preserving sufficient accuracy for the application
2Ease of operation
If the neural network architecture is optimized for reduced processing requirements, then the ease of deployment on edge devices is improved, but the model complexity and processing capability may be reduced
Solution Approach 1:
The patent segments the neural network into distinct functional components and applies different optimization techniques to different segments. Important feature extraction layers are preserved with higher fidelity while less critical layers are more aggressively pruned or quantized, maintaining overall accuracy while reducing complexity
Solution Approach 2:
The patent implements dynamic optimization where the network architecture and parameters are adaptively adjusted based on the specific edge device's capabilities. The optimization process dynamically identifies which components to prune or quantize based on performance requirements and hardware constraints
3Productivity
If pruning and compression techniques are applied to reduce model size, then the processing burden is alleviated, but the predictive accuracy may deteriorate
Solution Approach 1:
The patent employs feedback mechanisms during the pruning and quantization process where the model is evaluated on validation data after each optimization step. If accuracy degradation exceeds a threshold, the optimization parameters are adjusted and the process is repeated, ensuring that processing efficiency gains do not come at the cost of unacceptable accuracy loss
Solution Approach 2:
The patent performs preliminary training of the neural network to achieve high accuracy on a powerful device before applying pruning and compression techniques. This pre-trained model serves as the foundation for subsequent optimization, ensuring that the reduced model starts from a high-performance baseline and maintains accuracy through careful optimization
Data Source
AI summary
A computer-implemented method of optimising a student neural network (SNN), based on a previously-trained neural network (PTNN) trained on first data (FD) using a first processing system (FPS). The method includes using a second processing system (SPS) to generate reference output data (ROD) from the previously-trained neural network (PTNN) in response to inputting second data (SD) to the previously-trained neural network (PTNN). The method also includes optimising a student neural network (SNN) for processing the second data (SD) with the second processing system (SPS), by using the second processing system (SPS) to adjust a plurality of parameters of the student neural network (SNN) such that a difference (DIFF) between the reference output data (ROD), and second output data (SOD) generated by the student neural network (SNN) in response to inputting the second data (SD) to the student neural network (SNN), satisfies a stopping criterion.