Selective Neural Network Weight Quantization for Edge Deployment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) require significant computational resources and memory, making them costly and inefficient for deployment on resource-constrained devices such as edge devices, autonomous cars, and drones.
Innovation Solution
A method for training neural networks to selectively quantize weights as either binary or ternary weights, reducing the computational operations required and enabling deployment on resource-constrained devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full precision neural networks are used, then predictive accuracy is maintained, but computational resources and memory requirements increase significantly
Solution Approach 1:
The patent applies parameter changes by quantizing weight values from full precision (32-bit floating point) to low precision (1-bit binary or 2-bit ternary). This transformation fundamentally changes the parameter representation, reducing memory requirements and computational complexity while maintaining acceptable predictive accuracy through selective quantization of individual filters based on their importance to the network's output
Solution Approach 2:
The patent implements local quality by applying different quantization strategies to different filters within the neural network. Rather than uniformly quantizing all weights, the method selectively identifies and quantizes only those filters that have minimal impact on the network's predictive accuracy. This allows critical filters to maintain full precision while non-critical filters are quantized to low precision, optimizing the balance between accuracy and resource efficiency
2Productivity
If low-bit quantized DNNs are used, then computational complexity is reduced, but predictive accuracy deteriorates
Solution Approach 1:
The patent transforms the parameter representation from full precision to low precision (1-bit or 2-bit) through quantization, fundamentally changing how weight values are stored and processed. This parameter change reduces computational complexity and memory requirements while the selective application to individual filters ensures that accuracy-critical parameters maintain higher precision
Solution Approach 2:
The patent applies different precision levels to different parts of the network (individual filters) based on their importance. By analyzing each filter's contribution to the network output, the method assigns appropriate quantization levels locally, ensuring that filters critical to predictive accuracy maintain full precision while less important filters use low-bit representation, thus minimizing overall accuracy loss
Data Source
AI summary
A method and processing unit for training a neural network to selectively quantize weights of a filter of the neural network as either binary weights or ternary weights. A plurality of training iterations a performed that each comprise: quantizing a set of real-valued weights of a filter to generate a corresponding set of quantized weights; generating an output feature tensor based on matrix multiplication of an input feature tensor and the set of quantized weights; computing, based on the output feature tensor, a loss based on a regularization function that is configured to move the loss towards a minimum value when either: (i) the quantized weights move towards binary weights, or (ii) the quantized weights move towards a ternary weights; computing a gradient with an objective of minimizing the loss; updating the real-valued weights based on the computed gradient. When the training iterations are complete, a set of weights quantized from the updated real-valued weights is stored as either a set of binary weights or a set of ternary weights.