Neural Network Quantization via Loss-Optimized Subset Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural network (DNN) quantization techniques face challenges in directly applying weight quantization to deep learning processor units (DPUs) due to high computational complexity and precision requirements, making it difficult to improve power efficiency and reduce model size effectively.
Innovation Solution
A processor-implemented neural network operation method that generates a target subset of quantization points based on calculated quantization loss, using log-scale quantization and bitwidth to extract a subset of quantization points, which are then used to minimize quantization loss, thereby optimizing weight quantization for DNNs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If weight quantization is applied to reduce model size, then model size is reduced, but computational complexity and precision requirements increase making it difficult to apply directly to DPU
Solution Approach 1:
The patent segments the quantization process into multiple stages: generating candidate quantization points, extracting subsets based on bitwidth, calculating quantization loss, and generating target subsets. This segmentation allows the system to manage computational complexity by breaking down the overall quantization task into smaller, more manageable operations that can be efficiently executed on DPU hardware.
Solution Approach 2:
The patent performs preliminary actions by pre-generating candidate quantization points using log-scale quantization before the actual neural network operation. This preliminary generation of quantization points and their subsets prepares the data in advance, reducing the computational burden during real-time inference on the DPU and making the quantization more applicable to hardware constraints.
2Measurement precision
If quantization precision is increased to maintain accuracy, then quantization loss decreases, but power efficiency and computational speed are reduced
Solution Approach 1:
The patent implements dynamic quantization by generating target subsets of quantization points based on calculated quantization loss and specific bitwidth requirements. This dynamic approach allows the system to adapt the quantization precision to the actual needs of each layer or operation, using higher precision only where necessary to maintain accuracy while using lower precision elsewhere to preserve power efficiency and computational speed.
Solution Approach 2:
The patent changes the parameter of quantization precision by extracting subsets of quantization points based on different bitwidth values and optimizing them to minimize quantization loss. This parameter adjustment allows the system to find the optimal balance between accuracy and power efficiency by selectively applying different precision levels to different parts of the neural network.
3Productivity
If subset extraction is performed to reduce computational load, then processing speed increases, but quantization accuracy may deteriorate
Solution Approach 1:
The patent incorporates feedback by calculating quantization loss based on the extracted subset of quantization points and using this loss information to generate optimized target subsets. This feedback loop ensures that the subset extraction process maintains quantization accuracy by iteratively adjusting the selected quantization points to minimize the difference between original and quantized weights, even when processing speed is prioritized.
Data Source
AI summary
A neural network operation apparatus and method implementing quantization is disclosed. The neural network operation method may include receiving a weight of a neural network, a candidate set of quantization points, and a bitwidth for representing the weight, extracting a subset of quantization points from the candidate set of quantization points based on the bitwidth, calculating a quantization loss based on the weight of the neural network and the subset of quantization points, and generating a target subset of quantization points based on the quantization loss.