Neural Network Compression Using Layer-Wise Norm-Based Quantization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial Neural Networks (ANNs) are computationally demanding and energy-intensive due to their large memory footprint, making it challenging to implement them on devices with limited hardware capabilities, such as smartphones and IoT devices, and even in cloud computing where resource abundance is not the primary concern.
Innovation Solution
A data processing apparatus and method that compresses neural networks by quantizing and encoding neural network weights, where the quantization bin size is determined based on the norm of each processing layer, allowing for efficient compression and operation of neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the memory footprint of neural networks is reduced through compression techniques, then the neural network can be deployed on resource-constrained devices, but the accuracy of the neural network may deteriorate
Solution Approach 1:
The patent applies local quality by using layer-wise norm-based quantization where each layer is quantized independently based on its own norm characteristics. This allows different quantization parameters to be applied to different layers, optimizing the balance between compression and accuracy preservation for each specific layer rather than applying a uniform quantization strategy across all layers.
Solution Approach 2:
The patent changes the quantization parameter (bin size) based on the norm of each layer. By dynamically adjusting the quantization bin size according to the layer-specific norm, the system adapts the compression level to the importance and characteristics of each layer, thereby maintaining accuracy while achieving compression.
2Device complexity
If uniform quantization bin size is used for all layers, then the compression process is simpler, but the compression efficiency and accuracy preservation deteriorate
Solution Approach 1:
The patent implements local quality by computing and applying separate quantization bin sizes for each layer based on its norm. This approach increases computational complexity during the quantization setup phase but significantly improves compression efficiency and accuracy preservation by adapting to the specific characteristics of each layer.
Solution Approach 2:
The patent performs preliminary computation of layer norms and quantization bin sizes before the actual quantization process. This preliminary action allows the system to prepare optimized quantization parameters in advance, making the subsequent quantization process more efficient and accurate without requiring complex real-time adjustments.
Data Source
AI summary
A data processing apparatus for compressing a neural network is disclosed. The apparatus comprises a processing circuitry configured to operate the neural network that comprises a plurality of processing layers. Each processing layer comprises a plurality of neural network weights. The processing circuitry is further configured to compress the neural network by quantizing the plurality of neural network weights of each processing layer using a respective quantization bin size and by encoding the plurality of quantized neural network weights of each processing layer to obtain a compressed neural network. The processing circuitry is further configured to determine, for each processing layer, a norm based on the plurality of neural network weights of each processing layer and to determine the respective quantization bin size for each processing layer based on the norm of the processing layer.