Neural Network Weight Quantization for Mobile AI Memory Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require significant time to derive output values due to a large number of neurons, and their use is limited by memory capacity in mobile devices.
Innovation Solution
The solution involves quantizing and dequantizing trained connection strengths using iterative methods to lighten the neural network, reducing the bit resolution while maintaining accuracy through iterative quantization and retraining processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large number of neurons are used in deep neural networks, then accuracy of output value is improved, but time for deriving output value is delayed
Solution Approach 1:
The patent applies parameter changes by quantizing the connection strengths (weights) from high precision (e.g., 32-bit floating point) to low precision (e.g., 8-bit integer or lower). This parameter transformation reduces the computational burden and memory requirements while maintaining acceptable accuracy through iterative quantization and retraining processes that adapt the quantized weights to preserve network performance.
Solution Approach 2:
The patent extracts and removes unnecessary precision from the connection strengths by quantizing them to lower bit resolutions. By taking out the excess precision that is not critical for maintaining accuracy, the system reduces computational complexity and accelerates inference speed without significantly degrading model performance.
2Measurement precision
If a large number of neurons are used in deep neural networks, then accuracy of output value is improved, but memory capacity is exceeded in mobile devices
Solution Approach 1:
The patent changes the parameter of connection strength precision from high (32-bit floating point) to low (8-bit integer or lower precision). This parameter transformation dramatically reduces the memory footprint of the neural network model, enabling deployment on mobile devices with limited memory while maintaining acceptable accuracy through iterative quantization and retraining.
Solution Approach 2:
The patent uses low-precision quantized weights that can be stored and processed with minimal memory resources. By replacing high-precision floating-point weights with low-precision integer representations, the system creates a lightweight model that fits within the memory constraints of mobile devices.
3Quantity of substance
If bit resolution of connection strengths is reduced, then model size is reduced, but accuracy may be degraded
Solution Approach 1:
The patent employs feedback through iterative quantization and retraining processes. The system quantizes the weights to lower precision, evaluates the accuracy degradation, then retrains the network with quantized weights and iterates the process. This feedback loop allows the model to adapt to the reduced precision, minimizing accuracy loss while maintaining compact model size.
Solution Approach 2:
The patent performs preliminary quantization of connection strengths before final model deployment. By pre-quantizing the weights and performing iterative retraining in advance, the system prepares an optimized low-precision model that balances model size reduction with accuracy preservation, avoiding the need for complex runtime precision management.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Disclosed is an electronic device. The present electronic device includes: a memory; and a processor which quantizes a neural network, trained on the basis of deep learning, to generate a quantized neural network, and stores the quantized neural network in the memory, wherein the processor quantizes, in preset first bit units, trained connection strengths between neurons of the trained neural network, inverse-quantizes the quantized connection strengths in preset second bit units, retrains the inverse-quantized connection strengths, and quantizes the retrained connection strengths in the preset first bit units.