Neural Network Parameter Quantization for Memory Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning neural networks require significant computing power and memory, leading to performance degradation when implemented on microprocessors optimized for fixed-point data representation, necessitating a method to reduce memory size without compromising performance.
Innovation Solution
A method to analyze and modify the neural network's parameters, such as weight sizes and output data memory allocation, using a set of test input data to evaluate quality factors and compare them to ensure that performance is maintained within a tolerance, allowing for reduction in memory size while optimizing processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point representation is used for neural network parameters, then prediction performance is improved, but memory size increases
Solution Approach 1:
The patent applies parameter changes by systematically varying the precision representation (from 32-bit floating point to 16-bit and 8-bit fixed point) of neural network parameters and activation values. This allows finding an optimal balance between memory size and prediction accuracy, converting continuous precision parameters into discrete quantization levels that can be evaluated and optimized.
2Measurement precision
If neural network complexity is increased, then prediction performance is improved, but processing time increases
Solution Approach 1:
The patent changes parameters by optimizing the quantization precision at different stages of the neural network (input, hidden layers, output) and adjusting the bit-width representation. This parameter optimization reduces the computational complexity of operations while maintaining prediction accuracy, thereby reducing processing time on microprocessors.
3Measurement precision
If neural network complexity is increased, then prediction performance is improved, but computational load increases
Solution Approach 1:
The patent applies parameter changes by optimizing the precision representation of neural network parameters and activation values. By systematically varying bit-width from 32-bit floating point to lower precision formats, the computational load and energy consumption are reduced while maintaining acceptable prediction accuracy levels.
4Quantity of substance
If fixed-point representation is used, then memory size is reduced, but prediction performance deteriorates
Solution Approach 1:
The patent applies local quality by differentiating the precision requirements of different components within the neural network. Different layers, parameters, and activation values are assigned different quantization precisions based on their sensitivity to precision changes. This allows memory size to be reduced while maintaining prediction accuracy in critical regions of the network.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A method comprising an analysis of an initial parameter set defining an initial neural network (NN), the neural network (NN) being intended to be implemented by a processor (PROC), said analysis comprising a reduction of the initial memory size by at least one initial parameter so as to obtain a modified parameter set defining a neural network modified relative to the initial network, the method further comprising an implementation by said system (SYS) of the initial neural network (NN) and the modified neural network (MN) using a test input dataset (TD), an elaboration of a quality factor for the initial neural network (NN) and the modified neural network (MN) using said at least one test input dataset (TD), and a comparison between the two quality factors.