Neural Network Quantization via Codebook Lookup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale neural networks require significant storage capacity and result in high memory access energy consumption due to numerous parameters and computations.
Innovation Solution
The method involves quantizing weights and input neurons, creating weight and neuron dictionaries/codebooks, and determining a computational codebook for efficient table lookup operations, reducing the number of bits required to represent data and minimizing memory access and computation energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large-scale neural networks are used to improve recognition accuracy and processing capability, then the neural network can handle more complex tasks, but storage capacity requirements and memory access energy consumption increase significantly
Solution Approach 1:
The patent applies parameter changes by quantizing neural network weights and neurons from high precision (e.g., 32-bit floating point) to low precision (e.g., 8-bit integers or binary values). This parameter transformation reduces the storage bits per parameter from 32 bits to 8 bits or fewer, directly decreasing memory access energy consumption while preserving sufficient processing capability for practical applications
Solution Approach 2:
The patent creates codebooks that serve as compressed representations or copies of the original neural network parameters. Instead of storing and accessing all original high-precision weights and neurons, the system stores compact codebooks that can be used to reconstruct or approximate the original parameters during inference, reducing memory access requirements
2Measurement precision
If high precision weights and neurons are stored to maintain computational accuracy, then the neural network maintains high processing accuracy, but storage overhead and memory access requirements increase
Solution Approach 1:
The patent transforms the precision parameter of neural network weights and neurons from high precision (32-bit floating point) to low precision (8-bit integers, 4-bit values, or binary). This parameter change reduces storage overhead by 4x or more while maintaining sufficient computational accuracy through quantization algorithms that preserve important numerical properties
Solution Approach 2:
The patent creates codebooks as compact copies of quantized parameter sets. These codebooks store the essential information needed for neural network computation in a space-efficient format, allowing the system to maintain computational accuracy while significantly reducing the quantity of stored data
3Loss of energy
If quantization is applied to reduce parameters and computation amount, then storage and energy consumption are reduced, but the complexity of the quantization process and potential loss of precision increase
Solution Approach 1:
The patent performs quantization as a preliminary action during the training or model preparation phase. Codebooks are pre-computed and stored, so that during actual neural network inference, the system only needs to perform simple table lookups based on quantized indices rather than complex quantization calculations, reducing runtime computation energy consumption
Solution Approach 2:
The patent introduces codebooks as intermediary structures that mediate between the original high-precision neural network parameters and the low-precision computation process. The codebooks serve as lookup tables that translate quantized input indices into reconstructed parameter values, simplifying the quantization process during inference
Data Source
Figure 1~4
Figure 5~7
Figure 8~9
AI summary
The application provides an operation method and device. Quantized data is looked up to realize an operation, which simplifies the structure and reduces the computation energy consumption of the data, meanwhile, a plurality of operations are realized.