Neural Network Table Lookup Quantization for Lower Memory Energy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale neural networks require significant storage capacity and result in high memory access energy consumption due to numerous parameters and extensive computations, posing challenges in efficient data processing.
Innovation Solution
The method involves quantizing weights and input neurons using clustering algorithms to create weight and neuron codebooks, allowing for table lookup operations, which reduces the number of stored parameters and energy consumption by simplifying memory access and computation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large-scale neural networks are used to improve processing capability, then productivity is improved, but use of energy by stationary object increases due to high memory access energy consumption
Solution Approach 1:
The patent segments the continuous weight values into discrete codebooks with limited precision (e.g., 3-bit or 4-bit quantization). By dividing the weight space into discrete segments represented by codebook entries, the system reduces the memory bandwidth and energy required to access weight parameters while maintaining acceptable neural network processing capability.
Solution Approach 2:
The patent changes the precision parameter of weight representations from high-precision floating-point or fixed-point formats to low-precision integer codes. This parameter change reduces the number of bits that need to be accessed from memory, directly lowering memory access energy consumption while preserving the essential computational functionality.
2Measurement precision
If high-precision weights are stored to maintain computation accuracy, then measurement precision is improved, but use of energy by stationary object increases due to larger storage capacity requirements
Solution Approach 1:
The patent applies partial precision by using codebooks that provide just enough precision for acceptable neural network performance rather than storing full-precision weights. The codebook approach provides selective precision where only the most significant weight patterns are preserved, reducing storage energy while maintaining sufficient computational accuracy.
3Measurement precision
If full-precision weights and neurons are used to maintain computation accuracy, then measurement precision is improved, but device complexity increases due to large-scale computation requirements
Solution Approach 1:
The patent uses codebooks as simplified copies of the full weight space. Instead of storing and processing all possible weight values, the system maintains compact codebook representations that capture the essential weight patterns. This copying approach reduces computation unit complexity by working with discrete code indices rather than continuous weight values.
Data Source
AI summary
The application provides an operation method and device. Quantized data is looked up to realize an operation, which simplifies the structure and reduces the computation energy consumption of the data, meanwhile, a plurality of operations are realized.