Neural Network Weight Compression Using Lossless Exponents and LUT Mantissas
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern deep neural networks (DNNs) are hindered by the large number of parameters, which require significant processing resources and time for training, especially due to the need for multiple iterations to determine accurate parameter values, and existing compression methods like floating-point clustering algorithms are resource-intensive and iterative.
Innovation Solution
The system compresses floating-point parameters by separating them into mantissa and exponent values, using fixed-point arithmetic to compress mantissa values and encoding exponents losslessly, reducing the number of values to transmit and processing resources needed, while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If floating-point parameters are compressed using traditional clustering algorithms, then the number of parameters is reduced, but processing resources and time are significantly increased due to iterative optimization
Solution Approach 1:
The patent segments floating-point parameters into two distinct components: mantissa and exponent. This segmentation allows each component to be processed independently using different compression strategies, avoiding the need for iterative optimization of the entire parameter. The mantissa is compressed using lookup tables while the exponent is encoded separately, enabling non-iterative compression that reduces training time while maintaining parameter quantity reduction.
Solution Approach 2:
The patent changes the representation parameters of floating-point numbers by separating mantissa and exponent into independent compressible units. This parameter transformation enables the use of fixed-point arithmetic for mantissa compression and separate encoding for exponents, eliminating the computational complexity of traditional floating-point clustering while achieving comparable or better compression ratios.
2Productivity
If mantissa values are compressed using fixed-point values, then compression efficiency is improved, but precision may be lost
Solution Approach 1:
The patent introduces lookup tables (LUTs) as intermediary structures that map compressed mantissa indices to actual mantissa values. This intermediary mechanism allows fixed-point compression to achieve high efficiency while maintaining precision, as the LUT stores pre-computed accurate values that can be retrieved without iterative calculation, thus balancing compression efficiency with parameter precision.
Solution Approach 2:
The patent performs preliminary computation by pre-calculating and storing accurate mantissa values in lookup tables before the actual compression process. This preliminary action ensures that when compression is applied, the precision is already preserved in the LUT structure, eliminating the need for iterative optimization during training while maintaining high compression efficiency.
3Measurement precision
If exponent values are encoded losslessly, then accuracy is maintained, but compression ratio is reduced compared to lossy methods
Solution Approach 1:
The patent applies different compression qualities to different components of the floating-point parameter: lossless encoding for exponents and lossy compression for mantissas. This local quality differentiation is justified by the observation that mantissa values contribute more significantly to parameter accuracy than exponents. By applying higher compression ratio to the more critical mantissa component while maintaining exact precision for exponents, the overall system achieves both accuracy and efficiency.
4Reliability
If a large number of parameters are stored in floating-point format, then model accuracy is maintained, but memory area and power consumption increase
Solution Approach 1:
The patent segments floating-point parameters into mantissa and exponent components, allowing independent optimization of storage requirements. The mantissa, which carries the significant precision information, is compressed using lookup tables with configurable precision, while the exponent is stored using efficient encoding schemes. This segmentation enables significant reduction in memory area while preserving model accuracy by maintaining high-precision storage for the critical mantissa portion.
Solution Approach 2:
The patent changes the storage parameters from full floating-point format to a segmented representation with configurable precision levels. By adjusting the number of bits allocated to mantissa and exponent storage, the system can optimize the trade-off between model accuracy and memory area, allowing accurate model training with reduced memory footprint and lower power consumption.
Data Source
AI summary
Systems, methods, and apparatuses are provided for compressing values. A plurality of parameters may be obtained from a memory, each parameter comprising a floating-point number that is used in a relationship between artificial neurons or nodes in a model. A mantissa value and an exponent value may be extracted from each floating-point number to generate a set of mantissa values and a set of exponent values. The set of mantissa values may be compressed to generate a mantissa lookup table (LUT) and a plurality of mantissa LUT index values. The set of exponent values may be encoded to generate an exponent LUT and a plurality of exponent LUT index values. The mantissa LUT, mantissa LUT index values, exponent LUT, and exponent LUT index values may be provided to one or more processing entities to train the model.