Fixed-Point Matrix Compression for Deep Learning Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models require significant storage space and computational resources due to matrix multiplication operations, necessitating a method to compress floating-point number matrices while maintaining precision.
Innovation Solution
The method involves obtaining the minimum and maximum values of the floating-point number matrices, performing compression processing based on bit width to convert them into fixed-point number matrices, and then decompressing the results to maintain precision and reduce storage and computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point number matrices are used in deep learning models, then processing precision is maintained, but storage space and computational load increase significantly
Solution Approach 1:
The patent applies parameter changes by converting the number representation system from floating-point to fixed-point. This changes the fundamental parameter of how numerical data is stored and processed, allowing the model to maintain sufficient precision while significantly reducing storage requirements and computational complexity
Solution Approach 2:
The patent segments the fixed-point number representation into integer part and decimal part, with the decimal part being selectively retained based on precision requirements. This segmentation allows the system to keep only the necessary precision components while discarding redundant data, thereby reducing storage space while maintaining processing precision
2Measurement precision
If floating-point number matrices are used in deep learning models, then processing precision is maintained, but computational load increases significantly
Solution Approach 1:
By changing the numerical representation parameter from floating-point to fixed-point, the patent reduces computational load. Fixed-point arithmetic requires fewer computational resources and simpler hardware operations compared to floating-point arithmetic, while still maintaining the precision needed for deep learning model processing
Solution Approach 2:
The patent extracts and retains only the necessary decimal part of the fixed-point number based on precision requirements, removing redundant high-precision components. This extraction process reduces computational complexity by eliminating unnecessary calculation operations while preserving the precision needed for accurate model processing
3Quantity of substance
If fixed-point compression is applied to floating-point matrices, then storage space is reduced, but precision may be lost
Solution Approach 1:
The patent applies dynamics by adaptively determining the number of decimal bits to retain based on the specific precision requirements of different model parameters. Instead of using a static fixed-point representation, the system dynamically adjusts the precision level for different parts of the model, ensuring sufficient precision is maintained while maximizing storage compression
Solution Approach 2:
The patent applies local quality by allowing different parts of the model matrix to have different precision levels. The decimal part retention is customized based on local precision requirements of specific parameters, ensuring that critical parameters maintain higher precision while less critical parameters use lower precision, thereby optimizing the balance between storage space and precision
4Quantity of substance
If conventional compression methods are used, then storage space is reduced, but bit overflow and decimal point calibration errors occur
Solution Approach 1:
The patent introduces an intermediary mechanism - the sign bit and structured fixed-point representation - that mediates between the need for compression and the need for precision. The sign bit explicitly tracks the sign of numbers, and the structured fixed-point format with separate integer and decimal parts prevents overflow and calibration errors that occur in conventional compression methods
Solution Approach 2:
The patent applies preliminary action by pre-arranging the fixed-point number format with designated sign bit positions and decimal point locations before compression. This preliminary structuring prevents overflow and calibration errors during compression operations, as the format is designed in advance to accommodate the full range of values without loss of precision
Data Source
AI summary
The present disclosure provides a method and apparatus for processing a floating point number matrix, an apparatus and a computer readable storage medium. In embodiments of the present disclosure, the minimum value of the floating point number model matrix and the maximum value of the floating point number model matrix are obtained according to a floating point number model matrix to be compressed, and then, compression processing is performed for the floating point number model matrix to obtain the fixed point number model matrix according to the bit width, the minimum value of the floating point number model matrix and the maximum value of the floating point number model matrix. The compression processing is performed for the floating point number model matrix of the deep learning model by a fixed point method, to obtain the fixed point number model matrix and reduce the storage space and amount of operation of the deep learning model. Meanwhile, the present disclosure proposes a framework for implementing the apparatus in the deep learning network to maximize the deep learning network precision, that is, a multiplication portion of the matrix uses the apparatus, and operations of other portions such as activation function retain the floating point operation.