AI Model Matrix Quantization Using Group Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial intelligence models require vast amounts of data storage and high energy consumption, and quantization leads to performance deterioration.
Innovation Solution
An electronic apparatus that divides data matrices into groups, clusters them in a multi-dimensional space, determines representative values, and quantizes data using binary codes to minimize data capacity while maintaining performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If quantization is applied to reduce data capacity, then storage requirements decrease, but model performance deteriorates
Solution Approach 1:
The patent divides the weight matrix into multiple sub-matrices and further segments weights into integer and fraction parts. This segmentation allows different quantization strategies to be applied to different parts, reducing overall data capacity while preserving essential model performance through selective precision maintenance.
Solution Approach 2:
The patent applies different quantization precision levels to different parts of the model based on their importance. Critical weights maintain higher precision while less important weights are aggressively quantized. This local quality approach ensures performance is maintained in key areas while achieving compression overall.
2Quantity of substance
If high capacity memory is used to store vast amounts of data, then data storage requirements are met, but device complexity and cost increase
Solution Approach 1:
The patent changes the parameter of data precision from high-precision floating-point to low-precision integer representations. This parameter change dramatically reduces storage capacity requirements while maintaining acceptable model performance, eliminating the need for high-capacity memory systems.
3Quantity of substance
If aggressive quantization is applied to maximize compression, then data capacity is reduced, but manufacturing precision of model accuracy decreases
Solution Approach 1:
The patent performs preliminary calibration and optimization before final quantization. Representative values are pre-computed and optimization is performed on the quantized model structure. This preliminary action ensures that even aggressive quantization maintains acceptable accuracy by preparing the model to withstand the precision reduction.
Data Source
Figure 1a~2a
Figure 2b~4a
Figure 4b~5a
AI summary
An electronic apparatus is provided. The electronic apparatus includes a storage storing a matrix included in an artificial intelligence model, and a processor. The processor divides data included in at least a portion of the matrix by one of rows and columns of the matrix to form groups, clusters the groups into clusters based on data included in each of the groups, and quantizes data divided by the other one of rows and columns of the matrix among data included in each of the clusters.