AI Model Decompression Using Grouped Weight Matrix Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for decompressing deep learning models face challenges in maintaining accuracy while increasing compression rates and enabling parallel processing during the decompression process, as they require complex processes to restore original weight matrices from quantized data.
Innovation Solution
A decompression apparatus and method that includes a memory for storing compressed data, a decoder with logic circuits for decompressing the data, and a processor for obtaining neural network processible data, utilizing representative value matrices, pruning index matrices, and patch information to update and de-interleave the data, allowing for efficient neural network processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If quantization is applied to increase compression rate, then memory usage is reduced, but processing speed decreases due to inability to divide data into parallel groups
Solution Approach 1:
The patent divides the weight matrix into multiple groups based on magnitude ranges (e.g., first group for weights with absolute value ≤ first threshold, second group for weights with absolute value > first threshold and ≤ second threshold, etc.). This segmentation allows each group to be processed independently and in parallel, resolving the contradiction between compression and processing speed.
Solution Approach 2:
The patent applies different quantization parameters to different groups of weights. Each group uses quantization levels optimized for its specific magnitude range, allowing for both high compression ratios and accurate reconstruction. This parameter differentiation enables parallel processing while maintaining compression efficiency.
2Measurement precision
If complex restoration process is used to restore original weight matrix from quantized data, then accuracy is improved, but device complexity increases
Solution Approach 1:
The restoration process is segmented into independent group restoration operations. Each group's weights are restored using group-specific quantization parameters and offset values, avoiding the need for complex global restoration algorithms. This segmentation simplifies the overall device complexity while maintaining accuracy.
Solution Approach 2:
The patent pre-calculates and stores quantization parameters, offset values, and group boundaries during the compression phase. These preliminary computations enable simple and fast restoration operations during decompression, reducing the complexity of the restoration process while maintaining high accuracy.
3Quantity of substance
If high compression rate is achieved through pruning and quantization, then memory usage is reduced, but parallel processing capability is lost
Solution Approach 1:
The patent segments the weight matrix into multiple independent groups that can be processed in parallel. Each group maintains its own quantization parameters and can be decompressed independently, restoring parallel processing capability while preserving high compression rates achieved through pruning and quantization.
Solution Approach 2:
The patent applies quantization at different levels of granularity across different groups, allowing some groups to use more aggressive compression while others maintain higher precision. This partial differentiation enables parallel processing while achieving overall high compression rates.
Data Source
AI summary
A decompression apparatus is provided. The decompression apparatus includes a memory configured to store compressed data decompressed and used in neural network processing of an artificial intelligence model, a decoder configured to include a plurality of logic circuits related to a compression method of the compressed data, decompress the compressed data through the plurality of logic circuits based on an input of the compressed data, and output the decompressed data, and a processor configured to obtain data of a neural network processible form from the data output from the decoder.