AI Model Weight Compression via Pruning and Don't Care Substitution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models with large memory sizes are difficult to implement in normal electronic devices like smartphones due to memory limitations, and existing compression methods are inefficient in terms of memory bandwidth utilization and power consumption, especially when performing parallel operations.
Innovation Solution
A data compression method that prunes insignificant weight parameters, identifies unchanged and changed parameters, substitutes changed parameters with 'don't care' values, and applies quantization and the Viterbi algorithm to reduce the size of AI model weights, enabling efficient compression and decompression while maintaining pruning accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning models with large memory sizes are used, then recognition rate and user understanding are improved, but memory limitation in normal electronic devices prevents implementation
Solution Approach 1:
The patent extracts and removes insignificant weight parameters from the deep learning model through pruning operations. By identifying and eliminating parameters that contribute minimally to the recognition rate, the model size is reduced to fit within the memory constraints of normal electronic devices while preserving the essential functionality and accuracy of the model.
Solution Approach 2:
The patent applies quantization to change the precision parameters of the weight parameters from high-precision formats to lower-precision formats. This parameter transformation reduces the memory footprint of each weight parameter while maintaining the overall recognition rate, enabling the model to run on devices with limited memory resources.
2Quantity of substance
If previous compression methods are applied to deep learning, then model size is reduced, but memory bandwidth utilization becomes inefficient and power consumption increases
Solution Approach 1:
The patent performs pruning operations in advance before quantization and compression. By pre-identifying and removing insignificant parameters beforehand, the subsequent compression and quantization processes operate on a reduced dataset, improving memory bandwidth utilization and reducing the computational energy required during deployment.
Solution Approach 2:
The patent segments the weight parameters into significant and insignificant groups through pruning. This segmentation allows the compression system to focus resources on compressing only the essential parameters efficiently, while discarding redundant data, thereby optimizing both memory bandwidth usage and power consumption during model loading and inference.
3Quantity of substance
If pruning is applied to reduce model size, then memory footprint is reduced, but compression efficiency may be compromised
Solution Approach 1:
The patent performs pruning as a preliminary step before quantization and compression. By pre-removing insignificant parameters, the subsequent compression algorithms work on a smaller, more concentrated dataset of essential parameters, which improves compression efficiency and reduces the memory footprint simultaneously without compromising the essential model functionality.
Data Source
AI summary
A data compression method and a data decompression method are provided. The method includes pruning an original data including a plurality of weight parameters, identifying at least one first weight parameter of which at least one first value is not changed by the pruning, among multiple weight parameters included in the pruned original data, and obtaining a first index data including location information of the at least one first weight parameter of which the at least one first value is not changed, identifying at least one second weight parameter of which at least one second value is changed by the pruning, among the multiple weight parameters included in the pruned original data, and substituting the at least one second weight parameter of which the at least one second value is changed with a don't care parameter.