AI Weight Compression Using Viterbi Decoding for Memory-Limited Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models with large memory sizes are difficult to implement on normal electronic devices like smartphones due to memory size limitations, and previous compression methods are inefficient in terms of memory bandwidth utilization and power consumption, especially when performing parallel operations.
Innovation Solution
A data compression method that quantizes weight parameters of artificial intelligence models using the Viterbi algorithm, allowing for parallel compression while maintaining pruning rates, by substituting changed weight parameters with 'don't care' values and applying them to a Viterbi decompressor to achieve efficient compression and decompression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning models with large memory sizes are used, then recognition rate and user understanding accuracy are improved, but memory size limitation prevents implementation on normal electronic devices
Solution Approach 1:
The patent segments the weight parameters into multiple groups and applies different pruning rates to each group. This allows the model to maintain high recognition accuracy while reducing overall memory size by selectively pruning less important parameters across different segments of the weight matrix.
Solution Approach 2:
The patent applies different pruning rates to different groups of weight parameters based on their importance. By assigning higher pruning rates to less critical parameters and lower rates to more critical ones, the system achieves memory size reduction while preserving the recognition rate locally in important parameter groups.
2Productivity
If previous compression methods are applied to deep learning for parallel operation, then compression is achieved, but memory bandwidth utilization is poor and power consumption is large
Solution Approach 1:
The patent changes the parameter representation by converting weight parameters into a compressed format using different pruning rates. This parameter transformation enables more efficient memory bandwidth utilization during parallel operations and reduces the computational energy required for model inference.
Solution Approach 2:
The patent introduces dynamic pruning rates that can be adjusted for different operation modes. This allows the system to optimize between compression ratio and power consumption based on runtime requirements, enabling efficient parallel operation with adaptable resource usage.
3Quantity of substance
If compression ratio is increased to reduce model size, then memory size limitation is addressed, but accuracy maintenance becomes difficult
Solution Approach 1:
The patent divides weight parameters into multiple groups and applies different compression ratios to each segment. This segmented approach allows aggressive compression in less critical areas while maintaining higher fidelity in important parameter groups, achieving overall model size reduction without sacrificing accuracy.
Solution Approach 2:
The patent applies local quality control by assigning different pruning rates to different parameter groups based on their contribution to accuracy. This ensures that compression is applied selectively, maintaining high accuracy in critical regions while achieving significant model size reduction in less critical regions.
Data Source
Figure 1~3
Figure 4
Figure 5
AI summary
A data compression method and a data decompression method are provided. The method includes pruning an original data including a plurality of weight parameters, identifying at least one first weight parameter of which at least one first value is not changed by the pruning, among multiple weight parameters included in the pruned original data, and obtaining a first index data including location information of the at least one first weight parameter of which the at least one first value is not changed, identifying at least one second weight parameter of which at least one second value is changed by the pruning, among the multiple weight parameters included in the pruned original data, and substituting the at least one second weight parameter of which the at least one second value is changed with a don't care parameter.