Neural Network Weight Unification for Model Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for compressing Deep Neural Network (DNN) models are inefficient in reducing storage size and computational requirements, particularly for deployment on mobile devices and chips, as they do not effectively unify and optimize weight coefficients for further compression through quantization and entropy coding.
Innovation Solution
A method involving selective structured weight unification, where weight coefficients are unified within superblocks to minimize unification loss, allowing for iterative retraining to maintain original performance and facilitate further compression, applicable to both dense and pruned models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If weight coefficients are pruned to remove unimportant coefficients, then model size is reduced, but compression efficiency is insufficient for further quantization and entropy coding
Solution Approach 1:
The patent applies homogeneity by unifying weight coefficients within superblocks to have similar magnitude values. This is achieved by identifying superblocks where weight coefficients vary significantly in magnitude, then applying unification operations (such as setting coefficients to their mean or median value) to make them more homogeneous. This homogenization enables more effective subsequent quantization and entropy coding, as uniform magnitude distributions are more compressible than heterogeneous distributions.
Solution Approach 2:
The patent segments the weight coefficient tensor into superblocks for selective unification. Instead of uniformly processing all weight coefficients, the method divides them into manageable superblock units, evaluates each superblock's suitability for unification based on magnitude variation metrics, and applies unification operations selectively to specific superblocks. This segmentation enables targeted compression optimization without unnecessarily degrading the performance of important weight coefficients.
2Productivity
If unification operations are applied to weight coefficients, then compression efficiency is improved, but model performance may degrade
Solution Approach 1:
The patent applies local quality by selectively unifying weight coefficients only in specific superblocks where it is beneficial, rather than uniformly across the entire model. The method evaluates each superblock's magnitude variation and identifies those where unification will provide compression benefits without significantly impacting model performance. This localized approach allows different parts of the model to retain their original precision where needed while achieving compression in suitable regions.
Solution Approach 2:
The patent changes parameters by adjusting the unification threshold and superblock size to balance compression efficiency and model performance. The method allows dynamic control over the degree of unification applied, enabling optimization for different deployment scenarios. By adjusting these parameters, the system can achieve higher compression when performance requirements are less stringent, or reduce unification intensity when maintaining model accuracy is prioritized.
3Productivity
If selective structured unification is applied, then quantization and entropy coding efficiency is improved, but computational complexity increases
Solution Approach 1:
The patent segments the weight coefficient processing into discrete superblock units with clear boundaries and structured organization. This segmentation transforms the complex global optimization problem into manageable local operations on individual superblocks. Each superblock can be independently evaluated and processed, reducing the overall computational complexity compared to attempting to optimize all weight coefficients simultaneously. The structured nature of superblocks also facilitates efficient memory access patterns during processing.
Data Source
AI summary
A method, computer program, or computer system is provided for compressing a neural network model. One or more blocks are identified from among a superblock corresponding to a multi-dimensional tensor associated with a neural network. A set of weight coefficients associated with the superblock is unified. A model of the neural network is compressed based on the unified set of weight coefficients.