Bit-Level Pruning for Neural Network Power Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning neural networks face challenges in power and storage efficiency due to high calculation requirements, particularly in mobile devices, where lower bit-width quantization and higher sparsity pruning lead to decreased inference accuracy.
Innovation Solution
The approach involves bit-level pruning of neural network weights and activations, using a bit-pruning unit to dynamically prune activation data during inference, reducing bit-level switching and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If lower bit-width quantization and higher-sparsity word-level pruning are applied, then power consumption and storage requirements are reduced, but inference accuracy deteriorates
Solution Approach 1:
The patent segments the quantization and pruning process into bit-level operations rather than word-level operations. By processing individual bits separately and applying sparsity at the bit level within each weight element, the system achieves higher sparsity without the accuracy loss associated with traditional word-level pruning. This segmentation allows selective zeroing of specific bits while preserving the overall weight structure.
Solution Approach 2:
The patent applies different treatment to different bits within each weight element. Instead of uniformly pruning entire weight elements (word-level), the method selectively prunes individual bits based on their contribution to inference accuracy. This local quality approach allows the system to maintain accuracy for bits that matter while eliminating redundant bits, thereby reducing power consumption without sacrificing overall model performance.
2Quantity of substance
If lower bit-width quantization and higher-sparsity pruning are applied, then storage requirements are reduced, but inference accuracy deteriorates
Solution Approach 1:
The patent segments storage representation into individual bit levels rather than storing complete weight elements. By representing weights as collections of bit planes and storing only the significant bit patterns, the system achieves compact representation. The segmentation allows the model to store fewer bits per weight element while maintaining the essential information needed for accurate inference.
Solution Approach 2:
The patent extracts and removes redundant bit information from weight representations. By analyzing which bits contribute most to inference accuracy and eliminating the less significant bits through bit-level sparsity, the system reduces storage requirements. This extraction process selectively removes unnecessary data while preserving the core functional weights, thereby achieving compression without proportional accuracy loss.
3Loss of energy
If bit-level pruning is applied to reduce bit densities, then power efficiency is improved, but computational complexity increases
Solution Approach 1:
The patent employs periodic patterns in bit-level pruning where certain bit positions are systematically pruned based on their statistical importance. This periodic action creates regular computation patterns that can be efficiently implemented in hardware, reducing the overhead of complex bit-level operations. The periodic structure allows the system to achieve power efficiency through reduced bit switching while maintaining manageable computational complexity through pattern repetition.
Solution Approach 2:
The patent changes the parameter of sparsity from the traditional word level to the bit level, fundamentally altering how pruning is applied. This parameter change enables finer-grained control over the computation-reduction tradeoff. By adjusting the bit-level sparsity parameter, the system can optimize between power efficiency and computational complexity, achieving reduced power consumption through targeted bit elimination without requiring complete restructuring of the computational graph.
Data Source
AI summary
A method, system and apparatus provide bit-sparse neural network optimization. Rather than quantizing and pruning weight and activation elements at the word level, weight and activation elements are pruned at the bit level, which reduces the density of effective “set” bits in weight and activation data, which, advantageously, reduces the power consumption of the neural network inference process by reducing the degree of bit-level switching during inference.