Differentiable Quantized DNNs for Flexible Energy-Accuracy Trade-offs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning models for vision, speech, and natural language applications on mobile devices face high latencies and energy consumption due to computational costs and memory accesses, with previous model compression techniques like LightNN lacking flexibility in energy-accuracy trade-offs, resulting in discrete Pareto fronts.
Innovation Solution
Introducing a differentiable training algorithm that allows each convolutional filter to use a unique number of shift-and-add operations for approximation, enabling end-to-end optimization and gradual quantization to achieve a continuous Pareto front, with trainable thresholds and regularization to encourage sparsity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single k value is used across the whole network in LightNN approaches, then hardware implementation becomes simpler and faster, but the energy-accuracy trade-off becomes discrete and lacks flexibility
Solution Approach 1:
The patent divides the network into multiple segments (convolutional filters), each with its own independent k value. This segmentation allows different parts of the network to use different numbers of shift operations, enabling fine-grained control over the energy-accuracy trade-off for each filter while maintaining overall system functionality.
Solution Approach 2:
The patent introduces dynamic k values that can be adjusted during training and inference. By making k values variable rather than fixed, the system can adaptively optimize the balance between energy consumption and accuracy for different filters and different operating conditions, transforming a static discrete trade-off into a dynamic continuous one.
2Measurement precision
If more shift-and-add operations are used per multiplication, then accuracy improves, but energy consumption and latency increase
Solution Approach 1:
The patent applies local quality by allowing different k values (different numbers of shift operations) for different convolutional filters based on their specific requirements. Filters that benefit more from higher precision can use larger k values, while less critical filters use smaller k values, optimizing the overall accuracy-energy trade-off locally for each filter.
Solution Approach 2:
The patent changes the parameter k from a fixed global value to variable filter-specific values that can be continuously adjusted. This parameter change enables the system to fine-tune the accuracy-energy trade-off by modifying the number of shift operations for each filter, achieving a continuous Pareto front rather than discrete points.
3Use of energy by moving object
If model size is reduced through quantization, then energy consumption decreases, but accuracy may be compromised
Solution Approach 1:
The patent applies partial action by using different degrees of quantization (different k values) for different filters. Instead of uniformly applying aggressive quantization across the entire network, the system selectively applies higher precision (larger k) only where necessary to maintain accuracy, while using lower precision (smaller k) elsewhere to reduce energy consumption.
Data Source
AI summary
To improve the throughput and energy efficiency of Deep Neural Networks (DNNs) on customized hardware, lightweight neural networks constrain the weights of DNNs to be a limited combination of powers of 2. In such networks, the multiply-accumulate operation can be replaced with a single shift operation, or two shifts and an add operation. To provide even more design flexibility, the k for each convolutional filter can be optimally chosen instead of being fixed for every filter. The present invention formulates the selection of k to be differentiable and describes model training for determining k-based weights on a per-filter basis. The present invention can achieve higher speeds as compared to lightweight NNs with only minimal accuracy degradation, while also achieving higher computational energy efficiency for ASIC implementation.