Dynamic Fixed-Point Mapping for Low-Power DNN Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face challenges in low-power, low-latency implementations due to high precision requirements, leading to high power consumption and accuracy degradation when reduced to lower precision formats like binary or ternary, which are not suitable for embedded systems and large datasets.
Innovation Solution
Mapping floating-point DNNs to 8-bit dynamic fixed-point networks with integer power-of-two weights, allowing for multiplier-free hardware accelerators and using student-teacher learning to fine-tune networks, enabling low-power, low-latency inference with minimal accuracy loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point precision is used in DNNs, then classification accuracy is maintained, but power consumption and computational complexity increase
Solution Approach 1:
The patent changes the precision parameter from standard floating-point to dynamic fixed-point representation, allowing the system to adapt precision requirements dynamically. This parameter change enables lower power consumption while maintaining classification accuracy through optimized bit-width allocation for different network layers and operations.
Solution Approach 2:
The patent implements dynamic precision adjustment where the fixed-point format parameters (bit-width, scaling factors) are dynamically adapted based on the specific layer, operation type, and input data characteristics. This dynamic approach allows the system to use higher precision only where necessary for accuracy while using lower precision elsewhere to reduce power consumption.
2Use of energy by moving object
If precision is reduced to binary or ternary formats, then power consumption decreases, but accuracy degradation occurs and suitability for embedded systems is lost
Solution Approach 1:
The patent changes from fixed low-precision formats (binary/ternary) to dynamic fixed-point formats where the precision parameters can be adjusted. This allows the system to achieve lower power consumption than full floating-point while avoiding the severe accuracy degradation of binary/ternary formats, finding an optimal middle ground through parameter optimization.
Solution Approach 2:
The patent applies partial precision reduction rather than complete reduction to binary/ternary. By selectively applying fixed-point conversion only to specific layers and operations where it is most beneficial, the system achieves significant power savings while maintaining sufficient accuracy for the application.
3Device complexity
If standard fixed-point conversion is applied, then hardware implementation becomes simpler, but accuracy is lost due to quantization errors
Solution Approach 1:
The patent optimizes the fixed-point parameters (integer bits, fractional bits, scaling factors) specifically for neural network operations rather than using generic fixed-point conversion. This parameter optimization minimizes quantization errors while maintaining hardware simplicity, achieving better accuracy than standard fixed-point conversion.
Solution Approach 2:
The patent introduces intermediate representations and transformation layers that bridge the gap between floating-point training models and fixed-point inference. These intermediaries include quantization-aware training modules and calibration layers that adapt the model to fixed-point arithmetic, reducing accuracy loss during the conversion process.
Data Source
AI summary
A hardware neural network system includes an input buffer for input neurons (Nbin), an output buffer for output neurons (Nbout), and a third buffer for synaptic weights (SB) connected to a Neural Functional Unit (NFU) and a control logic (CP) for performing synapses and neurons computations. The NFU pipelines a computation into stages, the stages including weight blocks (WB), an adder tree, and a non-linearity function.