DNN Fixed-Point Format Selection for Low-Error Hardware Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware implementations of Deep Neural Networks (DNNs) face challenges in efficiently utilizing resources such as silicon area and processing power, and lack flexibility to support various DNN configurations.
Innovation Solution
A method for selecting fixed point number formats for DNN layers based on their dependencies, minimizing output error by configuring each layer's format in a sequence that considers the impact of previous layers, thereby optimizing hardware implementation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fixed point number formats are selected independently for each layer, then the selection process is simpler and faster, but the output error increases due to not considering inter-layer dependencies
Solution Approach 1:
The patent applies preliminary action by performing a two-stage format selection process: first selecting formats using a fast layer-independent method to establish initial configurations, then refining the selection through a slower layer-dependent method that considers inter-layer dependencies. This preliminary setup enables the system to achieve high precision without fully sacrificing selection speed, as the fast method provides a good starting point for the more accurate but time-consuming dependent selection.
Solution Approach 2:
The patent segments the format selection process into two distinct phases: a layer-independent selection phase that quickly establishes baseline formats, and a layer-dependent refinement phase that optimizes formats considering dependencies. This segmentation allows the system to balance between computational efficiency and accuracy by treating different aspects of the selection process separately rather than requiring full dependency analysis for every format decision.
2Adaptability or versatility
If hardware is designed to support multiple DNN configurations, then flexibility and adaptability improve, but device complexity increases
Solution Approach 1:
The patent applies dynamics by implementing a configurable hardware architecture where data format parameters can be dynamically adjusted based on the specific DNN configuration being deployed. The system uses format description structures that can be programmed at runtime to define custom data formats, allowing the same hardware to adapt to different DNN requirements without physical reconfiguration, thus achieving versatility without proportional increases in hardware complexity.
Solution Approach 2:
The patent changes parameters by allowing dynamic modification of data format specifications (such as integer bit widths, fractional bit positions, and scaling factors) to match different DNN layer requirements. This parameter-based configuration approach enables a single hardware design to support multiple DNN architectures and configurations by simply changing the format parameters rather than redesigning the hardware for each configuration.
3Measurement precision
If fixed point number formats are selected to minimize output error, then accuracy improves, but computational time and resources increase due to considering all layers in sequence
Solution Approach 1:
The patent applies preliminary action by using the layer-independent format selection as a preliminary step that quickly establishes baseline formats before performing the more time-consuming layer-dependent optimization. This preliminary configuration reduces the search space for the subsequent dependent selection, allowing the system to achieve minimal output error without exhaustively analyzing all possible format combinations across all layers.
Solution Approach 2:
The patent segments the format selection into two phases with different optimization goals: the first phase (layer-independent) focuses on speed and establishes reasonable baseline formats, while the second phase (layer-dependent) focuses on accuracy by refining formats based on inter-layer dependencies. This segmentation allows the system to allocate computational resources efficiently, spending more time only where it most impacts output accuracy.
Data Source
AI summary
Methods for selecting fixed point number formats for representing values input to and/or output from layers of a Deep Neural Network (DNN) which take into account the impact of the fixed point number formats for a particular layer in the DNN. The fixed point number format(s) used to represent sets of values input to and/or output from a layer are selected one layer at a time in a predetermined sequence wherein any layer is preceded in the sequence by the layer(s) from which it depends. The fixed point number format(s) for each layer is/are selected based on the error in the output of the DNN associated with the fixed point number formats. Once the fixed point number format(s) for a layer has/have been selected any calculation of the error in the output of the DNN for a subsequent layer in the sequence is based on that layer being configured to use the selected fixed point number formats.