Programmable Lookup Tables for Adaptive DNN Activation Functions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural networks (DNNs) face challenges in efficient implementation due to limited power resources and silicon area, particularly when their definition changes over time, necessitating a flexible and area-efficient system.
Innovation Solution
A method for configuring hardware with a programmable lookup table to approximate activation functions, allowing multiple activation functions to be performed efficiently by storing lookup data and switching between sets concurrently, with interpolation and extrapolation capabilities to handle input values within or outside the defined range.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If a fixed lookup table is used for activation functions, then hardware area is reduced, but adaptability to changing DNN definitions is lost
Solution Approach 1:
The patent implements a dynamically reconfigurable lookup table that can be programmed with different activation function data at runtime. The system monitors input data streams to determine the actual range of values encountered, then configures the lookup table accordingly. This allows the same hardware to adapt to different DNN definitions and activation functions without requiring a fixed, oversized table, thus resolving the contradiction between area efficiency and adaptability.
2Measurement precision
If a lookup table covers the full possible input range, then accuracy is maintained, but silicon area increases
Solution Approach 1:
The patent applies local quality by concentrating lookup table resources only where needed. Instead of uniformly covering the entire possible input range, the system identifies the actual local range of input values through monitoring and configures the lookup table to cover only that specific region with high precision. This localized approach maintains accuracy for the actual operating conditions while minimizing silicon area by not allocating resources to unused input ranges.
3Adaptability or versatility
If multiple activation functions are implemented with separate lookup tables, then functional versatility is improved, but hardware area and power consumption increase
Solution Approach 1:
The patent implements a universal lookup table that can serve multiple activation functions through dynamic reconfiguration. A single lookup table is programmed with activation function data corresponding to the currently needed function, determined by monitoring the input data stream. This multi-functional approach allows the hardware to support multiple activation functions sequentially, eliminating the need for multiple separate lookup tables and thus reducing both hardware area and power consumption while maintaining versatility.
Data Source
Figure 1~3
Figure 2
Figure 4a~4c
AI summary
A method for configuring hardware for implementing a Deep Neural Network (DNN) for performing an activation function, the hardware comprising, at an activation module for performing an activation function, a programmable lookup table for storing lookup data approximating the activation function over a first range of input values to the activation module, the method comprising: providing calibration data to a representation of the hardware; monitoring an input to an activation module of the representation of the hardware so as to determine a range of input values to the activation module; generating lookup data for the lookup table representing the activation function over the determined range of input values; and loading the generated lookup data into the lookup table of the hardware, thereby configuring the activation module of the hardware for performing the activation function over the determined range of input values.