Multi-Mode Neural Processing Unit for Activation Approximation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processing units (NPUs) face challenges in efficiently processing complex activation functions, leading to increased gate count, power consumption, and inference accuracy deterioration due to hard-wired processors' limitations and conventional approximation methods.
Innovation Solution
A neural processing unit (NPU) with a controller and converter circuit that selects modes based on activation function error rates, using a programmed activation function (PAF) to generate and convert activation values, allowing flexible and efficient approximation of activation functions through programmable segments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional approximation methods are used for activation functions, then device complexity is reduced, but inference accuracy deteriorates
Solution Approach 1:
The system dynamically selects between first mode (PAF approximation) and second mode (converter circuit) based on the characteristics of the activation function being processed. This dynamic mode selection allows the NPU to adaptively balance between computational efficiency and accuracy requirements for different activation functions.
Solution Approach 2:
The system changes the processing parameter (mode of operation) based on the error rate threshold of the activation function. When the error rate is below the threshold, first mode is used; otherwise, second mode is used. This parameter-based selection optimizes the trade-off between complexity and accuracy.
2Speed
If hard-wired processors are used for activation functions, then processing speed is improved, but adaptability deteriorates
Solution Approach 1:
The NPU is designed with multi-functionality to handle both PAF-based activation functions and converter circuit-based activation functions within a single processing architecture. This universal design allows the system to process different types of activation functions without requiring separate dedicated hardware for each type.
Solution Approach 2:
The system dynamically switches between processing modes depending on the activation function requirements, enabling the hard-wired processor to maintain high speed while adapting to different activation function types through configurable mode selection.
3Loss of energy
If conventional approximation methods are used, then power consumption is reduced, but manufacturing precision deteriorates
Solution Approach 1:
The system changes the processing mode parameter based on the precision requirements of different activation functions. For activation functions with low error rates, the energy-efficient first mode is used; for those requiring higher precision, the second mode is activated, optimizing the energy-accuracy trade-off.
Solution Approach 2:
The system uses error rate analysis as feedback to determine the appropriate processing mode. By evaluating the approximation error characteristics of each activation function, the system provides feedback-based mode selection that ensures adequate precision while minimizing power consumption.
Data Source
AI summary
A neural processing unit may be provided. The neural processing unit may comprise a controller circuit configured to select an activation function processing method among a first method or a second method, according to an activation function included in a neural network model, a programmed activation function execution unit (PAFE unit) configured to execute a programmed activation function (PAF) that approximate the activation function and output a first activation value, and a converter circuit configured to convert the first activation value and output a second activation value. In the first method, only the PAFE unit may operate. In the second method, both the PAFE unit and the converter may operate.