Neural Processing Element With N+1-Bit Operand Extension
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing general-purpose processors, such as GPUs, are unsuitable for long-duration DNN operations in low-power devices due to high computational complexity and power consumption, and existing ASICs do not adequately address the need for reduced power consumption and improved performance in DNN processing.
Innovation Solution
A neural processing element with increased precision calculations is implemented, using hardware AND gates and concatenation circuits to extend operands from N bits to N+1 bits, particularly for operands from DNN layers like ReLU, enabling higher precision computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors like GPUs are used for DNN operations, then computational performance is improved, but power consumption increases
Solution Approach 1:
The processor is divided into multiple processing elements (PEs), each specialized for specific DNN operations. Each PE contains dedicated multiply-accumulate units and activation function units, segmenting the computational tasks to improve efficiency while reducing overall power consumption through targeted optimization.
Solution Approach 2:
The processor dynamically adjusts operational parameters including precision settings (16-bit, 32-bit, 64-bit floating point), activation function selection, and precision control based on input characteristics. This allows the system to optimize the balance between computational performance and power consumption by selecting appropriate precision levels for different DNN layers and operations.
2Measurement precision
If standard precision N-bit operations are used, then device complexity is reduced, but measurement precision deteriorates
Solution Approach 1:
The processor implements dynamic precision control where the precision level (N-bit, N+1-bit, or extended precision) is adjusted based on the specific DNN layer and operation type. The system can switch between different precision modes including 16-bit, 32-bit, and 64-bit floating point operations, allowing high precision where needed while maintaining lower complexity elsewhere in the network.
Solution Approach 2:
The system changes the precision parameter dynamically based on input characteristics and layer requirements. For ReLU layers, the system can utilize all N bits of the operand for increased precision, while for other layers it may use standard N-bit operations, thereby adapting the computational precision to the specific requirements of each operation.
3Measurement precision
If N-bit operations are used for ReLU layer outputs, then device complexity is reduced, but measurement precision deteriorates
Solution Approach 1:
The processor applies different precision handling strategies to different parts of the computation pipeline. For ReLU layer outputs, the system locally increases precision by utilizing all N bits of the operand in subsequent operations, while maintaining standard precision handling for other operations. This localized precision enhancement improves overall DNN accuracy without requiring increased precision throughout the entire system.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Neural processing elements are configured with a hardware AND gate configured to perform a logical AND operation between a sign extend signal and a most significant bit ("MSB") of an operand. The state of the sign extend signal can be based upon a type of a layer of a deep neural network ("DNN") that generate the operand. If the sign extend signal is logical FALSE, no sign extension is performed. If the sign extend signal is logical TRUE, a concatenator concatenates the output of the hardware AND gate and the operand, thereby extending the operand from an N-bit unsigned binary value to an N+1 bit signed binary value. The neural processing element can also include another hardware AND gate and another concatenator for processing another operand similarly. The outputs of the concatenators for both operands are provided to a hardware binary multiplier.