Neural Network Unit Rotater Mechanism Reduces Multiplier Count
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processor architectures face inefficiencies in executing neural network operations due to the high number of multipliers and adders required for neural network computations, leading to increased resource utilization and performance bottlenecks in processing large numbers of connection inputs.
Innovation Solution
The processor incorporates a neural network unit (NNU) with a dynamically configurable architecture, featuring multiple neural processing units (NPUs) that perform multiply-accumulate operations in a pipelined fashion, utilizing a rotater mechanism to efficiently process data and weights, reducing the need for extensive multipliers and adders by rotating and accumulating products across clock cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional processor architectures are used for neural network computations, then the system can execute general-purpose instructions, but the resource utilization increases significantly due to the high number of multipliers and adders required
Solution Approach 1:
The processor is divided into a general-purpose processing unit and a dedicated neural network unit (NNU) with multiple neural processing units (NPUs). Each NPU is further segmented into functional components including multipliers, adders, and output registers organized in parallel arrays. This segmentation allows neural network computations to be handled by specialized sub-units, reducing the burden on the general-purpose processor and minimizing the total number of required computational elements.
Solution Approach 2:
The neural network unit is designed with multi-functional NPUs that can perform various neural network operations including multiply-accumulate operations, data rotation, and activation function computations. The NPUs can be dynamically configured to handle different neural network layers and operations, making the hardware versatile for various deep learning workloads while maintaining efficient resource utilization.
2Measurement precision
If extensive multipliers and adders are used for neural network operations, then computation accuracy is maintained, but performance bottlenecks occur when processing large numbers of connection inputs
Solution Approach 1:
The neural network unit implements dynamic data rotation through the rotater mechanism, which rotates data between NPUs based on the distance parameter. This dynamic rotation allows the system to efficiently process different neural network operations by reconfiguring data flow patterns, maintaining computation accuracy while adapting to different computational requirements and improving overall processing throughput.
Solution Approach 2:
The pipelined architecture of the NPUs enables continuous processing of neural network operations. Data flows continuously through the pipeline of multipliers, adders, and rotation units, eliminating idle cycles and maintaining high utilization of computational resources. This continuous action allows the system to process large numbers of connection inputs without performance bottlenecks while preserving computation accuracy.
3Ease of manufacture
If a fixed neural network unit architecture is used, then hardware implementation is simplified, but the unit cannot efficiently adapt to different neural network layers and operations
Solution Approach 1:
The neural network unit incorporates dynamically configurable NPUs that can adjust their operation mode based on the specific neural network layer being processed. The rotater mechanism can be configured with different distance parameters, and the data flow paths can be dynamically reconfigured to match the requirements of convolutional layers, fully connected layers, or other neural network operations, providing versatility while maintaining a relatively simple hardware structure.
Solution Approach 2:
The system changes operational parameters such as the rotation distance in the rotater mechanism and the configuration of data flow paths to adapt to different neural network operations. By modifying these parameters rather than changing the fundamental hardware architecture, the system achieves high adaptability to different neural network layers while keeping the hardware implementation relatively simple and manufacturable.
Data Source
Figure 1
Figure 2~3
Figure 4~6A
AI summary
N processing units (PU) each have an arithmetic unit (AU) that performs an operation on first, second and third inputs to generate a result to store in an accumulator having an output provided to the first input. A weight input is received by the AU second input. A multiplexed register has first, second, third and fourth data inputs and an output received by the third AU input. A first memory provides N weight words to the N weight inputs. A second memory provides N data words to the multiplexed register first data inputs. The multiplexed register output is also received by the second, third, and fourth data input of the multiplexed register one, 2^J, and 2^K PUs away, respectively. The N multiplexed registers collectively operate as an N-word rotater that rotates by one, 2^J, or 2^K words when the control input specifies the second, third, or fourth data input, respectively.