Neural Network Unit Rotater for High-Throughput Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial neural network (ANN) processors face inefficiencies in performing multiply-accumulate operations and activation functions, leading to suboptimal performance and resource utilization, especially in handling large numbers of connection inputs.
Innovation Solution
The processor design incorporates a neural network unit (NNU) with multiple neural processing units (NPUs) that operate as neurons, performing multiply-accumulate operations in a pipelined fashion and utilizing a rotater mechanism to efficiently process data and weight words, allowing for parallel execution of operations across multiple clock cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional processors are used to perform multiply-accumulate operations for neural networks, then general-purpose computation is achieved, but performance and efficiency for neural network computations are suboptimal
Solution Approach 1:
The processor is segmented into a general-purpose processing unit and a dedicated neural network unit (NNU). The NNU is further divided into multiple neural processing elements (NPEs) that can be independently configured. This segmentation allows neural network computations to be handled by specialized hardware while general-purpose tasks use the main processor, resolving the contradiction between optimization for specific tasks and architectural simplicity.
Solution Approach 2:
The neural network unit is designed with reconfigurable neural processing elements that can be dynamically configured to perform different neural network operations (convolution, pooling, fully connected layers) and support various neural network architectures. This multi-functionality allows a single dedicated unit to handle diverse neural network workloads, improving productivity without requiring separate specialized hardware for each operation type.
2Productivity
If the number of neural processing units is increased to handle large neural network layers, then processing capacity is improved, but resource utilization and efficiency deteriorate
Solution Approach 1:
The neural processing elements within the NNU are designed to be dynamically reconfigurable. The number and configuration of active NPEs can be adjusted based on the specific neural network layer being processed. This dynamic adaptation allows the system to scale processing capacity up or down as needed, maintaining high resource utilization efficiency while handling varying sizes of neural network layers without permanent over-provisioning of resources.
3Productivity
If dedicated hardware is designed for specific neural network operations, then performance is improved, but adaptability to different neural network architectures deteriorates
Solution Approach 1:
The neural processing elements incorporate reconfigurable parameters including the number of input connections per neuron, the number of neurons per layer, and the operational mode (convolution, pooling, fully connected). These parameters can be programmatically adjusted to match the specific requirements of different neural network architectures and operations, allowing the dedicated hardware to maintain high performance across diverse workloads without sacrificing adaptability.
Data Source
Figure 1
Figure 2~3
Figure 4~6A
AI summary
An array of N processing units (PU) each has: an accumulator; an arithmetic unit performs an operation on first, second and third inputs to generate a result to store in the accumulator, the first input receives the accumulator output; a weight input is received by the second input to the arithmetic unit; a multiplexed register has first and second data inputs, an output received by the third input to the arithmetic unit, and a control input that controls the data input selection. The multiplexed register output is also received by an adjacent PU's multiplexed register second data input. The N PU's multiplexed registers collectively operate as an N-word rotater when the control input specifies the second data input. Respective first/second memories hold W/D rows of N weight/data words and provide the N weight/data words to the corresponding weight/ multiplexed register first data inputs of the N PUs.