Neural Network Unit With Circular Shift Register For Parallel Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processor architectures face inefficiencies in performing neural network computations, particularly in deep learning applications, due to limitations in parallel processing and resource utilization, leading to suboptimal performance in tasks like speech and image recognition.
Innovation Solution
The processor incorporates a neural network unit (NNU) with multiple neural processing units (NPUs) that operate in parallel, utilizing a circular shifter mechanism to efficiently perform multiply-accumulate operations and activation functions, allowing for concurrent processing of large datasets with reduced resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional processor architectures are used for neural network computations, then general-purpose processing capability is maintained, but processing efficiency and parallel processing capability are insufficient
Solution Approach 1:
The processor is divided into separate functional units: a general-purpose processor for control and coordination, and a dedicated neural network unit with multiple neural processing units for computational tasks. This segmentation allows each component to be optimized for its specific function, improving overall processing efficiency while maintaining manageable complexity through modular design.
Solution Approach 2:
The neural network unit is designed as a multi-functional component that can perform various neural network operations (convolutions, matrix multiplications, activation functions) through a unified architecture of neural processing units, shift registers, and circular shift mechanisms. This universal design improves processing efficiency for diverse deep learning tasks without requiring separate specialized hardware for each operation type.
2Speed
If parallel processing is implemented for neural network computations, then computation speed is improved, but resource utilization becomes suboptimal
Solution Approach 1:
The neural processing units dynamically shift data between them using circular shift mechanisms, allowing flexible data flow patterns that adapt to different computational requirements. This dynamic data shifting enables efficient parallel processing where resources are actively utilized across multiple units simultaneously, improving both computation speed and resource utilization efficiency.
Solution Approach 2:
The shift registers and circular shift mechanisms enable continuous data flow and computation across neural processing units without idle periods. By maintaining continuous useful action through pipelined operations and overlapping computations, the system maximizes resource utilization while achieving high computation speed, eliminating the trade-off between speed and resource efficiency.
3Productivity
If conventional neural network units are used, then implementation simplicity is maintained, but performance in deep learning tasks is suboptimal
Solution Approach 1:
The neural network unit employs a nested structure where multiple neural processing units are organized within a larger unit that contains shift registers and circular shift mechanisms. This nested architecture allows the complex functionality required for high-performance deep learning to be achieved through hierarchical organization, where simpler components are combined to form more powerful computational structures.
Data Source
AI summary
A neural network unit has a first memory that holds elements of a data matrix and a second memory that holds elements of a convolution kernel. An array of neural processing units (NPU) each have a multiplexed register that receives a corresponding element of a row from the first memory and that also receives the multiplexed register output of an adjacent NPU. A register receives a corresponding element of a row from the second memory. An arithmetic unit receives the outputs of the register, the multiplexed register and an accumulator and performs a multiply-accumulate operation on them. For each sub-matrix of a plurality of sub-matrices of the data matrix, each arithmetic unit selectively receives either the element from the first memory or the adjacent NPU multiplexed register output and performs a series of the multiply-accumulate operations to accumulate into the accumulator a convolution of the sub-matrix with the convolution kernel.