Neural Network Unit Memory Array Configurable as Cache
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hardware implementations for neural networks face inefficiencies in computation performance and resource utilization, particularly in handling large numbers of connection inputs, which limits their ability to process neural network layers efficiently.
Innovation Solution
The design incorporates a neural network unit (NNU) with multiple neural processing units (NPUs) and a dynamically configurable architecture that allows for efficient multiply-accumulate operations and activation functions, utilizing a rotater mechanism to process data and weights in a pipelined fashion, reducing the need for extensive multipliers and adders and enabling high performance across varying network layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional hardware implementations use extensive multipliers and adders to handle large numbers of connection inputs, then computation performance improves, but device complexity and resource requirements increase
Solution Approach 1:
The patent implements a dynamically configurable NNU architecture where the number of active NPUs can be adjusted based on the specific neural network layer being processed. This allows the system to adapt resource allocation to match computational needs, avoiding the waste of static over-provisioning while maintaining high performance when needed.
Solution Approach 2:
The NNU architecture uses multiplexers to share data and weight inputs across multiple NPUs, allowing a single set of input lines to serve multiple processing units. This reduces the overall resource requirements while maintaining the ability to process large numbers of connection inputs through time-multiplexed access.
2Productivity
If the architecture is optimized for specific network layers, then processing efficiency for those layers improves, but adaptability to varying network layers decreases
Solution Approach 1:
The control logic dynamically configures the NNU architecture based on the specific neural network layer being processed. By adjusting the number of active NPUs and their configuration according to the layer's requirements, the system achieves high processing efficiency for each specific layer while maintaining overall versatility across different network architectures.
Solution Approach 2:
The NNU is divided into multiple independent NPUs that can be selectively activated. This segmentation allows the system to configure the optimal number of processing units for each specific network layer, achieving both specialized optimization and general adaptability.
3Productivity
If more NPUs are activated to process larger neural network layers, then computation performance improves, but resource utilization efficiency worsens when processing smaller layers
Solution Approach 1:
The system dynamically activates only the number of NPUs needed for the current processing task. When processing smaller neural network layers, fewer NPUs are activated, reducing resource consumption and improving utilization efficiency. When larger layers are processed, more NPUs are activated to maintain high computation performance.
Data Source
Figure 1
Figure 2~3
Figure 4~6A
AI summary
A processor comprising a mode indicator, a plurality of processing cores, and a neural network unit (NNU), comprising a memory array, an array of neural processing units (NPU), cache control logic, and selection logic that selectively couples the plurality of NPUs and the cache control logic to the memory array. When the mode indicator indicates a first mode, the selection logic enables the plurality of NPUs to read neural network weights from the memory array to perform computations using the weights. When the mode indicator indicates a second mode, the selection logic enables the plurality of processing cores to access the memory array through the cache control logic as a cache memory.