Neural Network Unit Memory Array Mode Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processor architectures face inefficiencies in performing neural network computations due to the need for extensive multipliers and adders, which limits performance and flexibility, especially in handling a large number of connection inputs in artificial neural networks.
Innovation Solution
The processor incorporates a neural network unit (NNU) with multiple neural processing units (NPUs) that operate as neurons, performing a multiply-accumulate function efficiently by rotating data and weights across NPUs, reducing the need for extensive multipliers and adders, and allowing for flexible configuration to handle varying numbers of connection inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional processor architectures are used for neural network computations, then general-purpose computation is maintained, but processing efficiency and performance for neural network operations deteriorate due to extensive multipliers and adders requirements
Solution Approach 1:
The processor is segmented into distinct functional units: a general-purpose processor and a specialized neural network unit (NNU). The NNU is further divided into multiple neural processing units (NPUs) that can be independently configured. This segmentation allows neural network computations to be handled by the optimized NNU while general-purpose tasks remain with the main processor, resolving the contradiction between general-purpose capability and neural network efficiency.
Solution Approach 2:
The NNU employs configurable parameters including the number of NPUs, connection inputs per NPU, and data word/weight word bit depths. These parameters can be adjusted to match specific neural network layer requirements. The ability to change these parameters dynamically allows the system to optimize for different computational workloads, improving processing efficiency without being locked into a fixed complex architecture.
2Adaptability or versatility
If the number of connection inputs per NPU is increased to handle larger neural networks, then versatility improves, but device complexity and resource requirements increase
Solution Approach 1:
The NNU architecture is designed to be dynamically reconfigurable. The number of NPUs and connection inputs per NPU can be adjusted based on the specific neural network layer being processed. This dynamic configuration allows the same hardware to adapt to different network sizes and complexities without requiring a fixed, overly complex design that must accommodate all possible scenarios.
Solution Approach 2:
Each NPU is designed as a universal computing element that can handle multiple functions: multiplying data words by weight words, accumulating results, and supporting different bit depths. This universal design allows a single NPU configuration to serve multiple purposes across different neural network layers, improving versatility without proportionally increasing complexity.
3Measurement precision
If extensive multipliers and adders are used to handle large numbers of connection inputs, then computational accuracy is maintained, but processing speed and efficiency deteriorate
Solution Approach 1:
The NPU merges the multiplier and adder functions into a single integrated computational unit. Instead of having separate multiplier and adder circuits that operate sequentially or require multiple clock cycles, the merged unit performs both operations in a unified manner, improving processing speed while maintaining the full precision of both multiplication and addition operations.
Solution Approach 2:
The NNU is designed to maintain continuous computational flow through pipelined operations and overlapping data transfers. While one NPU is performing multiplication, another can be accumulating results, and data can be being prepared for the next operation. This continuity eliminates idle time and maintains high processing speed without sacrificing computational accuracy through rushed or simplified operations.
Data Source
Figure 1
Figure 2~3
Figure 4~6A
AI summary
A processor comprises a neural network unit (NNU) and a processing complex (PC) comprising a processing core and cache memory. The NNU comprises neural processing units (NPU), cache control logic (CCL) and a memory array (MA). To transition from a first mode in which the MA operates to hold neural network weights for the array of NPUs to a second mode in which the MA and CCL operate as a victim cache, the CCL begins to cache evicted cache lines into the MA in response to eviction requests and begins to provide to the PC lines that hit in the MA in response to load requests. To transition from the second mode to the first mode, the CCL invalidates all lines of the MA, ceases to cache evicted lines into the MA in response to eviction requests, and ceases to provide to the PC lines in response to load requests.