Shared SRAM Architecture for Reconfigurable NPU Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processing unit (NPU) configurations face challenges in achieving optimal utilization due to the overhead associated with reconfiguration, particularly with segmented SRAMs, leading to inefficient usage and data transfer between memories.
Innovation Solution
A reconfigurable N×N array of NPUs is arranged in rows and columns, with each NPU having a memory and a convolutional multiply-accumulate circuit, processing input feature map data by storing along diagonals, broadcasting, multiplying, and adding values diagonally to form output feature maps, which are stored in corresponding locations without inter-memory data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If segmented SRAMs are used to save capacity, then SRAM usage is reduced, but data transfer overhead between memories increases
Solution Approach 1:
The patent merges segmented SRAMs into a unified shared SRAM pool that serves multiple NPU cores. Instead of each core having separate segmented SRAM, the system consolidates memory resources so that all NPUs can access the same SRAM space, eliminating the need for inter-memory data transfers when reshaping occurs.
Solution Approach 2:
The shared SRAM is designed to serve multiple functions and multiple NPU cores simultaneously. The same SRAM space is universally accessible by any NPU in the array, allowing dynamic resource allocation and eliminating dedicated memory segments for each core configuration.
2Productivity
If NPU configuration is changed to optimize for different network sizes, then utilization efficiency is improved, but reconfiguration overhead increases
Solution Approach 1:
The NPU array is designed with dynamic reconfigurability, allowing the system to change its operational configuration based on the specific neural network being processed. The array can dynamically adjust between different tile configurations (e.g., 8×8, 32×32, 64×64) without requiring physical reconfiguration, as the shared SRAM and control logic enable software-defined configuration changes.
Solution Approach 2:
The system changes operational parameters (array dimensions, tile sizes, memory allocation) rather than physical structure. By modifying control parameters and data flow patterns, the same hardware can optimize for different network architectures without physical reconfiguration overhead.
3Device complexity
If a single NPU configuration is used, then device complexity is reduced, but utilization efficiency deteriorates for different network sizes
Solution Approach 1:
A single NPU array design serves multiple functions by supporting different configuration modes. The same physical array can operate in 8×8 mode for MobileNetV2, 32×32 mode for Inception V3, or 64×64 mode for ResNet-50, making the hardware universally applicable to various network architectures without requiring multiple specialized designs.
Solution Approach 2:
The system transitions from static single-configuration designs to dynamic multi-configuration capability. The NPU array can adapt its operational characteristics in real-time based on workload requirements, maintaining low physical complexity while achieving high utilization efficiency across diverse workloads.
Data Source
AI summary
A system and a method is disclosed for processing input feature map (IFM) data of a current layer of a neural network model using an array of reconfigurable neural processing units (NPUs) and storing output feature map (OFM) data of the next layer of the neural network model at a location that does not involve a data transfer between memories of the NPUs according to the subject matter disclosed herein. The reconfigurable NPUs may be used to improve NPU utilization of NPUs of a neural processing system.