3D Neural-Network ASIC Memory Switching for Dynamic Bandwidth Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-core computer processors face challenges in efficiently distributing memory bandwidth and capacity among processing units, leading to inefficiencies in computation and power usage.
Innovation Solution
A three-dimensional application-specific integrated circuit (ASIC) with a switch matrix that allows each processing unit to communicate with proximate and remote memory banks via short and longer inter-die memory channels, enabling flexible allocation of memory speed and quantity among processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If memory bandwidth and capacity are distributed among processing units in traditional multi-core architectures, then each processing unit has access to its own memory resources, but memory access speed and overall computation efficiency are limited
Solution Approach 1:
The patent merges memory resources from multiple processing units into a shared pool that can be dynamically allocated. Instead of each processing unit having dedicated memory, the system consolidates memory banks and allows any processing unit to access any memory bank through a switching network, thereby increasing memory access speed without proportionally increasing physical memory capacity per unit.
Solution Approach 2:
The patent implements dynamic memory allocation where the mapping between processing units and memory banks is not fixed but can be reconfigured at runtime. A control mechanism dynamically routes memory access requests through a switching network, allowing the system to adapt memory bandwidth and capacity distribution based on current computational needs, thus improving overall memory access efficiency.
2Use of energy by moving object
If processing units are evenly distributed across the chip area in tiled architectures, then computation and power usage are evenly distributed avoiding hot spots, but memory bandwidth allocation becomes inefficient
Solution Approach 1:
The patent maintains the even spatial distribution of processing units for balanced power consumption but introduces dynamic routing through a switching network. This allows the system to flexibly allocate memory bandwidth to different processing units based on their current computational workload, thereby improving computation efficiency without creating power hot spots, as the physical distribution remains balanced while logical resource allocation becomes dynamic.
Solution Approach 2:
The patent creates a universal memory access architecture where any processing unit can access any memory bank through the switching network. This multi-functional routing capability allows the same physical memory infrastructure to serve different computational workloads efficiently, improving overall productivity while maintaining the balanced power distribution characteristics of the tiled architecture.
3Ease of operation
If each processing unit has dedicated memory bandwidth and capacity, then memory access is straightforward, but the system cannot efficiently handle varying computational demands of different applications
Solution Approach 1:
The patent introduces a switching network as an intermediary between processing units and memory banks. This intermediary component maintains the simplicity of memory access from the processing unit perspective while providing the flexibility to dynamically route requests. The switching network mediates between the simple access patterns desired by processing units and the complex dynamic allocation needs of varying applications, preserving ease of operation while enabling adaptability.
Data Source
AI summary
An application-specific integrated circuit for an artificial neural network is integrated with a high-bandwidth memory. A processing die with tiled neural-network processing units is bonded to a stack of memory dies with memory banks laid out to establish relatively short connections to overlying processing units. The memory banks form vertical groups of banks for each overlying processing unit. A switch matrix on the processing die allows each processing unit to communicate with its vertical group of banks via a short, fast inter-die memory channel or with more remote groups of banks under neighboring processing units.