Stacked ML Accelerator Dies With Reconfigurable Shared Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The disparity between processor speed improvements and memory bandwidth speed improvements has not been adequately addressed, leading to inefficiencies in processing operations.
Innovation Solution
A stacked die configuration is implemented, where a machine learning die with memory and accelerators is stacked with a processing core die, allowing the memory to be configured as either a cache or directly accessible memory, enhancing data transfer and processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If processor speed is improved, then processing capability is enhanced, but memory bandwidth speed does not improve proportionally, creating a performance bottleneck
Solution Approach 1:
The patent transitions from a traditional single-die horizontal architecture to a stacked 3D architecture with multiple dies vertically arranged. This dimensional change enables direct inter-die communication pathways that bypass traditional memory bottlenecks, allowing processing cores on one die to directly access memory on another die through vertical interconnects, thereby resolving the processor speed-memory bandwidth mismatch
Solution Approach 2:
The system is divided into separate functional dies: processing cores die and memory die, each optimized for its specific function. This segmentation allows independent optimization of processing speed and memory bandwidth, with each die specialized for its respective task while maintaining high-speed communication through direct vertical interconnects
2Speed
If memory is configured as cache, then processing speed is improved through faster data access, but memory capacity for direct access is reduced
Solution Approach 1:
The memory system implements dynamic reconfiguration capability, allowing the memory to switch between cache mode and direct access mode based on computational needs. This dynamic flexibility enables the same memory resources to serve dual purposes: providing fast cache access when needed for processing speed, or expanding available memory capacity for direct access operations, thus resolving the trade-off between speed and capacity
Data Source
AI summary
A device is disclosed. The device includes a machine learning die including a memory and one or more machine learning accelerators; and a processing core die stacked with the machine learning die, the processing core die being configured to execute shader programs for controlling operations on the machine learning die, wherein the memory is configurable as either or both of a cache and a directly accessible memory.