Stacked Memory Tile Layout for Locality-Driven Array Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current high-performance computing and graphics processing systems face memory bandwidth limitations, with a high byte to floating-point operation ratio, leading to increased energy consumption and latency in memory accesses.
Innovation Solution
A stacked memory system is implemented where multiple memory dies are stacked on a processor die, with vertically aligned memory tiles serving as local memory blocks for processing tiles, allowing for partitioning of dense arrays into sub-arrays and storing data in local memory to reduce memory access energy and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional memory systems are used, then memory capacity is sufficient, but memory bandwidth is insufficient leading to high energy consumption and latency
Solution Approach 1:
The patent transitions from a planar memory architecture to a three-dimensional stacked memory architecture, where memory dies are vertically stacked above processor dies. This vertical stacking enables significantly higher memory bandwidth by providing multiple memory channels in the vertical dimension, directly addressing the bandwidth limitation while reducing energy consumption through shorter access paths.
Solution Approach 2:
The patent divides the memory system into multiple independent memory dies, each stacked above a corresponding processor die. Each memory die can be independently accessed, enabling parallel memory operations and significantly increasing overall memory bandwidth. This segmentation also reduces energy consumption by allowing localized access to specific memory regions without accessing the entire memory space.
2Loss of time
If memory is accessed from distant locations, then memory capacity is accessible, but access latency increases
Solution Approach 1:
The patent implements a localized memory architecture where each processor die has dedicated memory dies stacked directly above it. This creates local memory pools that can be accessed with minimal latency, as the memory is physically adjacent to the processing units. The local quality principle is applied by optimizing the memory-access path for each processor, eliminating the need for long-distance memory accesses across the entire system.
Solution Approach 2:
By stacking memory dies vertically above processor dies, the patent creates a three-dimensional memory hierarchy that reduces access latency. The vertical stacking enables direct coupling between processor and memory through short interconnects, dramatically reducing access time compared to traditional planar architectures where memory is located far from processing units.
3Quantity of substance
If data is stored in remote memory, then total memory capacity is utilized, but energy per bit transfer increases
Solution Approach 1:
The patent implements local memory pools stacked directly above each processor die, enabling data to be stored and accessed locally. This dramatically reduces the energy per bit transfer by eliminating long-distance data movement across the system. Each processor accesses its locally attached memory, minimizing transfer distance and energy consumption while still providing access to large total memory capacity through the stacked architecture.
Data Source
AI summary
Embodiments of the present disclosure relate to application partitioning for locality in a stacked memory system. In an embodiment, one or more memory dies are stacked on the processor die. The processor die includes multiple processing tiles and each memory die includes multiple memory tiles. Vertically aligned memory tiles are directly coupled to and comprise the local memory block for a corresponding processing tile. An application program that operates on dense multi-dimensional arrays (matrices) may partition the dense arrays into sub-arrays associated with program tiles. Each program tile is executed by a processing tile using the processing tile's local memory block to process the associated sub-array. Data associated with each sub-array is stored in a local memory block and the processing tile corresponding to the local memory block executes the program tile to process the sub-array data.