3D Stacked HMC Cache Coherence for Balanced Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-processor systems with shared memory face challenges in memory access latency and bandwidth imbalance due to inefficient cache coherence protocols, which consume system bandwidth and result in uneven latency.
Innovation Solution
Implementing a Hybrid Memory Cube (HMC) with a three-dimensional stacked architecture and a memory coherence protocol that stores cache coherence directories within HMC storage vaults, allowing for atomic memory operations and unified management of memory and cache coherency traffic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a cache coherence protocol is implemented in multi-processor shared memory systems, then data correctness across processors is ensured, but system bandwidth is consumed and latency becomes uneven
Solution Approach 1:
The patent extracts the cache coherence directory from the traditional memory controller and places it within the HMC storage vaults themselves. This allows coherence management to be distributed across the memory subsystem, reducing the bandwidth burden on the system interconnect while maintaining data correctness through localized directory updates within the HMC architecture.
Solution Approach 2:
The patent transitions from a two-dimensional memory architecture to a three-dimensional stacked HMC architecture. This dimensional change enables multiple memory dies to be interconnected vertically, providing parallel access paths and increasing overall system bandwidth while distributing coherence management across multiple stacking layers, thereby reducing latency variability.
2Speed
If cache memory is added to improve memory access time, then frequently accessed data can be retrieved faster, but system complexity increases due to coherence protocol requirements
Solution Approach 1:
The patent merges the cache coherence directory functionality with the HMC storage vault structure. By combining these previously separate functions into a unified architecture where the directory resides within the storage vault, the system reduces the number of separate components and their associated interfaces, thereby simplifying the overall coherence protocol implementation while maintaining fast cache access.
3Ease of manufacture
If traditional two-dimensional memory architecture is used, then manufacturing is simpler, but bandwidth and latency performance are limited
Solution Approach 1:
The patent implements a three-dimensional stacked memory architecture using HMC technology, where multiple memory dies are stacked vertically and interconnected through silicon vias. This third dimension enables parallel access to multiple memory layers simultaneously, dramatically increasing bandwidth performance while maintaining manufacturing feasibility through established 3D stacking processes.
Solution Approach 2:
The patent divides the memory system into multiple independent but interconnected HMC dies stacked in three dimensions. Each die can be manufactured separately using standard processes, then interconnected through silicon vias. This segmentation allows for modular manufacturing and testing while achieving high bandwidth through parallel access to multiple stacked layers.
Data Source
AI summary
A system includes a plurality of host processors and a plurality of hybrid memory cube (HMC) devices configured as a distributed shared memory for the host processors. An HMC device includes a plurality of integrated circuit memory die including at least a first memory die arranged on top of a second memory die, and at least a portion of the memory of the memory die is mapped to include at least a portion of a memory coherence directory; and a logic base die including at least one memory controller configured to manage three-dimensional (3D) access to memory of the plurality of memory die by at least one second device, and logic circuitry configured to implement a memory coherence protocol for data stored in the memory of the plurality of memory die.