HMC Memory Coherence Directory Placement for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multi-processor systems with shared memory face challenges in memory access latency and cache coherence, where existing solutions consume precious system bandwidth and result in uneven latency due to complex cache coherence protocols and inefficient data management.
Innovation Solution
The implementation of Hybrid Memory Cube (HMC) devices, which provide a high-bandwidth, three-dimensional memory architecture with integrated logic and memory layers, enables efficient memory access and cache coherence by storing memory coherence directories within the HMC devices, allowing for uniform data access and reducing latency through atomic operations and packetized protocols.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If cache memory is used to improve memory access time, then memory access speed is improved, but system bandwidth is consumed and latency becomes uneven
Solution Approach 1:
The patent transitions from traditional two-dimensional cache coherence protocols to a three-dimensional memory architecture where memory coherence directories are stored within the HMC device itself. This dimensional change eliminates the need for separate cache coherence mechanisms and reduces system bandwidth consumption by integrating coherence management into the memory fabric.
2Reliability
If cache coherence protocols are implemented to maintain data correctness, then data consistency is improved, but device complexity increases
Solution Approach 1:
The patent merges memory coherence directory storage with the HMC device architecture, combining previously separate functions (cache coherence management and memory storage) into a unified system. This integration simplifies the overall device complexity while maintaining data consistency through atomic operations and packetized protocols.
3Reliability
If separate cache coherence mechanisms are used to ensure data correctness, then data consistency is improved, but system latency increases
Solution Approach 1:
The patent extracts cache coherence management from external separate mechanisms and embeds it directly within the HMC device. By taking out the coherence directory from external memory and placing it within the HMC fabric, the system eliminates additional memory access latency while maintaining data consistency through integrated coherence protocols.
Data Source
AI summary
A system includes a plurality of host processors and a plurality of HMC devices configured as a distributed shared memory for the host processors. An HMC device includes a plurality of integrated circuit memory die including at least a first memory die arranged on top of a second memory die and at least a portion of the memory of the memory die is mapped to include at least a portion of a memory coherence directory; and a logic base die including at least one memory controller configured to manage three-dimensional (3D) access to memory of the plurality of memory die by at least one second device, and logic circuitry configured to determine memory coherence state information for data stored in the memory of the plurality of memory die, communicate information regarding the access to memory, and include the memory coherence information in the communicated information.