Distributed Coherence Directory Subsystem for CPU-GPU Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional processing systems with a single coherence directory suffer from increased coherence directory access latencies for CPUs accessing shared memory, particularly when a GPU or high-memory-bandwidth processing unit is integrated with the memory controller, leading to degraded performance for other processing units due to long signaling distances.
Innovation Solution
A distributed coherence directory subsystem is implemented, partitioning the shared address space into multiple coherence directories, with one directory near the memory and another, the EDR coherence directory, closer to memory-intensive processing units, to reduce average access latencies by routing coherence probes to the nearest directory based on address regions accessed primarily by each set of processing units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If the coherence directory is integrated near the memory controller on the GPU to improve GPU memory bandwidth, then the GPU memory bandwidth is improved, but the CPU coherence directory access latency increases
Solution Approach 1:
The coherence directory is divided into multiple segments: a first coherence directory integrated with the memory controller on the GPU die for handling GPU memory access, and a second coherence directory integrated with the CPU interconnect for handling CPU coherence probes. This segmentation allows each directory to be optimally positioned for its primary function, resolving the contradiction between GPU bandwidth and CPU latency.
Solution Approach 2:
The patent introduces an intermediary mechanism where the first coherence directory on the GPU die acts as a local proxy for the second coherence directory. When CPU coherence probes arrive at the GPU memory controller, the first directory intercepts and handles them locally without requiring long-distance signaling to the second directory, thus reducing CPU access latency while maintaining GPU bandwidth optimization.
2Device complexity
If a single coherence directory is used for the entire shared address space, then the system structure is simple, but the access latency for distant processing units is long
Solution Approach 1:
The unified coherence directory is segmented into multiple distributed directories located at different physical positions in the system. The first coherence directory is positioned near the memory controller, while the second is positioned near the CPU interconnect. This segmentation reduces the average access latency by placing directories closer to the processing units that need them, while the overall system maintains a relatively simple structure through standardized interfaces between the directories.
Data Source
AI summary
A processing system includes a first set of one or more processing units including a first processing unit, a second set of one or more processing units including a second processing unit, and a memory having an address space shared by the first and second sets. The processing system further includes a distributed coherence directory subsystem having a first coherence directory to support a first subset of one or more address regions of the address space and a second coherence directory to support a second subset of one or more address regions of the address space. In some implementations, the first coherence directory is implemented in the system so as to have a lower access latency for the first set, whereas the second coherence directory is implemented in the system so as to have a lower access latency for the second set.