Cache Coherency Directory for Accelerator Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multiprocessing systems, maintaining cache coherence between CPUs and accelerators consumes significant inter-chip bandwidth due to the need for coordinated memory access across multiple nodes, leading to reduced efficiency.
Innovation Solution
Implementing a method where each node maintains a directory to track shared memory locations cached at other nodes, allowing only initial read, write, and synchronization operations to be transmitted, with intermediate accesses performed non-coherently, reducing the need for extensive communication across nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a snooping protocol is implemented to maintain cache coherence between CPUs and accelerators, then data consistency is ensured, but inter-chip bandwidth consumption increases significantly
Solution Approach 1:
The patent segments memory access operations into two categories: initial read/write/synchronization operations that require coherence protocol communication, and intermediate access operations that can be performed non-coherently. This segmentation allows the system to apply coherence protocols only when necessary, reducing unnecessary inter-chip communication while maintaining data consistency for critical operations.
Solution Approach 2:
The patent implements dynamic coherence management where the coherence protocol is selectively applied based on the operation type. Initial operations (read, write, synchronization) trigger coherence protocol communication, while intermediate operations bypass the protocol. This dynamic approach adapts the coherence enforcement to the specific operational context, optimizing bandwidth usage while preserving data consistency where required.
2Reliability
If all memory accesses are propagated to all coherent units for snooping, then cache coherence is maintained, but system efficiency decreases due to excessive communication
Solution Approach 1:
The patent applies partial action by implementing coherence protocol only for necessary operations (initial read, write, and synchronization) rather than all memory accesses. Intermediate access operations are permitted to proceed without full coherence protocol enforcement, reducing communication overhead while maintaining coherence for operations that require it.
Solution Approach 2:
The patent segments the memory access workflow into distinct phases: initial access phase where coherence protocol is fully enforced, and intermediate access phase where non-coherent operations are permitted. This segmentation allows the system to maintain coherence where critical while permitting efficient non-coherent operations elsewhere, thereby improving overall system efficiency.
3Loss of energy
If non-coherent intermediate accesses are permitted, then bandwidth consumption is reduced, but data consistency may be compromised
Solution Approach 1:
The patent dynamically adjusts coherence enforcement based on the operation phase. During the initial access phase, full coherence protocol is applied to establish consistent data state. During the intermediate access phase, non-coherent operations are permitted to reduce bandwidth consumption. This dynamic switching ensures data consistency is maintained when critical while allowing bandwidth optimization during intermediate processing.
Solution Approach 2:
The patent applies coherence protocol partially - only to initial read, write, and synchronization operations where data consistency is critical. Intermediate operations are permitted to proceed non-coherently, accepting reduced consistency guarantees in exchange for significant bandwidth savings. This partial application of coherence matches the consistency requirements to the operational needs.
Data Source
AI summary
Embodiments of the invention provide methods and systems for reducing the consumption of inter-node bandwidth by communications maintaining coherence between accelerators and CPUs. The CPUs and the accelerators may be clustered on separate nodes in a multiprocessing environment. Each node that contains a shared memory device may maintain a directory to track blocks of shared memory that may have been cached at other nodes. Therefore, commands and addresses may be transmitted to processors and accelerators at other nodes only if a memory location has been cached outside of a node. Additionally, because accelerators generally do not access the same data as CPUs, only initial read, write, and synchronization operations may be transmitted to other nodes. Intermediate accesses to data may be performed non-coherently. As a result, the inter-chip bandwidth consumed for maintaining coherence may be reduced.