Bias-Based Coherency in Interconnect Fabric for Accelerator Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computing systems face inefficiencies in managing memory for accelerators due to cache coherency mechanisms, which limit bandwidth and deployment options, especially when using PCIe protocols that do not support dynamic multiplexing of IO, cache, and mem protocols.
Innovation Solution
The Intel Accelerator Link (IAL) protocol extends the Rosetta Link protocol to support accelerators and IO devices by using a combination of Intel On-chip System Fabric, In-die Interconnect, and Scalable Memory Interconnect protocols, enabling coherent memory access and bypassing traditional cache coherency engines for improved performance and power management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional cache coherency mechanisms are used to manage memory for accelerators, then memory protection and system coherence are maintained, but bandwidth and access performance are limited
Solution Approach 1:
The system segments memory management into two distinct paths: a first path for memory accesses that requires cache coherency enforcement (going through the cache coherency engine), and a second path for memory accesses that bypasses the cache coherency engine entirely. This segmentation allows different types of memory operations to use different handling mechanisms, optimizing performance for operations that don't require coherence while maintaining correctness for those that do.
Solution Approach 2:
The patent introduces a memory management intermediary (such as a memory management unit or bridge) that sits between the accelerator and the cache coherency engine. This intermediary can intercept memory access requests and determine whether they need to go through the cache coherency path or can be handled directly, thereby reducing unnecessary coherence overhead and improving bandwidth for operations that don't require full coherence enforcement.
2Adaptability or versatility
If PCIe protocol is used for interconnect, then existing infrastructure is leveraged, but dynamic multiplexing of IO, cache, and mem protocols is not supported
Solution Approach 1:
The patent makes the PCIe interconnect universal by enabling it to handle multiple protocol types (IO, cache, and memory protocols) through a single interface. This is achieved by implementing protocol identification and routing logic that can recognize different protocol types incoming through PCIe and direct them to appropriate handling paths, including the bypass path for memory operations that don't require cache coherence. This multi-functionality allows the system to leverage existing PCIe infrastructure while gaining the versatility of dynamic protocol multiplexing.
3Productivity
If direct memory access paths are created for accelerators, then bandwidth and efficiency are enhanced, but cache coherency management becomes more complex
Solution Approach 1:
The system applies local quality by making memory access paths different depending on the specific requirements of the access. Rather than using a uniform path for all memory operations, the system creates specialized paths: a first path with full cache coherency enforcement for operations that require it, and a second optimized path that bypasses the coherency engine for operations that don't require coherence. This local differentiation optimizes performance for each case while maintaining overall system correctness.
Solution Approach 2:
The patent implements dynamic memory access path selection where the system can adaptively route memory operations based on real-time conditions and requirements. The routing logic can dynamically determine whether an operation should use the cache coherency path or the bypass path, allowing the system to optimize performance on-the-fly while maintaining the ability to enforce coherence when needed. This dynamic approach balances performance and correctness based on actual operational requirements.
Data Source
AI summary
A fabric controller to provide a coherent accelerator fabric, including: a host interconnect to communicatively couple to a host device; a memory interconnect to communicatively couple to an accelerator memory; an accelerator interconnect to communicatively couple to an accelerator having a last-level cache (LLC); and an LLC controller configured to provide a bias check for memory access operations.