NVLink-to-CHI Memory Bridge for ARM AI Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing architectures face limitations when integrating NVLink-connected accelerators with ARM CHI-based coherent interconnects due to protocol differences, preventing efficient memory sharing and resource access in ARM-based systems, particularly in edge computing and datacenter deployments.
Innovation Solution
Implementing Resource Provisioning Units (RPUs) that bridge the protocol gap between NVLink and CHI protocols, enabling NVLink-connected accelerators to access memory resources within ARM CHI-based coherent interconnect fabrics, maintaining high-bandwidth connectivity and cache coherency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If NVLink-connected accelerators are integrated with ARM CHI-based coherent interconnects, then high-bandwidth connectivity and memory sharing capability are improved, but protocol compatibility and system complexity worsen due to protocol differences between NVLink and CHI
Solution Approach 1:
The patent introduces a protocol translation layer or bridge component that mediates between NVLink and CHI protocols, enabling accelerators connected via NVLink to access memory resources in the CHI-based coherent fabric without requiring direct protocol compatibility. This intermediary handles protocol conversion, message formatting, and address mapping between the two different interconnect standards.
Solution Approach 2:
The system design enables the ARM-based platform to support multiple interconnect protocols (both NVLink and CHI) through a unified architecture, allowing the same memory subsystem and coherent fabric to serve both NVLink-connected accelerators and CHI-native components, thereby improving resource utilization and system flexibility.
2Adaptability or versatility
If protocol translation between NVLink and CHI is implemented, then interoperability and resource access capability are improved, but translation overhead and processing latency worsen
Solution Approach 1:
The system performs address translation and protocol mapping in advance during device initialization and memory allocation phases, pre-computing translation tables and establishing mapping relationships between NVLink address space and CHI coherent fabric address space. This preliminary setup reduces the complexity and latency of runtime protocol translation operations.
Solution Approach 2:
The patent employs translation lookaside buffers (TLBs) or cache structures that store pre-computed protocol translation mappings, allowing frequently accessed memory regions to be translated using cached entry information rather than performing full protocol conversion, thereby reducing average translation latency.
Data Source
AI summary
Modern AI/ML workloads demand efficient integration of GPUs and accelerators with ARM-based server architectures in datacenters and edge computing environments. Embodiments herein disclose systems incorporating RPUs that enable NVLink-connected accelerators to access memory resources within ARM CHI-based coherent interconnect fabrics. One embodiment comprises a CHI-based coherent interconnect with interconnect components routing CHI messages between processing cores and memory controllers supporting substantial memory capacities. The RPU bridges NVLink and CHI protocols, while performing protocol translation between NVLink and CHI messaging, enabling GPUs and accelerators to access system memory through the coherent fabric. Multiple RPUs optionally support scalable configurations with multiple NVLink-connected devices accessing shared memory resources. The embodiments address memory disaggregation challenges for GenAI inference, LLM training, and distributed computing, enabling accelerators to leverage ARM-based system memory beyond local device capacity while potentially maintaining cache coherency, suitable for heterogeneous computing deployments.