Forwarding Tables for CXL Multi-Link GPU Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The bottleneck of CPU-to-GPU data transfer in systems with high-speed GPU-GPU interconnects is caused by slower PCIe links, leading to underutilization of higher bandwidth CXL links and limiting overall system performance.
Innovation Solution
Implementing a memory access scheme that utilizes forwarding tables and CXL specification-compliant links to multiplex data transfer through both host and high-speed internal-connect links, enabling parallel access to host-managed device memory across multiple GPUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If PCIe links are used for CPU-to-GPU data transfer, then compatibility and ease of operation are maintained, but data bandwidth is limited and system performance is bottlenecked
Solution Approach 1:
The patent segments the data transfer path into multiple parallel links (PCIe link and HSIL link) that can be independently utilized. The memory access is divided into portions that can be transferred simultaneously over different links, effectively segmenting the bandwidth bottleneck into multiple narrower streams that collectively provide higher throughput.
Solution Approach 2:
The patent introduces an intermediary mechanism (the link multiplexing logic and forwarding tables) that mediates between the CPU and device memory by routing memory access requests through either the PCIe link or the HSIL link based on availability and performance metrics. This intermediary layer abstracts the complexity of multi-link management while enabling high-speed data transfer.
2Productivity
If HSILs are used for GPU-to-GPU communication, then high bandwidth is achieved, but CPU-to-GPU data transfer becomes bottlenecked by slower PCIe links
Solution Approach 1:
The patent makes the HSIL link serve multiple functions: it enables high-speed GPU-to-GPU communication while also being utilized for CPU-to-GPU data transfer when multiplexed through the device. The same high-bandwidth infrastructure serves dual purposes, eliminating the need for separate dedicated links for different communication patterns.
Solution Approach 2:
The patent implements dynamic link selection where the system adaptively chooses between PCIe link and HSIL link for CPU-to-GPU transfers based on real-time conditions such as link availability, current utilization, and performance requirements. This dynamic approach allows the system to optimize transfer speed by selecting the appropriate link for each operation.
3Productivity
If single link is used for memory access, then device complexity is minimized, but data bandwidth and resource utilization are underutilized
Solution Approach 1:
The patent performs preliminary actions by pre-configuring forwarding tables that map memory addresses to appropriate links (PCIe or HSIL) before actual data transfer occurs. This advance preparation enables the system to quickly route memory requests through the optimal path without real-time decision-making complexity, improving resource utilization while managing complexity through pre-computed routing information.
Data Source
AI summary
Examples include techniques to multiply memory access bandwidth using a plurality of links. Example techniques may include generation and use of forwarding tables separately maintained at devices coupled via high speed internal-connect links (HSILs). A forwarding table to enable a first device to route a memory request received by the first device to access a memory address of a memory at a second device. The memory request received by the first device from a host compute device via a link between the first device and the host compute device, the memory request to be forwarded to the second device via an HSIL coupled between the first and second devices.