Direct NIC-GPU Interconnect for CPU-Independent Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data communication between a network interface card (NIC) and a graphics processing unit (GPU) in computing systems is limited by the lowest bandwidth interconnect, which constrains throughput and shares high-bandwidth interconnects with other operations, reducing overall performance.
Innovation Solution
A direct interconnect is established between the NIC and the GPU, allowing data transfer without CPU constraints, using distinct address maps and hardware-enforced coherency mechanisms to manage concurrent access and maintain data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data communication between NIC and GPU goes through the CPU via interconnects, then the system can maintain compatibility and simplicity, but the throughput is limited by the lowest bandwidth interconnect (e.g., 64 GB/s Gen 5 PCIe)
Solution Approach 1:
The patent segments the data communication path into two independent routes: a high-bandwidth direct path between NIC and GPU for data transfers, and a separate CPU path for control operations. This segmentation allows each path to operate at its optimal bandwidth without being constrained by the weaker link in the series connection through the CPU.
Solution Approach 2:
The patent introduces a fabric interconnect as an intermediary component that enables direct communication between NIC and GPU. This intermediary bypasses the CPU for data transfer operations, allowing high-speed communication while the CPU remains involved only in configuration and control functions through separate interfaces.
2Productivity
If the high-bandwidth C2C interconnect is shared among DMA communications, I/O communications, and data transfers, then the system can simplify resource management, but the available bandwidth for NIC-GPU communication is reduced
Solution Approach 1:
The patent segments the interconnect bandwidth by creating a dedicated direct path between NIC and GPU that is separate from the CPU's C2C interconnect. This allows NIC-GPU data transfers to utilize the full capacity of the direct path without sharing bandwidth with CPU I/O operations or other DMA communications.
Solution Approach 2:
The patent extracts the data transfer function from the CPU-bound C2C interconnect path and places it on a dedicated NIC-to-GPU path. This extraction removes the bottleneck of shared bandwidth and allows high-speed data transfers independent of CPU operations.
3Productivity
If the NIC is upgraded to support higher bandwidth interconnect (e.g., Gen 6 PCIe with 128 GB/s), then the NIC capability increases, but the overall throughput is still limited by the CPU's lower-bandwidth interconnect
Solution Approach 1:
The patent segments the system interconnect architecture into a high-bandwidth direct path for NIC-GPU communication and a separate path for CPU communication. This allows the NIC to operate at its full upgraded capability (Gen 6 PCIe 128 GB/s) on the direct path while the CPU uses its own interconnect, eliminating the bottleneck of compatibility limitations.
Data Source
AI summary
Various embodiments include techniques for performing data transfer operations via a direct interconnect between a network interface and a graphics processor in a multiprocessor system that also includes a central processing unit (CPU). The CPU communicates with the graphics processor via a dedicated high-bandwidth interconnect to the memory in the graphics processor and a second interconnect to the graphics processor for various utility functions. The network interface communicates with the graphics processor via an interconnect to the memory in the graphics processor. The interconnect between the network interface and the graphics processor does not impact the throughput of the high-bandwidth interconnect from the CPU to the graphics processor, thereby improving CPU to graphics processor performance. Further, the interconnect between the CPU to the graphics processor does not impact the throughput of the interconnect from the network interface to the graphics processor, thereby improving network interface to graphics processor performance.