Remote Memory Access Abstraction for Accelerator Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Accelerator devices in data centers face limitations in accessing and communicating with a large percentage of available accelerator devices due to restricted communication faculties and differences in programming languages and architectures, hindering cooperative workload execution across disaggregated resources.
Innovation Solution
A system that provides communication abstraction for accelerator device kernels, enabling scale-up and scale-out by managing differences in kernels and architectures through a unified protocol, allowing logical communication paths across various physical paths, and facilitating memory access across distributed devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If accelerator device kernels are configured to establish connections with other accelerator device kernels through dedicated communication paths, then communication reliability between specific kernels is improved, but the ability to access and communicate with a large percentage of available accelerator devices deteriorates
Solution Approach 1:
The patent implements a universal communication interface that allows accelerator device kernels to communicate through multiple path types (PCIe, CXL, NVLink, InfiniBand, Ethernet) rather than dedicated single-purpose paths. This enables a single kernel to access diverse accelerator devices through a unified interface, resolving the contradiction between reliable dedicated communication and broad accessibility.
Solution Approach 2:
The patent introduces an intermediary communication layer that translates between different physical communication paths and a unified logical interface. This intermediary enables kernels to access remote memory and communicate across heterogeneous paths without requiring dedicated direct connections, thereby improving both reliability through multiple paths and versatility through broad device accessibility.
2Measurement precision
If kernel developers manually configure communication protocols and connections, then communication precision between specific kernels is improved, but the complexity of kernel development and deployment deteriorates
Solution Approach 1:
The patent implements automatic connection establishment and protocol negotiation where the communication system self-configures paths, addresses, and protocols without manual kernel developer intervention. The system automatically discovers available paths, negotiates communication parameters, and establishes connections, maintaining precision through automated configuration while eliminating development complexity.
Solution Approach 2:
The patent performs preliminary configuration of communication paths and protocols at system initialization rather than at kernel deployment time. Communication infrastructure, path routing, and protocol settings are pre-established, allowing kernels to simply request communication without manual configuration, thereby maintaining precision through pre-configured paths while reducing development complexity.
3Speed
If accelerator devices use specialized communication paths for specific kernel types, then communication speed between compatible kernels is improved, but the ability to cooperate across different accelerator device types deteriorates
Solution Approach 1:
The patent adds a logical communication dimension that sits above physical communication paths. Instead of being constrained to single-purpose physical paths, kernels can communicate through multiple path types simultaneously, with the system selecting optimal paths based on device compatibility and performance requirements. This enables both high-speed specialized communication and broad cross-device cooperation.
Solution Approach 2:
The patent dynamically changes communication parameters (path selection, protocol type, transmission mode) based on the specific kernel pair and workload requirements. The system can switch between different physical paths and protocols depending on whether speed or compatibility is the priority, allowing specialized high-speed paths when available while falling back to general-purpose paths for broader cooperation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Technologies for facilitating remote memory requests in accelerator devices are disclosed. The accelerator device includes circuitry to receive, from a kernel of the present accelerator device, a request through an application programming interface exposed to a high level software language in which the kernel of the present accelerator device is implemented, to establish a logical communication path between the kernel of the present accelerator device and a target accelerator device kernel, based on one or more physical communication paths. The communication protocol supported by the accelerator device may allow kernels operating on the accelerator device to send memory requests for memory locations at remote devices, with the communication protocol performing all of the operations necessary to carry out the memory request.