Virtual Accelerator Pairing for Cross-Host Compute Offload
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing applications deployed on hosts without hardware accelerators face challenges in utilizing hardware acceleration due to the lack of computation offloading capabilities.
Innovation Solution
A method is introduced to virtualize hardware accelerators by reconfiguring a programmable expansion bus device, such as a smart NIC, to emulate a virtual accelerator, allowing compute tasks to be offloaded to a target host with a hardware accelerator, thereby enabling applications to utilize hardware acceleration capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hardware accelerators are installed in some hosts, then computation tasks can be offloaded to hardware circuitry, but hosts without hardware accelerators cannot utilize hardware acceleration capabilities
Solution Approach 1:
The patent creates a virtual copy of the hardware accelerator in the form of a virtual accelerator device. This virtual accelerator emulates the functionality of the physical hardware accelerator, allowing applications on hosts without hardware accelerators to interact with and utilize acceleration capabilities through the virtualized interface. The virtual accelerator acts as a software abstraction that copies the essential functions of the hardware accelerator.
Solution Approach 2:
The virtual accelerator serves as an intermediary between applications on hosts without hardware accelerators and the actual hardware accelerator on target hosts. It mediates the interaction by receiving computation tasks from applications, forwarding them to appropriate target hosts with hardware accelerators, and returning results, thereby enabling indirect access to hardware acceleration capabilities.
2Adaptability or versatility
If hardware accelerators are virtualized across a host cluster, then applications on any host can access hardware acceleration, but system complexity increases due to virtualization overhead
Solution Approach 1:
The virtual accelerator is designed as a universal interface that can work with multiple different types of hardware accelerators across different host architectures. It provides a standardized method for applications to access hardware acceleration regardless of the specific underlying hardware implementation, reducing the need for application-specific virtualization configurations and simplifying the overall system.
3Productivity
If compute tasks are forwarded to target hosts for processing, then hosts without hardware accelerators can utilize acceleration capabilities, but communication overhead increases
Solution Approach 1:
The system performs preliminary actions by pre-establishing connections between virtual accelerators and target hardware accelerators, and by pre-configuring the virtualization infrastructure. This allows compute tasks to be quickly forwarded to target hosts without requiring setup or configuration at task execution time, reducing the communication overhead and latency.
Data Source
AI summary
An example method of offloading a compute task to an accelerator by an application executing in a host cluster includes: sending, by an application executing in an initiator host, the compute task to a virtual accelerator in the initiator host, the virtual accelerator paired with a hardware accelerator in a target host of the host cluster; forwarding, by the virtual accelerator, the compute task to the hardware accelerator in the target host over a network; and receiving, at the application via the virtual accelerator, a result of the compute task from the hardware accelerator over the network.