Virtualized Compute Accelerator Kernel Assignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Client devices with GPUs are underutilized due to bandwidth and latency constraints in cloud computing environments, leading to inefficient offloading of computational tasks to servers in datacenters.
Innovation Solution
Implementing a virtualized compute accelerator system that uses static analysis to identify dependencies between compute kernels and optimally assign them to available accelerators, both in datacenters and on edge devices, to maximize parallel execution and minimize latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If computational tasks are offloaded to servers in datacenters, then processing power and computational capability are improved, but bandwidth consumption increases and latency is introduced
Solution Approach 1:
The system segments computational tasks into compute kernels and divides them into different sets based on dependency relationships. Independent compute kernels are assigned to edge device GPUs for local execution, while dependent kernels are assigned to datacenter servers. This segmentation allows computation to be distributed between edge and cloud, reducing bandwidth consumption by keeping independent tasks local while maintaining access to datacenter processing power when needed.
2Power
If computational tasks are offloaded to servers in datacenters, then processing capability is improved, but latency increases
Solution Approach 1:
The system performs static analysis to identify dependency relationships between compute kernels in advance, before execution. This preliminary analysis allows the system to pre-determine which kernels can be executed independently at the edge and which require datacenter processing, optimizing the execution plan beforehand to minimize latency during actual runtime.
Solution Approach 2:
By segmenting compute kernels into independent and dependent sets based on static dependency analysis, the system enables parallel execution of independent kernels at the edge device without waiting for datacenter processing, thereby reducing overall latency while still utilizing datacenter processing capability for kernels that require it.
3Productivity
If client device GPUs are utilized for computation, then processing efficiency is improved, but resource coordination and task assignment complexity increases
Solution Approach 1:
The system introduces a management layer with a static analysis engine as an intermediary between the application and the distributed GPU resources. This intermediary analyzes compute kernel dependencies, determines execution sets, and assigns kernels to appropriate devices. By centralizing this coordination logic, the system simplifies resource management while enabling efficient utilization of both edge and datacenter GPUs.
Solution Approach 2:
The management layer receives information about available GPU resources from both edge devices and datacenter servers, uses static analysis to determine optimal assignment, and assigns compute kernels accordingly. This feedback loop allows the system to adapt to available resources while maintaining efficient utilization through informed decision-making about where to execute each compute kernel.
Data Source
AI summary
Disclosed are various embodiments for assigning compute kernels to compute accelerators that form an aggregated virtualized compute accelerator. A directed, acyclic graph (DAG) representing a workload assigned to a virtualized compute accelerator is generated. The workload can include a plurality of compute kernels and the DAG comprising a plurality of nodes and a plurality of edges, each of the nodes representing a respective compute kernel, each edge representing a dependency between a respective pair of the compute kernels, and the virtualized compute accelerator representing a logical interface for a plurality of compute accelerators. The DAG can be analyzed to identify sets of dependent compute kernels, each set of dependent compute kernels being independent of the other sets of dependent compute kernels and execution of at least one compute kernel in a set of dependent compute kernels depending on a previous execution of another computer kernel in the set of dependent compute kernels. Then, each set of dependent compute kernels can be assigned to a respective one of the plurality of compute accelerators.