Dynamic Intelligent Routing Engine for Accelerator Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hardware accelerator technologies face challenges in efficiently utilizing resources and implementing dynamic routing of virtual machine workloads due to sequential workload processing, limited concurrent processing of multiple VMs, and immature load balancing, leading to increased costs for cloud service providers and end users.
Innovation Solution
The Dynamic Intelligent Routing Engine (DIRE) partitions and groups accelerator resources by maintaining resource data structures and group information to dynamically route workloads across multiple accelerators, allowing parallel processing of multiple workloads and considering future workload forecasts and power management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hardware accelerators process workloads sequentially, then resource utilization is simplified to manage, but workload processing efficiency deteriorates due to inability to handle multiple VMs concurrently
Solution Approach 1:
The patent segments accelerator resources into multiple Virtual Functions (VFs), where each VF can independently handle workload from a different VM. This segmentation enables concurrent processing of multiple workloads across different VFs while maintaining simplified resource management through the virtualization layer. Each VF acts as an independent processing unit that can be allocated to different VMs, thus achieving parallelism without complicating the underlying hardware management.
Solution Approach 2:
The patent introduces a Virtualization I/O Interface (VIOIF) as an intermediary between the hardware accelerator and multiple VMs. This intermediary manages the distribution of workloads to appropriate VFs and handles the coordination required for concurrent processing. By placing this mediation layer, the system achieves multi-workload concurrency while keeping the hardware interface standardized and manageable.
2Quantity of substance
If multiple VMs share a single accelerator, then resource utilization improves, but workload processing deteriorates due to sequential handling of multiple VMs
Solution Approach 1:
The patent divides the shared accelerator into multiple Virtual Functions (VFs), each capable of independently processing workloads from different VMs simultaneously. This segmentation transforms a single sequential processing unit into multiple parallel processing units, enabling concurrent workload handling while maintaining efficient resource utilization through shared physical hardware.
Solution Approach 2:
The patent adds a virtualization dimension to the accelerator architecture by introducing Virtual Functions as an intermediate layer between the physical accelerator and VMs. This dimensional addition allows multiple VMs to access the accelerator concurrently through different VF instances, transforming single-threaded sequential access into multi-threaded parallel access without requiring multiple physical accelerators.
3Productivity
If load balancing technology is used for virtualization environments, then workload distribution improves, but system reliability deteriorates due to immature load balancing technology
Solution Approach 1:
The patent implements self-service load balancing where each Virtual Function automatically manages its own workload queue and processing state. The VIOIF enables workloads to be routed to appropriate VFs based on their current utilization state without requiring complex centralized load balancing algorithms. This self-managing approach provides reliable workload distribution while avoiding the instability associated with immature load balancing technologies.
Data Source
AI summary
Embodiments described herein are generally directed to assigning virtual machine (VM) workloads to groupings/partitions of accelerator resources. In an example, a processing resource of a host system maintains: (i) a resource data structure containing resource utilization information for each of one or more accelerators associated with the host system; and (ii) a group data structure containing information regarding each group of multiple groups of one or more virtual functions (VFs) of the one or more accelerators that has been assigned for use by a respective VM of multiple VMs running on a virtual machine monitor (VMM) of the processing resource. A request to deploy a workload associated with a first VM is received. Responsive to the request, the workload is assigned to a VF of a group of the multiple groups determined to have resource capacity available to satisfy expected resource utilization of the workload.