Dynamic Load Balancing for Heterogeneous Compute Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Programmable devices like FPGAs in data centers are often underutilized due to their inability to be easily repurposed for different workloads, leading to suboptimal total utilization.
Innovation Solution
Implementing a dynamic load balancing and configuration management system that allocates jobs among a pool of resources, monitors job queues to determine required hardware functionalities, provisions and configures hardware accelerators to meet these demands, and notifies workload management software of available resources, allowing for flexible reconfiguration and increased utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If programmable devices are deployed in specific servers for a single workload, then the device can be dedicated to a specific function, but the total utilization decreases due to inability to be easily repurposed
Solution Approach 1:
The patent implements a pool of hardware accelerators that can be dynamically reconfigured to perform multiple different workloads. Instead of dedicating each accelerator to a single function, the system allows accelerators to be provisioned and configured for various workloads based on instantaneous demand, enabling one accelerator to serve multiple purposes across different applications and time periods.
Solution Approach 2:
The system employs dynamic provisioning and reconfiguration of hardware accelerators based on real-time workload demands. The workload management software continuously monitors the job queue and dynamically allocates accelerators to different workloads as needed, transforming the static one-to-one mapping into a dynamic many-to-many relationship between accelerators and workloads.
2Device complexity
If hardware accelerators are statically configured for specific workloads, then the configuration is simple and stable, but the system cannot adapt to varying workload demands
Solution Approach 1:
The workload management software continuously monitors the job queue and workload demands, using this feedback information to dynamically provision and configure hardware accelerators. The system adjusts accelerator allocation and configuration based on real-time observations of workload patterns, ensuring optimal resource utilization while adapting to changing demands.
Solution Approach 2:
The system implements automated provisioning and configuration of hardware accelerators through workload management software that autonomously monitors job queues, determines required hardware functionalities, and configures appropriate accelerators without manual intervention. This self-service approach enables the system to automatically adapt to workload changes while maintaining manageable complexity.
Data Source
AI summary
An example method of managing a plurality of hardware accelerators in a computing system includes executing workload management software in the computing system configured to allocate a plurality of jobs in a job queue among a pool of resources in the computer system; monitoring the job queue to determine required hardware functionalities for the plurality of jobs; provisioning at least one hardware accelerator of the plurality of hardware accelerators to provide the required hardware functionalities; configuring a programmable device of each provisioned hardware accelerator to implement at least one of the required hardware functionalities; and notifying the workload management software that each provisioned hardware accelerator is an available resource in the pool of resources.