Accelerator Task Scheduling for Dynamic Resource Pooling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Accelerator devices often remain idle due to static resource allocation, leading to wastage of resources in data centers, especially in environments where workloads are not consistently utilized, such as in machine learning and genomics applications.
Innovation Solution
Implementing a data center architecture with dynamically pooled resources, such as accelerators, memory, and storage, that can be allocated on an as-needed basis through an optical fabric, allowing compute resources to access these resources as if they were local, and utilizing predictive utilization models to reallocate resources based on past usage patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If static resource allocation is used for accelerator devices, then resource assignment is simple and stable, but resource utilization is low and waste occurs
Solution Approach 1:
The patent implements dynamic resource allocation where accelerator devices are not permanently assigned to specific workloads but are instead allocated on-demand based on current needs. The system monitors workload requirements and dynamically assigns accelerator resources from a shared pool, allowing resources to transition between idle and active states based on real-time demand rather than static configuration.
Solution Approach 2:
The patent creates a universal accelerator resource pool where multiple types of accelerator devices (GPUs, FPGAs, ASICs) can serve multiple different workloads. Instead of dedicating specific accelerators to specific applications, the system allows any accelerator in the pool to be allocated to any workload that requires acceleration capabilities, making the infrastructure multi-functional and adaptable.
2Reliability
If accelerator devices are allocated for the duration of workload, then resource availability is guaranteed, but idle time increases and resources are wasted
Solution Approach 1:
The patent ensures continuous utilization of accelerator resources by implementing a resource pooling mechanism where accelerators that finish one workload are immediately made available for the next assignment. The system maintains a queue of pending workloads and continuously allocates resources from the pool to awaiting tasks, eliminating idle periods and ensuring that accelerator devices are constantly performing useful work rather than remaining dormant between allocations.
3Productivity
If dynamic resource pooling is implemented, then resource utilization improves, but system complexity increases
Solution Approach 1:
The patent introduces a resource manager as an intermediary component that handles the complexity of dynamic allocation, pooling, and assignment of accelerator resources. This mediator layer sits between the workloads and the physical accelerator devices, abstracting the complex management tasks away from both the applications and the hardware. The resource manager handles monitoring, allocation, de-allocation, and coordination, thereby enabling dynamic resource pooling without requiring complex changes to the underlying accelerator architecture or the workload applications.
Data Source
AI summary
Technologies for dividing work across one or more accelerator devices include a compute device. The compute device is to determine a configuration of each of multiple accelerator devices of the compute device, receive a job to be accelerated from a requester device remote from the compute device, and divide the job into multiple tasks for a parallelization of the multiple tasks among the one or more accelerator devices, as a function of a job analysis of the job and the configuration of each accelerator device. The compute engine is further to schedule the tasks to the one or more accelerator devices based on the job analysis and execute the tasks on the one or more accelerator devices for the parallelization of the multiple tasks to obtain an output of the job.