Preemptible Coprocessing for GPU Resource Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud computing systems face inefficiencies due to unused coprocessing resources, particularly in data centers where CPUs are engaged while GPUs and other specialized processors remain idle, leading to reduced overall efficiency.
Innovation Solution
Implementing a system that supports preemptible coprocessing by containerizing compute jobs and sub-tasks, allowing for efficient provisioning, de-provisioning, and migration of resources, including GPUs and FPGAs, to utilize available coprocessors more effectively and improve resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a cloud computing system employs servers with multiple processors including GPUs, then the system can provide high quality of service, but the GPUs may remain idle when the CPU is in use, resulting in unused computing capacity
Solution Approach 1:
The system enables GPUs to serve multiple purposes: they can be shared between different compute jobs and can handle various types of computing tasks. The GPU virtualization layer allows the same physical GPU to be allocated to different virtual machines and workloads dynamically, transforming a single-function resource into a multi-functional asset that improves both service quality and utilization efficiency
Solution Approach 2:
The system implements dynamic resource allocation where GPU assignments are not fixed but can be adjusted in real-time based on workload demands. The virtualization layer and scheduling mechanisms allow GPUs to be dynamically provisioned, de-provisioned, and re-allocated between different compute jobs, enabling the system to adapt to changing requirements and maximize resource utilization while maintaining service quality
2Productivity
If the system allocates dedicated GPUs to each compute job, then the compute job can execute efficiently, but the GPU cannot be utilized by other compute jobs, reducing overall data center efficiency
Solution Approach 1:
The system merges multiple compute jobs that require GPU resources into a shared infrastructure. By combining resource demands from multiple jobs and using virtualization to divide and allocate GPU capacity dynamically, the system ensures that GPUs remain actively utilized across different workloads rather than sitting idle between dedicated assignments, thereby reducing energy waste while maintaining job efficiency
Solution Approach 2:
The system implements self-service mechanisms where the virtualization layer and resource management system automatically allocate, monitor, and re-allocate GPU resources based on actual workload needs without manual intervention. This self-adjusting capability ensures that computing capacity is continuously optimized and utilized efficiently across multiple compute jobs
Data Source
AI summary
Methods, computer program products, and systems supporting preemptible coprocessing are disclosed. The method includes executing at least a portion of a first compute job, and executing at least a portion of a second compute job. The method further includes, prior to completing execution of the at least the portion of the second compute job, interrupting the execution of the second compute job, and scheduling at least a portion of a third compute job.