GPGPU Proxy Agent Dynamic Workload Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data center systems inefficiently manage general-purpose graphical processing units (GPGPUs) due to static allocation and underutilization, as workloads intermittently require GPGPU resources, leading to resource wastage and inefficiency.
Innovation Solution
A method for dynamically managing GPGPUs by a proxy agent that receives requests, selects suitable GPGPUs based on resource availability and workload requirements, establishes sessions, and collects performance profiles to identify underutilization, allowing for dynamic reassignment of workloads to optimize resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If GPGPUs are statically allocated to specific workloads until termination, then workload reliability is improved, but GPGPU utilization deteriorates
Solution Approach 1:
The system dynamically reassigns workloads to different GPGPUs based on real-time utilization metrics. Instead of static allocation, the cloud orchestrator continuously monitors performance data and redistributes workloads to maintain optimal utilization across the GPGPU pool, transforming the system from static to dynamic resource management.
Solution Approach 2:
The system changes the allocation parameter from dedicated one-to-one mapping to flexible many-to-many mapping. By adjusting the allocation strategy based on utilization thresholds and workload characteristics, the system optimizes resource distribution while maintaining sufficient reliability for each workload.
2Reliability
If entire GPGPUs are allocated to intermittently utilizing workloads, then workload reliability is improved, but resource efficiency deteriorates
Solution Approach 1:
GPGPUs are designed to serve multiple workloads sequentially rather than being dedicated to a single workload. The system enables each GPGPU to universally handle different types of computational tasks by dynamically reconfiguring resource allocation based on current utilization needs, eliminating waste during idle periods.
Solution Approach 2:
The system discards idle GPGPU resources from intermittently utilizing workloads and recovers them for assignment to other active workloads. By monitoring utilization patterns and reallocating underutilized GPGPUs to pending tasks, the system minimizes resource waste while maintaining reliability for workloads that currently need them.
3Reliability
If GPGPUs are not simultaneously shared amongst workloads, then workload reliability is improved, but system flexibility deteriorates
Solution Approach 1:
The system implements dynamic timesharing where GPGPUs are simultaneously allocated to multiple workloads through temporal multiplexing. The cloud orchestrator manages simultaneous allocations by monitoring utilization in real-time and dynamically adjusting which workloads occupy which GPGPUs, enabling both sharing and adaptability while maintaining reliability.
Solution Approach 2:
The cloud orchestrator acts as an intermediary between workloads and GPGPUs, managing simultaneous allocations and reassignments. This intermediary coordinates resource sharing, handles workload migration, and ensures that reliability requirements are met while enabling flexible simultaneous access to GPGPU resources across multiple workloads.
4Reliability
If dedicated GPGPUs are allocated to machine learning prediction workloads, then workload reliability is improved, but GPGPU utilization deteriorates
Solution Approach 1:
The system changes the allocation parameter for machine learning workloads from dedicated to on-demand based on prediction triggers. When prediction workloads receive input data requiring GPGPU processing, they are dynamically allocated appropriate resources. When idle, the same GPGPUs are reallocated to other workloads, optimizing utilization while maintaining reliability when needed.
Data Source
AI summary
A method for managing general-purpose graphical processing units (GPGPUs) in a data center system is described. The method includes receiving, by a proxy agent, a GPGPU request from an application; selecting a GPGPU from a set of GPGPUs for processing a workload of the application based on one or more of available resources of the set of GPGPUs and requirements of the workload as indicated by the GPGPU request; establishing a session between an application agent located on a compute node on which the application is located and the proxy agent, and a second session between the GPGPU and the proxy agent in response to selecting the GPGPU to allow the GPGPU to process the workload, including subsequent GPGPU requests associated with the workload; and collecting a performance profile to describe usage of resources of the GPGPU by the workload.