GPU Resource Sharing via Slice Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current large-scale compute infrastructures do not enable applications running on the same server or cluster to share GPUs, leading to decreased utilization, increased costs, and energy wastage due to the lack of a software mechanism for concurrent GPU sharing among applications.
Innovation Solution
The implementation of a GPU gatekeeper module that partitions GPU resources into slices, allowing multiple applications to share processing cores and memory in a fair and secure manner, enabling concurrent execution and optimizing resource utilization across both space and time dimensions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If whole GPU is allocated to a single application, then the application can utilize all GPU resources, but other applications cannot share the GPU leading to decreased utilization
Solution Approach 1:
The patent divides the GPU into multiple partitions, each with its own dedicated resources (processing cores, memory, caches). This segmentation allows multiple applications to simultaneously access different portions of the GPU without interfering with each other, thereby enabling resource sharing while maintaining high utilization rates.
Solution Approach 2:
The patent introduces a GPU partitioning mechanism that acts as an intermediary between multiple applications and the physical GPU resources. This intermediary layer manages resource allocation, enables concurrent access by multiple applications, and maintains isolation between them, thus allowing versatile resource sharing while preserving productivity.
2Productivity
If GPU resources are shared among multiple applications, then utilization increases, but resource hogging may occur where one application monopolizes resources
Solution Approach 1:
By dividing the GPU into fixed partitions with dedicated resources, the patent prevents any single application from monopolizing the entire GPU. Each partition has guaranteed resources that cannot be stolen by other applications, ensuring fair resource allocation while maintaining high overall utilization.
Solution Approach 2:
The patent assigns different resource characteristics to different partitions, allowing each application to have its own optimized resource allocation. This local differentiation ensures that each application receives appropriate resources for its needs while preventing resource hogging, thereby maintaining both productivity and fairness.
3Loss of energy
If GPU resources are shared among applications, then cost and energy consumption are reduced, but security and non-interference guarantees are not provided
Solution Approach 1:
The patent creates isolated partitions with dedicated resources that prevent applications from accessing or interfering with each other's memory and processing units. This physical isolation through segmentation provides security guarantees while enabling resource sharing that reduces energy consumption compared to allocating separate GPUs to each application.
Solution Approach 2:
The GPU partitioning mechanism serves as a secure intermediary that manages resource access between applications. It enforces isolation policies and prevents unauthorized access between partitions, providing security guarantees while enabling efficient resource sharing that reduces overall energy consumption.
4Productivity
If GPU resources are shared concurrently among applications, then utilization increases, but complexity of resource management increases
Solution Approach 1:
The patent simplifies resource management complexity by creating fixed, static partitions with predetermined resource allocations. This segmentation approach eliminates the need for complex dynamic scheduling and resource negotiation between applications, as each partition has its own dedicated resources that don't require complex coordination.
Data Source
AI summary
Examples of GPU resource sharing among applications are disclosed. In one example, a method includes receiving a first request from a first application of the plurality of applications for first requested GPU resources, and receiving a second request from a second application of the plurality of applications for second GPU resources. The method also includes, responsive to determining that the first requested GPU resources are available, allocating a first slice of the GPU resources with a first requested amount of resources to the first application and, responsive to determining that the second requested GPU resources are available, allocating a second slice of the GPU resources with a second requested amount of resources to the second application. Further, the method includes enabling the first application and the second application to execute concurrently within the first slice of the GPU and the second slice of the GPU respectively.