Multi-Process Service for GPU Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing parallel processing units (PPUs) like GPUs often have underutilized compute and memory-bandwidth capacity due to single processes not fully utilizing available resources.
Innovation Solution
A multi-process service (MPS) is implemented to configure and manage processor partitioning, allowing multiple processes to overlap their computational operations on a PPU, thereby maximizing resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single process utilizes a PPU, then the process can perform computational operations, but the compute and memory-bandwidth capacity is underutilized
Solution Approach 1:
The patent merges multiple processes onto a single PPU through the multi-process service, allowing concurrent execution of multiple process contexts. The MPS manager allocates and manages multiple process contexts (PCs) on the same PPU, enabling compute and memory bandwidth resources to be shared and utilized by multiple processes simultaneously, thereby reducing idle capacity and improving overall PPU utilization.
2Productivity
If multiple processes are allowed to overlap on a PPU, then PPU utilization increases, but the system complexity increases
Solution Approach 1:
The patent introduces a multi-process service (MPS) manager as an intermediary layer between the host CPU and the PPU. This manager handles the complexity of context switching, resource allocation, and process management, shielding individual processes from the underlying complexity. The MPS manager maintains process context structures and manages the overlapping execution, thereby enabling multi-process operation while encapsulating the system complexity within the manager layer.
3Adaptability or versatility
If compute contexts are switched between processes, then resource sharing is enabled, but context switching overhead increases
Solution Approach 1:
The patent implements preliminary action by pre-configuring process context structures and pre-establishing the multi-process service framework before actual computation begins. The MPS manager pre-allocates process contexts and sets up the necessary data structures in advance, so that when context switching is needed, the transition can occur more efficiently with minimal overhead, as the foundational infrastructure is already in place.
Data Source
AI summary
Apparatuses, systems, and techniques to configure processor partitioning for a multi-process service. In at least one embodiment, a multi-process service configures a set of streaming multiprocessors of one or more parallel processing units to perform one or more threads in response to an application programming interface (API).