Spatial Partitioning of GPU Physical Paths for Multi-tenancy Isolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional graphics processing units (GPUs) in cloud servers face the 'hourglass problem' where concurrently executing applications are unable to isolate resources, leading to performance interference due to shared paths, resulting in only one application accessing the GPU resources at a time.
Innovation Solution
Implementing spatial partitioning of resources such as command buses, caches, and memory channels to allocate exclusive paths for each application based on their characteristics, allowing multiple applications to execute concurrently without interference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple applications share GPU resources concurrently, then resource utilization improves, but performance interference occurs due to shared paths
Solution Approach 1:
The patent divides the GPU into multiple isolated partitions, each dedicated to a specific application. Command buses, caches, and memory channels are segmented into separate paths, preventing performance interference between applications while maintaining high resource utilization through parallel operation of multiple partitions.
2Reliability
If exclusive paths are allocated for each application, then performance isolation improves, but device complexity increases
Solution Approach 1:
The GPU partitions are designed with multi-functionality, where each partition can dynamically handle different application workloads while maintaining isolated paths. The command buses, caches, and memory channels are configured to serve multiple purposes across different partitions, reducing overall device complexity while preserving performance isolation.
3Device complexity
If shared paths are used for GPU resources, then device complexity is reduced, but performance interference occurs between applications
Solution Approach 1:
The patent transitions from a single-shared-dimension resource structure to a multi-dimensional partitioned structure. By adding the dimension of spatial separation through isolated paths for command buses, caches, and memory channels, the system maintains simplified resource management within each partition while enabling high-performance concurrent execution across multiple partitions.
Data Source
AI summary
An apparatus such as a graphics processing unit (GPU) includes a set of shader engines and a set of front end (FE) circuits. Subsets of the set of FE circuits schedule geometry workloads for subsets of the set of shader engines based on a mapping. The apparatus also includes a set of physical paths that convey information from the set of FE circuits to a memory via the set of shader engines. Subsets of the set of physical paths are allocated to the subsets of the set of FE circuits and the subsets of the set of shader engines based on the mapping. The mapping determines information stored in a set of registers used to configure the apparatus. In some cases, the set of registers store information indicating a spatial partitioning of the set of physical paths.