Parallel Processing Unit Partitioning for Multitenancy

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional GPUs face inefficiencies when handling multiple CPU processes simultaneously due to unfair resource allocation and interference between processing subcontexts, leading to reduced performance and multitenancy issues in cloud-based deployments.

Innovation Solution

A parallel processing unit (PPU) is partitioned into logical groups to support multiple processing contexts in functional isolation, allowing each context to utilize resources efficiently without interference, through the use of a hypervisor that allocates and configures PPU resources independently for each user.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple processing subcontexts share a parent context to enable simultaneous execution of multiple CPU processes on GPU, then resource utilization improves, but resource allocation fairness deteriorates

Engineering Contradiction:

ImproveGPU resource utilizationVSAvoidresource allocation fairness

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent divides a single GPU context into multiple isolated subcontexts, each capable of independent execution. This segmentation allows different CPU processes to run simultaneously in separate subcontexts while maintaining resource allocation fairness through isolation, preventing one subcontext from monopolizing GPU resources at the expense of others.

Inventive Principle:

Principle #1Segmentation

2Productivity

If processing subcontexts share a parent context to enable concurrent execution, then productivity improves, but fault isolation deteriorates

Engineering Contradiction:

Improveconcurrent process executionVSAvoidfault isolation

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent creates isolated subcontexts that function as separate execution environments within the parent context. Each subcontext maintains independent state and resource access, ensuring that faults in one subcontext cannot propagate to other subcontexts or the parent context, thereby achieving both concurrent execution and fault isolation.

Inventive Principle:

Principle #1Segmentation

3Device complexity

If a single context is used for multiple CPU processes to simplify management, then device complexity reduces, but resource utilization deteriorates

Engineering Contradiction:

Improvecontext management complexityVSAvoidGPU resource utilization

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The patent implements a parent context structure that can universally accommodate multiple subcontexts, each handling different CPU processes. This multi-functional design allows the single parent context to manage multiple subcontexts efficiently, simplifying overall context management while enabling full GPU resource utilization through concurrent subcontext execution.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11579925B2Techniques for reconfiguring partitions in a parallel processing system

Publication Date: 2023.02.14 NVIDIA CORP

AI summary

A parallel processing unit (PPU) can be divided into partitions. Each partition is configured to operate similarly to how the entire PPU operates. A given partition includes a subset of the computational and memory resources associated with the entire PPU. Software that executes on a CPU partitions the PPU for an admin user. A guest user is assigned to a partition and can perform processing tasks within that partition in isolation from any other guest users assigned to any other partitions. Because the PPU can be divided into isolated partitions, multiple CPU processes can efficiently utilize PPU resources.