PPU Fraction Allocation for Isolated Multi-Tenant Job Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems face challenges in optimizing job scheduling and data communication on specialized processors like GPUs, leading to sub-optimal performance, increased costs, and data isolation issues between different tenants.
Innovation Solution
Implementing a job scheduler that allocates jobs to PPU fractions, using an adapter to bypass the CPU for data transfer and enforce data isolation, and employing a smart NIC for efficient job scheduling and data communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If jobs are allocated to entire physical PPUs, then data isolation between tenants is ensured, but resource utilization efficiency deteriorates due to inability to share PPU resources among multiple jobs
Solution Approach 1:
The patent segments a physical PPU into multiple virtual PPUs (vPPUs), each capable of hosting one or more jobs from different tenants. This segmentation allows the physical PPU resources to be shared among multiple jobs while maintaining data isolation through virtualization, thus resolving the contradiction between data isolation and resource utilization efficiency
Solution Approach 2:
The patent introduces a PPU fraction as an intermediary layer between the physical PPU and jobs. The PPU fraction acts as a virtualization mechanism that enables multiple jobs to share the physical PPU while maintaining data isolation, thereby improving resource utilization without compromising security
2Productivity
If a job scheduler is implemented to optimize job allocation, then resource utilization improves, but system complexity increases due to additional scheduling overhead
Solution Approach 1:
The job scheduler is designed to handle multiple functions including job allocation, PPU fraction management, and data isolation enforcement through a unified mechanism. This multi-functionality reduces the need for separate specialized components, thereby improving resource utilization without proportionally increasing system complexity
3Device complexity
If data transfer goes through CPU, then system simplicity is maintained, but performance deteriorates due to CPU bottleneck and increased latency
Solution Approach 1:
The patent extracts the data transfer function from the CPU by implementing a direct memory access (DMA) path between the adapter and PPU memory. This extraction removes the CPU bottleneck from the data transfer path, significantly improving data transfer speed while maintaining system simplicity through the use of standard DMA mechanisms
Data Source
AI summary
In some examples, a controller receives a request to schedule a first job in a system including a plurality of physical parallel processing units (PPUs), where a physical PPU of the plurality of physical PPUs includes multiple PPU fractions. The controller allocates the first job to a first collection of PPU fractions of the multiple PPU fractions based on an operational cost reduction objective to reduce a cost associated with a usage of the plurality of physical PPUs. The controller triggers processing of the first job according to the allocation of the first job to the first collection of PPU fractions, where data isolation is provided between the first job allocated to the first collection of PPU fractions and a second job allocated to a second collection of PPU fractions of the multiple PPU fractions.