GPU Resource Allocation via Topology and Fragmentation Factors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPU resource allocation methods in high-performance calculation and AI platforms do not consider job types and resource characteristics, leading to inefficient resource utilization and fragmentation, resulting in wasted performance and reduced availability of GPU resources.
Innovation Solution
An efficient GPU resource allocation optimization method that calculates a target function value by determining a GPU topology communication factor and fragmentation factor, using a weighted calculation to optimize resource allocation based on physical topology and job information, ensuring minimal fragmentation and optimal resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If GPU resources are allocated based on static topology graph without considering job characteristics, then allocation simplicity is maintained, but resource utilization efficiency deteriorates due to fragmentation
Solution Approach 1:
The patent transforms the static topology graph into a dynamic allocation model that adapts to different job characteristics. The system dynamically adjusts allocation strategies based on job type (e.g., training vs. inference), resource requirements, and current system state, thereby resolving the contradiction between simple static allocation and efficient dynamic utilization.
Solution Approach 2:
The patent introduces multiple parameters including communication cost, fragmentation factor, and job-specific weights to optimize allocation. By changing these parameters based on job characteristics and system state, the system achieves both operational simplicity through standardized metrics and high efficiency through parameterized optimization.
2Reliability
If GPU resources are allocated to the first scheduled job with higher priority, then scheduling priority is respected, but resource availability for subsequent jobs deteriorates due to fragmentation
Solution Approach 1:
The patent performs preliminary evaluation of allocation options using the target function before finalizing assignments. This preliminary action considers both priority requirements and future availability, allowing the system to respect scheduling priorities while avoiding fragmentation that would harm subsequent job availability.
Solution Approach 2:
The patent implements a feedback mechanism where the allocation decision considers not only current job priorities but also the impact on future resource availability. The target function incorporates feedback about system state and job characteristics to balance priority respect with sustained resource availability.
3Speed
If non-uniform memory access packets are used for GPU allocation, then memory access efficiency is improved, but allocation complexity increases leading to fragmentation
Solution Approach 1:
The patent applies local quality by assigning different weights and characteristics to different NUMA packets based on their specific properties and job requirements. Rather than treating all packets uniformly, the system optimizes allocation locally for each packet based on its memory access characteristics, thereby maintaining efficiency while managing complexity through localized optimization.
4Adaptability or versatility
If multiple jobs with different resource requirements are scheduled simultaneously, then system versatility is improved, but resource allocation efficiency deteriorates due to conflicting requirements
Solution Approach 1:
The patent segments the allocation problem by considering each job's specific characteristics and requirements separately while optimizing the overall system. The target function is segmented into components that evaluate different aspects (communication cost, fragmentation, job-specific weights) independently, allowing versatile multi-job support while maintaining allocation efficiency through structured evaluation.
Data Source
AI summary
An efficient GPU resource allocation optimization method and system. The method includes: invoking an allocation interface for GPUs; acquiring GPU resources and data information needed for GPU allocation, the data information comprising a physical topology diagram structure, a NUMA packet structure, and operation information; determining a GPU topology communication factor according to a GPU static topology diagram in physical topology diagrams; determining a GPU fragmentation factor according to the NUMA packet structure and the operation information by adding a correction during GPU fragment computation; and performing weighted computation on the obtained communication factor and the obtained fragmentation factor to determine a target function value, the minimum target function value corresponding to an optimal GPU resource allocation scheme.