GPU Allocation for Single- and Multi-Machine Deep Learning Tasks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning training technologies struggle to efficiently utilize GPU resources across single-machine and multi-machine tasks, failing to ensure high utilization rates simultaneously.
Innovation Solution
A method and apparatus for allocating GPU resources based on remaining resource quantities and communication distances, optimizing allocation strategies for single-machine and multi-machine tasks by selecting GPUs with minimum remaining resources and shortest communication paths.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If GPU resources are allocated to single-machine tasks, then single-machine task processing efficiency is improved, but multi-machine task resource utilization deteriorates
Solution Approach 1:
The patent implements dynamic resource allocation by continuously monitoring GPU utilization rates and task queue states, and adjusting allocation strategies in real-time. The system transitions between different allocation modes (single-machine priority, multi-machine priority, balanced mode) based on current system state, making the resource allocation mechanism flexible and adaptive rather than static
Solution Approach 2:
The system changes allocation parameters dynamically by adjusting the proportion of resources allocated to single-machine versus multi-machine tasks based on GPU utilization thresholds. When utilization exceeds certain thresholds, the system automatically adjusts allocation weights to balance between the two task types, resolving the contradiction through parameter optimization
2Quantity of substance
If GPU resources are allocated to multi-machine tasks, then multi-machine task resource utilization is improved, but single-machine task processing efficiency deteriorates
Solution Approach 1:
The system employs dynamic allocation that monitors task queue depths and GPU utilization continuously. When multi-machine tasks accumulate in the queue, the system automatically increases allocation to multi-machine tasks while maintaining a minimum service level for single-machine tasks, preventing complete starvation of either task type
Solution Approach 2:
The patent applies partial allocation principles by dedicating a baseline portion of GPU resources to single-machine tasks while allowing flexible excess resources to be allocated to multi-machine tasks. This ensures single-machine tasks always have minimum guarantees while multi-machine tasks can utilize available capacity, resolving the efficiency-utilization contradiction
3Quantity of substance
If GPUs with minimum remaining resources are selected, then resource utilization rate is improved, but communication distance and overhead worsen
Solution Approach 1:
The system implements local quality optimization by considering both resource utilization and communication characteristics when selecting specific GPUs. Different GPU selection strategies are applied locally based on task type: single-machine tasks prioritize communication efficiency, while multi-machine tasks prioritize resource utilization, making the allocation locally optimal rather than globally uniform
Solution Approach 2:
The patent changes selection parameters dynamically by adjusting the weight between resource utilization and communication distance based on task characteristics. For communication-intensive single-machine tasks, the system prioritizes shorter communication distances, while for computation-intensive multi-machine tasks, it prioritizes minimum remaining resources, resolving the contradiction through parameter adaptation
4Length of moving object
If communication distance is minimized, then communication overhead is reduced, but GPU resource utilization rate deteriorates
Solution Approach 1:
The system applies different selection criteria locally based on task type: single-machine tasks use communication distance as the primary selection criterion, while multi-machine tasks use remaining resource quantity as the primary criterion. This local differentiation allows each task type to be optimized for its specific requirements without compromising overall utilization
Solution Approach 2:
The patent implements partial optimization by satisfying communication distance requirements only when necessary for single-machine tasks, while allowing greater flexibility in multi-machine task allocation to maximize resource utilization. The system takes excessive action on resource utilization for multi-machine tasks to ensure high GPU usage rates
Data Source
AI summary
A working method and device for a deep learning training task. GPUs are allocated to multiple deep learning training tasks according to the remaining resources of the GPUs in a single server node or multiple server nodes to achieve the effect of considering multiple deep learning training tasks while ensuring the utilization rate of the GPUs. The method comprises: obtaining a deep learning training task parameter input by a user; determining the type of the deep learning training task from the task parameter, the type of the deep learning training task type comprising: single model and multi-model; selecting GPUs by different policies according to different deep learning training task types; and selecting, according to the position of the GPU, a CPU having a shortest communication distance from the GPU for working.