A fine-grained approach to GPU resource management for mixed workloads

Abstract

The invention discloses a fine-grained GPU resource management method for mixed loads. A capacity-based streaming multiprocessor abstract model CapSM is proposed and serves as a basic unit of resource management; when the mixed loads (including an online task and an offline task) share GPU resources, the use of the GPU resources by different types of tasks is managed through fine granularity, and task resource allocation and resource online adjustment are supported; and while the GPU resources are shared, the service quality of the online task is ensured. The resources allocated to the tasks are determined according to the types of the tasks, a resource request and a current GPU resource state of a system; the use of the GPU resources by the offline task can be met under the condition of sufficient resources; and when the GPU resources are insufficient, the resource use of the offline task is dynamically adjusted to preferentially meet the resource demand of the online task, so that when the mixed loads run at the same time, the performance of the online task can be ensured and the GPU resources can be fully utilized.

Description

technical field [0001] The invention relates to the field of resource management and task scheduling in heterogeneous computing, in particular to a fine-grained GPU resource management method for mixed loads. Background technique [0002] Graphics Processing Unit (hereinafter referred to as GPU) has gradually become an indispensable part of high-performance computing, cloud computing and data centers due to its powerful peak computing capabilities. The use of GPUs to accelerate key services is becoming more and more Adopted by many institutions and organizations. In order to improve the utilization rate of the GPU, the infrastructure provider usually also allows multiple different types of tasks (online tasks and offline tasks) to share GPU resources, that is, adopts a mixed load operation mode. However, when mixed workloads share the GPU, there will be severe interference in the performance of online tasks since multiple tasks will compete for GPU resources. The fundament...

AI Technical Summary

Problems solved by technology

Based on the method of GPU preemption, although the waiting time of tasks can be reduced to a certain extent, the time overhead of preemption is related to the execution time of GPU kernel tasks.

[0014] To sum up, it can be seen that the hardware-based method needs to modify the GPU hardware structure to solve the performance problem when the load is mixed, and has low operability and poor practicability on the existing GPU equipment; although the software-based method can make online tasks Try to run with priority, but it cannot ensure that when online tasks need additional resources, the corresponding resources can be obtained in time

Therefore, a fine-grained GPU resource management method is needed to effectively control the use of GPU resources by different types of tasks under mixed loads, especially to support task resource quotas and online resource adjustments, so as to meet the requirements of service quality. technical report

Images