GPU task queue management method, system and device
By dividing GPU graphics card resources into multiple computing power units and building priority task queues, dynamically scheduling GPU resources, the problems of uneven resource allocation and low utilization in the existing technology are solved, and efficient management of GPU resource pools are realized and Maximize value utilization.
Patent Information
- Application Number
- CN202410968226.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-18
AI Technical Summary
The existing GPU resource management methods have uneven resource allocation, low utilization rate, low task scheduling efficiency and lack of automatic scaling capacity, which makes it difficult to adapt to different task needs, resulting in low resource waste and scheduling efficiency.
By dividing physical GPU graphics card resources into multiple GPU computing power units, a GPU resource pool is built, and a priority task queue is built based on the running status information of real-time monitoring. The dynamic resource scheduling strategy is used to manage the GPU task queue, and the task parameters and Resource parameters perform optimal task allocation and resource scheduling.
It improves the efficiency of GPU resource management and scheduling, ensures timely acquisition of resources for critical tasks, avoids resource waste, and maximizes the value utilization of GPU resource pool.
Smart Images

Figure CN119003149A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer software technology, and in particular to a GPU task queue management method, system and device. Background Art
[0002] The new generation of artificial intelligence technology based on large language models is developing rapidly. The training, fine-tuning and deployment of large language models require a large amount of graphics processing unit (GPU) computing power. Compared with CPU resources, GPU computing power resources are more expensive and scarce. In large-scale computing environments such as data centers, how to efficiently manage and schedule limited GPU resources has become an urgent problem to be solved. The current GPU resource management methods mainly have the following problems:
[0003] Uneven resource allocation: Traditional resource management methods often use static allocation or manual scheduling, which is difficult to adapt to the dynamic changes in task requirements, resulting in uneven resource allocation. Some tasks are waiting due to insufficient resources, while other resources may be idle.
[0004] Low resource utilization: Due to the lack of effective resource scheduling strategies, the utilization of GPU resources is generally low. Especially when the task load is uneven, some GPUs are idle for a long time, resulting in resource waste.
[0005] Low task scheduling efficiency: Existing task scheduling mechanisms are often based on simple priority or first-come-first-served principles, lacking comprehensive consideration of task characteristics and resource requirements, resulting in low scheduling efficiency and difficulty in meeting the needs of high-performance computing.
[0006] Lack of automatic scaling capabilities: In the face of tasks of different scales and characteristics, the existing GPU resource management method lacks the ability to automatically scale up and down, and cannot dynamically adjust resource allocation according to the actual needs of the task, affecting the execution efficiency of the task and the rational use of resources.
[0007] In order to solve the above problems and improve the management and scheduling efficiency of GPU resources, there is an urgent need for a new GPU task queue implementation mechanism that can achieve automatic expansion and contraction of resources, efficient scheduling, and adapt to the needs of different tasks. Summary of the invention
[0008] The present invention provides a GPU task queue management method, system and device, which improve the management and scheduling efficiency of GPU resources and realize the maximum value utilization of GPU resource pool.
[0009] In a first aspect, the present invention discloses a GPU task queue management method, the method comprising:
[0010] According to the GPU division rules, physical GPU graphics card resources are divided into multiple GPU computing units;
[0011] Based on the GPU computing unit, a GPU resource pool is constructed;
[0012] Monitor the running status information of the GPU resource pool in real time, obtain task parameters in the GPU resource pool according to the running status information, and build a task queue with priority according to the task parameters;
[0013] Based on the resource scheduling strategy, the tasks to be scheduled in the GPU task queue and the GPU computing power unit are dynamically allocated, and the resource scheduling management of the GPU task queue is performed based on the resource parameter information of the GPU computing power unit.
[0014] In some embodiments, the GPU computing power units are differentiated based on resource parameter information, where the resource parameter information includes the GPU graphics card manufacturer name, GPU graphics card type, unit type, GPU computing power quantity and / or GPU graphics memory quantity.
[0015] In some implementations, the resource scheduling strategy includes:
[0016] Determine the optimal execution task and the optimal GPU computing unit according to the task parameters of the task to be scheduled and the resources of the GPU computing unit, and perform task allocation; the task parameters include task priority and the shortest task execution time and waiting time;
[0017] When a new task is received, the new task is inserted into any queue of tasks to be scheduled and queued for scheduling. The GPU computing unit in the current GPU resource pool is periodically queried to see whether there are idle resources available. The optimal execution task and the optimal GPU computing unit are determined based on the task parameters of the task to be scheduled and the resource parameter information of the GPU computing unit.
[0018] In some implementations, performing resource scheduling management on the GPU task queue based on the resource parameter information includes:
[0019] Traverse the currently executing task queue, update the task status that meets the end conditions, and release the corresponding GPU resources to the GPU resource pool;
[0020] Traverse the GPU resource pool, match the idle GPU resources according to the task parameters of the task to be scheduled, allocate the corresponding GPU computing unit, and start the corresponding task process;
[0021] If there are still tasks to be scheduled in the task queue, determine the tasks that meet the end conditions, update the task status that meets the end conditions, and release the corresponding GPU resources to the GPU resource pool;
[0022] The corresponding resources are allocated to the tasks to be scheduled, and the tasks that meet the end conditions are placed in the queue of tasks to be scheduled.
[0023] In some embodiments, the GPU computing power unit includes hard partitioning based on MIG and VTPU technologies, soft partitioning based on CUDA, RoCm, and oneAPI programming frameworks, and resource-sharing computing power units.
[0024] In some implementations, determining the optimal execution task and the optimal GPU computing unit according to the task parameters of the task to be scheduled and the resources of the GPU computing unit also includes:
[0025] Determine the task priority, minimum execution time, waiting time, number of required computing units, and elastic expansion type of the tasks to be scheduled, and determine the matching idle GPU computing units;
[0026] When there are multiple tasks to be scheduled that match the same idle GPU computing unit, the optimal execution task of the optimal GPU computing unit is determined based on the matching distance and priority.
[0027] In some embodiments, the matching distance is
[0028] gpu_distance=(request_core-target_core)*10+(request_memory-target_memory)
[0029] Among them, gpu_distance is the matching distance, request_core is the amount of GPU computing power required for the task to be scheduled, target_core is the amount of idle GPU computing power, request_memory is the memory capacity of the task to be scheduled, and target_core is the idle GPU memory capacity.
[0030] In some implementations, the termination condition is that the task has been completed or the actual running time of the task exceeds a preset running time.
[0031] In a second aspect, the present invention discloses a GPU task queue management system, comprising:
[0032] A GPU resource pool construction module is used to divide the physical GPU graphics card resources into multiple GPU computing units according to the GPU division rule; and to construct a GPU resource pool based on the GPU computing units;
[0033] A GPU task queue construction module, used to monitor the running status information of the GPU resource pool in real time, obtain task parameters in the GPU resource pool according to the running status information, and construct a GPU task queue with priority according to the task parameters;
[0034] The GPU task scheduling module dynamically allocates the tasks to be scheduled in the GPU task queue and the GPU computing power unit based on the resource scheduling strategy, and performs resource scheduling management on the GPU task queue based on the resource parameter information of the GPU computing power unit.
[0035] In a third aspect, the present invention discloses a GPU task queue management device, the device comprising:
[0036] A memory storing executable program code;
[0037] a processor coupled to the memory;
[0038] The processor calls the executable program code stored in the memory to execute a GPU task queue management method as described in any one of the above items.
[0039] Compared with the prior art, the present invention has the following beneficial effects:
[0040] The present invention provides a GPU task queue management method, system and device. Through resource scheduling strategy and task priority management, GPU queues can be managed. GPU resources can be dynamically allocated according to actual task requirements, ensuring that key tasks can obtain required resources in a timely manner, thereby improving overall scheduling efficiency, avoiding waste and idleness of resources, improving GPU resource management and scheduling efficiency, and realizing maximum value utilization of GPU resource pool. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 A schematic diagram of the process provided by the present invention;
[0042] Figure 2 A schematic diagram of the structure of the GPU resource pool provided by the present invention;
[0043] Figure 3 This is a schematic diagram of the structure of the GPU task scheduling module provided by the present invention;
[0044] Figure 4 This is a schematic diagram of the relationship between the GPU Driver plugin, GPU Task Operator and Kubernetes components provided by the present invention;
[0045] Figure 5 A schematic diagram of the process of GPU task scheduling provided by the present invention. DETAILED DESCRIPTION
[0046] For better understanding and implementation, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0047] The terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products or apparatus.
[0048] The embodiment of the present invention discloses a GPU task queue management method, which improves the management and scheduling efficiency of GPU resources and realizes the maximum value utilization of the GPU resource pool.
[0049] Specifically, the method comprises the following steps:
[0050] Step S1: divide the physical GPU graphics card resources into multiple GPU computing units according to the GPU division rule.
[0051] In traditional practice, the industry generally used a complete graphics card as a scheduling unit, that is, only a complete graphics card could be scheduled during task scheduling, and each graphics card performed the corresponding task. This scheduling method generally leads to low utilization of GPU resources, especially when the task load is uneven, some GPUs are idle for a long time, resulting in resource waste.
[0052] Therefore, in this application, according to the GPU division rules, the physical GPU graphics card resources are divided into multiple GPU computing units of different specifications, and a schedulable GPU resource pool is formed based on these computing units.
[0053] Specifically, the GPU computing unit can be implemented through GPU hard slicing technology (Nvidia MIG, AMD MxGPU), or by intercepting the GPU computing power and memory call API at the software level, such as interception based on the kernel graphics driver level, interception based on CUDA driver and SDK, etc.
[0054] The GPU computing units divided in the above way are all regarded as a basic computing unit, which is uniformly managed in the GPU resource pool by distinguishing categories and quantities. It should be noted that the GPU in this application also includes NPU and TPU devices, such as Huawei Ascend NPU and Google TPU.
[0055] In this application, the scheduling in GPU task queue management is based on the GPU computing unit as the granularity. Different types of GPU computing units are suitable for different types of tasks. The CPU computing units are named in a unified and standardized way, which can greatly simplify the matching algorithm between tasks and resources and simplify the complexity of GPU resource elastic scheduling.
[0056] GPU computing units are distinguished based on resource parameter information, which includes the manufacturer name of the GPU graphics card, the GPU graphics card type, the unit type, the GPU computing power quantity and / or the GPU video memory quantity. For example, the resource name of the GPU computing unit of Nvidia's A100 graphics card divided by MIG slicing technology can be nvidia / a100.mig.4c.20gb, where 4c means that the computing unit has 4 GPU computing cores, and 20gb means that the available video memory of the computing unit is 20gb; the resource name of the GPU computing unit is huawei / ascend910.vnpu.4c.20gb, which means that it is Huawei's Ascend 910NPU card, a computing unit divided by vnpu slicing technology. Similarly, if the resource name of the GPU computing unit is soft.4c.10gb, it means a 4-core 10gb GPU computing unit implemented by the graphics card driver or CUDA call interception. The naming of shared.10c.80gb indicates a shared computing power and video memory GPU computing unit. The tasks scheduled to this computing unit share the computing power and video memory of the same unit.
[0057] Furthermore, the GPU computing unit can automatically and elastically scale up and down. The scaling dimensions include the scaling of the GPU computing unit specifications and the scaling of the number of GPU computing units. In addition, the GPU task also needs to set its own priority level, the shortest duration of GPU resource occupation for each scheduling operation, and the maximum cumulative duration of GPU resource occupation, and other important parameters.
[0058] Step S2: constructing a GPU resource pool based on the GPU computing unit;
[0059] The GPU resource pool has a two-level structure. The first level is the server nodes that contain GPUs in the cluster. The second level is the various GPU computing units owned by each node. Taking a cluster of three nodes as an example, the corresponding GPU resource pool structure is as follows: Figure 2 shown.
[0060] Step S3: monitor the running status information of the GPU resource pool in real time, obtain task parameters in the GPU resource pool according to the running status information, and construct a task queue with priority according to the task parameters.
[0061] Tasks include GPU tasks in different states, including tasks to be scheduled, tasks in progress, and tasks that have ended. For tasks that require GPU resources, the number of GPU computing units required will be displayed in the task parameters. The GPU resource pool will provide all types of GPU computing units currently supported for selection by the tasks to be scheduled. According to the task parameter information and the resource parameter information of the GPU computing unit, select the most appropriate GPU computing unit and the corresponding number of computing units. In addition, all tasks also need to set their own priority, the shortest duration of GPU resources occupied by each scheduling operation, the maximum cumulative duration of GPU resources occupied, and other task parameters.
[0062] When applying for resources, a GPU task needs to specify the name of the required GPU computing unit and the corresponding number of computing units.
[0063] For example, it indicates applying for 2 GPU units of nvidia / a100.mig.4c.20gb specification:
[0064] gpu-class=nvidia / a100.mig.4c.20gb
[0065] gpu-count=2
[0066] At the same time, you also need to specify the following three parameters related to task scheduling:
[0067] task-priority=100, priority, the larger the value, the higher the priority
[0068] task-one-run-period=3600, the minimum duration of GPU resources occupied by each scheduled run, in seconds
[0069] task-one-run-total=360000, the maximum cumulative time limit for occupying GPU resources. If this time is exceeded, the task needs to be terminated.
[0070] In this application, the GPU task queue includes the task queue to be scheduled and the task queue being executed. The user creates a GPU task through the interactive interface or API call and submits it to the GPU task scheduling module. The structure of the GPU task scheduling module Figure 3As shown in the figure, after the task receiver receives the GPU task, the task enters the waiting task queue of the GPU task scheduling module and waits for scheduling. The executing task queue is also set with a scheduling daemon thread to complete the specific task scheduling logic. The scheduling daemon thread manages the waiting task queue, the running task queue, and the GPU resource pool.
[0071] Step S4: dynamically allocate the tasks to be scheduled in the GPU task queue and the GPU computing power unit based on the resource scheduling strategy, and perform resource scheduling management on the GPU task queue based on the resource parameter information of the GPU computing power unit.
[0072] The scheduling daemon thread of the GPU task scheduling module will dynamically allocate tasks to be scheduled in the GPU task queue and the GPU computing unit to improve the efficiency of GPU resources. The resource scheduling strategy includes:
[0073] Determine the optimal execution task and the optimal GPU computing unit according to the task parameters of the task to be scheduled and the resources of the GPU computing unit, and perform task allocation; the task parameters include task priority and the shortest task execution time and waiting time;
[0074] When a new task is received, the new task is inserted into any queue of tasks to be scheduled and queued for scheduling. The GPU computing unit in the current GPU resource pool is periodically queried to see whether there are idle resources available. The optimal execution task and the optimal GPU computing unit are determined based on the task parameters of the task to be scheduled and the resource parameter information of the GPU computing unit.
[0075] In the above process, the optimal execution task and the optimal GPU computing unit are determined according to the task parameters of the task to be scheduled and the resources of the GPU computing unit, which also includes:
[0076] Determine the task priority, minimum execution time, waiting time, number of required computing units, and elastic expansion type of the tasks to be scheduled, and determine the matching idle GPU computing units;
[0077] When there are multiple tasks to be scheduled that match the same idle GPU computing unit, the optimal execution task of the optimal GPU computing unit is determined based on the matching distance and priority.
[0078] Performing resource scheduling management on the GPU task queue based on the resource parameter information includes:
[0079] Traverse the currently executing task queue, update the task status that meets the end conditions, and release the corresponding GPU resources to the GPU resource pool;
[0080] Traverse the GPU resource pool, match the idle GPU resources according to the task parameters of the task to be scheduled, allocate the corresponding GPU computing unit, and start the corresponding task process;
[0081] If there are still tasks to be scheduled in the task queue, determine the tasks that meet the end conditions, update the task status that meets the end conditions, and release the corresponding GPU resources to the GPU resource pool;
[0082] The corresponding resources are allocated to the tasks to be scheduled, and the tasks that meet the end conditions are placed in the queue of tasks to be scheduled.
[0083] The specific logic of the GPU task scheduling module is explained below in conjunction with a specific embodiment:
[0084] 1. Traverse the currently running task queue, set the task status to ended for tasks that meet the end conditions, that is, completed tasks and tasks that have timed out (the cumulative running time exceeds task-one-run-total), and release the corresponding GPU resources to the resource pool;
[0085] 2. Traverse the GPU resource pool, find the tasks to be scheduled that match the idle GPU resources, allocate the corresponding GPU computing units to them, and start the corresponding task process.
[0086] 3. If there are still tasks to be scheduled in the current task queue, search for task A whose actual running time is greater than task-one-run-period in the currently running task queue. If the GPU resources occupied by task A satisfy a task B to be scheduled, force task A to end, update the task status of task A, allocate the corresponding resources to task B, start the corresponding task process, and put task A into the task queue to be scheduled and wait for the next round of scheduling.
[0087] In the above task scheduling process, the focus is on how to determine the optimal execution task and the optimal GPU computing unit based on the task parameters of the task to be scheduled and the resources of the GPU computing unit.
[0088] Assume that the current idle GPU resources are two GPU computing units with the specification of huawei / ascend910.vnpu.4c.20gb on Node A.
[0089] The tasks to be scheduled for huawei / ascend910.vnpu are as follows:
[0090] Task task1 requires 3 blocks of huawei / ascend910.vnpu.4c.20gb, which can be elastically expanded or reduced Task task2 requires 1 block of huawei / ascend910.vnpu.4c.20gb, which can be elastically expanded or reduced Task task3 requires 2 blocks of huawei / ascend910.vnpu.4c.30gb, which can be elastically expanded or reduced Task task4 requires 2 blocks of huawei / ascend910.vnpu.5c.20gb, which can be elastically expanded or reduced
[0091] Task task1 requires 3 computing units, which does not meet the conditions of Node A and cannot be scheduled in the GPU resources. However, since tasks task2, task3, and task4 can be elastically expanded and reduced, they meet the conditions of Node A. Since the GPU computing units required by tasks task3 and task4 are closest to the number of computing units actually available, task3 and task4 are given priority. On this basis, their respective matching distances are calculated using the following formula:
[0092] The matching distance is
[0093] gpu_distance=(request_core-target_core)*10+(request_memory-target_memory)
[0094] Among them, gpu_distance is the matching distance, request_core is the amount of GPU computing power required for the task to be scheduled, target_core is the amount of idle GPU computing power, request_memory is the memory capacity of the task to be scheduled, and target_core is the idle GPU memory capacity.
[0095] According to the above matching distance calculation formula, the calculation results are as follows:
[0096] task3_gpu_distance=(4-4)*10+(30-20)=20
[0097] task4_gpu_distance=(5-4)*10+(20-20)=50
[0098] Therefore, task task3 is scheduled first.
[0099] Furthermore, in some cases, if the matching distances of task 3 or task 4 are the same, the following logic is used for scheduling:
[0100] 1. Prioritize the tasks that have not been scheduled in the previous task scheduling cycle and have been waiting the longest;
[0101] 2. Priority scheduling has high priority;
[0102] That is, when task task3 was not scheduled in the last task scheduling cycle, and task task4 did not exist in the last task cycle, or there was a last task cycle but the waiting time did not exceed that of task task3, even if task task4 has a higher priority, task task3 will still be scheduled first.
[0103] When both task task3 and task task4 have not been scheduled in the previous task scheduling cycle and have the same waiting time, and task task4 has a higher priority, task task4 will be scheduled first.
[0104] Through the above two logics, it is ensured that there will be no low-priority tasks that are always in the waiting state. If they have not been scheduled in the previous scheduling cycle, they will be scheduled first in this round of scheduling cycle. In addition, it is also ensured that high-priority tasks are scheduled first in the same round of scheduling cycle.
[0105] Furthermore, considering that most current digital transformation applications use cloud-native architecture platforms, and mainstream cloud-native architecture platforms are implemented based on Kubernetes, this application will take Kubernetes as an example to illustrate the specific implementation process and pay attention to details.
[0106] First, implement a specific Kubernetes GPU Driver plugin, which is the GPU driver plugin for Kubernetes. This plugin runs on each Kubernetes node with GPU resources, divides the GPU graphics card on the current node according to the requirements of the GPU computing unit, and injects it into the Kubernetes API server.
[0107] As a standard kubernetes container resource, the GPU computing unit is allocated and scheduled by kubernetes. At the same time, the allocated resources and remaining resources can also be queried through the kubernetes API. The GPU task scheduling module is implemented with the kubernetes Operator mechanism. It is specifically implemented as the GPU Task Operator, which is a mechanism for extending the Kubernetes API. It allows developers to use custom resources to manage more complex applications. The Operator usually consists of one or more custom resource definitions (CRDs) and an application running in a Kubernetes cluster. This application is responsible for implementing the logic of custom resources. The relationship between the GPU Driver plugin, GPU Task Operator and kubernetes components is as follows: Figure 4 shown.
[0108] After the user writes a YAML file that defines a GPU task (GpuJob), it is submitted to the API Server through the Kubernetes command line tool or API. The GPU Task Operator monitors in real time that a new task to be scheduled has been generated, parses the properties and task parameters of the GPU task, generates the corresponding Task, and puts it into the queue of tasks to be scheduled. When the Task is successfully scheduled, the GPU Task Operator generates the corresponding Pod instance and writes the corresponding GPU resource information into the resource application attribute request of the Pod. Subsequently, the Pod instance is scheduled by Kubernetes to a corresponding node for execution. The whole process is as follows: Figure 5 shown.
[0109] The CRD object GpuJob that describes the GPU task is modeled as a GpuTask object when it enters the GPU Task Operator. GpuTask contains some status data related to task scheduling, some of which comes from its corresponding Pod instance. For example, if the Pod is in the pending, Running, Succeeded, or Failed state, the GPU Task Operator will automatically synchronize the GpuTask with the corresponding Pod instance status to complete the logic of resource release and rescheduling.
[0110] Furthermore, the present application may also be implemented directly in a Docker container system, which is not limited here.
[0111] Based on the same inventive concept, the present application also provides a GPU task queue management system, including:
[0112] A GPU resource pool construction module is used to divide the physical GPU graphics card resources into multiple GPU computing units according to the GPU division rule; and to construct a GPU resource pool based on the GPU computing units;
[0113] A GPU task queue construction module, used to monitor the running status information of the GPU resource pool in real time, obtain task parameters in the GPU resource pool according to the running status information, and construct a GPU task queue with priority according to the task parameters;
[0114] The GPU task scheduling module dynamically allocates the tasks to be scheduled in the GPU task queue and the GPU computing power unit based on the resource scheduling strategy, and performs resource scheduling management on the GPU task queue based on the resource parameter information of the GPU computing power unit.
[0115] The present invention provides a GPU task queue management method, which manages the GPU queue through resource scheduling strategy in combination with task priority, can dynamically allocate GPU resources according to the actual needs of the tasks, ensure that key tasks can obtain the required resources in a timely manner, improve the overall scheduling efficiency, avoid waste and idleness of resources, improve the management and scheduling efficiency of GPU resources, and realize the maximum value utilization of the GPU resource pool.
[0116] The present application also provides a GPU task queue management device, the device comprising:
[0117] A memory storing executable program code;
[0118] a processor coupled to the memory;
[0119] The processor calls the executable program code stored in the memory to execute the GPU task queue management method as described above.
[0120] An embodiment of the present invention further provides a non-transitory machine-readable storage medium, on which an executable program is stored. When the executable program is executed by a processor, the processor executes the method provided in the above embodiment.
[0121] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the described method.
[0122] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the described method.
[0123] The embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, and may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.
[0124] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.
[0125] Finally, it should be noted that the methods and systems disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features therein may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A GPU task queue management method, characterized in that: The method comprises: According to the GPU division rules, physical GPU graphics card resources are divided into multiple GPU computing units; Based on the GPU computing unit, a GPU resource pool is constructed; Monitor the running status information of the GPU resource pool in real time, obtain task parameters in the GPU resource pool according to the running status information, and build a task queue with priority according to the task parameters; The tasks to be scheduled in the GPU task queue and the GPU computing power unit are dynamically allocated based on the resource scheduling strategy, and the GPU task queue is managed for resource scheduling based on the resource parameter information of the GPU computing power unit.
2. A GPU task queue management method according to claim 1, characterized in that: The GPU computing power unit is distinguished according to resource parameter information, and the resource parameter information includes the manufacturer name of the GPU graphics card, the GPU graphics card type, the unit type, the GPU computing power quantity and / or the GPU video memory quantity.
3. A GPU task queue management method according to claim 2, characterized in that: The resource scheduling strategy includes: Determine the optimal execution task and the optimal GPU computing unit according to the task parameters of the task to be scheduled and the resources of the GPU computing unit, and perform task allocation; the task parameters include task priority and the shortest task execution time and waiting time; When a new task is received, the new task is inserted into any queue of tasks to be scheduled and queued for scheduling. The GPU computing unit in the current GPU resource pool is periodically queried to see whether there are idle resources available. The optimal execution task and the optimal GPU computing unit are determined based on the task parameters of the task to be scheduled and the resource parameter information of the GPU computing unit.
4. A GPU task queue management method according to claim 1, characterized in that: Performing resource scheduling management on the GPU task queue based on the resource parameter information includes: Traverse the currently executing task queue, update the task status that meets the end conditions, and release the corresponding GPU resources to the GPU resource pool; Traverse the GPU resource pool, match the idle GPU resources according to the task parameters of the task to be scheduled, allocate the corresponding GPU computing unit, and start the corresponding task process; If there are still tasks to be scheduled in the task queue, determine the tasks that meet the end conditions, update the task status that meets the end conditions, and release the corresponding GPU resources to the GPU resource pool; The corresponding resources are allocated to the tasks to be scheduled, and the tasks that meet the end conditions are placed in the queue of tasks to be scheduled.
5. A GPU task queue management method according to any one of claims 1 to 4, characterized in that: GPU computing power units include hard partitioning based on MIG and VTPU technologies, soft partitioning based on CUDA, RoCm, and oneAPI programming frameworks, and resource-sharing computing power units.
6. A GPU task queue management method according to claim 4, characterized in that: Determining the optimal execution task and the optimal GPU computing unit according to the task parameters of the task to be scheduled and the resources of the GPU computing unit, further comprising: Determine the task priority, minimum execution time, waiting time, number of required computing units, and elastic expansion type of the tasks to be scheduled, and determine the matching idle GPU computing units; When there are multiple tasks to be scheduled that match the same idle GPU computing unit, the optimal execution task of the optimal GPU computing unit is determined based on the matching distance and priority.
7. A GPU task queue management method according to claim 6, characterized in that: The matching distance is gpu_distance=(request_core-target_core)*10+(request_memory-target_memory) Among them, gpu_distance is the matching distance, request_core is the amount of GPU computing power required for the task to be scheduled, target_core is the amount of idle GPU computing power, request_memory is the memory capacity of the task to be scheduled, and target_core is the idle GPU memory capacity.
8. A GPU task queue management method according to claim 4, characterized in that: The termination condition is that the task has been completed or the actual running time of the task exceeds the preset running time.
9. A GPU task queue management system, characterized in that: include: A GPU resource pool construction module is used to divide the physical GPU graphics card resources into multiple GPU computing units according to the GPU division rule; and to construct a GPU resource pool based on the GPU computing units; A GPU task queue construction module, used to monitor the running status information of the GPU resource pool in real time, obtain task parameters in the GPU resource pool according to the running status information, and construct a GPU task queue with priority according to the task parameters; The GPU task scheduling module dynamically allocates the tasks to be scheduled in the GPU task queue and the GPU computing power unit based on the resource scheduling strategy, and performs resource scheduling management on the GPU task queue based on the resource parameter information of the GPU computing power unit.
10. A GPU task queue management device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute a GPU task queue management method as described in any one of claims 1-8.
Citation Information
Patent Citations
GPU computing resource management method and device, electronic equipment and readable storage medium
CN115562878A
Container scheduling system and scheduling method for sharing GPU (Graphics Processing Unit) resources
CN116069481A
GPU resource scheduling method and device
CN116069496A
AI computing power automatic scheduling method and device and medium
CN116991585A
Dynamic scheduling method and terminal based on resource application
CN117032974A
Cited By
Computing power resource scheduling method and device
CN119739539A
Intelligent computing center model development method and device oriented to popularity
CN119902904A
Intelligent Computing Center Model Development Method and Device for Inclusive Computing Power
CN119902904B
GPU resource virtualization computing power scheduling method and device
CN120196421A
Task execution method, system and device, storage medium and program product
CN120295798A
