Workflow-based GPU scheduling method, device, equipment and medium

Abstract

The application relates to the computer technical field, and particularly provides a GPU scheduling method and device based on a work flow, equipment and a medium, aiming at solving the problem that when GPU memory is insufficient, tasks in a work node and other tasks in a message queue cannot be executed in time, thereby causing task timeout. For this purpose, the GPU scheduling method based on the work flow comprises the following steps: acquiring a plurality of idle GPU queues; screening a target GPU queue corresponding to the memory requirement of the work node from the plurality of idle GPU queues; acquiring a target task from a task queue, and processing the target task by using the target GPU queue.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically providing a workflow-based GPU scheduling method, apparatus, device, and medium. Background Technology

[0002] When performing tasks such as artificial intelligence data annotation, model inference, image processing, and video analysis, it is usually necessary to use the computer's graphics processing unit (GPU) for computation to improve data processing efficiency.

[0003] In traditional distributed workflow or task queue systems, the scheduler packages tasks and pushes them into a message queue based on resource availability. Worker nodes then actively pull a task from the message queue, prepare the execution environment according to the task, copy data from memory to GPU memory, and call the GPU for computation. However, when GPU memory is insufficient, tasks on worker nodes cannot be executed, and other tasks in the message queue cannot be executed in a timely manner, leading to task timeouts.

[0004] Accordingly, there is a need in the field for a new workflow-based GPU scheduling method to address the above problems. Summary of the Invention

[0005] In order to overcome the above-mentioned defects, this application is made to provide a solution or at least a partial solution to the problem in the prior art that when GPU memory is insufficient, tasks in the worker node and other tasks in the message queue cannot be executed in a timely manner, thus causing task timeouts.

[0006] Firstly, this application provides a workflow-based GPU scheduling method, including:

[0007] Obtain multiple idle GPU queues;

[0008] Select the target GPU queue that corresponds to the memory requirements of the working node from the plurality of idle GPU queues;

[0009] The target task is retrieved from the task queue and processed using the target GPU queue.

[0010] In one technical solution of the above workflow-based GPU scheduling method, obtaining multiple idle GPU queues includes:

[0011] Get the video memory capacity;

[0012] The video memory capacity is divided into multiple GPU queues;

[0013] Obtain the occupancy status of the multiple GPU queues;

[0014] The GPU queues that are currently idle are designated as the idle GPU queues.

[0015] In one technical solution of the above-mentioned workflow-based GPU scheduling method, the step of selecting a target GPU queue from the idle GPU queue that corresponds to the memory requirements of the worker node includes:

[0016] Iterate through the multiple idle GPU queues;

[0017] Obtain the video memory of the multiple idle GPU queues;

[0018] The memory size of each of the multiple idle GPU queues is compared with the memory requirement of the worker node to obtain the comparison result;

[0019] The idle GPU queues whose comparison results are greater than or equal to are combined to obtain an idle GPU queue set.

[0020] Select an idle GPU queue from the set of idle GPU queues as the target GPU queue.

[0021] In one technical solution of the above workflow-based GPU scheduling method, selecting an idle GPU queue as the target GPU queue from the set of idle GPU queues includes:

[0022] The idle GPU queues in the idle GPU queue set are sorted in descending order of video memory to obtain the sorted idle GPU queues;

[0023] The last idle GPU queue in the sorted idle GPU queue is taken as the target GPU queue.

[0024] In one technical solution of the above-mentioned workflow-based GPU scheduling method, before obtaining the target task from the task queue and processing the target task using the target GPU queue, the method further includes:

[0025] Obtain GPU status information;

[0026] The state information is detected to obtain the detection result;

[0027] The test result was determined to be normal.

[0028] In one technical solution of the above-mentioned workflow-based GPU scheduling method, after detecting the state information and obtaining the detection result, the method further includes:

[0029] If the detection result is determined to be abnormal, the GPU will be marked as unusable.

[0030] The worker nodes corresponding to the GPU queue in the GPU are stopped.

[0031] In one technical solution of the above-mentioned workflow-based GPU scheduling method, the method further includes:

[0032] Obtain the execution progress of the target task;

[0033] If the execution progress is determined to be complete, the target GPU queue is released.

[0034] Secondly, this application provides a GPU scheduling device, comprising:

[0035] The acquisition module is used to acquire multiple idle GPU queues;

[0036] A filtering module is used to filter target GPU queues from the plurality of idle GPU queues that correspond to the memory requirements of the working node.

[0037] The processing module is used to obtain the target task from the task queue and process the target task using the target GPU queue.

[0038] Thirdly, this application provides a GPU scheduling device, including at least one processor and a memory; wherein,

[0039] The memory stores computer-executed instructions;

[0040] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method described in any one of the first aspects.

[0041] Fourthly, this application provides a computer-readable storage medium storing a plurality of program codes adapted to be loaded and run by a processor to perform the method described in any one of the first aspects.

[0042] This application provides a workflow-based GPU scheduling method, apparatus, device, and medium. The method specifically includes: acquiring multiple idle GPU queues; selecting a target GPU queue from the multiple idle GPU queues that corresponds to the memory requirements of the worker node; acquiring a target task from a task queue; and processing the target task using the target GPU queue, thereby ensuring that the task is executed in a timely manner. Attached Figure Description

[0043] The disclosure of this application will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. Furthermore, similar numbers in the drawings are used to denote similar components, wherein:

[0044] Figure 1 This is a flowchart illustrating a workflow-based GPU scheduling method according to an embodiment of this application.

[0045] Figure 2 This is a flowchart illustrating a second embodiment of a workflow-based GPU scheduling method provided in this application.

[0046] Figure 3 This is a flowchart illustrating a third embodiment of a workflow-based GPU scheduling method provided in this application.

[0047] Figure 4 This is a flowchart illustrating a fourth embodiment of a workflow-based GPU scheduling method provided in this application.

[0048] Figure 5 This is a flowchart illustrating a fifth embodiment of a workflow-based GPU scheduling method provided in this application.

[0049] Figure 6 This is a flowchart illustrating a sixth embodiment of a workflow-based GPU scheduling method provided in this application.

[0050] Figure 7 This is a flowchart illustrating embodiment seven of a workflow-based GPU scheduling method provided in this application.

[0051] Figure 8 This is a schematic diagram of the structure of a GPU scheduling device provided in an embodiment of this application;

[0052] Figure 9 This is a schematic diagram of the structure of a GPU scheduling device provided in an embodiment of this application.

[0053] List of reference numerals in the attached diagram:

[0054] 11: Acquisition module; 12: Filtering module; 13: Processing module; 21: Processor; 22: Memory. Detailed Implementation

[0055] Some embodiments of this application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of this application and are not intended to limit the scope of protection of this application.

[0056] In the description of this application, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and can also include software components, such as program code, or a combination of software and hardware. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Non-transitory computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular terms "a" or "this" can also include plural forms.

[0057] When performing tasks such as AI data annotation, model inference, image processing, and video analysis, GPUs are typically required to process these tasks. Specifically, worker nodes usually pull tasks from a queue first, and then attempt to acquire GPU resources within the task. When the GPU's video memory is insufficient, the task in the worker node cannot be executed by the GPU, causing the task to wait for the GPU to complete its current task before it can be processed, thus leading to task timeouts.

[0058] Based on this, in order to solve the above-mentioned technical problems, the technical concept of this application is to provide a new workflow-based GPU scheduling method to achieve timely task processing.

[0059] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0060] Figure 1 This is a flowchart illustrating an embodiment of a workflow-based GPU scheduling method provided in this application. Figure 1 As shown, specifically, the method includes:

[0061] Step S101: Obtain multiple idle GPU queues.

[0062] In this embodiment, the GPU's video memory is divided into multiple GPU queues based on the task's video memory requirements. The GPU queues that are not executing tasks are designated as idle GPU queues.

[0063] Step S102: Select the target GPU queue from multiple idle GPU queues that corresponds to the memory requirements of the working node.

[0064] In this embodiment, each working node has a set video memory requirement. When a working node receives a task, if the video memory required by the task is greater than or equal to the video memory requirement, the working node will process the task; otherwise, it will not process the task.

[0065] Therefore, the video memory of the idle GPU queue is compared with the video memory requirements of the worker nodes, and the GPU queues with video memory greater than or equal to the video memory requirements are selected from the idle GPU queues as the target GPU queues.

[0066] Step S103: Obtain the target task from the task queue and process the target task using the target GPU queue.

[0067] In this embodiment, after determining the target GPU queue that satisfies the work node, the target task is obtained from the task queue, and the target GPU queue is used to perform calculations on the target task.

[0068] In this embodiment, multiple idle GPU queues are obtained; a target GPU queue corresponding to the memory requirements of the worker node is selected from the multiple idle GPU queues; a target task is obtained from the task queue, and the target task is processed using the target GPU queue. Compared with the prior art, where tasks in the worker node and other tasks in the message queue cannot be executed in time when GPU memory is insufficient, thus causing task timeout, this application first selects the target GPU queue corresponding to the memory requirements of the worker node from multiple idle GPU queues, then obtains the target task from the task queue, and processes the target task using the target GPU queue. Only when it is determined that the GPU queue corresponding to the target task is idle will the target task be obtained and processed using the target GPU queue, avoiding the suspension of processing due to insufficient GPU memory, thereby ensuring that the task is executed in time.

[0069] Figure 2 This is a flowchart illustrating a second embodiment of a workflow-based GPU scheduling method provided in this application. Figure 2 As shown, the specific implementation of step S101 includes:

[0070] Step S201: Obtain the video memory capacity.

[0071] In this embodiment, after the worker node starts up, the local GPU is detected to obtain the GPU's video memory capacity.

[0072] Step S202: Divide the video memory capacity to obtain multiple GPU queues.

[0073] In this embodiment, the video memory capacity is divided according to the video memory size required by the task, resulting in multiple GPU queues.

[0074] For example, the GPU queue can have 2GB, 4GB, 8GB, and 12GB of video memory. Each GPU queue is also bound to a resource slot provider and the amount of video memory for that resource slot provider is declared.

[0075] Step S203: Obtain the occupancy status of multiple GPU queues.

[0076] In this embodiment, the running status of multiple GPU queues is detected to obtain the GPU queue occupancy status.

[0077] Step S204: Designate the GPU queues that are currently idle as the idle GPU queues.

[0078] In this embodiment, the video memory capacity is obtained; the video memory capacity is divided to obtain multiple GPU queues; the occupancy status of multiple GPU queues is obtained; and the GPU queues with idle occupancy status are designated as idle GPU queues. By dividing the GPU video memory into multiple GPU queues and selecting idle GPU queues from them, the utilization rate of the GPU can be improved.

[0079] Figure 3 This is a flowchart illustrating a third embodiment of a workflow-based GPU scheduling method provided in this application. Figure 3 As shown, the specific implementation of step S102 includes:

[0080] Step S301: Traverse multiple idle GPU queues.

[0081] Step S302: Obtain the video memory of multiple idle GPU queues.

[0082] In this embodiment, multiple idle GPU queues are traversed one by one, and the video memory of multiple idle GPU queues is obtained during the traversal.

[0083] Step S303: Compare the memory size of each of the multiple idle GPU queues with the memory requirement of the worker node to obtain the comparison result.

[0084] In this embodiment, during the traversal, the memory of the idle GPU queue is compared with the memory requirements of the worker nodes to obtain the comparison result.

[0085] Step S304: Combine the idle GPU queues whose comparison results are greater than or equal to obtain an idle GPU queue set.

[0086] Step S305: Select an idle GPU queue from the set of idle GPU queues as the target GPU queue.

[0087] In this embodiment, if there are multiple idle GPU queues in the set of idle GPU queues, the idle GPU queue with the best running status is selected as the target GPU queue.

[0088] In this embodiment, multiple idle GPU queues are traversed; the video memory of multiple idle GPU queues is obtained; the video memory size of each of the multiple idle GPU queues is compared with the video memory requirement of the working node to obtain the comparison result; the idle GPU queues whose comparison result is greater than or equal to the task are combined to obtain a set of idle GPU queues; an idle GPU queue is selected from the set of idle GPU queues as the target GPU queue, thereby obtaining the GPU queue corresponding to the video memory required by the task.

[0089] Figure 4 This is a flowchart illustrating a fourth embodiment of a workflow-based GPU scheduling method provided in this application. Figure 4 As shown, the specific implementation of step S305 includes:

[0090] Step S401: Sort the idle GPU queues in the idle GPU queue set according to the amount of video memory from largest to smallest to obtain the sorted idle GPU queue.

[0091] Step S402: Select the last idle GPU queue in the sorted idle GPU queue as the target GPU queue.

[0092] In this embodiment, since the sorted idle GPU queues are ordered from largest to smallest video memory, the last GPU queue in the sorted idle GPU queues has the smallest video memory. This GPU queue with the smallest video memory is the target GPU queue.

[0093] In this embodiment, the idle GPU queues in the idle GPU queue set are sorted in descending order of video memory to obtain a sorted idle GPU queue. The last idle GPU queue in the sorted idle GPU queue is selected as the target GPU queue. Selecting an idle GPU queue that matches the video memory requirements of the working node as the target GPU queue is sufficient to meet the task execution requirements. Other GPU queues with larger video memory can execute other tasks, thereby improving the utilization of the GPU.

[0094] Figure 5This is a flowchart illustrating embodiment five of a workflow-based GPU scheduling method provided in this application. Figure 5 As shown, specifically, before step S103, the method further includes:

[0095] Step S501: Obtain the GPU status information.

[0096] In this embodiment, the check can be performed before the GPU executes the task, or the GPU can be checked periodically. The GPU's status information may include driver information, device visibility, and memory lookup information.

[0097] Step S502: Detect the status information and obtain the detection result.

[0098] In this embodiment, the status information is compared with a preset normal value corresponding to the status information to obtain a detection result. If the status information is within the preset normal value range, the detection result is normal; otherwise, the detection result is abnormal.

[0099] Step S503: Determine that the test result is normal.

[0100] In this embodiment, the status information of the GPU is obtained; the status information is detected to obtain the detection result; the detection result is determined to be normal, and then the GPU is used to execute the task. In this embodiment, the GPU is detected before it executes the task, and the task is only executed after the detection result is determined to be normal. This can avoid the task from being unable to be executed due to GPU failure, thereby ensuring that the task can be processed in a timely manner.

[0101] Figure 6 This is a flowchart illustrating a sixth embodiment of a workflow-based GPU scheduling method provided in this application. Figure 6 As shown, specifically, after step S502, the method further includes:

[0102] Step S601: If the detection result is determined to be abnormal, then mark the GPU as unavailable.

[0103] In this embodiment, if the detection result is abnormal, the node status of the GPU is obtained and the node status is marked as GPU unavailable.

[0104] Step S602: Stop the worker node corresponding to the GPU queue in the GPU.

[0105] In this embodiment, if the detection result of the GPU is abnormal, the working node corresponding to the GPU queue is stopped, and the unfinished tasks in the working node are transferred to other working nodes for processing.

[0106] In this embodiment, if the detection result is determined to be abnormal, the GPU is marked as unavailable; the working node corresponding to the GPU queue in the GPU is stopped. Thus, when the GPU malfunctions, by stopping the working node corresponding to the GPU and transferring the unfinished tasks of the working node to other working nodes, the tasks can be processed in a timely manner, thereby improving task processing efficiency.

[0107] Figure 7 This is a flowchart illustrating embodiment seven of a workflow-based GPU scheduling method provided in this application. Figure 7 As shown, specifically, after step S103, the method further includes:

[0108] Step S701: Obtain the execution progress of the target task.

[0109] Step S702: If the execution progress is determined to be completed, the target GPU queue is released.

[0110] In this embodiment, if the target task has been completed, the worker node releases the target GPU queue according to the displacement flag in the resource slot supplier. If the task is abnormal or the worker node crashes, the timer is used to automatically reclaim the timed-out allocation to prevent resource leakage.

[0111] In this embodiment, the execution progress of the target task is obtained; if the execution progress is determined to be completed, the target GPU queue is released. After the task is completed, the GPU queue is released in a timely manner so that the GPU queue can execute other tasks, thereby improving the task processing efficiency of the GPU.

[0112] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effect of this application, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of this application.

[0113] Furthermore, this application also provides a GPU scheduling device.

[0114] Figure 8 This is a schematic diagram of a GPU scheduling device provided in an embodiment of this application. Figure 8As shown, the GPU scheduling device in this embodiment mainly includes an acquisition module 11, a filtering module 12, and a processing module 13. In some embodiments, one or more of the acquisition module 11, filtering module 12, and processing module 13 can be combined into a single module. In some embodiments, the acquisition module 11 can be configured to acquire multiple idle GPU queues. The filtering module 12 can be configured to filter target GPU queues corresponding to the memory requirements of the worker node from the multiple idle GPU queues. The processing module 13 can be configured to acquire target tasks from the task queue and process the target tasks using the target GPU queue.

[0115] The aforementioned GPU scheduling mechanism is used for execution Figures 1 to 7 The workflow-based GPU scheduling method embodiments shown are similar in technical principle, the technical problems they solve, and the technical effects they produce. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the GPU scheduling device can be found in the embodiments of the workflow-based GPU scheduling method, which will not be repeated here.

[0116] Those skilled in the art will understand that all or part of the processes in the method of the above-described embodiment can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0117] Furthermore, this application also provides a GPU scheduling device.

[0118] Figure 9 This is a schematic diagram of the structure of a GPU scheduling device provided in an embodiment of this application. Figure 9As shown, the GPU scheduling device includes a processor 21 and a memory 22. The memory 22 can be configured to store a program that executes the workflow-based GPU scheduling method of the above-described method embodiments. The processor 21 can be configured to execute the program stored in the memory, which includes, but is not limited to, a program that executes the workflow-based GPU scheduling method of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The GPU scheduling device can be a control device device comprising various electronic devices.

[0119] Furthermore, this application also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to this application, the computer-readable storage medium can be configured to store a program that executes the workflow-based GPU scheduling method of the above-described method embodiments. This program can be loaded and run by a processor to implement the above-described workflow-based GPU scheduling method. For ease of explanation, only the parts related to the embodiments of this application are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of this application. The computer-readable storage medium can be a storage device device comprising various electronic devices. Optionally, in the embodiments of this application, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0120] Furthermore, it should be understood that since the various modules are only provided to illustrate the functional units of the device described in this application, the physical devices corresponding to these modules may be the processor itself, or a part of the processor's software, hardware, or a combination of both. Therefore, the number of modules shown in the figures is merely illustrative.

[0121] Those skilled in the art will understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principles of this application; therefore, the technical solutions after splitting or combining will fall within the protection scope of this application.

[0122] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A workflow-based GPU scheduling method, characterized in that, include: Obtain multiple idle GPU queues; Select the target GPU queue that corresponds to the memory requirements of the working node from the plurality of idle GPU queues; The target task is retrieved from the task queue and processed using the target GPU queue.

2. The method according to claim 1, characterized in that, The process of acquiring multiple idle GPU queues includes: Get the video memory capacity; The video memory capacity is divided into multiple GPU queues; Obtain the occupancy status of the multiple GPU queues; The GPU queues that are currently idle are designated as the idle GPU queues.

3. The method according to claim 2, characterized in that, The step of selecting a target GPU queue from the idle GPU queue that corresponds to the memory requirements of the worker node includes: Iterate through the multiple idle GPU queues; Obtain the video memory of the multiple idle GPU queues; The memory size of each of the multiple idle GPU queues is compared with the memory requirement of the worker node to obtain the comparison result; The idle GPU queues whose comparison results are greater than or equal to are combined to obtain an idle GPU queue set. Select an idle GPU queue from the set of idle GPU queues as the target GPU queue.

4. The method according to claim 3, characterized in that, Selecting an idle GPU queue from the set of idle GPU queues as the target GPU queue includes: The idle GPU queues in the idle GPU queue set are sorted in descending order of video memory to obtain the sorted idle GPU queues. The last idle GPU queue in the sorted idle GPU queue is taken as the target GPU queue.

5. The method according to claim 1, characterized in that, Before retrieving the target task from the task queue and processing the target task using the target GPU queue, the method further includes: Obtain GPU status information; The state information is detected to obtain the detection result; The test result was determined to be normal.

6. The method according to claim 5, characterized in that, After detecting the state information and obtaining the detection result, the method further includes: If the detection result is determined to be abnormal, the GPU will be marked as unusable. The worker nodes corresponding to the GPU queue in the GPU are stopped.

7. The method according to claim 1, characterized in that, The method further includes: Obtain the execution progress of the target task; If the execution progress is determined to be complete, the target GPU queue is released.

8. A GPU scheduling device, characterized in that, include: The acquisition module is used to acquire multiple idle GPU queues; A filtering module is used to filter target GPU queues from the plurality of idle GPU queues that correspond to the memory requirements of the working node. The processing module is used to obtain the target task from the task queue and process the target task using the target GPU queue.

9. A GPU scheduling device, comprising at least one processor and memory; wherein, The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the method of any one of claims 1 to 7.

Images