GPU (Graphics Processing Unit) scheduling method, device, equipment, medium and product

Abstract

The embodiment of the invention provides a GPU scheduling method and device, equipment, a medium and a product, and relates to the field of big data. The method comprises the steps of receiving a to-be-executed first task at a first moment, obtaining required running time and required video memory of the first task, traversing each moment in a time period from the first moment to the required running time, determining a GPU group meeting the required video memory, equally dividing the required video memory of the first task by each GPU in the GPU group meeting the required video memory, and executing the first task according to the required video memory. If the time period is equal to the required video memory, and the residual video memory of each GPU at the time period is greater than or equal to the uniformly-distributed required video memory, determining the time corresponding to the GPU group containing the minimum number of GPUs in the time period as a target time, and averagely deploying the first task on each GPU in the GPU group containing the minimum number of GPUs at the target time and executing the first task. Through the method, the influence of parallel deceleration is reduced to the minimum, the generation of resource fragments is reduced, the utilization efficiency of the GPU is improved, and the total time for completing all tasks is optimized.

Description

Technical Field

[0001] The present application relates to the field of big data, and in particular to a GPU scheduling method, device, equipment, medium and product. Background Art

[0002] Artificial intelligence, especially machine learning technology represented by deep learning, is gradually maturing. The process of deep learning can be simply divided into two parts, namely training and reasoning. Training refers to the process of generating the deep learning algorithm itself, while reasoning refers to the process of deploying the deep learning algorithm on the reasoning card for calculation and generating results. Nowadays, the reasoning problem of deep learning business is gradually becoming important. The reasoning process of deep learning can be summarized as follows:

[0003] 1. The CPU (Central Processing Unit) copies the deep learning algorithm (i.e. the neural network itself) to the GPU (Graphics Processing Unit) video memory.

[0004] 2. The CPU reads the inference request from the client, unpacks the request content and copies it to the GPU memory. Taking CV (Computer Vision) tasks as an example, the inference request generally contains several images to be calculated, as well as the values ​​of some hyperparameters.

[0005] 3. The GPU reads the deep learning algorithm in the video memory and loads it onto the computing unit.

[0006] 4. The GPU reads the inference request in the video memory and passes it as input to the deep learning algorithm in the computing unit.

[0007] 5. The GPU completes the calculation and returns the result.

[0008] According to the description of the above process, we can find that the GPU's computing unit and video memory are involved in the deep learning reasoning process. Considering that the current servers suitable for deep learning reasoning tasks generally have multiple GPU cards, it is particularly important to arrange each reasoning task to achieve the highest possible computing efficiency.

[0009] In the above task flow, if multi-GPU business deployment is taken into consideration, the problem will be further complicated. On the one hand, how to select a GPU with idle resources to complete business deployment is a difficult problem in itself. On the other hand, multi-GPU distributed scheduling will face the problem of "parallel deceleration", that is, the acceleration ratio of the GPU is not completely equal to the number of GPUs, but will be less than the number of GPUs. In addition, the unreasonable allocation of GPUs will lead to resource fragmentation, which seriously interferes with the operation of the GPU computing cluster, causing a large amount of resources to be wasted and unusable.

[0010] In summary, a GPU scheduling method that reduces the impact of parallel slowdown and reduces resource fragmentation is urgently needed. Summary of the invention

[0011] Based on the above technical problems, the embodiments of the present application provide a GPU scheduling method, device, equipment, medium and product, aiming to conveniently and accurately perform GPU scheduling so that when GPU business deployment is performed, the impact of parallel deceleration on multi-GPU business deployment can be reduced, and the generation of resource fragmentation can be reduced.

[0012] A first aspect of an embodiment of the present application provides a GPU scheduling method, the method comprising:

[0013] Receiving a first task to be performed at a first moment;

[0014] Obtaining the required running time and required video memory of the first task;

[0015] Traversing each moment in a time period from the first moment to the required running time, determining a GPU group that meets the required video memory of the first task, wherein each GPU in the GPU group that meets the required video memory of the first task evenly shares the required video memory of the first task, and the remaining video memory of each GPU at this moment is greater than or equal to the evenly shared required video memory;

[0016] Determine the time corresponding to the GPU group containing the minimum number of GPUs within the time period as the target time;

[0017] At the target time, the first task is evenly deployed on each GPU in the GPU group with the minimum number of GPUs and executed.

[0018] Optionally, the method further comprises:

[0019] In the case where multiple tasks are received at the first moment, determining a task with the largest video memory requirement among the multiple tasks as the first task;

[0020] When multiple tasks are received at the first moment, and there are multiple tasks with the largest video memory requirements among the multiple tasks, any task with the longest required running time among the tasks with the largest video memory requirements is determined as the first task.

[0021] Optionally, before traversing each moment in a time period from the first moment to the required running time to determine a GPU group that meets the required video memory of the first task, the method further includes:

[0022] Obtaining GPU video memory, and determining the ratio of the required video memory of the first task to the GPU video memory as the GPU required quantity of the first task;

[0023] When the number of idle GPUs at the first moment is greater than or equal to the required number of GPUs, at the first moment, the first task is evenly deployed and executed in a first number of the idle GPUs, where the first number is the minimum number greater than the required number of GPUs.

[0024] Optionally, the method further comprises:

[0025] When the number of idle GPUs at the first moment is less than the required number of GPUs, traverse each moment in a time period from the first moment to the required running time to determine whether there is a second moment, where the second moment is a moment when the number of idle GPUs is greater than or equal to the required number of GPUs;

[0026] In the case where the second moment exists, the first task is evenly deployed on a second number of idle GPUs, where the second number is a minimum number greater than the required number of GPUs;

[0027] Traversing each moment in a time period from the first moment to the required running time to determine a GPU group that meets the required video memory of the first task, including:

[0028] In the case that the second moment does not exist, each moment in a time period from the first moment to the required running time is traversed to determine a GPU group that meets the required video memory of the first task.

[0029] Optionally, when the second moment exists, the first task is evenly deployed on each idle GPU, and the method includes:

[0030] In the case where the second moment exists, the first task is evenly divided and deployed in the task queues of the second number of idle GPUs;

[0031] At the second moment, the evenly distributed tasks are executed by the second number of idle GPUs.

[0032] Optionally, the method further comprises:

[0033] If there is no GPU group that meets the required video memory of the first task within the time period, a new first task is obtained to deploy and execute the new first task.

[0034] A second aspect of an embodiment of the present application provides a GPU scheduling device, the device comprising:

[0035] A first task receiving module, used for receiving a first task to be executed at a first moment;

[0036] An information acquisition module, used to acquire the required running time and required video memory of the first task;

[0037] A GPU group determination module is used to traverse each moment in a time period from the first moment to the required running time, and determine a GPU group that meets the required video memory of the first task, wherein each GPU in the GPU group that meets the required video memory of the first task evenly shares the required video memory of the first task, and the remaining video memory of each GPU at this moment is greater than or equal to the evenly shared required video memory;

[0038] A target time determination module, used to determine the time corresponding to the GPU group containing the minimum number of GPUs within the time period as the target time;

[0039] The first task deployment module is used to evenly deploy and execute the first task on each GPU in the GPU group with the minimum number of GPUs at the target time.

[0040] Optionally, the first task receiving module further includes:

[0041] A first determining submodule is configured to, when receiving multiple tasks at the first moment, determine a task with the largest video memory requirement among the multiple tasks as the first task;

[0042] The second determination submodule is used to, when multiple tasks are received at the first moment and there are multiple tasks with the largest video memory requirements among the multiple tasks, determine any task with the longest required running time among the tasks with the largest video memory requirements as the first task.

[0043] Optionally, the GPU scheduling device further includes:

[0044] A GPU requirement quantity determination module, used for obtaining GPU video memory, and determining a ratio of the required video memory of the first task to the GPU video memory as the GPU requirement quantity of the first task;

[0045] A first quantity task deployment module is used to evenly deploy and execute the first task in a first number of idle GPUs at the first moment when the number of idle GPUs at the first moment is greater than or equal to the required number of GPUs, and the first number is a minimum number greater than the required number of GPUs.

[0046] Optionally, the GPU scheduling device further includes:

[0047] A second moment determination module is configured to, when the number of idle GPUs at the first moment is less than the required number of GPUs, traverse each moment in a time period from the first moment to the required running time to determine whether there is a second moment, where the second moment is a moment when the number of idle GPUs is greater than or equal to the required number of GPUs;

[0048] A second number of task deployment module, configured to deploy the first task on a second number of idle GPUs on average when the second moment exists, where the second number is a minimum number greater than the required number of GPUs;

[0049] Optionally, the GPU scheduling device further includes:

[0050] A second number of task deployment module, configured to evenly divide and deploy the first task in respective task queues of the second number of idle GPUs when the second moment exists;

[0051] The second-moment task execution module is used to execute the evenly distributed tasks through the second number of idle GPUs at the second moment.

[0052] Optionally, the GPU scheduling device further includes:

[0053] The task replacement module is used to obtain a new first task in the case that there is no GPU group that meets the required video memory of the first task within the time period, so as to deploy and execute the new first task.

[0054] A third aspect of an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the GPU scheduling method of the first aspect of the embodiment of the present application is implemented.

[0055] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the GPU scheduling method of the first aspect of the embodiment of the present application is implemented.

[0056] A fifth aspect of an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the GPU scheduling method of the first aspect of the embodiment of the present application.

[0057] Through the GPU scheduling method of the embodiment of the present application, after receiving the first task, it is first determined whether it can be deployed on an idle GPU by deploying it on the idle GPU or waiting for a time period of required running time. If not, each moment in the time period is traversed to determine whether there is a moment when the remaining video memory of each GPU in the GPU group is greater than or equal to the required video memory evenly distributed to the first task. The moment with the smallest number of GPUs among all moments that meet the conditions is selected and determined as the target moment. At the target moment, the first task is evenly deployed on each GPU in the corresponding GPU group for execution.

[0058] In this application, the scheduling and deployment of GPUs in related technologies are often not generalizable, and do not take into account the problems such as parallel deceleration that will occur when multiple GPUs execute tasks. It is proposed to select as few GPUs as possible when considering multiple GPUs to execute the same task at the same time, so as to minimize the impact of parallel deceleration. In addition, it is also proposed that when the number of idle GPUs is insufficient to complete the task, the idle parts of multiple non-idle GPUs are selected and evenly distributed to complete the processing task. In this way, the task deployment scenarios of multiple GPUs are comprehensively considered, which not only minimizes the impact of parallel deceleration, but also reduces the resource fragmentation generated in GPU deployment, improves the utilization efficiency of GPUs, and optimizes the total time to complete all tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0060] Figure 1 is a flow chart of a GPU scheduling method proposed in one embodiment of the present application;

[0061] Figure 2 This is a schematic diagram of GPU scheduling proposed in an embodiment of the present application;

[0062] Figure 3 This is a schematic diagram of GPU scheduling when the GPU is idle, proposed in an embodiment of the present application;

[0063] Figure 4 This is a schematic diagram of GPU scheduling at the second moment proposed in an embodiment of the present application;

[0064] Figure 5 This is a schematic diagram of a GPU scheduling process proposed in an embodiment of the present application.

[0065] Figure 6 is a structural block diagram of a GPU scheduling device provided in an embodiment of the present application;

[0066] Figure 7 It is a schematic diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0068] In the drawings, the size of the constituent elements, the thickness of the layer or the area may be exaggerated for the sake of clarity. Therefore, any implementation of the present disclosure is not necessarily limited to the size shown in the drawings, and the shapes and sizes of the components in the drawings do not reflect the true proportions. In addition, the drawings schematically show ideal examples, and any implementation of the present disclosure is not limited to the shapes or values ​​shown in the drawings.

[0069] Please refer to Figure 1 , Figure 1 1 is a flowchart of a GPU scheduling method proposed in one embodiment of the present application. Figure 1 As shown, the method may include steps S101 to S105:

[0070] Step S101: receiving a first task to be executed at a first moment;

[0071] Step S102: Obtaining the required running time and required video memory of the first task;

[0072] Step S103: traversing each moment in a time period from the first moment to the required running time, determining a GPU group that meets the required video memory of the first task, each GPU in the GPU group that meets the required video memory of the first task equally shares the required video memory of the first task, and the remaining video memory of each GPU at this moment is greater than or equal to the equally shared required video memory;

[0073] Step S104: determining the time corresponding to the GPU group containing the minimum number of GPUs within the time period as the target time;

[0074] Step S105: at the target time, evenly deploy the first task on each GPU in the GPU group with the minimum number of GPUs and execute them.

[0075] In the related art, when facing the problem of multi-GPU business deployment, there is often a problem of parallel deceleration. The so-called parallel deceleration means that when multiple GPUs process a task at the same time, the time to complete the task is not equal to the ratio of the total task volume to the number of GPUs. Due to factors such as communication delays, information exchange delays, and storage differences between the GPUs, the time consumed when actually processing the same task together is greater than the ratio of the total task volume to the number of GPUs. Therefore, assigning fewer GPUs to execute the same task can further reduce the impact of parallel deceleration. At the same time, since the number of computing units and the size of video memory in each GPU are fixed, and the task volume of each task carried is not the same, when assigning tasks to multiple GPUs, there is often a situation where the computing units and video memory in a single GPU are not fully utilized, but the remaining part cannot carry a new task due to its small capacity. The idle resources in the GPU caused by this reason are the resource fragments of the GPU. The problems of parallel deceleration and resource fragmentation have greatly affected the efficiency of multiple GPUs in completing inference tasks. In related technologies, there are methods such as optimized inference methods for image tasks and scheduling based on association-based neural networks. Although these methods are also methods for allocating and scheduling inference tasks, they are limited to a certain range and do not have good generalization. In addition, these methods do not consider the allocation scenarios of multiple GPUs.

[0076] Therefore, a method for scheduling multiple GPUs is proposed in the present application. Through the method proposed in the present application, when multiple GPUs are faced with task allocation, the parallel deceleration problem between the GPUs can be taken into account, and the generation of resource fragmentation can be minimized. First, when a first task to be executed is received, it is necessary to obtain the running time and required video memory required by the first task, wherein, by allocating tasks to homogeneous multiple GPUs, the running time required for the task can be determined according to the total task amount and the task amount that a single computing unit can bear, as well as the number of computing units accommodated in each GPU, and the video memory size required for each task is also information that can be obtained while receiving the task, and the task needs to be allocated to a video memory space sufficient to accommodate its size. If the video memory required by a task is greater than the video memory space of a single GPU, the task needs to be split. At the same time, since multiple GPUs are homogeneous structures, it is necessary to evenly divide the tasks that require more video memory than the video memory space of a single GPU, and evenly divide them into multiple parts and allocate them to multiple video memories for joint execution.

[0077] like Figure 2 As shown, Figure 2 This is a GPU scheduling diagram proposed in an embodiment of the present application, wherein the height of GPU0 to GPU3 represents the size of the GPU's video memory space. Since multiple GPUs are of isomorphic structure, the size of the video memory space of each GPU is consistent. The height of tasks a to d and the first task represents the size of the video memory space required by the task, i.e., the required video memory. The horizontal width represents the time required to complete the task, i.e., the required running time. At the first moment, tasks a to d are being executed in GPU0 to GPU3 respectively. At this time, the first task to be executed is obtained, such as Figure 2 According to the required video memory and required time of the first task, all moments within the required time are traversed from the first moment, and there are two situations to determine the moment when the first task can be assigned, as follows: Figure 2 (B) and Figure 2 (C) shown.

[0078] exist Figure 2 In (B), there are GPU groups: GPU0, GPU2, GPU3, so that the first task can be divided into three parts. The remaining video memory of each GPU in the GPU group is greater than the required video memory of the first task. Therefore, the first task can be divided into three parts at this moment and deployed in GPU0, GPU2, and GPU3 respectively. It should be noted that when the tasks to be processed are divided into multiple GPUs for simultaneous processing, the above-mentioned parallel deceleration problem will be encountered. Specifically, Figure 2It is reflected as the loss part after the first task after equalization, that is, the time required to complete the first task is the sum of the required running time of the first task and the loss time.

[0079] exist Figure 2 In (C), there are GPU groups: GPU0 and GPU2, so that the first task can be divided into two parts. The remaining video memory of each GPU in the GPU group is greater than the required video memory of the first task. Therefore, the first task can be divided into two parts at this moment and deployed in GPU0 and GPU2 respectively. Figure 2 In the scheduling method shown in (C), the first task is processed by two GPUs simultaneously, which also faces the problem of parallel deceleration. However, due to the small number of GPUs allocated, the loss time caused by parallel deceleration is relatively small. Figure 2 The situation shown in (B) would be more transient.

[0080] According to the GPU scheduling method proposed in this application, Figure 2 When there are multiple moments when the first task shown in can be deployed, the number of GPUs included in its GPU group is compared, Figure 2 (B) has a quantity of 3, and Figure 2 The number in (C) is 2, so the first task will be Figure 2 The method shown in (C) is deployed in GPU0 and GPU2, and when the target time is reached, GPU0 and GPU2 execute the deployed first task.

[0081] Step S101: receiving a first task to be executed at a first moment.

[0082] In an embodiment of the present application, it is first necessary to receive tasks to be executed. In the present application, only the first task is taken as an example. In the actual execution process, there may be multiple tasks to be executed in the GPU group, and multiple new tasks will be dispatched to the GPU group at the same time. When faced with a large number of tasks to be executed, the GPU scheduling method proposed in the present application can improve the efficiency of the GPU group in processing tasks by adjusting the task deployment strategy, thereby reducing the generation of resource fragments in the GPU group while reducing parallel deceleration.

[0083] Step S102: Obtain the required running time and required video memory of the first task.

[0084] In the embodiment of the present application, after obtaining the first task to be executed, it is also necessary to obtain the required running time and required video memory of the first task, which are attributes of each task. The required video memory refers to how much computing space the task needs to occupy. Since the video memory size in each GPU is fixed and equal, it is necessary to determine whether the task can be deployed in one or several GPUs for execution based on the required video memory; the required running time is used to measure the workload of the task, how much computing resources are needed and how much time the GPU needs to occupy. It is usually believed that the computing efficiency of each computing unit is certain, so the required running time can be used to characterize the total amount of computing.

[0085] Step S103: Traverse each moment in the time period from the first moment to the required running time, determine the GPU group that meets the required video memory of the first task, each GPU in the GPU group that meets the required video memory of the first task equally shares the required video memory of the first task, and the remaining video memory of each GPU at this moment is greater than or equal to the required video memory equally shared.

[0086] In an embodiment of the present application, after obtaining the required video memory and required running time of the first task to be executed, the first task can be deployed according to the current usage of the GPU, wherein since the tasks existing in each GPU may be executed and completed in a short time, it is necessary to consider all possible GPU situations at each moment within a certain time period, and this time period refers to the required running time of the task to be executed. Within this time period, determine whether there is a situation at each moment that allows the first task to be deployed, and consider the required video memory of the first task. Since each GPU is isomorphic, it is necessary to consider that when deploying the first task in multiple GPUs at the same time, each GPU must divide the first task equally, and the remaining video memory space in each GPU must be greater than or equal to the required video memory of the first task after the equal division. For example, if Figure 2 In the situation shown in (B), at the first moment, although the total amount of remaining video memory in GPU0 to GPU3 is greater than the required video memory of the first task, since the remaining video memory in GPU1, GPU2, and GPU3 is less than half of the required video memory of the first task, the first task cannot be divided into two parts for deployment, but can only be divided into three parts and deployed in GPU0, GPU2, and GPU3 respectively.

[0087] Step S104: Determine the time corresponding to the GPU group containing the minimum number of GPUs within the time period as the target time.

[0088] In an embodiment of the present application, after traversing each moment in the time period from the first moment to the required running time, it is necessary to determine the specific moment at which the first task is deployed for execution. Based on the idea proposed in the present application to minimize the interference caused by parallel deceleration, the first task is deployed in a manner that minimizes the interference of parallel deceleration, that is, the moment with the least number of GPUs in the GPU group where the first task can be deployed is selected as the target moment.

[0089] Step S105: at the target time, evenly deploy the first task on each GPU in the GPU group with the minimum number of GPUs and execute them.

[0090] In the embodiment of the present application, since the target moment may not be the first moment, it is necessary to first evenly deploy the first task on each GPU in the GPU group corresponding to the target moment. In this way, when running to the target moment, each GPU can start executing the first task.

[0091] In combination with the above embodiments, in one implementation, the present application further provides a GPU scheduling method, which specifically includes the following contents:

[0092] First, when multiple tasks are received at the first moment, a task with the largest video memory requirement among the multiple tasks is determined as the first task.

[0093] In an embodiment of the present application, when a task is received, GPU scheduling needs to be performed based on the occupancy of the GPU. In steps S101 to S105, it is explained how to deploy the first task. However, in actual use, the tasks dispatched to the GPU group may not be dispatched one by one. In most cases, multiple tasks will be dispatched to the GPU group at one time. Therefore, how to determine the order of task execution among multiple tasks is also one of the key factors affecting task completion.

[0094] In the present application, it is proposed to first sort the tasks to be executed according to the required video memory, and determine the task with the largest video memory requirement as the first task. Since the total video memory owned by each GPU is the same, the tasks with a larger video memory requirement are allocated first, so that a larger part of the GPU video memory space can be utilized, and the subsequent deployment of other tasks with smaller video memory requirements can utilize the remaining video memory space, so that the utilization rate of the video memory in the GPU is higher. For example, if the tasks with small video memory requirements are deployed first, the remaining available video memory in a single GPU will be compressed. When deploying tasks with larger existing requirements, the total video memory of the GPU may meet the deployment requirements, but because other tasks have been deployed, the remaining available video memory in the GPU is insufficient to deploy the tasks with larger video memory requirements, resulting in more resource fragmentation, and may even cause the task to be unable to be deployed within the required running time period, delaying the overall completion time of all tasks.

[0095] Then, when multiple tasks are received at the first moment and there are multiple tasks with the largest video memory requirement, any task with the longest required running time among the tasks with the largest video memory requirement is determined as the first task.

[0096] In the embodiment of the present application, on the basis of giving priority to GPU scheduling for tasks with greater video memory requirements as described above, there is also a situation where there are multiple tasks with the largest video memory requirements. In this case, it is also necessary to consider the required running time of each task, and determine any task with the longest required running time as the first task. The longer the required time of a task, the more workload this task requires to complete. Therefore, it is necessary to prioritize its deployment so that tasks with greater requirements can be started earlier, to avoid other tasks being completed, while only tasks with long required running time are still being executed, thus delaying the overall task completion time.

[0097] In combination with the above embodiments, in one implementation, the present application further provides a GPU scheduling method, which specifically further includes the following contents before traversing each moment in a time period from the first moment to the required running time to determine a GPU group that meets the required video memory of the first task:

[0098] First, a GPU video memory is obtained, and a ratio of the required video memory of the first task to the GPU video memory is determined as the GPU required quantity of the first task.

[0099] In the embodiment of the present application, to schedule multiple GPUs, it is first necessary to determine the video memory size of each GPU, and then after determining the first task in the above steps, the ratio of the required video memory of the first task to the total video memory of a single GPU is determined as the GPU required number of the first task. The GPU required number is used to express the total amount of GPU video memory occupied by the first task. By obtaining the GPU required number, the GPU can be scheduled more intuitively, and tasks can be preferentially assigned to idle GPUs.

[0100] Then, when the number of idle GPUs at the first moment is greater than or equal to the required number of GPUs, at the first moment, the first task is evenly deployed and executed in a first number of the idle GPUs, where the first number is the minimum number greater than the required number of GPUs.

[0101] In the embodiment of the present application, after the required number of GPUs for the first task is determined, it will be preferentially compared with the idle GPUs in the GPU group. In order to enable each GPU to distribute work tasks as evenly as possible, when a new task is obtained, it is necessary to preferentially allocate it to the idle GPUs. However, due to the different video memory requirements of each task, the idle GPUs may not be enough to accommodate the first task. Therefore, it is necessary to compare the number of idle GPUs with the required number of GPUs for the first task. When the number of idle GPUs is larger, it means that there are enough idle GPUs to fully accommodate the first task. Therefore, the first task needs to be allocated to the idle GPUs. Similarly, the idea of ​​equal distribution is adopted, and at the same time, the impact of parallel deceleration is reduced, and the first task needs to be allocated to fewer GPUs.

[0102] like Figure 3 As shown, Figure 3 This is a schematic diagram of GPU scheduling when a GPU is idle, proposed in an embodiment of the present application. When the first task is received, GPU0 to GPU2 in the GPU group are all in idle state. Since the number of GPUs required by the first task is greater than 1 but less than 2, it is necessary to deploy them in as few GPUs as possible. As shown in the figure, they are evenly deployed in GPU0 and GPU1 to minimize the loss caused by parallel deceleration.

[0103] In combination with the above embodiments, in one implementation, the present application further provides a GPU scheduling method, which specifically includes the following contents:

[0104] First, when the number of idle GPUs at the first moment is less than the required number of GPUs, traverse each moment in the time period from the first moment to the required running time to determine whether there is a second moment, where the second moment is the moment when the number of idle GPUs is greater than or equal to the required number of GPUs.

[0105] In the embodiment of the present application, as described above, if the number of idle GPUs in the GPU group is greater than the number of GPUs required by the first task when the first task is received, the idle GPUs are directly scheduled to execute the first task, and when the number of idle GPUs is less than the number of GPUs required by the first task at the first moment, it is necessary to consider whether there is a moment within a certain period of time when the occupied GPU completes all its tasks and turns to an idle state, so that the number of idle GPUs is greater than the number of GPUs required. According to the same logic as above, each moment in the time period from the first moment to the required running time of the first task is traversed to determine whether there is a second moment when the number of idle GPUs is greater than or equal to the number of GPUs required, so that the first task can be assigned to the idle GPU for execution.

[0106] Then, in the case where the second moment exists, the first task is evenly deployed on a second number of idle GPUs, where the second number is a minimum number greater than the required number of GPUs;

[0107] In an embodiment of the present application, as described above, in order to reduce the impact of parallel deceleration, the first task needs to be allocated to fewer GPUs. Therefore, if there are multiple moments when the number of idle GPUs is greater than or equal to the required number of GPUs, the moment when the number of idle GPUs is the smallest needs to be selected as the second moment.

[0108] Traversing each moment in a time period from the first moment to the required running time to determine a GPU group that meets the required video memory of the first task, including:

[0109] In the case that the second moment does not exist, each moment in a time period from the first moment to the required running time is traversed to determine a GPU group that meets the required video memory of the first task.

[0110] In the embodiment of the present application, when the number of idle GPUs is greater than or equal to the number of GPU requirements at the first moment or the second moment as described above, the first task will be deployed in the idle GPU first, which is also for better utilization of GPU resources. However, if there is no moment when the number of idle GPUs is greater than or equal to the number of GPU requirements at the first moment and during the time period after the first moment and after the required running time, it is necessary to select the method in steps S101 to S105 described above to perform GPU scheduling.

[0111] like Figure 4 As shown, Figure 4This is a schematic diagram of GPU scheduling at the second moment proposed in an embodiment of the present application. At the first moment of acquiring the first task, all GPUs are in an occupied state, but within the required running time of the first task, tasks b and c executed in GPU1 and GPU2 are completed respectively, that is, GPU1 and GPU2 become idle at the second moment, and the first task is deployed at the second moment of GPU1 and GPU2 for execution accordingly.

[0112] In combination with the above embodiments, in one implementation, the present application further provides a GPU scheduling method, which, in the presence of the second moment, evenly deploys the first task on each idle GPU, and specifically includes the following contents:

[0113] In the case where the second moment exists, the first task is evenly divided and deployed in the task queues of the second number of idle GPUs;

[0114] At the second moment, the evenly distributed tasks are executed by the second number of idle GPUs.

[0115] In the embodiment of the present application, as described above, when the number of idle GPUs at the first moment is less than the number of GPUs required for the first task, the second moment will be searched for and the first task will be assigned to the idle GPU. If the second moment exists, the first task will be deployed in the corresponding idle GPU and can be executed at the second moment. In this way, on the one hand, the deployment of the tasks to be executed can be completed quickly, and the deployment of the next task can be considered, thereby improving the efficiency of GPU scheduling. On the other hand, the GPU memory space at the second moment can be occupied in advance to prevent the remaining memory space of these GPUs from being directly used for deployment when subsequent tasks are deployed, resulting in insufficient remaining memory space in the GPU to deploy the first task when the second moment is reached, causing the first task to be further postponed. In this way, multiple tasks to be executed can be reasonably and orderly scheduled for GPUs, so that all tasks to be executed can be completed with maximum efficiency.

[0116] like Figure 5 As shown, Figure 5The present invention is a schematic diagram of a GPU scheduling process proposed in an embodiment of the present application. First, it is determined whether all tasks to be executed have been deployed. When there are tasks to be executed that have not been deployed, they need to be sorted according to the required video memory and the required running time, determine the first task, and then perform GPU scheduling execution on the first task. The specific steps are: first, it is determined whether there is a number of idle GPUs that is not less than the number of GPUs required by the first task at the first moment. If so, the first task is deployed in the idle GPU; if not, consider whether there is a second moment at each moment in the time period from the first moment to the required running time, so that the number of idle GPUs is not less than the number of GPUs required by the first task. If so, the first task is deployed in the idle GPU corresponding to the second moment; if not, consider whether there is a target moment at each moment in the time period from the first moment to the required running time, so that the required video memory of the first task can be evenly divided by the remaining video memory of each GPU in the GPU group, and the remaining video memory of each GPU at the target moment is greater than or equal to the evenly divided required video memory. If so, the first task is deployed in the GPU corresponding to the target moment according to the method described above; if not, the next task is deployed.

[0117] In summary, in order to implement the scheduling of multiple GPUs through the method proposed in this application, an integer linear programming model is proposed in the example of this application, wherein the variables of the model include: the total number of GPUs, the number of computing units owned by a single GPU, and the size of the video memory of a single GPU, which reflect the computing power and task-carrying capacity of the GPU; the model variables also include: the number of tasks to be executed, the required video memory of the tasks to be executed, and the required running time of the tasks to be executed, which reflect the task volume of the tasks to be executed. The parameter that the model needs to consider is the parallel reduction ratio of the GPU. Ideally, there is no parallel reduction. If the same task is assigned to two identical GPUs for simultaneous execution, it only takes half the time to complete, but due to the existence of parallel reduction, it actually takes longer time, and the ratio between this ideal time and the actual time is the parallel reduction ratio. This integer linear programming model also has certain constraints, including at least: at the same time, the sum of the required video memory of all tasks being executed in the same GPU should not exceed the video memory size of a single GPU, and all tasks to be executed need to be completed. The ultimate optimization goal of the model is the total time required to complete all tasks to be executed. Through the method proposed in this application, the total time to complete all tasks is shortened as much as possible. The integer linear programming model proposed above can be applied to various practical scenarios to complete the GPU scheduling method proposed in this application.

[0118] In combination with the above embodiments, in one implementation, the present application further provides a GPU scheduling method, which specifically includes the following contents:

[0119] If there is no GPU group that meets the required video memory of the first task within the time period, a new first task is obtained to deploy and execute the new first task.

[0120] In the embodiment of the present application, there is also a possible situation, that is, within the time period from the first moment to the required running time of the first task, there is no any moment at which the first task can be scheduled for execution on the GPU by any of the methods described above. In this case, the GPU scheduling of the next task is performed, and the undeployed first task needs to try to be deployed again after its corresponding required running time has passed.

[0121] Based on the same design concept, an embodiment of the present application provides a GPU scheduling device. Figure 6 , Figure 6 : is a structural block diagram of a GPU scheduling device provided by an embodiment of the present application. Figure 6 As shown, the device comprises:

[0122] A first task receiving module, used for receiving a first task to be executed at a first moment;

[0123] An information acquisition module, used to acquire the required running time and required video memory of the first task;

[0124] A GPU group determination module is used to traverse each moment in a time period from the first moment to the required running time, and determine a GPU group that meets the required video memory of the first task, wherein each GPU in the GPU group that meets the required video memory of the first task evenly shares the required video memory of the first task, and the remaining video memory of each GPU at this moment is greater than or equal to the evenly shared required video memory;

[0125] A target time determination module, used to determine the time corresponding to the GPU group containing the minimum number of GPUs within the time period as the target time;

[0126] The first task deployment module is used to evenly deploy and execute the first task on each GPU in the GPU group with the minimum number of GPUs at the target time.

[0127] Optionally, the first task receiving module further includes:

[0128] A first determining submodule is configured to, when receiving multiple tasks at the first moment, determine a task with the largest video memory requirement among the multiple tasks as the first task;

[0129] The second determination submodule is used to, when multiple tasks are received at the first moment and there are multiple tasks with the largest video memory requirements among the multiple tasks, determine any task with the longest required running time among the tasks with the largest video memory requirements as the first task.

[0130] Optionally, the GPU scheduling device further includes:

[0131] A GPU requirement quantity determination module, used for obtaining GPU video memory, and determining a ratio of the required video memory of the first task to the GPU video memory as the GPU requirement quantity of the first task;

[0132] A first quantity task deployment module is used to evenly deploy and execute the first task in a first number of idle GPUs at the first moment when the number of idle GPUs at the first moment is greater than or equal to the required number of GPUs, and the first number is a minimum number greater than the required number of GPUs.

[0133] Optionally, the GPU scheduling device further includes:

[0134] A second moment determination module is configured to, when the number of idle GPUs at the first moment is less than the required number of GPUs, traverse each moment in a time period from the first moment to the required running time to determine whether there is a second moment, where the second moment is a moment when the number of idle GPUs is greater than or equal to the required number of GPUs;

[0135] A second number of task deployment module, configured to deploy the first task on a second number of idle GPUs on average when the second moment exists, where the second number is a minimum number greater than the required number of GPUs;

[0136] Optionally, the GPU scheduling device further includes:

[0137] A second number of task deployment module, configured to evenly divide and deploy the first task in respective task queues of the second number of idle GPUs when the second moment exists;

[0138] The second-moment task execution module is used to execute the evenly distributed tasks through the second number of idle GPUs at the second moment.

[0139] Optionally, the GPU scheduling device further includes:

[0140] The task replacement module is used to obtain a new first task in the case that there is no GPU group that meets the required video memory of the first task within the time period, so as to deploy and execute the new first task.

[0141] Based on the same design concept, another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the GPU scheduling method described in any of the above embodiments of the present application are implemented.

[0142] Based on the same design concept, another embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in the GPU scheduling method described in any of the above embodiments of the present application.

[0143] Based on the same design concept, another embodiment of the present application provides an electronic device, such as Figure 7 shown. Figure 7 1 is a schematic diagram of an electronic device shown in an embodiment of the present application. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes, the steps in the GPU scheduling method described in any of the above embodiments of the present application are implemented.

[0144] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0145] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0146] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0147] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0148] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0150] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0151] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0152] The above is a detailed introduction to a GPU scheduling method, device, equipment, medium and product provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A GPU scheduling method, characterized in that: include: Receiving a first task to be performed at a first moment; Obtaining the required running time and required video memory of the first task; Traversing each moment in a time period from the first moment to the required running time, determining a GPU group that meets the required video memory of the first task, wherein each GPU in the GPU group that meets the required video memory of the first task evenly shares the required video memory of the first task, and the remaining video memory of each GPU at this moment is greater than or equal to the evenly shared required video memory; Determine the time corresponding to the GPU group containing the minimum number of GPUs within the time period as the target time; At the target time, the first task is evenly deployed on each GPU in the GPU group with the minimum number of GPUs and executed.

2. The GPU scheduling method according to claim 1, characterized in that: Also includes: In the case where multiple tasks are received at the first moment, determining a task with the largest video memory requirement among the multiple tasks as the first task; When multiple tasks are received at the first moment, and there are multiple tasks with the largest video memory requirements among the multiple tasks, any task with the longest required running time among the tasks with the largest video memory requirements is determined as the first task.

3. The GPU scheduling method according to claim 1, characterized in that: Before traversing each moment in a time period from the first moment to the required running time to determine a GPU group that meets the required video memory of the first task, the method further includes: Obtaining GPU video memory, and determining the ratio of the required video memory of the first task to the GPU video memory as the GPU required quantity of the first task; When the number of idle GPUs at the first moment is greater than or equal to the required number of GPUs, at the first moment, the first task is evenly deployed and executed in a first number of the idle GPUs, where the first number is the minimum number greater than the required number of GPUs.

4. The GPU scheduling method according to claim 3, characterized in that: Also includes: When the number of idle GPUs at the first moment is less than the required number of GPUs, traverse each moment in a time period from the first moment to the required running time to determine whether there is a second moment, where the second moment is a moment when the number of idle GPUs is greater than or equal to the required number of GPUs; In the case where the second moment exists, the first task is evenly deployed on a second number of idle GPUs, where the second number is a minimum number greater than the required number of GPUs; Traversing each moment in a time period from the first moment to the required running time to determine a GPU group that meets the required video memory of the first task, including: In the case that the second moment does not exist, each moment in a time period from the first moment to the required running time is traversed to determine a GPU group that meets the required video memory of the first task.

5. The GPU scheduling method according to claim 4, characterized in that: In the case where the second moment exists, evenly deploying the first task on each idle GPU includes: In the case where the second moment exists, the first task is evenly divided and deployed in the task queues of the second number of idle GPUs; At the second moment, the evenly distributed tasks are executed by the second number of idle GPUs.

6. The GPU scheduling method according to any one of claims 1 to 5, characterized in that: The method further comprises: If there is no GPU group that meets the required video memory of the first task within the time period, a new first task is obtained to deploy and execute the new first task.

7. A GPU scheduling device, characterized in that: The device comprises: A first task receiving module, used for receiving a first task to be executed at a first moment; An information acquisition module, used to acquire the required running time and required video memory of the first task; A GPU group determination module is used to traverse each moment in a time period from the first moment to the required running time, and determine a GPU group that meets the required video memory of the first task, wherein each GPU in the GPU group that meets the required video memory of the first task evenly shares the required video memory of the first task, and the remaining video memory of each GPU at this moment is greater than or equal to the evenly shared required video memory; A target time determination module, used to determine the time corresponding to the GPU group containing the minimum number of GPUs within the time period as the target time; The first task deployment module is used to evenly deploy and execute the first task on each GPU in the GPU group with the minimum number of GPUs at the target time.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by the processor, the GPU scheduling method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the GPU scheduling method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the GPU scheduling method according to any one of claims 1 to 6 is implemented.

Images