A deadline-aware deep learning training task scheduling method and system for efficiency

Abstract

The present invention discloses a deadline-aware deep learning training task scheduling method and system for efficiency. The method mainly includes two steps: deep learning training task running time estimation and deep learning training task scheduling optimization. First, a threshold-based adaptive sampling technology is used to obtain an accurate estimation result of the deep learning training task running time. Then, the scheduling problem in the heterogeneous GPU cluster is converted into a search problem for task arrangement on each GPU. The optimized scheduling result that meets the scheduling goal is searched, and the search space is reduced through a variety of pruning methods. Compared with the prior art, the present invention innovatively proposes a unified expression form of scheduling goals, which can simultaneously meet the scheduling goals of minimizing the deadline violation amount of the task and the overall task completion time, and can efficiently schedule deep learning tasks, which is conducive to improving the efficiency of task deployment and operation in heterogeneous GPU clusters.

Description

Technical Field

[0001] The present invention belongs to the field of cloud computing technology, and relates to an efficiency-oriented deadline-aware deep learning training task scheduling method and system. The method uses adaptive sampling technology to estimate the running time information of deep learning training tasks; uses a unified cost function to simultaneously express the scheduling goals of minimizing the task deadline violation amount and minimizing the overall task completion time, and uses a branch-and-bound search method to search for the optimal cost and the corresponding task scheduling result. Background Art

[0002] In recent years, deep learning technology has developed rapidly and has been widely used in many fields, such as computer vision, natural language processing, and speech recognition. Deep learning uses large data sets to update the parameters of deep learning models (generally deep neural networks) by running deep learning training jobs, and finally generates models that meet the accuracy requirements of the above fields. Deep learning training tasks are computationally intensive tasks, and using GPUs (Graphics Processing Units) to accelerate deep learning tasks has become one of the key technologies to improve the performance of deep learning training tasks. Many large-scale enterprises and research institutions have established large-scale GPU clusters to run deep learning training tasks, and GPU clusters are heterogeneous (the cluster contains multiple models of GPUs). In large-scale heterogeneous GPU clusters, a reasonably designed scheduling system is needed to determine the binding relationship between deep learning training tasks and computing resources, so as to run task loads with high quality, improve GPU utilization efficiency, and reduce infrastructure costs.

[0003] The main objectives of deep learning training task scheduling include minimizing the overall task completion time (efficiency), that is, the sum of the completion time of all tasks in the GPU cluster is as small as possible; minimizing task deadline violations (deadline awareness), that is, requiring the completion time of as many tasks as possible to be less than or equal to their deadlines. However, in a heterogeneous GPU cluster environment, achieving these two scheduling goals at the global level is still challenging. Relevant experimental data show that the running time difference of the same task on different models of GPUs can be up to 4 times, which is highly uncertain. In this environment, if high-quality scheduling cannot be performed in combination with the running time difference of tasks on heterogeneous GPUs, it may lead to an extension of the overall task time and increase the cost of infrastructure use; or it may cause tasks to violate deadlines, causing significant losses to core businesses. Therefore, it is urgent to design an efficiency-oriented deadline-aware deep learning training task scheduling method in a heterogeneous GPU cluster environment.

[0004] In a heterogeneous GPU cluster environment, scheduling deep learning training tasks presents the following challenges:

[0005] (1) In order to perform deadline-aware scheduling of deep learning training tasks and meet the scheduling target of task scheduling deadlines, it is necessary to estimate the running time of the tasks. However, existing research work ignores the "cold start" overhead caused by the initialization of deep learning training tasks, so it is difficult for existing work to accurately estimate the running time of tasks.

[0006] (2) Deep learning training task scheduling needs to meet multiple scheduling objectives. However, existing research work can often only meet a single scheduling objective (such as minimizing the overall task completion time) or cannot accurately quantify multiple objectives at the same time.

[0007] (3) In a heterogeneous GPU cluster environment, the running time of tasks is highly uncertain. How to reduce the deadline violations of tasks at the global level and minimize the overall task running time in a heterogeneous GPU cluster environment remains a challenge.

[0008] In recent years, related work to solve the scheduling problem of deep learning training tasks mainly includes the following work:

[0009] (1) Efficiency-oriented task scheduling method. Allox (Le Tan N, Sun Xiao, Chowdhury Mosharaf, et al.: Allox: compute allocation in hybrid clusters@EuroSys20.) is a deep learning training task scheduling method for hybrid heterogeneous device environments. Its scheduling goal is to minimize the overall task completion time. The system uses classical graph theory algorithms to transform the scheduling problem of deep learning training tasks into an optimal matching problem of weighted bipartite graphs, and uses the Hungarian algorithm to solve the problem, and then transforms it to obtain the scheduling result. Due to the limitations of the problem modeling of this method, it can only support the scheduling goal of minimizing the overall task completion time, and cannot meet the task deadline goal.

[0010] (2) Deadline-oriented task scheduling method. Chronus (Gao Wei, Ye Zhisheng, Sun, Peng, et al.: Chronus: A Novel Deadline-aware Scheduler for Deep Learning Training Jobs@Socc21.) sets a qualitative binary reward function for tasks with deadlines and rewards tasks that meet the deadline. The scheduling problem of deep learning training tasks is then modeled as a mixed integer linear programming problem and solved to obtain the scheduling result. However, this method is only effective for homogeneous GPU cluster environments, and this method can only qualitatively model whether a task violates the deadline, and cannot accurately reflect the amount of deadline violated by the task. Therefore, there is still room for improvement in the scheduling results.

[0011] (3) Scheduling method in heterogeneous GPU cluster environment. Gandiva fair (Chaudhary Shubham, Ramjee Ramachandran, Sivathanu Muthiand, etc.: Balancing efficiency and fairness inheterogeneous GPU clusters for deep learning@EuroSys20.) In a heterogeneous GPU resource environment, a heterogeneous resource exchange strategy based on "second-price auction" was proposed to increase the throughput of cluster GPUs while ensuring fairness among users. However, this method does not take the task deadline into consideration and therefore cannot meet the task deadline target.

[0012] In summary, existing work either performs scheduling for homogeneous GPU environments or fails to accurately model and express the scheduling objectives of this scenario, thereby efficiently obtaining high-quality scheduling results. Summary of the invention

[0013] The purpose of the present invention is to overcome the shortcomings of the existing deep learning training task scheduling method, and obtain the running time of the deep learning training task through the deep learning task running time estimation technology based on adaptive sampling. In order to solve the problem of insufficient expression ability of minimizing task deadline violations, a quantitative expression that comprehensively considers deadline violations and overall task completion time is proposed, that is, a unified cost is proposed to quantitatively express the scheduling target in a heterogeneous GPU environment; in order to solve the problem that the search space of the scheduling scheme is too large, a branch-and-bound search technology optimized by multiple pruning strategies is used to narrow the search space, quickly obtain the optimal task scheduling sequence that meets the scheduling target in the heterogeneous GPU cluster, and improve the deployment and execution efficiency of tasks in the GPU cluster.

[0014] The technical solution adopted by the present invention is as follows:

[0015] A deadline-aware deep learning training task scheduling method for efficiency includes the following steps:

[0016] The running time of the deep learning training task is estimated through adaptive sampling technology, and the running time of the deep learning training task is added to the task information;

[0017] Schedule deep learning training tasks based on task information and GPU information. The scheduling goal is to minimize the overall task completion time while minimizing the task deadline violations.

[0018] Deploy the deep learning training tasks to the corresponding GPUs in the GPU cluster in the order specified in the scheduling results for execution.

[0019] Furthermore, the running time of the deep learning training task is estimated by the adaptive sampling technology, and the sampling process is performed when the iterative running time of the training task enters a stable state; and the difference method is used to determine whether the iterative running time of the training task enters a stable state.

[0020] Furthermore, scheduling the deep learning training task according to the task information and GPU information includes:

[0021] Step 1: The task scheduler obtains all unscheduled tasks and adds each unscheduled task j to the task set of each GPU.

[0022] Step 2: The task scheduler generates a task sequence for each GPU in a short-term task-first manner.

[0023] Step 3: Schedule each GPU according to the task sequence in step 2, and determine whether there is a task on each GPU that violates the deadline. If there is no task deadline violation on the GPU, jump to step 4, otherwise jump to step 5;

[0024] Step 4: Calculate the cost of the task sequence obtained in step 2 and directly use it as the minimum cost of the GPU;

[0025] Step 5: Search for the optimal task sequence with minimum cost and the corresponding minimum cost;

[0026] Step 6: In steps 4 and 5, the minimum cost and the corresponding task sequence have been calculated for each GPU. Select the minimum minimum cost, the task j' and GPU g' corresponding to this cost, and the corresponding task sequence on GPU g' is S'. Then schedule task j' to GPU g', set the task sequence on GPU g' to S', and mark the status of task j' as scheduled;

[0027] Step 7: Repeat steps 1-6 until all tasks are scheduled.

[0028] Furthermore, a unified cost function is used to simultaneously express the scheduling objectives of minimizing task deadline violations and minimizing overall task completion time, defining GPU g m Task sequence on The cost is:

[0029]

[0030] in, For task j n In GPU g m Waiting time on For task j n In GPU g m Running time on For task j n The deadline constraint is: f is a piecewise function defined as:

[0031]

[0032] is the penalty coefficient for task deadline violation.

[0033] Furthermore, a branch-and-bound search algorithm with pruning optimization is used to search for an optimal task sequence with minimum cost and a minimum cost corresponding thereto; the branch-and-bound search algorithm with pruning optimization comprises the following steps:

[0034] a) For each GPU, create a minimum heap to store nodes. Each node contains an incomplete task sequence and its corresponding value. The value of the node is defined as the cost of the complete task sequence generated by adding all tasks in the task set that are not in the incomplete task sequence of the node to the end of the incomplete task sequence in the order of short task priority. When the minimum heap is created, it only contains one node with an empty task sequence and a value of 0.

[0035] b) Each time a node is popped from the top of the minimum heap, for each task in the task set that is not in the incomplete task sequence of the node, it is added to the end of the incomplete task sequence to generate a new extended node. When the extended node meets the conditions corresponding to the pruning strategy, the extended node is pruned, otherwise, the extended node is added to the minimum heap;

[0036] c) When the task sequence in a node popped from the top of the minimum heap is a complete task sequence, the node is called the answer node; the node with the minimum value in the answer node has the minimum cost on GPU g, and its task sequence is the optimal task sequence on GPU g.

[0037] Furthermore, the branch-and-bound search algorithm with pruning optimization uses the following pruning strategies during operation to reduce the running time of the algorithm:

[0038] i) Relative position pruning: When there are two tasks without deadlines in the task sequence of a node, and their relative positions in the task sequence do not satisfy the short task priority order, the node is discarded;

[0039] ii) Constrained pruning: When the cost of a node’s task sequence is greater than the minimum cost of the current record, the node is discarded;

[0040] iii) Optimal prediction pruning: When a complete task sequence is generated on a node and there is no deadline violation in the second half of the task sequence, the cost of the task sequence is the minimum cost that the node can generate. In this case, the node does not need to be expanded;

[0041] iv) Lower bound pruning: When a complete task sequence is generated on a node, the lower bound of the cost of the task sequence of the node is calculated, that is, the cost without considering the penalty item of the second half of the tasks violating the deadline, and if the cost is greater than the minimum cost of the current record, the node is discarded.

[0042] An efficiency-oriented deadline-aware deep learning training task scheduling system, comprising:

[0043] A deep learning training task running time estimation module is used to estimate the running time of the deep learning training task through adaptive sampling technology, and add the running time of the deep learning training task to the task information;

[0044] The deep learning training task scheduling optimization module is used to schedule deep learning training tasks according to task information and GPU information. The scheduling goal is to minimize the overall task completion time while minimizing the task deadline violations, and deploy the deep learning training tasks to the corresponding GPUs in the GPU cluster in the order of the scheduling results for execution.

[0045] Compared with the prior art, the present invention has the following technical advantages:

[0046] 1) The deep learning training task running time estimation method based on adaptive sampling proposed in the present invention has a higher estimation accuracy than existing work because it takes into account the "cold start" overhead caused by the initialization of the deep learning training task, and serves as an important basis for scheduling.

[0047] 2) The method of the present invention is aimed at a heterogeneous GPU cluster environment, and proposes a cost formula to uniformly quantify the scheduling objectives of minimizing task deadline violations and overall task completion time. Compared with the existing scheduling methods with a single goal and qualitative expression methods, it can obtain more optimized scheduling results and improve the quality and efficiency of task deployment and operation.

[0048] 3) To address the problem that the search space for scheduling solutions is too large, the method of the present invention designs a variety of pruning strategies when using the branch-and-bound method to search for the optimal scheduling solution, thereby greatly reducing the search space and lowering the overhead of task scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 .Schematic diagram of the structure of the method of the present invention.

[0050] Figure 2 .The task running time estimator operation flow chart of the method of the present invention.

[0051] Figure 3 .Task scheduler operation flow chart of the method of the present invention. DETAILED DESCRIPTION

[0052] The technical solution of the present invention will be further described below in conjunction with the accompanying drawings of the present invention. The described embodiments are only part of the embodiments of the present invention and do not represent all the embodiments. For those skilled in the art, some well-known technologies may not be described in detail.

[0053] The present invention provides an efficiency-oriented deadline-aware deep learning training task scheduling method, which mainly includes two steps: deep learning training task running time estimation; deep learning training task scheduling optimization.

[0054] (1) Estimation of running time of deep learning training tasks

[0055] In this method, when a user submits a deep learning training task, the task running time estimator first uses an adaptive sampling estimation method to estimate the running time of the deep learning training task.

[0056] Existing work has pointed out that deep learning training tasks are iterative and highly predictable. Because the training iterations have the same execution logic and execution time, the running time of the task can be estimated using sampling methods. However, in the actual running process of the task, the running time of the first few iterations is longer than that of the subsequent iterations. An important reason for this phenomenon is the "cold start", that is, when the task starts running, there is a certain initialization overhead, mainly including: initialization of Python and CUDA library functions, copying between memory and video memory, hardware preheating, etc.

[0057] Therefore, the present invention proposes a threshold-based adaptive sampling technology that takes into account the initialization overhead of the task, thereby improving the accuracy of estimating the task running time. The overall idea is that when the running time of the training iteration enters a stable state, the sampling process is performed. This technology uses a differential method to determine whether the iterative running time of the task has entered a stable state, that is, the running time of two adjacent iterations of the task is T1 and T2, then the difference between the running time of the two adjacent iterations is During the iterative run, by setting a stable state threshold (set to 50% in this system), when the calculation is When , it is considered that the iteration time of the task has entered a stable state. At this time, resampling I S Iterations are used to predict the running time of deep learning training tasks in a steady state.

[0058] Based on the above, the estimated running time T of the task obtained by this threshold-based adaptive sampling method is:

[0059]

[0060] Among them, I B represents the number of iterations before the task enters a stable state, T B represents the iteration running time of the task before entering the stable state, I represents the total number of iterations specified by the user, and I S represents the number of sampling iterations, T S Represents the running time of the sampling iteration.

[0061] (2) Deep learning training task scheduling optimization

[0062] Furthermore, the task scheduler schedules the deep learning training task by obtaining the basic information of the task, the task running time information output by the task running time estimator, and the GPU information. This method sets the scheduling goal to minimize the overall task completion time under the premise of minimizing the task deadline violation, and defines the cost of the scheduling scheme according to the scheduling goal. The scheme with the minimum cost among all scheduling schemes is searched through branch and bound, and is used as the final scheduling result. In order to reduce the overall search space, a variety of pruning strategies are also used in the branch and bound search to reduce the overhead of the scheduling process. The specific steps of this scheduling method include:

[0063] 1) According to the unscheduled tasks and available resources in the GPU cluster, a formal expression of the problem is established: there are N unscheduled tasks in the cluster, denoted by J = {j1, j2, …, j N The GPU cluster contains M heterogeneous GPUs, which are represented by g1, g2, …, g M , that is, the available GPU resources can be expressed as G = {g1,g2,…,g M}, Task j n In GPU g m The running time on From step (1), we can get task j n In GPU g m The waiting time on (waiting for the previous task to complete) is Task J n The deadline constraint is

[0064] 2) Then, for the allocation to GPU g m The tasks on the GPU will be arranged in a certain order and executed in the order of the arrangement, and this arrangement is defined as the task sequence of the GPU. According to this task sequence, define GPUg m Task sequence on The cost is:

[0065]

[0066] Where f is a piecewise function defined as:

[0067]

[0068] is the penalty factor for task deadline violation (the default value is 10 9, indicating that the task deadline is a more important scheduling goal). This cost quantitatively expresses the scheduling goal, that is, according to the cost definition, the lower the cost of a task sequence on a GPU, the shorter the deadline violation amount and the shorter the overall task completion time. If the cost of the task sequence on each GPU can be minimized, the corresponding task scheduling sequence on each GPU is the scheduling result expected by this method.

[0069] 3) Then, in each round of the algorithm, a task will be selected and assigned to a GPU. The specific process is as follows: for each unscheduled task j∈J, task j is added to a temporary task set generated by each GPU. On each GPU g∈G, the tasks in the temporary set are first generated into a task sequence in the order of short tasks first (sorted according to the running time of task j on GPU g, and tasks with short running time are placed at the front of the task sequence). If no task in this task sequence violates its deadline, the cost of this task sequence is used as the minimum cost of GPU g. Otherwise, all different task sequences of the tasks in the temporary set are generated. In the process of generating different task sequences, a branch-and-bound search algorithm optimized with multiple pruning strategies is used to reduce the search space and quickly obtain the optimal task sequence and the corresponding minimum cost on GPU g.

[0070] Among them, the specific steps of "branch-and-bound search algorithm optimized using multiple pruning strategies" include:

[0071] a) For each GPU, create a minimum heap to store nodes. Each node contains an incomplete task sequence and its corresponding value. The value of the node is defined as the cost of the complete task sequence generated by adding all tasks in the task set that are not in the incomplete task sequence of the node to the end of the incomplete task sequence in the order of short task priority. When the minimum heap is created, it only contains one node with an empty task sequence and a value of 0.

[0072] b) Each time a node is popped from the top of the minimum heap, for each task in the task set that is not in the incomplete task sequence of the node, it is added to the end of the incomplete task sequence to generate a new extended node. When the extended node meets the conditions corresponding to the pruning strategy, the extended node is pruned, otherwise, the extended node is added to the minimum heap.

[0073] c) When the task sequence in a node popped from the top of the minimum heap is a complete task sequence, the node is called the answer node. Among the answer nodes, the node with the minimum value has the minimum cost on GPU g, and its task sequence is the optimal task sequence on GPU g.

[0074] During the operation of the branch-and-bound search algorithm, the following pruning strategies are used to reduce the running time of the algorithm:

[0075] i) Relative position pruning: When there are two tasks without deadlines in the task sequence of a node, and their relative positions in the task sequence do not satisfy the short task priority order, the node is discarded.

[0076] ii) Constrained pruning: When the cost of a node’s task sequence is greater than the minimum cost of the current record, the node is discarded.

[0077] iii) Optimal prediction pruning: When a complete task sequence is generated on a node, and the second half of the task sequence (the second half includes all tasks in the task set that are not in the current node, and these tasks form a task sequence according to the short task priority method) does not cause any deadline violations, then the cost of the task sequence is the minimum cost that the node can generate. In this case, the node does not need to be expanded.

[0078] iv) Lower bound pruning: When a complete task sequence is generated on a node, the lower bound of the cost of the task sequence of the node is calculated, that is, the cost without considering the penalty item of the second half of the tasks violating the deadline, and if the cost is greater than the minimum cost of the current record, the node is discarded.

[0079] 4) Select the smallest minimum cost from all the minimum costs obtained in step 3). Note that this cost is calculated when adding task j' to the task set of GPU g', and the corresponding task sequence on GPU g' is S'. Then schedule task j' to GPU g', and set the task sequence on GPU g' to S'.

[0080] 5) Repeat steps 3) and 4) for N rounds until all tasks are scheduled.

[0081] Furthermore, the task deployer deploys the task to the GPU cluster according to the scheduling results obtained by the task scheduler. This system uses Kubernetes as a resource manager, and the task is deployed to the GPU cluster in the form of Pod, using the corresponding GPU resources.

[0082] In one embodiment of the present invention, a component structure of a deadline-aware deep learning training task scheduling method for efficiency is provided, such as Figure 1 As shown, the steps of the method are as follows:

[0083] Step 101: Based on the task set submitted by the user, generate corresponding task information for each task, including the task name, the number of training iterations of the task, and the deadline of the task; based on the available GPU resources in the GPU cluster, generate corresponding GPU information for each GPU, including the GPU model and the number of GPUs.

[0084] Step 102: Submit all task information to the runtime estimator. The runtime estimator samples the training iterations of the task using an adaptive sampling technique, and estimates the runtime of the task based on the sampling results.

[0085] Step 103: The runtime estimator returns the runtime of the task and adds the runtime of the task to the task information.

[0086] Step 104: Submit all task information and GPU information to the task scheduler, and the task scheduler creates corresponding internal data according to the task information and GPU information.

[0087] Step 105: The task scheduler runs the scheduling algorithm to generate a corresponding scheduling result.

[0088] Step 106: The task deployer deploys the tasks to the corresponding GPUs in the GPU cluster in the order of the scheduling results obtained in step 105 for execution.

[0089] In this embodiment, in a deadline-aware deep learning training task scheduling method for efficiency, the task running time estimator workflow is as follows: Figure 2 As shown, the specific implementation steps are as follows:

[0090] Step 201: The task running time estimator runs a training iteration, and during the iteration process, calculates the percentage difference between the running times of two adjacent iterations.

[0091] Step 202: If the percentage difference is less than the set threshold, it means that the training iteration time of the task has entered a stable state, and jump to step 203; otherwise, jump to step 201.

[0092] Step 203: The task running time estimator samples a fixed number of iterations to obtain sampled data.

[0093] Step 204: The task running time estimator calculates the estimated task running time according to the sampled data.

[0094] In this embodiment, in a method for scheduling deep learning training tasks with deadline awareness for efficiency, the workflow of the task scheduler is as follows: Figure 3 As shown, the specific implementation steps are as follows:

[0095] Step 301: The task scheduler obtains all unscheduled tasks, and adds each unscheduled task j to the task set of each GPU.

[0096] Step 302: The task scheduler generates a task sequence for the tasks on each GPU in a manner that gives priority to short tasks.

[0097] Step 303: Schedule each GPU according to the task sequence in step 302, and determine whether any task on each GPU violates the deadline. If no task deadline is violated on the GPU, jump to step 304; otherwise, jump to step 305.

[0098] Step 304: Calculate the cost of the task sequence obtained in step 302 and directly use it as the minimum cost of the GPU.

[0099] Step 305: Use a branch-and-bound search algorithm with pruning optimization to search for an optimal task sequence with minimum cost and the corresponding minimum cost.

[0100] Step 306: In steps 304 and 305, the minimum cost and the corresponding task sequence have been calculated for each GPU, and the minimum minimum cost, the task j' and GPU g' corresponding to this cost are selected, and the corresponding task sequence on GPU g' is S'. Then, task j' is scheduled to GPU g', the task sequence on GPU g' is set to S', and the status of task j' is marked as scheduled.

[0101] Step 307: Repeat steps 301-306 until all tasks are scheduled.

[0102] Based on the same inventive concept, another embodiment of the present invention provides an efficiency-oriented deadline-aware deep learning training task scheduling system, which is characterized by comprising:

[0103] A deep learning training task running time estimation module is used to estimate the running time of the deep learning training task through adaptive sampling technology, and add the running time of the deep learning training task to the task information;

[0104] The deep learning training task scheduling optimization module is used to schedule deep learning training tasks according to task information and GPU information. The scheduling goal is to minimize the overall task completion time while minimizing the task deadline violations, and deploy the deep learning training tasks to the corresponding GPUs in the GPU cluster in the order of the scheduling results for execution.

[0105] The specific implementation process of each module refers to the above description of the method of the present invention.

[0106] Based on the same inventive concept, another embodiment of the present invention provides an electronic device (computer, server, smart phone, etc.), which includes a memory and a processor, the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program includes instructions for executing each step in the method of the present invention.

[0107] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, CD), which stores a computer program. When the computer program is executed by a computer, it implements the various steps of the method of the present invention.

[0108] The above description of the specific implementation method of the embodiment of the present invention is only an example, and the protection scope of the present invention is explained by the claims. Any changes and modifications made by those skilled in the art based on the understanding of the above description shall fall within the protection scope of the present invention.

Claims

1. A deadline-aware deep learning training task scheduling method for efficiency, characterized in that: The following steps are involved: The running time of the deep learning training task is estimated through adaptive sampling technology, and the running time of the deep learning training task is added to the task information; Schedule deep learning training tasks based on task information and GPU information. The scheduling goal is to minimize the overall task completion time while minimizing the task deadline violations. Deploy the deep learning training tasks to the corresponding GPUs in the GPU cluster according to the order in the scheduling results for execution; The scheduling of the deep learning training task according to the task information and the GPU information includes: Step 1: The task scheduler obtains all unscheduled tasks and adds each unscheduled task j to the task set of each GPU; Step 2: The task scheduler generates a task sequence for each GPU in a short-term task priority manner; Step 3: Schedule each GPU according to the task sequence in step 2, and determine whether there is a task on each GPU that violates the deadline. If there is no task deadline violation on the GPU, jump to step 4, otherwise jump to step 5; Step 4: Calculate the cost of the task sequence obtained in step 2 and directly use it as the minimum cost of the GPU; Step 5: Search for the optimal task sequence with minimum cost and the corresponding minimum cost; Step 6: In steps 4 and 5, the minimum cost and the corresponding task sequence have been calculated for each GPU. Select the minimum minimum cost, the task j′ and GPU g corresponding to this cost. ′ , and GPUg ′ The corresponding task sequence is S′, then task j′ is scheduled to GPU g ′ On the GPU, set ′ The task sequence on is S ′ , and mark the status of task j′ as scheduled; Step 7: Repeat steps 1-6 until all tasks are scheduled; Among them, step 4 uses a unified cost function to simultaneously express the scheduling objectives of minimizing the task deadline violation and minimizing the overall task completion time, defining GPU g m Task sequence on The cost is: in, For task j n In GPU g m Waiting time on For task j n In GPU g m Running time on For task j n The deadline constraint is: f is a piecewise function defined as: is the penalty coefficient for task deadline violation.

2. The method according to claim 1, characterized in that The running time of the deep learning training task estimated by the adaptive sampling technology is to perform the sampling process when the iterative running time of the training task enters a stable state; the difference method is used to determine whether the iterative running time of the training task enters a stable state, that is: Let the running time of two adjacent iterations of the task be T1 and T2, then the difference in running time of two adjacent iterations is During the iteration process, set a stable state threshold When calculated When , it is considered that the iteration time of the task has entered a stable state.

3. The method according to claim 2, characterized in that The running time T of the deep learning training task is calculated using the following formula: Among them, I B represents the number of iterations before the task enters a stable state, T B represents the iteration running time of the task before entering the stable state, I represents the total number of iterations specified by the user, and I S represents the number of sampling iterations, T S Represents the running time of the sampling iteration.

4. The method according to claim 1, characterized in that: Step 5 uses a branch-and-bound search algorithm with pruning optimization to search for an optimal task sequence with minimum cost and a minimum cost corresponding thereto; the branch-and-bound search algorithm with pruning optimization includes the following steps: a) For each GPU, create a minimum heap to store nodes. Each node contains an incomplete task sequence and its corresponding value. The value of the node is defined as the cost of the complete task sequence generated by adding all tasks in the task set that are not in the incomplete task sequence of the node to the end of the incomplete task sequence in the order of short task priority. When the minimum heap is created, it only contains one node with an empty task sequence and a value of 0. b) Each time a node is popped from the top of the minimum heap, for each task in the task set that is not in the incomplete task sequence of the node, it is added to the end of the incomplete task sequence to generate a new extended node. When the extended node meets the conditions corresponding to the pruning strategy, the extended node is pruned, otherwise, the extended node is added to the minimum heap; c) When the task sequence in a node popped from the top of the minimum heap is a complete task sequence, the node is called the answer node; the node with the minimum value in the answer node has the minimum cost on GPU g, and its task sequence is the optimal task sequence on GPU g.

5. The method according to claim 4, characterized in that The branch-and-bound search algorithm with pruning optimization uses the following pruning strategies during operation to reduce the running time of the algorithm: i) Relative position pruning: When there are two tasks without deadlines in the task sequence of a node, and their relative positions in the task sequence do not satisfy the short task priority order, the node is discarded; ii) Constrained pruning: When the cost of a node’s task sequence is greater than the minimum cost of the current record, the node is discarded; iii) Optimal prediction pruning: When a complete task sequence is generated on a node and there is no deadline violation in the second half of the task sequence, the cost of the task sequence is the minimum cost that the node can generate. In this case, the node does not need to be expanded; iv) Lower bound pruning: When a complete task sequence is generated on a node, the lower bound of the cost of the task sequence of the node is calculated, that is, the cost without considering the penalty item of the second half of the tasks violating the deadline, and if the cost is greater than the minimum cost of the current record, the node is discarded.

6. An efficiency-oriented deadline-aware deep learning training task scheduling system, characterized in that: include: A deep learning training task running time estimation module is used to estimate the running time of the deep learning training task through adaptive sampling technology, and add the running time of the deep learning training task to the task information; The deep learning training task scheduling optimization module is used to schedule deep learning training tasks according to task information and GPU information. The scheduling goal is to minimize the overall task completion time while minimizing the task deadline violations, and deploy the deep learning training tasks to the corresponding GPUs in the GPU cluster in the order of the scheduling results for execution. The scheduling of the deep learning training task according to the task information and the GPU information includes: Step 1: The task scheduler obtains all unscheduled tasks and adds each unscheduled task j to the task set of each GPU; Step 2: The task scheduler generates a task sequence for each GPU in a short-term task priority manner; Step 3: Schedule each GPU according to the task sequence in step 2, and determine whether there is a task on each GPU that violates the deadline. If there is no task deadline violation on the GPU, jump to step 4, otherwise jump to step 5; Step 4: Calculate the cost of the task sequence obtained in step 2 and directly use it as the minimum cost of the GPU; Step 5: Search for the optimal task sequence with minimum cost and the corresponding minimum cost; Step 6: In steps 4 and 5, the minimum cost and the corresponding task sequence have been calculated for each GPU. Select the minimum minimum cost, the task j′ and GPU g corresponding to this cost. ′ , and GPUg ′ The corresponding task sequence is S′, then task j′ is scheduled to GPU g ′ On the GPU, set ′ The task sequence on is S ′ , and mark the status of task j′ as scheduled; Step 7: Repeat steps 1-6 until all tasks are scheduled; Among them, step 4 uses a unified cost function to simultaneously express the scheduling objectives of minimizing the task deadline violation and minimizing the overall task completion time, defining GPU g m Task sequence on The cost is: in, For task j n In GPU g m Waiting time on For task j n In GPU g m Running time on For task j n The deadline constraint is: f is a piecewise function defined as: is the penalty coefficient for task deadline violation.

7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method according to any one of claims 1 to 5 is implemented.

Images