A slo-aware throughput optimization GPU scheduling method and system for data parallel training

Abstract

The application discloses a kind of SLO perception throughput optimization GPU scheduling method and system for data parallel training, including obtaining the task set to be scheduled and its hyperparameter, and define the number of schedulable GPU device;Throughput model is constructed for each task;The minimum GPU quantity required for each task to meet its service level target SLO constraint is calculated;With the total throughput of all scheduled tasks as the goal, the cluster GPU resource capacity as the constraint, formalize the SLO perception throughput optimization problem;Using the method of grouping greed and marginal benefit driven, the optimization problem is converted into the solution of GPU allocation strategy.The application realizes the reasonable scheduling of data parallel training task in GPU cluster under the condition of meeting the SLO constraint of each scheduled task and the cluster GPU resource capacity constraint, improves the total throughput of system and guarantees the service quality of task.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of distributed computing, and particularly relates to an SLO-aware throughput optimization GPU scheduling method and system for data parallel training. BACKGROUND

[0002] With the rapid development of deep learning technology, the model size and the training data volume present exponential growth. The parameter size of large language models has broken through the trillion level, and training these models needs to consume a large amount of computing resources. In the face of the physical limitations of single card computing power and memory capacity, distributed training has become an inevitable choice for large-scale deep learning training. Among them, data parallel training has become the most widely used parallel strategy due to its simplicity and strong scalability.

[0003] In data parallel training, gradient synchronization is a key link that affects training efficiency. The all-reduce architecture avoids the center node bottleneck problem of the parameter server architecture by decentralizing the design, so that the devices in the cluster form a logical loop and directly exchange data with each other. However, in a multi-tenant graphics processing unit (GPU) cluster environment, how to efficiently schedule computing resources to maximize system throughput has become a core problem that the industry and academia are concerned about.

[0004] The existing scheduling scheme has two limitations: first, the strategy that takes cluster throughput as a single guide often ignores the quality of service requirements of individual tasks in a multi-task concurrent scene; second, the strategy that takes service level objective (SLO) as a hard constraint often sacrifices cluster throughput at the cost, and fails to establish a collaborative optimization model between task internal execution efficiency and cluster global scheduling. Therefore, there is an urgent need for a GPU scheduling method that can maximize the total throughput of the cluster while meeting the SLO constraints of each scheduled task. SUMMARY

[0005] To solve the problems existing in the prior art, the application provides an SLO-aware throughput optimization GPU scheduling method and system for data parallel training.

[0006] The technical scheme of the application is as follows: An SLO-aware throughput optimization GPU scheduling method for data parallel training is applied to a GPU cluster, and the method comprises the following steps: acquiring a task set to be scheduled and task hyperparameters, and defining the total number of schedulable GPU devices in the GPU cluster; Based on the task set, task hyperparameters, and total number of GPU devices, a throughput model is constructed for each task. The throughput model is used to characterize the functional relationship between the number of GPUs allocated to a task and the throughput of that task. Based on the throughput model, the minimum number of GPUs required for each task to meet its Service Level Target (SLO) constraint is calculated. The SLO constraint is defined by the task's expected runtime, the number of required iteration cycles, and the SLO sensitivity coefficient. With the goal of maximizing the total throughput of all scheduled tasks and constrained by the cluster's GPU resource capacity, we formalize the SLO-aware throughput optimization problem. Based on the SLO-aware throughput optimization problem, a method combining grouped greedy algorithm and marginal benefit-driven approach is adopted to transform the SLO-aware throughput optimization problem into a GPU allocation strategy problem and solve the GPU allocation strategy. The GPU allocation strategy includes task selection decision and GPU quantity allocation decision, thus completing the GPU scheduling of the data parallel training task.

[0007] Furthermore, the total number of schedulable GPUs in the cluster is defined as follows: The number of tasks to be scheduled is The set of tasks to be scheduled is ; For each task The number of iterations required to complete is Expected running time is SLO sensitivity coefficient is And the throughput on a single GPU is .

[0008] Furthermore, for each task Building a throughput model , Indicates when assigned to a task The number of GPUs is The task throughput at that time is calculated using the following formula: ; in, It is a task Given a number of GPUs The additional overhead incurred by using data parallelism is shown in the fitting function: ; and These are the first and second positive constants, respectively.

[0009] Furthermore, the task The SLO constraint is expressed as: ; Assigned to task Number of GPUs As decision variables, task Minimum number of GPUs required to satisfy SLO constraints Represented as: .

[0010] Furthermore, the formalized SLO-aware throughput optimization problem Represented as: ; , ; ; , ; in, Represents the set of non-negative integers.

[0011] Furthermore, the specific method for transforming the SLO-aware throughput optimization problem into a GPU allocation strategy problem by combining grouped greedy algorithms with marginal reward-driven approaches, and then solving the GPU allocation strategy, includes: The first stage involves GPU allocation based on a grouping greedy strategy. It calculates the minimum number of GPUs required to satisfy the SLO constraint for each task, sorts the task set in non-increasing order of unit throughput for each task, and iterates through all tasks whose size does not exceed a preset parameter. The task subset is assigned the minimum number of GPUs that satisfy the SLO constraint for each task in the subset. Then, resources are allocated to the remaining tasks in a non-increasing order of unit throughput. The allocation scheme that produces the maximum total throughput is recorded as the initial allocation result. The second phase involves the marginal revenue-driven reallocation of remaining GPUs. Based on the initial allocation results, if GPUs remain, they are then allocated one by one to the tasks with the highest current unit marginal throughput, from among the tasks with already allocated GPUs and the unallocated tasks requiring a minimum of one GPU to satisfy the SLO constraint. This continues until all GPUs are allocated. The task... The unit marginal throughput is .

[0012] Furthermore, the preset parameters This is used to control the enumeration range of the grouping greedy strategy.

[0013] A SLO-aware throughput optimization GPU scheduling system for data-parallel training includes a predefined module, a throughput model building module, a computation module, a formalization module, and a solution allocation module. The predefined module is used to obtain the set of tasks to be scheduled and the task hyperparameters, and to define the total number of schedulable GPU devices in the GPU cluster. The throughput model construction module is used to construct a throughput model for each task based on the task set, task hyperparameters, and the total number of GPU devices. The throughput model is used to characterize the functional relationship between the number of GPUs allocated to a task and the throughput of that task. The computing module is used to calculate, based on the throughput model, the minimum number of GPUs required for each task to meet its Service Level Target (SLO) constraint, which is defined by the task's expected runtime, the number of required iteration cycles, and the SLO sensitivity coefficient. The formal module is used to formalize the SLO-aware throughput optimization problem with the goal of maximizing the total throughput of all scheduled tasks and the constraint of cluster GPU resource capacity. The solution allocation module is used to transform the SLO-aware throughput optimization problem into a GPU allocation strategy problem by combining a grouped greedy approach with a marginal benefit-driven approach, and to solve the GPU allocation strategy. The GPU allocation strategy includes task selection decision and GPU quantity allocation decision, thereby completing the GPU scheduling of the data parallel training task.

[0014] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor being configured to invoke and run the computer program stored in the memory to perform any of the methods described above.

[0015] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention constructs a data-parallel training GPU scheduling model that includes SLO constraints and GPU resource capacity constraints. It formally defines the SLO-aware throughput optimization problem with the goal of maximizing the total task throughput, and realizes the collaborative optimization between task internal execution efficiency and cluster global scheduling.

[0017] 2. The GPU scheduling algorithm designed in this invention can quickly find a high-quality initial solution through a GPU allocation phase based on a grouping greedy strategy; and further optimize resource allocation through a marginal benefit-driven remaining GPU reallocation phase, thereby improving the throughput of the scheduling scheme. The time complexity of this algorithm is O(n log n). It can meet the solution requirements in actual scheduling scenarios.

[0018] 3. This invention introduces the SLO sensitivity coefficient. This allows the scheduling method to flexibly adapt to the sensitivity of different application scenarios to SLO constraints. In hard constraint scenarios, it prioritizes task timeliness, while in soft constraint scenarios, it focuses on improving resource utilization and overall throughput, thereby improving the availability and adaptability of the method. Attached Figure Description

[0019] Figure 1 A flowchart illustrating the process of optimizing GPU scheduling for SLO-aware throughput in data-parallel training. Figure 2 This is a schematic diagram of the system structure. Detailed Implementation

[0020] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0021] Example 1: This invention provides a SLO-aware throughput optimization GPU scheduling method for data-parallel training, applied to GPU clusters, comprising: Obtain the set of tasks to be scheduled and the task hyperparameters, and define the total number of schedulable GPU devices in the GPU cluster; Based on the task set, task hyperparameters, and total number of GPU devices, a throughput model is constructed for each task. The throughput model is used to characterize the functional relationship between the number of GPUs allocated to a task and the throughput of that task. Based on the throughput model, the minimum number of GPUs required for each task to meet its Service Level Target (SLO) constraint is calculated. The SLO constraint is defined by the task's expected runtime, the number of required iteration cycles, and the SLO sensitivity coefficient. With the goal of maximizing the total throughput of all scheduled tasks and constrained by the cluster's GPU resource capacity, we formalize the SLO-aware throughput optimization problem. Based on the SLO-aware throughput optimization problem, a method combining grouped greedy algorithm and marginal benefit-driven approach is adopted to transform the SLO-aware throughput optimization problem into a GPU allocation strategy problem and solve the GPU allocation strategy, which includes task selection decision and GPU quantity allocation decision, to complete the GPU scheduling of data parallel training tasks.

[0022] Example 2: This embodiment, based on Embodiment 1, is further designed in that the total number of schedulable GPUs in the cluster is defined as follows: The number of tasks to be scheduled is The set of tasks to be scheduled is ; For each task The number of iterations required to complete is Expected running time is SLO sensitivity coefficient is And the throughput on a single GPU is .

[0023] Example 3: This embodiment, based on embodiment two, is further designed in that, in this example, each task... Building a throughput model , Indicates when assigned to a task The number of GPUs is The task throughput at that time is calculated using the following formula: ; in, It is a task Given a number of GPUs The additional overhead incurred by using data parallelism is shown in the fitting function: ; and These are the first and second positive constants, respectively.

[0024] Example 4: This embodiment is a further design based on embodiment three, in that the task in this example... The SLO constraint is expressed as: ; Assigned to task Number of GPUs As decision variables, task Minimum number of GPUs required to satisfy SLO constraints Represented as: .

[0025] Example 5: This embodiment, based on Embodiment 4, further designs the formalized SLO-aware throughput optimization problem in this example. Represented as: ; , ; ; , ; in, Represents the set of non-negative integers.

[0026] in, The optimization objective is to maximize the total throughput of all scheduled tasks; This represents the SLO constraint for each task. Due to cluster GPU resource capacity constraints, For non-negative integer constraints on decision variables, Represents the set of non-negative integers. Indicates task Not scheduled during the current scheduling period; when At that time, the task The corresponding SLO constraints should be met.

[0027] Example 6: This embodiment, based on Embodiment 5, further designs the following: In this example, based on the SLO-aware throughput optimization problem, a combination of grouped greedy algorithms and marginal reward-driven methods is used to transform the SLO-aware throughput optimization problem into a GPU allocation strategy problem. The specific methods for solving the GPU allocation strategy include: The first stage involves GPU allocation based on a grouping greedy strategy. It calculates the minimum number of GPUs required to satisfy the SLO constraint for each task, sorts the task set in non-increasing order of unit throughput for each task, and iterates through all tasks whose size does not exceed a preset parameter. The task subset is assigned the minimum number of GPUs that satisfy the SLO constraint for each task in the subset. Then, resources are allocated to the remaining tasks in a non-increasing order of unit throughput. The allocation scheme that produces the maximum total throughput is recorded as the initial allocation result. The second phase involves a marginal revenue-driven reallocation of remaining GPUs. Based on the initial allocation, if GPUs remain, then... or In this task, the remaining GPUs are allocated one by one to the task with the highest current unit marginal throughput, until all GPUs are allocated, where the unit marginal throughput of task i is... .

[0028] The specific method of the above two-stage algorithm is as follows: Phase 1: GPU allocation based on a grouping greedy strategy, including: Step 1) Calculate each task Minimum number of GPUs required to satisfy SLO constraints ; Step 2) Set up the task set By each task Unit throughput Non-increasing sequential sorting; Step 3) Traverse all cases whose size does not exceed the preset parameter. For each subset of tasks, steps 3.1) to 3.2) are performed. ; Step 3.1) For each task in the subset distribute One GPU; Step 3.2) Process the remaining tasks in non-incremental order of unit throughput; when the number of remaining GPUs is not less than the minimum number of GPUs required for the current task to satisfy the SLO constraint, allocate the corresponding number of GPUs to the current task; otherwise, skip the current task and continue to process the next task until the GPU resources are exhausted or the task allocation is completed. Step 4) Record the GPU allocation scheme that maximizes the current total throughput that can be generated in Step 3) as the allocation result of Phase 1; Phase 2: Marginal revenue-driven reallocation of surplus GPUs, including: Step 5) Based on the allocation results of Phase 1, if there are still GPUs remaining, allocate the remaining GPUs to tasks one by one according to the method in Step 6) until all GPUs are allocated. Step 6) Each time an allocation is performed, if the conditions are met... or Select the task with the highest unit marginal throughput from the task list, and allocate 1 GPU to the selected task. The unit marginal throughput is ; Step 7) Return the GPU allocation scheme.

[0029] Example 7: This embodiment, based on Embodiment Six, is further designed in that it presets parameters. This is used to control the enumeration range of the grouping greedy strategy.

[0030] Example 8: The present invention provides a SLO-aware throughput optimization GPU scheduling system for data-parallel training, comprising a predefined module, a throughput model construction module, a computation module, a formalization module, and a solution allocation module; A predefined module is used to obtain the set of tasks to be scheduled and the task hyperparameters, and to define the total number of schedulable GPU devices in the GPU cluster; The throughput model building module is used to build a throughput model for each task based on the task set, task hyperparameters, and the total number of GPU devices. The throughput model is used to characterize the functional relationship between the number of GPUs allocated to a task and the throughput of that task. The computation module is used to calculate the minimum number of GPUs required for each task to meet its Service Level Target (SLO) constraints based on the throughput model. The SLO constraints are defined by the task's expected runtime, the number of required iteration cycles, and the SLO sensitivity coefficient. The formal module is used to formalize the SLO-aware throughput optimization problem with the goal of maximizing the total throughput of all scheduled tasks and constrained by the cluster's GPU resource capacity. The allocation module is used to solve the SLO-aware throughput optimization problem. It adopts a combination of grouped greedy algorithm and marginal benefit-driven approach to transform the SLO-aware throughput optimization problem into a GPU allocation strategy problem and solve the GPU allocation strategy. The GPU allocation strategy includes task selection decision and GPU quantity allocation decision, and completes the GPU scheduling for data parallel training tasks.

[0031] Example 9: An electronic device includes a memory and a processor. The memory stores a computer program, and the processor is used to invoke and run the computer program stored in the memory to perform the methods of any of the above embodiments.

[0032] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above embodiments.

[0033] Example 10: This embodiment provides a GPU scheduling method and system for SLO-aware throughput optimization oriented towards data-parallel training, such as... Figure 1 and Figure 2 As shown, it includes: S1: Obtain the set of tasks to be scheduled and define the number of GPU devices.

[0034] In an optional embodiment, consider a homogeneous GPU cluster system, where the total number of schedulable GPUs in the cluster is... Tasks arrive at the system's task queue sequentially. At the beginning of each scheduling cycle, the scheduler retrieves tasks from the task queue. A set of tasks to be scheduled And read the hyperparameters for each task. For each task Extract task parameter information, including: the number of iteration cycles required to complete the task. Expected running time SLO sensitivity coefficient and throughput on a single GPU .in, The estimation is obtained by starting model training on a single GPU, running a certain number of batches, and recording the total time consumed.

[0035] S2: Build a throughput model for each task.

[0036] In an optional embodiment, for each task Building a throughput model This indicates that when the number of GPUs allocated to this task is The model considers the communication and synchronization overhead in data-parallel training, and its calculation formula is as follows: ; in, It is a task Given a number of GPUs The additional overhead incurred by using data parallelism techniques. Based on hardware scaling trend studies, this overhead function can be fitted as: ; in, and All values ​​are positive constants, obtained by experimentally measuring the actual training throughput under different numbers of GPUs and performing curve fitting. From the above throughput model and additional overhead fitting function, it can be seen that... It exhibits the characteristic of diminishing marginal returns, meaning that as the number of GPUs allocated increases, the throughput gain from each additional GPU gradually decreases.

[0037] S3: Calculate the minimum number of GPUs required for each task to satisfy the SLO constraint.

[0038] In an optional embodiment, the task The SLO constraint means that the actual runtime does not exceed the expected runtime. (Task) Actual running time It can be calculated as: ; Therefore, SLO constraint This can be converted into a throughput constraint: ; To accommodate the varying degrees of sensitivity to SLO constraints across different application scenarios, an SLO sensitivity coefficient is introduced. Then the SLO constraint can be rewritten as: ; Assigned to task Number of GPUs For decision variables, the minimum number of GPUs required to satisfy the SLO constraint. for: ; S4: Formal SLO-aware throughput optimization problem.

[0039] In an optional embodiment, the primal scheduling optimization problem is formalized with the objective of maximizing the total throughput of all scheduled tasks and constrained by the cluster's GPU resource capacity: ; , ; ; , ; in, Represents the set of non-negative integers.

[0040] The optimization objective is to maximize the total throughput of all scheduled tasks. The first constraint is a conditional SLO constraint, which indicates that when... At that time, the task If not scheduled within the current scheduling period, when At that time, the task The corresponding SLO constraints should be met; the second constraint is the cluster GPU resource capacity constraint, and the third constraint is the non-negative integer constraint of the decision variables.

[0041] S5: A method combining grouped greedy algorithm and marginal reward-driven approach is used to solve the GPU allocation strategy.

[0042] Since solving the SLO-aware throughput optimization problem is quite complex, this invention proposes a GPU scheduling algorithm for solving it. The GSA (Grouping Greedy Scheduling Algorithm) algorithm consists of two stages: Stage 1 is GPU allocation based on a grouping greedy strategy, and Stage 2 is the reallocation of remaining GPUs driven by marginal benefits.

[0043] Input parameters include: task set Total number of GPUs SLO sensitivity coefficient Algorithm order Each task Number of iteration cycles Expected running time Throughput under single GPU .

[0044] Phase 1: GPU allocation based on a grouping greedy strategy: Step 1) Calculate each task Minimum number of GPUs required to satisfy SLO constraints ; Step 2) Set up the task set By each task Unit throughput A non-increasing order sort yields an ordered set of tasks. ; Step 3) Traverse all cases whose size does not exceed the preset parameter. For each subset of tasks, steps 3.1) to 3.2) are performed. ; Step 3.1) For each task in the subset distribute One GPU; Step 3.2) Process the remaining tasks in non-incremental order of unit throughput; when the number of remaining GPUs is not less than the minimum number of GPUs required for the current task to satisfy the SLO constraint, allocate the corresponding number of GPUs to the current task; otherwise, skip the current task and continue to process the next task until the GPU resources are exhausted or the task allocation is completed. Step 4) Record the GPU allocation scheme that maximizes the current total throughput that can be generated in Step 3) as the allocation result of Phase 1; Phase 2: Marginal revenue-driven reallocation of surplus GPUs, including: Step 5) Based on the allocation results of Phase 1, if there are still GPUs remaining, allocate the remaining GPUs to tasks one by one according to the method in Step 6) until all GPUs are allocated. Step 6) Each time an allocation is performed, if the conditions are met... or Select the task with the highest unit marginal throughput from the task list, and allocate 1 GPU to the selected task. The unit marginal throughput is ; Step 7) Return the GPU allocation scheme.

[0045] To further illustrate the specific implementation process of the method of this invention, a numerical example involving three tasks is also provided. This example demonstrates how, under conditions of limited GPU resources, the scheduling algorithm prioritizes high-yield tasks and abandons low-yield tasks, thereby maximizing total throughput.

[0046] Suppose that the set of tasks to be scheduled in a certain scheduling period is: ; The total number of schedulable GPUs in the cluster is: ; SLO sensitivity coefficient is: ; The order of the algorithm is: ; The communication overhead coefficient is taken as follows: ; The communication growth index is taken from: ; The parameters for each task are shown in Table 1: Table 1. Basic Parameters of the Task Step 1) Calculate the minimum number of GPUs required for each task to satisfy the SLO constraint: For task 1, the minimum required throughput is Based on the throughput model, additional overhead fitting function, and SLO constraints described above, calculate the minimum number of GPUs required to satisfy the SLO constraints: hour, Not satisfied; therefore, Solve the following inequalities: ; ; ; therefore .

[0047] For task 2, the minimum required throughput is Calculate the minimum number of GPUs required to satisfy the SLO constraint: hour, Not satisfied; therefore, Solve the following inequalities: ; ; ; therefore .

[0048] For task 3, the minimum required throughput is Calculate the minimum number of GPUs required to satisfy the SLO constraint: hour, Not satisfied; therefore, Solve the following inequalities: ; ; ; therefore .

[0049] The minimum number of GPUs required to meet the SLO for each task is summarized in Table 2: Table 2 Task SLO Constraints and Throughput Parameters Step 2) Calculate and sort the unit throughput; By unit throughput Non-increasing sorting yields an ordered set of tasks. .

[0050] Phase 1: GPU allocation based on a grouping greedy strategy; Iterate through all sizes not exceeding task subset And satisfy Where H' represents the total throughput corresponding to the current GPU allocation scheme, and N' represents the number of GPUs currently remaining.

[0051] subset : Assign to Task 1 One GPU, , ; Process remaining task 3 ( ), allocate 2 GPUs, , ; Process remaining task 2 ( ),jump over; Current total throughput Remaining GPUs .

[0052] subset : Assign to Task 3 One GPU, , ; Process remaining task 1 ( ), allocate 4 GPUs, , ; Process remaining task 2 ( ),jump over; Current total throughput Remaining GPUs .

[0053] subset : Assign to Task 2 One GPU, , ; Process remaining task 1 ( ), allocate 4 GPUs, , ; Process remaining task 3 ( ),jump over; Current total throughput Remaining GPUs .

[0054] The maximum total throughput generated by each subset is 21.01. Phase 1 output allocation scheme: , , Remaining GPUs .

[0055] Phase 2: Marginal Revenue-Driven Redistribution of Surplus GPUs Current number of remaining GPUs These GPUs need to be allocated to tasks one by one, selecting the task with the highest unit marginal throughput each time.

[0056] First, we calculate the throughput of each task with different numbers of GPUs to facilitate subsequent calculations of marginal returns. Based on the throughput model and the additional overhead fitting function described above, the throughput of each task with different numbers of GPUs is calculated as follows: Task 1 ( ): , ; , ; Task 3 ( ): , ; , ; Now, the remaining GPUs will be allocated round by round. The number of GPUs currently allocated to each task is: , .

[0057] Round 1 allocation: Calculate the unit marginal throughput for each task: Task 1: ; Task 3: .

[0058] The task with the greatest marginal benefit is task 1 (3.64), which adds one GPU to task 1. , .

[0059] Round 2 allocation: current , : Task 1: ; Task 3: .

[0060] The task with the greatest marginal benefit is task 1 (3.59), which adds one GPU to task 1. , .

[0061] Step 3) Output the scheduling result: The final GPU allocation scheme is as follows: Task 1 allocates 6 GPUs, throughput ; Task 2 is allocated 0 GPUs, throughput ; Task 3 allocates 2 GPUs, throughput ; Total throughput is .

[0062] This embodiment demonstrates that the GPU scheduling algorithm, through a greedy grouping strategy in Phase 1, ensures that each scheduled task receives at least the minimum GPU resources required to satisfy the SLO constraint, while the number of GPUs allocated to unscheduled tasks is 0. Then, through a marginal benefit-driven reallocation of remaining GPUs in Phase 2, the remaining resources are preferentially allocated to the task with the highest marginal benefit, thereby maximizing the total system throughput while satisfying the SLO constraints of each scheduled task. When the value is large, stage 1 can explore more task subsets and further approach the global optimal solution.

[0063] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A SLO-aware throughput optimization GPU scheduling method for data-parallel training, applied to GPU clusters, characterized in that, include: Obtain the set of tasks to be scheduled and the task hyperparameters, and define the total number of schedulable GPU devices in the GPU cluster; Based on the task set, task hyperparameters, and total number of GPU devices, a throughput model is constructed for each task. The throughput model is used to characterize the functional relationship between the number of GPUs allocated to a task and the throughput of that task. Based on the throughput model, the minimum number of GPUs required for each task to meet its Service Level Target (SLO) constraint is calculated. The SLO constraint is defined by the task's expected runtime, the number of required iteration cycles, and the SLO sensitivity coefficient. With the goal of maximizing the total throughput of all scheduled tasks and constrained by the cluster's GPU resource capacity, we formalize the SLO-aware throughput optimization problem. Based on the SLO-aware throughput optimization problem, a method combining grouped greedy algorithm and marginal benefit-driven approach is adopted to transform the SLO-aware throughput optimization problem into a GPU allocation strategy problem and solve the GPU allocation strategy. The GPU allocation strategy includes task selection decision and GPU quantity allocation decision, thus completing the GPU scheduling of the data parallel training task.

2. The SLO-aware throughput optimization GPU scheduling method for data-parallel training according to claim 1, characterized in that, Define the total number of schedulable GPUs in the cluster as follows: The number of tasks to be scheduled is The set of tasks to be scheduled is ; For each task The number of iterations required to complete is Expected running time is SLO sensitivity coefficient is And the throughput on a single GPU is .

3. The SLO-aware throughput optimization GPU scheduling method for data-parallel training according to claim 2, characterized in that, For each task Building a throughput model , Indicates when assigned to a task The number of GPUs is The task throughput at that time is calculated using the following formula: ； in, It is a task Given a number of GPUs The additional overhead incurred by using data parallelism is shown in the fitting function: ; and These are the first and second positive constants, respectively.

4. The SLO-aware throughput optimization GPU scheduling method for data-parallel training according to claim 3, characterized in that, Task The SLO constraint is expressed as: ; Assigned to task Number of GPUs As decision variables, task Minimum number of GPUs required to satisfy SLO constraints Represented as: .

5. The SLO-aware throughput optimization GPU scheduling method for data-parallel training according to claim 4, characterized in that, Formal SLO-aware throughput optimization problem Represented as: ； ， ； ； ， ； in, Represents the set of non-negative integers.

6. The SLO-aware throughput optimization GPU scheduling method for data-parallel training according to claim 5, characterized in that, The specific method for transforming the SLO-aware throughput optimization problem into a GPU allocation strategy problem, and then solving the GPU allocation strategy, based on a combination of grouped greedy algorithms and marginal reward-driven approaches, includes: The first stage involves GPU allocation based on a grouping greedy strategy. It calculates the minimum number of GPUs required to satisfy the SLO constraint for each task, sorts the task set in non-increasing order of unit throughput for each task, and iterates through all tasks whose size does not exceed a preset parameter. The task subset is assigned the minimum number of GPUs that satisfy the SLO constraint for each task in the subset. Then, resources are allocated to the remaining tasks in a non-increasing order of unit throughput. The allocation scheme that produces the maximum total throughput is recorded as the initial allocation result. The second phase involves the marginal revenue-driven reallocation of remaining GPUs. Based on the initial allocation results, if GPUs remain, they are then allocated one by one to the tasks with the highest current unit marginal throughput, from among the tasks with already allocated GPUs and the unallocated tasks requiring a minimum of one GPU to satisfy the SLO constraint. This continues until all GPUs are allocated. The task... The unit marginal throughput is .

7. The SLO-aware throughput optimization GPU scheduling method for data-parallel training according to claim 6, characterized in that, The preset parameters This is used to control the enumeration range of the grouping greedy strategy.

8. A GPU scheduling system for SLO-aware throughput optimization oriented to data-parallel training, characterized in that, It includes predefined modules, throughput model building modules, calculation modules, formalization modules, and solution allocation modules; The predefined module is used to obtain the set of tasks to be scheduled and the task hyperparameters, and to define the total number of schedulable GPU devices in the GPU cluster. The throughput model construction module is used to construct a throughput model for each task based on the task set, task hyperparameters, and the total number of GPU devices. The throughput model is used to characterize the functional relationship between the number of GPUs allocated to a task and the throughput of that task. The computing module is used to calculate, based on the throughput model, the minimum number of GPUs required for each task to meet its Service Level Target (SLO) constraint, which is defined by the task's expected runtime, the number of required iteration cycles, and the SLO sensitivity coefficient. The formal module is used to formalize the SLO-aware throughput optimization problem with the goal of maximizing the total throughput of all scheduled tasks and the constraint of cluster GPU resource capacity. The solution allocation module is used to transform the SLO-aware throughput optimization problem into a GPU allocation strategy problem by combining a grouped greedy approach with a marginal benefit-driven approach, and to solve the GPU allocation strategy. The GPU allocation strategy includes task selection decision and GPU quantity allocation decision, thereby completing the GPU scheduling of the data parallel training task.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor being configured to invoke and run the computer program stored in the memory to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7 above.

Images