Deep learning load-oriented carbon efficient cluster scheduling system, method and product

Abstract

The invention provides a deep learning load-oriented carbon efficient cluster scheduling system, method and product, and relates to the technical field of cluster scheduling, and the method comprises a GPU scheduler which is used for generating a scheduling scheme for a GPU cluster; the GPU scheduler at least comprises a first scheduling module used for calculating a first marginal income of each deep learning training task and generating a first scheduling scheme according to the first marginal income; the first scheduling scheme is a scheduling scheme aiming at increasing the number of deep learning training tasks meeting the deadline requirement; the second scheduling module is used for calculating a second marginal income of each deep learning training task and generating a second scheduling scheme according to the second marginal income; the third scheduling module is used for calculating a third marginal income of each deep learning training task and generating a third scheduling scheme according to the third marginal income; and the GPU cluster is used for executing a deep learning training task according to the scheduling scheme generated by the GPU scheduler.

Description

Technical Field

[0001] The present application relates to the field of cluster scheduling technology, and in particular to a carbon-efficient cluster scheduling system, method and product for deep learning loads. Background Art

[0002] Deep learning is widely used in various applications and services used in daily life. Therefore, training deep learning models has become an important workload in data centers. This growing computing demand has led to a simultaneous increase in energy demand and increased carbon emissions.

[0003] However, the current method of scheduling deep learning training tasks in GPU clusters does not consider energy consumption or carbon emissions during the scheduling process. Not only does it lack flexibility in the process of scheduling GPUs, it is also easy to generate unnecessary carbon emissions. Therefore, it is necessary to develop a carbon-efficient cluster scheduling system, method and product for deep learning loads to improve the carbon emission efficiency of deep learning training tasks and increase operational benefits. Summary of the invention

[0004] In view of the above problems, the embodiments of the present application provide a carbon-efficient cluster scheduling system, method and product for deep learning loads to overcome the above problems or at least partially solve the above problems.

[0005] In a first aspect of an embodiment of the present application, a carbon-efficient cluster scheduling system for deep learning loads is provided, the system comprising: A GPU scheduler is used to generate a scheduling plan for the GPU cluster according to the current deep learning training task and cluster status information, wherein the scheduling plan includes: the execution priority order of the deep learning training task, and the number of GPUs and GPU frequency used by each deep learning training task; The GPU scheduler includes at least: a first scheduling module, a second scheduling module and a third scheduling module; The first scheduling module is used to calculate the first marginal benefit of each deep learning training task, and generate a first scheduling plan according to the first marginal benefit; the first scheduling plan is a scheduling plan aimed at increasing the number of deep learning training tasks that meet the deadline requirements; the first marginal benefit is calculated based on the ratio between the reduction in the number of tasks that can be completed by the GPU cluster after adding m GPUs to the deep learning training task and the increase in carbon emissions after adding m GPUs; wherein m represents the minimum number of GPUs required to complete the deep learning training task within the deadline; The second scheduling module is used to calculate the second marginal benefit of each deep learning training task, and generate a second scheduling plan according to the second marginal benefit; the second scheduling plan is a scheduling plan aimed at improving the fairness of completion time; the second marginal benefit is calculated based on the ratio between the reduction in the fairness of completion time after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU; The third scheduling module is used to calculate the third marginal benefit of each deep learning training task, and generate a third scheduling plan according to the third marginal benefit; the third scheduling plan is a scheduling plan aimed at reducing the average job completion time of the GPU cluster; the third marginal benefit is calculated based on the ratio between the reduction in job completion time after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU; The GPU cluster is used to execute deep learning training tasks according to the scheduling scheme generated by the GPU scheduler.

[0006] The second aspect of the embodiment of the present application further provides a carbon-efficient cluster scheduling method, which is applied to the carbon-efficient cluster scheduling system for deep learning loads described in the first aspect of the embodiment of the present application, and the method includes: Generate a scheduling plan for the GPU cluster according to the current deep learning training task and cluster status information, wherein the scheduling plan includes: the execution priority order of the deep learning training tasks, and the number of GPUs and GPU frequency used by each deep learning training task; Execute deep learning training tasks according to the scheduling scheme generated by the GPU scheduler; The generating of a scheduling scheme for a GPU cluster includes: Calculate the first marginal benefit of each deep learning training task, and generate a first scheduling plan according to the first marginal benefit; the first scheduling plan is a scheduling plan aimed at increasing the number of deep learning training tasks that meet the deadline requirements; the first marginal benefit is calculated based on the ratio between the reduction in the number of tasks that can be completed by the GPU cluster after adding m GPUs to the deep learning training task and the increase in carbon emissions after adding m GPUs; wherein m represents the minimum number of GPUs required to complete the deep learning training task within the deadline; Calculate the second marginal benefit of each deep learning training task, and generate a second scheduling plan according to the second marginal benefit; the second scheduling plan is a scheduling plan aimed at improving the fairness of completion time; the second marginal benefit is calculated based on the ratio between the reduction in the fairness of completion time after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU; Calculate the third marginal benefit of each deep learning training task, and generate a third scheduling plan according to the third marginal benefit; the third scheduling plan is a scheduling plan aimed at reducing the average job completion time of the GPU cluster; the third marginal benefit is calculated based on the ratio between the reduction in job completion time after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU.

[0007] A third aspect of an embodiment of the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the steps in the carbon-efficient cluster scheduling method described in the second aspect of an embodiment of the present application.

[0008] The fourth aspect of the embodiments of the present application further provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps in the carbon-efficient cluster scheduling method described in the second aspect of the embodiments of the present application are implemented.

[0009] The fifth aspect of the embodiments of the present application further provides a computer program product. When the computer program product is run on an electronic device, the processor is enabled to implement the steps in the carbon-efficient cluster scheduling method as described in the second aspect of the embodiments of the present application.

[0010] An embodiment of the present application provides a carbon-efficient cluster scheduling system for deep learning loads, the system comprising: a GPU scheduler, for generating a scheduling scheme for a GPU cluster according to current deep learning training tasks and cluster status information, the scheduling scheme comprising: an execution priority order for deep learning training tasks, and the number of GPUs and GPU frequency used by each deep learning training task; the GPU scheduler comprises at least: a first scheduling module, a second scheduling module and a third scheduling module; the first scheduling module is used to calculate a first marginal benefit for each deep learning training task, and generate a first scheduling scheme according to the first marginal benefit; the first scheduling scheme is a scheduling scheme aimed at increasing the number of deep learning training tasks that meet deadline requirements; the first marginal benefit is calculated based on the ratio between the reduction in the number of tasks that can be completed by the GPU cluster after adding m GPUs to the deep learning training task and the increase in carbon emissions after adding m GPUs; wherein m represents the carbon emission of the deep learning training task at the deadline. time; the second scheduling module is used to calculate the second marginal benefit of each deep learning training task, and generate a second scheduling plan according to the second marginal benefit; the second scheduling plan is a scheduling plan aimed at improving the fairness of completion time; the second marginal benefit is calculated based on the ratio between the reduction in the fairness of completion time after adding 1 GPU to the increase in carbon emissions after adding 1 GPU to the deep learning training task; the third scheduling module is used to calculate the third marginal benefit of each deep learning training task, and generate a third scheduling plan according to the third marginal benefit; the third scheduling plan is a scheduling plan aimed at reducing the average job completion time of the GPU cluster; the third marginal benefit is calculated based on the ratio between the reduction in the job completion time after adding 1 GPU to the deep learning training task and the increase in carbon emissions after adding 1 GPU; the GPU cluster is used to execute the deep learning training task according to the scheduling plan generated by the GPU scheduler.

[0011] The embodiment of the present application utilizes three modules (first scheduling module, second scheduling module and third scheduling module) in the GPU scheduler to generate scheduling schemes for the GPU cluster from three aspects respectively, thereby determining the scheduling scheme to be executed in the end based on the three generated schemes (first scheduling scheme, second scheduling scheme and third scheduling scheme), and then enabling the GPU cluster to execute the deep learning training task according to the scheduling scheme. The three scheduling modules respectively execute different scheduling algorithms to calculate the marginal benefit of each task, and the carbon emissions generated by executing the task are taken into account when calculating the marginal benefit, thereby incorporating carbon emissions into the consideration factors of the scheduling scheme, which realizes more flexible GPU scheduling on the one hand, and avoids unnecessary carbon consumption in the GPU scheduling process on the other hand. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0013] Figure 1 It is a structural diagram of a carbon-efficient cluster scheduling system for deep learning loads provided in an embodiment of the present application; Figure 2 This is a flow chart of a GPU cluster scheduling method provided in an embodiment of the present application; Figure 3 It is a schematic diagram of a cluster scheduling process provided by an embodiment of the present application; Figure 4 It is a flowchart of the steps of a carbon efficient cluster scheduling method provided by an embodiment of the present application; Figure 5 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] The exemplary embodiments of the present application will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to enable the scope of the present application to be fully communicated to those skilled in the art.

[0015] Deep learning has been widely used in various applications and services used in daily life. Therefore, training deep learning models has become an important load in data centers. In order to improve the accuracy of deep learning models, developers need a large amount of training data; at the same time, the structure of deep learning models has become more and more complex, and the number of parameters contained is increasing. However, this growing computing demand has led to greater energy demand and carbon emissions. Many cloud computing service providers have formulated plans to reduce carbon emissions. Among them, reducing the total carbon emissions of deep learning training jobs is crucial to achieving these plans. However, the current method of scheduling deep learning training tasks in GPU clusters does not consider either energy consumption or carbon emissions during the scheduling process. Not only does it lack flexibility in the process of scheduling GPUs, it is also prone to unnecessary carbon emissions.

[0016] In view of the above problems, the embodiments of the present application propose a carbon-efficient cluster scheduling system, method and product for deep learning loads to improve the carbon emission efficiency of deep learning training tasks and increase operating benefits. In conjunction with the accompanying drawings, the carbon-efficient cluster scheduling system for deep learning loads provided by the embodiments of the present application is described in detail through some embodiments and their application scenarios.

[0017] The first aspect of the embodiment of the present application provides a carbon efficient cluster scheduling system for deep learning loads, referring to Figure 1 , Figure 1 A structural diagram of a carbon-efficient cluster scheduling system for deep learning loads provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the system comprises: The GPU scheduler is used to generate a scheduling plan for the GPU cluster based on the current deep learning training task and cluster status information. The scheduling plan includes: the execution priority order of the deep learning training tasks, and the number of GPUs and GPU frequency used by each deep learning training task.

[0018] The GPU scheduler includes at least: a first scheduling module, a second scheduling module and a third scheduling module; The first scheduling module is used to calculate the first marginal benefit of each deep learning training task, and generate a first scheduling plan based on the first marginal benefit; the first scheduling plan is a scheduling plan aimed at increasing the number of deep learning training tasks that meet the deadline requirements; the first marginal benefit is calculated based on the ratio between the reduction in the number of tasks that can be completed by the GPU cluster after adding m GPUs to the deep learning training task, and the increase in carbon emissions after adding m GPUs; wherein m represents the minimum number of GPUs required to complete the deep learning training task within the deadline.

[0019] The second scheduling module is used to calculate the second marginal benefit of each deep learning training task, and generate a second scheduling plan based on the second marginal benefit; the second scheduling plan is a scheduling plan aimed at improving the fairness of completion time; the second marginal benefit is calculated based on the ratio between the reduction in completion time fairness after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU.

[0020] The third scheduling module is used to calculate the third marginal benefit of each deep learning training task, and generate a third scheduling plan based on the third marginal benefit; the third scheduling plan is a scheduling plan aimed at reducing the average job completion time of the GPU cluster; the third marginal benefit is calculated based on the ratio between the reduction in job completion time after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU.

[0021] The GPU cluster is used to execute deep learning training tasks according to the scheduling scheme generated by the GPU scheduler.

[0022] In this embodiment, the specific processing flow of the carbon efficient cluster scheduling system is as follows: First, the developer of the deep learning training task submits the code, data set, termination conditions, optimizer and other information required for the deep learning training task (no resource usage configuration is required). Each deep learning training task needs to be executed by one or more GPUs. Different jobs have different parameter quantities and data set sizes, and require different execution times and different energy consumption to complete the execution.

[0023] Then, the GPU scheduler in the system allocates GPU resources for the submitted multiple deep learning training tasks and generates the latest scheduling plan for the GPU cluster. Specifically, the GPU scheduler receives cluster status information from the GPU cluster in real time through the cluster monitoring module, and calculates the number of cluster resources and resource-related configurations that can be used by each current deep learning training task through the resource allocation module, that is, according to the carbon emission budget of the GPU cluster and the carbon intensity of the power grid at the current moment, allocates the number of GPU resources used for each deep learning training task, sets the frequency of the GPU used, and outputs the scheduling plan. Among them, the GPU scheduler also needs to obtain the carbon intensity at the current moment to generate the scheduling plan. Carbon intensity represents the amount of carbon dioxide emissions generated by each unit of energy consumption on the power grid. It should be noted that carbon intensity changes in real time, that is, the carbon emissions corresponding to each unit of energy consumption are different at different times. For example, for the same GPU, the same deep learning training task is executed in different time periods, and the same amount of energy consumption is generated, and the carbon emissions generated are different (in the time period with high carbon intensity, the carbon emissions generated are higher, and in the time period with low carbon intensity, the carbon emissions generated are lower). In this embodiment, the system also includes: a carbon intensity acquisition module, which obtains in real time from the power grid data system the amount of carbon dioxide generated by each unit of electricity when the power grid generates electricity (i.e., carbon intensity), and sends this data to the scheduler, so that the scheduler can adjust the GPU scheduling plan in real time according to the change in carbon intensity.

[0024] Finally, the GPU cluster allocates specific GPUs and GPU frequencies in the GPU cluster for each deep learning training task according to the scheduling plan finally output by the GPU scheduler, so that the execution module in the GPU cluster uses the allocated GPU to execute the corresponding deep learning training task.

[0025] Reference Figure 2 , Figure 2 A schematic diagram of a process flow generated by a GPU cluster scheduling method is shown in FIG. Figure 2 As shown, the system obtains real-time carbon intensity data through the carbon intensity acquisition module, and the resource allocation module (including the first scheduling module, the second scheduling module and the third scheduling module) in the GPU scheduler generates a suitable scheduling plan based on the deep learning training task to be executed, the carbon intensity data, and the cluster status. The job placement module sends the scheduling plan to the GPU cluster, and the execution module in the GPU cluster executes the scheduling plan. In addition, the cluster state manager updates the cluster status according to the scheduling plan issued by the scheduler. The GPU cluster sends information related to the job performance of the cluster to the performance model manager, which performs performance prediction and sends the predicted information to the scheduler. The scheduler calculates the marginal benefit based on the performance prediction information (for example, the execution time and carbon emissions required for a deep learning training task to be executed under a certain number of GPUs), and then generates a scheduling plan. In addition, the embodiment of the present application also proposes a carbon intensity prediction module for predicting future carbon intensity, so that the scheduler can use the prediction results to adjust the scheduling plan.

[0026] In this embodiment, the GPU scheduler includes at least three scheduling modules: a first scheduling module, a second scheduling module and a third scheduling module, each scheduling module generates a scheduling scheme with different objectives. The following embodiments 1 to 3 respectively illustrate the specific execution process of each scheduling module.

[0027] Embodiment 1 This first embodiment is used to illustrate the specific execution process of the first scheduling module.

[0028] The first scheduling module is used to calculate the first marginal benefit of each deep learning training task, and generate a first scheduling plan based on the first marginal benefit. Among them, the first scheduling plan is a scheduling plan aimed at increasing the number of deep learning training tasks that meet the deadline requirements. That is, for each deep learning training task, a deadline is set, and the task is considered successfully completed only if the task is completed within the deadline. The first scheduling module generates the first scheduling plan with the scheduling goal of completing as many tasks as possible within the deadline, and the total carbon emissions of the GPU cluster not exceeding the carbon emission budget. The specific process of generating the first scheduling plan is as follows: Step S101: The input of the first scheduling module is the information related to the current deep learning training task (the deadline of each task), the cluster status information (including the number of available GPUs), and the carbon intensity value at the current moment, so that the first scheduling module calculates the first marginal benefit according to the input.

[0029] Among them, the first marginal benefit is calculated based on the ratio between the reduction in the number of tasks that can be completed by the GPU cluster after adding m GPUs to the deep learning training task, and the increase in carbon emissions after adding m GPUs; wherein m represents the minimum number of GPUs required to complete the deep learning training task within the deadline.

[0030] In a possible implementation manner, the first marginal benefit is calculated according to the following formula: ; in, Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f At 0:00, the number of deep learning training tasks that meet the deadline requirements of the GPU cluster, Indicates that n+m GPUs are used in deep learning training task j and the GPU frequency is f At 0, the number of deep learning training tasks that the GPU cluster meets the deadline requirements, m represents the minimum number of GPUs required to complete the deep learning training task j within the deadline based on the already configured n GPUs, and N0 represents the total number of deep learning training tasks; Indicates that n+m GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are C 0 represents the average carbon emissions generated by all deep learning training tasks. Among them, it can be calculated using the performance model manager. For each deep learning training task j , , and .

[0031] Step S102: Generate a first GPU quantity priority queue according to the first marginal benefit of each deep learning training task. That is, the larger the first marginal benefit, the higher the priority in the queue.

[0032] Step S103, re-allocate m GPUs for the deep learning training task with the highest priority in the first GPU number priority queue. That is, for the task j with the highest priority in the queue, n GPUs have been allocated in step S101, and m GPUs are added on this basis.

[0033] Step S104, repeating the above steps until a first termination condition is reached, wherein the first termination condition is: all GPUs in the GPU cluster are allocated, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget; Specifically, the embodiment of the present application uses a greedy method to allocate the number of GPUs used for each deep learning job in the GPU cluster. First, the number of GPUs for all deep learning training tasks in the GPU cluster is set to 0. Then, by calculating the first marginal benefit, a priority queue is maintained for all tasks, and then the idle GPUs are allocated to the tasks with the largest first marginal benefits. The task is removed from the head of the queue, and steps S101-S103 are re-executed, that is, the first marginal benefits of all tasks are recalculated, the priority queue is regenerated, and m GPUs are added to the task with the highest priority. After each iteration (i.e., steps S101-S103) is completed, it is determined whether the first termination condition is met.

[0034] Step S105, increase the unit frequency of the GPU frequency allocated to the deep learning training task with the highest priority. Specifically, increase the unit frequency based on the allocated GPU frequency. In this embodiment, the specific numerical value of the unit frequency is not limited.

[0035] Step S106, calculate the first marginal benefit of each deep learning training task and generate a first GPU frequency priority queue. Similarly, the larger the first marginal benefit, the higher the priority of the task in the queue.

[0036] Step S107: Increase the GPU frequency assigned to the deep learning training task with the highest priority in the first GPU frequency priority queue by one unit frequency. .

[0037] Step S108, recalculate the first marginal benefit and generate a new first GPU frequency priority queue until a second termination condition is reached, wherein the second termination condition is: the frequencies of all GPUs in the GPU cluster reach the maximum frequency value, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget.

[0038] Specifically, maintain a priority queue for all tasks, and then place a unit frequency Assign to the task with the largest first marginal benefit. Remove the task from the head of the queue and re-execute steps S106-S107, that is, recalculate the first marginal benefits of all tasks, regenerate the priority queue, and increase the unit frequency for the task with the highest priority. After each iteration (i.e., step S106-step S107) is completed, the carbon emissions generated by the target scheduling solution are calculated (the total carbon emissions of the scheduling solution of the current iteration can be calculated through the performance model manager and real-time carbon intensity) to determine whether the second termination condition is met.

[0039] Step S109: Generate a first scheduling scheme according to the first GPU frequency priority queue finally obtained.

[0040] The first scheduling scheme includes the number of GPUs and GPU frequency allocated to each deep learning training task, which ensures that the final generated scheduling scheme does not exceed the carbon emission budget, achieves carbon efficiency, and maximizes the number of deep learning training tasks that meet the deadline requirements.

[0041] Embodiment 2 The second embodiment is used to illustrate the specific execution process of the second scheduling module.

[0042] The second scheduling module is used to calculate the second marginal benefit of each deep learning training task and generate a second scheduling plan based on the second marginal benefit. The second scheduling module is a scheduling plan with the goal of improving the fairness of completion time. That is, for the second scheduling module, the second scheduling plan is generated with the highest fairness of completion time and the total carbon emissions of the GPU cluster not exceeding the carbon emission budget as the scheduling target. The specific process of generating the second scheduling plan is as follows: Step S201: The input of the second scheduling module is the information related to the current deep learning training task (the deadline of each task), the cluster status information (including the number of available GPUs), and the carbon intensity value at the current moment, so that the second scheduling module calculates the second marginal benefit according to the input.

[0043] The second marginal benefit is calculated based on the ratio between the reduction in the fairness of the completion time of the deep learning training task after adding one GPU and the increase in carbon emissions after adding one GPU.

[0044] In a possible implementation manner, the second marginal benefit is calculated according to the following formula: ; in, Indicates that n GPUs are used in deep learning training task j and the GPU frequency is fThe fairness of the completion time at 0 o'clock, Indicates that n+1 GPUs are used in deep learning training task j and the GPU frequency is f The fairness of the completion time of 0 hours, Indicates the completion time fairness of the GPU cluster; Indicates that n+1 GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f 0, the carbon emissions generated, C 0 represents the average carbon emissions generated by all deep learning training tasks. Among them, it can be calculated using the performance model manager. For each deep learning training task j , .

[0045] Among them, the fairness of completion time can be calculated according to the following formula: ;in, It represents the time required to complete a deep learning training task using a GPU that is assigned to it. It is the time required to complete the deep learning training task when all tasks are executed together in the GPU cluster. It also includes the slowdown time caused by task placement due to the order in which tasks are executed, as well as the queue delay time when tasks are scheduled in the GPU cluster.

[0046] Step S202: Generate a second GPU quantity priority queue according to the second marginal benefit of each deep learning training task. That is, the larger the second marginal benefit, the higher the priority in the queue.

[0047] Step S203, allocate another GPU to the deep learning training task with the highest priority in the second GPU number priority queue. That is, for the task j with the highest priority in the queue, n GPUs have been allocated in step S201, and one GPU is added on this basis.

[0048] Step S204, repeating the above steps until a first termination condition is reached, wherein the first termination condition is: all GPUs in the GPU cluster are allocated, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget; Step S205: Increase the GPU frequency allocated to the deep learning training task with the highest priority by one unit frequency. Specifically, increase the unit frequency based on the allocated GPU frequency. In this embodiment, the specific numerical value of the unit frequency is not limited.

[0049] Step S206, calculate the second marginal benefit of each deep learning training task and generate a second GPU frequency priority queue. Similarly, the larger the second marginal benefit, the higher the priority of the task in the queue.

[0050] Step S207: Increase the GPU frequency assigned to the deep learning training task with the highest priority in the second GPU frequency priority queue by one unit frequency. .

[0051] Step S208, recalculate the second marginal benefit and generate a new second GPU frequency priority queue until a second termination condition is reached, wherein the second termination condition is: the frequencies of all GPUs in the GPU cluster reach the maximum frequency value, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget.

[0052] Step S209: Generate a second scheduling scheme according to the second GPU frequency priority queue finally obtained.

[0053] The second scheduling scheme includes the number of GPUs and GPU frequency allocated to each deep learning training task, which ensures that the final generated scheduling scheme does not exceed the carbon emission budget, achieves carbon efficiency, and maximizes the fairness of the completion time of deep learning training tasks.

[0054] Embodiment 3 This third embodiment is used to illustrate the specific execution process of the third scheduling module.

[0055] The third scheduling module is used to calculate the third marginal benefit of each deep learning training task and generate a third scheduling plan based on the third marginal benefit. The third scheduling plan is a scheduling plan that aims to reduce the average job completion time of the GPU cluster. That is, for the third scheduling module, the third scheduling plan is generated with the minimum average job completion time and the total carbon emissions of the GPU cluster not exceeding the carbon emission budget as the scheduling target. The specific generation process of the third scheduling plan is as follows: Step S301: The input of the third scheduling module is the information related to the current deep learning training task (the deadline of each task), the cluster status information (including the number of available GPUs), and the carbon intensity value at the current moment, so that the third scheduling module calculates the third marginal benefit according to the input.

[0056] The third marginal benefit is calculated based on the ratio between the reduction in job completion time after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU.

[0057] In a possible implementation manner, the third marginal benefit is calculated according to the following formula: ; in, Indicates that deep learning training task j is using n GPUs and the GPU frequency is f The duration of the job completion at 0 o'clock; Indicates that deep learning training task j uses n+1 GPUs with GPU frequency of f The job completion time at 0:00; JCT0 represents the average job completion time of all deep learning training tasks; Indicates that n+1 GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f 0, the carbon emissions generated, C 0 represents the average carbon emissions generated by all deep learning training tasks.

[0058] Step S302: Generate a third GPU quantity priority queue according to the third marginal benefit of each deep learning training task. That is, the greater the third marginal benefit, the higher the priority in the queue.

[0059] Step S303, allocate another GPU to the deep learning training task with the highest priority in the third GPU quantity priority queue. That is, for the task j with the highest priority in the queue, n GPUs have been allocated in step S301, and one GPU is added on this basis.

[0060] Step S304, repeating the above steps until a first termination condition is reached, wherein the first termination condition is: all GPUs in the GPU cluster are allocated, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget; Step S305: Increase the unit frequency of the GPU frequency allocated to the deep learning training task with the highest priority by one. Specifically, increase the unit frequency based on the allocated GPU frequency. In this embodiment, the specific numerical value of the unit frequency is not limited.

[0061] Step S306, calculate the third marginal benefit of each deep learning training task and generate a third GPU frequency priority queue. Similarly, the greater the third marginal benefit, the higher the priority of the task in the queue.

[0062] Step S307: Increase the GPU frequency assigned to the deep learning training task with the highest priority in the third GPU frequency priority queue by one unit frequency. .

[0063] Step S308, recalculate the third marginal benefit and generate a new third GPU frequency priority queue until the second termination condition is reached, wherein the second termination condition is: the frequencies of all GPUs in the GPU cluster reach the maximum frequency value, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget.

[0064] Step S309: Generate a third scheduling scheme according to the third GPU frequency priority queue finally obtained.

[0065] The third scheduling scheme includes the number of GPUs and GPU frequency allocated to each deep learning training task, which ensures that the final generated scheduling scheme does not exceed the carbon emission budget, achieves carbon efficiency, and minimizes the average job completion time of deep learning training tasks.

[0066] The following fourth and fifth embodiments are used to illustrate a method for determining a scheduling scheme output by a final GPU scheduler (ie, a scheduling scheme ultimately executed by a GPU cluster) based on three scheduling modules and three different scheduling schemes generated.

[0067] Embodiment 4: In a possible implementation, the system further includes: The user demand determination module is used to select a scheduling scheme to be finally executed by the GPU cluster from the first scheduling scheme, the second scheduling scheme and the third scheduling scheme according to the scheduling target selected by the user.

[0068] Reference Figure 3 , Figure 3 A schematic diagram of a cluster scheduling process is shown, such as Figure 3 As shown in the figure, each deep learning developer submits a job (i.e., a deep learning training task), and the cluster manager selects a scheduling target. The scheduler selects the corresponding module from the first scheduling module, the second scheduling module, and the third scheduling module according to different scheduling targets to generate a scheduling plan. Finally, the GPU cluster executes the job according to the final output scheduling plan.

[0069] In this embodiment, the scheduling target input by the user can be received, so that according to the user's scheduling target (such as reducing the average job completion time, improving the fairness of the completion time, and increasing the number of jobs that meet the deadline requirements), the corresponding scheduling module is selected to generate a scheduling scheme. Exemplarily, if the user's scheduling target is to increase the number of jobs that meet the deadline requirements, the first scheduling scheme generated by the first scheduling module is selected as the scheduling scheme output by the final GPU scheduler, that is, the scheduling scheme finally executed by the GPU cluster. If the user's scheduling target is to improve the fairness of the completion time, the second scheduling scheme generated by the second scheduling module is selected as the scheduling scheme output by the final GPU scheduler, that is, the scheduling scheme finally executed by the GPU cluster.

[0070] Embodiment 5 In a possible implementation, the GPU scheduler further includes: a fourth scheduling module; The fourth scheduling module is used to generate a fourth scheduling scheme according to the following steps: Step S401, calculate the first marginal benefit, the second marginal benefit and the third marginal benefit of each deep learning training task. Specifically, the first marginal benefit, the second marginal benefit and the third marginal benefit can be calculated using the methods in the above-mentioned embodiments 1, 2 and 3.

[0071] Step S402, performing a preset weight ratio, and calculating a fourth marginal benefit according to the first marginal benefit, the second marginal benefit and the third marginal benefit.

[0072] Step S403: Generate a first priority queue according to the fourth marginal benefit of each deep learning training task. That is, the higher the fourth marginal benefit, the higher the priority of the task in the queue.

[0073] Step S404, allocate a GPU to the deep learning training task with the highest priority in the first priority queue. That is, for the task j with the highest priority in the queue, n GPUs have been allocated in step S401, and 1 GPU is added on this basis.

[0074] Step S405, repeat the above steps until a first termination condition is reached, wherein the first termination condition is: all GPUs in the GPU cluster are allocated, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget.

[0075] Step S406: Increase the GPU frequency allocated to the deep learning training task with the highest priority by one unit frequency.

[0076] Step S407, calculate the fourth marginal benefit of each deep learning training task and generate a second priority queue.

[0077] Step S408: Increase the GPU frequency allocated to the deep learning training task with the highest priority in the second priority queue by one unit frequency.

[0078] Step S409, recalculate the fourth marginal benefit and generate a new second priority queue until the second termination condition is reached, wherein the second termination condition is: the frequency of all GPUs in the GPU cluster reaches the maximum frequency value, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget.

[0079] Step S410: Generate a fourth scheduling scheme according to the finally obtained second priority queue.

[0080] In this embodiment, when the user's scheduling target is not received, the fourth scheduling module can be used to merge the first marginal benefit, the second marginal benefit and the third marginal benefit according to the pre-set weight to obtain the fourth marginal benefit, so as to determine the number of GPUs and GPU frequency allocated to each task according to the fourth marginal benefit, and obtain the fourth scheduling scheme, so as to use the fourth scheduling scheme as the scheduling scheme finally executed by the GPU cluster. It not only ensures that the final generated scheduling scheme will not exceed the carbon emission budget and achieve carbon efficiency, but also optimizes the GPU scheduling performance from three aspects (reducing the average job completion time, improving the fairness of the completion time, and increasing the number of jobs that meet the deadline requirements).

[0081] Embodiment 6 This sixth embodiment is used to illustrate a scheme for adjusting the scheduling scheme during execution according to the characteristic of real-time changes in carbon intensity during execution of the scheduling scheme.

[0082] In a possible implementation, the system further includes: The carbon intensity acquisition module is used to acquire and store real-time carbon intensity.

[0083] The carbon intensity prediction module is used to predict a first carbon intensity value at a time point reaching the first preset duration based on the carbon intensities at the most recent multiple moments when the total execution duration of the scheduling scheme output by the GPU scheduler exceeds the first preset duration.

[0084] The scheduling scheme evaluation module is used to predict whether the scheduling scheme output by the GPU scheduler exceeds the carbon emission budget after reaching a first preset time period according to the first carbon intensity value.

[0085] The GPU scheduler is further configured to regenerate a scheduling plan after reaching a first preset time duration when a prediction result indicates that the scheduling plan output by the GPU scheduler exceeds the carbon emission budget after reaching the first preset time duration.

[0086] In this embodiment, considering that when there are many deep learning tasks, the execution cycle of the scheduling plan is long, and there may be a situation where the carbon intensity changes significantly during the execution process. For example, the carbon intensity at the current moment t1 is 10, and the generated scheduling plan changes from 10 to 20 after being executed for a certain time (for example, one hour) and reaching the moment t2, resulting in the scheduling plan no longer being suitable for the current scenario, and there is a risk that the carbon emissions of the GPU cluster will exceed the carbon emission budget.

[0087] In order to solve the above problems, Figure 2 As shown, the embodiment of the present application proposes to train a carbon intensity prediction module for future carbon intensity prediction, that is, to predict the carbon intensity after the first preset time in advance. Considering that the daily changes in carbon intensity are relatively regular, a large amount of historical carbon intensity information recorded can be used in advance to train a prediction model (such as a time series model ARIMA) as a carbon intensity prediction module. The input of this module is a time series composed of carbon intensity values ​​at multiple moments closest to the current moment, and the output is the predicted carbon intensity value at the moment after the first preset time at the current moment. Therefore, the scheduling scheme evaluation module is used to analyze whether the scheduling scheme in execution exceeds the carbon emission budget after reaching the first preset time based on the predicted first carbon intensity value.

[0088] Specifically, predicting, according to the first carbon intensity value, whether the scheduling scheme output by the GPU scheduler exceeds the carbon emission budget after reaching a first preset time, includes: First, a cluster state manager is used (the cluster state manager can be trained by the scheme described in Example 8), and the cluster state is updated according to the scheduling scheme output by the GPU scheduler to obtain the candidate cluster state (the number of GPUs executing tasks, the GPU frequency) corresponding to the moment after the first preset time is reached.

[0089] Then, a performance model manager is used (the performance model manager can be obtained by training the solution steps S601 to S605 described in the seventh embodiment) to calculate the carbon emissions generated by the GPU cluster when the candidate cluster state is at the first carbon intensity value.

[0090] Finally, determine whether the carbon emissions exceed the carbon emission budget.

[0091] Therefore, by predicting whether the carbon emissions of the GPU cluster will exceed the carbon emission budget due to changes in carbon intensity during the execution of scheduling plan A, if it does not exceed the budget, scheduling plan A will continue to be executed; if it exceeds the carbon emission budget, scheduling plan B will be regenerated after reaching the first preset time based on the first carbon intensity value, so that after executing scheduling plan A for the first preset time, it will be converted to executing scheduling plan B to ensure that the GPU cluster can always control the carbon emissions generated by task execution within the carbon emission budget.

[0092] In a possible implementation manner, the carbon intensity prediction module is trained through the following steps: Step S501, obtaining a first training sample set, wherein each first training sample in the first training sample set is a time series composed of historical carbon intensity values ​​at multiple consecutive moments, and the label of each first training sample is the carbon intensity value after the first preset time interval.

[0093] Step S502: input the first training sample into the model to be trained to obtain a prediction result output by the model.

[0094] Step S503, using the mean square error between the prediction result and the label as the loss function, calculating a first loss function value.

[0095] Step S504: update the parameters of the model to be trained according to the first loss function value.

[0096] Step S505, repeat the above steps until a preset number of training times is reached or the loss function converges, and then the training is terminated to obtain a trained carbon intensity prediction module.

[0097] In this embodiment, the model is trained using the first training sample (a time series consisting of historical carbon intensity values ​​at multiple consecutive moments) to enable the model to learn the changing pattern of carbon intensity. The trained model is used as a carbon intensity prediction module to predict the carbon intensity value after a first preset period of time based on the changes in carbon intensity in the current period.

[0098] Embodiment 7 This embodiment 7 is used to illustrate the specific implementation of the performance model manager.

[0099] In a possible implementation, the system further includes: The performance model manager is used to predict the execution time and carbon emissions of a deep learning training task based on the number of GPUs configured for each deep learning training task, the GPU frequency, and the carbon intensity at the current moment.

[0100] The performance model manager is trained according to the following steps: Step S601, obtain a second training sample set, each second training sample in the second training sample set is the number of GPUs used for a deep learning training task, the GPU frequency, the number of task parameters, and the carbon intensity value, and the label of each second training sample is the time required to execute the deep learning training task and the carbon emissions generated.

[0101] Step S602: input the second training sample into the performance model manager to be trained to obtain a calculation result.

[0102] Step S603, calculating a loss function according to the calculation result and the label to obtain a second loss function value.

[0103] Step S604: update parameters of the performance model manager according to the second loss function value.

[0104] Step S605, repeat the above steps until the preset number of training times is reached or the loss function converges, and then the training is terminated to obtain a performance model manager that has completed the training.

[0105] In this embodiment, the model is trained using the second training sample (the number of GPUs used by a deep learning training task, the GPU frequency, the number of task parameters, and the carbon intensity value, and may also include relevant information of the task), and the trained model is used as a performance model manager to achieve, for each deep learning training task, predicting the completion time required for the task to be executed under a specific GPU configuration, as well as the carbon emissions generated when the task is completed under a specific carbon intensity.

[0106] Embodiment 8 This eighth embodiment is used to illustrate the specific implementation of the cluster state manager.

[0107] In a possible implementation, the system further includes: A cluster state manager, configured to update a cluster state according to a scheduling scheme output by the GPU scheduler; the cluster state includes: cluster state information at multiple moments in the process of executing the scheduling scheme; The cluster status manager is also used to compare the cluster status with the actual cluster status obtained through real-time monitoring during the execution of the scheduling plan by the GPU cluster, and to issue an abnormal warning when the difference between the two reaches a preset threshold, and / or to re-acquire the carbon intensity value at the current moment, and use the GPU scheduler to regenerate a new scheduling plan based on the carbon intensity value at the current moment and the actual cluster status.

[0108] In this embodiment, the cluster state manager predicts the cluster state that changes during the scheduling process based on the currently executed scheduling plan and the initial cluster state. Specifically, the cluster state manager can be trained according to the following steps: Step S701, obtaining a third training sample set, wherein each third training sample in the third training sample set is an initial cluster state and a scheduling scheme, and the label of each third training sample is cluster state information at multiple moments during the execution of the scheduling scheme (which may include a cluster state after reaching a first preset time length), and the cluster state information at least includes: GPU usage; Step S702: input the third training sample into the cluster state manager to be trained to obtain a predicted cluster state.

[0109] Step S703: Calculate a loss function according to the predicted cluster state and label to obtain a third loss function value.

[0110] Step S704: update parameters of the cluster state manager according to the third loss function value.

[0111] Step S705, repeat the above steps until the preset number of training times is reached or the loss function converges, and then the training is terminated to obtain a cluster state manager that has completed the training.

[0112] In this embodiment, the model is trained using the third training sample (initial cluster state and scheduling scheme), and the trained model is used as a cluster state manager to achieve the prediction of the cluster state after multiple moments (or after reaching the first preset time length) based on the initial cluster state and scheduling scheme at the current moment. Therefore, during the execution of the scheduling scheme, the cluster state manager can be used to update the cluster state, and the cluster state updated by the cluster state manager represents the state that the GPU cluster should reach when executing the scheduling scheme under normal conditions, and the cluster state updated by the cluster state manager is compared with the actual cluster state obtained by real-time monitoring. If the difference between the two is large (the difference reaches a preset threshold), it means that the current GPU cluster has an abnormality in the process of executing the scheduling scheme and fails to execute the task according to the ideal state, then an abnormality warning is automatically issued, or a new scheduling scheme is regenerated (the scheduling scheme can be regenerated in any of the ways proposed in the above embodiments one to seven).

[0113] The second aspect of the embodiment of the present application further proposes a carbon-efficient cluster scheduling method, which is applied to the carbon-efficient cluster scheduling system for deep learning loads described in the first aspect of the embodiment of the present application, referring to Figure 4 , Figure 4 A flowchart of the steps of a carbon efficient cluster scheduling method is shown, such as Figure 4 As shown, the method includes: Step S801, generating a scheduling scheme for the GPU cluster according to the current deep learning training task and cluster status information, wherein the scheduling scheme includes: the execution priority order of the deep learning training task, and the number of GPUs and GPU frequency used by each deep learning training task.

[0114] Step S802: Execute a deep learning training task according to a scheduling scheme generated by the GPU scheduler.

[0115] The generating of a scheduling scheme for a GPU cluster includes: Calculate the first marginal benefit of each deep learning training task, and generate a first scheduling plan according to the first marginal benefit; the first scheduling plan is a scheduling plan aimed at increasing the number of deep learning training tasks that meet the deadline requirements; the first marginal benefit is calculated based on the ratio between the reduction in the number of tasks that can be completed by the GPU cluster after adding m GPUs to the deep learning training task and the increase in carbon emissions after adding m GPUs; wherein m represents the minimum number of GPUs required to complete the deep learning training task within the deadline.

[0116] Calculate the second marginal benefit of each deep learning training task, and generate a second scheduling plan according to the second marginal benefit; the second scheduling plan is a scheduling plan aimed at improving the fairness of completion time; the second marginal benefit is calculated based on the ratio between the reduction in completion time fairness after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU.

[0117] Calculate the third marginal benefit of each deep learning training task, and generate a third scheduling plan according to the third marginal benefit; the third scheduling plan is a scheduling plan aimed at reducing the average job completion time of the GPU cluster; the third marginal benefit is calculated based on the ratio between the reduction in job completion time after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU.

[0118] In a possible implementation manner, the first marginal benefit is calculated according to the following formula: ; in, Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f At 0:00, the number of deep learning training tasks that meet the deadline requirements of the GPU cluster, Indicates that n+m GPUs are used in deep learning training task j and the GPU frequency is fAt 0, the number of deep learning training tasks that the GPU cluster meets the deadline requirements, m represents the minimum number of GPUs required to complete the deep learning training task j within the deadline based on the already configured n GPUs, and N0 represents the total number of deep learning training tasks; Indicates that n+m GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f 0, the carbon emissions generated, C 0 represents the average carbon emissions generated by all deep learning training tasks; The second marginal benefit is calculated according to the following formula: ; in, Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f The fairness of the completion time at 0 o'clock, Indicates that n+1 GPUs are used in deep learning training task j and the GPU frequency is f The fairness of the completion time of 0 hours, Indicates the completion time fairness of the GPU cluster; Indicates that n+1 GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f 0, the carbon emissions generated, C 0 represents the average carbon emissions generated by all deep learning training tasks; The third marginal benefit is calculated according to the following formula: ; in, Indicates that deep learning training task j is using n GPUs and the GPU frequency is f The duration of the job completion at 0 o'clock; Indicates that deep learning training task j is using n+1 GPUs and the GPU frequency is f The job completion time at 0:00; JCT0 represents the average job completion time of all deep learning training tasks; Indicates that n+1 GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are C 0 represents the average carbon emissions generated by all deep learning training tasks.

[0119] In a possible implementation, the method further includes: According to the scheduling target selected by the user, a scheduling scheme finally executed by the GPU cluster is selected from the first scheduling scheme, the second scheduling scheme, and the third scheduling scheme.

[0120] In a possible implementation, generating a scheduling scheme for a GPU cluster includes: Generate the fourth scheduling plan according to the following steps: Calculating the first marginal benefit, the second marginal benefit, and the third marginal benefit of each deep learning training task; A preset weight ratio is performed, and a fourth marginal benefit is calculated according to the first marginal benefit, the second marginal benefit and the third marginal benefit; Generate a first priority queue according to the fourth marginal benefit of each deep learning training task; Allocate a GPU to the deep learning training task with the highest priority in the first priority queue; Repeat the above steps until a first termination condition is reached, where the first termination condition is: all GPUs in the GPU cluster are allocated, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget; Increase the GPU frequency assigned to the highest priority deep learning training task by one unit frequency; Calculate the fourth marginal benefit of each deep learning training task and generate a second priority queue; Increase the GPU frequency allocated to the deep learning training task with the highest priority in the second priority queue by one unit frequency; Recalculating the fourth marginal benefit and generating a new second priority queue until a second termination condition is reached, wherein the second termination condition is: the frequencies of all GPUs in the GPU cluster reach a maximum frequency value, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget; A fourth scheduling scheme is generated according to the finally obtained second priority queue.

[0121] In a possible implementation, the method further includes: Obtain real-time carbon intensity and store it; Using the carbon intensity prediction module, when the total execution time of the scheduling plan exceeds the first preset time, predicting the first carbon intensity value at the time point when the first preset time is reached according to the carbon intensities at the most recent multiple moments; According to the first carbon intensity value, predict whether the scheduling scheme exceeds the carbon emission budget after reaching a first preset time; When the prediction result indicates that the scheduling plan exceeds the carbon emission budget when the first preset time is reached, a scheduling plan is regenerated after the first preset time is reached.

[0122] In a possible implementation manner, the carbon intensity prediction module is trained through the following steps: Obtain a first training sample set, wherein each first training sample in the first training sample set is a time series consisting of historical carbon intensity values ​​at a plurality of consecutive moments, and the label of each first training sample is a carbon intensity value after the first preset time interval; Inputting the first training sample into the model to be trained to obtain a prediction result output by the model; Using the mean square error between the prediction result and the label as the loss function, calculating a first loss function value; Update the parameters of the model to be trained according to the first loss function value; Repeat the above steps until the preset number of training times is reached or the loss function converges, then end the training and obtain a trained carbon intensity prediction module.

[0123] In a possible implementation, the method further includes: Using the performance model manager, the execution time and carbon emissions of each deep learning training task are predicted based on the number of GPUs configured for each deep learning training task, the GPU frequency, and the carbon intensity at the current moment. The performance model manager is trained according to the following steps: Obtain a second training sample set, where each second training sample in the second training sample set is the number of GPUs used by a deep learning training task, the GPU frequency, the number of task parameters, and the carbon intensity value, and the label of each second training sample is the time required to execute the deep learning training task and the amount of carbon emissions generated; Inputting the second training sample into the performance model manager to be trained to obtain a calculation result; Calculate the loss function according to the calculation result and the label to obtain a second loss function value; updating parameters of the performance model manager according to the second loss function value; Repeat the above steps until the preset number of training times is reached or the loss function converges, then end the training and obtain a performance model manager that has completed the training.

[0124] In a possible implementation, the method further includes: Using a cluster state manager, according to a scheduling scheme, the cluster state is updated; the cluster state includes: cluster state information at multiple moments in the process of executing the scheduling scheme; The cluster status manager is also used to compare the cluster status with the actual cluster status obtained through real-time monitoring during the execution of the scheduling plan by the GPU cluster, and to issue an abnormal warning when the difference between the two reaches a preset threshold, and / or to re-acquire the carbon intensity value at the current moment, and use the GPU scheduler to regenerate a new scheduling plan based on the carbon intensity value at the current moment and the actual cluster status.

[0125] The present application also provides an electronic device, referring to Figure 5 , Figure 5 Schematic diagram of an electronic device proposed in an embodiment of the present application. Figure 5 As shown, the electronic device 100 includes: a memory 110 and a processor 120. The memory 110 and the processor 120 are connected via a bus communication. A computer program is stored in the memory 110. The computer program can be run on the processor 120 to implement the steps in the carbon-efficient cluster scheduling method disclosed in the embodiment of the present application.

[0126] The embodiment of the present application also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps in the carbon-efficient cluster scheduling method disclosed in the embodiment of the present application are implemented.

[0127] The embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, the processor is enabled to implement the steps of the carbon-efficient cluster scheduling method disclosed in the embodiment of the present application.

[0128] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0129] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0132] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0133] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0134] The above is a detailed introduction to a carbon-efficient cluster scheduling system, method and product for deep learning loads provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A carbon-efficient cluster scheduling system for deep learning workloads, characterized in that: The system comprises: A GPU scheduler is used to generate a scheduling plan for the GPU cluster according to the current deep learning training task and cluster status information, wherein the scheduling plan includes: the execution priority order of the deep learning training task, and the number of GPUs and GPU frequency used by each deep learning training task; The GPU scheduler includes at least: a first scheduling module, a second scheduling module and a third scheduling module; The first scheduling module is used to calculate the first marginal benefit of each deep learning training task, and generate a first scheduling plan according to the first marginal benefit; the first scheduling plan is a scheduling plan aimed at increasing the number of deep learning training tasks that meet the deadline requirements; the first marginal benefit is calculated based on the ratio between the reduction in the number of tasks that can be completed by the GPU cluster after adding m GPUs to the deep learning training task and the increase in carbon emissions after adding m GPUs; wherein m represents the minimum number of GPUs required to complete the deep learning training task within the deadline; The second scheduling module is used to calculate the second marginal benefit of each deep learning training task, and generate a second scheduling plan according to the second marginal benefit; the second scheduling plan is a scheduling plan aimed at improving the fairness of completion time; the second marginal benefit is calculated based on the ratio between the reduction in the fairness of completion time after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU; The third scheduling module is used to calculate the third marginal benefit of each deep learning training task, and generate a third scheduling plan according to the third marginal benefit; the third scheduling plan is a scheduling plan aimed at reducing the average job completion time of the GPU cluster; the third marginal benefit is calculated based on the ratio between the reduction in job completion time after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU; The GPU cluster is used to execute deep learning training tasks according to the scheduling scheme generated by the GPU scheduler.

2. The carbon-efficient cluster scheduling system for deep learning loads according to claim 1 is characterized in that: The first marginal benefit is calculated according to the following formula: ； in, Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f At 0:00, the number of deep learning training tasks that meet the deadline requirements of the GPU cluster, Indicates that n+m GPUs are used in deep learning training task j and the GPU frequency is f At 0, the number of deep learning training tasks that the GPU cluster meets the deadline requirements, m represents the minimum number of GPUs required to complete the deep learning training task j within the deadline based on the already configured n GPUs, and N0 represents the total number of deep learning training tasks; Indicates that n+m GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f 0, the carbon emissions generated, C 0 represents the average carbon emissions generated by all deep learning training tasks; The second marginal benefit is calculated according to the following formula: ； in, Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f The fairness of the completion time at 0 o'clock, Indicates that n+1 GPUs are used in deep learning training task j and the GPU frequency is f The fairness of the completion time of 0 hours, Indicates the completion time fairness of the GPU cluster; Indicates that n+1 GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f 0, the carbon emissions generated, C 0 represents the average carbon emissions generated by all deep learning training tasks; The third marginal benefit is calculated according to the following formula: ； in, Indicates that deep learning training task j is using n GPUs and the GPU frequency is f The duration of the job completion at 0 o'clock; Indicates that deep learning training task j is using n+1 GPUs and the GPU frequency is f The job completion time at 0:00; JCT0 represents the average job completion time of all deep learning training tasks; Indicates that n+1 GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are Indicates that n GPUs are used in deep learning training task j and the GPU frequency is f At 0, the carbon emissions generated are C 0 represents the average carbon emissions generated by all deep learning training tasks.

3. The carbon-efficient cluster scheduling system for deep learning loads according to claim 1 is characterized in that: The system further comprises: The user demand determination module is used to select a scheduling scheme to be finally executed by the GPU cluster from the first scheduling scheme, the second scheduling scheme and the third scheduling scheme according to the scheduling target selected by the user.

4. The carbon-efficient cluster scheduling system for deep learning loads according to claim 1 is characterized in that: The GPU scheduler further includes: a fourth scheduling module; The fourth scheduling module is used to generate a fourth scheduling scheme according to the following steps: Calculating the first marginal benefit, the second marginal benefit, and the third marginal benefit of each deep learning training task; A preset weight ratio is performed, and a fourth marginal benefit is calculated according to the first marginal benefit, the second marginal benefit and the third marginal benefit; Generate a first priority queue according to the fourth marginal benefit of each deep learning training task; Allocate a GPU to the deep learning training task with the highest priority in the first priority queue; Repeat the above steps until a first termination condition is reached, where the first termination condition is: all GPUs in the GPU cluster are allocated, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget; Increase the GPU frequency assigned to the highest priority deep learning training task by one unit frequency; Calculate the fourth marginal benefit of each deep learning training task and generate a second priority queue; Increase the GPU frequency allocated to the deep learning training task with the highest priority in the second priority queue by one unit frequency; Recalculating the fourth marginal benefit and generating a new second priority queue until a second termination condition is reached, wherein the second termination condition is: the frequencies of all GPUs in the GPU cluster reach a maximum frequency value, and / or the total carbon emissions of the GPU cluster exceed the carbon emission budget; A fourth scheduling scheme is generated according to the finally obtained second priority queue.

5. The carbon-efficient cluster scheduling system for deep learning loads according to claim 1 is characterized in that: The system further comprises: A carbon intensity acquisition module is used to acquire and store real-time carbon intensity; A carbon intensity prediction module, configured to predict, when the total execution time of the scheduling scheme output by the GPU scheduler exceeds a first preset time, a first carbon intensity value at a time point when the first preset time is reached based on the carbon intensities at the most recent multiple moments; a scheduling scheme evaluation module, configured to predict, based on the first carbon intensity value, whether the scheduling scheme output by the GPU scheduler exceeds the carbon emission budget after reaching a first preset time; The GPU scheduler is further configured to regenerate a scheduling plan after reaching a first preset time duration when a prediction result indicates that the scheduling plan output by the GPU scheduler exceeds the carbon emission budget after reaching the first preset time duration.

6. The carbon-efficient cluster scheduling system for deep learning loads according to claim 5 is characterized in that: The carbon intensity prediction module is trained through the following steps: Obtain a first training sample set, wherein each first training sample in the first training sample set is a time series consisting of historical carbon intensity values ​​at a plurality of consecutive moments, and the label of each first training sample is a carbon intensity value after the first preset time interval; Inputting the first training sample into the model to be trained to obtain a prediction result output by the model; Using the mean square error between the prediction result and the label as the loss function, calculating a first loss function value; Update the parameters of the model to be trained according to the first loss function value; Repeat the above steps until the preset number of training times is reached or the loss function converges, then end the training and obtain a trained carbon intensity prediction module.

7. The carbon-efficient cluster scheduling system for deep learning loads according to claim 1, characterized in that: The system further comprises: Performance model manager, used to predict the execution time and carbon emissions of each deep learning training task based on the number of GPUs configured for each deep learning training task, GPU frequency and the carbon intensity at the current moment; The performance model manager is trained according to the following steps: Obtain a second training sample set, where each second training sample in the second training sample set is the number of GPUs used by a deep learning training task, the GPU frequency, the number of task parameters, and the carbon intensity value, and the label of each second training sample is the time required to execute the deep learning training task and the amount of carbon emissions generated; Inputting the second training sample into the performance model manager to be trained to obtain a calculation result; Calculate the loss function according to the calculation result and the label to obtain a second loss function value; updating parameters of the performance model manager according to the second loss function value; Repeat the above steps until the preset number of training times is reached or the loss function converges, then end the training and obtain a performance model manager that has completed the training.

8. The carbon-efficient cluster scheduling system for deep learning loads according to claim 1, characterized in that: The system further comprises: A cluster state manager, configured to update a cluster state according to a scheduling scheme output by the GPU scheduler; the cluster state includes: cluster state information at multiple moments in the process of executing the scheduling scheme; The cluster status manager is also used to compare the cluster status with the actual cluster status obtained through real-time monitoring during the execution of the scheduling plan by the GPU cluster, and to issue an abnormal warning when the difference between the two reaches a preset threshold, and / or to re-acquire the carbon intensity value at the current moment, and use the GPU scheduler to regenerate a new scheduling plan based on the carbon intensity value at the current moment and the actual cluster status.

9. A carbon efficient cluster scheduling method, characterized in that: The carbon-efficient cluster scheduling system for deep learning loads according to any one of claims 1 to 8, the method comprising: Generate a scheduling plan for the GPU cluster according to the current deep learning training task and cluster status information, wherein the scheduling plan includes: the execution priority order of the deep learning training tasks, and the number of GPUs and GPU frequency used by each deep learning training task; Execute deep learning training tasks according to the scheduling scheme generated by the GPU scheduler; The generating of a scheduling scheme for a GPU cluster includes: Calculate the first marginal benefit of each deep learning training task, and generate a first scheduling plan according to the first marginal benefit; the first scheduling plan is a scheduling plan aimed at increasing the number of deep learning training tasks that meet the deadline requirements; the first marginal benefit is calculated based on the ratio between the reduction in the number of tasks that can be completed by the GPU cluster after adding m GPUs to the deep learning training task and the increase in carbon emissions after adding m GPUs; wherein m represents the minimum number of GPUs required to complete the deep learning training task within the deadline; Calculate the second marginal benefit of each deep learning training task, and generate a second scheduling plan according to the second marginal benefit; the second scheduling plan is a scheduling plan aimed at improving the fairness of completion time; the second marginal benefit is calculated based on the ratio between the reduction in the fairness of completion time after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU; Calculate the third marginal benefit of each deep learning training task, and generate a third scheduling plan according to the third marginal benefit; the third scheduling plan is a scheduling plan aimed at reducing the average job completion time of the GPU cluster; the third marginal benefit is calculated based on the ratio between the reduction in job completion time after adding one GPU to the deep learning training task and the increase in carbon emissions after adding one GPU.

10. An electronic device, characterized in that: It comprises a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the carbon efficient cluster scheduling method according to claim 9.

Images