A workflow scheduling method and device based on GPU time-multiplexing and a storage medium

Abstract

The application discloses a workflow scheduling method and device based on GPU time-sharing multiplexing and a storage medium, and aims to solve the technical problem that GPU resources cannot be dynamically allocated in the prior art, resulting in performance degradation. The method comprises the following steps: dividing all GPU memory resources of a plurality of nodes in a cluster into a plurality of time segments; creating a workflow, determining the working time required to complete the workflow, and calculating the first GPU memory resource required to complete the workflow; allocating the first GPU memory resource to the workflow, and marking the time segment corresponding to the allocated first GPU memory resource, so as to prompt the system that the first GPU memory resource has been occupied. The application realizes that resources are not wasted due to periodic calling of idle time to occupy GPU and GPU memory resources, and can effectively control the scheduling of GPU and GPU memory by setting the time segments of GPU time sharing, so that the business can normally and efficiently run.

Description

Technical Field

[0001] This application relates to the field of GPU time-sharing technology, and in particular to a workflow scheduling method, device and storage medium based on GPU time-sharing. Background Technology

[0002] With the development of artificial intelligence, computer application technology has also continuously evolved, becoming an indispensable part of current social development. Currently, leading distributed architecture solutions deployed in multi-node clusters are widely used in deployment environments. The operation of numerous training and inference scripts requires sufficient resources, but when resources are limited, reasonable optimization is necessary.

[0003] Currently, it is possible to reuse and isolate GPUs and GPU memory, but technically it cannot flexibly adapt to application scenarios. When workflows in a business run according to a cycle, the GPUs and GPU memory allocated during idle periods cannot be effectively utilized.

[0004] Therefore, there is an urgent need for a workflow scheduling method to solve the problem that GPU resources cannot be dynamically allocated in existing technologies, which leads to a decrease in computing performance. Summary of the Invention

[0005] This application provides a workflow scheduling method, device, and storage medium based on GPU time-sharing multiplexing to solve the technical problem that GPU resources cannot be dynamically allocated in the prior art, resulting in a decrease in computing performance.

[0006] On one hand, embodiments of this application provide a workflow scheduling method based on GPU time-sharing multiplexing. The method includes: dividing all GPU memory resources of several nodes in a cluster into several time segments; creating a workflow, determining the working time required to complete the workflow, and calculating the first GPU memory resources required to complete the workflow; allocating the first GPU memory resources to the workflow, and assigning a value to the time segment corresponding to the allocated first GPU memory resources to indicate to the system that the first GPU memory resources have been occupied.

[0007] In one implementation of this application, before dividing the total GPU memory resources of several nodes in the cluster into several time segments, the method further includes: real-time monitoring of all GPUs and GPU memory resources of several nodes in the cluster, and aggregating the GPU memory resources into a resource pool. Since the GPU memory resources carried or externally mounted by each server are different, aggregating them into a resource pool allows for reasonable overall planning and facilitates resource reallocation.

[0008] In one implementation of this application, the method further includes: determining the utilization rate of the first GPU memory resource; and dynamically reallocating the GPU memory resource based on a preset utilization rate value.

[0009] In one implementation of this application, the dynamic allocation of GPU memory resources based on a preset utilization rate value specifically includes: when the utilization rate of the first GPU memory resource is lower than a first preset utilization rate value, allocating a second GPU memory resource from the first GPU memory resource and releasing the second GPU memory resource back to the resource pool. By setting automatic resource release, resource waste can be avoided, thereby achieving the purpose of dynamically allocating GPU memory resources.

[0010] In one implementation of this application, the method further includes: when the utilization rate of the first GPU memory resource is higher than a second preset utilization rate value, retrieving a third GPU memory resource from the GPU resources in the resource pool, and merging the third GPU memory resource into the first GPU memory resource. By setting to monitor the utilization rate of existing GPU resources and allocating additional GPU resources when the utilization rate is high, dynamic allocation of GPU memory resources can be achieved, ensuring the smooth completion of tasks.

[0011] In one implementation of this application, the method further includes: monitoring whether the third GPU memory resource has been successfully allocated; if the third GPU memory resource has not been successfully allocated, analyzing the cause and generating platform alarm information.

[0012] In one implementation of this application, the method further includes: monitoring whether a third GPU resource has been successfully retrieved from the resource pool and merging the third GPU memory resource into the first GPU memory resource; if the third GPU resource has not been successfully retrieved from the resource pool and the third GPU memory resource has not been merged into the first GPU memory resource, then analyzing the cause and generating platform alarm information.

[0013] In one implementation of this application, the method further includes: receiving a newly created workflow and determining the pre-running cycle required to complete the newly created workflow; if the pre-running cycle is consistent with the time segment of the first GPU memory resource, then hiding the first GPU memory resource for the newly created workflow. By setting a time segment assignment flag, it is possible to prevent other newly created workflows from occupying the current first GPU memory resource, causing resource overlap and resulting in workflow failure.

[0014] This application embodiment also provides a workflow scheduling device based on GPU time-sharing multiplexing. The device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: divide all GPU memory resources of several nodes in a cluster into several time segments; create a workflow, determine the working time required to complete the workflow, and calculate the first GPU memory resources required to complete the workflow; allocate the first GPU memory resources to the workflow, and assign a value to the time segment corresponding to the allocated first GPU memory resources to indicate to the system that the first GPU memory resources have been occupied.

[0015] This application embodiment also provides a non-volatile computer storage medium for workflow scheduling based on GPU time-sharing multiplexing, storing computer-executable instructions. The computer-executable instructions are configured to: divide all GPU memory resources of several nodes in the cluster into several time segments; create a workflow, determine the working time required to complete the workflow, and calculate the first GPU memory resources required to complete the workflow; allocate the first GPU memory resources to the workflow, and assign a value to the time segment corresponding to the allocated first GPU memory resources to indicate to the system that the first GPU memory resources have been occupied.

[0016] This application provides a workflow scheduling method, device, and storage medium based on GPU time-sharing multiplexing. By dynamically adjusting GPU memory resources, the success rate of training script execution can be improved, avoiding failures due to insufficient GPU memory allocation caused by fixed GPU memory quotas during GPU memory multiplexing. By using GPU time-sharing multiplexing time segments, the effective use of GPU and GPU memory resources is controlled efficiently. Resources are not wasted due to periodic calls occupying GPU and GPU memory resources during idle periods, while during busy periods, GPU and GPU memory scheduling can be effectively controlled by setting GPU time-sharing time segments, allowing the business to run normally and efficiently, ultimately achieving better computational results. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 A flowchart of a workflow scheduling method based on GPU time-sharing multiplexing provided in this application embodiment;

[0019] Figure 2This is a schematic diagram of a workflow scheduling device based on GPU time-sharing multiplexing, provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] With the development of artificial intelligence, computer application technology has also continuously evolved, becoming an indispensable part of current social development. Currently, leading distributed architecture solutions deployed in multi-node clusters are widely used in deployment environments. The operation of numerous training and inference scripts requires sufficient resources, but when resources are limited, reasonable optimization is necessary.

[0022] Currently, while it's possible to reuse and isolate GPU and GPU memory allocation, this technology lacks flexibility to adapt to various application scenarios. When workflows run periodically, the allocated GPU and GPU memory during idle periods cannot be effectively utilized. In other words, it's not currently possible to dynamically assign GPU memory resources based on the actual workflow execution. During a workflow task, GPU memory resources may have idle periods, but because this cannot be dynamically adjusted, these resources are wasted. Therefore, a workflow scheduling method is urgently needed to address the problem of degraded computing performance caused by the inability to dynamically allocate GPU resources in existing technologies.

[0023] This application provides a method to solve a technical problem.

[0024] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0025] Figure 1 A flowchart illustrating a method provided in an embodiment of this application. Figure 1 As shown, the main steps include:

[0026] Step 101: Divide all GPU resources of several nodes in the cluster into several time segments.

[0027] In this embodiment, the same cluster contains multiple nodes. These nodes contain multiple servers, and each server contains a large number of GPU resources. These GPU resources are applied to a variety of tasks. The running time and resource requirements of each task are different, so it is necessary to divide the GPU resources into several different time segments to coordinate and utilize all GPU resources in a reasonable manner.

[0028] It should be noted that when dividing all GPU memory resources into time segments, the division can be based on the size of the GPU memory resources. For example, some complex workflow business processes require a large amount of computing power, so some large-capacity GPU memory resources can be allocated, and idle time segments can be allocated to the large-capacity GPU memory resources, such as certain time periods between 21:00 and 05:00.

[0029] For certain workflows with simpler functional structures, large amounts of GPU memory resources are not required. Therefore, it is still necessary to allocate some small amounts of GPU memory resources and divide these small amounts of GPU memory resources into short time segments, such as 9:10-9:35, a 25-minute time segment, to process certain business flows with simpler structures and functions. This ensures that GPU memory resources can be rationally allocated and utilized, reducing the occurrence of GPU memory waste.

[0030] In this embodiment, before dividing GPU resources into multiple time segments, it is necessary to aggregate the GPU resources into a resource pool. Since each server's built-in or external GPU cards and video memory resources are different, they need to be placed in the resource pool for overall planning and redistribution.

[0031] Step 102: Determine the working time required to complete the workflow and calculate the first GPU memory resources required to complete the workflow.

[0032] In this embodiment, when a periodic task such as a workflow is created, the system automatically calculates the time required to complete the workflow task within that period, then presets a pre-running period and calculates the amount of GPU resources required to run the periodic workflow task. For example, if the time required to complete a certain periodic workflow task is calculated to be 2 to 3 hours, then the preset pre-running period can be 3 hours.

[0033] Furthermore, if the GPU resources required to complete this periodic workflow task are calculated to be 5GB, then 5GB of GPU resources need to be scraped from the entire GPU resource pool. Therefore, the scraped 5GB of GPU resources is the size of the first GPU resource described in this article.

[0034] Step 103: Allocate the first GPU memory resources to the workflow and assign a value to the time segment corresponding to the allocated first GPU memory resources.

[0035] In this embodiment, a periodic workflow task is bound to the allocated first GPU resource, and a value is assigned to the first GPU resource to indicate to the system that the first GPU resource has been used and a task is running.

[0036] By setting the assignment flag for the first GPU memory resource, it can be ensured that the currently running workflow is not affected by other newly created workflows, which would cause multiple workflows to be superimposed on the first GPU resource, resulting in the failure of the currently running workflow.

[0037] It should be noted that the specific markings could be: First GPU resource size: 5GB, running time period: 14:00-17:00. This way, during the same time period (14:00-17:00), other periodic workflow tasks will automatically avoid creating this first GPU resource. The first GPU resource will then display as occupied and unable to be created.

[0038] In this embodiment of the application, when a workflow task is created and successfully scheduled within the period, the background monitoring unit will perform real-time summarization and monitoring, and detect the utilization of the first GPU resources.

[0039] In this embodiment, for example, the functional structure of the periodic workflow task is relatively simple, or the GPU graphics card has a high model and performance. It can be completed quickly. After completion, the utilization rate of the first GPU resource will drop rapidly. When it drops to a preset second utilization rate value, such as 10%, then the resource utilization of the first GPU will remain idle.

[0040] In other words, completing this periodic workflow task does not require that many GPU resources to be occupied, but a certain number of GPU resources are still allocated to this periodic workflow task, which will result in a waste of GPU resources.

[0041] Furthermore, when the system detects that the utilization rate of the first GPU resource is low, it will automatically generate an alarm message and send it to the user. At the same time, the alarm message will show the reason analyzed by the system as being due to the relatively simple work structure or the GPU graphics card model and performance being better, so that the current workflow can be completed in less time.

[0042] After receiving the alarm message, the user can manually adjust the size and time segment of the first GPU graphics card resource, for example, change the preset 14:00-17:00 to 14:00-15:00, and adjust the size of the first GPU graphics card resource to 2GB.

[0043] It's important to note that adjusting the size of the first GPU memory resource to 2GB can be viewed as follows: The first GPU resource is re-partitioned, allocating a second GPU memory resource of 3GB from it. This second GPU memory resource is then returned to the resource pool. Since this portion of GPU memory is unused, it can be utilized by other periodic workflows during their creation. By releasing excess GPU memory resources back into the resource pool, dynamic allocation of GPU memory resources is achieved.

[0044] In this embodiment, the background system also monitors the first GPU memory resources in real time to determine whether excess second GPU memory resources are released into the resource pool to avoid wasting GPU memory resources. If the second GPU memory resources are not successfully released, the background monitoring system analyzes the cause and generates an alarm message to notify the user. After seeing the alarm message, the user can manually release GPU resources into the resource pool.

[0045] In this embodiment, the process is similar to the one described above when the first GPU resource utilization is high. For example, if the pre-running cycle of this periodic workflow task is one hour, the system will automatically estimate that the task can be completed within one hour.

[0046] However, in reality, the task flow may not be completed even when it is almost an hour in. At this point, the system will automatically calculate the utilization rate of the first GPU resource and compare it with the preset utilization rate value. For example, if the preset utilization rate value is 95%, and the utilization rate of the first GPU resource has reached 100%, the system will analyze the reason for this situation, such as the temperature of the server room being too high or other reasons, and then generate an alarm message to notify the user.

[0047] Furthermore, users can manually adjust GPU resources based on the messages displayed in the alarm. For example, they can extend the time period by another hour or set the size of the first GPU resource to 10GB.

[0048] It's important to note that setting the size of the first GPU memory resource to 10GB can be viewed as follows: the background system automatically retrieves a 5GB third GPU memory resource from the resource pool of GPUs that haven't yet been allocated workflow tasks. This third GPU memory resource is then merged into the first GPU memory resource, resulting in a new 10GB first GPU resource. This large GPU memory resource can then support the execution of faster, periodic workflow tasks, ensuring the achievement of computational results.

[0049] In this embodiment, the background system also monitors in real time whether a 5GB third GPU memory resource has been successfully retrieved from the resource pool. If the third GPU memory resource is not successfully retrieved, the background monitoring system analyzes the reason for the situation and generates an alarm message to notify the user. When the user confirms that the third GPU memory resource has not been successfully retrieved, or that the retrieved third GPU memory resource has not been merged into the first GPU memory resource, the user can manually select a third GPU memory resource of appropriate size and time segment from the GPU memory resources in the resource pool.

[0050] This application can efficiently utilize GPU and GPU memory resources under fixed GPU memory resources. When business processes such as current workflows need to run periodically, GPU memory resources and memory quotas are used in time periods according to the actual running situation, which greatly reduces the quota occupation and enables GPU memory resources to be used well.

[0051] Furthermore, by using time-sharing multiplexing of GPU time segments, the effective use of GPU and GPU memory resources can be controlled efficiently, preventing the waste of GPU memory resources caused by periodic calls occupying GPU memory during idle periods. During busy periods, the allocation of GPU and GPU memory can be effectively controlled by setting sufficient time-sharing periods, allowing the business to run normally and efficiently, ultimately achieving the best computing results.

[0052] The above are method embodiments provided in this application. Based on the same inventive concept, this application also provides a schematic diagram of a workflow scheduling device based on GPU time-sharing multiplexing. Figure 2As shown, the device mainly includes: at least one processor 201; and a memory 202 communicatively connected to the at least one processor 201; wherein the memory 202 stores instructions executable by the at least one processor 201, the instructions being executed by the at least one processor 201 to enable the at least one processor 201 to: divide all GPU memory resources of several nodes in the cluster into several time segments; create a workflow, determine the working time required to complete the workflow, and calculate the first GPU memory resources required to complete the workflow; allocate the first GPU memory resources to the workflow, and assign a value to the time segment corresponding to the allocated first GPU memory resources to indicate to the system that the first GPU memory resources have been occupied.

[0053] Based on the same inventive concept, this application also provides a non-volatile computer storage medium for workflow scheduling based on GPU time-sharing multiplexing, storing computer-executable instructions. The computer-executable instructions are configured to: divide all GPU memory resources of several nodes in the cluster into several time segments; create a workflow, determine the working time required to complete the workflow, and calculate the first GPU memory resources required to complete the workflow; allocate the first GPU memory resources to the workflow, and assign a value to the time segment corresponding to the allocated first GPU memory resources to indicate to the system that the first GPU memory resources have been occupied.

[0054] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0055] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0056] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0057] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0058] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0059] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0060] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0061] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0062] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A workflow scheduling method based on GPU time-sharing multiplexing, characterized in that, The method includes: Divide all GPU memory resources of several nodes in the cluster into several time segments according to a preset time granularity. Create a workflow, determine the estimated runtime required to complete the workflow, and calculate the capacity of the first GPU memory resources required to complete the workflow; In response to the creation of the workflow, a target time segment matching the estimated runtime is allocated to the workflow from the plurality of time segments, and the first GPU memory resource is allocated from the total GPU memory resources; The allocated first GPU memory resources are bound to the target time segment, and an occupancy flag is set for the target time segment. The occupancy flag is used to indicate to the scheduling system that the first GPU memory resources have been occupied in the target time segment, so as to prevent other workflows from attempting to allocate the first GPU memory resources in the target time segment. The method further includes: Determine the utilization rate of the first GPU memory resources; Based on a preset utilization rate value, the GPU memory resources are dynamically reallocated.

2. The workflow scheduling method based on GPU time-sharing multiplexing according to claim 1, characterized in that, Before dividing all GPU memory resources of several nodes in the cluster into several time segments, the method further includes: Real-time monitoring of all GPUs and GPU memory resources on several nodes within the cluster, and aggregation of the GPU memory resources into a resource pool.

3. The workflow scheduling method based on GPU time-sharing multiplexing according to claim 1, characterized in that, The dynamic allocation of GPU memory resources based on a preset utilization rate value specifically includes: When the utilization rate of the first GPU memory resource is lower than the first preset utilization rate value, the second GPU memory resource is allocated from the first GPU memory resource. Release the second GPU memory resources back into the resource pool.

4. The workflow scheduling method based on GPU time-sharing multiplexing according to claim 3, characterized in that, The method further includes: Monitor whether the second GPU memory resources have been successfully allocated; If the second GPU memory resources are not successfully allocated, the cause will be analyzed and a platform alarm message will be generated.

5. A workflow scheduling method based on GPU time-sharing multiplexing according to claim 1, characterized in that, The method further includes: When the utilization rate of the first GPU memory resource is higher than the second preset utilization rate value, the third GPU memory resource is retrieved from the GPU resources in the resource pool and merged into the first GPU memory resource.

6. A workflow scheduling method based on GPU time-sharing multiplexing according to claim 5, characterized in that, The method further includes: Monitor whether the third GPU memory resources have been successfully captured from the resource pool; If the third GPU resource is not successfully retrieved from the resource pool, or if the third GPU memory resource is not merged into the first GPU memory resource, the cause will be analyzed and a platform alarm message will be generated.

7. A workflow scheduling method based on GPU time-sharing multiplexing according to claim 1 or 2, characterized in that, The method further includes: Receive the newly created workflow and determine the pre-run cycle required to complete the newly created workflow; If the pre-run cycle is consistent with the time segment of the first GPU memory resource, then the first GPU memory resource is hidden in the newly created workflow.

8. A workflow scheduling device based on GPU time-sharing multiplexing, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

9. A non-volatile computer storage medium for workflow scheduling based on GPU time-sharing multiplexing, storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed by a processor, they implement the method of any one of claims 1 to 7.

Images