Graphical processor resource scheduling method and system, computer device and storage medium

Abstract

The application relates to a graphics processor resource scheduling method and system, computer equipment and a storage medium. The method comprises the following steps: acquiring a plurality of preparation nodes and the node idle bandwidth of each preparation node within a preset time, and acquiring the group average idle bandwidth of each non-uniform memory access group within the preparation node within a preset time; determining a target task and a task type thereof, calculating the node score of each preparation node under different task types, and determining a target node according to the task type and the node score; and selecting a target graphics processor from the target node according to the group average idle bandwidth, so as to execute the target task through the target graphics processor, thereby improving the graphics processor resource utilization rate.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence training, and in particular to a method, system, computer device, and storage medium for scheduling graphics processor resources. Background Technology

[0002] With the development of AI technology, the data in AI training scenarios is constantly increasing, and the training models are becoming larger. Currently, a single network card (NIC) cannot meet these growing demands. Distributed training can solve this problem by expanding the number of accelerator cards. Deep learning frameworks (TensorFlow, PyTorch, Horovod, etc.) all support distributed training on multiple machines and with multiple NICs, and have done a lot of work on optimizing inter-card communication and inter-node communication. For example, the NCCL communication framework, the Ring-Allreduce algorithm, gradient fusion, and gradient compression are all designed to maximize the use of network bandwidth and reduce the overlap of computation and communication time, thereby improving computational efficiency and accelerating model convergence.

[0003] For AI cluster management platforms, it is necessary to manage and utilize computing power, high-performance network cards, and data storage for AI training. The core task is to optimize the allocation of accelerator card resources, thereby maximizing the utilization of high-performance networks, datasets, and accelerator cards to improve accelerator card utilization. Containerized AI clusters built on Kubernetes often have multiple distributed tasks with several containers assigned to the same node. When multiple containers perform gradient reduction, they consume significant high-performance network bandwidth, potentially making hardware network bandwidth a bottleneck. Some studies indicate that when the hardware network bandwidth available for a single task is less than 25Gb, performance is significantly impacted, reducing training task performance. However, at this point, there may be GPU (Graphics Processing Unit) nodes within the cluster with idle network bandwidth or low bandwidth utilization that are not being scheduled for use, resulting in low GPU resource utilization. Summary of the Invention

[0004] Based on this, this application provides a graphics processor resource scheduling method, system, computer device, and storage medium to improve the utilization rate of graphics processor resources.

[0005] On the one hand, a method for scheduling graphics processor resources is provided, the method comprising:

[0006] Obtain multiple preparatory nodes and the node idle bandwidth of each preparatory node within a preset time period, and obtain the group average idle bandwidth of each non-consistent memory access group within the preparatory nodes within a preset time period.

[0007] Determine the target task and its task type, calculate the node score of each preparatory node under different task types, and determine the target node based on the task type and node score;

[0008] A target graphics processor is selected from the target nodes based on the average idle bandwidth of the group, so as to execute the target task through the target graphics processor.

[0009] In one embodiment, obtaining the plurality of reserve nodes and the node idle bandwidth of each reserve node within a preset time includes:

[0010] Determine resource requirements and select multiple reserve nodes that meet the resource requirements based on a preset scheduling mechanism;

[0011] The idle bandwidth of each of the prepared nodes within a preset time period is obtained based on the node dimension.

[0012] In one embodiment, the target task's task type includes single-machine task type and multi-machine task type. Determining the target task and its task type, calculating the node score of each candidate node under different task types, and determining the target node based on the task type and node score includes:

[0013] Determine the target task and its task type;

[0014] In response to the fact that the target task is a single-machine task, the node score of each candidate node is calculated based on the principle of minimizing the idle bandwidth of the preset node, and the candidate node with the lowest node score is determined as the target node.

[0015] In response to the fact that the target task is a multi-machine task, the node score of each candidate node is calculated based on the principle of maximizing the idle bandwidth of the preset node, and the candidate node with the highest node score is determined as the target node.

[0016] In one embodiment, the single-machine task type includes a single-machine single-card task type and a single-machine multi-card task type. The step of selecting a target graphics processor from the target nodes based on the group's average idle bandwidth, and executing the target task through the target graphics processor, includes:

[0017] In response to the fact that the task type of the target task is a single-machine single-card task type, the target group with the lowest average idle bandwidth is determined from the target nodes;

[0018] Select any graphics processor from the target group as the target graphics processor to perform the target task through the target graphics processor.

[0019] In one embodiment, the step of selecting graphics processor resources from the target nodes based on the average idle bandwidth of the group, and executing the target task using the graphics processor resources, further includes:

[0020] In response to the fact that the task type of the target task is a single-machine multi-card task type, the target group with the lowest average idle bandwidth is determined from the target nodes;

[0021] Select at least two graphics processors from the target group as target graphics processors to perform the target task through the target graphics processors;

[0022] In response to the fact that the graphics processor resources in the target group cannot meet the target task, a sub-target group with the second lowest average idle bandwidth is determined from the target nodes, and a corresponding number of graphics processors are selected from the sub-target group as target graphics processors to execute the target task through the target graphics processors.

[0023] In one embodiment, the multi-machine task type includes a multi-machine single-card task type and a multi-machine multi-card task type. The step of selecting a target graphics processor from the target nodes based on the average idle bandwidth of the group, and executing the target task through the target graphics processor, further includes:

[0024] In response to the fact that the task type of the target task is a multi-machine single-card task type, the target group with the highest average idle bandwidth is determined from the target nodes;

[0025] Select any graphics processor from the target group as the target graphics processor to perform the target task through the target graphics processor.

[0026] In one embodiment, the step of selecting a target graphics processor from the target nodes based on the average idle bandwidth of the group, so as to execute the target task through the target graphics processor, further includes:

[0027] In response to the fact that the task type of the target task is a multi-machine multi-card task type, the target group with the highest average idle bandwidth is determined from the target nodes;

[0028] Select at least two graphics processors from the target group as target graphics processors to perform the target task through the target graphics processors;

[0029] In response to the fact that the graphics processor resources in the target group cannot meet the target task, a sub-target group with the second highest average idle bandwidth in the target group is determined from the target nodes, and a corresponding number of graphics processors are selected from the sub-target group as target graphics processors to execute the target task through the target graphics processors.

[0030] On the other hand, a graphics processor resource scheduling system is provided. The system includes a resource scheduler and multiple nodes. Each node includes a monitoring component and multiple non-consistent memory access groups. Each non-consistent memory access group is communicatively connected to the monitoring component, and the resource scheduler is communicatively connected to the monitoring component. Each non-consistent memory access group includes multiple network interface cards (NICs) and a graphics processor. The NICs are communicatively connected to the graphics processors.

[0031] In another aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0032] Obtain multiple preparatory nodes and the node idle bandwidth of each preparatory node within a preset time period, and obtain the group average idle bandwidth of each non-consistent memory access group within the preparatory nodes within a preset time period.

[0033] Determine the target task and its task type, calculate the node score of each preparatory node under different task types, and determine the target node based on the task type and node score;

[0034] A target graphics processor is selected from the target nodes based on the average idle bandwidth of the group, so as to execute the target task through the target graphics processor.

[0035] In another aspect, a computer-readable storage medium is provided, the computer-readable storage medium storing a program that, when executed by a processor, causes the processor to perform the following steps:

[0036] Obtain multiple preparatory nodes and the node idle bandwidth of each preparatory node within a preset time period, and obtain the group average idle bandwidth of each non-consistent memory access group within the preparatory nodes within a preset time period.

[0037] Determine the target task and its task type, calculate the node score of each preparatory node under different task types, and determine the target node based on the task type and node score;

[0038] A target graphics processor is selected from the target nodes based on the average idle bandwidth of the group, so as to execute the target task through the target graphics processor.

[0039] The technical solution described in this application has the following advantages over the prior art:

[0040] The aforementioned graphics processor resource scheduling method, system, computer equipment, and storage medium, based on network interface card (NIC) bandwidth, achieves graphics processor resource scheduling. First, it obtains the NIC's idle bandwidth resources from two dimensions: the node dimension and the non-consistent memory access group dimension. Based on the node dimension, it obtains the node's idle bandwidth within a preset time period; based on the non-consistent memory access group dimension, it obtains the group average idle bandwidth of the non-consistent memory access group within the preset time period. Then, it determines the target task's task type, calculates the node scores of the candidate nodes for different task types, and determines the target node based on the target task's task type and node scores. Finally, it selects the target graphics processor from the target nodes based on the group average idle bandwidth. This effectively improves the utilization rate of idle bandwidth resources and graphics processor resources, further enhancing the performance and efficiency of the training task. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is an application scenario diagram of the distributed program execution method provided in the embodiments of this application;

[0043] Figure 2 This is a flowchart of the first method of the graphics processor resource scheduling method provided in the embodiments of this application;

[0044] Figure 3 This is a flowchart of the second method of the graphics processor resource scheduling method provided in the embodiments of this application;

[0045] Figure 4 This is a system architecture diagram of the graphics processor resource scheduling system provided in the embodiments of this application;

[0046] Figure 5 This is a device structure diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] Example 1

[0049] Reference Figure 1 As shown, Figure 1This is a diagram illustrating an application scenario of the distributed program execution method provided in an embodiment of this application.

[0050] The distributed program execution method provided in this application can be applied to application scenarios of distributed application development, testing, and operation. For example, as shown in the embodiments of this application... Figure 1 As shown, the execution entity of the method provided in this application embodiment can be a distributed system, wherein the distributed system includes multiple computing nodes, and each computing node is a unit for providing computing power, such as a server, or a processor or a kernel within the server, etc., without specific limitations. Exemplarily, the distributed system includes a control node, and the method in this embodiment can also be executed through a fixed control node. Further, the target program is a distributed program, and during the execution of the target program, the target program is implemented in the form of a workflow. The workflow of the target program includes multiple work steps, corresponding to the functional implementation steps of the target program, wherein the work steps are executed by different computing nodes, as shown in the reference... Figure 1 The target program's workflow includes three steps: step #1, step #2, and step #3. Step #1 is executed by computing node A, step #2 by computing node B, and step #3 by computing node C, ultimately achieving the target program's specific functionality.

[0051] Obtain multiple preparatory nodes and the node idle bandwidth of each preparatory node within a preset time period, and obtain the group average idle bandwidth of each non-consistent memory access group within the preparatory nodes within a preset time period.

[0052] Determine the target task and its task type, calculate the node score of each preparatory node under different task types, and determine the target node based on the task type and node score;

[0053] A target graphics processor is selected from the target nodes based on the average idle bandwidth of the group, so as to execute the target task through the target graphics processor.

[0054] Example 2

[0055] Reference Figure 2 As shown, Figure 2 A flowchart of the first method of the graphics processor resource scheduling method provided in the embodiments of this application.

[0056] The graphics processor resource scheduling method includes the following steps:

[0057] S101, obtain multiple preparatory nodes and the node idle bandwidth of each preparatory node within a preset time, and obtain the group average idle bandwidth of each non-consistent memory access group within the preparatory node within a preset time.

[0058] Specifically, this application manages multiple nodes based on a Kubernetes cluster. The Kubernetes cluster includes multiple GPU (Graphics Processing Unit) nodes, referred to as nodes in this application. Each node may include multiple or a single network interface card (NIC). The NICs are high-performance NICs, such as Infiniband or RoCE high-performance NICs, collectively referred to as NICs in this application. To improve the utilization of bandwidth resources in the cluster and enhance training task performance, this application proposes a GPU resource scheduling method. This method schedules GPU resources based on NIC bandwidth. First, it obtains idle bandwidth through two dimensions: node dimension and Non-Uniform Memory Access Group (NUMA) dimension. The node dimension obtains the node's idle bandwidth within a preset time period, and the NUMA dimension obtains the group average idle bandwidth of the NUMA group within the preset time period. Furthermore, the preset time can be set according to requirements. For example, setting it to 10 minutes will obtain the average idle bandwidth of nodes within 10 minutes, as well as the average idle bandwidth of non-consistent memory access groups within 10 minutes. The average idle bandwidth of nodes is calculated on a node-by-node basis; the average idle bandwidth of groups is calculated on a non-consistent memory access group-by-group basis. Specifically, when calculating the average idle bandwidth of groups at the non-consistent memory access group level, it is necessary to obtain the topology of the non-consistent memory access groups, and then obtain the average idle bandwidth of the groups. The data representation format of the non-consistent memory access group topology is as follows:

[0059] [{"numa1":"mlx5_0:20Gb,mlx5_1:80Gb","gpu":"0,1,2,3"},

[0060] {"numa2":"mlx5_2:40Gb,mlx5_3:90Gb","gpu":"4,5,6,7"}].

[0061] The above representation format refers to the following: numa1 (inconsistent memory access group) includes two network cards, mlx5_0 and mlx5_1; and four GPUs, gpu0, gpu1, gpu2, and gpu3; numa2 includes two network cards, mlx5_2 and mlx5_3; and four GPUs, gpu4, gpu5, gpu6, and gpu7. Furthermore, if the GPU card is compatible with multiple network cards, the deep learning framework will automatically select the appropriate network card. It can average the bandwidth of network cards with the same affinity (during training tasks, the training task selects the network card closest to the used GPU card) and combine them into one. This simplifies the data representation format of the inconsistent memory access group topology, reducing data transfer volume, as shown in the following format:

[0062] [{"numa1":"mlx5_0_1:500Gb","gpu":"0,1,2,3"},

[0063] {"numa2":"mlx5_2_3:65Gb","gpu":"4,5,6,7"}].

[0064] The formula for calculating the average idle bandwidth of each non-consistent memory access group within the preparatory node over a preset time period is as follows:

[0065]

[0066] Among them, gpufreebandwidth_numa i For numa i Group average free bandwidth; freebandwidth_num i For numa i Total free bandwidth of network cards within the group; gpufree_num i For numa i The number of graphics processors in the group that are currently idle.

[0067] Specifically, after obtaining the idle bandwidth of each candidate node within a preset time period based on the node dimension, the average idle bandwidth of the group is then calculated based on the dimension of inconsistent memory access groups. The specific calculation method is as follows: the average idle bandwidth of the group is equal to the total idle bandwidth of the network cards in the group divided by the number of graphics processors in the group that are in an idle state.

[0068] S102, determine the target task and its task type, calculate the node score of each preparatory node under different task types, and determine the target node based on the task type and node score;

[0069] Specifically, after obtaining the idle bandwidth of the candidate nodes and the average idle bandwidth of the group, the task type of the target task needs to be determined. Different task types have different requirements for graphics processing unit (GPU) resources, so it is necessary to schedule the corresponding GPU resources according to the task type. After determining the target task and its task type, for different task types of target tasks, it is necessary to calculate the node score of the candidate nodes under different task types. Then, based on the task type of the target task and the node score, the target node is determined so that GPU resources can be selected from the target node, and then the target task is executed using the GPU resources.

[0070] S103, Select a target graphics processor from the target nodes based on the average idle bandwidth of the group, so as to execute the target task through the target graphics processor.

[0071] After assigning target nodes to the target task based on the task type and node score, the corresponding target graphics processor (GPU) can be selected from the target nodes based on the average idle bandwidth of the group, so as to execute the target task through the GPU resources.

[0072] The graphics processor resource scheduling method of this application schedules graphics processor resources based on the network interface card (NIC) bandwidth. First, it obtains the NIC's idle bandwidth resources from two dimensions: the node dimension and the non-consistent memory access group dimension. Based on the node dimension, it obtains the node's idle bandwidth within a preset time period; based on the non-consistent memory access group dimension, it obtains the group average idle bandwidth of the non-consistent memory access group within the preset time period. Then, it determines the task type of the target task, calculates the node scores of the candidate nodes for different task types, and determines the target node based on the target task's task type and node scores. Finally, it selects the target graphics processor from the target nodes based on the group average idle bandwidth. This effectively improves the utilization rate of idle bandwidth resources and graphics processor resources, further enhancing the performance and efficiency of the training task.

[0073] In one implementation, the step of obtaining the reserve nodes, which involves obtaining the node idle bandwidth of each reserve node within a preset time period based on the node dimension, includes:

[0074] Determine resource requirements and select multiple reserve nodes that meet the resource requirements based on a preset scheduling mechanism;

[0075] Specifically, before obtaining the idle bandwidth of a node within a preset time period based on the node dimension, the resource requirements must first be determined, that is, the bandwidth resources needed to execute the target task. Then, a preset scheduling mechanism is used to select reserve nodes that meet the resource requirements, so that the idle bandwidth of each reserve node within the preset time period can be obtained based on the node dimension. The preset scheduling mechanism is an existing scheduling mechanism, that is, a mechanism that selects nodes based on CPU size, memory size, and number of GPUs.

[0076] The idle bandwidth of each of the prepared nodes within a preset time period is obtained based on the node dimension;

[0077] Specifically, after selecting multiple reserve nodes that meet the resource requirements according to the preset scheduling mechanism, the idle bandwidth of each reserve node within a preset time is obtained based on the node dimension, so as to score each reserve node by means of the idle bandwidth.

[0078] In one implementation, the target task's task type includes single-machine task type and multi-machine task type. The steps of determining the target task and its task type, calculating the node score of each candidate node under different task types, and determining the target node based on the task type and node score include:

[0079] Determine the target task and its task type;

[0080] Specifically, the target task can be categorized into single-machine task types and multi-machine task types, with multi-machine task types being distributed computing tasks. The methods for obtaining node scores for reserve nodes differ under different task types; therefore, it is essential to first determine the target task and its task type.

[0081] In response to the fact that the target task is a single-machine task, the node score of each candidate node is calculated based on the principle of minimizing the idle bandwidth of the preset node, and the candidate node with the lowest node score is determined as the target node.

[0082] Specifically, for single-machine task types (single container), high-performance network bandwidth is not required. Therefore, it is necessary to calculate the node score of each candidate node based on the principle of minimizing idle bandwidth, and then determine the candidate node with the lowest node score as the target node. Furthermore, the formula for calculating the node score based on the principle of minimizing idle bandwidth is as follows:

[0083] score node_i =score base +score bandwidth_least

[0084] in, totalbandwidth is the node's total bandwidth, freebandwidth is the node's idle bandwidth, and score is the node's free bandwidth. bandwidth_least The score is the bandwidth score calculated based on the principle of minimizing idle bandwidth of preset nodes. base The base score node_i The node score is calculated based on the principle of minimizing the idle bandwidth of the preset node. The base score is related to the number of central processing units (CPUs), memory, and graphics processing units (GPUs) in the prepared node, and ω is the weight.

[0085] The scoring mechanism considers not only the idle bandwidth of nodes, but also the number of CPUs, memory, and GPUs in the reserve nodes. It comprehensively evaluates each reserve node, that is, it establishes a base score by considering the number of CPUs, memory, and GPUs in the reserve node, and then obtains the corresponding node score by considering the base score of the reserve node and the node's idle bandwidth.

[0086] In response to the fact that the target task is a multi-machine task, the node score of each reserve node is calculated based on the principle of maximizing the idle bandwidth of the preset nodes, and the target node is determined according to the multi-machine task type and the node score.

[0087] Specifically, for multi-machine task types (multiple containers), gradient reduction (data transfer) is required between containers via RDMA (Remote Direct Memory Access) network. Therefore, this task type requires high-performance network bandwidth. This necessitates calculating the node score of each candidate node based on the principle of maximizing idle bandwidth, and then determining the candidate node with the highest score as the target node. Furthermore, the formula for calculating the node score based on the principle of maximizing idle bandwidth is as follows:

[0088] score node_i =score base +score bandwidth_most

[0089] in, score bandwidth_most The score is the bandwidth score calculated based on the principle of maximizing the idle bandwidth of preset nodes. node_i This is the node score calculated based on the principle of maximizing the idle bandwidth of a preset node. For the explanation of other symbols, please refer to the explanation of the formula symbols for calculating the node score based on the principle of minimizing the idle bandwidth of a preset node mentioned above.

[0090] In one implementation, the single-machine task type includes a single-machine single-card task type and a single-machine multi-card task type. The step of selecting a target graphics processor from the target nodes based on the group's average idle bandwidth, and executing the target task through the target graphics processor, includes:

[0091] In response to the fact that the task type of the target task is a single-machine single-card task type, the target group with the lowest average idle bandwidth is determined from the target nodes;

[0092] Specifically, single-machine task types can be further divided into single-machine single-card task types and single-machine multi-card task types. If the target task is determined to be a single-machine single-card task type (one container and one graphics processor), then this type of task does not require the use of high-performance network bandwidth. Since the average idle bandwidth of each non-consistent memory access group in each prepared node of the cluster within the preset time has been obtained before, the non-consistent memory access group with the lowest average idle bandwidth in the target node should be selected as the target group.

[0093] Select any graphics processor from the target group as the target graphics processor to perform the target task through the target graphics processor.

[0094] Specifically, since the task type is a single-machine, single-card task, one graphics processor can be randomly selected from the target group. This graphics processor is the target graphics processor, and the target task is finally executed through the target processor.

[0095] In one embodiment, the step of selecting a target graphics processor from the target nodes based on the average idle bandwidth of the group, and executing the target task through the target graphics processor, further includes:

[0096] In response to the fact that the task type of the target task is a single-machine multi-card task type, the target group with the lowest average idle bandwidth is determined from the target nodes;

[0097] Specifically, if the target task is a single-machine multi-GPU task (one container with multiple GPUs), then the task does not need to use high-performance network bandwidth. Since the average idle bandwidth of each non-consistent memory access group in each preparatory node of the cluster within the preset time has been obtained, the non-consistent memory access group with the lowest average idle bandwidth in the target node should be selected as the target group.

[0098] Select at least two graphics processors from the target group as target graphics processors to perform the target task through the target graphics processors;

[0099] Specifically, since the target task is a single-machine, multi-GPU task, at least two graphics processors (GPUs) need to be selected. The target nodes include multiple non-consistent memory access groups. Therefore, after determining the target nodes, the non-consistent memory access group with the lowest average idle bandwidth is selected from among the target nodes; this non-consistent memory access group is the target group. Since the task type is a single-machine, multi-GPU task, at least two GPUs need to be selected from the target group; these selected GPUs are called target GPUs. Finally, the target task is executed through the target GPUs.

[0100] In response to the fact that the graphics processor resources in the target group cannot meet the target task, a sub-target group with the second lowest average idle bandwidth in the target group is determined from the target nodes, and a corresponding number of graphics processors are selected from the sub-target group as target graphics processors to execute the target task through the target graphics processors.

[0101] Specifically, since the target task is a single-machine, multi-GPU task, at least two graphics processors (GPUs) need to be selected. When selecting at least two processors from the target group, there may be situations where the GPU resources within the target group are insufficient to meet the GPU resource requirements of the target task. Therefore, in this case, GPUs need to be selected from another non-uniform memory group. Specifically, a sub-target group with the second lowest average idle bandwidth needs to be determined from the target nodes, and a corresponding number of GPUs are selected from this sub-target group as the target GPUs to execute the target task.

[0102] In one embodiment, the multi-machine task type includes a multi-machine single-card task type and a multi-machine multi-card task type. The step of selecting a target graphics processor from the target nodes based on the average idle bandwidth of the group, and executing the target task through the target graphics processor, further includes:

[0103] In response to the fact that the task type of the target task is a multi-machine single-card task type, the target group with the highest average idle bandwidth is determined from the target nodes;

[0104] Specifically, if the target task is a multi-machine single-card task (multiple containers and one graphics processor), the containers need to perform gradient reduction (data transfer) based on the RDMA network. Therefore, this type of task requires high-performance network bandwidth. Since the average idle bandwidth of each non-consistent memory access group in each preparatory node in the cluster has been obtained before, the non-consistent memory access group with the highest average idle bandwidth in the target node should be selected as the target group.

[0105] Select any graphics processor from the target group as the target graphics processor to perform the target task through the target graphics processor.

[0106] Specifically, since the target node includes multiple non-consistent access memory groups, after determining the target node, the non-consistent access memory group with the highest average idle bandwidth is selected from the target nodes. This non-consistent access memory group is the target group. Since the task type is a multi-machine, multi-card task, only one graphics processor needs to be selected from the target group. This graphics processor is the target graphics processor, and finally, the target task is executed through the target processor.

[0107] In one embodiment, the step of selecting a target graphics processor from the target nodes based on the average idle bandwidth of the group, and executing the target task through the target graphics processor, further includes:

[0108] In response to the fact that the task type of the target task is a multi-machine multi-card task type, the target group with the highest average idle bandwidth is determined from the target nodes;

[0109] Specifically, if the target task is a multi-machine, multi-GPU task (multiple containers and multiple GPUs), the containers need to perform gradient reduction (data transfer) based on the RDMA network. Therefore, this type of task requires high-performance network bandwidth. Since the average idle bandwidth of each non-consistent memory access group in each preparatory node of the cluster within the preset time has been obtained before, the non-consistent memory access group with the highest average idle bandwidth in the target node should be selected as the target group.

[0110] Select at least two graphics processors from the target group as target graphics processors to perform the target task through the target graphics processors;

[0111] Specifically, since the target task is a multi-machine, multi-GPU task, at least two graphics processors (GPUs) need to be selected. The target nodes include multiple non-consistent memory access groups. Therefore, after determining the target nodes, the non-consistent memory access group with the highest average idle bandwidth is selected from among the target nodes; this non-consistent memory access group is the target group. Because the task is a multi-machine, multi-GPU task, at least two GPUs need to be selected from the target group; these selected GPUs are called target GPUs. Finally, the target task is executed through the target GPUs.

[0112] In response to the fact that the graphics processor resources in the target group cannot meet the target task, a sub-target group with the second highest average idle bandwidth in the target group is determined from the target nodes, and a corresponding number of graphics processors are selected from the sub-target group as target graphics processors to execute the target task through the target graphics processors.

[0113] Specifically, since the target task is a multi-machine, multi-GPU task, at least two graphics processors (GPUs) need to be selected. When selecting at least two processors from the target group, there may be situations where the GPU resources within the target group are insufficient to meet the GPU requirements of the target task. Therefore, in this case, GPUs need to be selected from another non-uniform memory group. Specifically, a secondary target group with the second-highest average idle bandwidth needs to be identified from the target nodes, and a corresponding number of GPUs are selected from this secondary target group as the target GPUs to execute the target task.

[0114] Example 3

[0115] Reference Figure 3 As shown, Figure 3 This is a flowchart of a second method for a bandwidth resource scheduling method provided in an embodiment of this application. Figure 3 In the method shown, with Figure 2 For content that is the same or similar to the method shown, please refer to... Figure 2 The method description will not be repeated here.

[0116] S201, Determine resource requirements, and select multiple reserve nodes that meet the resource requirements according to the preset scheduling mechanism of resource requirements;

[0117] Before obtaining the node's idle bandwidth within a preset time based on the node dimension, the resource requirements must first be determined, that is, the bandwidth resources required to execute the target task. Then, reserve nodes that meet the resource requirements are selected according to the preset scheduling mechanism, so as to obtain the node's idle bandwidth of each reserve node within a preset time based on the node dimension.

[0118] S202, Calculate the idle bandwidth of each of the prepared nodes within a preset time based on the node dimension;

[0119] After selecting multiple reserve nodes that meet the resource requirements according to the preset scheduling mechanism, the idle bandwidth of each reserve node within a preset time is obtained based on the node dimension, so as to score each reserve node by means of the idle bandwidth.

[0120] S203, obtain the average idle bandwidth of each non-consistent memory access group within the preparatory node within a preset time period based on the dimension of non-consistent memory access groups.

[0121] After obtaining the node idle bandwidth of each prepared node within a preset time based on the node dimension, the average idle bandwidth of the group is then calculated based on the dimension of the non-consistent memory access group.

[0122] S204, Determine the target task and its task type;

[0123] The target task can be categorized into single-machine task type and multi-machine task type, with multi-machine task type being distributed computing task. The method for obtaining node scores for reserve nodes differs under different task types; therefore, it is essential to first determine the target task and its task type.

[0124] S205, in response to the target task being a single-machine task, the node score of each candidate node is calculated based on the principle of minimum idle bandwidth of preset nodes, and the candidate node with the lowest node score is determined as the target node.

[0125] For single-machine task types (single container), high-performance network bandwidth is not required. Therefore, it is necessary to calculate the node score of each candidate node based on the principle of minimizing the idle bandwidth of the preset nodes, and then determine the candidate node with the lowest node score as the target node.

[0126] S206, in response to the target task being a single-machine single-card task type, determine the target group with the lowest average idle bandwidth from the target nodes;

[0127] If the target task is a single-machine, single-card task (one container, one graphics processor), then this type of task does not require high-performance network bandwidth. Since the average idle bandwidth of each non-consistent memory access group in each preparatory node of the cluster within the preset time has been obtained before, the non-consistent memory access group with the lowest average idle bandwidth in the target node should be selected as the target group.

[0128] S207, Select any graphics processor from the target group as the target graphics processor, so as to execute the target task through the target graphics processor;

[0129] Since the task type is either a single-machine single-card task or a multi-level single-card task, only one graphics processor needs to be selected from the target group. This graphics processor is the target graphics processor, and the target task is finally executed through the target processor.

[0130] S208, in response to the target task being a single-machine multi-card task type, determine the target group with the lowest average idle bandwidth from the target nodes;

[0131] If the target task is a single-machine multi-GPU task (one container with multiple GPUs), then this type of task does not require high-performance network bandwidth. Since the average idle bandwidth of each non-consistent memory access group in each preparatory node of the cluster within the preset time has been obtained, the non-consistent memory access group with the lowest average idle bandwidth in the target node should be selected as the target group.

[0132] S209, Select at least two graphics processors from the target group as target graphics processors to execute the target task through the target graphics processors;

[0133] Since the target task is a single-machine, multi-GPU task, at least two graphics processors (GPUs) need to be selected. The target nodes include multiple non-consistent memory access groups. Therefore, after determining the target nodes, the non-consistent memory access group with the lowest average idle bandwidth is selected from among the target nodes; this non-consistent memory access group is the target group. Because the task type is a single-machine, multi-GPU task, at least two GPUs need to be selected from the target group; these selected GPUs are called target GPUs. Finally, the target task is executed through the target GPUs.

[0134] S210, in response to the fact that the graphics processor resources in the target group cannot meet the target task, determine the sub-target group with the second lowest average idle bandwidth from the target nodes;

[0135] Since the target task is a single-machine, multi-GPU task, at least two graphics processors (GPUs) need to be selected. When selecting at least two processors from the target group, there may be situations where the GPU resources within the target group are insufficient to meet the GPU resource requirements of the target task. Therefore, in this case, GPUs need to be selected from another non-uniform memory group, specifically the sub-target group with the second lowest average idle bandwidth among the target nodes.

[0136] S211, Select a corresponding number of graphics processors from the sub-target group as target graphics processors, so as to execute the target task through the target graphics processors;

[0137] After determining the sub-target group from the target nodes, select a corresponding number of graphics processors from the sub-target group as target graphics processors to execute the target task through the target graphics processors.

[0138] S212, in response to the target task being a multi-machine task, the node score of each candidate node is calculated based on the principle of maximizing the idle bandwidth of a preset node, and the candidate node with the highest node score is determined as the target node.

[0139] For multi-machine task types (multiple containers), gradient reduction (data transmission) between containers is required based on RDMA network. Therefore, this type of task requires the use of high-performance network bandwidth. Based on the principle of maximizing the idle bandwidth of preset nodes, the node score of each candidate node is calculated, and then the candidate node with the highest node score is determined as the target node.

[0140] S213, in response to the target task being a multi-machine single-card task type, determine the target group with the highest average idle bandwidth from the target nodes, and then execute step S207.

[0141] If the target task is a multi-machine single-card task (multiple containers and one graphics processor), the containers need to perform gradient reduction (data transfer) based on the RDMA network. Therefore, this task type requires high-performance network bandwidth. Since the average idle bandwidth of each non-consistent memory access group in each preparatory node of the cluster within a preset time has been obtained, the non-consistent memory access group with the highest average idle bandwidth in the target node needs to be selected as the target group. Then, step S207 is executed: select any graphics processor from the target group as the target graphics processor to execute the target task through the target graphics processor.

[0142] S214, in response to the target task being a multi-machine multi-card task type, determine the target group with the highest average idle bandwidth from the target nodes;

[0143] If the target task is a multi-machine, multi-GPU task (multiple containers and multiple GPUs), the containers need to perform gradient reduction (data transfer) based on the RDMA network. Therefore, this type of task requires high-performance network bandwidth. Since the average idle bandwidth of each non-consistent memory access group in each preparatory node of the cluster within the preset time has been obtained, the non-consistent memory access group with the highest average idle bandwidth in the target node should be selected as the target group.

[0144] S215, Select at least two graphics processors from the target group as target graphics processors to execute the target task through the target graphics processors;

[0145] Since the target task is a multi-machine, multi-GPU task, at least two graphics processors (GPUs) need to be selected. The target nodes include multiple non-consistent memory access groups. Therefore, after determining the target nodes, the non-consistent memory access group with the highest average idle bandwidth is selected from among the target nodes; this non-consistent memory access group is the target group. Because the task is a multi-machine, multi-GPU task, at least two GPUs need to be selected from the target group; these selected GPUs are the target GPUs. Finally, the target task is executed through the target GPUs.

[0146] S216, in response to the fact that the graphics processor resources in the target group cannot meet the target task, determine the second target group with the second highest average idle bandwidth from the target nodes, and then execute step S211.

[0147] Since the target task is a multi-machine, multi-GPU task, at least two graphics processors (GPUs) need to be selected. When selecting at least two processors from the target group, there may be situations where the GPU resources within the target group are insufficient to meet the GPU requirements of the target task. Therefore, in this case, GPUs need to be selected from another non-uniform memory group. In this case, a sub-target group with the second-highest average idle bandwidth needs to be determined from the target nodes. Then, step S211 is executed: a corresponding number of GPUs are selected from the sub-target group as target GPUs to execute the target task.

[0148] It should be understood that, although Figures 1-2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 1-2At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0149] Example 4

[0150] Reference Figure 4 As shown, Figure 4 A system architecture diagram of a graphics processor resource scheduling system provided in an embodiment of this application.

[0151] The graphics processor resource scheduling system of this embodiment includes:

[0152] The system includes a resource scheduler and multiple nodes, all of which are communicatively connected to the resource scheduler; each node includes a monitoring component and multiple non-consistent memory access groups, all of which are communicatively connected to the monitoring component; each non-consistent memory access group includes multiple network interface cards (NICs) and graphics processors (GPUs), with the NICs communicatively connected to the GPUs.

[0153] like Figure 4 The graphics processing unit (GPU) resource scheduling system shown, taking a two-node example, includes two nodes. Each node comprises two NUMA groups (NUMA0 and NUMA1) and a monitoring component. Each NUMA group includes a CPU, a network interface card (NIC), a GPU, and a PCIe switch. The CPU, NIC, and GPU are all communicatively connected to the PCIe switch. Different NUMA groups are connected via QPI communication between CPUs. Figure 4 The solid lines represent PCIe connection signals, and the dashed lines represent QPI (Quick Path Interconnect) connection signals. The monitoring component is used to obtain the average idle bandwidth of nodes and the average idle bandwidth of groups. The monitoring component communicates with the resource scheduler so that it can report the average idle bandwidth of nodes and groups to the resource scheduler, and the resource scheduler can schedule graphics processor resources based on network card bandwidth.

[0154] The following steps are achieved using a graphics processor resource scheduling system with the above structure:

[0155] Obtain multiple preparatory nodes and the node idle bandwidth of each preparatory node within a preset time period, and obtain the group average idle bandwidth of each non-consistent memory access group within the preparatory nodes within a preset time period.

[0156] Determine the target task and its task type, calculate the node score of each preparatory node under different task types, and determine the target node based on the task type and node score;

[0157] A target graphics processor is selected from the target nodes based on the average idle bandwidth of the group, so as to execute the target task through the target graphics processor.

[0158] Specific limitations regarding the graphics processor resource scheduling system can be found in the method limitations section above, and will not be repeated here. Each module in the aforementioned graphics processor resource scheduling system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in the computer device, or stored in software within the computer device's memory, so that the processor can invoke and execute the corresponding operations of each module.

[0159] Example 5

[0160] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a graphics processor resource scheduling method.

[0161] This computer device can be a terminal, and its internal structure diagram can be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a graphics processor resource scheduling method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0162] Those skilled in the art should understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0163] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0164] Obtain multiple preparatory nodes and the node idle bandwidth of each preparatory node within a preset time period, and obtain the group average idle bandwidth of each non-consistent memory access group within the preparatory nodes within a preset time period.

[0165] Determine the target task and its task type, calculate the node score of each preparatory node under different task types, and determine the target node based on the task type and node score;

[0166] A target graphics processor is selected from the target nodes based on the average idle bandwidth of the group, so as to execute the target task through the target graphics processor.

[0167] Example 6

[0168] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps:

[0169] Obtain multiple preparatory nodes and the node idle bandwidth of each preparatory node within a preset time period, and obtain the group average idle bandwidth of each non-consistent memory access group within the preparatory nodes within a preset time period.

[0170] Determine the target task and its task type, calculate the node score of each preparatory node under different task types, and determine the target node based on the task type and node score;

[0171] A target graphics processor is selected from the target nodes based on the average idle bandwidth of the group, so as to execute the target task through the target graphics processor.

[0172] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0173] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0174] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for scheduling graphics processor resources, characterized in that, The method includes: Obtain multiple preparatory nodes and the node idle bandwidth of each preparatory node within a preset time period, and obtain the group average idle bandwidth of each non-consistent memory access group within the preparatory nodes within a preset time period. Determine the target task and its task type, calculate the node score of each preparatory node under different task types, and determine the target node based on the task type and node score; A target graphics processor is selected from the target nodes based on the average idle bandwidth of the group, so as to execute the target task through the target graphics processor; The target task includes single-machine task type and multi-machine task type. Determining the target task and its task type, calculating the node score of each candidate node under different task types, and determining the target node based on the task type and node score includes: Determine the target task and its task type; In response to the fact that the target task is a single-machine task, the node score of each candidate node is calculated based on the principle of minimizing the idle bandwidth of the preset node, and the candidate node with the lowest node score is determined as the target node. In response to the fact that the target task is a multi-machine task, the node score of each candidate node is calculated based on the principle of maximizing the idle bandwidth of the preset nodes, and the candidate node with the highest node score is determined as the target node.

2. The graphics processor resource scheduling method according to claim 1, characterized in that, The step of acquiring multiple reserve nodes and the node idle bandwidth of each reserve node within a preset time includes: Determine resource requirements and select multiple reserve nodes that meet the resource requirements based on a preset scheduling mechanism; The idle bandwidth of each of the prepared nodes within a preset time period is obtained based on the node dimension.

3. The graphics processor resource scheduling method according to claim 1, characterized in that, The single-machine task type includes single-machine single-card task type and single-machine multi-card task type. The step of selecting a target graphics processor from the target nodes based on the average idle bandwidth of the group, and executing the target task through the target graphics processor, includes: In response to the fact that the task type of the target task is a single-machine single-card task type, the target group with the lowest average idle bandwidth is determined from the target nodes; Select any graphics processor from the target group as the target graphics processor to perform the target task through the target graphics processor.

4. The graphics processor resource scheduling method according to claim 3, characterized in that, The step of selecting a target graphics processor from the target nodes based on the average idle bandwidth of the group, and executing the target task through the target graphics processor, further includes: In response to the fact that the task type of the target task is a single-machine multi-card task type, the target group with the lowest average idle bandwidth is determined from the target nodes; Select at least two graphics processors from the target group as target graphics processors to perform the target task through the target graphics processors; In response to the fact that the graphics processor resources in the target group cannot meet the target task, a sub-target group with the second lowest average idle bandwidth is determined from the target nodes, and a corresponding number of graphics processors are selected from the sub-target group as target graphics processors to execute the target task through the target graphics processors.

5. The graphics processor resource scheduling method according to claim 1, characterized in that, The multi-machine task types include multi-machine single-card task types and multi-machine multi-card task types. The step of selecting a target graphics processor from the target nodes based on the average idle bandwidth of the group, and executing the target task through the target graphics processor, further includes: In response to the fact that the task type of the target task is a multi-machine single-card task type, the target group with the highest average idle bandwidth is determined from the target nodes; Select any graphics processor from the target group as the target graphics processor to perform the target task through the target graphics processor.

6. The graphics processor resource scheduling method according to claim 5, characterized in that, The step of selecting a target graphics processor from the target nodes based on the average idle bandwidth of the group, and executing the target task through the target graphics processor, further includes: In response to the fact that the task type of the target task is a multi-machine multi-card task type, the target group with the highest average idle bandwidth is determined from the target nodes; Select at least two graphics processors from the target group as target graphics processors to perform the target task through the target graphics processors; In response to the fact that the graphics processor resources in the target group cannot meet the target task, a sub-target group with the second highest average idle bandwidth in the target group is determined from the target nodes, and a corresponding number of graphics processors are selected from the sub-target group as target graphics processors to execute the target task through the target graphics processors.

7. A graphics processor resource scheduling system that implements the graphics processor resource scheduling method as described in any one of claims 1 to 6, characterized in that, The system includes: The system includes a resource scheduler and multiple nodes. Each node includes a monitoring component and multiple non-consistent memory access groups. Each non-consistent memory access group is communicatively connected to the monitoring component, and the resource scheduler is communicatively connected to the monitoring component. Each non-consistent memory access group includes multiple network interface cards (NICs) and graphics processors (GPUs). Each NIC is communicatively connected to the graphics processor.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program that, when executed by a processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 6.

Images