A container scheduling method, device, medium and equipment based on GPU topology structure

By using GPU topology to filter and bind virtual GPU resources in GPU shared virtualization technology, the problem of uneven resource allocation in the existing technology is solved, and the overall performance and usage rate of the GPU are improved.

CN118656177BActive Publication Date: 2024-12-03WUHAN AI RES +1
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411153824.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-12-03
Estimated Expiration
2044-08-21

AI Technical Summary

Technical Problem

In the existing virtualization technology based on GPU sharing, the GPU topology is not fully utilized, resulting in uneven resource allocation and reducing the GPU usage and performance.

Method used

By receiving the virtual GPU usage request initiated by the container, obtain the node's virtual GPU allocation information and physical GPU topology information, filter out the nodes that can meet the request, and select the target computing node based on the topology information between the physical GPUs, The target container is bound to this node for scheduling.

Benefits of technology

It realizes more accurate resource allocation, improves data exchange performance and overall GPU performance and usage among multi-card GPUs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118656177B_ABST
    Figure CN118656177B_ABST
Patent Text Reader

Abstract

The present invention discloses a container scheduling method, device, medium and equipment based on GPU topology structure, wherein the method first receives a virtual GPU usage request initiated by a container, and obtains virtual GPU allocation information of all nodes and topology information between physical GPUs from a cluster; then, based on the relationship between the virtual GPU allocation information of the node and the virtual GPU resource application amount, the node whose idle virtual GPU resources can meet the virtual GPU usage request is screened out, and then the target computing node is screened out based on the topology information between the physical GPUs in the schedulable node; finally, the target container is bound to the target computing node, and each idle virtual GPU resource in the target computing node is mounted to the target container. The present invention can realize more accurate resource allocation, fully utilize the data exchange performance between multi-card GPUs, thereby improving the overall performance and utilization rate of the GPU.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a container scheduling method, device, medium and equipment based on a GPU topology structure. Background Art

[0002] With the development of deep learning technology, various types of deep learning models have emerged, and the models are being used more and more widely in various industries. If the model needs to achieve good results, in addition to the two elements of data and algorithm, powerful computing power is also indispensable. However, the current mainstream GPUs are relatively expensive, and in general, the exclusive card mode will waste precious computing resources such as GPUs, that is, different users have different depths of understanding of the model, resulting in independent cards being applied for but not fully utilizing the resources. Therefore, in order to solve the above problems, improving resource utilization through GPU sharing virtualization technology has gradually become a current research hotspot.

[0003] In the GPU sharing-based virtualization technology used in the prior art, the container scheduling method based on the GPU topology structure relies on the classic polling and Binpack scheduling strategies, which do not fully explore and utilize the topological structure advantages of the GPU. This limitation leads to the inability to fully utilize this advantage when allocating GPU resources, resulting in relatively low data exchange efficiency between multiple vgpus (virtual GPUs, each vgpu is on a different GPU), which fails to fully utilize the performance of the GPU and reduces the utilization rate of the GPU. Summary of the invention

[0004] In view of the technical problems of low performance and utilization rate of GPU in the prior art, the present invention proposes a container scheduling method, device, medium and equipment based on GPU topology structure.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A first aspect provides a container scheduling method based on a GPU topology structure, including:

[0007] Receive a virtual GPU usage request initiated by a container, where the virtual GPU usage request includes a virtual GPU resource application amount;

[0008] Get the virtual GPU allocation information and topology information between physical GPUs of all nodes in the cluster;

[0009] According to the relationship between the virtual GPU allocation information of the node and the virtual GPU resource application amount, a node whose idle virtual GPU resources can satisfy the virtual GPU usage request is selected as a schedulable node;

[0010] Filtering a target computing node from the schedulable nodes according to topology information between physical GPUs in the schedulable nodes, and binding a target container to the target computing node, wherein the target container is a container that initiates the virtual GPU use request;

[0011] The physical GPU of the target computing node bound to the target container is mounted to the target container according to the use request to schedule the target container.

[0012] In one embodiment, the virtual GPU allocation information of the node includes allocation information of all physical GPUs on the node, and the node whose idle virtual GPU resources can satisfy the virtual GPU usage request is screened out according to the relationship between the virtual GPU allocation information of the node and the virtual GPU resource application amount, including:

[0013] According to the allocation information of all physical GPUs on the node and the amount of virtual GPU resource applications, the physical GPUs whose idle virtual GPU resources can meet the virtual GPU usage request are selected;

[0014] According to the relationship between the number of physical GPUs whose idle virtual GPU resources can satisfy the virtual GPU usage request and the virtual GPU application amount, the schedulable nodes are screened out.

[0015] In one embodiment, the step of screening out a physical GPU whose idle virtual GPU resources can satisfy the virtual GPU usage request according to the allocation information of all physical GPUs on the node and the virtual GPU resource application amount includes:

[0016] According to the allocation information of all physical GPUs on the node, obtain the idle virtual GPU amount of the physical GPU;

[0017] According to whether the idle virtual GPU amount of the physical GPU is greater than or equal to the virtual GPU resource application amount and whether the physical GPU type matches the virtual GPU application type, a physical GPU whose idle virtual GPU resources on the node can satisfy the virtual GPU usage request is screened out.

[0018] In one embodiment, the screening out schedulable nodes according to the relationship between the number of physical GPUs that can satisfy the virtual GPU usage request and the virtual GPU application amount based on the screened idle virtual GPU resources includes:

[0019] Count the number of physical GPUs on a node that satisfy virtual GPU usage requests;

[0020] It is determined whether the total number of physical GPUs on a node that meet the virtual GPU usage request is greater than or equal to the virtual GPU application amount. If so, the node is considered a schedulable node; otherwise, the node is considered an unschedulable node.

[0021] In one implementation, the virtual GPU resources requested by the container include the virtual GPU application amount, the video memory application amount, and the computing power application amount. The idle virtual GPU amount of the GPU application type physical GPU includes the remaining virtual GPU amount, the remaining video memory, and the remaining computing power on the physical GPU. The method of screening out the physical GPU whose idle virtual GPU resources on the node can meet the virtual GPU usage request based on whether the idle virtual GPU amount of the physical GPU is greater than or equal to the virtual GPU resource application amount and whether the physical GPU type matches the virtual GPU application type includes:

[0022] Determine whether the remaining virtual GPU amount on the physical GPU is greater than or equal to the virtual GPU application amount, whether the remaining video memory on the physical GPU is greater than or equal to the video memory application amount, and whether the remaining computing power on the physical GPU is greater than or equal to the computing power application amount; if all are greater than or equal to, determine whether the type of the physical GPU matches the type of the virtual GPU application; if they match, use the physical GPU as the physical GPU on the node that satisfies the virtual GPU usage request.

[0023] In one implementation, the step of selecting a target computing node based on topological information between physical GPUs in the schedulable nodes includes:

[0024] The physical GPUs whose remaining idle virtual GPUs on the schedulable node can satisfy the virtual GPU usage request are grouped according to the virtual GPU application amount, and each group is scored according to the topological information between the grouped physical GPUs, and the group with the highest score is selected as the score of the schedulable node;

[0025] According to the scores of the schedulable nodes, the nodes with the highest scores are selected from all schedulable nodes as the target computing nodes.

[0026] In one implementation, the topology information between the physical GPUs includes a connection mode between the physical GPUs, and the physical GPUs whose idle virtual GPU remaining amount on the schedulable node can satisfy the virtual GPU usage request are grouped according to the virtual GPU application amount, and each group is scored according to the topology information between the grouped physical GPUs, including:

[0027] The physical GPUs that can satisfy the virtual GPU usage request are grouped using the virtual GPU application amount as the number of physical GPUs in the group;

[0028] Set the corresponding score according to the type of connection between physical GPUs;

[0029] The two physical GPUs in the group are scored according to the connection mode and the set score between the two physical GPUs, and the sum of the scores of all pairs of physical GPUs in the group is used as the score of the group.

[0030] Based on the same inventive concept, the second aspect of the present invention provides a container scheduling method and device based on a GPU topology structure, including:

[0031] A virtual GPU usage request receiving module, configured to receive a virtual GPU usage request initiated by a container, wherein the virtual GPU usage request includes a virtual GPU resource application amount;

[0032] The allocation information and topology information acquisition module is used to obtain the virtual GPU allocation information of all nodes and the topology information between physical GPUs from the cluster;

[0033] A schedulable node screening module, configured to screen out nodes whose idle virtual GPU resources can satisfy the virtual GPU usage request as schedulable nodes according to the relationship between the virtual GPU allocation information of the node and the virtual GPU resource application amount;

[0034] A target computing node screening module, configured to screen out a target computing node from the schedulable nodes according to topology information between physical GPUs in the schedulable nodes, and bind a target container to the target computing node, wherein the target container is a container that initiates the virtual GPU use request;

[0035] The scheduling module is used to mount the physical GPU of the target computing node bound to the target container to the target container according to the use request, so as to schedule the target container.

[0036] Based on the same inventive concept, the third aspect of the present invention provides a non-volatile storage medium, which stores multiple instructions, and the instructions are suitable for being loaded by a processor and executing the container scheduling method based on GPU topology structure described in the first aspect.

[0037] Based on the same inventive concept, the fourth aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the container scheduling method based on GPU topology structure described in the first aspect.

[0038] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0039] In an embodiment of the present invention, when the scheduler performs container scheduling, it first receives a virtual GPU usage request initiated by a receiving container, and obtains virtual GPU allocation information of all nodes and topological information between physical GPUs from a cluster; then, according to the relationship between the virtual GPU allocation information of the node and the virtual GPU resource application amount, the node whose idle virtual GPU resources can meet the virtual GPU usage request is screened out; then, according to the topological information between the physical GPUs in the schedulable nodes, the target computing node is screened out from the schedulable nodes, and the target container is bound to the target computing node, and finally the physical GPU of the target computing node bound to the target container is mounted to the target container according to the usage request to schedule the target container. Through the above method, when performing container scheduling, compared with the classic polling and Binpack scheduling strategies that rely on the existing vgpu scheduling, the present invention takes into account the topological information between physical GPUs, supports the topological structure of the perceived GPU, and can allocate resources more accurately when scheduling and allocating multiple vgpus (each vgpu is on a different GPU), and can give full play to the data exchange performance between multi-card GPUs, thereby improving the overall performance and utilization of the GPU. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A flowchart of a container scheduling method based on a GPU topology structure provided by an embodiment of the present invention;

[0042] Figure 2 A flowchart of another container scheduling method based on GPU topology structure provided by an embodiment of the present invention;

[0043] Figure 3 A flowchart of implementing physical GPU screening and target computing node screening provided in an embodiment of the present invention;

[0044] Figure 4 A schematic diagram of the structure of a container scheduling method and device based on a GPU topology structure provided by an embodiment of the present invention;

[0045] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0048] The main drawback of the container scheduling method based on GPU topology disclosed in the prior art is that it fails to fully explore and utilize the topological structure advantages of the GPU. This limitation makes it impossible to fully utilize this advantage when allocating GPU resources. As a result, the data exchange efficiency between multiple vgpus (each vgpu is on a different GPU) is relatively low, the performance of the GPU is not fully utilized, and the utilization rate of the GPU is reduced.

[0049] To solve the virtualization and scheduling problems of GPU resources in containers, the present invention uses volcano (a container batch computing platform based on Kubernetes) and plug-ins (devices) provided by volcano to manage GPUs. It supports dividing a physical GPU into multiple vgpus (each vgpu is on a physical GPU) to allocate to applications for use, which improves the utilization rate of the GPU. However, it is found during use that vgpu scheduling mainly relies on classic polling and Binpack scheduling strategies. However, the scheduling strategy did not take into account the complex topological structure inside the GPU at the beginning of the design, which is undoubtedly a shortcoming of resource utilization. Due to this neglect, the applicant found that in practical applications, resource allocation is not always optimal, especially when scheduling and allocating multi-card vgpus (each vgpu is on a different GPU), it is impossible to allocate resources more accurately, give full play to the data exchange performance between multi-card GPUs, and improve the overall performance and utilization of the GPU. Therefore, there is an urgent need for a new scheduling mechanism that can understand and utilize the topological structure of the GPU so that we can allocate resources more accurately and give full play to the data exchange performance between multiple vgpus (each vgpu is on a different GPU).

[0050] Embodiment 1

[0051] Based on the above technical problems, an embodiment of the present invention provides an embodiment of a container scheduling method based on a GPU topology structure. The steps shown in the flowchart in the accompanying drawing can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0052] Figure 1 is a flowchart of a container scheduling method based on a GPU topology structure according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0053] S101: receiving a virtual GPU usage request initiated by a container, where the virtual GPU usage request includes a virtual GPU resource application amount;

[0054] S102: Acquire virtual GPU allocation information of all nodes and topology information between physical GPUs from the cluster;

[0055] S103: Filter out nodes whose idle virtual GPU resources can satisfy the virtual GPU usage request as schedulable nodes according to the relationship between the virtual GPU allocation information of the node and the virtual GPU resource application amount;

[0056] S104: Filtering a target computing node from the schedulable nodes according to topology information between physical GPUs in the schedulable nodes, and binding a target container to the target computing node, wherein the target container is a container that initiates the virtual GPU use request;

[0057] S105: Mounting the physical GPU of the target computing node bound to the target container to the target container according to the use request, so as to schedule the target container.

[0058] Specifically, the container scheduling of the present invention refers to the scheduling of pods by k8s (Kubernetes, an open source, is used to manage containerized applications on multiple hosts in a cloud platform). A container, namely a pod, is actually a group of closely cooperating containers, that is, where the application executes code.

[0059] The above scheme of the present invention mainly includes the above five steps, virtual GPU usage request reception, allocation information and topology information acquisition, schedulable node screening, target computing node screening and container scheduling. The execution subject is the scheduler (container scheduling device) in the cluster, which can be a scheduler based on volcano that supports virtual GPU topology scheduling; the scheduler runs inside the cluster and is responsible for the scheduling and management of resources (cpu / memory / gpu) in the cluster.

[0060] Before step S101 , a container for applying to use a virtual GPU is created, thereby starting to schedule the container.

[0061] Specifically, virtual GPU resources include the following types: requested vgpu video memory size, requested vgpu video memory ratio, requested vgpu computing power, and requested number of vgpus (corresponding to the requested vgpu video memory, requested vgpu computing power, and requested number of vgpus).

[0062] The code to apply for vgpu video memory size is: volcano.sh / vgpu-cores

[0063] The code to apply for vgpu memory ratio is: volcano.sh / vgpu-memory-percentage

[0064] Apply for vgpu computing power: integer type (0-100), the default is 0, which means the percentage of computing power reserved for each task by default. If it is set to 0, it means that the task may be assigned to any GPU that meets the video memory requirements; if it is set to 100, it means that the task has exclusive use of the entire graphics card.

[0065] The code for applying for the number of vgpus is: volcano.sh / vgpu-number, which is an integer type and defaults to 1. If it is configured to 0, the configuration will not take effect.

[0066] In step S102, all nodes in the cluster will summarize and report the topological connection information between physical GPUs to the cluster. The allocation cache in the cluster will also have the virtual GPU allocation information of the nodes. Therefore, the scheduler can obtain the virtual GPU allocation information of all nodes and the topological information between physical GPUs from the cluster.

[0067] During the implementation of step S103, nodes are selected from the currently available nodes in turn, and the idle virtual GPU resources of the nodes can be obtained according to the virtual GPU allocation information of the nodes. It is determined whether the idle virtual GPU resources of the nodes meet the virtual GPU resource application amount. If so, the node is a node whose idle virtual GPU resources can meet the virtual GPU usage request as a schedulable node. If not, the next node is selected and the above operation is repeated until all available nodes are screened.

[0068] In step S104, the target computing node is selected based on the topological information between the physical GPUs in the schedulable nodes;

[0069] In one implementation, the virtual GPU allocation information of the node includes allocation information of all physical GPUs on the node. S103 may be implemented in the following manner:

[0070] According to the allocation information of all physical GPUs on the node and the amount of virtual GPU resource applications, the physical GPUs whose idle virtual GPU resources can meet the virtual GPU usage request are selected;

[0071] According to the relationship between the number of physical GPUs whose idle virtual GPU resources can satisfy the virtual GPU usage request and the virtual GPU application amount, the schedulable nodes are screened out.

[0072] The allocation information of a physical GPU refers to the virtual GPU information that has been allocated to the physical GPU, including the number of allocated virtual GPUs, video memory, computing power, etc. This implementation first screens out the physical GPUs whose idle virtual GPU resources can satisfy the virtual GPU usage request, and then screens the schedulable nodes based on whether the number of the screened physical GPUs meets the virtual GPU application amount.

[0073] In one embodiment, the step of screening out a physical GPU whose idle virtual GPU resources can satisfy the virtual GPU usage request according to the allocation information of all physical GPUs on the node and the virtual GPU resource application amount includes:

[0074] According to the allocation information of all physical GPUs on the node, obtain the idle virtual GPU amount of the physical GPU;

[0075] According to whether the idle virtual GPU amount of the physical GPU is greater than or equal to the virtual GPU resource application amount and whether the physical GPU type matches the virtual GPU application type, a physical GPU whose idle virtual GPU resources on the node can satisfy the virtual GPU usage request is screened out.

[0076] Specifically, when the idle virtual GPU amount of the physical GPU is greater than or equal to the virtual GPU resource application amount, it means that the idle virtual GPU amount (remaining virtual GPU amount) of the physical GPU can meet the virtual GPU resource usage request, and then GPU type matching is performed.

[0077] In one embodiment, the screening out schedulable nodes according to the relationship between the number of physical GPUs that can satisfy the virtual GPU usage request and the virtual GPU application amount based on the screened idle virtual GPU resources includes:

[0078] Count the number of physical GPUs on a node that satisfy virtual GPU usage requests;

[0079] It is determined whether the total number of physical GPUs on a node that meet the virtual GPU usage request is greater than or equal to the virtual GPU application amount. If so, the node is considered a schedulable node; otherwise, the node is considered an unschedulable node.

[0080] In the specific implementation process, the physical GPUs whose idle virtual GPU resources can meet the virtual GPU usage request can be marked, and then all the physical GPUs that meet the conditions on a node are counted to determine whether it is a schedulable node. If the number of physical GPUs in the virtual GPU usage request after screening is less than the number of virtual GPUs (vgpu) applied for by the container, the node cannot schedule the container.

[0081] In one implementation, the virtual GPU resources requested by the container include the virtual GPU application amount, the video memory application amount, and the computing power application amount. The idle virtual GPU amount of the GPU application type physical GPU includes the remaining virtual GPU amount, the remaining video memory, and the remaining computing power on the physical GPU. The method of screening out the physical GPU whose idle virtual GPU resources on the node can meet the virtual GPU usage request based on whether the idle virtual GPU amount of the physical GPU is greater than or equal to the virtual GPU resource application amount and whether the physical GPU type matches the virtual GPU application type includes:

[0082] Determine whether the remaining virtual GPU amount on the physical GPU is greater than or equal to the virtual GPU application amount, whether the remaining video memory on the physical GPU is greater than or equal to the video memory application amount, and whether the remaining computing power on the physical GPU is greater than or equal to the computing power application amount; if all are greater than or equal to, determine whether the type of the physical GPU matches the type of the virtual GPU application; if they match, use the physical GPU as the physical GPU on the node that satisfies the virtual GPU usage request.

[0083] In a specific implementation, the selection process of a physical GPU that meets the virtual GPU usage request can be implemented by the following steps:

[0084] S1031: Get the information of all physical GPUs on the node, traverse all physical GPUs on the node in reverse order, set index to the number of physical GPUs on the node, and get the physical GPU numbered as index minus 1. If index minus 1 is greater than or equal to 0, it means that there are still physical GPUs that have not been screened, then perform the screening process S1032.

[0085] S1032: The number of virtual GPUs (vgpu) on the current physical GPU is less than or equal to the number of allocated GPUs. If so, the next physical GPU is selected.

[0086] S1033: The remaining video memory of the current physical GPU is less than or equal to the requested video memory. If so, select the next physical GPU;

[0087] S1034: The remaining computing power of the current physical GPU is less than or equal to the computing power application amount. If so, select the next physical GPU;

[0088] S1035: The current physical GPU type does not match the requested GPU type. If so, select the next physical GPU.

[0089] S1036: If the physical GPU passes the screening of steps S1032, S1033, S1034, and S1035, it is considered that the physical GPU meets the application usage (application amount) of the container virtual GPU (vgpu), and the physical GPU is marked.

[0090] S1037: If the index is less than 0 after being reduced by 1, it is considered that all physical GPUs on the current node have been screened.

[0091] See also Figure 3 , which is a flow chart for implementing physical GPU screening and target computing node screening provided in an embodiment of the present invention.

[0092] In one implementation, the step of selecting a target computing node based on topological information between physical GPUs in the schedulable nodes includes:

[0093] The physical GPUs whose remaining idle virtual GPUs on the schedulable node can satisfy the virtual GPU usage request are grouped according to the virtual GPU application amount, and each group is scored according to the topological information between the grouped physical GPUs, and the group with the highest score is selected as the score of the schedulable node;

[0094] According to the scores of the schedulable nodes, the nodes with the highest scores are selected from all schedulable nodes as the target computing nodes.

[0095] Specifically, the physical GPUs that meet the usage requests are grouped according to the virtual GPU application amount, and then scored, and the target computing nodes are screened out according to the scoring results.

[0096] See also Figure 2 , is a flowchart of another container scheduling method based on GPU topology structure provided by an embodiment of the present invention, which specifically includes the following steps:

[0097] Step S1, the nodes in the cluster report the physical GPU topology connection information to the cluster;

[0098] Step S2, receiving a virtual GPU usage request initiated by a container;

[0099] Step S3, selecting virtual GPU allocation information and physical GPU topology information of a node in the cluster in sequence;

[0100] Step S4, screening out nodes whose idle virtual GPU remaining capacity can satisfy the usage request according to the node virtual GPU allocation information;

[0101] Step S5, grouping the physical GPUs on the node according to the virtual GPU usage request (the virtual GPUs are logically isolated from the physical GPUs), and then scoring the topological connections between the two physical GPUs in the group, and selecting the group with the highest score as the node score. If the remaining amount of idle virtual GPUs of the node meets the demand, execute step S6, otherwise execute step S3;

[0102] Step S6, selecting the node with the highest score from all the nodes whose idle virtual GPU remaining capacity can satisfy the use request, and binding the target container (the container that initiates the virtual GPU use request) to the node;

[0103] Step S7: mount the physical GPU group (with the highest score) bound to the target container to the target container according to the usage request.

[0104] Specifically, step S1 requires all nodes in the cluster to summarize and report the topological connection information between the physical GPUs on the nodes to the cluster;

[0105] Step S2 creates a container that applies to use the virtual GPU, thereby starting to schedule the container;

[0106] Step S3 selects a node from the first node among the available nodes in turn;

[0107] In step S4, when the remaining amount of idle virtual GPUs in the node cannot meet the usage request, return to step S3 to select the next node, otherwise, execute step S5, and group the physical GPUs on the node according to the topological structure information according to the usage request of the container virtual GPU. Among them, in step S1, the node reports the physical GPU topology information to the cluster. At the beginning of scheduling, the physical GPU topology of the node can be obtained, and the allocation information of the node can be obtained from the allocation cache.

[0108] In S5, if the number of physical GPUs that meet the application usage after screening is greater than or equal to the number of virtual GPUs (vgpus) applied for by the container, the physical GPUs are grouped based on the number of applied virtual GPUs (vgpus). After grouping, the connection methods of the physical GPUs in each group are scored. If the number of physical GPUs that meet the application usage after screening is less than the number of virtual GPUs (vgpus) applied for by the container, the node cannot schedule the container.

[0109] In step S6, if the number of nodes that meet the usage request amount screened in S5 is greater than 0, the node with the highest score is selected from all nodes that meet the requirements, and the container is bound to the node, and the group with the highest score is allocated to the container. Otherwise, the container cannot be scheduled.

[0110] In step S7, after the container is scheduled to the node with the highest score, the physical GPU group bound to the container (with the highest score) is mounted to the target container according to the usage request.

[0111] In one implementation, the topology information between the physical GPUs includes a connection mode between the physical GPUs, and the physical GPUs whose idle virtual GPU remaining amount on the schedulable node can satisfy the virtual GPU usage request are grouped according to the virtual GPU application amount, and each group is scored according to the topology information between the grouped physical GPUs, including:

[0112] The physical GPUs that can satisfy the virtual GPU usage request are grouped using the virtual GPU application amount as the number of physical GPUs in the group;

[0113] Set the corresponding score according to the type of connection between physical GPUs;

[0114] The two physical GPUs in the group are scored according to the connection mode and the set score between the two physical GPUs, and the sum of the scores of all pairs of physical GPUs in the group is used as the score of the group.

[0115] In the specific implementation process, if the number of physical GPUs that meet the application usage after screening is greater than or equal to the number of virtual GPUs (vgpus) applied for by the container, the physical GPUs are grouped based on the number of applied virtual GPUs (vgpus). After grouping, the connection mode of the physical GPUs in each group is scored. The corresponding score is set according to the type of connection mode between the physical GPUs. The two physical GPUs in the group are scored according to the connection mode and the set score between the two physical GPUs. If the connection between the two physical GPUs is of unknown type, 0 points are added; if it is of SYS type, 10 points are added; if it is of NODE type, 20 points are added; if it is of PHB type, 30 points are added; if it is of PXB type, 40 points are added; if it is of PIX type, 50 points are added; if it is of NV# type, 100 points are added. After calculating the scores of all groups, the group with the highest score is selected as the score of the node.

[0116] For example, if a physical server (node) has 8 physical GPUs, these 8 physical GPUs are numbered 0, 1, 2, 3, 4, 5, 6, 7; each physical GPU is logically divided into 10 virtual GPUs (vgpu), and a pod applies for 2 virtual GPUs (the number of virtual GPUs applied is 2). Assuming that the physical GPUs that meet the container GPU requirements (the remaining number of virtual GPUs and video memory / computing power are greater than or equal to the container application) are numbered 0, 1, 3, 5, 7, according to the relationship between the internal topology of the GPU, a combination method is needed. These physical GPUs are grouped based on 2, and the following grouping can be obtained.

[0117] •(0, 1)

[0118] •(0, 3)

[0119] •(0, 5)

[0120] •(0, 7)

[0121] •(1, 3)

[0122] •(1, 5)

[0123] •(1, 7)

[0124] •(3, 5)

[0125] •(3, 7)

[0126] •(5, 7)

[0127] For the above groups, each group has multiple GPUs, and the connection methods between GPUs may be different, and the connection efficiency is different. According to the connection method between two physical GPUs, the groups are scored. After calculating the scores of all groups, the group with the highest score is selected as the node score.

[0128] Then the node with the highest score is selected, the target container is bound to the node, and the group with the highest score is assigned to the container. Otherwise, the container cannot be scheduled. After the container is scheduled to the node with the highest score, the physical GPU group (with the highest score) bound to the container is mounted to the target container according to the usage request.

[0129] The professional terms involved in the present invention are explained as follows:

[0130] Cluster: Generally refers to the combination or joint work of multiple computers in a computer system. In the context of Kubernetes, a cluster refers to a system consisting of a group of nodes that work together to run and manage containerized applications.

[0131] Container (pod): is a virtualization technology that allows an application and all its dependencies to run in an isolated environment. Containers are lighter than traditional virtual machines and are often used to deploy microservice architectures and cloud-native applications.

[0132] vgpu: short for "virtual GPU", which refers to a virtualized graphics processing unit. vgpu allows sharing of physical GPU resources in a virtualized environment, which is very useful for applications that require graphics acceleration.

[0133] NVIDIA provides 6 physical communication methods for GPU cards:

[0134] SYS (System Copy Command): Communicates across NUMA (Non-Uniform Memory Access) nodes via QPI (QuickPath Interconnect) and PCIe. This communication method is used in complex multi-CPU, multi-GPU environments, similar to the connection from GPU1 to GPU5 in the figure.

[0135] NODE: Communicates within a single NUMA node through the Host Bridge PCIe bus. Although there is no specific example, there can be multiple CPU chips on a NUMA node, and the PCIe bus is used to connect these chips with the GPU.

[0136] PHB: PCIe bus communication through Host Bridge (part of Root Complex). This method is used for communication between multiple PCIe buses under the same Root Complex.

[0137] PXB: spans multiple PCIe Bridges (switches) but does not pass through the Host Bridge PCIe bus. Usually used for communication between multiple PCIe Bridges, similar to the connection from GPU3 to GPU4 in the figure.

[0138] PIX: communicates through a PCIe Bridge (switch), that is, the communication between GPU cards mounted on the same PCIe Bridge. Similar to the connection from GPU1 to GPU2 in the figure.

[0139] NV#: Use NVLink communication. NVLink is a high-speed communication technology developed by NVIDIA that allows GPUs to communicate at higher speeds.

[0140] The present invention aims to solve the problem of uneven resource allocation and limited performance caused by ignoring the GPU topology structure in the existing GPU scheduling technology. Through the secondary development of volcano and the plug-ins (devices) provided by volcano, it supports the topology structure of the GPU perception. When scheduling and allocating multiple vgpus (each vgpu is on a different GPU), resources can be allocated more accurately, and the data exchange performance between multiple GPUs can be fully utilized, thereby improving the overall performance and utilization rate of the GPU.

[0141] Embodiment 2

[0142] Based on the same inventive concept, this embodiment discloses a container scheduling method and device based on GPU topology structure, see Figure 4 , the device comprises:

[0143] A virtual GPU usage request receiving module 401 is used to receive a virtual GPU usage request initiated by a container, wherein the virtual GPU usage request includes a virtual GPU resource application amount;

[0144] Allocation information and topology information acquisition module 402, used to acquire virtual GPU allocation information of all nodes and topology information between physical GPUs from the cluster;

[0145] The schedulable node screening module 403 is used to screen out nodes whose idle virtual GPU resources can satisfy the virtual GPU usage request as schedulable nodes according to the relationship between the virtual GPU allocation information of the node and the virtual GPU resource application amount;

[0146] A target computing node screening module 404 is used to screen out a target computing node from the schedulable nodes according to the topology information between the physical GPUs in the schedulable nodes, and bind a target container to the target computing node, wherein the target container is the container that initiates the virtual GPU use request;

[0147] The scheduling module 405 is used to mount the physical GPU of the target computing node bound to the target container to the target container according to the use request, so as to schedule the target container.

[0148] Since the device introduced in the second embodiment of the present invention is a device used to implement the container scheduling method based on the GPU topology structure in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, the person skilled in the art can understand the specific structure and deformation of the device, so it is not repeated here. All devices used in the method in the first embodiment of the present invention belong to the scope of protection of the present invention.

[0149] Embodiment 3

[0150] Based on the same inventive concept, the present invention also provides a non-volatile storage medium, which stores multiple instructions, and the instructions are suitable for being loaded by a processor and executing the container scheduling method based on GPU topology structure described in Example 1.

[0151] Since the non-volatile storage medium introduced in the third embodiment of the present invention is the storage medium used to implement the container scheduling method based on the GPU topology structure in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, the person skilled in the art can understand the specific structure and deformation of the storage medium, so it is not repeated here. All storage media used in the method of the first embodiment of the present invention belong to the scope of protection of the present invention.

[0152] Embodiment 4

[0153] The present invention also provides an electronic device, see Figure 5 , including a memory 501 and a processor 502, wherein the memory stores a computer program 503, and the processor is configured to run the computer program to execute the container scheduling method based on the GPU topology structure described in Example 1.

[0154] Since the electronic device introduced in the fourth embodiment of the present invention is an electronic device used to implement the container scheduling method based on the GPU topology structure in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, the person skilled in the art can understand the specific structure and deformation of the electronic device, so it is not repeated here. All electronic devices used in the method of the first embodiment of the present invention belong to the scope of protection of the present invention.

[0155] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0157] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic creative concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A container scheduling method based on GPU topology structure, characterized in that: include: Receive a virtual GPU usage request initiated by a container, where the virtual GPU usage request includes a virtual GPU resource application amount; Get the virtual GPU allocation information and topology information between physical GPUs of all nodes in the cluster; According to the relationship between the virtual GPU allocation information of the node and the virtual GPU resource application amount, a node whose idle virtual GPU resources can satisfy the virtual GPU usage request is selected as a schedulable node; Filtering a target computing node from the schedulable nodes according to topology information between physical GPUs in the schedulable nodes, and binding a target container to the target computing node, wherein the target container is a container that initiates the virtual GPU use request; Mounting the physical GPU of the target computing node bound to the target container to the target container according to the use request to schedule the target container; The step of selecting a target computing node according to topology information between physical GPUs in the schedulable nodes includes: The physical GPUs whose idle virtual GPU remaining amount on the schedulable node can satisfy the virtual GPU usage request are grouped according to the virtual GPU application amount, and each group is scored according to the topological information between the grouped physical GPUs, and the group with the highest score is selected as the score of the schedulable node, wherein the scoring of each group according to the topological information between the grouped physical GPUs comprises: scoring each group according to the connection mode between the physical GPUs; According to the scores of the schedulable nodes, the nodes with the highest scores are selected from all schedulable nodes as the target computing nodes.

2. The container scheduling method based on GPU topology structure according to claim 1, characterized in that: The virtual GPU allocation information of a node includes the allocation information of all physical GPUs on the node.

3. The container scheduling method based on GPU topology structure according to claim 2, characterized in that: The step of screening out nodes whose idle virtual GPU resources can satisfy the virtual GPU usage request according to the relationship between the virtual GPU allocation information of the node and the virtual GPU resource application amount includes: According to the allocation information of all physical GPUs on the node and the amount of virtual GPU resource applications, the physical GPUs whose idle virtual GPU resources can meet the virtual GPU usage request are selected; According to the relationship between the number of physical GPUs whose idle virtual GPU resources can satisfy the virtual GPU usage request and the virtual GPU application amount, the schedulable nodes are screened out.

4. The container scheduling method based on GPU topology structure according to claim 3, characterized in that: The step of screening out a physical GPU whose idle virtual GPU resources can satisfy the virtual GPU usage request according to the allocation information of all physical GPUs on the node and the virtual GPU resource application amount includes: According to the allocation information of all physical GPUs on the node, obtain the idle virtual GPU amount of the physical GPU; According to whether the idle virtual GPU amount of the physical GPU is greater than or equal to the virtual GPU resource application amount and whether the physical GPU type matches the virtual GPU application type, a physical GPU whose idle virtual GPU resources on the node can satisfy the virtual GPU usage request is screened out.

5. The container scheduling method based on GPU topology structure according to claim 3, characterized in that: The screening out of schedulable nodes according to the relationship between the number of physical GPUs that can satisfy the virtual GPU usage request and the virtual GPU application amount based on the screened out idle virtual GPU resources includes: Count the number of physical GPUs on a node that satisfy virtual GPU usage requests; It is determined whether the total number of physical GPUs on a node that meet the virtual GPU usage request is greater than or equal to the virtual GPU application amount. If so, the node is considered a schedulable node; otherwise, the node is considered an unschedulable node.

6. The container scheduling method based on GPU topology structure according to claim 1, characterized in that: The topology information between physical GPUs includes the connection method between physical GPUs.

7. The container scheduling method based on GPU topology structure according to claim 6, characterized in that: The step of grouping the physical GPUs whose idle virtual GPU remaining amount on the schedulable node can satisfy the virtual GPU usage request according to the virtual GPU application amount, and scoring each group according to the topology information between the grouped physical GPUs, includes: The physical GPUs that can satisfy the virtual GPU usage request are grouped using the virtual GPU application amount as the number of physical GPUs in the group; Set the corresponding score according to the type of connection between physical GPUs; The two physical GPUs in the group are scored according to the connection mode and the set score between the two physical GPUs, and the sum of the scores of all pairs of physical GPUs in the group is used as the score of the group.

8. A container scheduling device based on GPU topology structure, characterized in that: include: A virtual GPU usage request receiving module, configured to receive a virtual GPU usage request initiated by a container, wherein the virtual GPU usage request includes a virtual GPU resource application amount; The allocation information and topology information acquisition module is used to obtain the virtual GPU allocation information of all nodes and the topology information between physical GPUs from the cluster; A schedulable node screening module, configured to screen out nodes whose idle virtual GPU resources can satisfy the virtual GPU usage request as schedulable nodes according to the relationship between the virtual GPU allocation information of the node and the virtual GPU resource application amount; A target computing node screening module, configured to screen out a target computing node from the schedulable nodes according to topology information between physical GPUs in the schedulable nodes, and bind a target container to the target computing node, wherein the target container is a container that initiates the virtual GPU use request; A container scheduling module, used to mount the physical GPU of the target computing node bound to the target container to the target container according to the use request, so as to schedule the target container; The target computing node screening module is specifically used for: The physical GPUs whose idle virtual GPU remaining amount on the schedulable node can satisfy the virtual GPU usage request are grouped according to the virtual GPU application amount, and each group is scored according to the topological information between the grouped physical GPUs, and the group with the highest score is selected as the score of the schedulable node, wherein the scoring of each group according to the topological information between the grouped physical GPUs comprises: scoring each group according to the connection mode between the physical GPUs; According to the scores of the schedulable nodes, the nodes with the highest scores are selected from all schedulable nodes as the target computing nodes.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the container scheduling method based on GPU topology structure as described in any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the container scheduling method based on GPU topology structure according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Method and device for sharing GPU, electronic equipment and readable storage medium

    CN111913794A

  • Dynamic allocation method and system for multiple GPUs

    CN112527513A

  • Cluster node resource scheduling method and device

    CN113157379A

  • Resource scheduling method and device, equipment and storage medium

    CN113377520A

  • Container scheduling method and device, cluster system, electronic equipment and storage medium

    CN114610497A