Resource allocation method, apparatus, device, and storage medium

Abstract

The disclosure provides a resource allocation method, device, equipment and storage medium. It is related to the technical field of computers, and in particular to the fields of resource management and task allocation in computer technology. The specific implementation scheme is: creating a pod for a target task; obtaining GPU resource requirement information of the target task; obtaining available node information and available GPU resource information of a target cluster; based on the available node information and the available GPU resource information, assigning a first target node and a second target node to the pod to meet the GPU resource requirement information, wherein the first target node is a node where the target GPU resource allocated to the pod is located, and the second target node is a node where the pod allocated to the pod is located. According to the technical scheme of the disclosure, the resource utilization rate can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of resource management and task allocation in computer technology, specifically to a resource allocation method, apparatus, device, and storage medium. Background Technology

[0002] In machine learning scenarios, computing resources often require unified management. When some clusters (such as Kubernetes clusters, or K8S clusters) execute tasks, they allocate resources to the corresponding set of containers (pods), specifically allocating nodes and Graphics Processing Units (GPUs) to the pods. In related technologies, only nodes with GPU resources can be allocated to pods during allocation. However, this allocation method results in low resource utilization. Summary of the Invention

[0003] This disclosure provides a resource allocation method, apparatus, device, and storage medium.

[0004] According to a first aspect of this disclosure, a resource allocation method is provided, comprising:

[0005] Create a pod for the target task;

[0006] Obtain the GPU resource requirements for this target task;

[0007] Obtain information on available nodes and available GPU resources in the target cluster;

[0008] Based on available node information and available GPU resource information, a first target node and a second target node that meet the GPU resource requirements are allocated to the pod. The first target node is the node where the target GPU resource allocated to the pod is located, and the second target node is the node where the pod is located.

[0009] According to a second aspect of this disclosure, a resource allocation apparatus is provided, comprising:

[0010] Create a module to create a pod for the target task;

[0011] The first acquisition module is used to acquire the GPU resource requirement information for the target task;

[0012] The second acquisition module is used to acquire information on available nodes and available GPU resources in the target cluster.

[0013] The allocation module is used to allocate a first target node and a second target node to the pod based on available node information and available GPU resource information, which meet the GPU resource requirements. The first target node is the node where the target GPU resource allocated to the pod is located, and the second target node is the node where the pod is located.

[0014] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] The memory is communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method provided in the first aspect above.

[0018] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method provided in the first aspect above.

[0019] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided in the first aspect described above.

[0020] The technical solutions of this disclosure can improve resource utilization.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0022] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0023] Figure 1 This is a flowchart illustrating a resource allocation method according to an embodiment of the present disclosure;

[0024] Figure 2 This is a schematic diagram illustrating the relationship between nodes in a computer room according to an embodiment of the present disclosure;

[0025] Figure 3 This is a general flowchart of a resource allocation method according to an embodiment of the present disclosure;

[0026] Figure 4 This is a schematic diagram of the composition of a resource allocation device according to an embodiment of the present disclosure. Figure 1 ;

[0027] Figure 5 This is a schematic diagram of the composition of a resource allocation device according to an embodiment of the present disclosure. Figure 2 ;

[0028] Figure 6 This is a schematic diagram of a resource allocation scenario according to an embodiment of this disclosure;

[0029] Figure 7 This is a block diagram of an electronic device used to implement the resource allocation method of the embodiments of this disclosure. Detailed Implementation

[0030] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0031] The terms "first," "second," and "third," etc., used in the embodiments, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0032] This disclosure provides a resource allocation method. Figure 1 This is a flowchart illustrating a resource allocation method according to an embodiment of the present disclosure. This resource allocation method can be applied to a resource allocation device. The resource allocation device is located in an electronic device, which includes, but is not limited to, fixed devices and / or mobile devices. The electronic device has resource scheduling capabilities. For example, the electronic device can be deployed on a target cluster, or it can be independent of the target cluster but connected to it. In some possible implementations, the resource allocation method can also be implemented by a processor calling computer-readable instructions stored in memory. Figure 1 As shown, the resource allocation method includes:

[0033] S101: Create a pod for the target task;

[0034] S102: Obtain GPU resource requirements information for the target task;

[0035] S103: Obtain information on available nodes and available GPU resources in the target cluster;

[0036] S104: Based on available node information and available GPU resource information, allocate a first target node and a second target node that meet the GPU resource requirements of the pod. The first target node is the node where the target GPU resource allocated to the pod is located, and the second target node is the node where the pod is located.

[0037] Here, the target cluster can be a cluster delivered via a private cloud, such as a cluster built on Kubernetes (K8S), or a cluster built on an open-source distributed resource management framework (e.g., Mesos) and application testing tools (e.g., Marathon). The above is merely an illustrative example and is not intended to limit all possible types of target clusters; it is simply not an exhaustive list.

[0038] K8S is a container cluster management platform based on a cluster manager. It provides unified management of underlying host, network and storage resources based on container technology. It provides functions such as application deployment, maintenance and expansion mechanisms. K8S can be used to easily manage containerized applications running across machines.

[0039] In this embodiment of the disclosure, the target cluster includes multiple nodes, each with a different identifier. Nodes can be bare metal, physical machines, virtual machines, etc.

[0040] In this embodiment of the disclosure, the target task is a task that needs to be executed by the target cluster. For example, the target task can be a detection task. Another example is a classification task. Yet another example is a segmentation task. The above are merely illustrative examples and are not intended to limit all possible types of target tasks; they are simply not exhaustive.

[0041] In this embodiment of the disclosure, the target task is a task that requires the use of GPU resources. The target task runs in a pod.

[0042] In this embodiment of the disclosure, the target task can be a task created by a user through the user interface of the target cluster. The target task can also be a target task created by other devices forwarded by the terminal. This disclosure does not limit the source method of the target task.

[0043] In this embodiment of the disclosure, there is no limitation on the specific implementation method of creating a pod for the target task.

[0044] In this embodiment of the disclosure, the GPU resource requirement information includes information related to the GPU resources requested by the target task. The GPU resource requirement information includes at least the amount of GPU resources used. It may also include GPU resource usage conditions such as the type of GPU card to which the requested GPU resources belong, and the topology of the GPU cards to which the GPU resources belong.

[0045] Here, the types of GPU cards include, but are not limited to, NVIDIA GPU cards and Bitmain GPU cards.

[0046] Here, the GPU card topology refers to the relationships between GPU cards, including but not limited to communication relationships and sequential relationships (order of being called). For example, communication relationships include the communication methods between a particular GPU and other GPUs. For example, topology relationships include which GPU is preceding and following a particular GPU.

[0047] When two or more GPUs have a topological relationship, the GPU card to be called can be selected based on this topological relationship. For example, if node 1 has P GPU cards, and there is a topological relationship between Q GPU cards, then if the sum of the GPU resources of R GPU cards among the Q GPU cards can meet the GPU resource usage requested by the target task, then node 1 can be one of the candidate nodes.

[0048] In some embodiments, obtaining available node information and available GPU resource information of the target cluster includes: determining available node information and available GPU resource information of the target cluster based on the reported information from the GPU resource pool and the dynamic information of the target cluster.

[0049] Here, available node information can be understood as nodes that are in an idle state.

[0050] Here, available GPU resource information can be understood as information about GPU resources that are in an idle state.

[0051] In this embodiment of the disclosure, the first target node and the second target node are two nodes among the currently available nodes in the target cluster.

[0052] In this embodiment of the disclosure, the execution order of S102 and S103 is not limited. S102 and S103 may be executed simultaneously; or S102 may be executed before S103; or S102 may be executed after S103.

[0053] The technical solution described in this disclosure receives a pod created for a target task; obtains GPU resource requirement information for the target task; obtains available node information and available GPU resource information of the target cluster; and, based on the available node information and available GPU resource information, allocates a first target node and a second target node that meet the GPU resource requirement information to the pod. The first target node is the node where the target GPU resource allocated to the pod resides, and the second target node is the node where the pod resides. Thus, compared to the allocation method of only allocating a single node with GPU resources to the pod, this disclosure adopts an allocation method of allocating two nodes to the pod: one is the first target node where the target GPU resource allocated to the pod resides, and the other is the second target node where the pod resides. The second target node may not have GPU resources, thereby enabling the second target node to execute the target task corresponding to the pod based on the GPU resources of the first target node. By adopting an allocation method of allocating different nodes to the pod and GPU resources, the decoupling between the node where the GPU resource resides and the node where the pod resides is achieved, eliminating the limitation of only allocating a node with GPU resources to the pod and improving resource utilization.

[0054] In some embodiments, GPU resource requirement information includes GPU resource usage, GPU card type, and GPU card topology. Based on available node information and available GPU resource information, allocating a first target node and a second target node that meet the GPU resource requirement information to a pod includes: determining multiple candidate nodes that meet the GPU resource usage for the pod based on the available node information and available GPU resource information; determining a first set of nodes that meets the GPU card type and GPU card topology from the multiple candidate nodes; and determining the first target node and the second target node based on the first set of nodes.

[0055] Here, GPU resource usage includes the number of GPUs and / or computing power requested.

[0056] For example, the target cluster consists of N nodes, and M of these N nodes have at least one GPU card. M and N are both positive integers, where M is less than or equal to N and N is a positive integer greater than or equal to 2. Assume the target cluster currently has k available nodes: n1, n2, ..., nk. If the available nodes n1, n2, n3, n4, n5, and ni can satisfy the GPU resource usage requested by the target task, then n1, n2, n3, n4, n5, and ni are determined as candidate nodes. If n1, n2, n3, n4, and n5 satisfy the GPU card type and GPU card topology requested by the target task, then n1, n2, n3, n4, and n5 are determined as the first set of nodes. Here, k and i are both positive integers.

[0057] Thus, by first determining the first set of nodes that meet the GPU resource usage requirements for the pod, and then determining the first target node and the second target node based on the first set of nodes, the selection speed and accuracy of the first target node and the second target node can be improved.

[0058] In some embodiments, determining a first target node and a second target node based on a first node set includes: determining the amount of GPU resource idle time corresponding to each GPU of each node in the first node set; selecting nodes from the first node set whose GPU resource idle time satisfies the GPU resource usage as a second node set; and determining the first target node and the second target node based on the second node set.

[0059] Here, GPU resource idle time can include both video memory idle time and computing power idle time.

[0060] Here, the second set of nodes includes the nodes where the target GPU resources allocated to the pod reside.

[0061] For example, if nodes n1, n2, n3, n4, and n5 are determined as the first node set, and the GPU resources of n1, n2, n3, and n4 in the first node set can meet the video memory and computing power requested by the target task, then nodes n1, n2, n3, and n4 are determined as the second node set.

[0062] Therefore, by first filtering the second set of nodes from the first set of nodes, and then determining the first and second target nodes based on the second set of nodes, the filtering accuracy of the first and second target nodes can be improved.

[0063] In some embodiments, determining the first target node and the second target node based on the second node set includes: obtaining information about the switch corresponding to each node in the second node set; determining the third node set and the fourth node set corresponding to each node in the second node set based on the information about the switch corresponding to each node in the second node set; and determining the first target node and the second target node from the second node set, the third node set, and the fourth node set.

[0064] Here, the information about the switch may include at least one of the following: the switch's identifier, the switch's network information, and the target cluster's network information.

[0065] Figure 2 This diagram illustrates the relationships between the nodes in the computer room, such as... Figure 2As shown, the data center contains multiple server racks, which communicate with each other via switches. Each rack corresponds to one switch, and each rack includes one or more nodes. Nodes within the same rack communicate with each other through the same switch, while nodes in different racks communicate by forwarding relevant information through switches. Each node may or may not have a GPU card. Figure 2 In the diagram, nodes marked with solid lines have GPU cards, while nodes marked with dashed lines do not.

[0066] Thus, by first filtering the third and fourth node sets based on the second node set, and then determining the first and second target nodes from the second, third, and fourth node sets, the filtering range of the first and second target nodes can be increased, thereby helping to improve resource utilization.

[0067] In some embodiments, based on the information of the switch corresponding to each node in the second node set, the third node set and the fourth node set corresponding to each node in the second node set are determined, including: for any node in the second node set, the nodes corresponding to the same switch as that node are determined as the third node set corresponding to that node; for any node in the second node set, the nodes corresponding to different switches as that node are determined as the fourth node set corresponding to that node.

[0068] In practical applications, the third and fourth node sets can be determined simultaneously for the second node set. Alternatively, the third node set can be determined first, followed by the fourth node set. Or, the fourth node set can be determined first, followed by the third node set.

[0069] In this way, the third and fourth node sets can be determined based on the information of the switch corresponding to each node in the second node set, which can increase the range of options for the first and second target nodes, thereby helping to improve the speed of allocating nodes and GPU resources to pods.

[0070] In some embodiments, determining the first target node and the second target node from the second node set, the third node set, and the fourth node set includes: obtaining the load conditions corresponding to the second node set, the third node set, and the fourth node set respectively; and determining the first target node and the second target node based on the load conditions corresponding to the second node set, the third node set, and the fourth node set respectively.

[0071] Here, load information can include the usage of various workloads. These workloads include, but are not limited to, CPU, disk, and memory. For example, CPU usage can be represented by CPU utilization rate. Disk usage can be represented by disk space occupied. Memory usage can be represented by the amount of memory used.

[0072] In this way, while increasing the selection range of the first and second target nodes, the first and second target nodes can be determined according to the load conditions corresponding to the second, third, and fourth node sets, respectively, thus ensuring the availability of the allocated first and second target nodes.

[0073] In some embodiments, determining the first target node and the second target node based on the load conditions corresponding to the second node set, the third node set, and the fourth node set respectively includes: determining various attributes of each node in the second node set, the third node set, and the fourth node set, and the weight values ​​corresponding to each attribute; determining the total weight value of each node based on the weight values ​​corresponding to the various attributes of each node; determining the maximum total weight value based on the total weight value of each node; and determining the first target node where the GPU resource is located and the second target node where the pod is located based on the maximum total weight value.

[0074] In this embodiment of the disclosure, the attributes corresponding to each node may include at least one of the following attributes: set (located in the second node set, the third node set, or the fourth node set), network throughput of the corresponding switch, network usage of the corresponding switch, GPU memory free amount, GPU computing power free amount, disk free amount, central processing unit (CPU) free amount, and GPU dynamic priority.

[0075] Here, a weight value can be set in advance for each attribute.

[0076] When a node is located in the second, third, or fourth node set, the weight values ​​are set from high to low.

[0077] Among them, the network throughput and usage of the switch, the amount of free GPU memory, the amount of free GPU computing power, the amount of free disk, and the amount of free CPU are all considered. The larger the amount of free time, the higher the weight value.

[0078] Among them, the higher the GPU priority, the higher the weight value.

[0079] For example, when nodes are located in the second, third, and fourth batches, the weight values ​​are determined as follows: the target GPU resources used by the pod are located on nodes 1, 2, or 3; the pod is located on nodes 1, 2, 3... or 7. When both the GPU and the pod are on node 1 (the second node set), the weight is the highest, for example, 100, because they are on the same node. When the GPU is on node 1 and the pod is on node 2 or 4 (the third node set), the weight is slightly lower, for example, 30, because they share the same switch. When the GPU is on node 1 and the pod is on node 3, 5, 6, or 7 (the fourth node set), the weight is the lowest, for example, 10, because they do not share the same switch. The same logic applies when the GPU is on other nodes. The relationship between the second, third, and fourth node sets and the weights is shown in Table 1.

[0080] node 1 2 3 4 5 6 7 1 100 30 10 30 10 10 10 2 30 100 10 30 10 10 10 3 10 10 100 10 30 30 10

[0081] Table 1

[0082] In Table 1, the leftmost number represents the node where the GPU is located, and the topmost number represents the node where the pod is located.

[0083] When calculating the weight values, you can refer to the following steps:

[0084] For the case where the GPU is located on node 1, determine the weight values ​​for the cases where the pod is located on nodes 1 to 7, that is, add up the weight values ​​corresponding to each attribute to obtain the total weight;

[0085] For the case where the GPU is located on node 2, determine the weight values ​​for the cases where the pod is located on nodes 1 to 7, that is, add up the weight values ​​corresponding to each attribute to obtain the total weight;

[0086] For the case where the GPU is located on node 3, determine the weight values ​​for the cases where the pod is located on nodes 1 to 7, that is, add up the weight values ​​corresponding to each attribute to obtain the total weight.

[0087] Based on the above steps, a total of 21 total weights are obtained. First, the maximum value of the total weight is determined from the 21 total weights. Then, the nodes where the GPU and pod are located are determined based on the maximum total weight value, which is the optimal solution.

[0088] Based on this result, scheduling is performed. For example, if the GPU is located on node 2 and the pod is located on node 1, the total weight value is the largest. In this case, node 1 is assigned to the pod, and the GPU service address of node 2 is assigned to the pod so that the pod can use GPU resources based on the service address.

[0089] This allows for GPU resource scheduling even when the pod and the target GPU resource are on different nodes, solving the problem of not being able to schedule nodes for a pod when GPU resources are insufficient. Furthermore, it improves the efficiency of GPU resource utilization and enables dynamic resource scheduling.

[0090] In some embodiments, each node's various attributes include at least one of the following attributes:

[0091] Each node belongs to a set, and the weight value varies depending on the set.

[0092] Network throughput of the switch corresponding to each node;

[0093] The usage of the switch corresponding to each node;

[0094] The amount of free GPU memory for each node;

[0095] The amount of idle computing power of the GPU corresponding to each node;

[0096] The amount of free disk space corresponding to each node;

[0097] CPU idle time for each node;

[0098] The GPU priority for each node.

[0099] Here, different sets correspond to different weight values. The weight value of the nodes in the second node set is greater than the weight value of the nodes in the third node set, and the weight value of the nodes in the third node set is greater than the weight value of the nodes in the fourth node set.

[0100] Here, the higher the network throughput of the switch, the higher its weight value.

[0101] Here, the more switches are used, the smaller the weight value.

[0102] Here, for various idle quantities, the larger the idle quantity, the greater the weight value.

[0103] Here, GPU priority refers to the priority at which the GPU is scheduled. The higher the GPU priority, the larger the weight value.

[0104] In this way, by assigning different weight values ​​to different attributes, the determination of the first target node and the second target node is standardized, thereby improving the determination speed of the first target node and the second target node.

[0105] In some embodiments, the resource allocation method may further include: notifying a second target node to perform a target task using the target GPU resources of the first target node.

[0106] This disclosure does not limit the notification method.

[0107] Here, notifying the second target node to use the target GPU resources of the first target node to execute the target task includes: notifying the second target node of the service address of the target GPU resources of the first target node, so that the second target node can call the target GPU resources of the first target node based on the service address of the target GPU resources, and execute the target task based on the target GPU resources.

[0108] In this way, when pods and GPUs are on different nodes, GPU resource scheduling is performed, solving the problem of not being able to schedule nodes for pods when GPU resources are insufficient. Furthermore, it improves the efficiency of GPU resource utilization and achieves dynamic scheduling.

[0109] Figure 3 A flowchart illustrating the resource allocation process is shown, such as... Figure 3 As shown, the process includes:

[0110] S301: The target cluster obtains GPU resource requirement information for the target task;

[0111] Here, GPU resource requirements include: usage requirements (video memory requirements and computing power requirements), usage conditions, GPU allocation method, and GPU priority.

[0112] Here, the usage conditions include the type of GPU card and the topology of the GPU cards. The types of GPU cards include, but are not limited to: NVIDIA GPU cards and Bitmain GPU cards.

[0113] Here, there are two GPU allocation methods: Method a. The pod and GPU are on the same node; Method b. The pod and GPU can be on different nodes.

[0114] Here, GPU priority is the set priority, which can be set to a value from 0 to 9. The higher the priority, the earlier it is scheduled.

[0115] Here, the target task can be a user-created task that requires GPU resources. A pod is generated for the task, and the task can run in the pod.

[0116] S302: The target cluster uses information from the GPU resource pool, the node information where the GPU is located, and the cluster dynamic information to find a suitable first set of nodes for the pod corresponding to the target task.

[0117] S303: The target cluster assigns specific nodes to pods based on the first set of nodes and allocates GPU service addresses.

[0118] Here, a specific node is assigned to the pod, and a GPU service address is allocated, including determining the node where the pod resides and the node where the GPU resides.

[0119] For method A, where the pod and GPU are on the same node, the allocation steps include:

[0120] Compare the GPU usage requirements (memory and computing power requirements) with the GPU usage information in the GPU resource pool to identify nodes that meet the criteria. Specifically, select nodes where both available memory and computing power exceed the requirements, and which also meet usage conditions such as GPU card type and node name.

[0121] For method b, the pod and GPU can be on different nodes, and the allocation steps include:

[0122] Step b1: Compare the GPU usage demand with the GPU usage information in the GPU resource pool to identify nodes that meet the criteria. Based on the reported information and cluster dynamic information, determine the available video memory and computing power. Compare the available video memory and computing power with the GPU usage demand; if the conditions are met, select the corresponding nodes. Then, based on the type of GPU card and the topology of the GPU cards in the usage conditions, obtain the first set of nodes.

[0123] Step b2: Continue filtering in the first node set to determine the available video memory and computing power of each GPU in each node. If the available video memory and computing power of a GPU meet the GPU resource usage requirements, the node is retained; otherwise, the node is discarded, and the second node set is determined. The service address of the GPU is determined based on the second node set.

[0124] Step b3: For each node in the second node set, determine the switch that the node is connected to, and then determine the corresponding third and fourth node sets based on the switch (while comprehensively considering the network information of the node in the node information, the network information of the switch, and the network information of the Kubernetes cluster). Details are as follows:

[0125] refer to Figure 2 In terms of hardware, there are a total of 3 server racks in the data center. Server rack 1 has three nodes 1, 2, and 4; server rack 2 has three nodes 3, 5, and 6; and the third server rack has one node 7.

[0126] For example, the selected second set of nodes includes node 1, node 2, and node 3. For each second set of nodes, different nodes under the same switch constitute the third set of nodes; nodes under different switches constitute the fourth set of nodes.

[0127] For node 1, the third set of nodes is nodes 2 and 4; the fourth set of nodes is nodes 3, 5, 6, and 7.

[0128] For node 2, the third set of nodes is nodes 1 and 4; the fourth set of nodes is nodes 3, 5, 6, and 7.

[0129] For node 3, the third set of nodes is nodes 5 and 6; the fourth set of nodes is nodes 1, 2, 4, and 7.

[0130] Step b4: Based on the second, third, and fourth node sets and cluster dynamic information, determine the load (CPU, disk, network) usage of each node, select the optimal node, determine the optimal solution node for the node where the GPU is located and the node where the pod is located, and pre-allocate GPU service addresses for the pod.

[0131] In practical applications, GPU resource pools can report GPU resource information and node information of the nodes where the GPUs reside through Kubernetes plugins or standalone programs.

[0132] The GPU resource information includes: the GPU model and number of each node, the video memory and computing power of each GPU, and the topological relationship between GPUs.

[0133] The node information of the node where the GPU is located includes: the node's CPU, memory, disk, network information of the node (number of network cards, network card throughput), network information of the switch where the node is located (switch throughput, switch networking information), and network information of the Kubernetes cluster where the node is located.

[0134] In practical applications, cluster dynamic information of the target cluster can be collected through a separate Kubernetes plugin or a standalone program.

[0135] The cluster dynamic information includes the allocation status of each GPU in the node (which have been allocated and which have not), the usage status of the GPU (allocated GPUs may not be used), the usage status of resources on the node (CPU, disk, network), and the network status of Kubernetes (used bandwidth and remaining bandwidth).

[0136] In this way, by classifying nodes based on their relative positional relationships (second node set, third node set, fourth node set), determining the corresponding weight values, and summing them with the weights corresponding to other attributes, the optimal node is found, enabling pods and GPUs to reside on different nodes; this further enables the over-allocation of GPU resources, that is, if a GPU changes from busy to idle, it can be used by pods on the same node or other nodes, improving the utilization efficiency of GPU resources and realizing dynamic scheduling.

[0137] It should be understood that Figure 2 and Figure 3 The schematic diagrams shown are merely illustrative and not restrictive; those skilled in the art can interpret them based on... Figure 2 and Figure 3Even with various obvious changes and / or substitutions to the examples, the resulting technical solutions still fall within the scope of this disclosure.

[0138] This disclosure provides a resource allocation device, such as... Figure 4 As shown, the resource allocation device includes: a creation module 401 for creating a container set for a target task; a first acquisition module 402 for acquiring GPU resource requirement information of the target task; a second acquisition module 403 for acquiring available node information and available GPU resource information of the target cluster; and an allocation module 404 for allocating a first target node and a second target node that meet the GPU resource requirement information to the pod based on the available node information and available GPU resource information, wherein the first target node is the node where the target GPU resource allocated to the pod is located, and the second target node is the node where the pod is located.

[0139] In some embodiments, the GPU resource requirement information includes GPU resource usage, GPU card type, and GPU card topology. The allocation module 404 includes: a first determining submodule, used to determine multiple candidate nodes that meet the GPU resource usage requirements for a pod based on available node information and available GPU resource information; a second determining submodule, used to determine a first set of nodes that meet the GPU card type and GPU card topology from the multiple candidate nodes; and a third determining submodule, used to determine a first target node and a second target node based on the first set of nodes.

[0140] In some embodiments, the third determining submodule is configured to: determine the amount of GPU resource idle time corresponding to each GPU of each node in the first node set; select nodes from the first node set whose GPU resource idle time satisfies the GPU resource usage as a second node set; and determine a first target node and a second target node based on the second node set.

[0141] In some embodiments, the third determining submodule is configured to: obtain information about the switch corresponding to each node in the second node set; determine the third node set and the fourth node set corresponding to each node in the second node set based on the information about the switch corresponding to each node in the second node set; and determine the first target node and the second target node from the second node set, the third node set and the fourth node set.

[0142] In some embodiments, the third determining submodule is configured to: for any node in the second node set, determine the nodes that correspond to the same switch as that node as the third node set corresponding to that node; and for any node in the second node set, determine the nodes that correspond to different switches as the fourth node set corresponding to that node.

[0143] In some embodiments, the third determining submodule is configured to: obtain the load conditions corresponding to the second node set, the third node set, and the fourth node set respectively; and determine the first target node and the second target node based on the load conditions corresponding to the second node set, the third node set, and the fourth node set respectively.

[0144] In some embodiments, the third determining submodule is configured to: determine the attributes of each node in the second node set, the third node set, and the fourth node set, and the weight value corresponding to the attributes; determine the total weight value of each node based on the attributes of each node and the weight value corresponding to the attributes; determine the maximum total weight value based on the total weight value of each node; and determine the first target node and the second target node based on the maximum total weight value.

[0145] In some embodiments, each node's attributes include at least one of the following attributes:

[0146] The set to which each node belongs;

[0147] Network throughput of the switch corresponding to each node;

[0148] The usage of the switch corresponding to each node;

[0149] The amount of free video memory on the GPU card corresponding to each node;

[0150] The amount of idle computing power of the GPU card corresponding to each node;

[0151] The amount of free disk space corresponding to each node;

[0152] CPU idle time for each node;

[0153] The GPU priority for each node.

[0154] In some embodiments, such as Figure 5 As shown, the resource allocation device may further include: a notification module 405, used to send the service address of the target GPU resource in the first target node to the second target node, so that the second target node can call the target GPU resource to execute the target task based on the service address.

[0155] Those skilled in the art should understand that the functions of each processing module in the resource allocation device of this disclosure embodiment can be understood with reference to the relevant description of the resource allocation method described above. Each processing module in the resource allocation device of this disclosure embodiment can be implemented by an analog circuit that implements the functions described in the embodiments of this disclosure, or by running software that performs the functions described in the embodiments of this disclosure on an electronic device.

[0156] The resource allocation device in this embodiment can improve the resource utilization of the target cluster.

[0157] Figure 6 A schematic diagram of a resource allocation scenario is shown, from... Figure 6 As can be seen, electronic devices such as cloud servers receive target tasks sent from various terminals, which include GPU resource requirement information; electronic devices create pods for target tasks; obtain GPU resource requirement information of target tasks, available node information of target clusters, and available GPU resource information, and allocate the first target node where the target GPU resource is located and the second target node where the pod is located to the pod.

[0158] This disclosure does not limit the number of terminals or electronic devices; in practical applications, it may include multiple terminals or multiple electronic devices.

[0159] It should be understood that Figure 6 The scene diagrams shown are merely illustrative and not restrictive; those skilled in the art can interpret them based on... Figure 6 Even with various obvious changes and / or substitutions to the examples, the resulting technical solutions still fall within the scope of this disclosure.

[0160] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0161] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0162] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0163] like Figure 7As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. An input / output (I / O) interface 705 is also connected to bus 704.

[0164] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0165] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as resource allocation methods. For example, in some embodiments, the resource allocation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the resource allocation method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform a resource allocation method by any other suitable means (e.g., by means of firmware).

[0166] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0167] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0168] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0169] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0170] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0171] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0172] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0173] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A resource allocation method, comprising: Create a container set pod for the target task; Obtain the GPU resource requirements of the target task; Obtain information on available nodes and available GPU resources in the target cluster; Based on the available node information and the available GPU resource information, a first target node and a second target node that meet the GPU resource requirements are allocated to the pod. The first target node is the node where the target GPU resource allocated to the pod is located, and the second target node is the node where the pod is located. The second target node and the first target node are different nodes. The address of the target GPU resource in the first target node is sent to the second target node so that the second target node can call the target GPU resource based on the address to execute the target task running on the pod.

2. The method according to claim 1, wherein, The GPU resource requirement information includes GPU resource usage, GPU card type, and GPU card topology. The process of allocating a first target node and a second target node that meet the GPU resource requirement information to the pod based on the available node information and the available GPU resource information includes: Based on the available node information and the available GPU resource information, multiple candidate nodes that meet the GPU resource usage requirements are determined for the pod. A first set of nodes that satisfies the type of the GPU card and the topology of the GPU card is determined from the plurality of candidate nodes; The first target node and the second target node are determined based on the first set of nodes.

3. The method according to claim 2, wherein, The step of determining the first target node and the second target node based on the first node set includes: Determine the amount of free GPU resources for each GPU in each node of the first node set; Nodes whose GPU resource availability satisfies the GPU resource usage are selected from the first node set and used as the second node set; The first target node and the second target node are determined based on the second set of nodes.

4. The method according to claim 3, wherein, The step of determining the first target node and the second target node based on the second node set includes: Obtain the information of the switch corresponding to each node in the second node set; Based on the information of the switch corresponding to each node in the second node set, determine the third node set and the fourth node set corresponding to each node in the second node set; The first target node and the second target node are determined from the second set of nodes, the third set of nodes, and the fourth set of nodes.

5. The method according to claim 4, wherein, The step of determining the third and fourth node sets corresponding to each node in the second node set based on the information of the switch where each node in the second node set is located includes: For any node in the second node set, the nodes that share the same switch with that node are identified as the third node set corresponding to that node; For any node in the second node set, the nodes that correspond to different switches as that node are determined as the fourth node set corresponding to that node.

6. The method according to claim 4, wherein, Determining the first target node and the second target node from the second node set, the third node set, and the fourth node set includes: Obtain the load information corresponding to the second node set, the third node set, and the fourth node set respectively; The first target node and the second target node are determined based on the load conditions corresponding to the second node set, the third node set, and the fourth node set, respectively.

7. The method according to claim 6, wherein, The step of determining the first target node and the second target node based on the load conditions corresponding to the second node set, the third node set, and the fourth node set respectively includes: Determine the attributes of each node in the second node set, the third node set, and the fourth node set, as well as the weight value corresponding to the attribute; Based on the attributes of each node and the weight values ​​corresponding to those attributes, the total weight value of each node is determined. Determine the maximum total weight value based on the total weight value of each node; The first target node and the second target node are determined based on the maximum total weight value.

8. The method according to claim 7, wherein, Each node's attributes must include at least one of the following: The set to which each node belongs; Network throughput of the switch corresponding to each node; The usage of the switch corresponding to each node; The amount of free GPU memory for each node; The amount of idle computing power of the GPU corresponding to each node; The amount of free disk space corresponding to each node; The amount of CPU idle time for each node; The GPU priority for each node.

9. A resource allocation device, comprising: Create a module to create a set of container pods for the target task; The first acquisition module is used to acquire the graphics processing unit (GPU) resource requirements information of the target task; The second acquisition module is used to acquire information on available nodes and available GPU resources in the target cluster. The allocation module is used to allocate a first target node and a second target node to the pod based on the available node information and the available GPU resource information, which meet the GPU resource requirements. The first target node is the node where the target GPU resource allocated to the pod is located, and the second target node is the node where the pod is located. The second target node and the first target node are different nodes. The notification module is used to send the address of the target GPU resource in the first target node to the second target node, so that the second target node can call the target GPU resource based on the address to execute the target task running in the pod.

10. The apparatus according to claim 9, wherein, The GPU resource requirement information includes GPU resource usage, GPU card type, and GPU card topology. The allocation module includes: The first determining submodule is used to determine multiple candidate nodes that meet the GPU resource usage requirements for the pod based on the available node information and the available GPU resource information; The second determining submodule is used to determine a first set of nodes from the plurality of candidate nodes that satisfy the type of the GPU card and the topology of the GPU card; The third determining submodule is used to determine the first target node and the second target node based on the first node set.

11. The apparatus according to claim 10, wherein, The third determining submodule is used for: Determine the amount of free GPU resources for each GPU in each node of the first node set; Nodes whose GPU resource availability satisfies the GPU resource usage are selected from the first node set and used as the second node set; The first target node and the second target node are determined based on the second set of nodes.

12. The apparatus according to claim 11, wherein, The third determining submodule is used for: Obtain the information of the switch corresponding to each node in the second node set; Based on the information of the switch corresponding to each node in the second node set, determine the third node set and the fourth node set corresponding to each node in the second node set; The first target node and the second target node are determined from the second set of nodes, the third set of nodes, and the fourth set of nodes.

13. The apparatus according to claim 12, wherein, The third determining submodule is used for: For any node in the second node set, the nodes that share the same switch with that node are identified as the third node set corresponding to that node; For any node in the second node set, the nodes that correspond to different switches as that node are determined as the fourth node set corresponding to that node.

14. The apparatus according to claim 12, wherein, The third determining submodule is used for: Obtain the load information corresponding to the second node set, the third node set, and the fourth node set respectively; The first target node and the second target node are determined based on the load conditions corresponding to the second node set, the third node set, and the fourth node set, respectively.

15. The apparatus according to claim 14, wherein, The third determining submodule is used for: Determine the attributes of each node in the second node set, the third node set, and the fourth node set, as well as the weight value corresponding to the attribute; Based on the attributes of each node and the weight values ​​corresponding to those attributes, the total weight value of each node is determined. Determine the maximum total weight value based on the total weight value of each node; The first target node and the second target node are determined based on the maximum total weight value.

16. The apparatus according to claim 15, wherein, Each node's attributes must include at least one of the following: The set to which each node belongs; Network throughput of the switch corresponding to each node; The usage of the switch corresponding to each node; The amount of free GPU memory for each node; The amount of idle computing power of the GPU corresponding to each node; The amount of free disk space corresponding to each node; The amount of CPU idle time for each node; The GPU priority for each node.

17. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images