AI Application Scheduling Method, Device and Electronic Device Based on K8s

Abstract

An embodiment of the present invention relates to an AI application scheduling method, device, and electronic device based on K8s. The method includes: when receiving a resource scheduling request for creating an AI application, parsing the resource scheduling request to obtain resource scheduling conditions and identification information corresponding to the resource scheduling application scenario; determining a resource scheduling object according to the identification information corresponding to the resource scheduling application scenario; when the resource type to be scheduled includes heterogeneous acceleration cards, screening out nodes with heterogeneous acceleration cards from the resource scheduling objects; after reserving the resources of the nodes with heterogeneous acceleration cards according to a pre-configured reservation ratio, determining the remaining resources of each node with heterogeneous acceleration cards, where the remaining resources include the type of remaining resources and the resource quantity corresponding to each type of remaining resources; when the remaining resources of a first node meet the scheduling conditions, using the first node as the target node to complete the scheduling of the AI application.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular, to an AI application scheduling method, apparatus, and electronic device based on K8s. Background Art

[0002] With the continuous popularization of Kubernetes (abbreviated as K8s) technology, more and more manufacturers have abandoned the method of deploying applications on virtual machines or physical machines and gradually started to adopt container technology to reduce the difficulty of deploying or launching applications through containerization. The CPU and memory resources managed by the K8s cluster can meet the needs of most applications. However, in the field of artificial intelligence (AI), these resources are not enough, and most AI applications need to rely on heterogeneous acceleration cards to exert their maximum service capabilities.

[0003] However, in many cases, it may affect the scheduling and deployment of AI applications because there are not enough resources available for heterogeneous acceleration cards. Summary of the Invention

[0004] The present application provides an AI application scheduling method, apparatus, and electronic device based on K8s to solve some or all of the above problems in the prior art.

[0005] In a first aspect, the present application provides an AI application scheduling method based on K8s, and the method includes:

[0006] When receiving a resource scheduling request for creating an AI application, parse the resource scheduling request to obtain resource scheduling conditions and identification information corresponding to the resource scheduling application scenario; wherein, the resource scheduling conditions include the types of resources to be scheduled and the amount of resources to be scheduled for each type of resource, and the identification information is used to indicate the type of resource scheduling object;

[0007] Determine a resource scheduling object according to the identification information corresponding to the resource scheduling application scenario;

[0008] When the types of resources to be scheduled include heterogeneous acceleration cards, screen out the nodes with the heterogeneous acceleration cards from the resource scheduling objects;

[0009] After reserving the resources of the nodes with the heterogeneous acceleration cards according to a pre-configured reservation ratio, determine the remaining resources of each node with the heterogeneous acceleration cards, and the remaining resources include the types of the remaining resources and the amount of resources corresponding to each type of the remaining resources;

[0010] When the remaining resources of the first node meet the scheduling condition, the first node is used as the target node to complete the scheduling of the AI application. Among them, when the resource type in the remaining resources of the first node includes the to-be-scheduled resource type, and the remaining resource amount corresponding to the to-be-scheduled resource type is greater than or equal to the to-be-scheduled resource amount, it is determined that the remaining resources meet the scheduling condition. The first node is any node among the nodes with the heterogeneous acceleration card.

[0011] Optionally, when there are multiple nodes with the heterogeneous acceleration card and the remaining resources of the current node with the heterogeneous acceleration card do not meet the scheduling condition, the method further includes:

[0012] Traverse the next node with the heterogeneous acceleration card in the resource scheduling object until the target node is obtained to complete the scheduling of the AI application; or, when the remaining resources of all nodes with the heterogeneous acceleration card included in the resource scheduling object do not meet the scheduling condition, it is determined that the AI application scheduling fails.

[0013] Optionally, the resource scheduling object includes a node cluster or one or more namespaces included in the node cluster.

[0014] Optionally, when the resource scheduling object is one or more namespaces and the to-be-scheduled resource type includes a heterogeneous acceleration card, screening out candidate nodes with heterogeneous acceleration cards from the resource scheduling object includes:

[0015] Screen out candidate namespaces that meet the resource scheduling conditions from one or more namespaces;

[0016] Screen out candidate nodes from the candidate namespaces.

[0017] Optionally, screening out candidate namespaces that meet the resource scheduling conditions from one or more namespaces specifically includes:

[0018] Count the resource types in each namespace and the resource amount corresponding to each resource type;

[0019] When the resource type in the first namespace includes the to-be-scheduled resource type, and the remaining resource amount corresponding to the to-be-scheduled resource type in the first namespace is greater than or equal to the to-be-scheduled resource amount, it is determined that the first namespace meets the resource scheduling condition, and the first namespace is used as the candidate namespace. The first namespace is any namespace among one or more namespaces.

[0020] Optionally, when the to-be-scheduled resource type does not include a heterogeneous acceleration card, the method further includes:

[0021] Filter nodes that do not include heterogeneous acceleration cards and meet the scheduling conditions from the candidate namespaces as target nodes;

[0022] Or,

[0023] When there are no nodes in the candidate namespaces that do not include heterogeneous acceleration cards, or the nodes that do not include heterogeneous acceleration cards do not meet the scheduling conditions, select nodes that include heterogeneous acceleration cards and meet the scheduling conditions from the namespaces as target nodes.

[0024] Optionally, when there are multiple candidate namespaces and each of the multiple candidate namespaces includes at least one common candidate node, the method further includes:

[0025] After reserving the resources of the first common candidate node according to the greedy algorithm and the reserved ratio of the first common candidate node pre-configured for each namespace, determine the remaining resource types of the first common candidate node and the resource amount corresponding to each resource type;

[0026] When it is determined that the remaining resources of the first common candidate node meet the scheduling conditions according to the remaining resource types of the first common candidate node and the resource amount corresponding to each resource type, use the first common candidate node as the target node to complete the scheduling of the AI application, where the first common candidate node is any candidate node among at least one common candidate node.

[0027] Optionally, when there are one or more first-type candidate namespaces that include only one common candidate node and the common candidate node meets the scheduling conditions, and there is at least one second-type candidate namespace that includes, in addition to the common candidate node, candidate nodes that meet the scheduling conditions, the method further includes:

[0028] Select candidate nodes other than the common candidate node from the second-type candidate namespaces as target nodes.

[0029] In a second aspect, the present application provides an AI application scheduling device based on K8s, and the device includes:

[0030] A receiving module, configured to receive a resource scheduling request for creating an AI application;

[0031] An analysis module, configured to analyze the resource scheduling request, obtain resource scheduling conditions, and identification information corresponding to the resource scheduling application scenario; where the resource scheduling conditions include the resource types to be scheduled and the amount of resources to be scheduled corresponding to each resource type, and the identification information is used to indicate the type of resource scheduling object;

[0032] A determination module, configured to determine a resource scheduling object according to the identification information corresponding to the resource scheduling application scenario;

[0033] A screening module, configured to screen out nodes with the heterogeneous acceleration card from the resource scheduling objects when the resource type to be scheduled includes a heterogeneous acceleration card;

[0034] A processing module, configured to reserve resources of nodes with the heterogeneous acceleration card according to a preconfigured reservation ratio, and then determine the remaining resources of each node with the heterogeneous acceleration card. The remaining resources include the type of the remaining resources and the amount of resources corresponding to each type of the remaining resources. When the remaining resources of a first node meet the scheduling condition, the first node is used as a target node to complete the scheduling of the AI application. Wherein, when the resource type in the remaining resources of the first node includes the resource type to be scheduled, and the amount of the remaining resources corresponding to the resource type to be scheduled is greater than or equal to the amount of the resource to be scheduled, it is determined that the remaining resources meet the scheduling condition, and the first node is any node among the nodes with the heterogeneous acceleration card.

[0035] In a third aspect, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus;

[0036] The memory is used to store a computer program;

[0037] The processor is configured to implement the steps of the K8s-based AI application scheduling method according to any one of the embodiments of the first aspect when executing the program stored on the memory.

[0038] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the K8s-based AI application scheduling method according to any one of the embodiments of the first aspect are implemented.

[0039] The above technical solutions provided by the embodiments of the present application have the following advantages compared with the prior art:

[0040] When the method provided by the embodiment of the present application receives a resource scheduling request of an AI application, it analyzes the resource scheduling request to obtain resource scheduling conditions and identification information corresponding to the resource scheduling application scenario. Then, according to the identification information corresponding to the resource scheduling application scenario, it determines a resource scheduling object, and when it is determined that the resource type to be scheduled includes heterogeneous acceleration cards, it filters out nodes with heterogeneous acceleration cards from the resource scheduling object, and then reserves resources of the nodes with heterogeneous acceleration cards according to the reservation ratio. And it determines the remaining resources of each node with a heterogeneous acceleration card. If the remaining resources meet the scheduling conditions, this node can be used as the target node to complete the scheduling of the AI application. Throughout the process, considering the situation where resources need to be reserved when there are heterogeneous acceleration cards, so as to avoid the situation that the acceleration cards cannot be used due to the consumption of CPU and memory resources on the nodes, affecting the scheduling and deployment of the AI application, and ensuring the normal operation and deployment of the AI application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic flowchart of a method for scheduling an AI application based on K8s provided by an embodiment of the present invention;

[0042] Figure 2 It is a schematic flowchart of another method for scheduling an AI application based on K8s provided by an embodiment of the present invention;

[0043] Figure 3 It is a schematic flowchart of another method for scheduling an AI application based on K8s provided by an embodiment of the present invention;

[0044] Figure 4 It is a schematic flowchart of another method for scheduling an AI application based on K8s provided by an embodiment of the present invention;

[0045] Figure 5 It is a schematic structural diagram of a device for scheduling an AI application based on K8s provided by an embodiment of the present invention;

[0046] Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0048] To facilitate the understanding of the embodiments of the present invention, the following will further explain with specific embodiments in conjunction with the drawings. The embodiments do not constitute a limitation to the embodiments of the present invention.

[0049] In response to the technical problems mentioned in the background art, the embodiments of the present application provide an AI application scheduling method based on K8s. Specifically, refer to Figure 1 as shown in Figure 1 which is a schematic flowchart of an AI application scheduling method based on K8s provided by the embodiments of the present invention. Before executing the method steps, some preparatory work needs to be done, such as: performing an initialization operation on the server node and joining the node to the K8s cluster. The K8s cluster identifies the resource types and quantities on the node. Among them, the resources include: CPU, memory, disk, and heterogeneous acceleration cards. The identification of the acceleration card is registered to the K8s cluster through the device-plugin plugin. After performing this operation, the method steps of the embodiments of the present application can be referred to as follows.

[0050] The method steps include:

[0051] Step 110, when receiving a resource scheduling request for creating an AI application, parse the resource scheduling request to obtain the resource scheduling conditions and the identification information corresponding to the resource scheduling application scenario.

[0052] Step 120, determine the resource scheduling object according to the identification information corresponding to the resource scheduling application scenario.

[0053] Specifically, the resource scheduling conditions include the resource types to be scheduled and the amount of resources to be scheduled corresponding to each resource type. The identification information is used to indicate the type of the resource scheduling object.

[0054] In an optional example, the resource types may include one or more of CPU, memory, disk, and heterogeneous acceleration cards. The amount of resources to be scheduled may require 28 cores of CPU, 100G of memory, and so on.

[0055] Among them, heterogeneous computing is a type of distributed computing. It can use either a single independent computer that can support both the single instruction multiple data (SIMD) mode and the multiple instruction stream multiple data (MIMD) mode, or a group of independent computers interconnected by a high-speed network to complete the computing tasks. Supercomputers adopting heterogeneous computing architectures will use at least two types of processors. In the heterogeneous computing architecture, the general-purpose CPU is responsible for the scheduling of complex logic and serial tasks, and the accelerator is responsible for tasks with high parallelism to achieve computing acceleration. The acceleration card, that is, the physical processor, is a literal translation of the English term Physics Processing Unit, with the English abbreviation PPU. It is a specially designed processor product used to accelerate the execution of physical simulation algorithms. The simulation algorithms it can accelerate include rigid body dynamics, collision detection, fluid simulation, soft object and object fracture simulation. Through this hardware device, we can liberate the processors in the current computer system from the physical simulation and artificial intelligence algorithms they are not good at, allowing the CPU, GPU, and PPU to each be responsible for the parts they are best at in the game. The heterogeneous acceleration card is an acceleration card that performs heterogeneous computing.

[0056] In the resource scheduling conditions, in addition to including the types of resources to be scheduled and the amount of resources to be scheduled corresponding to each type of resource, it also includes identification information used to indicate the type of resource scheduling object. In an optional example, the type of resource scheduling object can, for example, include nodes in a K8s cluster, and another can be one or more namespaces included in the K8s cluster. Therefore, in an optional example, the resource scheduling object can, for example, include a node cluster or one or more namespaces included in the node cluster.

[0057] Step 130, when the type of resource to be scheduled includes a heterogeneous acceleration card, screen out the nodes with heterogeneous acceleration cards from the resource scheduling objects.

[0058] Step 140, after reserving the resources of the nodes with heterogeneous acceleration cards according to the pre-configured reservation ratio, determine the remaining resources of each node with a heterogeneous acceleration card.

[0059] Specifically, the heterogeneous acceleration cards in the resource types to be scheduled need to be specifically explained here. Considering that if heterogeneous acceleration card resources need to be called during AI calls, in order to ensure that the heterogeneous acceleration cards can be normally called, it is necessary to reserve the resources on the nodes, so as to prevent the resources on the nodes with acceleration cards from being unable to use the acceleration cards due to the consumption of CPU and memory resources, affecting the scheduling and deployment of AI applications. Specifically, because the scheduling of AI applications will occupy a relatively large amount of resources. This is also the reason why it is necessary to reserve the resources on the nodes with heterogeneous acceleration cards. Once an AI application scheduling is applied and the resources on the nodes with heterogeneous acceleration cards are not reserved, it is very easy for the acceleration cards to become unusable after the AI application scheduling occupies a large amount of resources, thus affecting the scheduling and deployment of AI applications.

[0060] Therefore, first, it is necessary to screen out the nodes with heterogeneous acceleration cards from the resource scheduling objects. And, after reserving the resources in the nodes with heterogeneous acceleration cards according to the pre-configured resource reservation ratio on the nodes with heterogeneous acceleration cards, determine the remaining resources of each node with heterogeneous acceleration cards. Among them, the remaining resources include the type of remaining resources and the amount of resources of each type of remaining resources.

[0061] It should be noted that other resource types are all resource types common to nodes. Therefore, instead of judging whether there are other types of resources on the nodes, it is necessary to judge whether the remaining resources of other types of resources can meet the resource scheduling conditions.

[0062] In an optional example, when determining the remaining resources of each node with heterogeneous acceleration cards, it can be achieved through the following method. Specifically, see the following:

[0063] For example, if the acceleration card is nvidia.com / gpu and the resource reservation ratio is 1:8:8. This means that one nvidia.com / gpu card reserves 8 cores of CPU and 8G of memory. Taking the CPU as an example, the calculation method of the remaining CPU resources is: Remaining CPU resources = Total CPU cores of the node - Allocated cores - Number of acceleration cards × Reserved CPU core ratio. For example: The node has a total of 64 cores of CPU, 20 cores have been allocated, and there are 2 heterogeneous acceleration cards (GPU cards) on this node, and the reservation ratio is 1:8. Then the remaining cpu resources on this node are: 64 - 20 - 2 × 8 = 28 cores.

[0064] The heterogeneous acceleration card and the resource reservation ratio can also be other examples besides the above examples. For example, for the acceleration card nvidia.com / v100-sxm2-16gb, the corresponding reservation ratio is 1:16:16. The specific method for calculating the remaining resources is similar and will not be elaborated here. Of course, besides the CPU, the above formula can also be applied to other types of resources, but adaptation adjustments need to be made according to the resource type. For example, if the node memory is 100G, the number of acceleration cards is 2, the allocated memory is 40G, and the reservation ratio is 1:8. Then, the remaining memory is 100 - 40 - 2×8 = 44G.

[0065] Step 150, when the remaining resources of the first node meet the scheduling conditions, use the first node as the target node to complete the scheduling of the AI application.

[0066] In a specific example, to determine whether the remaining resources of the first node meet the scheduling conditions, mainly check whether it conforms to the following rules:

[0067] When the resource type in the remaining resources of the first node includes the resource type to be scheduled, and the remaining resource quantity corresponding to the resource type to be scheduled is greater than or equal to the resource quantity to be scheduled, it is determined that the remaining resources meet the scheduling conditions; otherwise, it is determined that the remaining resources do not meet the scheduling conditions. Here, the first node is any node included in the candidate nodes.

[0068] Moreover, when the remaining resources of the first node meet the scheduling conditions, use the first node as the target node to complete the scheduling of the AI application.

[0069] In an optional example, assume that when there are multiple nodes with heterogeneous acceleration cards and the remaining resources of the current node with heterogeneous acceleration cards do not meet the scheduling conditions, the method may further include:

[0070] Traverse the next node with a heterogeneous acceleration card in the resource scheduling object until a target node is obtained to complete the scheduling of the AI application; or, when the remaining resources of all nodes with heterogeneous acceleration cards included in the resource scheduling object do not meet the scheduling conditions, determine that the AI application scheduling fails.

[0071] That is, if there are multiple nodes with heterogeneous acceleration cards, then continue to traverse the next node with a heterogeneous acceleration card and determine whether the remaining resources of the next node with a heterogeneous acceleration card meet the resource scheduling conditions in the manner described above. If they meet, it can also be used as the target node to complete the AI scheduling. Repeat the above operations until a target node that meets the resource scheduling conditions is obtained. Of course, if no suitable node is found among the multiple nodes with heterogeneous acceleration cards, it means that the AI application scheduling fails.

[0072] Further optionally, if the remaining resources of multiple nodes with heterogeneous acceleration cards all meet the resource scheduling conditions, a node can be randomly selected from the multiple nodes that meet the resource scheduling conditions as the target node. Alternatively, a node with remaining resources closer to the AI scheduling application can be selected as the target node. Specifically, how to select the target node from multiple nodes with heterogeneous acceleration cards and with remaining resources that all meet the resource scheduling conditions can be set according to the actual situation and will not be elaborated here.

[0073] The AI application scheduling method based on K8s provided by the embodiments of the present invention, when receiving a resource scheduling request for an AI application, parses the resource scheduling request to obtain the resource scheduling conditions and the identification information corresponding to the resource scheduling application scenario. Then, according to the identification information corresponding to the resource scheduling application scenario, determines the resource scheduling object, and when it is determined that the resource type to be scheduled includes heterogeneous acceleration cards, filters out the nodes with heterogeneous acceleration cards from the resource scheduling object, and then reserves the resources of the nodes with heterogeneous acceleration cards according to the reservation ratio. And determines the remaining resources of each node with a heterogeneous acceleration card. If the remaining resources meet the scheduling conditions, the node can be used as the target node to complete the scheduling of the AI application. During the whole process, considering the situation where resources need to be reserved when there are heterogeneous acceleration cards, so as to avoid the situation that the acceleration cards cannot be used due to the consumption of CPU and memory resources on the node, affecting the scheduling and deployment of the AI application, and ensuring the normal operation and deployment of the AI application.

[0074] It should be noted that in the above embodiments, if the resource scheduling object is a K8s cluster, then directly filter out nodes from the K8s cluster and perform the above operations. However, considering that if the resource scheduling object is one or more namespaces in the K8s cluster and the resource type to be scheduled includes heterogeneous acceleration cards, filtering out the nodes with heterogeneous acceleration cards from the resource scheduling object can specifically include the following method steps, see Figure 2 As shown, the method steps include:

[0075] Step 210, filter out candidate namespaces that meet the resource scheduling conditions from one or more namespaces.

[0076] Specifically, when the K8s cluster needs to create a namespace, the resource quantities of the entire K8s cluster, including CPU, memory, disk, and heterogeneous acceleration cards, can be aggregated in advance, and then the overall resources in the K8s cluster can be divided according to the resource requirements of each namespace. Among them, a namespace is also composed of one or more nodes in the K8s cluster. Therefore, if the namespace requests resources including acceleration card resources, resource reservation also needs to be performed on the nodes with acceleration card resources. For the specific reservation method, please refer to the above text and will not be elaborated here. After reservation, the remaining resources in each namespace also need to be counted. It should be noted that the nodes in the namespace can include nodes with acceleration card resources and nodes without acceleration card resources.

[0077] Filtering out candidate namespaces that meet the resource scheduling conditions from one or more namespaces can be achieved through the following methods. For specific details, please refer to Figure 3 As shown, the method steps include:

[0078] Step 310, count the resource types in each namespace and the corresponding resource quantities for each resource type.

[0079] Step 320, when the resource types in the first namespace include the resource types to be scheduled, and the remaining resource quantity corresponding to the resource types to be scheduled in the first namespace is greater than or equal to the resource quantity to be scheduled, determine that the first namespace meets the resource scheduling conditions and use the first namespace as a candidate namespace.

[0080] Specifically, the counting process can be identified by the K8s cluster. The remaining resources can be calculated in a similar way as above, so it will not be elaborated here. And how to determine whether the first namespace meets the resource scheduling conditions has also been described in detail above, so no further explanation will be given here.

[0081] The remaining resources in the namespace need to be greater than or equal to the resource scheduling conditions, considering that different AI applications may apply for resources without heterogeneous acceleration cards in addition to applying for resources with heterogeneous acceleration cards. To ensure flexible adaptation to different situations, the remaining resources in the namespace need to be greater than or equal to the resource scheduling conditions.

[0082] Step 220, screen out candidate nodes from the candidate namespaces.

[0083] For the specific process of selecting candidate nodes, please refer to the process of selecting candidate nodes in the K8s cluster above. The difference between the two is that when selecting candidate nodes in the K8s cluster, there is no need to deliberately distinguish which namespace it is. When selecting candidate nodes in the candidate namespaces, it is only to select candidate nodes in one candidate namespace.

[0084] Further, as introduced above, AI calls may not request heterogeneous acceleration card resources. Then, when the type of resources to be scheduled does not include heterogeneous acceleration cards, the method further includes the following method steps:

[0085] Screen nodes that do not include heterogeneous acceleration cards and meet the scheduling conditions from the candidate namespaces as target nodes. Alternatively, when there are no nodes that do not include heterogeneous acceleration cards in the candidate namespaces, or the nodes that do not include heterogeneous acceleration cards do not meet the scheduling conditions, select nodes that include heterogeneous acceleration cards and meet the scheduling conditions from the namespaces as target nodes.

[0086] For the specific selection process, refer to the above, and it will not be elaborated here.

[0087] Further optionally, the nodes included in different namespaces may be the same node, that is, the resources of the same node can be called by multiple namespaces. Then, when there are multiple candidate namespaces and all of the multiple candidate namespaces include at least one common candidate node, the method further includes the following method steps, specifically refer to Figure 4 as shown, including:

[0088] Step 410, after reserving the resources of the first common candidate node according to the greedy algorithm and the reservation ratio of the first common candidate node pre-configured for each namespace, determine the remaining resource types of the first common candidate node and the resource amount corresponding to each resource type.

[0089] Step 420, when it is determined that the remaining resources of the first common candidate node meet the scheduling conditions according to the remaining resource types of the first common candidate node and the resource amount corresponding to each resource type, use the first common candidate node as the target node to complete the scheduling of the AI application.

[0090] Specifically, the first common candidate node is any one of the at least one common candidate node.

[0091] In a specific example, for instance, calculate based on the largest reservation ratio among all namespaces. For example, 3 namespaces have all applied for nvidia.com / gpu accelerator card resources, and the reservation ratios are 1:2:2, 1:4:4, and 1:8:8 respectively; the reservation ratio of the remaining resources on this node is first calculated according to 1:8:8; if the remaining resources can meet the resource application requirements of this AI application, then this node can be determined as the target node. If the remaining resources are insufficient, continue to select other nodes according to the above logic; when all nodes in the cluster still have no available nodes according to the greedy algorithm, continue to select nodes according to the second-largest ratio, that is, when no available nodes are selected according to the reservation ratio of 1:8:8, then select nodes according to the reservation ratio of 1:4:4. Continue until a suitable target node is finally selected to complete the scheduling of the AI application. Of course, if no suitable target node can be found even according to the lowest reservation ratio, it is determined that the AI application scheduling fails.

[0092] Further optionally, when there are one or more first-type candidate namespaces that only include one common candidate node and this common candidate node meets the scheduling conditions, and there is at least one second-type candidate namespace that includes, in addition to the common candidate node, candidate nodes that meet the scheduling conditions, the method further includes:

[0093] Select the candidate nodes other than the common candidate node from the second-type candidate namespaces as the target nodes.

[0094] The execution of this method step is mainly to prevent the situation that once a namespace A has only one node and this node is shared with other namespace B. And when the AI scheduling selects the target node and selects this node, the resources of this node will be occupied, and the remaining resources do not meet the resource application situation of namespace A, which will cause the node list of namespace A to be empty and the namespace will lose its meaning of existence. If there is another node in namespace B that meets the AI scheduling application, then another node can be selected to complete this AI call.

[0095] Optionally, after each AI call is completed, it is also necessary to update the remaining resources in each namespace to facilitate the selection of suitable nodes for the next AI call.

[0096] It should also be noted that whether the resource scheduling object is a node cluster or one or more namespaces included in the node cluster, when it is determined that the AI call does not require a heterogeneous acceleration card, nodes without a heterogeneous acceleration card should be preferentially selected to execute resource calls to minimize the working cost. That is, when the resource scheduling object is a node cluster, if a node does not have heterogeneous acceleration card resources, the node is directly used as an alternative node and added to the list of nodes that can be called by the AI. Only when a node has a heterogeneous acceleration card, will the resource reservation be completed according to the scheme mentioned above, and then it will be further determined whether the remaining resources meet the resource scheduling conditions, and if it is determined that the resource scheduling conditions are met, the node will be added to the list of nodes that can be called by the AI. However, if the resource types required for the AI call include a heterogeneous acceleration card, the target node can only be selected from the nodes including the heterogeneous acceleration card. Nodes without a heterogeneous acceleration card can only be used as target nodes when the AI call does not require a heterogeneous acceleration card.

[0097] The above are several method embodiments of the AI application scheduling based on K8s provided by this application. The following will introduce other embodiments of the AI application scheduling based on K8s provided by this application. For details, see the following.

[0098] Figure 5 An AI application scheduling device based on K8s provided by an embodiment of the present invention includes: a receiving module 501, an analysis module 502, a determination module 503, a screening module 504, and a processing module 505.

[0099] Among them, the receiving module 501 is configured to receive a resource scheduling request for creating an AI application;

[0100] The analysis module 502 is configured to analyze the resource scheduling request to obtain resource scheduling conditions and identification information corresponding to the resource scheduling application scenario; where the resource scheduling conditions include the resource types to be scheduled and the amount of resources to be scheduled corresponding to each resource type, and the identification information is used to indicate the type of the resource scheduling object;

[0101] The determination module 503 is configured to determine a resource scheduling object according to the identification information corresponding to the resource scheduling application scenario;

[0102] The screening module 504 is configured to screen out nodes with the heterogeneous acceleration card from the resource scheduling objects when the resource types to be scheduled include a heterogeneous acceleration card;

[0103] The processing module 505 is configured to reserve resources of a node with the heterogeneous acceleration card according to a pre-configured reservation ratio, and then determine the remaining resources of each node with the heterogeneous acceleration card. The remaining resources include the type of the remaining resources and the amount of resources corresponding to each type of the remaining resources. When the remaining resources of a first node meet the scheduling conditions, the first node is used as the target node to complete the scheduling of the AI application. Among them, when the resource type in the remaining resources of the first node includes the to-be-scheduled resource type and the amount of the remaining resources corresponding to the to-be-scheduled resource type is greater than or equal to the to-be-scheduled resource amount, it is determined that the remaining resources meet the scheduling conditions, and the first node is any node among the nodes with the heterogeneous acceleration card.

[0104] Optionally, the processing module 505 is further configured to, when there are multiple nodes with the heterogeneous acceleration card and the remaining resources of the current node with the heterogeneous acceleration card do not meet the scheduling conditions, traverse the next node with the heterogeneous acceleration card in the resource scheduling object until the target node is obtained to complete the scheduling of the AI application; or, when the remaining resources of all nodes with the heterogeneous acceleration card included in the resource scheduling object do not meet the scheduling conditions, it is determined that the AI application scheduling fails.

[0105] Optionally, the resource scheduling object includes a node cluster or one or more namespaces included in the node cluster.

[0106] Optionally, when the resource scheduling object is one or more namespaces and the to-be-scheduled resource type includes a heterogeneous acceleration card, the screening module 504 is further configured to screen out candidate namespaces that meet the resource scheduling conditions from the one or more namespaces; and screen out candidate nodes from the candidate namespaces.

[0107] Optionally, the screening module 504 is specifically configured to count the resource types in each namespace and the amount of resources corresponding to each resource type;

[0108] When the resource type in the first namespace includes the to-be-scheduled resource type and the amount of the remaining resources corresponding to the to-be-scheduled resource type in the first namespace is greater than or equal to the to-be-scheduled resource amount, it is determined that the first namespace meets the resource scheduling conditions, and the first namespace is used as the candidate namespace, where the first namespace is any namespace among the one or more namespaces.

[0109] Optionally, when the to-be-scheduled resource type does not include a heterogeneous acceleration card, the screening module 504 is further configured to screen out nodes that do not include a heterogeneous acceleration card and meet the scheduling conditions from the candidate namespaces as the target nodes;

[0110] Or,

[0111] When there is no node without a heterogeneous acceleration card in the candidate namespace, or the nodes without a heterogeneous acceleration card do not meet the scheduling conditions, select a node that includes a heterogeneous acceleration card and meets the scheduling conditions from the namespace as the target node.

[0112] Optionally, when there are multiple candidate namespaces and each of the multiple candidate namespaces includes at least one common candidate node, the processing module 505 is further configured to reserve the resources of the first common candidate node according to the greedy algorithm and the reserved ratio of the first common candidate node pre-configured for each namespace, and then determine the remaining resource types of the first common candidate node and the amount of resources corresponding to each resource type;

[0113] When it is determined that the remaining resources of the first common candidate node meet the scheduling conditions according to the remaining resource types of the first common candidate node and the amount of resources corresponding to each resource type, use the first common candidate node as the target node to complete the scheduling of the AI application, where the first common candidate node is any one of the at least one common candidate nodes.

[0114] Optionally, when there are one or more first-class candidate namespaces that include only one common candidate node and the common candidate node meets the scheduling conditions, and there is at least one second-class candidate namespace that includes candidate nodes that meet the scheduling conditions in addition to the common candidate node, the processing module 505 is further configured to select candidate nodes other than the common candidate node from the second-class candidate namespace as the target nodes.

[0115] The functions performed by each component in the AI application scheduling device based on K8s provided by the embodiments of the present invention have been described in detail in any of the above method embodiments, so they will not be repeated here.

[0116] An AI application scheduling device based on K8s provided by an embodiment of the present invention, when receiving a resource scheduling request for an AI application, analyzes the resource scheduling request to obtain resource scheduling conditions and identification information corresponding to the resource scheduling application scenario. Then, according to the identification information corresponding to the resource scheduling application scenario, determines the resource scheduling object, and when it is determined that the resource type to be scheduled includes a heterogeneous acceleration card, filters out nodes with a heterogeneous acceleration card from the resource scheduling object, and then reserves the resources of the nodes with a heterogeneous acceleration card according to the reserved ratio. And determine the remaining resources of each node with a heterogeneous acceleration card. If the remaining resources meet the scheduling conditions, the node can be used as the target node to complete the scheduling of the AI application. During the whole process, considering the situation that resources need to be reserved when there is a heterogeneous acceleration card, so as to avoid the situation that the acceleration card cannot be used due to the consumption of CPU and memory resources on the node, affecting the scheduling and deployment of the AI application, and ensuring the normal operation and deployment of the AI application.

[0117] As shown Figure 6 In the embodiment of the present application, an electronic device is provided, including a processor 111, a communication interface 112, a memory 113, and a communication bus 114. Among them, the processor 111, the communication interface 112, and the memory 113 complete mutual communication through the communication bus 114.

[0118] The memory 113 is used to store computer programs;

[0119] In an embodiment of the present application, when the processor 111 executes the program stored on the memory 113, it implements the AI application scheduling method based on K8s provided by any one of the foregoing method embodiments, including:

[0120] When receiving a resource scheduling request for creating an AI application, parsing the resource scheduling request to obtain resource scheduling conditions and identification information corresponding to the resource scheduling application scenario; wherein, the resource scheduling conditions include the types of resources to be scheduled and the amount of resources to be scheduled corresponding to each type of resource, and the identification information is used to indicate the type of resource scheduling object;

[0121] Determine a resource scheduling object according to the identification information corresponding to the resource scheduling application scenario;

[0122] When the types of resources to be scheduled include heterogeneous acceleration cards, filter out the nodes with the heterogeneous acceleration cards from the resource scheduling objects;

[0123] After reserving the resources of the nodes with the heterogeneous acceleration cards according to a pre-configured reservation ratio, determine the remaining resources of each node with the heterogeneous acceleration cards. The remaining resources include the types of the remaining resources and the amount of resources corresponding to each type of the remaining resources;

[0124] When the remaining resources of a first node meet the scheduling conditions, use the first node as the target node to complete the scheduling of the AI application. Among them, when the types of resources in the remaining resources of the first node include the types of resources to be scheduled and the amount of remaining resources corresponding to the types of resources to be scheduled is greater than or equal to the amount of resources to be scheduled, it is determined that the remaining resources meet the scheduling conditions, and the first node is any one of the nodes with the heterogeneous acceleration cards.

[0125] Optionally, when there are multiple nodes with the heterogeneous acceleration cards and the remaining resources of the current node with the heterogeneous acceleration cards do not meet the scheduling conditions, the method further includes:

[0126] Traverse the next node with the heterogeneous acceleration card in the resource scheduling object until the target node is obtained, and complete the scheduling of the AI application; or, when the remaining resources of all nodes with the heterogeneous acceleration card included in the resource scheduling object do not meet the scheduling conditions, determine that the AI application scheduling fails.

[0127] Optionally, the resource scheduling object includes a node cluster or one or more namespaces included in the node cluster.

[0128] Optionally, when the resource scheduling object is one or more namespaces and the resource type to be scheduled includes a heterogeneous acceleration card, screening out candidate nodes with heterogeneous acceleration cards from the resource scheduling object includes:

[0129] Screen out candidate namespaces that meet the resource scheduling conditions from one or more namespaces;

[0130] Screen out candidate nodes from the candidate namespaces.

[0131] Optionally, screening out candidate namespaces that meet the resource scheduling conditions from one or more namespaces specifically includes:

[0132] Count the resource types in each namespace and the resource quantity corresponding to each resource type;

[0133] When the resource type in the first namespace includes the resource type to be scheduled and the remaining resource quantity corresponding to the resource type to be scheduled in the first namespace is greater than or equal to the resource quantity to be scheduled, determine that the first namespace meets the resource scheduling conditions and use the first namespace as a candidate namespace, where the first namespace is any one of the one or more namespaces.

[0134] Optionally, when the resource type to be scheduled does not include a heterogeneous acceleration card, the method further includes:

[0135] Screen out nodes that do not include a heterogeneous acceleration card and meet the scheduling conditions from the candidate namespaces as the target nodes;

[0136] Or,

[0137] When there are no nodes that do not include a heterogeneous acceleration card in the candidate namespaces, or the nodes that do not include a heterogeneous acceleration card do not meet the scheduling conditions, select nodes that include a heterogeneous acceleration card and meet the scheduling conditions from the namespaces as the target nodes.

[0138] Optionally, when there are multiple candidate namespaces and all of the multiple candidate namespaces include at least one common candidate node, the method further includes:

[0139] After reserving the resources of the first common candidate node according to the greedy algorithm and the reservation ratio of the first common candidate node pre-configured for each namespace, determine the remaining resource types of the first common candidate node and the resource quantity corresponding to each resource type;

[0140] When it is determined that the remaining resources of the first common candidate node meet the scheduling conditions according to the remaining resource types of the first common candidate node and the resource quantity corresponding to each resource type, use the first common candidate node as the target node to complete the scheduling of the AI application, where the first common candidate node is any candidate node among at least one common candidate node.

[0141] Optionally, when there are one or more first-type candidate namespaces that only include one common candidate node and the common candidate node meets the scheduling conditions, and there is at least one second-type candidate namespace that includes candidate nodes that meet the scheduling conditions in addition to the common candidate node, the method further includes:

[0142] Select the candidate nodes other than the common candidate node from the second-type candidate namespaces as the target nodes.

[0143] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the AI application scheduling method based on K8s provided in any of the foregoing method embodiments are implemented.

[0144] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0145] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A K8s-based AI application scheduling method, characterized in that: The method comprises: Upon receiving a resource scheduling request for creating an AI application, the system parses the resource scheduling request to obtain resource scheduling conditions and identification information corresponding to the resource scheduling application scenario; wherein the resource scheduling conditions include the type of resources to be scheduled and the amount of resources to be scheduled corresponding to each resource type, and the identification information indicates the type of resource scheduling object; Determining a resource scheduling object according to identification information corresponding to the resource scheduling application scenario; When the resource type to be scheduled includes a heterogeneous accelerator card, filtering out nodes having the heterogeneous accelerator card from the resource scheduling object; After reserving resources of the nodes having the heterogeneous accelerator card according to a preconfigured reservation ratio, determining remaining resources of each node having the heterogeneous accelerator card, the remaining resources including a type of the remaining resources and a resource amount corresponding to each type of the remaining resources; When there is a first node with remaining resources that meets the scheduling condition, the first node is used as the target node to complete the scheduling of the AI application, wherein when the resource type in the remaining resources of the first node includes the type of resources to be scheduled, and the amount of remaining resources corresponding to the type of resources to be scheduled is greater than or equal to the amount of resources to be scheduled, it is determined that the remaining resources meet the scheduling condition, and the first node is any node with the heterogeneous accelerator card.

2. The method according to claim 1, characterized in that When there are multiple nodes having the heterogeneous accelerator card, and remaining resources of the current node having the heterogeneous accelerator card do not meet the scheduling condition, the method further includes: The next node with the heterogeneous accelerator card in the resource scheduling object is traversed until the target node is obtained, thereby completing the scheduling of the AI application. Alternatively, when the remaining resources of all nodes with the heterogeneous accelerator card included in the resource scheduling object do not meet the scheduling condition, it is determined that the AI application scheduling has failed.

3. The method according to claim 1, characterized in that The resource scheduling object includes a node cluster or one or more namespaces included in the node cluster.

4. The method according to claim 3, characterized in that When the resource scheduling object is one or more namespaces and the resource type to be scheduled includes the heterogeneous accelerator card, screening out a candidate node having the heterogeneous accelerator card from the resource scheduling object includes: Filtering candidate namespaces that meet the resource scheduling conditions from one or more namespaces; The candidate node is filtered out from the candidate namespace.

5. The method according to claim 4, characterized in that The step of selecting a candidate namespace that meets the resource scheduling condition from one or more namespaces specifically includes: Count the resource types in each namespace and the resource quantity corresponding to each resource type; When the resource types in the first namespace include the resource type to be scheduled, and the remaining resource amount corresponding to the resource type to be scheduled in the first namespace is greater than or equal to the resource amount to be scheduled, it is determined that the first namespace meets the resource scheduling condition, and the first namespace is used as the candidate namespace, wherein the first namespace is any one of the one or more namespaces.

6. The method according to claim 4 or 5, characterized in that When the type of resources to be scheduled does not include the heterogeneous accelerator card, the method further includes: Filtering a node from the candidate namespace that does not include the heterogeneous accelerator card and meets the scheduling condition as the target node; or, When there is no node in the candidate namespace that does not include the heterogeneous accelerator card, or the node that does not include the heterogeneous accelerator card does not meet the scheduling condition, a node that includes the heterogeneous accelerator card and meets the scheduling condition is selected from the namespace as the target node.

7. The method according to claim 4 or 5, characterized in that When the candidate namespaces include multiple ones, and each of the multiple candidate namespaces includes at least one common candidate node, the method further includes: After reserving resources of the first public candidate node according to a greedy algorithm and a reservation ratio of the first public candidate node preconfigured for each namespace, determine the remaining resource types of the first public candidate node and the resource amounts corresponding to each resource type; When it is determined that the remaining resources of the first public candidate node meet the scheduling conditions based on the remaining resource types of the first public candidate node and the resource quantities corresponding to each resource type, the first public candidate node is used as the target node to complete the scheduling of the AI application, wherein the first public candidate node is any one candidate node among at least one public candidate node.

8. The method according to claim 7, characterized in that When there are one or more first-category candidate namespaces that include only one public candidate node and the public candidate node satisfies the scheduling condition, and there is at least one second-category candidate namespace that includes candidate nodes that meet the scheduling condition in addition to the public candidate node, the method further includes: A candidate node other than the public candidate node is selected from the second type of candidate namespace as the target node.

9. A K8s-based AI application scheduling device, characterized in that: The device comprises: A receiving module, used to receive resource scheduling requests for creating AI applications; a parsing module, configured to parse the resource scheduling request and obtain resource scheduling conditions and identification information corresponding to the resource scheduling application scenario; wherein the resource scheduling conditions include the type of resources to be scheduled and the amount of resources to be scheduled corresponding to each resource type, and the identification information is used to indicate the type of resource scheduling object; A determination module, configured to determine a resource scheduling object according to identification information corresponding to the resource scheduling application scenario; a screening module, configured to, when the resource type to be scheduled includes a heterogeneous accelerator card, screen out nodes having the heterogeneous accelerator card from the resource scheduling object; A processing module is used to reserve resources for the nodes with the heterogeneous accelerator card according to a preconfigured reservation ratio, and then determine the remaining resources of each node with the heterogeneous accelerator card, where the remaining resources include the type of the remaining resources and the resource amount corresponding to each remaining resource type; when there is a first node whose remaining resources meet the scheduling condition, use the first node as the target node to complete the scheduling of the AI application, wherein, when the resource type in the remaining resources of the first node includes the resource type to be scheduled, and the remaining resource amount corresponding to the resource type to be scheduled is greater than or equal to the resource amount to be scheduled, it is determined that the remaining resources meet the scheduling condition, and the first node is any node among the nodes with the heterogeneous accelerator card.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; The processor is used to implement the steps of the K8s-based AI application scheduling method described in any one of claims 1-8 when executing the program stored in the memory.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the K8s-based AI application scheduling method are implemented.

Images