Node scheduling method and apparatus for container, and device, storage medium and program product

Abstract

Disclosed in the present embodiment are a node scheduling method and apparatus for a container, and a device, a storage medium and a program product. The method comprises: determining pre-selected nodes from among nodes at least on the basis of information of resources in each node and attributes required to be satisfied by resources requested during the creation of a Pod; sending identifiers of the pre-selected nodes to a controller, such that the controller determines a first fitness score between each of the pre-selected nodes and a scheduling node for the Pod; and receiving the first fitness scores sent by the controller, and on the basis of a second fitness score between each of the pre-selected nodes and the scheduling node for the Pod which is determined by a scheduler, and the first fitness scores, selecting from among the pre-selected nodes the scheduling node for the Pod.

Description

Container node scheduling methods, apparatus, devices, storage media, and program products

[0001] Cross-reference to related applications

[0002] This application is based on and claims priority to Chinese Patent Application No. 202510096662.1, filed on January 21, 2025, entitled “Node Scheduling Method, Apparatus, Device, Storage Medium and Program Product for Containers”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of cloud computing technology, and includes, but is not limited to, a method, apparatus, device, storage medium, and program product for scheduling nodes in a container. Background Technology

[0004] In related technologies, the Dynamic Resource Allocation (DRA) mechanism provided by Kubernetes can be used to achieve node scheduling of container groups (Pods). For example, Pod node scheduling can be completed through the negotiation process between the Kubernetes cluster scheduler (Kube-scheduler) and the Resource Driver Controller. However, during the negotiation process between Kube-scheduler and Resource Driver Controller, Kube-scheduler can only determine the Pod node scheduling scheme by guessing multiple times, which reduces scheduling efficiency. Summary of the Invention

[0005] This application provides a method, apparatus, device, storage medium, and program product for scheduling nodes in containers.

[0006] This application provides a method for scheduling container nodes, applied in a Kubernetes cluster scheduler, the method comprising:

[0007] Based at least on the information of resources in each node and the attributes that the resources requested when creating a Pod need to satisfy, the pre-selected nodes are determined from the nodes.

[0008] The identifiers of the pre-selected nodes are sent to the controller, so that the controller determines the first fit score between each of the pre-selected nodes and the scheduling node of the Pod;

[0009] The system receives a first fit score from the controller, and selects a scheduling node for the Pod from the pre-selected nodes based on a second fit score between each node in the pre-selected nodes and the scheduling node of the Pod, and the first fit score.

[0010] In some embodiments, determining the pre-selected nodes among the nodes based at least on the information of resources in each node and the attributes that the resources requested when creating a Pod need to satisfy includes: obtaining preset filtering rules for each type of resource; and determining the pre-selected nodes among the nodes based on the filtering rules for each type of resource, the information of resources in each node, and the attributes that the resources requested when creating a Pod need to satisfy.

[0011] In some embodiments, the filtering rules for each resource are judgment statements organized using logical operations.

[0012] In some embodiments, before determining the pre-selected nodes among the nodes, based at least on information about resources in each node and the attributes that the resources requested when creating the Pod need to satisfy, the method further includes: when obtaining the scheduling request of the Pod, creating a resource declaration, and determining at least two types of resources requested when creating the Pod based on the resource declaration.

[0013] This application also provides another method for scheduling nodes in a container, applied in a controller, the method comprising:

[0014] The system receives the identifiers of pre-selected nodes sent by the scheduler of the Kubernetes cluster. The scheduler determines the pre-selected nodes from among the nodes based at least on information about the resources in each node and the attributes that the resources requested when creating a Pod need to satisfy.

[0015] Determine the first fit score between each of the pre-selected nodes and the scheduling node of the Pod;

[0016] The first fit score is sent to the scheduler, so that the scheduler selects the scheduling node for the Pod from the pre-selected nodes based on the second fit score between each node in the pre-selected nodes and the scheduling node of the Pod, and the first fit score.

[0017] In some embodiments, determining the first compatibility score between each of the pre-selected nodes and the scheduling node of the Pod includes: determining a resource allocation scheme for each node based at least on the topological relationship between different types of resources in each node; and determining the first compatibility score between each node and the scheduling node of the Pod based on the resource allocation scheme of each node.

[0018] In some embodiments, determining a first compatibility score between each node and the scheduling node of the Pod based on the resource allocation scheme of each node includes: determining resource attribute data corresponding to the resource allocation scheme of each node, wherein the resource attribute data includes at least one of the following: the failure rate of each resource in each node, the minimum bandwidth between different resources in each node, and the matching degree between each resource in each node and the task characteristics of the Pod;

[0019] Based on the resource attribute data corresponding to the resource allocation scheme of each node, a first compatibility score is determined between each node and the scheduling node of the Pod.

[0020] This application also provides a container node scheduling device, applied in a Kubernetes cluster scheduler, the device comprising:

[0021] The first processing module is configured to determine the pre-selected nodes from the nodes based at least on the information of resources in each node and the attributes that the resources requested when creating the Pod need to satisfy.

[0022] The second processing module is configured to send the identifiers of the pre-selected nodes to the controller, so that the controller determines a first fit score between each of the pre-selected nodes and the scheduling node of the Pod; receive the first fit score sent by the controller, and select the scheduling node of the Pod from the pre-selected nodes based on the second fit score between each of the pre-selected nodes and the scheduling node of the Pod determined by the scheduler, and the first fit score.

[0023] This application embodiment also provides another node scheduling device for containers, applied in a controller, the device comprising:

[0024] The receiving module is configured to receive the identifiers of pre-selected nodes sent by the scheduler of the Kubernetes cluster. The scheduler is used to determine the pre-selected nodes from among the nodes based at least on the information of resources in each node and the attributes that the resources requested when creating a Pod need to satisfy.

[0025] The third processing module is configured to determine a first fit score between each node in the pre-selected nodes and the scheduling node of the Pod; send the first fit score to the scheduler, so that the scheduler selects the scheduling node of the Pod from the pre-selected nodes based on the second fit score between each node in the pre-selected nodes and the scheduling node of the Pod, and the first fit score.

[0026] This application also provides an electronic device, which includes a processor and a memory for storing a computer program that can run on the processor; wherein the processor is used to run the computer program to execute any of the above-described container node scheduling methods.

[0027] This application also provides a computer storage medium storing a computer program that, when executed by a processor, implements any of the above-described container node scheduling methods.

[0028] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described container node scheduling methods.

[0029] As can be seen, in this embodiment of the application, the Kubernetes cluster scheduler does not need to make multiple blind guesses to determine the nodes that the controller will not reject. The controller only scores the pre-selected nodes for the Kubernetes cluster scheduler to refer to. Finally, the Kubernetes cluster scheduler can determine the scheduling node of the Pod in one go based on the first fit score and the second fit score, which improves the scheduling efficiency. Attached Figure Description

[0030] Figure 1 is a schematic diagram of the resource management and allocation implementation of Kubernetes provided by related technologies;

[0031] Figure 2 is a schematic diagram of resource allocation to a container provided by related technologies;

[0032] Figure 3 is another schematic diagram of resource allocation to a container provided by related technologies;

[0033] Figure 4 is another schematic diagram of resource allocation to a container provided by related technologies;

[0034] Figure 5 is a flowchart of a node scheduling method for containers applied to a Kubernetes cluster scheduler according to an embodiment of this application;

[0035] Figure 6 is a schematic diagram of selecting scheduling nodes through pre-selection rules and preferred rules according to an embodiment of this application;

[0036] Figure 7 is a flowchart of a node scheduling method for a container applied to a controller according to an embodiment of this application;

[0037] Figure 8 is a schematic diagram of the structure of a node scheduling device for a Kubernetes cluster applied to a scheduler according to an embodiment of this application;

[0038] Figure 9 is a schematic diagram of the structure of a node scheduling device for a container applied to a controller according to an embodiment of this application;

[0039] Figure 10 is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application.

[0040] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0041] With the development of large-scale models and cloud-native technologies, a common technical solution for training and inference of large-scale models is to use Kubernetes for container orchestration. This involves allocating intelligent computing resources such as Graphics Processing Units (GPUs), Remote Direct Memory Access (RDMA) devices, and Neural Processing Units (NPUs) to containers for training and inference. For example, common tools like kubeflow and argo are based on this technical solution. Training models using containers generally requires distributed training with multiple instances to fully utilize the GPU resources of the entire cluster. For instance, OpenAI uses a Kubernetes cluster with tens of thousands of nodes to train its Large Language Model (LLM).

[0042] In distributed training, the topology of resources allocated to each subtask affects the overall speedup. This is because data transfer between tasks is necessary, and bandwidth becomes a bottleneck for training efficiency. Taking NVIDIA's Collective Communication Library (NCCL) as an example, when planning transmission paths, NCCL prioritizes GPUs connected via NVLink, offering higher bandwidth compared to GPUs connected via Peripheral Component Interconnect Express (PCIe). Additionally, NVIDIA's Direct RDMA technology enables fast memory swapping via RDMA, but requires the RDMA device to be connected to the GPU solely via PCIe to ensure sufficient bandwidth. Therefore, when using containers for distributed training, allocating resources such as GPUs, RDMA devices, and NPUs is only the foundation. To achieve higher speedup, the topology between resources must be considered during allocation. For example, when allocating multiple GPUs, prioritize GPUs connected via NVLink or NVSwitch; when allocating RDMA network cards, prioritize RDMA network cards connected to the GPU solely via PCIe.

[0043] In related technologies, Kubernetes typically manages and allocates GPU and RDMA resources using a device plugin mechanism. Referring to Figure 1, the device plugin reports device information to the kubelet, including the device name and its Non-Uniform Memory Access (NUMA) architecture. The kubelet then reports this device information to the Kubernetes cluster's process service (Kube-apiserver). The scheduler, based on the device information in the kube-apiserver, schedules the device to the corresponding node. The kubelet on the node then allocates the device. After allocating the device, the kubelet sends an allocation request for the specified device to the device plugin. Based on the information returned by the device plugin, the kubelet invokes the Container Runtime Interface (CRI), which starts and configures the container.

[0044] Device plugins manage and allocate devices and are decoupled from Kubernetes, facilitating expansion and development. Common GPU and RDMA device plugins include the Kubernetes Device Plugin provided by NVIDIA for managing NVIDIA GPUs and the RDMA Device Plugin for managing Mellanox RDMA.

[0045] In related technologies, the Container-Device-Interface (CDI) mechanism can also be used to allocate resources to containers. Referring to Figure 2, Kubelet can call CRI to create a container; the CRI component reads the CDI json file on the specified path based on the information passed by Kubelet; and the CRI calls runc to create the container based on the information in the CDI json file.

[0046] Compared to device plugins, CDI also supports the configuration of network devices, storage devices, and more, making it suitable for a wider range of applications. A common tool for allocating GPUs using CDI is the NVIDIA GPU container runtime, nvidia-container-toolkit. Currently, with Containerd and Device Plugin also supporting CDI, the CDI mechanism is gradually aligning with the Device Plugin mechanism.

[0047] Kubernetes' newly provided Dynamic Resource Allocation (DRA) mechanism can also manage and allocate devices such as GPUs and RDMA. Referring to Figures 3 and 4, when a Pod is created, a Pod scheduling request is generated, and a corresponding ResourceClaim is created. The ResourceClaim represents the resources required by the Pod. kube-scheduler can negotiate with the resource-driven controller to schedule Pod nodes.

[0048] For example, the process of scheduling nodes for a Pod may include the following steps:

[0049] Step S1: kube-scheduler selects the optimal node based on its built-in scheduling rules and creates a PodScheduling resource, writing the selected optimal node to the SelectedNode of the PodScheduling resource. kube-scheduler's scheduling rules include pre-selection rules and optimization rules. Pre-selection rules filter all nodes, selecting those that meet hard criteria; for example, pre-selection rules can filter nodes with sufficient CPU resources to run Pods. Optimization rules score and rank the pre-selected nodes; for example, nodes can be scored based on whether the required image for a Pod already exists on the node. After scoring and ranking the nodes, the node with the highest score is selected.

[0050] Step S2: The resource-driven controller reads the SelectedNode of the PodScheduling resource and queries the ResourceClaim corresponding to the Pod.

[0051] Step S3: If the SelectedNode does not meet the resource requirements (e.g., the SelectedNode device resources are insufficient or the device model is not the required model), the resource-driven controller informs the kube-scheduler through the DeallocationRequested message of the PodScheduling resource that the node needs to be reselected, and writes the nodes that do not meet the resource requirements into the UnsuitableNodes field of the PodScheduling resource. Here, the resource requirements can be determined based on the content in the resource declaration.

[0052] Step S4: If the SelectedNode meets the resource requirements, the resource-driven controller informs the kube-scheduler through the Allocation message of PodScheduling.

[0053] Step S5: When kube-scheduler determines that the SelectedNode does not meet the resource requirements, it returns to step S1 and considers the UnsuitableNodes field of the PodScheduling resource when reselecting a node.

[0054] Step S6: When kube-scheduler determines that the SelectedNode meets the resource requirements, it re-executes the scheduling rules for the SelectedNode to ensure that the SelectedNode still meets the resource requirements. (This step prevents changes in the SelectedNode's status during steps S2 and S4, such as the CPU being occupied by other nodes, causing the SelectedNode to no longer meet the resource requirements.) If the SelectedNode still meets the resource requirements, proceed to step S7; otherwise, return to step S1, where kube-scheduler selects a new node to continue negotiating with the resource-driven controller.

[0055] Step S7: kube-scheduler binds the SelectedNode to the Pod, completing the node scheduling of the Pod.

[0056] Referring to Figure 4, after the node scheduling of the Pod is completed, the kubelet of the corresponding node calls the Resource Driver Plugin to allocate the device and create the container.

[0057] Compared to Device Plugins, CDI, and Resource-Driven Controllers (DRCs), the DRA scheduling scheme participates in cluster scheduling. Therefore, it can rationally schedule containers based on device information. In contrast, when using Device Plugins or CDIs for container scheduling, kube-scheduler only considers whether the number of devices is sufficient. Furthermore, in scheduling schemes using Device Plugins and CDIs, the specific device allocation is primarily determined by the kubelet, while in the DRA scheduling scheme, the allocation logic is delegated to the resource-driven plugin, improving allocation flexibility.

[0058] In related technologies, the nvidia-container-toolkit based on the CDI mechanism is merely a device configuration tool for containers, lacking device management capabilities and having limited use cases. While the k8s Device Plugin and RDMA Device Plugin based on the Device Plugin mechanism can provide device configuration and management and are currently widely used solutions, they do not involve the scheduling layer. Furthermore, the specific device allocation is determined solely by the kubelet, which can only consider NUMA topology and cannot further consider NVLink, PCIe, or other topology information. Therefore, it cannot prioritize allocating NVLink-connected GPUs to containers for higher training efficiency, nor can it allocate GPUs and RDMA connected only via PCIe to containers to achieve GPU Direct RDMA functionality.

[0059] The k8s-dra-driver based on the DRA mechanism can participate in container scheduling decisions, scheduling containers to nodes with better topologies (for example, if a container requests two GPUs, and there are two remaining GPU resources, a node with two GPUs connected via NVlink is better than a node with two GPUs connected via PCIe). Furthermore, it can customize resource allocation logic with the resource driver plugin, and can prioritize the allocation of a group of GPUs with higher bandwidth on the local node. However, two problems still exist: First, managing only a single resource cannot meet the needs of scenarios requiring coordination of multiple resources. For example, in the GPU Direct RDMA scenario, the topology affinity between the GPU and the RDMA device needs to be considered. This requires the DRA plugin managing GPU resources to negotiate with the DRA plugin managing RDMA resources. The final resource allocation made by both must meet the topology affinity, but the DRA mechanism does not have a collaborative mechanism between one DRA plugin and another. Second, the efficiency is too low. During the negotiation process between kube-scheduler and the resource-driven controller, kube-scheduler can only guess. In the optimal case, kube-scheduler needs to calculate the scheduling rules twice. In non-optimal cases, kube-scheduler needs to make more attempts. For example, when the SelectedNode does not meet the resource requirements, or when the SelectedNode meets the resource requirements but the node state changes, the kube-scheduler's scheduling rules will no longer be met in the second scheduling.

[0060] To address the technical problems existing in related technologies, this application proposes a technical solution that solves the problem of low scheduling efficiency of DRA while retaining the advantages of DRA's ability to participate in scheduling and customize resource allocation logic. By unifying the management and topology computation of various intelligent computing resources, affinity between different intelligent computing resources is achieved.

[0061] The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the embodiments provided herein are merely illustrative of the embodiments of this application and are not intended to limit the embodiments of this application. Furthermore, the embodiments provided below are some embodiments for implementing this application, and not all embodiments for implementing this application. Unless otherwise specified, the technical solutions described in the embodiments of this application can be implemented in any combination.

[0062] It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other related elements (e.g., steps in the method or units in the apparatus, such as portions of circuitry, processors, programs, or software, etc.) in the method or apparatus that includes that element.

[0063] The container node scheduling method provided in this application includes a series of steps, but the container node scheduling method provided in this application is not limited to the steps described. Similarly, the container node scheduling device provided in this application includes a series of modules, but the device provided in this application is not limited to the modules explicitly described, and may also include modules that need to be set for obtaining relevant information or processing based on information.

[0064] Figure 5 is a flowchart of a node scheduling method for containers applied to a Kubernetes cluster scheduler according to an embodiment of this application. As shown in Figure 5, the process includes:

[0065] Step 501: Based at least on the information of resources in each node and the attributes that the resources requested when creating the container group Pod need to satisfy, determine the pre-selected nodes in each node.

[0066] In this embodiment of the application, each node may include at least one hardware resource. For example, the resources of each node include, but are not limited to, GPU, CPU, RDMA device, NPU, etc.

[0067] For example, after the resource-driven plugin on each node starts, it can scan the intelligent computing devices on that node to obtain device information, which is the information about the resources in the node. After obtaining the device information, it can be stored as ResourcePerNode objects in kube-apiserver. The information stored in ResourcePerNode includes, but is not limited to, device type, device model, device bandwidth, device characteristics, local device topology, and whether the device instance has been allocated. The Kubernetes cluster scheduler can obtain the resource information of each node from kube-apiserver.

[0068] For example, the resource-driven plugin can also query the monitoring system in the Kubernetes cluster to obtain fault information of each device on the local node, calculate the failure rate of each device on the local machine, and record it in the ResourcePerNode object.

[0069] In some embodiments, the attributes that the resources requested when creating a container group Pod need to meet include, but are not limited to: the number of GPUs required, the GPU model required, the GPU memory size required, the GPU features required, the number of RDMA devices required, the RDMA device model required, the RDMA device features required, the number of NPUs required, the NPU model required, and the NPU features required.

[0070] In practical applications, the resources requested when creating a container group Pod can be determined by creating resource declarations. For example, when obtaining the Pod creation request, resource declarations can be created to determine at least two resources requested when creating a container group Pod.

[0071] When a user creates a Pod, the Kubernetes cluster scheduler receives the Pod's scheduling request. At this point, the Kubernetes cluster scheduler can create a corresponding resource declaration. This resource declaration can specify multiple resource categories, and the ParametersRef parameter specifies the attributes that each requested resource must satisfy. It can be seen that since at least two resources requested when creating a Pod can be determined based on the resource declaration, this embodiment can more accurately and reasonably determine the pre-selected nodes based on considering the attributes that at least one or two resources must satisfy. This facilitates the accurate selection of the Pod's scheduling node from the pre-selected nodes.

[0072] In some embodiments, the process of determining the pre-selected nodes in each node based at least on the information of resources in each node and the attributes that the resources requested when creating a Pod need to satisfy may include: obtaining preset filtering rules for each type of resource; and determining the pre-selected nodes in each node based on the filtering rules for each type of resource, the information of resources in each node, and the attributes that the resources requested when creating a container group Pod need to satisfy.

[0073] Here, the filtering rules for each resource can be judgment statements organized using logical operations. For example, logical operations can be OR operations or AND operations, and judgment statements include, but are not limited to, equal to, include, intersect, exclude, greater than, less than, etc.

[0074] For example, the resource-driven controller can create filtering rules for each resource, and the filtering rules can be in YAML format.

[0075] Referring to Figure 6, after receiving a Pod's scheduling request, kube-scheduler no longer negotiates with the resource-driven controller; instead, the resource-driven controller assists in the scheduling. For example, kube-scheduler can obtain the ResourcePerNode object and the filtering rules created by the resource-driven controller, and then determine the pre-selected nodes from among the various nodes based on the pre-selection rules. For instance, for each resource in a node, kube-scheduler will additionally filter each node using the resource declaration, the corresponding filtering rules, and the node's ResourcePerNode object to obtain the pre-selected nodes.

[0076] In one example, if the ParametersRef in the resource declaration sets the GPU memory size to memoryValue, then devices with a memory value greater than memoryValue can be selected based on the ResourcePerNode object. If the resource declaration specifies a device with a particular function or model, then the corresponding device can be selected based on the ResourcePerNode object. After selecting devices that meet the above conditions, if the number of devices selected in a node is less than the number requested in the resource declaration, then this node is discarded; otherwise, if the number of devices selected in a node is greater than or equal to the number requested in the resource declaration, then this node is determined as one of the pre-selected nodes.

[0077] As can be seen, in this embodiment of the application, the pre-selected nodes can be determined more accurately in each node according to the filtering rules of each resource.

[0078] Step 502: Send the identifiers of the pre-selected nodes to the controller, so that the controller can determine the first fit score between each of the pre-selected nodes and the Pod's scheduling node.

[0079] The controller in this step can be the resource-driven controller described above. After sending the identifiers of the pre-selected nodes to the controller, the controller can score each of the pre-selected nodes to obtain the first fit score between each of the pre-selected nodes and the scheduling node of the Pod. The controller can then return the first fit score to the scheduler of the Kubernetes cluster.

[0080] Step 503: Receive the first fit score sent by the controller, and select the Pod's scheduling node from the pre-selected nodes based on the second fit score of each node and the Pod's scheduling node in the pre-selected nodes determined by the scheduler, as well as the first fit score.

[0081] Referring to Figure 6, after determining the pre-selected nodes, kube-scheduler can select the scheduling node for the Pod from among the pre-selected nodes according to the selection rules. For example, kube-scheduler can perform a weighted summation of its own determined second fit score and the first fit score sent by the resource-driven controller to determine the comprehensive score of each pre-selected node, and select the node with the highest comprehensive score as the scheduling node for the Pod.

[0082] In some embodiments, kube-scheduler can determine a second fit score between each node and the scheduling node of the Pod based on the runtime status data of each node. Here, the runtime status data includes, but is not limited to, the failure rate of each node, the matching degree of the task characteristics between each node and the Pod, etc. For example, kube-scheduler can calculate the failure rate of each node; the higher the failure rate, the lower the second fit score of the node. kube-scheduler can also determine whether the image required by the Pod already exists on the node. If the image required by the Pod exists on the node, the second fit score of the node is relatively high; otherwise, the second fit score of the node is relatively low.

[0083] In some embodiments, kube-scheduler notifies the resource-driven controller of the pod's scheduling node. The resource-driven controller can then write the resource allocation scheme (i.e., the device allocation result) from the pod's scheduling node into the resource declaration. After the pod's node scheduling is completed, the kubelet of the corresponding node calls the resource-driven plugin to allocate resources (devices). The resource-driven plugin can perform the allocation operation based on the device allocation result written in the resource declaration by the resource-driven controller.

[0084] In practical applications, steps 501 to 503 can be implemented based on a processor, which can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), CPU, controller, microcontroller, and microprocessor.

[0085] In related technologies, the scheduling node is determined through negotiation between kube-scheduler and the resource-driven controller. During this process, both kube-scheduler and the resource-driven controller have hard filtering rules that can reject scheduling to a particular node. Since kube-scheduler is the one that ultimately performs the scheduling operation (binding the Pod to the node), it needs to try to select a node that the resource-driven controller does not reject and perform the scheduling operation, which can easily lead to multiple scheduling attempts. However, in this embodiment, the Kubernetes cluster scheduler does not need to negotiate with the controller and can select nodes that meet the filtering rules in one go, thus determining the pre-selected nodes. The Kubernetes cluster scheduler does not need to rely on multiple guesses to determine nodes that the controller does not reject. The controller only scores the pre-selected nodes for the Kubernetes cluster scheduler's reference. Finally, the Kubernetes cluster scheduler can determine the scheduling node for the Pod in one go based on the first and second fit scores, improving scheduling efficiency.

[0086] Figure 7 is a flowchart of a node scheduling method for a container applied to a controller according to an embodiment of this application. As shown in Figure 7, the process includes:

[0087] Step 701: Receive the identifiers of the pre-selected nodes sent by the scheduler of the Kubernetes cluster. The scheduler uses this identifier to determine the pre-selected nodes based on at least the information of the resources in each node and the attributes that the resources requested when creating a Pod must satisfy.

[0088] Step 702: Determine the first fit score between each node in the pre-selected nodes and the Pod's scheduling node.

[0089] Step 703: Send the first fit score to the scheduler, so that the scheduler selects the Pod's scheduling node from the pre-selected nodes based on the second fit score of each node in the pre-selected nodes and the Pod's scheduling node, as well as the first fit score.

[0090] In practical applications, steps 701 to 703 can be implemented based on a processor, which can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0091] As can be seen, in this embodiment of the application, the Kubernetes cluster scheduler does not need to make multiple blind guesses to determine the nodes that the controller will not reject. The controller only scores the pre-selected nodes for the Kubernetes cluster scheduler to refer to. Finally, the Kubernetes cluster scheduler can determine the scheduling node of the Pod in one go based on the first fit score and the second fit score, which improves the scheduling efficiency.

[0092] In some embodiments of this application, the process of determining the first fit score between each node in the pre-selected nodes and the scheduling node of the Pod may include: determining the resource allocation scheme of each node based at least on the topological relationship between different types of resources in each node; and determining the first fit score between each node and the scheduling node of the Pod based on the resource allocation scheme of each node.

[0093] The controller's scoring logic takes into account the topological relationships between various resources. For example, in implementing the optimal allocation scheme for GPU and RDMA devices, the selection of GPU and RDMA follows this process:

[0094] a. Select GPU devices that meet the filtering rules as GPUs to be allocated in the following order (bg) until the number of GPUs to be allocated meets the quantity requirements in the resource declaration.

[0095] b. When an NVSwitch device exists, select as many GPUs as possible from one NVSwitch device as GPUs to be assigned.

[0096] c. Select the GPU that is connected to the currently assigned GPU device via NVLink and add it to the set of GPUs to be assigned.

[0097] d. Select the GPU that is connected to the currently assigned GPU device via PCIe and add it to the set of GPUs to be assigned.

[0098] e. Select the GPU that is on the same CPU as the currently assigned GPU device and add it to the GPU set to be assigned.

[0099] f. Select a GPU that is in the same NUMA as the currently assigned GPU device and add it to the GPU set to be assigned.

[0100] g. Select other GPU devices on the node and add them to the GPU set to be allocated.

[0101] h. Select RDMA devices based on the GPU devices to be allocated. Select RDMA devices that meet the filtering rules in the following im order as RDMA devices to be allocated until the number of RDMA devices to be allocated meets the quantity requirements in the resource declaration.

[0102] i. Select the RDMA device that is connected to the GPU device to be assigned via NVLink and add it to the RDMA set to be assigned.

[0103] j. Select the RDMA device that is connected to the GPU device to be assigned via PCIe and add it to the RDMA set to be assigned.

[0104] k. Select the RDMA device that is on the same CPU as the GPU device to be assigned and add it to the RDMA set to be assigned.

[0105] l. Select the RDMA device under the same NUMA as the GPU device to be assigned and add it to the RDMA set to be assigned.

[0106] m. Select other RDMA devices on the node to add to the RDMA set to be allocated.

[0107] For example, in implementing the optimal allocation scheme for NPU devices, the NPU is selected according to the following process:

[0108] a. Select NPU devices that meet the filtering rules as NPUs to be assigned in the following bf order until the number of NPU devices to be assigned meets the quantity requirements in the resource declaration.

[0109] b. Select the NPU device that is connected to the currently assigned NPU device through HCCN (Huawei Cache Coherence Network) and add it to the set of NPUs to be assigned.

[0110] c. Select the NPU device that is connected to the currently assigned NPU device via PCIe and add it to the NPU set to be assigned.

[0111] d. Select the NPU device that is on the same CPU as the currently assigned NPU device and add it to the NPU set.

[0112] e. Select the NPU device that is under the same NUMA as the currently assigned NPU device and add it to the NPU set to be assigned.

[0113] f. Select other NPU devices on the node and add them to the NPU set to be assigned.

[0114] It can be seen that since the resource allocation scheme of each node can be determined based on the topological relationship between different types of resources in each node, it is beneficial to achieve reasonable resource allocation of nodes by coordinating multiple types of resources.

[0115] In some embodiments of this application, determining the first fit score between each node and the scheduling node of the Pod based on the resource allocation scheme of each node may include:

[0116] Determine the resource attribute data corresponding to the resource allocation scheme for each node. The resource attribute data includes at least one of the following: the failure rate of each resource in each node, the minimum bandwidth between different resources in each node, and the matching degree between each resource in each node and the task characteristics of the Pod.

[0117] Based on the resource attribute data corresponding to the resource allocation scheme of each node, determine the first fit score between each node and the scheduling node of the Pod.

[0118] For example, after generating the optimal allocation scheme for GPU, RDMA, or NPU, the nodes are scored according to the set to be allocated, and the scoring rules are as follows:

[0119] a. Calculate the minimum bandwidth between two devices in the set to be assigned. The higher the minimum bandwidth, the higher the score, and vice versa.

[0120] b. Scoring is based on task characteristics. When the Pod requesting resources is a training Pod, the total video memory of the GPUs or NPUs in the allocation set is calculated, and the higher the total video memory, the higher the score. When the Pod requesting resources is an inference Pod, the total floating-point operation capability (Floating Point Operations Per Second, FLOPS) of the GPUs or NPUs in the allocation set is calculated, and the higher the total floating-point operation capability, the higher the score.

[0121] c. Score based on the device failure rates recorded in ResourcesPerNode. Calculate the total failure rate (the sum of all device failure rates) of all devices in the set to be assigned. The higher the total failure rate, the lower the score.

[0122] After the controller determines the first fit score, it can return the first fit score to the scheduler of the Kubernetes cluster. At the same time, the optimal allocation scheme for each node is recorded in the resource declaration so that the resource-driven plugin can allocate the device in the future.

[0123] In summary, the embodiments of this application can achieve unified management and unified topology computing of various intelligent computing resources. Unlike related technical solutions, the embodiments of this application support Pods to apply for multiple resources through a single resource declaration. That is, in an instance of a resource declaration, a Pod can simultaneously apply for RDMA devices and GPU resources. This is mainly achieved by changing the relationship between resource declarations and resource types from one-to-one to one-to-many. In the pre-selection nodes of the scheduling, nodes can be filtered according to different resources. In the preferred nodes of the scheduling, the topological relationships between various resources can be considered and scored.

[0124] This application improves upon the DRA mechanism in related technologies, resolving the issue of kube-scheduler needing to coordinate multiple times with the resource-driven controller. In this embodiment, the resource-driven controller only informs kube-scheduler how to select nodes during the pre-selection phase and how many points to add during the optimization phase; it does not reject the scheduling results of kube-scheduler. Furthermore, this application supports the unified management of multiple resources, enabling unified topology calculations for various intelligent computing resources during optimization scoring.

[0125] This application proposes an improved DRA (Distributed Automated Resource Allocation) intelligent computing resource allocation process to enhance DRA scheduling efficiency. The process includes: informing the kube-scheduler of pre-selection rules through filtering rules; and providing the kube-scheduler with additional optimization scoring items through a resource-driven controller. This application can solve the problem of coordinated allocation between different types of resources, enabling unified topology computation for various intelligent computing resources. In this application, multiple resource types can be applied for through resource declarations; nodes are filtered through filtering rules for each resource; and the resource-driven controller generates an allocation set containing multiple resources according to certain rules and performs scoring.

[0126] Compared to Kubernetes Device Plugins and RDMA Device Plugins implemented based on driver plugins, the embodiments of this application have superior scheduling capabilities, possessing the ability to coordinate multiple types of resources to achieve optimal allocation. Compared to Kubernetes-dra-driver based on the DRA mechanism, the embodiments of this application do not require multiple negotiations, resulting in higher efficiency; they do not only manage one type of resource but can uniformly manage various intelligent computing resources and coordinate the allocation of various resources, which is beneficial to improving intelligent computing performance.

[0127] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0128] Figure 8 is a schematic diagram of the structure of a node scheduling device for a Kubernetes cluster scheduler according to an embodiment of this application. As shown in Figure 8, the device includes:

[0129] The first processing module 801 is configured to determine the pre-selected nodes among the nodes based at least on the information of resources in each node and the attributes that the resources requested when creating the Pod need to satisfy.

[0130] The second processing module 802 is configured to send the identifiers of the pre-selected nodes to the controller, so that the controller determines a first fit score between each of the pre-selected nodes and the scheduling node of the Pod; receive the first fit score sent by the controller, and select the scheduling node of the Pod from the pre-selected nodes based on the second fit score between each of the pre-selected nodes and the scheduling node of the Pod determined by the scheduler, and the first fit score.

[0131] In some embodiments, the first processing module 801 is configured to determine pre-selected nodes among the nodes based at least on information about resources in each node and attributes that the resources requested when creating a Pod must satisfy, including:

[0132] Get the preset filtering rules for each type of resource;

[0133] Based on the filtering rules for each resource, the resource information in each node, and the attributes that the resources requested when creating a Pod need to satisfy, the pre-selected nodes are determined among the nodes.

[0134] In some embodiments, the filtering rules for each resource are judgment statements organized using logical operations.

[0135] In some embodiments, the first processing module 801 is further configured to, before determining the pre-selected nodes among the nodes based at least on the information of resources in each node and the attributes that the resources requested when creating the Pod need to satisfy, create a resource declaration when obtaining the scheduling request of the Pod, and determine at least two types of resources requested when creating the Pod based on the resource declaration.

[0136] In practical applications, the first processing module 801 and the second processing module 802 can be implemented based on a processor and a communication device.

[0137] Figure 9 is a schematic diagram of the node scheduling device for a container applied to a controller according to an embodiment of this application. As shown in Figure 9, the device includes:

[0138] The receiving module 901 is configured to receive the identifier of the pre-selected node sent by the scheduler of the Kubernetes cluster. The scheduler is used to determine the pre-selected node from the nodes based at least on the information of resources in each node and the attributes that the resources requested when creating a container group Pod need to satisfy.

[0139] The third processing module 902 is configured to determine a first fit score between each node in the pre-selected nodes and the scheduling node of the Pod; send the first fit score to the scheduler, so that the scheduler selects the scheduling node of the Pod from the pre-selected nodes based on the second fit score between each node in the pre-selected nodes and the scheduling node of the Pod, and the first fit score.

[0140] In some embodiments, the third processing module 902 is configured to determine a first fit score between each of the pre-selected nodes and the scheduling node of the Pod, including:

[0141] The resource allocation scheme for each node is determined based at least on the topological relationships between different types of resources in each node;

[0142] Based on the resource allocation scheme of each node, a first compatibility score is determined between each node and the scheduling node of the Pod.

[0143] In some embodiments, the third processing module 902 is configured to determine a first compatibility score between each node and the scheduling node of the Pod based on the resource allocation scheme of each node, including:

[0144] Determine the resource attribute data corresponding to the resource allocation scheme of each node, wherein the resource attribute data includes at least one of the following: the failure rate of each resource in each node, the minimum bandwidth between different resources in each node, and the matching degree between each resource in each node and the task characteristics of the Pod;

[0145] Based on the resource attribute data corresponding to the resource allocation scheme of each node, a first compatibility score is determined between each node and the scheduling node of the Pod.

[0146] In practical applications, the receiving module 901 and the third processing module 902 can be implemented based on a processor and a communication device.

[0147] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0148] It should be noted that, in the embodiments of this application, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a terminal, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0149] Correspondingly, this application embodiment further provides a computer program product, the computer program product including computer executable instructions, which are used to implement any of the container node scheduling methods provided in this application embodiment.

[0150] Accordingly, this application embodiment further provides a computer storage medium storing computer-executable instructions, which are used to implement any of the container node scheduling methods provided in the above embodiments.

[0151] This application also provides an electronic device. Figure 10 is a schematic diagram of the composition structure of an electronic device provided in this application embodiment. As shown in Figure 10, the electronic device 100 may include:

[0152] Memory 1001 is used to store executable instructions;

[0153] The processor 1002 is used to implement any of the above-described container node scheduling methods when executing executable instructions stored in the memory 1001.

[0154] The processor 1002 mentioned above can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0155] The aforementioned computer-readable storage medium and memory 1002 may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it may also be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0156] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0157] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0158] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0159] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0160] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0161] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0162] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims. All of these forms are within the protection scope of this application.

Claims

1. A method for scheduling container nodes, applied in a scheduler of a Kubernetes cluster, the method comprising: Based at least on the information of resources in each node and the attributes that the resources requested when creating a container group Pod need to satisfy, the pre-selected nodes are determined from the nodes. The identifiers of the pre-selected nodes are sent to the controller, so that the controller determines the first fit score between each of the pre-selected nodes and the scheduling node of the Pod; The system receives a first fit score from the controller, and selects a scheduling node for the Pod from the pre-selected nodes based on a second fit score between each node in the pre-selected nodes and the scheduling node of the Pod, and the first fit score.

2. The method according to claim 1, wherein, The process of determining the pre-selected nodes from among the nodes, based at least on information about resources in each node and the attributes that the resources requested when creating the Pod must satisfy, includes: Get the preset filtering rules for each type of resource; Based on the filtering rules for each resource, the resource information in each node, and the attributes that the resources requested when creating a Pod need to satisfy, the pre-selected nodes are determined among the nodes.

3. The method according to claim 2, wherein, The filtering rules for each resource are judgment statements organized using logical operations.

4. The method according to claim 1, wherein, Before determining the pre-selected nodes from among the nodes, based at least on information about resources in each node and the attributes that the resources requested when creating the Pod must satisfy, the method further includes: When a Pod's scheduling request is obtained, a resource declaration is created, and at least two resources requested when creating the Pod are determined based on the resource declaration.

5. A node scheduling method for containers, applied in a controller, the method comprising: The system receives the identifiers of pre-selected nodes sent by the scheduler of the Kubernetes cluster. The scheduler is used to determine the pre-selected nodes from among the nodes based at least on the information of resources in each node and the attributes that the resources requested when creating a container group Pod need to satisfy. Determine the first fit score between each of the pre-selected nodes and the scheduling node of the Pod; The first fit score is sent to the scheduler, so that the scheduler selects the scheduling node for the Pod from the pre-selected nodes based on the second fit score between each node in the pre-selected nodes and the scheduling node of the Pod, and the first fit score.

6. The method according to claim 5, wherein, The determination of the first compatibility score between each of the pre-selected nodes and the scheduling node of the Pod includes: The resource allocation scheme for each node is determined based at least on the topological relationships between different types of resources in each node; Based on the resource allocation scheme of each node, a first compatibility score is determined between each node and the scheduling node of the Pod.

7. The method according to claim 6, wherein, The step of determining the first compatibility score between each node and the scheduling node of the Pod based on the resource allocation scheme of each node includes: Determine the resource attribute data corresponding to the resource allocation scheme of each node, wherein the resource attribute data includes at least one of the following: the failure rate of each resource in each node, the minimum bandwidth between different resources in each node, and the matching degree between each resource in each node and the task characteristics of the Pod; Based on the resource attribute data corresponding to the resource allocation scheme of each node, a first compatibility score is determined between each node and the scheduling node of the Pod.

8. A container node scheduling device, applied in a scheduler of a Kubernetes cluster, the device comprising: The first processing module is configured to determine the pre-selected nodes among the nodes based at least on the information of resources in each node and the attributes that the resources requested when creating the container group Pod need to satisfy. The second processing module is configured to send the identifiers of the pre-selected nodes to the controller, so that the controller determines a first fit score between each of the pre-selected nodes and the scheduling node of the Pod; receive the first fit score sent by the controller, and select the scheduling node of the Pod from the pre-selected nodes based on the second fit score between each of the pre-selected nodes and the scheduling node of the Pod determined by the scheduler, and the first fit score.

9. A node scheduling device for containers, applied in a controller, the device comprising: The receiving module is configured to receive the identifier of the pre-selected node sent by the scheduler of the Kubernetes cluster. The scheduler is used to determine the pre-selected node from the nodes based at least on the information of resources in each node and the attributes that the resources requested when creating a container group Pod need to satisfy. The third processing module is configured to determine a first fit score between each node in the pre-selected nodes and the scheduling node of the Pod; send the first fit score to the scheduler, so that the scheduler selects the scheduling node of the Pod from the pre-selected nodes based on the second fit score between each node in the pre-selected nodes and the scheduling node of the Pod, and the first fit score.

10. An electronic device comprising a processor and a memory for storing a computer program capable of running on the processor; wherein, The processor is used to run the computer program to perform the method according to any one of claims 1 to 7.

11. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1 to 7.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method of any one of claims 1 to 7.

Images