Container management system and methods, equipment, media and products for scheduling heterogeneous devices

Abstract

This specification provides a container management system and a method, device, medium, and product for scheduling heterogeneous devices. The method is applied to the control plane of the system for managing a cluster of worker nodes, including: listening to device resource objects sent by worker nodes; writing information of the connected heterogeneous devices into the resource objects of worker nodes according to a control plane-compatible protocol based on custom device resource objects; obtaining Pod creation requests; converting resource requirement information into a data format compatible with the control plane protocol; scheduling target worker nodes and target heterogeneous devices adapted to the resource requirement information from the cluster based on the conversion result and the resource objects in the control plane; converting the scheduling result of the target heterogeneous devices into a plugin-compatible format of the target heterogeneous devices and writing it into the configuration information corresponding to the Pod creation request, so as to trigger the target worker nodes to use the plugin to allocate resources of the target heterogeneous devices to the Pod to be created according to the configuration information.

Description

Technical Field

[0001] This specification relates to one or more embodiments in the field of cloud-native technology, and in particular to a container management system and a method, apparatus, medium and product for scheduling heterogeneous devices. Background Technology

[0002] Container management systems can automate the deployment, scheduling, and operation and maintenance of containerized applications. Taking Kubernetes (K8s) as an example, it divides the cluster into a control plane and a worker node cluster. The control plane is responsible for global management and scheduling decisions, while the worker nodes carry out the actual container operation.

[0003] In terms of resource scheduling, container management systems natively support the scheduling of general-purpose resources such as CPU and memory. These resources have a unified usage and management method at the operating system level, and the scheduler can accurately perceive the resource availability of each node. With the development of AI applications, heterogeneous devices such as Graphics Processing Units (GPUs) and Remote Direct Memory Access (RDMA) network cards are increasingly being introduced into clusters. These devices are not natively supported by container management systems and require device plugins provided by device manufacturers to connect to the cluster.

[0004] Device plugins can identify local heterogeneous devices and report resources to the container management system. However, each vendor uses its own custom resource formats and access protocols, resulting in a lack of a unified heterogeneous device scheduling solution for the cluster. Summary of the Invention

[0005] In view of the above, one or more embodiments of this specification provide the following technical solutions: According to a first aspect of the embodiments of this specification, a method for scheduling heterogeneous devices in a container management system is provided. The method is applied to the control plane of the container management system, the control plane being used to manage a cluster of worker nodes. The control plane deploys several resource objects for each worker node, and the non-native heterogeneous devices accessed by the worker nodes are bound to plugins. The method includes: Listen for device resource objects sent by worker nodes. Device resource objects are resource objects that worker nodes convert into control plane-defined objects when they obtain device attribute information of heterogeneous devices. Based on a custom device resource object, and in accordance with a control plane compatible protocol, information about the accessed heterogeneous devices is written into the resource object of the worker node. Obtain the Pod creation request. The Pod creation request contains user-declared resource requirement information. The resource requirement information is created by the user based on a protocol that is compatible with the heterogeneous devices that has been pre-published in the control plane. The resource requirement information is converted into a data format compatible with the control plane protocol. Based on the conversion result and the resource objects in the control plane, target worker nodes and target heterogeneous devices that are adapted to the resource requirement information are scheduled from the cluster. The scheduling results of the target heterogeneous device are converted into a plugin-compatible format for the target heterogeneous device and written into the configuration information corresponding to the Pod creation request. This triggers the target worker node to use the plugin to allocate resources of the target heterogeneous device to the Pod to be created based on the configuration information.

[0006] According to a second aspect of the embodiments of this specification, a method for scheduling heterogeneous devices in a container management system is provided. The method is applied to worker nodes in a worker node cluster managed by a control plane within the container management system, wherein the non-native heterogeneous devices accessed by the worker nodes are bound to plugins. The method includes: Obtain the device attribute information of the accessed heterogeneous devices, convert the device attribute information into a device resource object customized by the control plane, and send it to the control plane; Obtain the configuration information issued by the control plane for the Pod to be created; The plugin allocates resources from the target heterogeneous devices to the Pod to be created based on the configuration information. The control plane is used to perform the steps of any embodiment of the first aspect.

[0007] According to a third aspect of the embodiments of this specification, a container management system is provided. The system includes a cluster of worker nodes and a control plane for managing the cluster of worker nodes. Non-native heterogeneous devices connected to the worker nodes are bound to plugins. The control plane is used to execute the steps of any embodiment of the first aspect, and the worker nodes are used to execute the steps of any embodiment of the second aspect.

[0008] According to a fourth aspect of the embodiments of this specification, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any embodiment of the first or second aspect.

[0009] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the embodiments of the first or second aspect.

[0010] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of any embodiment of the first or second aspect.

[0011] The technical solutions in the embodiments of this specification may include the following beneficial effects: In the embodiments described in this specification, the control plane monitoring worker nodes convert the device attribute information of heterogeneous devices and report device resource objects. These device resource objects use a unified format defined by the control plane, masking the differences in resource formats defined by different vendors' device plugins. Based on these device resource objects, heterogeneous device information is written into the resource objects of the worker nodes according to a control plane-compatible protocol. This allows the cluster to form a unified view of heterogeneous device resources, and the control plane scheduler can directly perceive the availability of heterogeneous devices on each worker node based on this information.

[0012] When a Pod creation request is received, the resource requirements declared by the user are created based on a unified protocol pre-published in the control plane. Users do not need to learn the access protocols of different vendors, thus lowering the barrier to entry for heterogeneous devices. After converting the resource requirements into a control plane-compatible protocol data format, the scheduler can make scheduling decisions based on a unified resource view, selecting suitable target worker nodes and target heterogeneous devices from the cluster. After scheduling, the scheduling result is converted into a format compatible with the plugins bound to the target heterogeneous device and written into the Pod's configuration information. This allows the worker node's plugins to read the configuration information and execute resource scheduling for heterogeneous devices just like they would with the decision results of their own scheduler, thereby achieving centralized scheduling of heterogeneous devices. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the architecture of a container management system provided in an exemplary embodiment.

[0014] Figure 2A This is a flowchart illustrating a method for scheduling heterogeneous devices in a container management system according to an exemplary embodiment.

[0015] Figure 2B This is a schematic diagram of the architecture of a container management system illustrated in this specification according to an exemplary embodiment.

[0016] Figure 2C This is a schematic diagram illustrating a container management system according to an exemplary embodiment of this specification.

[0017] Figure 2D This specification illustrates an interactive diagram of scheduling heterogeneous devices in a container management system according to an exemplary embodiment.

[0018] Figure 3 This is a flowchart illustrating a method for scheduling heterogeneous devices on the worker node side according to an exemplary embodiment of this specification.

[0019] Figure 4 This is a schematic structural diagram of a device provided in an exemplary embodiment.

[0020] Figure 5 This is a block diagram illustrating an apparatus for scheduling heterogeneous devices in a container management system according to an exemplary embodiment.

[0021] Figure 6 This is a block diagram illustrating an apparatus for scheduling heterogeneous devices in another container management system according to an exemplary embodiment of this specification. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0023] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0024] like Figure 1 As shown, Figure 1 This is a schematic diagram illustrating the architecture of a container management system according to an exemplary embodiment. The container management system in this embodiment includes, but is not limited to, various systems capable of cluster management, such as Kubernetes (or K8s). Figure 1 As shown, taking Kubernetes as an example, the container management system can divide the machines in the cluster into a control plane (Master) and several worker nodes (Node). Figure 1 For the sake of simplicity, only one Master and three Nodes (Node1, Node2 and Node3) are shown in the example.

[0025] The control plane runs a set of processes related to cluster management. Taking Kubernetes as an example, this might include components such as the API Server, Controller Manager, and Scheduler. The API Server is the entry point for cluster control and the sole entry point for operations on all resource objects. The Scheduler is responsible for scheduling Pods to run on appropriate worker nodes. The Controller Manager implements management functions such as resource management, elastic scaling, and security control. On worker nodes, service processes such as kubelet and kube-proxy run. kubelet is responsible for creating, starting, monitoring, and destroying Pods, as well as implementing load balancing in a software model. On a Node, the smallest running unit managed by the container management system is the Pod. A Pod can be viewed as a "logical host" at the application layer by a containerized environment, and a Pod contains one or more closely related containers.

[0026] In terms of resource scheduling, container management systems natively support the scheduling of general-purpose resources such as CPU and memory. These general-purpose resources have a unified usage and management method at the operating system level. The scheduler can accurately perceive the total amount and allocated amount of CPU and memory on each worker node and match scheduling according to the resource requirements declared by the Pod. This is because CPU and memory have sufficient generality, and their usage and management methods are consistent at the operating system level.

[0027] However, with the development of AI applications and high-performance computing, heterogeneous devices such as GPUs, RDMA high-speed network cards, and hardware accelerators need to be introduced into the cluster. These devices are not natively supported by container management systems and require device plugins provided by device manufacturers to connect to the cluster. Under the device plugin mechanism, the device plugin runs on the worker node, identifies local heterogeneous devices, reports them as custom resources to the allocatable resource attributes of the corresponding resource object on the worker node, and registers this resource with the container agent component on the worker node. In this way, the container management system can be aware of the existence of heterogeneous device resources on the worker node.

[0028] However, the resource information reported by device plugins is relatively limited. The plugins primarily report the resource's identifier and a total quantity, such as xxx.com / gpu: 800. In practice, some GPUs can be shared by multiple Pods, and device plugins typically report a larger value to ensure Pods can be allocated to suitable nodes through general resource scheduling. Other GPUs may not support sharing by multiple Pods, in which case the reported quantity is the actual number of GPUs (i.e., the number of GPU cards). Device plugins do not report detailed device attributes, such as the device's specific computing power, memory capacity, hardware topology relationships between devices, or the device's health status.

[0029] Furthermore, different vendors' device plugins use different custom resource names and access protocols. For example, one GPU vendor might define a resource as nvidia.com / gpu, while another might define it as xxx.com / xgpu. Users with heterogeneous devices need to learn each vendor's specific resource declaration format and fill in the resource declaration content in the Pod according to the vendor's specifications. User application deployment schemes need to be designed and implemented separately for different vendors, resulting in poor portability.

[0030] From an operations and maintenance perspective, there is no unified method for querying device information and health status of heterogeneous devices from different vendors in a cluster. When a device on a worker node fails, the administrator needs to first identify which vendor's plugin is installed on that node, then consult that vendor's troubleshooting manual, and finally analyze the vendor's plugin logs to locate the problem. If a cluster contains devices from multiple vendors, multiple troubleshooting processes need to be maintained.

[0031] At the scheduling level, the native scheduler of a container management system can only perceive the total amount of heterogeneous device resources on each worker node and allocate appropriate worker nodes to Pods based on resource availability. When there are multiple devices on a worker node, the native scheduler cannot determine the specific device allocation. This means that the scheduler cannot allocate a set of devices with the highest communication efficiency to a Pod based on the hardware interconnection topology between devices, nor can it achieve shared allocation of devices, such as in scenarios where multiple Pods share a single GPU and each uses a portion of computing power and GPU memory.

[0032] To address the aforementioned limitations of the native scheduler, some device manufacturers offer scheduling plugins to complement their device plugins, enabling centralized scheduling for their devices. However, centralized scheduling protocols remain customized by each manufacturer, with different vendors' scheduling plugins implemented independently, resulting in varying capabilities and making it difficult to reuse common capabilities across vendors. More importantly, these vendor-specific scheduling plugins operate independently and cannot collaborate. When a Pod requires both a GPU and an RDMA network card, the two devices are allocated independently by their respective scheduling plugins, potentially assigning the GPU to one set of nodes and the network card to another, failing to leverage the communication efficiency advantages of a direct physical connection between the GPU and the network card. Furthermore, vendor-provided scheduling plugins often integrate via Scheduler Extenders. The native scheduler needs to remotely call the Extender via HTTP for each Pod scheduled, resulting in long call chains that can easily become performance bottlenecks in high-concurrency scheduling scenarios across large-scale clusters.

[0033] Another approach is to use a multi-scheduler architecture, assigning different schedulers to different types of Pods. This approach improves scheduling performance to some extent, but it further increases the complexity of heterogeneous computing power integration. Users need to specify the schedulerName in the Pod, and there are also coordination issues between multiple schedulers.

[0034] The above analysis reveals that the root of the problem lies in the lack of a unified protocol abstraction layer to integrate the discovery, modeling, scheduling decisions, and protocol conversion of heterogeneous devices within a single framework. Device plugins excel at low-level hardware operations, such as actually mounting devices into containers, but they shouldn't be left to each vendor to define its own resource declaration protocols and scheduling strategies. If a unified protocol layer could be built on top of the device plugins, responsible for collecting device information in a standardized format, abstracting device specification definitions, intercepting user resource requests and converting them to a unified format, making topology-aware scheduling decisions under a unified view, and finally writing back the scheduling results in a format recognizable by the vendor plugins, then users would only need to learn one protocol to use all heterogeneous devices in the cluster. The cluster would have a unified resource view and topology-aware scheduling capabilities, while the vendor plugins would still handle the low-level hardware operations without requiring any code modifications.

[0035] Based on the above analysis, this specification proposes a method for scheduling heterogeneous devices in a container management system. By constructing a unified device resource management mechanism and protocol conversion mechanism on the control plane, it achieves unified management and scheduling of heterogeneous devices while maintaining compatibility with the existing device plug-in ecosystem. The embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0036] Figure 2A As shown, Figure 2AThis is a flowchart illustrating a method for scheduling heterogeneous devices in a container management system according to an exemplary embodiment. This method can be applied to the control plane of a container management system.

[0037] like Figure 2B As shown, Figure 2B This specification illustrates an architecture diagram of a container management system based on an exemplary embodiment. The container management system may include a control plane and a cluster of worker nodes. The control plane manages the worker node cluster and is responsible for global scheduling decisions and resource management. The worker node cluster includes several worker nodes, each of which can connect to native devices (such as CPUs and memory) and non-native heterogeneous devices (such as GPUs, RDMA network cards, FPGAs, or other hardware accelerators). The control plane deploys several resource objects for each worker node to record the node's attribute information and resource status. The heterogeneous devices connected to the worker nodes are bound to plugins provided by the device manufacturers to perform the underlying allocation operations of the heterogeneous devices on the worker nodes.

[0038] The heterogeneous devices in the embodiments of this specification may include various types, such as GPUs, RDMA network cards for connecting and enabling direct memory access between GPUs, and FPGAs (Field-Programmable Gate Arrays). The number of heterogeneous devices connected to a node may also be one or more, and this specification does not limit this.

[0039] like Figure 2A As shown, the method may include the following steps: Step 202: Listen for the device resource objects sent by the worker nodes. These device resource objects are custom resource objects created by the worker nodes after obtaining the device attribute information of heterogeneous devices, which are then converted into control plane-defined resource objects.

[0040] Step 204: Based on the custom device resource object, and in accordance with the control plane compatible protocol, write the information of the accessed heterogeneous devices into the resource object of the worker node.

[0041] Step 206: Obtain the Pod creation request. This request contains user-declared resource requirements, which are created based on protocols pre-published by the user on the control plane that are compatible with the connected heterogeneous devices.

[0042] Step 208: Convert the resource requirement information into a data format compatible with the control plane protocol. Based on the conversion result and each resource object in the control plane, schedule target worker nodes and target heterogeneous devices that are adapted to the resource requirement information from the cluster.

[0043] Step 210: Convert the scheduling result of the target heterogeneous device into a plugin-compatible format for the target heterogeneous device and write it into the configuration information corresponding to the Pod creation request, so as to trigger the target worker node to use the plugin to allocate resources of the target heterogeneous device to the Pod to be created according to the configuration information.

[0044] Regarding step 202, after detecting the accessed heterogeneous device, the working node can obtain the device attribute information of the heterogeneous device. In this embodiment, the device attribute information may include any data describing the physical characteristics and operating status of the heterogeneous device, including but not limited to device classification information (such as GPU, network card, etc.), device resource quantity (such as computing power value, video memory capacity), hardware topology relationship between devices, and / or device health status, etc.

[0045] There are several ways for worker nodes to obtain device attribute information. For example, they can use device query tools provided by the manufacturers of heterogeneous devices, or they can directly read the information through hardware interfaces provided by the operating system. This specification does not limit the specific method for obtaining device attribute information in the embodiments. For example, as one implementation, worker nodes can use device query tools provided by the manufacturers of heterogeneous devices. For instance, for GPU devices, manufacturers typically install a command-line query tool (such as nvidia-smi) with the driver. Worker nodes can use this tool to obtain detailed information such as the GPU model, memory capacity, health status, and interconnection topology with other devices. This method does not require modification of the manufacturer-provided components and can utilize the manufacturer's existing tool capabilities. As another implementation, worker nodes can also directly read device attribute information through hardware interfaces provided by the operating system, such as by reading the device information file in the / sys directory to obtain the device model and status. This specification does not limit the specific method for obtaining device attribute information in the embodiments.

[0046] After obtaining the device attribute information, the worker node converts the device attribute information into a device resource object defined by the control plane and submits it to the control plane. The device resource object adopts a data format uniformly defined by the control plane. Regardless of the manufacturer or type of device at the underlying level, it is organized according to the same format, thereby shielding the differences between manufacturers and eliminating the need for subsequent processing steps to concern themselves with the specific origin of the underlying device.

[0047] Regarding the specific implementation of step 202 of the work node execution, it can be completed by a single functional component to acquire, convert, and report device attribute information, or it can be completed collaboratively by multiple functional components. The specific division method of functional components is not limited in the embodiments of this specification.

[0048] The device resource object monitoring mechanism in the control plane can be implemented based on the list / watch mechanism of the container management system. For example, relevant components in the control plane continuously listen for the creation and update events of device resource objects to obtain information from the device resource objects of heterogeneous devices reported by each worker node in real time. Each connected heterogeneous device can correspond to an independent device resource object. For example, if a worker node has 8 GPUs and 2 RDMA network cards connected, it can generate 10 device resource objects, and each device resource object can describe the attribute information of a specific device.

[0049] Regarding step 204, after the control plane detects the device resource object, it can write information about the heterogeneous device into the resource object of the worker node according to the device information recorded in the device resource object and a control plane-compatible protocol. In this embodiment, the resource object of the worker node is a data object maintained by the container management system for each worker node, used to record the worker node's attributes (such as tags) and allocable resources. The control plane-compatible protocol refers to the resource types and tag specifications uniformly defined by the control plane, which can be independent of specific device manufacturers. For example, for GPU-type devices, regardless of the manufacturer, the video memory resources can be defined in a uniform format (such as ic / gpu-memory), and the computing resources can also be defined in a uniform format (such as ic / gpu-compute).

[0050] In this embodiment, the control plane compatible protocol can refer to resource types and specifications uniformly defined by the control plane, and can be independent of specific equipment manufacturers. By writing heterogeneous device information into the resource objects of worker nodes, the cluster can form a unified view of heterogeneous device resources, and the scheduler on the control plane can directly perceive the availability of heterogeneous devices on each worker node based on this information.

[0051] There are several ways to write information about heterogeneous devices into the resource objects of worker nodes. For example, the device classification identifier can be written into the tag attribute of the resource object so that the scheduler can identify the device type on the node by matching tags; alternatively, the resource quantity information of the devices can be summarized and written into the allocatable resource attribute of the resource object so that the scheduler can understand the total resource quantity of heterogeneous devices on each node. The embodiments in this specification do not limit the specific method of writing.

[0052] The significance of writing heterogeneous device information into the resource objects of worker nodes is that the control plane scheduler can directly utilize the existing scheduling mechanisms of the container management system to perceive heterogeneous device resources. For example, the node filtering function of the native scheduler can perform preliminary filtering based on the information in the worker node resource objects, excluding nodes that do not have the required heterogeneous devices or have insufficient resources. In this way, the solution of the embodiments in this specification can fully utilize the existing scheduling capabilities of the container management system without completely replacing the native scheduling logic.

[0053] In practical applications, only one model of each device type can be connected to a single working node. However, the embodiments in this specification do not limit this. When multiple models exist on the same node, corresponding device information can be maintained for each model separately, and the resource quantity can be summarized separately.

[0054] Regarding step 206, the control plane can obtain the Pod creation request submitted by the user. The Pod creation request may contain resource requirement information declared by the user. In this embodiment, the resource requirement information is created by the user based on a pre-published protocol compatible with heterogeneous devices. The pre-published protocol can refer to a unified resource declaration specification defined by the control plane. Users can use this protocol to declare the required heterogeneous device resources without needing to understand the specific access methods of various underlying vendor device plugins, thus lowering the barrier to entry for using heterogeneous devices.

[0055] Resource requirement information can include a variety of content. For example, it can include the identifier of the device specification object that the user needs to use, the number of devices required for the corresponding specification (such as the number of GPUs), the number of other types of heterogeneous devices required, and the constraints on the hardware topology between heterogeneous devices.

[0056] In practical applications, some heterogeneous devices can be interconnected. In this case, different interconnection methods may affect the communication efficiency between heterogeneous devices. Therefore, if a user only needs one heterogeneous device, there is no need to declare hardware topology constraints, as there is no interconnection issue between devices. If a user needs multiple heterogeneous devices, or needs different types of devices simultaneously (such as GPUs and RDMA network cards), topology requirements between devices can be declared as needed. For example, high-speed interconnection between allocated GPUs, or GPUs and network cards on the same PCIe slot to achieve the shortest communication path.

[0057] The embodiments in this specification do not limit the specific content of the resource requirement information, which depends on the specific definition of the protocol published by the control plane and the actual needs of the user.

[0058] Regarding the protocols pre-published by the control plane that are compatible with connected heterogeneous devices, the control plane can, based on step 204, generate a device specification object for users to use before submitting a Pod creation request, based on the device information contained in the device resource object. The device specification object uniquely corresponds to a device category identifier and is used to describe the resource attributes of that model of heterogeneous device. By viewing the existing device specification object, users can understand which heterogeneous device models and their resource attributes are available in the cluster and create a Pod creation request accordingly. The specific content and generation method of the device specification object will be described in detail in subsequent embodiments.

[0059] In step 208, the control plane converts the user-declared resource requirement information into a data format compatible with the control plane protocol. Since the user's resource requirement information is based on a protocol declaration pre-published by the control plane, and the control plane scheduler needs to identify specific resource types and quantities during scheduling, format conversion is necessary.

[0060] The conversion includes, but is not limited to: calculating the resource request quantity that the control plane scheduler can recognize based on the device specifications defined in the protocol and the number of devices declared by the user; generating the node filtering conditions required for scheduling based on the device model information in the device specifications; and converting the number of devices into resource identifiers that can be recognized by the plug-ins bound to the heterogeneous devices. The specific content of the conversion in the embodiments of this specification is not limited, and it depends on the protocol published by the control plane and the format required by the plug-ins.

[0061] After the transformation is complete, the scheduler in the control plane schedules target worker nodes and target heterogeneous devices from the cluster that match the resource requirements based on the transformation results and the resource objects in the control plane. As an example, the scheduler filters worker nodes in the cluster according to the conditions in the resource requirements information to obtain a set of candidate nodes. Then, based on the detailed information recorded by the device resource objects on the candidate nodes, it selects the target worker nodes from the candidate node set and determines which heterogeneous devices to allocate to the Pod to be created.

[0062] Regarding step 210, after the scheduler determines the target worker node and the target heterogeneous device, the scheduling result needs to be converted into a format compatible with the plugins bound to the target heterogeneous device. As an example, the scheduling result may include the device identifier of the target heterogeneous device and its corresponding resource allocation quota.

[0063] The purpose of the transformation is to enable plugins on worker nodes to recognize scheduling results and perform device allocation accordingly. Plugins are provided by device vendors, each defining different protocol formats to describe device allocation information. For example, vendor A's plugin might require specific key-value pairs to be written into the Pod's annotations to identify the allocated device, while vendor B's plugin might use a different annotation format. The control plane can write the scheduling results into the configuration information corresponding to the Pod creation request, according to the data format agreed upon by the plugins bound to the target heterogeneous device.

[0064] After the configuration information is written, when a Pod is scheduled to a target worker node, the target worker node (such as a container proxy component on the node) can detect the completion of Pod scheduling and recognize that the Pod has declared resource requests defined by the plugin. It can then request the corresponding plugin to obtain the required injected resources. The plugin can read the scheduling result from the configuration information and perform the actual device allocation operation according to the device allocated by the scheduler, such as mounting a specified GPU into the Pod's container. The container proxy component (such as kubelet) completes the container creation and startup process according to the device injection information returned by the plugin.

[0065] Through the format conversion mechanism in step 210, the solution in this embodiment can transmit the decision result of the unified scheduler to the vendor plugin for execution without modifying the vendor plugin code. The vendor plugin does not need to know that the scheduling decision is actually made by an external unified scheduler; it only needs to read the allocation information according to its own agreed protocol format and complete the device allocation. Therefore, it can be compatible with and reuse the existing device access ecosystem.

[0066] As can be seen from the above embodiments, the embodiments in this specification form a cluster-level heterogeneous device resource view by listening to the unified format of device resource objects reported by the control plane working nodes. Users declare resource requirements based on the unified protocol pre-published by the control plane, without needing to learn the access protocols of different vendors. The control plane converts the resource requirement information into a format recognizable by the scheduler for unified scheduling, and then converts the scheduling results into a plug-in compatible format, allowing vendor plug-ins to complete device allocation without modifying their code. The entire process achieves unified management and scheduling of heterogeneous devices, while maximizing compatibility with the existing device plug-in ecosystem.

[0067] In some examples, worker nodes may deploy a first daemon and a second daemon. The first daemon is used to obtain device attribute information by calling the device query interface provided by the heterogeneous device manufacturer, and write it to a device information file stored locally on the worker node according to a preset general format. The second daemon is used to read the device information file, convert it into a device resource object customized by the control plane, and submit it to the control plane. Accordingly, listening for device resource objects sent by the worker node in step 202 may include: listening for device resource objects submitted by the second daemon on the worker node.

[0068] like Figure 2C As shown, Figure 2C This is a schematic diagram illustrating a container management system according to an exemplary embodiment. The diagram shows the components running on the worker nodes, including a first daemon (which may be named IC DeviceDaemon), a second daemon (which may be named IC Daemon), a device plugin, a container agent component (kubelet), and a container runtime interface (CRI). The diagram also shows the components running in the control plane, including ICOperator, IC Webhook, and IC Scheduler.

[0069] The first daemon (IC Device Daemon) is deployed on worker nodes and requires high-level operating system privileges to access hardware devices and invoke vendor tools. For example, for a GPU device from a specific vendor, the first daemon can invoke the vendor's command-line tools to obtain the model, memory capacity, computing power, health status, and hardware topology relationships with other GPUs and RDMA network cards for each GPU. For RDMA network card devices, the first daemon can invoke the network card manufacturer's query tools to obtain information such as the network card's model, bandwidth, and health status. The first daemon then writes the collected device attribute information into the local device information file on the worker node according to a common format.

[0070] The second daemon (IC Daemon) is also deployed on worker nodes, but its required permissions may differ from those of the first daemon. The second daemon requires control plane API access to create and update device resource objects, but does not need direct access to the hardware devices. The second daemon continuously reads the device information file written by the first daemon, converts the device attribute information in the file into control plane-defined device resource objects (such as custom Device resources), and submits them to the cluster via the control plane API.

[0071] The advantage of splitting the device information collection function into two daemons lies in permission isolation and separation of concerns. The first daemon focuses on hardware interaction and requires only operating system permissions; the second daemon focuses on communication with the control plane and requires only API access permissions. Both transfer data through the local file system and operate under minimal privileges. Even if one component malfunctions, it will not affect the normal operation of the other. In practical applications, the two functions can also be merged into a single component; this specification does not limit this approach.

[0072] In some examples, a worker node may connect to one or more heterogeneous devices. Writing information about the connected heterogeneous devices into the worker node's resource object may include: extracting the device classification identifiers of the heterogeneous devices contained in each device resource object and writing them into the node tag of the worker node's resource object; and summarizing the resource quantity information contained in each device resource object and writing it into the allocatable resource attribute of the worker node's resource object.

[0073] In this embodiment, the device classification identifier refers to information used to distinguish heterogeneous device categories, and may include one or more combinations of manufacturer name, device model, and device type. For example, the device classification identifier may be a combination of vendor=na, model=h100, and type=gpu, that is, a combination of manufacturer identifier, model identifier, and device type identifier. After these identifiers are written into the node labels, the scheduler can quickly locate the working node with the corresponding device when filtering nodes based on the node label selector declared on the Pod.

[0074] In this embodiment, the device classification identifier (rather than resource quantity) is written into the node label because the main purpose of the node label is to identify the characteristic attributes of the node for node filtering during scheduling. For example, if a worker node has 8 GPU cards of the same manufacturer and model connected to it, only one device classification identifier needs to be written into the node label to identify that the node has that model of GPU, without having to write it repeatedly for each card. This approach keeps the node label concise while meeting the requirements of scheduling filtering.

[0075] The resource quantity information is summarized by adding up the resource quantities of the same type of heterogeneous devices on the same working node and writing it into the allocatable resource attribute. For example, if a working node has 8 H100 GPUs, each with 128 Gi of video memory, then the total video memory resource quantity is 1024 Gi, and it is written into the allocatable resource attribute as ic / gpu-memory: 1024Gi. Here, "ic" represents the unified resource namespace of this system. In actual applications, other identifiers can be removed or used as needed, and this embodiment does not limit this. Similarly, if there are 2 RDMA network cards on the node, the number of network cards is summarized and written as ic / rdma: 2. The allocatable resource attribute can be recorded using a common resource type defined by the control plane. Regardless of the manufacturer of the underlying device, resources of the same type of device can use a unified resource type name.

[0076] The allocatable resource attribute records the total physical resources of heterogeneous devices on worker nodes. This value does not change with the creation or deletion of Pods. It can only be updated when the physical devices themselves change, such as when a new device is added, a device is removed, or the device's health status changes. For example, when a GPU changes from a healthy state to an unhealthy state, the control plane can deduct the corresponding resources of that GPU from the allocatable resource attribute, causing the scheduler to no longer schedule new Pods to that unhealthy device.

[0077] In some examples, the protocols pre-published by the control plane that are compatible with connected heterogeneous devices may include: device specification objects for the connected heterogeneous devices. The control plane can also generate device specification objects based on device category identifiers contained in device resource objects. Each device specification object uniquely corresponds to a device category identifier and is used to describe the resource attributes of that model of heterogeneous device. This is used by users before submitting a Pod creation request to declare resource requirement information according to the device specification object.

[0078] As an example, the role of a device specification object can include providing users with a standardized view of device information. By viewing existing device specification objects in the cluster, users can understand which manufacturers and models of heterogeneous devices are available in the current cluster, as well as the physical resource attributes of each device (such as memory capacity, computing power, etc.). Users do not need to know the specific device naming and parameter formats of each manufacturer at the underlying level; they only need to declare their resource requirements according to the unified format defined in the device specification object.

[0079] Device specification objects can be generated based on device category identifiers contained in device resource objects. For example, when the control plane detects the creation of a new device resource object, it can extract the manufacturer name and device model information to check if a corresponding device specification object already exists. If it doesn't exist, a new device specification object is created; if it already exists, it is not created again. For instance, if 20 H100 GPUs from a certain manufacturer are connected to multiple worker nodes in the cluster, although 20 device resource objects will be generated, only one device specification object corresponding to the H100 from that manufacturer can be created. This design makes the device specification object represent a device model category, rather than a specific device instance.

[0080] The device specification object can contain various information, such as device manufacturer, device model, device type, physical memory capacity, and physical computing power. Users can understand the resource capabilities of a particular device model based on the device specification object and decide what resource requirements to declare in the Pod creation request accordingly.

[0081] In some examples, for heterogeneous devices that provide resources as a whole on a single device basis, the device specification object may only include a physical device specification object describing the physical resource attributes of the heterogeneous device. For heterogeneous devices that support resource partitioning on a single device basis, the device specification object may include a physical device specification object and a corresponding virtual device specification object. The virtual device specification object describes the allocatable virtual resource quota for Pod instances, and the virtual resource quota is less than or equal to the physical resources represented by the corresponding physical device specification object.

[0082] In this embodiment, the Physical Device Specification Object (PGS) describes the physical resource attributes of a heterogeneous device model. For example, for an H100 GPU from a certain manufacturer, its PGS can record that the model's memory capacity is 128 Gi, its computing power is a specific value, etc. The PGS is automatically generated by the control plane when a new device is detected, and it uniquely corresponds to the combination of manufacturer and model.

[0083] Virtual Device Specification Objects (VGS) are created based on Physical Device Specification Objects and are used to define allocable virtual resource quotas for Pod instances. The reason for introducing VGS is that in some scenarios, certain heterogeneous devices can support resource partitioning. For example, with GPUs, in many AI applications, users may not need the full resources of a single GPU. For instance, a lightweight inference task might only require half the computing power and a quarter of the VRAM of a GPU. Allocating resources only as a whole GPU would be wasteful. VGS allows defining various partitioning methods, such as "half a GPU" (50% computing power, 50% VRAM) or "a quarter a GPU" (25% computing power, 25% VRAM).

[0084] Virtual resource quotas can be less than or equal to the total physical resources represented by the corresponding physical device specification object, ensuring that the virtual specification does not exceed the actual capacity of the physical device. After the user declares the use of a certain virtual device specification object and quantity in the Pod creation request, the control plane can calculate the actual required resource request amount in step 208 based on the correspondence between virtual device specification objects and physical device specification objects.

[0085] When generating a physical device specification object, the control plane can automatically create a default virtual device specification object corresponding to the physical device specification object, whose virtual resource quota can be equal to the total physical resources. Furthermore, virtual device specification objects with various partitioning specifications can be pre-created based on the physical device specification object according to actual needs. The specific specifications are determined based on the resource partitioning methods actually supported by the heterogeneous device. For example, for a GPU with 128Gi of video memory, if it supports "half" and "quarter" partitioning methods, "half-card" (50% compute power, 50% video memory) and "quarter-card" (25% compute power, 25% video memory) virtual device specification objects can be pre-created. Considering that the partitioning methods for heterogeneous devices are usually limited, this embodiment can pre-create all the necessary specifications for users to declare when submitting a Pod creation request. This method ensures that users only use pre-planned specifications, rather than arbitrarily specifying them, and also helps avoid problems such as resource fragmentation. Additionally, when users need to use the entire device, they can directly use these default virtual device specification objects without creating additional ones.

[0086] For heterogeneous devices that do not require resource partitioning (such as RDMA network interface cards), the device specification object may only include the physical device specification object. These devices are typically allocated as a whole device and do not support allocating partial resources to multiple Pods on the same device. This specification does not limit this approach; if a certain type of network interface card also supports virtualization partitioning in the future, virtual device specification objects can also be created for it.

[0087] In practical applications, when a user submits a Pod creation request, the name of the required virtual device specification object is declared in the resource requirement information. For example, the control plane can pre-maintain a list of created VGSs and look up the corresponding virtual device specification object by name. If it exists, subsequent resource calculations and scheduling are performed based on its correspondence with the physical device specification object; if it does not exist, an error message is returned.

[0088] After the VGS is determined, the control plane will then perform the subsequent resource requirement information conversion steps to calculate the actual required general resource request quantity based on the virtual resource quotas and request quantities defined in the VGS.

[0089] In some examples, worker nodes deploy native container proxy components; the scheduler converts the scheduling results of the target heterogeneous device into a plugin-compatible format for the target heterogeneous device and writes it to the configuration information corresponding to the Pod creation request, thereby triggering the target worker node to use the plugin to allocate resources of the target heterogeneous device to the Pod to be created according to the configuration information, including: Determine the scheduling protocol type of the plug-in bound to the target heterogeneous device; Based on the scheduling results, determine the device identifier of the target heterogeneous device and the corresponding resource allocation quota of the target heterogeneous device in the Pod to be created; According to the data format agreed upon by the scheduling protocol type, the device identifier and resource allocation quota of the target heterogeneous device are written into the annotation corresponding to the Pod creation request. This enables the container proxy component to call the plugin when it hears that the Pod to be created has been scheduled to the target worker node and recognizes the device identifier in the annotation. The plugin then allocates the resources of the target heterogeneous device to the Pod to be created based on the device identifier and resource allocation quota of the target heterogeneous device.

[0090] Different vendors' plugins define different scheduling protocol formats to describe device allocation information. For example, vendor A's plugin might require writing a list of allocated device IDs with a specific prefix in the Pod's annotations, while vendor B's plugin might use a different annotation key-value format. The control plane needs to determine the scheduling protocol type of the plugins bound to the target heterogeneous devices, and then write the scheduling results according to the corresponding format.

[0091] In practice, the control plane can maintain a plugin protocol adaptation table, recording the scheduling protocol types of plugins from various vendors and their corresponding annotation formats. Once the scheduling result is determined, the control plane looks up the plugin protocol type corresponding to the target device and writes the device identifier and resource allocation quota into the Pod's annotation according to the corresponding format. This design allows the protocol adaptation logic to be extended; when a new vendor's device is connected, only the corresponding protocol format definition needs to be added to the adaptation table.

[0092] After a container proxy component (such as kubelet) detects that a Pod has been scheduled to a target worker node, it can check whether the Pod's resource request contains resource identifiers defined by the plugin. If it detects that the annotation contains a device identifier and resource allocation quota, the container proxy component will call the corresponding plugin. The plugin reads this information and performs the actual device allocation operation, such as mounting the specified GPU into the Pod's container. Throughout this process, the plugin does not need to know that the scheduling decision is made by an external unified scheduler; it only needs to read the allocation information according to its own agreed-upon protocol format.

[0093] In some examples, device attribute information may include device classification information, device resource quantity, device health status, and hardware topology relationships between heterogeneous devices. The hardware topology relationships characterize the connection methods between heterogeneous devices within the same working node. And / or, heterogeneous devices may include graphics processors (GPUs) and network interface cards (NICs) used to connect and enable direct memory access between GPUs.

[0094] Device classification information can be used to identify the categories of heterogeneous devices. For example, it can distinguish between GPUs and RDMA network cards, as well as heterogeneous devices from different manufacturers and with different signals.

[0095] Device resource quantity can be used to describe the specific resource capabilities of a device. For example, for a GPU device, it can include video memory capacity and computing power, while for an RDMA network card, it can include bandwidth, etc.

[0096] Device health status indicates whether a device is currently functioning properly, typically categorized into two basic states: healthy and unhealthy. The specific reasons for an unhealthy state can be presented as error codes or error messages; different manufacturers may have different definitions, and this embodiment does not limit this. Optionally, based on this, this embodiment uses a unified device resource object to record device health status. The control plane can query the health status of all devices through a single API, eliminating the need to differentiate between manufacturers and improving the troubleshooting process.

[0097] Hardware topology can describe the connection methods between heterogeneous devices in the same working node, and different connection methods can represent the communication rates between heterogeneous devices.

[0098] The hardware topology in this embodiment can mainly focus on the interconnection of heterogeneous devices within a single worker node, without involving interconnection across worker nodes, because the scheduler considers which GPUs to allocate on a specific worker node. Therefore, this embodiment does not need to consider the case of multiple worker nodes.

[0099] For example, in AI training scenarios, a single worker node can typically carry multiple GPU cards (e.g., eight) and an RDMA network card for cross-node communication. Since the CPU provides a limited number of PCIe lanes, multiple GPUs cannot all be directly connected to the CPU; they need to be connected via a PCIe switch chip. A PCIe switch can be understood as a signal switching chip through which the CPU expands multiple PCIe ports to connect to devices such as GPUs.

[0100] When GPUs connected to the same PCIe switch communicate, data only needs to be exchanged through that switch, without passing through the CPU, resulting in a short communication path and low latency. However, when GPUs connected to different PCIe switches communicate, data must first pass through their respective switches and then be forwarded through the CPU or inter-processor bus, resulting in a longer communication path and higher latency. Furthermore, RDMA network cards are also plugged into the PCIe slots of the worker nodes, establishing a physical connection with the GPUs—RDMA network cards located on the same PCIe switch as the GPUs offer higher data exchange efficiency with these GPUs.

[0101] Based on this, in the embodiments of this specification, hardware topology can refer to the device interconnection topology within a single machine, used to characterize the physical connection method and communication rate level of heterogeneous devices (such as between GPUs, or between a GPU and an RDMA network card) on the same working node, without involving cross-node network topology. When the scheduler allocates multiple heterogeneous devices to a Pod, it selects devices within a defined target working node, using the topology within that node to combine the Pod into a group of devices with optimal communication efficiency, rather than combining devices across multiple nodes.

[0102] In addition, for RDMA network cards, there can be two roles in this embodiment: As a heterogeneous device subject to unified management: Like GPUs, RDMA network cards are also heterogeneous devices and need to be managed by the unified protocol of this system. Users can declare their requirements for RDMA network cards (such as quantity and model) through a unified specification protocol, without having to learn the specific declaration methods of RDMA network card manufacturers.

[0103] As an object jointly scheduled with the GPU: In distributed AI training (such as GDR scenarios), a Pod typically needs to request both a GPU and an RDMA network interface card (NIC) simultaneously. When making allocation decisions, the scheduler can consider whether the GPU and RDMA NIC requested by the Pod have optimal communication paths within a single machine. For example, it can prioritize allocating the RDMA NIC, which is located on the same PCIe switch as the GPU, to the Pod, thereby maximizing the data interaction efficiency between the GPU and the RDMA NIC within the Pod. Which remote node's RDMA NIC the Pod subsequently communicates with across nodes is a matter for the Pod's runtime and is not within the scope of the scheduler's topology-aware scheduling.

[0104] In some examples, the scheduler deployed on the control plane includes both native scheduling plugins and heterogeneous device scheduling plugins. Based on the transformation results and resource objects in the control plane, scheduling target worker nodes and target heterogeneous devices adapted to the resource requirements from the cluster can include: using the native scheduling plugin to perform preliminary filtering of each worker node in the cluster based on the transformation results and the resource objects of each worker node to obtain a candidate node set; and using the heterogeneous device scheduling plugin to schedule target worker nodes and target heterogeneous devices adapted to the resource requirements from the candidate node set based on the device resource objects on each worker node in the candidate node set.

[0105] The scheduler deployed on the control plane can be implemented based on the scheduler framework of the container management system. The scheduler framework is an extensible scheduling architecture provided by the container management system, allowing custom scheduling plugins to be inserted into the scheduling process. Compared to the Scheduler Extender approach, plugins implemented based on the scheduler framework run within the scheduler's main process. Scheduling decisions are made through in-process function calls, eliminating the need for remote calls via HTTP. Therefore, they offer better performance and stability in high-concurrency scheduling scenarios across large-scale clusters.

[0106] The scheduler includes native scheduling plugins that can be built into the container management system's scheduling capabilities, such as node affinity plugins and resource matching plugins. The node affinity plugin matches worker nodes with the node label selector on the Pod and the labels on the worker nodes to filter out worker nodes with the required device model. The resource matching plugin compares the resource requests declared by the Pod with the available resources on the worker nodes, excluding nodes with insufficient resources. After this initial filtering by the native scheduling plugins, a set of candidate nodes that meet the basic criteria is obtained.

[0107] The heterogeneous device scheduling plugin is an extension of the scheduling capabilities described in the embodiments of this specification. After the native scheduling plugin completes the initial filtering, the heterogeneous device scheduling plugin can make more granular scheduling decisions based on the device resource objects on each worker node in the candidate node set. Specifically, the heterogeneous device scheduling plugin can view the resource availability, health status, and hardware topology of each device on each candidate node. After comprehensively considering these factors, it selects the target worker node from the candidate nodes and determines which heterogeneous devices on the target worker node are specifically allocated to the Pod to be created.

[0108] The scheduler internally maintains a device allocation database (commonly known as a "ledger") to record the current resource allocation status of various heterogeneous devices. This mechanism follows the design philosophy of the native scheduler in container management systems. While the native scheduler originally only recorded the allocation of general resources such as CPU and memory, the embodiments in this specification extend this to heterogeneous device resources. When a Pod is successfully created, the scheduler updates the allocated resources of the corresponding device in the ledger; when a Pod is deleted, the scheduler listens for the deletion event, reads the previous device allocation records from the Pod's annotations, and releases the occupied resources of the corresponding device in the ledger. Through the ledger mechanism, the scheduler can accurately grasp the real-time resource availability of each device, ensuring the correctness of scheduling decisions.

[0109] In some examples, converting resource requirement information into a control plane-compatible protocol data format may include: converting the resource requirement information to obtain target heterogeneous device classification identifiers and resource request quantities; and using the target heterogeneous device classification identifiers to generate node label filtering conditions for the Pods to be created. The node label filtering conditions are used to limit the scheduling scope to worker nodes whose node labels contain the target heterogeneous device classification identifier. Correspondingly, preliminary filtering of each worker node in the cluster may include: filtering each worker node in the cluster based on the node label filtering conditions and resource request quantities.

[0110] In this embodiment, the purpose of the conversion is to translate the resource requirements declared by the user based on the device specification object into native scheduling parameters that the scheduler can recognize. For example, if a user declares that they need two H100 GPUs from a certain manufacturer based on a virtual device specification object, the node label filtering condition after conversion could be `nodeSelector: {gpu-model: h100}`, and the resource request amount could be `ic / gpu: 2` and `ic / gpu-memory: 64Gi` (assuming the virtual specification is 1 / 4 of the video memory). The native scheduling plugin can filter out worker nodes whose labels contain `gpu-model=h100` based on the node label filtering condition, and then exclude worker nodes with insufficient allocable resources based on the resource request amount, thus obtaining a candidate node set.

[0111] In some examples, device attribute information may include device classification information, device resource quantity, device health status, and hardware topology relationships between heterogeneous devices. The control plane maintains device allocation data that records the allocation status of each heterogeneous device. Based on the device resource objects on each worker node in the candidate node set, scheduling target worker nodes and target heterogeneous devices that match the resource requirement information from the candidate node set may include: scheduling target worker nodes and target heterogeneous devices that match the resource requirement information from the candidate node set based on one or more of the following: the resource reserve of each heterogeneous device recorded in the device allocation data, and the device health status and hardware topology relationships between devices recorded in the device resource objects.

[0112] In this embodiment, the heterogeneous device scheduling plugin can integrate multiple pieces of information when making scheduling decisions. For example, device allocation data (ledger) records the current resource availability of each device, which is the basis for determining whether a device can accommodate a new Pod. Device health status can be used to exclude unhealthy devices and avoid scheduling Pods to faulty devices. The hardware topology relationship between devices can be used to optimize device allocation to schedule heterogeneous device combinations with better communication efficiency.

[0113] As an example, this process can take into account the available resource balance of each device and determine the final allocation choice based on the global scheduling policy configured on the scheduler. For example, if the Binpack scheduling policy is used, the scheduler tends to allocate Pods to heterogeneous devices with higher resource usage (few remaining resources but still meeting the demand) to fill individual heterogeneous devices first and reduce resource fragmentation; if the Spread scheduling policy is used, the scheduler tends to allocate Pods to the currently idle heterogeneous device to achieve load balancing of heterogeneous devices within a single machine.

[0114] In some examples, scheduling target worker nodes and target heterogeneous devices that are compatible with the resource requirement information from the candidate node set includes: filtering out heterogeneous devices marked as unhealthy based on the device resource objects of each heterogeneous device accessed by the candidate node set; after filtering, if the resource requirement information contains multiple heterogeneous devices and includes constraints on the hardware topology between heterogeneous devices, selecting the target heterogeneous device that meets the constraints and has the best inter-device communication efficiency from the heterogeneous devices of the candidate nodes according to the hardware topology relationship recorded by each device resource object.

[0115] As an example, when the resource requirement information includes constraints on the hardware topology between heterogeneous devices, the control plane can select the target heterogeneous device from the candidate nodes that satisfies the constraints and has the optimal inter-device communication efficiency. For example, the implementation can be as follows: The control plane pre-constructs a device topology graph for each candidate working node based on the hardware topology relationships recorded in the device resource object. For example, the topology graph uses each available heterogeneous device on a node as a vertex and the interconnection relationship between heterogeneous devices as edges, with edge weights assigned according to the connection type or communication rate level between devices. For scenarios requiring the allocation of multiple devices of the same type (e.g., 4 GPUs), the control plane extracts a subgraph containing only that type of device from the device topology graph, traverses all device combinations that meet the quantity requirement, calculates the sum of the shortest path weights between all heterogeneous device pairs in each combination, and selects the combination with the smallest sum of weights as the target device combination with the optimal communication efficiency. For scenarios requiring the allocation of different types of devices (e.g., GPUs and RDMA network cards), the control plane calculates the shortest path between each type of device to be allocated on the complete topology graph containing multiple device types, starting from the device to be allocated of each type, and selects the combination that minimizes the maximum communication path between each type of device as the optimal combination. Of course, in practical applications, any other reasonable algorithm can be used to determine communication efficiency, and this specification does not limit this.

[0116] Specifically, this involves selecting target heterogeneous devices from among the candidate nodes that meet the constraints, have the best inter-device communication efficiency, and conform to the current scheduling strategy. For example, referring to the aforementioned explanation, the available resource reserves of each device can be comprehensively considered, and the final allocation selection can be determined based on the global scheduling strategy configured on the scheduler.

[0117] For example, suppose a Pod requires 4 GPUs and 2 RDMA network cards, and requires high-speed interconnection between GPUs and the shortest communication path between GPUs and network cards. The heterogeneous device scheduling plugin can combine the current global scheduling policy and the available resource margins of each heterogeneous device to traverse the device resource objects of each node in the candidate node set and find a node that simultaneously meets the following conditions: the node has at least 4 healthy GPUs and 2 healthy RDMA network cards, the 4 GPUs are directly connected via NVLink, and the 2 RDMA network cards and 4 GPUs are located on the same PCIe switch.

[0118] like Figure 3 As shown, Figure 3 This is a flowchart illustrating a method for scheduling heterogeneous devices on the worker node side, according to an exemplary embodiment. The method is applied to worker nodes in a worker node cluster managed by the control plane in a container management system. The non-native heterogeneous devices accessed by the worker nodes are bound to corresponding plugins. The method includes the following steps: Step 302: Obtain the device attribute information of the accessed heterogeneous devices, convert the device attribute information into a device resource object customized by the control plane, and send it to the control plane.

[0119] Step 304: Obtain the configuration information issued by the control plane to the Pod to be created.

[0120] Step 306: Use the plugin to allocate resources from the target heterogeneous device to the Pod to be created based on the configuration information.

[0121] The control plane is used to perform the steps of any of the embodiments of the first aspect described above.

[0122] Regarding step 302, after detecting a newly connected heterogeneous device, the worker node can obtain device attribute information by calling the device query interface provided by the manufacturer. As mentioned earlier, device attribute information may include device classification information, device resource quantity, device health status, and hardware topology relationships. After obtaining the device attribute information, the worker node converts it into a device resource object according to a common format and sends it to the control plane. For specific implementation details, please refer to the relevant description of step 202, which will not be repeated here.

[0123] Regarding step 304, after the control plane completes the scheduling decision, it writes the scheduling result (including the device identifier and resource allocation quota of the target heterogeneous device) into the Pod's configuration information in a plugin-compatible format and distributes it to the target worker node. The container proxy component on the worker node can obtain this configuration information after it detects that the Pod has been scheduled to this node.

[0124] In step 306, after the container proxy component on the worker node recognizes that the Pod has declared a resource request defined by the plugin, it invokes the corresponding plugin. The plugin reads the scheduling result from the configuration information and, based on the device identifier and resource allocation quota recorded therein, performs the actual device allocation operation. For example, for a GPU device, the plugin can mount the specified GPU into the Pod's container; for an RDMA network card device, the plugin can configure the specified network card into the Pod's network namespace. After the allocation is completed, the container proxy component completes the container creation and startup process according to the device injection information returned by the plugin.

[0125] During the Pod's lifecycle, resource management also involves resource reclamation. When a Pod is deleted, the scheduler on the control plane detects the deletion event, reads the previous device allocation records from the Pod's annotations, updates the internal device allocation data (ledger), and releases the occupied resources of the corresponding devices. Pod deletion follows the standard container management system deletion process without requiring additional interception or protocol conversion. This is because all necessary information is injected into the Pod's configuration during creation and accompanies the Pod throughout its lifecycle; upon deletion, the scheduler can directly read the allocation records to complete resource reclamation.

[0126] The following is combined with Figure 2C The diagram shown illustrates the container management system, and Figure 2DThe diagram illustrates the interaction of scheduling heterogeneous devices in a container management system. Taking the access of a new GPU device (hereinafter referred to as xGPU) in a K8s cluster as an example, it explains the workflow of the embodiments in this specification.

[0127] Figure 2C The diagram illustrates all relevant components and their interaction links involved in the system described in this embodiment. Some abbreviations are explained below: l / w: an abbreviation for List / Watch, representing a component continuously querying / listening to an object to obtain its latest state in real time; DP: Abbreviation for Device Plugin; PGS: an abbreviation for Physical GPU Spec; VGS: Abbreviation for Virtual GPU Spec.

[0128] The following section details the workflow of this system based on the diagram above. This is divided into two parts: the first part explains how heterogeneous computing devices are managed by this system (corresponding to...). Figure 2C The second part introduces how users can use the computing power of managed heterogeneous computing devices through this system (corresponding to the "background persistent link"). Figure 2C (The "Creating a Link for Computing Pods" section).

[0129] (I) Part One: Management of Heterogeneous Computing Power Devices (Permanent Backend Links) 1. Node access device: The K8s cluster administrator adds a node with a new heterogeneous computing device (taking a certain GPU as an example, hereinafter referred to as xGPU) to the K8s cluster, or installs a new heterogeneous computing device on an existing node.

[0130] 2. Node Component Identification and Processing: Each relevant component on this node identifies the device and begins executing its respective logic: a. Handling of vendor device plugins: Once the device plugin provided by the device manufacturer identifies the device (the identification logic is that it already has the capability; the information reported here is only the identifier and total resource amount, to let the container agent component (kubelet) know that it will be used for final resource mounting after specific resources have been scheduled), it reports the resource as a custom resource to the allocatable resource attribute in the Kubernetes node object (e.g., xxx.com / xgpu: 800; this resource quantity generally does not represent the actual physical resource quantity. Since a GPU can be shared by multiple Pods, it is usually reported as a large number to ensure that the Pod can be scheduled through Kubernetes general resource scheduling, for example, reporting the number of GPUs * 100). At the same time, the device plugin registers this resource with the container agent component (kubelet), that is, it declares that any Pod requesting this resource needs to be processed by it (i.e., the device plugin).

[0131] b. Handling of the first daemon process (IC Device Daemon) in this system: After the first daemon component of this system identifies the device (in this embodiment, a separate component is needed to implement different identification logics for devices from different manufacturers, using binary tools provided by the manufacturer or Linux tools (installed with the driver) for identification), it writes information such as the number of devices, resource quantity (e.g., computing power, video memory capacity), hardware topology (referring to the connection relationships between devices, indicating different communication speed levels between GPUs), and health status into the device information file (Device InfoFiles) on the node according to the general standard format defined by this system. This logic can be extended to identify different devices from various manufacturers. Regarding the file writing method, information for multiple devices (such as information for multiple GPUs and network cards on the node) can be written on a single node, or a single file can be written for each device; this embodiment does not limit this. It should be noted that GPU communication between nodes requires a network card, while GPU communication within a node does not require a network card; the role of health status information is that the scheduler will actively avoid unhealthy devices during scheduling, and maintenance personnel can understand the health status of devices by checking the device resource object (Device CR).

[0132] c. Handling of the second daemon process (IC Daemon) in this system: The second daemon of this system reads the contents of the device information file and converts it into a custom Kubernetes object called a Device Resource Object (Device CR) and submits it to the Kubernetes API Server (which can be understood as storing data in the cluster database) for use by other components of this system deployed in Kubernetes; each newly inserted heterogeneous device has an independent Device Resource Object.

[0133] 3. Control plane management component (IC Operator) processing: The control plane management component (IC Operator) of this system, deployed in the Kubernetes cluster, listens for newly created (or updated) device resource objects (Device CRs) via the List / Watch mechanism (this object is used by the scheduler when scheduling resources, as it needs to decide which heterogeneous resources to allocate to which Pod), and executes the following logic: a. Write node label: Write the basic device information (such as manufacturer, model, device type, etc.) from the device resource object into the label of the node object (Node) corresponding to the K8s node where the heterogeneous device is located.

[0134] b. Write Allocatable Resources: The device resource information (such as the number of GPUs, computing power, and video memory) from the device resource object is summarized and written to the Allocatable attribute of the node object. Since the device resource object is a generic definition, its custom resources are also generic. For example, regardless of the GPU manufacturer, its video memory resources are defined in a generic format like ic / gpu-memory: 128Gi. This writes the total resource amount of all heterogeneous devices on the node; typically, a node will only connect to one type of GPU and will not connect to multiple types simultaneously.

[0135] c. Creating a Physical Device Spec Object (PGS): If the device resource object contains a GPU, an additional Physical Device Spec Object (PGS) will be created specifically to represent that particular physical GPU. This PGS contains information such as the GPU's manufacturer, model, and physical resources. A PGS can be uniquely identified by "manufacturer + model," and the same PGS will not be created repeatedly (i.e., a manufacturer can have multiple GPU cards under a single model, but each corresponds to only one PGS).

[0136] d. Create a default Virtual Device Spec (VGS): For each new physical device spec object, a default virtual device spec object (VGS) representing that GPU card specification is created. The virtual device spec object contains its corresponding physical device spec object and virtual resource information. For a full-card specification virtual device spec object, its virtual resources are equivalent to the physical resources of the corresponding physical device spec object.

[0137] (2) Part Two, Computing Pod Creation Link.

[0138] 1. The user prepares the virtual device specification object: A user is preparing to submit a Pod that requires the computing power of the new heterogeneous computing devices xGPU described in Part 1. If the user expects the Pod to be allocated only a portion of the computing power of a specific xGPU card (e.g., 1 / 2 of the computing power and 1 / 4 of the video memory), then the user needs to pre-create the corresponding Virtual Device Specs (VGS). The virtual resources of this VGS are the resources that actually need to be allocated. If the user needs to use the entire card, they can directly use the default full-card VGS automatically created by the system.

[0139] 2. User submits Pod declaration: When a user submits a computing power Pod, the following information should be declared in the Pod's tags and annotations according to the general protocol defined by this system: a. The required virtual device specifications (the manufacturers and models supported by this platform must be provided to the user through pre-created virtual device specifications). b. For the specifications defined in the virtual device specification object in a, the required quantity (number of cards); c. The number of other general-purpose computing devices (such as RDMA network cards) required (if only one card is needed, then it is not necessary to declare the network card device and hardware topology requirements). d. Hardware topology requirements for computing devices, including topology requirements between the same type of devices and between different devices (e.g., requiring GPUs to be interconnected via NVLink at high speed, and GPUs and RDMA network cards to be located under the same PCIe Switch to achieve the shortest communication path, etc. The specific topology description method shall be agreed upon by the general protocol defined by this system).

[0140] 3. Control plane interception component (IC Webhook) interception and conversion: The control plane interception component (IC Webhook) of this system, deployed in the Kubernetes cluster, intercepts the creation request of the computing power Pod through the Kubernetes Webhook mechanism and makes the following modifications to the Pod: a. Based on the definitions of physical device specification objects and virtual device specification objects, convert the information from steps 2.a and 2.b into: i. Custom resource requests for Pods defined by this system and the vendor's device plugin (e.g., if a Pod requests two xGPUs with 1 / 4 of the video memory, and each xGPU has 128Gi of physical video memory, then it will ultimately be converted into resource requests defined by this system as ic / gpu: 2 and ic / gpu-memory: 128 / 4*2=64Gi, and xxx.com / xgpu: 2 defined by the vendor's device plugin). ii. Node Selector on the Pod used to select nodes for this type of GPU card model.

[0141] b. Convert the information in step 2.c into custom resource requests for the Pod as defined by this system and the vendor's device plugin (for example, if the Pod requests 2 RDMA network cards, it will eventually be converted into resource requests ic / rdma: 2 defined by this system and xxx.com / rdma: 2 defined by the vendor's device plugin).

[0142] 4. Control plane scheduler (IC Scheduler) scheduling decisions: The control plane scheduler (IC Scheduler) of this system, deployed in the Kubernetes cluster, schedules the computing Pods. The control plane scheduler is an extended Kubernetes scheduler implemented based on the Kubernetes Scheduler Framework. It includes all the scheduling plugins of the native Kubernetes scheduler (native scheduling plugins) and the general heterogeneous computing device scheduling plugins that are additionally extended by this system (heterogeneous device scheduling plugins).

[0143] The following explains the logic of the plugins that play a key role in the scheduling of computing power Pods: a. Native Node Affinity Plugin: Filters nodes with the required GPU model based on the node label selector on the Pod and the labels on the node objects.

[0144] b. Native Node Resource Plugin: Based on all custom resource requests on the Pod and allocable custom resources on the node object, filter out nodes with sufficient remaining computing power to meet the needs of the Pod.

[0145] c. The heterogeneous device scheduling plugin of this system (the native scheduler of Kubernetes can only find nodes that meet the conditions, while this embodiment needs to further determine which specific heterogeneous devices on which nodes to use): i. Based on the resource requests defined by this system on the Pod, the hardware topology requirements for computing devices declared in the Pod annotations, and the information of node objects and device resource objects, comprehensively consider the remaining resources of each device and the hardware topology between different devices, allocate the globally optimal node and its specific computing devices according to the Pod's requirements, and write the scheduling result; ii. For vendor device plugins that support obtaining device allocation results from the scheduler through extended scheduling mechanisms, additionally write the scheduling result according to the format natively recognized by the vendor device plugin (this logic can be extended to achieve adaptation to various vendor device plugins).

[0146] 5. The container proxy component (kubelet) requests device injection: When the container proxy component (kubelet) on the target node of the Pod scheduling hears that the Pod scheduling is complete, it also recognizes that the Pod has declared a resource request defined by the vendor's device plugin, and then requests the corresponding device plugin to obtain the required injected resources.

[0147] 6. Device plugin returns device information: Based on the scheduling results written in step 4.c.ii, the manufacturer's device plugin returns the corresponding device resources (such as the specific GPU card index, device path, etc.) according to the scheduler's allocation results.

[0148] 7. The container proxy component completes container creation: The container proxy component completes the remaining container creation and startup process according to the device injection information returned by the device plugin (this process is a native implementation of K8s, and you can refer to relevant technologies, which will not be elaborated here).

[0149] Figure 4 This is a schematic structural diagram of a device provided in an exemplary embodiment. For example... Figure 4As shown, device 400 mainly consists of a communication interface 402, a user interface 404, a processor 406, and a data storage 408. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 410. The communication interface 402 enables device 400 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 402 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 402 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 402 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 402 may also include multiple physical communication interfaces, such as Wi-Fi, Bluetooth, and wide-area wireless interfaces.

[0150] User interface 404 includes receiving user input and providing output to the user. Therefore, user interface 404 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 404 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 404 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, device 400 may support remote access from other devices via communication interface 402 or another physical interface (not shown). User interface 404 may be configured to receive user input, the position and movement of which may be indicated by an indicator or cursor described herein. User interface 404 may also be configured as a display device for rendering or displaying text fragments.

[0151] Processor 406 may contain one or more general-purpose processors and / or special-purpose processors.

[0152] Data storage 408 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 406. Data storage 408 may include removable and non-removable components.

[0153] Processor 406 is capable of executing program instructions 418 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 408 to perform the various functions described herein. Data storage 408 may comprise a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 400, enable device 400 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Processor 406 executing program instructions 418 may result in processor 406 using data 412.

[0154] For example, program instructions 418 may include an operating system 422 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 400 and one or more applications 420 (e.g., a browser, social application, or game application). Similarly, data 412 may include operating system data 416 and application data 414. Operating system data 416 is primarily accessible to the operating system 422, while application data 414 is primarily accessible to one or more applications 420. Application data 414 may reside in a file system visible or hidden from the user of device 400.

[0155] Application 420 can communicate with operating system 422 through one or more application programming interfaces (APIs). These APIs help application 420 read and / or write application data 414, transmit or receive information via communication interface 402, receive or display information on user interface 404, etc.

[0156] In some terminology, application 420 may be simply referred to as "app". Furthermore, application 420 can be downloaded to device 400 through one or more online app stores or app markets. However, applications can also be installed on device 400 in other ways, such as through a web browser or a physical interface on device 400 (e.g., a USB port).

[0157] like Figure 5 As shown, Figure 5 This specification is a block diagram illustrating an apparatus for scheduling heterogeneous devices in a container management system according to an exemplary embodiment. The apparatus is applied to the control plane of the container management system and includes: The device resource monitoring module 51 is used to: monitor the device resource objects sent by the worker nodes. The device resource objects are resource objects that the worker nodes convert into control plane-defined objects when they obtain the device attribute information of heterogeneous devices.

[0158] The resource information writing module 52 is used to: write the information of the accessed heterogeneous devices into the resource object of the working node based on a custom device resource object and in accordance with a control plane compatible protocol.

[0159] The request acquisition module 53 is used to: acquire Pod creation requests. The Pod creation requests contain user-declared resource requirement information. The resource requirement information is created by the user based on a protocol that is compatible with the heterogeneous devices that has been pre-published in the control plane.

[0160] The scheduling module 54 is used to: convert resource demand information into a data format compatible with the control plane protocol, and based on the conversion result and each resource object in the control plane, schedule target worker nodes and target heterogeneous devices that are adapted to the resource demand information from the cluster.

[0161] The result conversion module 55 is used to convert the scheduling result of the target heterogeneous device into a plugin-compatible format of the target heterogeneous device and write it into the configuration information corresponding to the Pod creation request, so as to trigger the target worker node to use the plugin to allocate the resources of the target heterogeneous device to the Pod to be created according to the configuration information.

[0162] like Figure 6 As shown, Figure 6 This specification is a block diagram illustrating an apparatus for scheduling heterogeneous devices on the worker node side according to an exemplary embodiment. The apparatus is applied to worker nodes in a worker node cluster managed by the control plane in a container management system, and includes: The device information reporting module 61 is used to: obtain the device attribute information of the accessed heterogeneous devices, convert the device attribute information into a device resource object customized by the control plane, and send it to the control plane.

[0163] Configuration acquisition module 62 is used to: acquire configuration information issued by the control plane to the Pod to be created.

[0164] The device allocation module 63 is used to: allocate resources of target heterogeneous devices to the Pod to be created based on the configuration information using the plugin.

[0165] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0166] Based on the same concept as the above methods, this specification also provides a container management system, the system including a cluster of worker nodes and a control plane for managing the cluster of worker nodes, wherein the non-native heterogeneous devices connected to the worker nodes are bound with plug-ins; the control plane is used to execute the steps of the method described in any of the above embodiments, and the worker nodes are used to execute the steps of the method described in any of the above embodiments.

[0167] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0168] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0169] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0170] What those skilled in the art will understand is: In this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded.

[0171] In this specification, “a,” “an,” and “the” do not specifically refer to the singular, but may also include the plural.

[0172] In this specification, ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0173] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0174] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0175] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0176] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

Claims

1. A method for scheduling heterogeneous devices in a container management system, the method being applied to the control plane of the system, the control plane being used to manage a cluster of worker nodes, the control plane deploying several resource objects for each worker node, and the non-native heterogeneous devices connected to the worker nodes being bound to plugins; the method comprising: Listen for the device resource object sent by the worker node. The device resource object is a resource object customized by the control plane that the worker node converts into the device attribute information when it obtains the device attribute information of the heterogeneous device. Based on the custom device resource object, and in accordance with the control plane compatible protocol, the information of the connected heterogeneous devices is written into the resource object of the working node. Obtain a Pod creation request, which contains user-declared resource requirement information. This resource requirement information is created by the user based on a protocol that is pre-published by the control plane and is compatible with connected heterogeneous devices. The resource requirement information is converted into a data format compatible with the control plane protocol. Based on the conversion result and each resource object in the control plane, target worker nodes and target heterogeneous devices adapted to the resource requirement information are scheduled from the cluster. The scheduling result of the target heterogeneous device is converted into a plugin-compatible format for the target heterogeneous device and written into the configuration information corresponding to the Pod creation request, so as to trigger the target worker node to use the plugin to allocate the resources of the target heterogeneous device to the Pod to be created according to the configuration information.

2. The method according to claim 1, wherein the working node is deployed with a first daemon process and a second daemon process; The first daemon process is used to: obtain the device attribute information by calling the device query interface provided by the manufacturer of the heterogeneous device, and write it to the device information file stored locally on the working node in a preset general format; The second daemon process is used to: read the device information file, convert it into a device resource object customized by the control plane, and submit it to the control plane; The device resource object being monitored sent by the worker node includes: Listen for device resource objects submitted by the second daemon process on the worker node.

3. The method according to claim 1, wherein the heterogeneous devices connected to the working node are one or more; The step of writing the information of the connected heterogeneous devices into the resource object of the working node includes: Extract the device classification identifier of the heterogeneous device contained in each of the device resource objects, and write it into the node tag of the resource object of the working node; After summarizing the resource quantity information contained in each of the device resource objects, it is written into the allocatable resource attribute of the resource object of the working node.

4. The method according to claim 1, wherein the pre-published protocols compatible with connected heterogeneous devices in the control plane include: Device specifications that have been connected to heterogeneous devices; The method further includes: Based on the equipment classification identifiers of heterogeneous devices contained in the equipment resource object, the equipment specification object is generated; The device specification object uniquely corresponds to the device category identifier and is used to describe the resource attributes of the heterogeneous device of that model. This is used by the user before submitting the Pod creation request to declare the resource requirement information according to the device specification object.

5. The method according to claim 4, for heterogeneous devices that provide resources as a whole on a single device basis, the device specification object includes: A physical device specification object used to describe the physical resource attributes of heterogeneous devices; For heterogeneous devices that support resource partitioning of a single device, the device specification object includes: the physical device specification object and one or more corresponding virtual device specification objects. The virtual device specification object is used to describe the allocatable virtual resource quota for Pod instances, and the virtual resource quota is less than or equal to the physical resource represented by the corresponding physical device specification object.

6. The method according to claim 1, wherein the working node is deployed with a native container proxy component; The step of converting the scheduling result of the target heterogeneous device into a plugin-compatible format for the target heterogeneous device and writing it into the configuration information corresponding to the Pod creation request, so as to trigger the target worker node to use the plugin to allocate resources of the target heterogeneous device to the Pod to be created according to the configuration information, includes: Determine the scheduling protocol type of the plug-in bound to the target heterogeneous device; Based on the scheduling result, determine the device identifier of the target heterogeneous device and the resource allocation quota corresponding to the target heterogeneous device in the Pod to be created; According to the data format agreed upon by the scheduling protocol type, the device identifier and resource allocation quota of the target heterogeneous device are written into the annotation corresponding to the Pod creation request. This allows the container proxy component to call the plugin when it detects that the Pod to be created is scheduled to the target worker node and recognizes the device identifier in the annotation. The plugin then allocates the resources of the target heterogeneous device to the Pod to be created based on the device identifier and resource allocation quota of the target heterogeneous device.

7. The method according to claim 1, wherein the device attribute information includes: The heterogeneous devices include device classification information, device resource quantity, device health status, and hardware topology relationships between heterogeneous devices; wherein, the hardware topology relationships characterize the connection methods between heterogeneous devices in the working node; And / or, The heterogeneous devices include: a graphics processor and a network interface card (NIC) for connecting and enabling direct memory access between the graphics processors.

8. The method according to any one of claims 1 to 7, wherein the scheduler deployed on the control plane includes a native scheduling plugin and a heterogeneous device scheduling plugin; The process of scheduling target worker nodes and target heterogeneous devices adapted to the resource requirement information from the cluster based on the transformation results and resource objects in the control plane includes: Using the native scheduling plugin, based on the transformation results and the resource objects of each worker node, the worker nodes in the cluster are initially filtered to obtain a set of candidate nodes; The heterogeneous device scheduling plugin schedules the target working node and the target heterogeneous device that are adapted to the resource requirement information from the candidate node set based on the device resource objects on each working node in the candidate node set.

9. The method according to claim 8, wherein converting the resource requirement information into a data format compatible with the control plane protocol comprises: The resource demand information is converted to obtain the target heterogeneous device classification identifier and resource request quantity. The target heterogeneous device classification identifier is used to generate node label filtering conditions for the Pod to be created. The node label filtering conditions are used to limit the scheduling scope to working nodes whose node labels contain the target heterogeneous device classification identifier. The preliminary filtering of each worker node in the cluster includes: The worker nodes in the cluster are filtered according to the node label filtering criteria and the resource request volume.

10. The method according to claim 8, wherein the device attribute information includes: The heterogeneous devices include device classification information, device resource quantity, device health status, and hardware topology relationships between heterogeneous devices; wherein, the hardware topology relationships represent the connection methods between heterogeneous devices in the same working node; The control plane maintains device allocation data that records the allocation status of each heterogeneous device. The step of scheduling the target working node and the target heterogeneous device that match the resource requirement information from the candidate node set based on the device resource objects on each working node in the candidate node set includes: Based on the resource availability of each heterogeneous device recorded in the device allocation data record, and one or more pieces of information such as device health status and hardware topology relationship between devices recorded in the device resource object, target working nodes and target heterogeneous devices that are compatible with the resource requirement information are scheduled from the candidate node set.

11. The method according to claim 10, wherein scheduling target worker nodes and target heterogeneous devices adapted to the resource requirement information from the candidate node set comprises: Based on the device resource objects of each heterogeneous device accessed by the candidate node set, heterogeneous devices marked as unhealthy are filtered out. After filtering, if the resource requirement information contains multiple heterogeneous devices and includes constraints on the hardware topology between heterogeneous devices, then based on the hardware topology relationships recorded by each device resource object, a target heterogeneous device that satisfies the constraints and has the best inter-device communication efficiency is selected from the heterogeneous devices of the candidate nodes.

12. A method for scheduling heterogeneous devices in a container management system, the method being applied to worker nodes in a worker node cluster managed by a control plane in the system, wherein the non-native heterogeneous devices connected to the worker nodes are bound to plugins; the method comprising: Obtain the device attribute information of the connected heterogeneous devices, convert the device attribute information into a device resource object customized by the control plane, and send it to the control plane; Obtain the configuration information issued by the control plane to the Pod to be created; The plugin is used to allocate resources from target heterogeneous devices to the Pod to be created based on the configuration information. The control surface is used to perform the steps of the method according to any one of claims 1 to 11.

13. A container management system, the system comprising a cluster of worker nodes and a control plane for managing the cluster of worker nodes, wherein non-native heterogeneous devices connected to the worker nodes are bound to plugins; the control plane is used to execute the steps of the method according to any one of claims 1 to 11, and the worker nodes are used to execute the steps of the method according to claim 12.

14. An electronic device, characterized in that, include: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as claimed in any one of claims 1 to 12 by executing the executable instructions.

15. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 12.

16. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 12.

Images