A device topology adaptive scheduling method, electronic device, medium and product

Abstract

The application discloses a device topology adaptive scheduling method, an electronic device, a medium and a product, relates to the technical field of computers, and acquires the physical topology relationship between the device plug-ins and constructs a device topology tree by configuring topology-aware device plug-ins in nodes, so that the system can perform perceptive device allocation based on topology information when a container group is created. The method judges whether the devices applied by the container group are located in the same topology domain in the allocation stage, returns the allocation result only when the devices are all located in the same topology domain, and automatically re-applies the devices otherwise, so that the topology proximity of the devices in the container group is ensured. The data transmission delay and bandwidth competition between the devices are significantly reduced, the internal communication efficiency of the container group is improved, the efficiency, predictability and adaptability of device scheduling are realized, and the problems of low communication efficiency and slow service response in the traditional scheme are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a device topology adaptive scheduling method, an electronic device, a medium and a product. BACKGROUND

[0002] According to the existing device plugin mechanism of k8s, a pod only needs to fill in the number of resources applied for to apply for device resources, and then the node proxy component (kubelet) will automatically allocate GPU device resources to the pod. For the device plugin of GPU, some manufacturers have implemented internal topology-aware allocation. When a pod applies for multiple GPUs, the devices within the same topology domain (pciswitch) are preferentially allocated. However, for cross-manufacturer device allocation, different device plugins are often involved, and there is no suitable solution in the industry; for example, a pod simultaneously applies for an RDMA network card and two GPUs, and it cannot be ensured that the devices allocated to the pod are within the same pciswitch. Therefore, when a user creates a pod to apply for multiple devices, the devices are randomly allocated by the respective device plugins, and it cannot be ensured that the devices allocated to the pod are within the same pciswitch, resulting in low efficiency of data communication between devices and reducing the response speed of model inference services in large model inference scenarios. SUMMARY

[0003] The present application provides a device topology adaptive scheduling method, an electronic device, a medium and a product to at least solve the problem in the related art that when a user creates a pod to apply for multiple devices, the devices are randomly allocated by the respective device plugins, and it cannot be ensured that the devices allocated to the pod are within the same topology domain, resulting in low efficiency of data communication between devices and reducing the response speed of model inference services in large model inference scenarios.

[0004] The present application provides a device topology adaptive scheduling method, in which a topology-aware device plugin, a proxy component and at least one device plugin are set on a node, and the method comprises:

[0005] The topology-aware device plugin is used to obtain the topology relationship between the device plugins of the node, and a device topology tree is formed for each node;

[0006] In response to a user applying to create a pod, a target node is selected from multiple nodes, and a task of creating a pod is allocated to the target node;

[0007] The proxy component of the target node obtains the number of devices contained in each container of a pre-created pod, and each container applies for the corresponding number of devices from the device plugins one by one through the topology-aware device plugin, and obtains multiple target devices returned by the application;

[0008] determine whether the multiple target devices returned by the application are in the same topology domain according to the device topology tree of the target node;

[0009] In response to the multiple target devices returned by the application being in the same topology domain, return the device allocation result to the agent component, and the agent component creates a container group according to the device allocation result, otherwise, reapply for devices to the device plugin.

[0010] The application further provides an electronic device, comprising a memory for storing a computer program and a processor for executing the computer program to implement the steps of the device topology adaptive scheduling method.

[0011] Obtain the topology relationship between the device plugins of the node through the topology-aware device plugin, and form a device topology tree for each node.

[0012] In response to a user applying to create a container group, select a target node from multiple nodes, and allocate the task of creating a container group to the target node.

[0013] The agent component of the target node obtains the number of devices contained in each container in a pre-created container group, applies for a corresponding number of devices to the device plugin for each container through the topology-aware device plugin, and obtains multiple target devices returned by the application.

[0014] Determine whether the multiple target devices returned by the application are in the same topology domain according to the device topology tree of the target node.

[0015] In response to the multiple target devices returned by the application being in the same topology domain, return the device allocation result to the agent component, and the agent component creates a container group according to the device allocation result, otherwise, reapply for devices to the device plugin.

[0016] The application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the device topology adaptive scheduling method.

[0017] Obtain the topology relationship between the device plugins of the node through the topology-aware device plugin, and form a device topology tree for each node.

[0018] In response to a user applying to create a container group, select a target node from multiple nodes, and allocate the task of creating a container group to the target node.

[0019] The agent component of the target node obtains the number of devices contained in each container in the pre-created container group, and applies for the corresponding number of devices from the device plugin one by one through the topology-aware device plugin for each container, to obtain a plurality of target devices returned by the application;

[0020] According to the device topology tree of the target node, it is judged whether the plurality of target devices returned by the application are in the same topology domain;

[0021] In response to the plurality of target devices returned by the application being in the same topology domain, a device allocation result is returned to the agent component, and the agent component creates a container group according to the device allocation result, otherwise, the device is re-applied to the device plugin.

[0022] The application also provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned device topology adaptive scheduling methods:

[0023] The topology relationship between the device plugins of the node is obtained through the topology-aware device plugin, and a device topology tree is formed for each node;

[0024] In response to a user applying to create a container group, a target node is selected from a plurality of nodes, and a container group creation task is assigned to the target node;

[0025] The agent component of the target node obtains the number of devices contained in each container in the pre-created container group, and applies for the corresponding number of devices from the device plugin one by one through the topology-aware device plugin for each container, to obtain a plurality of target devices returned by the application;

[0026] According to the device topology tree of the target node, it is judged whether the plurality of target devices returned by the application are in the same topology domain;

[0027] In response to the plurality of target devices returned by the application being in the same topology domain, a device allocation result is returned to the agent component, and the agent component creates a container group according to the device allocation result, otherwise, the device is re-applied to the device plugin.

[0028] By configuring a topology-aware device plugin in a node, obtaining the physical topology relationship between device plugins, and constructing a device topology tree, the system can perform aware device allocation based on topology information when a container group is created. This method determines whether the devices applied for by the container group are located in the same topology domain during the allocation stage, and only returns the allocation result when the devices are in the same topology domain, otherwise automatically re-applies for devices, thereby ensuring that the devices in the container group have topology proximity. In this way, the cross-topology domain communication problem caused by random allocation of device plugins in the prior art is avoided, the data transmission delay and bandwidth competition between devices are significantly reduced, and the internal communication efficiency of the container group is improved. In scenarios such as large model inference that require high parallel computing and communication performance, this method can effectively improve the throughput and response speed of model inference tasks, realize the efficiency, predictability and adaptability of device scheduling, and solve the problems of low communication efficiency and slow service response in traditional solutions. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0030] Figure 1 A device plugin working timing diagram in the existing k8s;

[0031] Figure 2 An application environment diagram of the device topology adaptive scheduling method in one embodiment of the present application;

[0032] Figure 3 A flowchart of the device topology adaptive scheduling method in one embodiment of the present application;

[0033] Figure 4 A structure diagram of setting two device topology trees in one node in one embodiment of the present application;

[0034] Figure 5 A flowchart of the step of determining whether the multiple target devices returned by the application are in the same topology domain according to the device topology tree of the target node in one embodiment of the present application;

[0035] Figure 6 A decision process diagram of the interface information decision module in one embodiment of the present application;

[0036] Figure 7 A structure block diagram of the device topology adaptive scheduling device in one embodiment of the present application;

[0037] Figure 8 Figure 1 is a diagram of an internal structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0039] It should be noted that in the description of the present application, the terms "comprise", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms "first", "second" and the like in the present application are used to distinguish similar objects, not to describe a specific order or sequence.

[0040] In order to make the skilled in the art better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] Kubernetes(k8s) is an open-source container orchestration platform developed by Google. It is based on the Google's internal container cluster management system (Borg) and provides a complete set of functions for containerized applications, such as deployment, resource scheduling, service discovery, and dynamic scaling. It improves the convenience of large-scale container cluster management. Kubernetes is a complete distributed system support platform with complete cluster management capabilities, multi-layer security protection and access mechanism, multi-tenant application support, transparent service registration and discovery mechanism, built-in intelligent load balancer, powerful fault detection and self-repair capabilities, service rolling upgrade and online expansion capabilities, scalable resource automatic scheduling mechanism, and multi-granularity resource quota management capabilities. At the same time, Kubernetes provides complete management tools covering all aspects of development, deployment testing, and operation and maintenance. With the gradual landing of deep learning research in various fields, the industry has begun to explore a complete system from data acquisition to model training and model landing, reducing the cost of computing resources for model training and shortening the cycle of model development. To achieve this goal, major cloud service providers have launched their own deep learning cloud platforms. These deep learning cloud platforms solve the problems of high computing resource cost, lack of unified resource monitoring system, and algorithm personnel needing to participate in operation and maintenance (such as setting up hardware and software environment) caused by traditional methods of training directly on PC or server. At the same time, they are based on k8s unified resource monitoring and scheduling, which greatly improves the resource utilization of the computing center and effectively reduces the cost.

[0042] GPU stands for Graphics Processing Unit, which is a type of processor designed specifically for accelerating image and video processing, 3D rendering, and scientific computing tasks. Compared to traditional central processing units (CPUs), GPUs have more powerful processing capabilities and can handle large amounts of data and perform parallel computing quickly. Initially, GPUs were widely used in computer games and graphics design fields, but with the rise of artificial intelligence and machine learning, GPUs have become an important part of high-performance computing and scientific computing.

[0043] The architecture of a GPU is different from that of a CPU. The core design idea of a GPU is to organize a large number of computing units in parallel to achieve efficient parallel computing. A modern GPU can have thousands of computing units, each with its own memory and control unit, and can execute instructions independently. This highly parallel design makes the GPU perform very well in scientific computing and machine learning. In order to better support scientific computing and machine learning applications, GPU manufacturers have also developed a series of software and tools, such as CUDA, OpenCL, etc., to make it easier for developers to program and compute using GPUs.

[0044] In a computer, a GPU is usually connected to the motherboard using a PCIe interface. The PCIe interface provides a high-bandwidth, low-latency data transmission channel, allowing the GPU to quickly read and write data in the main memory. Pci (Peripheral Component Interconnect) is a computer hardware interface standard used to connect various external devices inside a computer, such as graphics cards, network cards, sound cards, etc. At the computer operating system level, you can easily query the pci device information connected to the current motherboard using commands such as lspci, where the key information is the pci id. The ID of a PCI device is a set of naming specifications developed by the PCI-SIG (PCI Special Interest Group) organization to identify the manufacturer and device model of a PCI device. The PCI device ID consists of two 16-bit hexadecimal numbers, the first number representing the manufacturer ID, and the second number representing the device ID. For example, the ID 0x10DE:0x1B81 represents the manufacturer ID as NVIDIA (0x10DE) and the device ID as GeForce GTX 1080 Ti (0x1B81).

[0045] Based on the existing interpretation of the fields of the lspci command, through the combination of bus:slot.func, a physical device can be uniquely determined, and the manufacturer and device model and version information of the device can be determined according to the pci number of the device.

[0046] In the field of high-performance GPU computing, key components such as CPUs, memory modules, NVMe storage devices, GPUs, and network adapters are connected efficiently and smoothly through PCIe (Peripheral Component Interconnect Express) buses or specially designed PCIe switch chips. After five generations of technological innovation, the latest Gen5 version ensures highly efficient interconnection performance between devices. For AI servers, there may be multiple pci switches between different pci devices in the physical layer, each of which connects multiple different devices to achieve effective expansion of pci devices. In the physical topology diagram of a typical 8-card server, two CPUs manage two pci switches, each of which manages two GPUs, two IB network cards, and two NVME solid-state disks, forming efficient interconnection of in-node devices; the communication efficiency of devices within the same pci switch is much higher than that of devices across pci switches

[0047] Device Plugin is a mechanism in Kubernetes that allows users to register custom devices in the cluster and expose them to Pods for use. It is implemented through custom kubelet plugins and provides device information to the Kubernetes cluster through the Kubelet API. This mechanism can be used to support various hardware devices such as GPUs, FPGAs, and TPUs. Among them, Pod is the application that the workload runs on Kubernetes.

[0048] Device Plugin can help Kubernetes better manage node resources, allowing applications to use hardware devices for accelerated computing. Its working principle is to register device plugins in the cluster and inform Kubernetes of the type and quantity of devices. Kubernetes can automatically schedule Pods to nodes with the required devices based on the information provided by the device plugin. During Pod execution, Device Plugin creates corresponding device files or sockets within the container for application use. Socket is a combination of IP address and port, providing a mechanism for transmitting data packets to application layer processes.

[0049] For GPU devices, Device Plugin can manage GPU resources through Nvidia GPU device plugins or other third-party plugins. These plugins can detect and allocate available GPU resources so that applications can use them for computation. At the same time, Device Plugin also supports resource quotas and resource reservations to ensure GPU resource isolation and fair allocation between different applications.

[0050] AsFigure 1 as shown, Figure 1 is a device plugin working timing diagram in k8s, the device plugin sends the device resources on the current node to the node agent (kubelet) plugin to register the device, the kubelet plugin updates the resource state through the node state and feeds back to all request unified entrances (API servers), users create pods (containers), request device resources from all request unified entrances, all request unified entrances schedule pods according to the node state, and call allocators to allocate resources.

[0051] With the continuous maturity of cloud native technology and AI computing technology, it has become a common practice for pods on k8s to use various acceleration devices for AI inference computing, and the way pods use GPUs has become very flexible with business needs. A pod can mount multiple GPU devices and multiple smart network cards at the same time to provide model loading and inference services externally.

[0052] According to the existing deviceplugin mechanism of k8s, pod application for using device resources only needs to fill in the number of resources applied, and then kubelet will automatically allocate GPU device resources to the pod for use. Through the existing mechanism, for the allocation of GPUs in the node, it is actually completed on kubelet and deviceplugin. The current interface design of kubelet can only handle quantity requests, such as the number of RDMA network cards, the number of GPU cards, etc. For the deviceplugin of GPU, some manufacturers have implemented internal topology-aware allocation. When a pod applies for multiple GPUs, the devices within the same pciswitch are preferentially allocated. However, for cross-vendor device allocation, different device-plugins are often involved, and there is no suitable solution in the industry; for example, a pod simultaneously applies for an RDMA network card and two GPUs, which cannot ensure that the devices allocated to the pod are within the same pciswitch. Therefore, users need to be able to create pods to apply for multiple devices, and need to be able to ensure that the devices allocated to the pod are within the same pciswitch as much as possible, rather than being randomly allocated by the respective deviceplugin, to avoid the problem of low efficiency of data communication between devices.

[0053] The device topology adaptive scheduling method provided by the present application can be applied to, for example, Figure 2The application environment shown. Among them, by increasing a device-plugin intermediate layer that realizes cross-device topology awareness: topology-aware device plugin, downward compatible with device plugins (device-plugin) of multiple manufacturers in the form of plugins, it can guarantee that when the device resources are sufficient, multiple devices within the same pci switch are allocated to the pod, thereby improving the efficiency of data communication between devices, and in large model inference scenarios, the response speed of model inference services can be greatly improved, without modifying other deviceplugin code, making up for the gap in the community solution, and having high practical value.

[0054] Among them, the node is provided with a topology-aware device plugin, a proxy component and a peripheral component interconnect express bus (PCIe bus), the topology-aware device plugin includes an interface information decision module and a resource key module, the information of device plugins of multiple manufacturers is obtained through the interface information decision module, registration is completed to the proxy component kubelet, the device plugin is not directly managed, but the resource key resourcekey created by other device plugins deviceplugin is listened to, a fixed prefix "topoaware" (topology awareness) is added in front, and registration is performed on the node as a new resourcekey; the number of keys changes with the change of the corresponding key in the normal deviceplugin; when the pod applies for a device, if the resourcekey applied has the topoaware prefix, the request will be sent to the topology-aware device plugin (topoaware-deviceplugin) for additional pci topology judgment, and finally the device information is returned to the pod by the topoaware-deviceplugin.

[0055] The resource key module is mainly responsible for the creation and update of the resource key. The module has three main functions:

[0056] 1) When the deviceplugin is initialized, the list of device plugin device plugins already existing in the node is queried to obtain the native resourcekey, for example, hygon.com / dcu;

[0057] 2) The topology-aware resourcekey (resource key) of the node (node) is registered, and the format is that the prefix is added to the native resourcekey, for example, topoaware.hygon.com / dcu, and the same number is registered to the kubelet, and the kubelet adds resource information to the node;

[0058] 3) Listen to the change of the number of resources corresponding to the original resourcekey, and update the value corresponding to the topology-aware resourcekey, for example, hygon.com / dcu is changed to 6, then update the value of topoaware.hygon.com / dcu to 6.

[0059] The interface information decision module is mainly responsible for obtaining the topology relationship between the devices of the node, making a decision on the rationality of the topology, and reporting the use of the pci device of the node and the topology information to the topology-aware scheduler (topoaware-scheduler);

[0060] As shown in Figure 3 The embodiments of the present application provide a device topology adaptive scheduling method, which sets a topology-aware device plug-in, an agent component and at least one device plug-in on a node, and the method comprises the following steps:

[0061] Step S1, obtaining the topology relationship between the device plug-ins of the node through the topology-aware device plug-in, forming a device topology tree for each node;

[0062] Step S2, in response to a user's application for creating a container group, selecting a target node from the multiple nodes and assigning the task of creating the container group to the target node;

[0063] Step S3, the agent component of the target node obtains the number of devices contained in each container in the pre-created container group, and applies for the corresponding number of devices to the device plug-in one by one through the topology-aware device plug-in for each container, and obtains multiple target devices returned by the application;

[0064] Step S4, judging whether the multiple target devices returned by the application are in the same topology domain according to the device topology tree of the target node;

[0065] Step S5, in response to the multiple target devices returned by the application being in the same topology domain, returning the device allocation result to the agent component, and the agent component creates the container group according to the device allocation result, otherwise reapplying for the devices to the device plug-in.

[0066] This method involves configuring topology-aware device plugins in nodes to acquire the physical topology relationships between device plugins and construct a device topology tree. This enables the system to perform topology-aware device allocation based on topology information when container groups are created. During the allocation phase, the method determines whether the devices requested by the container group are located within the same topology domain. Allocation results are only returned if all devices are in the same topology domain; otherwise, devices are automatically re-requested, ensuring topological proximity among devices within the container group. This approach avoids cross-topology domain communication problems caused by random device plugin allocation in existing technologies, significantly reducing data transmission latency and bandwidth contention between devices, and improving communication efficiency within the container group. In scenarios with extremely high requirements for parallel computing and communication performance, such as large model inference, this method effectively improves the throughput and response speed of model inference tasks, achieving high efficiency, predictability, and adaptability in device scheduling, and solving the problems of low communication efficiency and slow service response in traditional solutions.

[0067] In this context, a topology domain is a set of devices with similar network characteristics located under a gateway device, preferably a switch. These devices are logically grouped into the same area and enjoy similar network services and policies. Through a topology-aware device plugin, the system can automatically identify and classify the topology domain to which a device belongs, and then intelligently allocate devices based on this key information when creating a container group. This topology domain-based device allocation strategy not only optimizes the utilization of network resources but also further enhances the overall performance and stability of the system.

[0068] In this embodiment, the topological relationships between device plugins of this node are obtained through the topology-aware device plugin, and a device topology tree is formed for each node, including:

[0069] The topology-aware device plugin obtains the device plugin information of the node and the topological relationship between the device plugins.

[0070] Based on the device plug-in information, the device information and device type attributes are filled into the topology relationship to form a device topology tree.

[0071] In this way, by forming a structured topology tree to represent the connection hierarchy of devices within a node, accurate modeling of device resources can be achieved, enabling subsequent scheduling to make topology judgments at the logical structure level, thereby improving scheduling accuracy and the spatial rationality of device allocation.

[0072] like Figure 4 As shown, Figure 4 This demonstrates the structure of setting up two device topologies in a single node, with the root node of the device topology tree being the switch and the child nodes being device plug-ins.

[0073] In the embodiment, the device plugin information of the node and the topology relationship between the device plugins are obtained by the topology-aware device plugin, including:

[0074] The extended interface ID information corresponding to the device plugin ID is obtained by the topology-aware device plugin using the device driver command.

[0075] The topology relationship between the device plugins connected by the extended interface ID of the node is obtained using the operating system command.

[0076] The physical connection relationship is accurately detected by the system-level instruction, the accuracy and real-time performance of the topology information are ensured, the generated topology structure can truly reflect the bottom-layer hardware connection state, and reliable basic data support is provided for the topology adaptive scheduling.

[0077] In the embodiment, the device information and the device type attribute are filled for the topology relationship based on the device plugin information, including:

[0078] The native resource key registered by the device plugin in the agent component is obtained according to the device plugin information.

[0079] The resource quantity corresponding to the native resource key is obtained as the device information of the device plugin.

[0080] The device type and name corresponding to the device plugin are obtained as the device type attribute of the device plugin.

[0081] The topology tree not only reflects the connectivity, but also has the expression ability of the resource dimension by combining the topology structure with the device attribute, so that the scheduler can consider the device type matching and the quantity constraint when making a decision, and the multi-dimensional resource optimization allocation is realized.

[0082] In the embodiment, the native resource key registered by the device plugin in the agent component is obtained according to the device plugin information, including:

[0083] When the topology-aware device plugin is initialized, the system file directory of the node is scanned, and all registered native resource key instances in the agent component are identified based on the scanning result, a native resource key list is generated and cached in the local memory.

[0084] The resource key information of the node is cached, so that the system can quickly access the device registration state and reduce the overhead of repeated queries, thereby improving the device information acquisition efficiency and providing data support for real-time topology update.

[0085] In the embodiment, the resource quantity corresponding to the native resource key is obtained as the device information of the device plugin, including:

[0086] A prefix resource key is formed by adding a prefix to the native resource key, and the prefix resource key is registered on the agent component as a new resource key.

[0087] obtaining a resource quantity corresponding to the native resource key, and setting a value of the prefix resource key according to the resource quantity corresponding to the native resource key;

[0088] In the filling of the device information for the topology relationship, the value of the prefix resource key is used as the device information of the device plugin to represent the resource quantity provided by the device plugin.

[0089] The prefix resource key mechanism is established to realize hierarchical management and index expansion of the device resource information, improve the uniqueness and traceability of the device resource identification, and thus enable the scheduler to more efficiently parse and access the node device resources.

[0090] In the embodiment, the method further includes:

[0091] The topology-aware device plugin periodically listens to the resource quantity corresponding to the native resource key, and when the resource quantity of any device plugin changes, the value of the prefix resource key is updated according to the resource quantity corresponding to the native resource key, and the update event is reported to the topology-aware scheduler of the container orchestration scheduling platform through the interface information decision module.

[0092] Through real-time listening and reporting of the resource state, the system can timely reflect the change of device availability, realize dynamic consistency of the topology information and the actual hardware state, and thus guarantee the real-time effectiveness of the scheduling decision and the self-adaptability of the scheduling system.

[0093] In the embodiment, the target node is selected from the plurality of nodes, including:

[0094] The occupied quantity and the idle quantity of the device topology tree in each node are obtained to determine the remaining resource quantity of each node;

[0095] The node with the largest remaining resource quantity is obtained from the plurality of nodes as the target node;

[0096] In response to the existence of a plurality of nodes with the largest remaining resource quantity, the integrity score of the device topology tree of the node with the largest remaining resource quantity is obtained;

[0097] The node with the largest integrity score of the device topology tree is taken as the target node.

[0098] Through the introduction of the dual evaluation standards of the topology integrity and the remaining resource quantity, the node selection is more scientific and reasonable, and thus the problem of resource concentration or uneven distribution is effectively avoided, and the overall resource utilization and the computing load balancing of the cluster are improved.

[0099] In the embodiment, in response to the existence of a plurality of nodes with the largest remaining resource quantity, the integrity score of the device topology tree of the node with the largest remaining resource quantity is obtained, including:

[0100] Obtaining the number of unused devices and the total number of devices in each node within the same topology domain;

[0101] Determining the idle rate h of the topology domain by dividing the number of unused devices by the total number of devices within the same topology domain;

[0102] Determining the score of the device topology tree of each node by score =∑e 5*h Determining the integrity score of the device topology tree of each node.

[0103] The intra-node topology health is quantified by the idle rate at the switch level, ensuring that the selected node has high communication aggregation and low device conflict probability, thereby further optimizing the deployment efficiency of the container group at the physical topology level.

[0104] In this embodiment, the agent component of the target node obtains the number of devices contained in each container in the pre-created container group, and applies for the corresponding number of target devices from the device plugin for each container through the topology-aware device plugin one by one, to obtain a plurality of target devices returned by the application, including:

[0105] The agent component of the target node obtains the number of containers contained in the pre-created container group;

[0106] Obtaining the number of devices contained in each container;

[0107] Assigning a device plugin to each container in turn, and applying for the corresponding number of target devices from the respective corresponding device plugin through the topology-aware device plugin for each container;

[0108] Determining whether the containers in the pre-created container group all return the target devices applied for by each container;

[0109] If yes, proceed to the next step, if not, reassign a new device plugin to the container that does not return the target device applied for, and apply for the corresponding number of target devices from the new device plugin.

[0110] The fine-grained control and dynamic compensation of resource allocation between containers are realized through the one-by-one device application and retry mechanism at the container level, the device allocation success rate is improved, and the integrity and stability of the container group creation are ensured.

[0111] In this embodiment, a device plugin is assigned to each container in turn, including:

[0112] Obtaining the load rate of the device plugin of the node, and arranging the device plugins in the node in ascending order according to the load rate to form a first device plugin list;

[0113] Arranging the containers in the pre-created container group in descending order according to the number of devices contained in each container to form a first container list;

[0114] corresponding to each container in the first container list, a device plugin is assigned to each container;

[0115] When the number of containers in the first container list is greater than the number of device plugins in the first device plugin list, after the device plugin assignment in the device plugin list is completed, the containers in the first container list that are not assigned a device plugin are sorted to form a second container list, the load rate of the device plugins of the node is reacquired, and the device plugins in the node are arranged in ascending order according to the load rate to form a second device plugin list;

[0116] corresponding to each container in the first container list, a device plugin is assigned to each container.

[0117] Wherein, the device assignment sequence is optimized by the load balancing algorithm, the device plugin overload is maximally avoided, the resource scheduling is more fair and efficient, and the node internal device utilization and task parallel execution performance are improved.

[0118] As shown in Figure 5 In the embodiment, it is judged whether the multiple target devices returned according to the container application are in the same topology domain according to the device topology tree of the target node, including:

[0119] At least one target device returned according to the first container application is parsed, and it is judged whether the at least one target device returned according to the first container application is in the same topology domain;

[0120] In response to the at least one target device returned according to the first container application being in the same topology domain, at least one target device returned according to the second container application is parsed, and it is judged whether the at least one target device returned according to the second container application is in the same topology domain as the at least one target device returned according to the first container application, and the cycle is repeated in turn until the at least one target device returned according to all container applications of the pre-created container group is in the same topology domain;

[0121] In response to the at least one target device returned according to any target container application not being in the same topology domain, a new device plugin is re-assigned to the target container, and the corresponding number of target devices of the target container is re-applied.

[0122] Wherein, the physical location proximity of the devices in the same container group is ensured, thereby reducing the network overhead and delay caused by cross-topology domain communication, and making the container group have higher data transmission efficiency and predictable performance when running.

[0123] In the embodiment, the re-assignment of a new device plugin to the target container includes:

[0124] If the device plug-ins in the same topology domain in the device topology tree are all allocated, the device in the adjacent topology domain is selected as the target container to re-allocate the new device plug-in according to the shortest topology distance principle.

[0125] The connection tightness between devices is maintained by the shortest topology distance strategy, the delay compensation is minimized when allocating across the topology domain, the communication cost is effectively reduced, and the continuity of task calculation is maintained.

[0126] As shown in the embodiment, in response to the at least one target device returned by the first container application being in the same topology domain, if the at least one target device returned by the other container application is not in the same topology domain, the method further comprises: Figure 6

[0127] The target node applies a target device to the original device plug-in, obtains a target device ID, finds an interface device topology tree of the node, and judges whether the target device ID is in the same topology domain;

[0128] If yes, the target device ID is returned to create a target container and release the devices corresponding to the other IDs;

[0129] If no, the current device is kept without being released, and the device plug-in is re-applied for a device, it is judged whether a new device is returned, if a new device is returned, a new device ID is obtained, the interface device topology tree of the node is found, and it is judged whether the new device ID is in the same topology domain, if no new device is returned, the current device is returned to the other container.

[0130] The scheme realizes the fault tolerance and dynamic correction mechanism of device allocation, avoids the scheduling interruption caused by single allocation failure, and improves the stability and task creation success rate of the system.

[0131] In the embodiment, the device allocation result is returned to the agent component, and the agent component creates a container group according to the device allocation result, including:

[0132] The target device ID in the device allocation result is mapped to the runtime instance of the created container;

[0133] The device path corresponding to the target device ID is mounted to the specified mounting directory of the created container running environment;

[0134] The configuration information of the target device is injected into the runtime configuration file of the container through the runtime interface (CRI) or engine (Containerd, CRI-O, etc.) of the container;

[0135] Before the container is started, the resource descriptor of the container runtime is synchronously updated according to the injected device configuration information, so that the container can directly access and call the corresponding physical device resources during the running period.​

[0136] Wherein, through the automation device mounting and runtime injection, the container can directly access the corresponding physical device resources at startup, reducing the risk of manual configuration and interface mismatch, thereby realizing efficient and safe device resource availability management.

[0137] In the embodiment, the method further comprises:

[0138] Updating the number of occupied and idle device plugins of the device topology tree of the target node according to the device allocation result.

[0139] Wherein, the scheme ensures that the topology structure and the resource usage state are updated synchronously, maintains the accuracy of the system scheduling information and the real-time of the topology model, provides a reliable basis for subsequent scheduling, and realizes closed-loop management of topology data.

[0140] In the embodiment, after the agent component completes the container group creation according to the device allocation result, the method further comprises:

[0141] Periodically detecting the running indicators of the running container, the running indicators including device access delay, bandwidth usage rate and resource utilization rate;

[0142] Obtaining preset threshold values, the preset threshold values including delay threshold value, bandwidth threshold value and device load rate threshold value;

[0143] When any of the running indicators exceeds the preset threshold value, re-evaluating the device topology integrity score of the current node through the topology-aware device plugin, and if the device topology integrity score of the current node is lower than the preset threshold value, triggering the topology-aware scheduler to perform container group rescheduling operation.

[0144] Wherein, the scheme ensures that the topology structure and the resource usage state are updated synchronously, maintains the accuracy of the system scheduling information and the real-time of the topology model, provides a reliable basis for subsequent scheduling, and realizes closed-loop management of topology data.

[0145] Wherein, the topology-aware device plugin further comprises, in the node initialization stage:

[0146] Collecting power consumption information and temperature information of all PCI devices on the node,

[0147] and generating a device energy consumption topology graph based on the power consumption distribution;

[0148] The topology-aware scheduler takes the average power consumption in the energy consumption topology graph as a weighted parameter in the node scoring process, so as to preferentially select nodes with low energy consumption and complete topology in multi-node scheduling.

[0149] Wherein, when forming the device energy consumption topology graph, the topology-aware device plugin:

[0150] A device stability index S is calculated according to the power consumption change rate ΔP and the temperature change rate ΔT of each device, and the calculation formula is: S = 1 / (1+|ΔP|+|ΔT|);

[0151] The topology-aware scheduler weights and fuses the stability index S and the topology integrity score when calculating the node score to form a node comprehensive stability score, which is used for scheduling and sorting.

[0152] In the device topology adaptive scheduling method, a topology-aware device plug-in is configured in the node to obtain the physical topology relationship between the device plug-ins and construct a device topology tree, so that the system can perform sensing-type device allocation based on topology information when a container group is created. In the allocation stage, the method determines whether the devices applied by the container group are located in the same topology domain, and only returns the allocation result when the devices are in the same topology domain, otherwise the devices are automatically re-applied, thereby ensuring the topology proximity of the devices in the container group. In this way, the cross-topology domain communication problem caused by random allocation of device plug-ins in the prior art is avoided, the data transmission delay and bandwidth competition between devices are significantly reduced, and the internal communication efficiency of the container group is improved. In scenarios such as large model inference that have very high requirements for parallel computing and communication performance, the method can effectively improve the throughput and response speed of the model inference task, realize the efficiency, predictability and adaptability of device scheduling, and solve the problems of low communication efficiency and slow service response in traditional solutions.

[0153] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0154] In one embodiment, as shown in Figure 7 a device topology adaptive scheduling apparatus 10 is provided, comprising a topology construction module 1, a target node selection module 2, a device application module 3, a topology verification module 4, and a device allocation module 5.

[0155] The topology construction module 1 is configured to obtain the topology relationship between the device plug-ins of the node through the topology-aware device plug-in, and form a device topology tree for each node.

[0156] The target node selection module 2 is configured to select a target node from a plurality of nodes in response to a user applying to create a container group, and assign the task of creating the container group to the target node.

[0157] The device application module 3 is configured to obtain the number of devices contained in each container in the pre-created container group by the proxy component of the target node, and apply for a corresponding number of devices to the device plug-in one by one through the topology-aware device plug-in for each container, and obtain a plurality of target devices returned by the application.

[0158] The topology verification module 4 is configured to determine whether the multiple target devices returned by the application are in the same topology domain according to the device topology tree of the target node.

[0159] The device allocation module 5 is configured to return a device allocation result to the agent component in response to the multiple target devices returned by the application being in the same topology domain, and the agent component creates a container group according to the device allocation result, or re-applies for devices from the device plugin.

[0160] In this embodiment, the topology-aware device plugin is used to obtain the topology relationship between the device plugins of the node, and a device topology tree is formed for each node, including:

[0161] The topology-aware device plugin is used to obtain the device plugin information of the node and the topology relationship between the device plugins.

[0162] The device information and the device type attribute are filled for the topology relationship based on the device plugin information, and a device topology tree is formed.

[0163] In this embodiment, the topology-aware device plugin is used to obtain the device plugin information of the node and the topology relationship between the device plugins, including:

[0164] The topology-aware device plugin is used to obtain the extended interface ID information corresponding to the device plugin ID by using a device driver command.

[0165] An operating system command is used to obtain the topology relationship between the device plugins connected by the extended interface ID of the node.

[0166] In this embodiment, the device information and the device type attribute are filled for the topology relationship based on the device plugin information, including:

[0167] The native resource key registered by the device plugin in the agent component is obtained according to the device plugin information.

[0168] The number of resources corresponding to the native resource key is obtained as the device information of the device plugin.

[0169] The device type and the name corresponding to the device plugin are obtained as the device type attribute of the device plugin.

[0170] In this embodiment, the native resource key registered by the device plugin in the agent component is obtained according to the device plugin information, including:

[0171] When the topology-aware device plugin is initialized, the system file directory of the node is scanned, and all registered native resource key instances in the agent component are identified based on the scanning result, a native resource key list is generated and cached in the local memory.

[0172] In the embodiment, the number of resources corresponding to the native resource key is acquired as the device information of the device plugin, including:

[0173] A prefix is added to the native resource key to form a prefix resource key, and the prefix resource key is registered on the proxy component as a new resource key;

[0174] The number of resources corresponding to the native resource key is acquired, and the value of the prefix resource key is set according to the number of resources corresponding to the native resource key;

[0175] When filling the device information for the topology relationship, the value of the prefix resource key is used as the device information of the device plugin to represent the number of resources provided by the device plugin.

[0176] In the embodiment, the method further includes:

[0177] The topology-aware device plugin periodically listens to the number of resources corresponding to the native resource key, and when the number of resources of any device plugin changes, the value of the prefix resource key is updated according to the number of resources corresponding to the native resource key, and the update event is reported to the topology-aware scheduler of the container orchestration scheduling platform through the interface information decision module.

[0178] In the embodiment, the target node is selected from the plurality of nodes, including:

[0179] The number of occupied device plugins and the number of idle device plugins of the device topology tree in each node are acquired to determine the remaining resource number of each node;

[0180] The node with the largest remaining resource number is acquired as the target node from the plurality of nodes;

[0181] In response to the existence of a plurality of nodes with the largest remaining resource number, the integrity score of the device topology tree of the node with the largest remaining resource number is acquired;

[0182] The node with the largest integrity score of the device topology tree is selected as the target node.

[0183] In the embodiment, in response to the existence of a plurality of nodes with the largest remaining resource number, the integrity score of the device topology tree of the node with the largest remaining resource number is acquired, including:

[0184] The number of unused devices and the total number of devices in each node in the same topology domain are acquired;

[0185] The idle rate h of the topology domain is determined by dividing the number of unused devices in the same topology domain by the total number of devices;

[0186] The integrity score of the device topology tree of each node is determined by score=∑e 5*h

[0187] ​In the embodiment, the agent component of the target node obtains the number of devices contained in each container in the pre-created container group, and applies for the corresponding number of target devices from the device plugin through the topology-aware device plugin for each container.

[0188] The agent component of the target node obtains the number of containers contained in the pre-created container group;

[0189] The number of devices contained in each container is obtained.

[0190] A device plugin is assigned to each container in sequence, and the corresponding number of target devices is applied for from the respective corresponding device plugin through the topology-aware device plugin for each container.

[0191] It is judged whether the containers in the pre-created container group all return the target devices applied for by each container.

[0192] If yes, the next step is entered, and if no, a new device plugin is re-assigned to the container that does not return the target devices applied for, and the corresponding number of target devices is applied for from the new device plugin.

[0193] In the embodiment, a device plugin is assigned to each container in sequence, including:

[0194] The load rate of the device plugin of the node is obtained, and the device plugins in the node are arranged in ascending order according to the load rate to form a first device plugin list;

[0195] The containers in the pre-created container group are arranged in descending order according to the number of devices contained in each container to form a first container list;

[0196] The device plugins in the first device plugin list are corresponded to the containers in the first container list one by one, and a device plugin is assigned to each container;

[0197] When the number of containers in the first container list is greater than the number of device plugins in the first device plugin list, the containers in the first container list that are not assigned a device plugin are arranged to form a second container list after the assignment of the device plugins in the device plugin list is completed, the load rate of the device plugin of the node is re-obtained, and the device plugins in the node are arranged in ascending order according to the load rate to form a second device plugin list;

[0198] The device plugins in the second device plugin list are corresponded to the containers in the second container list one by one, and a device plugin is assigned to the container that is not assigned a device plugin.

[0199] In the embodiment, it is judged according to the device topology tree of the target node whether the plurality of target devices applied for and returned are in the same topology domain, including:

[0200] analyzing the at least one target device returned by the first container application, and determining whether the at least one target device returned by the first container application is in the same topology domain;

[0201] in response to the at least one target device returned by the first container application being in the same topology domain, analyzing the at least one target device returned by the second container application, and determining whether the at least one target device returned by the second container application is in the same topology domain as the at least one target device returned by the first container application, and sequentially repeating the above process until all the target devices returned by the container applications of the pre-created container group are in the same topology domain;

[0202] in response to the at least one target device returned by any target container application not being in the same topology domain, re-allocating a new device plug-in to the target container, and re-applying the target container to the target devices corresponding to the target container.

[0203] In the embodiment, the re-allocating of the new device plug-in to the target container includes:

[0204] If all the device plug-ins in the same topology domain in the device topology tree are allocated, a device in a neighboring topology domain is selected as the new device plug-in of the target container according to the shortest topology distance principle.

[0205] In the embodiment, in response to the at least one target device returned by the first container application being in the same topology domain, if the at least one target device returned by the other container applications is not in the same topology domain, the method further includes:

[0206] applying, by the target node, for the target devices to the original device plug-in, obtaining a target device ID, searching the interface device topology tree of the node, and determining whether the target device ID is in the same topology domain;

[0207] if yes, returning the target device ID to create the target container and releasing the devices corresponding to the other IDs;

[0208] if no, keeping the current device without releasing and continuing to apply for the devices to the device plug-in, determining whether a new device is returned, if a new device is returned, obtaining a new device ID and searching the interface device topology tree of the node, and determining whether the new device ID is in the same topology domain, and if no new device is returned, using the current device to return to the other containers.

[0209] In the embodiment, the device allocation result is returned to the agent component, and the agent component creates the container group according to the device allocation result, including:

[0210] mapping the target device ID in the device allocation result and the runtime instance of the created container;

[0211] Mount the device path corresponding to the target device ID to the specified mounting directory of the created container running environment;

[0212] Inject the configuration information of the target device into the runtime configuration file of the container through a runtime interface (CRI) or engine (Containerd, CRI-O, etc.) of the container;

[0213] Before the container is started, the resource descriptor of the container runtime is synchronously updated according to the injected device configuration information, so that the corresponding physical device resources can be directly accessed and called by the container during running.

[0214] In the embodiment, as shown in FIG. 1, the device topology adaptive scheduling apparatus 10 further comprises a topology updating module 6, which is configured to: Figure 7

[0215] Update the number of occupied and idle device plug-ins of the device topology tree of the target node according to the device allocation result.

[0216] In the embodiment, after the container group is created by the agent component according to the device allocation result, the method further comprises:

[0217] Periodically detect the running index of the running container, and the running index comprises a device access delay, a bandwidth usage rate, and a resource utilization rate;

[0218] Obtain a preset threshold, and the preset threshold comprises a delay threshold, a bandwidth threshold, and a device load rate threshold;

[0219] When any index in the running index exceeds the preset threshold, reevaluate the device topology integrity score of the current node through the topology-aware device plug-in, and if the device topology integrity score of the current node is lower than the preset threshold, trigger the topology-aware scheduler to perform a container group rescheduling operation.

[0220] ​In the device topology adaptive scheduling apparatus, a topology-aware device plug-in is configured in a node, a physical topology relationship between device plug-ins is obtained, and a device topology tree is constructed, so that the system can perform sensing-based device allocation based on topology information when a container group is created. In the allocation stage, it is judged whether the devices applied by the container group are located in the same topology domain. Only when the devices are located in the same topology domain, the allocation result is returned, otherwise the devices are automatically re-applied, so as to ensure that the devices in the container group have topology proximity. In this way, the cross-topology domain communication problem caused by random allocation of device plug-ins in the prior art is avoided, the data transmission delay and bandwidth competition between devices are significantly reduced, and the internal communication efficiency of the container group is improved. In a scene such as large model inference, which has very high requirements on parallel computing and communication performance, the method can effectively improve the throughput and response speed of the model inference task, realize the efficiency, predictability and adaptability of device scheduling, and solve the problems of low communication efficiency and slow service response in the traditional scheme.

[0221] The features of the embodiments of the device topology adaptive scheduling apparatus can be referred to the related descriptions of the embodiments of the device topology adaptive scheduling method, which will not be repeated here.

[0222] Embodiments of the present application also provide an electronic device, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above device topology adaptive scheduling method embodiments.

[0223] In one embodiment, the electronic device can be a server, and its internal structure diagram can be as shown in Figure 8 The electronic device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the electronic device is used to store device topology adaptive scheduling data. The network interface of the electronic device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a device topology adaptive scheduling method.

[0224] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the above device topology adaptive scheduling method embodiments when running:

[0225] The topology-aware device plug-in obtains the topology relationship between the device plug-ins in the node, and forms a device topology tree for each node;

[0226] In response to a user applying to create a container group, a target node is selected from the plurality of nodes, and the task of creating the container group is assigned to the target node;

[0227] The proxy component of the target node obtains the number of devices contained in each container in the pre-created container group, and applies to the device plugin for the corresponding number of devices for each container through the topology-aware device plugin one by one, to obtain a plurality of target devices returned by the application;

[0228] It is judged according to the device topology tree of the target node whether the plurality of target devices returned by the application are in the same topology domain;

[0229] In response to the plurality of target devices returned by the application being in the same topology domain, a device allocation result is returned to the proxy component, and the proxy component creates the container group according to the device allocation result, otherwise, the device is re-applied to the device plugin.

[0230] In an example embodiment, the computer readable storage medium described above can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0231] The embodiment of the application also provides a computer program product, the computer program product comprising a computer program, the computer program being executed by a processor to implement the steps in any of the device topology adaptive scheduling method embodiments described above:

[0232] The topology relationship between the device plugins of the node is obtained through the topology-aware device plugin, and a device topology tree is formed for each node;

[0233] In response to a user applying to create a container group, a target node is selected from the plurality of nodes, and the task of creating the container group is assigned to the target node;

[0234] The proxy component of the target node obtains the number of devices contained in each container in the pre-created container group, and applies to the device plugin for the corresponding number of devices for each container through the topology-aware device plugin one by one, to obtain a plurality of target devices returned by the application;

[0235] It is judged according to the device topology tree of the target node whether the plurality of target devices returned by the application are in the same topology domain;

[0236] In response to the plurality of target devices returned by the application being in the same topology domain, a device allocation result is returned to the proxy component, and the proxy component creates the container group according to the device allocation result, otherwise, the device is re-applied to the device plugin.

[0237] Embodiments of the present application also provide another computer program product comprising a non-transitory computer-readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps in any of the above-mentioned device topology adaptive scheduling method embodiments:

[0238] obtain the topology relationship between the device plugins of the node through the topology-aware device plugin, and form a device topology tree for each node;

[0239] In response to a user applying to create a container group, a target node is selected from the plurality of nodes, and the task of creating the container group is assigned to the target node;

[0240] The agent component of the target node obtains the number of devices contained in each container in the pre-created container group, and applies to the device plugin for the corresponding number of devices for each container through the topology-aware device plugin, to obtain a plurality of target devices returned by the application;

[0241] According to the device topology tree of the target node, it is determined whether the plurality of target devices returned by the application are in the same topology domain;

[0242] In response to the plurality of target devices returned by the application being in the same topology domain, a device allocation result is returned to the agent component, and the agent component creates the container group according to the device allocation result, otherwise, the device plugin is re-applied for devices.

[0243] Those skilled in the art will further appreciate that the functions described in the examples described herein can be implemented in electronic hardware, computer software, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, the above description has generally been described in terms of examples of functions that can be implemented as hardware or software. Whether such functions are implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art can implement the described functions in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0244] The above provides a detailed description of the kernel module log redirection method, electronic device, storage medium and product provided by the present application. The principles and implementation methods of the present application are described in the specific examples. The above description of the examples is only applicable to help understand the method and core idea of the present application. It should be noted that, for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A method for adaptive scheduling of device topology, the method comprising: The method comprises the following steps of: setting a topology-aware device plug-in, an agent component and at least one device plug-in on a node, wherein the method comprises the following steps of: obtaining, by the topology-aware device plug-in, a topology relationship between device plug-ins of the node, and forming a device topology tree for each node; selecting a target node from multiple nodes and assigning a container group creation task to the target node in response to a user application for creating a container group; obtaining, by the agent component of the target node, a number of devices contained in each container in a pre-created container group, and applying, by the agent component of the target node, to the device plug-in for each container through the topology-aware device plug-in, a corresponding number of devices to obtain a plurality of target devices returned by the application; determining, according to the device topology tree of the target node, whether the plurality of target devices returned by the application are in the same topology domain switch; in response to the plurality of target devices returned by the application being in the same topology domain, returning a device allocation result to the agent component, and creating a container group according to the device allocation result by the agent component, otherwise reapplying for devices to the device plug-in; wherein the step of selecting a target node from multiple nodes comprises the following steps of: obtaining an occupied number and a free number of device plug-ins of a device topology tree in each node to determine a remaining resource number of each node; selecting a node with a maximum remaining resource number as a target node from multiple nodes; in response to there being multiple nodes with a maximum remaining resource number, obtaining a completeness score of a device topology tree of the node with a maximum remaining resource number; selecting a node with a maximum completeness score of a device topology tree as a target node; wherein the step of, in response to there being multiple nodes with a maximum remaining resource number, obtaining a completeness score of a device topology tree of the node with a maximum remaining resource number, comprises the following steps of: obtaining a number of unused devices and a total number of devices in each node in the same topology domain; determining a free rate h of the same topology domain by dividing the number of unused devices by the total number of devices; By score =∑e 5*h A completeness score of the device topology tree for each node is determined.

2. The method of claim 1, wherein, the step of obtaining, by the topology-aware device plug-in, a topology relationship between device plug-ins of the node, and forming a device topology tree for each node, comprises the following steps of: obtaining, by the topology-aware device plug-in, device plug-in information and a topology relationship between device plug-ins of the node; filling device information and a device type attribute into the topology relationship based on the device plug-in information to form a device topology tree.

3. The method of claim 2, wherein, the step of obtaining, by the topology-aware device plug-in, device plug-in information and a topology relationship between device plug-ins of the node, comprises the following steps of: obtaining, by the topology-aware device plug-in, extended interface ID information corresponding to a device plug-in ID by using a device driver command; obtaining a topology relationship between device plug-ins connected by an extended interface ID of the node by using an operating system command.

4. The method of claim 2, wherein, the step of filling device information and a device type attribute into the topology relationship based on the device plug-in information, comprises the following steps of: obtaining a native resource key registered by the device plug-in in the agent component according to the device plug-in information; obtaining a resource number corresponding to the native resource key as device information of the device plug-in; obtaining a device type and a name corresponding to the device plug-in as a device type attribute of the device plug-in.

5. The method of claim 4, wherein, The method further comprises: The proxy component of the target node obtains the number of devices contained in each container in the pre-created container group, and applies for a corresponding number of target devices to the device plugin through the topology-aware device plugin for each container, and obtains a plurality of target devices returned by the application, comprising:

6. The method of claim 4, wherein, The proxy component of the target node obtains the number of containers contained in the pre-created container group; Obtain the number of devices contained in each container; In turn, assign a device plugin to each container, and apply for a corresponding number of target devices to the respective corresponding device plugin through the topology-aware device plugin for each container; Determine whether the containers in the pre-created container group have returned the target devices applied for by each container; 7. The method of claim 6, wherein, If yes, proceed to the next step, if not, reassign a new device plugin to the container that has not returned the target device applied for, and apply for a corresponding number of target devices to the new device plugin. The method further comprises:

8. The method of claim 1, wherein, The topology-aware device plugin periodically listens to the number of resources corresponding to the native resource key, and when the number of resources of any device plugin changes, updates the value of the prefix resource key according to the number of resources corresponding to the native resource key, and reports the update event to the topology-aware scheduler of the container orchestration scheduling platform through the interface information decision module. The proxy component of the target node obtains the number of devices contained in each container in the pre-created container group, and applies for a corresponding number of target devices to the device plugin through the topology-aware device plugin for each container, and obtains a plurality of target devices returned by the application, comprising: The proxy component of the target node obtains the number of containers contained in the pre-created container group; Obtain the number of devices contained in each container; In turn, assign a device plugin to each container, and apply for a corresponding number of target devices to the respective corresponding device plugin through the topology-aware device plugin for each container; Determine whether the containers in the pre-created container group have returned the target devices applied for by each container; 9. The method of claim 8, wherein, If yes, proceed to the next step, if not, reassign a new device plugin to the container that has not returned the target device applied for, and apply for a corresponding number of target devices to the new device plugin. The method further comprises: The proxy component of the target node obtains the number of devices contained in each container in the pre-created container group, and applies for a corresponding number of target devices to the device plugin through the topology-aware device plugin for each container, and obtains a plurality of target devices returned by the application, comprising: The proxy component of the target node obtains the number of containers contained in the pre-created container group; Obtain the number of devices contained in each container; In turn, assign a device plugin to each container, and apply for a corresponding number of target devices to the respective corresponding device plugin through the topology-aware device plugin for each container; Determine whether the containers in the pre-created container group have returned the target devices applied for by each container; If yes, proceed to the next step, if not, reassign a new device plugin to the container that has not returned the target device applied for, and apply for a corresponding number of target devices to the new device plugin. Corresponding to the device plug-ins in the second device plug-in list and the containers in the second container list, a device plug-in is assigned to a container without an assigned device plug-in.

10. The method of adaptive scheduling of device topology according to claim 8, wherein, The method further comprises: Corresponding to the at least one target device returned by the first container application, determining whether the at least one target device returned by the first container application is in the same topology domain; In response to the at least one target device returned by the first container application being in the same topology domain, corresponding to the at least one target device returned by the second container application, determining whether the at least one target device returned by the second container application is in the same topology domain as the at least one target device returned by the first container application, and sequentially repeating the above steps until the at least one target device returned by all container applications of the pre-created container group is in the same topology domain; In response to the at least one target device returned by any target container application not being in the same topology domain, re-assigning a new device plug-in to the target container and re-applying the target container for a corresponding number of target devices.

11. The method of adaptive scheduling of device topology according to claim 10, wherein, The method further comprises: If all device plug-ins in the same topology domain in the device topology tree are assigned, a device in a neighboring topology domain is selected as the new device plug-in re-assigned to the target container according to the shortest topology distance principle.

12. The method of adaptive scheduling of device topology according to claim 10, wherein, In response to the at least one target device returned by the first container application being in the same topology domain, if the at least one target device returned by other container applications is not in the same topology domain, the method further comprises: In response to the at least one target device returned by the first container application being in the same topology domain, if the at least one target device returned by other container applications is not in the same topology domain, the method further comprises: If not, the current device is maintained without being released, and a device is re-applied to the device plug-in, and it is determined whether a new device is returned; if a new device is returned, a new device ID is obtained and the interface device topology tree of the node is searched, and it is determined whether the new device ID is in the same topology domain; if no new device is returned, the current device is returned to the other container. The method further comprises:

13. The method of claim 1, wherein, The target device ID in the device assignment result is mapped to the runtime instance of the created container; The device path corresponding to the target device ID is mounted to the specified mounting directory of the created container runtime environment; The configuration information of the target device is injected into the runtime configuration file of the container through the runtime interface or engine of the container; Before the container is started, the resource descriptor of the container runtime is synchronously updated according to the injected device configuration information, so that the container can directly access and call the corresponding physical device resource during operation. The method further comprises:

14. The method of claim 1, wherein, ​ According to the device allocation result, the device plug-in occupied number and the idle number of the device topology tree of the target node are updated.

15. The method of claim 1, wherein, After the container group creation is completed by the agent component according to the device allocation result, the method further includes: Periodically detecting running indexes of running containers, the running indexes including device access delay, bandwidth usage rate and resource utilization rate; A preset threshold is obtained, the preset threshold including a delay threshold, a bandwidth threshold and a device load rate threshold; When any index in the running indexes exceeds the preset threshold, the device topology integrity score of the current node is re-evaluated by a topology-aware device plug-in, and if the device topology integrity score of the current node is lower than the preset threshold, a container group rescheduling operation is triggered by a topology-aware scheduler.

16. An electronic device, comprising: Comprise: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the device topology adaptive scheduling method according to any one of claims 1 to 15.

17. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, wherein the computer program is executed by the processor to implement the steps of the device topology adaptive scheduling method according to any one of claims 1 to 15.

18. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the device topology adaptive scheduling method according to any one of claims 1 to 15.

Images