A topology-aware scheduling system, method, electronic device, and storage medium

Abstract

The present disclosure relates to a topology-aware scheduling system, method, electronic device and storage medium, wherein the system comprises: a request receiving module, receiving a resource application request submitted by a user, wherein the resource application request contains definition information of a container group Pod, wherein the definition information includes the topological scheduling requirements of the Pod for resources; a scheduler, obtaining the topological scheduling requirements of the Pod for resources, and the topological information of the resources of each node; the scheduler, based on the topological information of the resources of each node, selects a node that meets the topological scheduling requirements from each node as a candidate node of the Pod; the scheduler, based on the candidate node, allocates resources to the Pod. The embodiments of the present disclosure can improve the stability of the cluster and the flexibility of resource scheduling, and realize efficient use of resources.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a topology-aware scheduling system, method, electronic device, and storage medium. Background Art

[0002] With the rapid development of cloud computing technology, containerization technology has been widely used in the field of cloud computing due to its lightweight, highly portable, and fast deployment characteristics. In container orchestration systems such as Kubernetes, Pod, as the smallest deployable unit, is responsible for carrying containerized applications and their related dependencies. In order to meet the needs of different business scenarios, how to efficiently and reasonably schedule Pod resources to ensure cluster stability and efficient resource utilization has become a hot topic in the industry.

[0003] In the existing Pod scheduling strategy, topology scheduling is an important consideration. Topology scheduling mainly focuses on how to allocate Pods to different physical nodes or physical areas to optimize network latency, improve data access speed, meet compliance requirements, etc. For example, in some scenarios, you may want to deploy Pods that need to communicate frequently on the same physical node to reduce network transmission overhead; in other scenarios, in order to ensure data security, you may need to deploy business-critical Pods in physically isolated areas.

[0004] However, the existing Pod topology scheduling strategy can only be set uniformly at the cluster level. Although this global scheduling strategy can meet some common needs, it is often not flexible enough in actual applications. Specifically, due to the large differences in requirements between different businesses and different Pods, it is difficult for a global topology scheduling strategy to meet the personalized needs of all Pods. For example, some businesses may have extremely high requirements for network latency, while other businesses may be more concerned with the physical isolation of data; at the same time, different physical nodes or regions in the cluster may also have different performance characteristics and resource limitations, requiring refined Pod scheduling based on these characteristics.

[0005] Therefore, the existing Pod topology scheduling strategy has poor flexibility. Summary of the invention

[0006] The present disclosure proposes a topology-aware scheduling technology solution.

[0007] According to one aspect of the present disclosure, a topology-aware scheduling system is provided, comprising:

[0008] A request receiving module receives a resource application request submitted by a user, wherein the resource application request includes definition information of a container group Pod, and the definition information includes a topology scheduling requirement of the Pod for resources;

[0009] The scheduler obtains the topological scheduling requirements of the Pod for resources and the topological information of the resources of each node;

[0010] The scheduler selects a node that meets the topology scheduling requirement from each node based on the topology information of the resources of each node as a candidate node for the Pod;

[0011] The scheduler allocates resources to the Pod based on the candidate nodes.

[0012] In a possible implementation, the additional information of the Pod includes the topology scheduling requirements of the Pod for resources;

[0013] The scheduler obtains the topology scheduling requirements of the Pod for resources from the additional information of the Pod.

[0014] In a possible implementation, when there is a unified topology scheduling requirement in the cluster-level default configuration of the node agent, the scheduler uses the topology scheduling requirement obtained from the additional information of the Pod to obtain the topology scheduling requirement configuration at the Pod level.

[0015] In a possible implementation, the scheduler determines a target node from the candidate nodes and allocates the Pod to the target node;

[0016] The scheduler obtains bandwidth information between GPUs in the target node;

[0017] The scheduler determines a GPU combination with optimal performance based on the bandwidth information and allocates the GPU combination to the Pod.

[0018] In a possible implementation, the resource application request includes a target number of requested GPUs, where the target number is M, and the number of GPUs in the target node is N, where both M and N are positive integers;

[0019] The scheduler selects a plurality of GPU combinations including M GPUs from the N GPUs of the target node;

[0020] The scheduler determines the minimum bandwidth of each GPU combination respectively;

[0021] A maximum bandwidth value is determined from each minimum bandwidth value, and a GPU combination corresponding to the maximum bandwidth value is used as a GPU combination with optimal performance.

[0022] In a possible implementation, the system further includes:

[0023] The bandwidth information acquisition module is used to obtain the bandwidth information between GPUs and report the bandwidth information between GPUs to the label of the node;

[0024] The scheduler obtains bandwidth information between GPUs from the labels of the nodes.

[0025] In a possible implementation, the scheduler adds the identifier of the GPU combination with the best performance to the additional information of the Pod;

[0026] The system further includes: a resource allocation module, which obtains the identification of the GPU combination in the additional information of the Pod, and allocates the GPU resources corresponding to the identification to the Pod.

[0027] According to one aspect of the present disclosure, a topology-aware scheduling method is provided, including:

[0028] Receive a resource application request submitted by a user, wherein the resource application request includes definition information of a container group Pod, and the definition information includes a topology scheduling requirement of the Pod for resources;

[0029] Obtain the topological scheduling requirements of the Pod for resources and the topological information of the resources of each node;

[0030] Based on the topological information of the resources of each node, a node that meets the topological scheduling requirements is selected from each node as a candidate node for the Pod;

[0031] Based on the candidate nodes, resources are allocated to the Pod.

[0032] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0033] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0034] In the disclosed embodiment, the request receiving module receives the resource application request submitted by the user, wherein the resource application request includes the definition information of the container group Pod, and the definition information includes the topological scheduling requirements of the Pod for resources; the scheduler obtains the topological scheduling requirements of the Pod for resources, and the topological information of the resources of each node, and selects the node that meets the topological scheduling requirements from each node based on the topological information of the resources of each node as the candidate node of the Pod, and then allocates resources to the Pod based on the candidate node. Thus, the resource application request submitted by the user is received by the request receiving module, which includes the definition information of the Pod, and these definition information include the topological scheduling requirements of the Pod for resources, that is, the user can customize its unique topological scheduling requirements for each Pod according to the needs of his own business; then, based on the topological scheduling requirements of the Pod and the resource topological information of each node, the scheduler can screen out the candidate nodes that meet the Pod requirements from the many nodes of the cluster, realizing the transition from global unified scheduling to refined scheduling based on the personalized needs of the Pod; since the candidate nodes are selected based on the personalized topological scheduling requirements of the Pod, this resource allocation method is more flexible and efficient than the traditional global unified scheduling method.

[0035] By introducing Pod-level topology scheduling requirements and a scheduling mechanism based on node resource topology information, the problem of poor flexibility in existing technologies, which can only set Pod topology scheduling policies at the cluster level, is solved. The system makes Pod topology scheduling more refined and personalized, which can better meet the differences in requirements between different businesses and different Pods, and improve cluster stability and efficient resource utilization.

[0036] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0038] Figure 1 A framework diagram of a topology-aware scheduling system according to an embodiment of the present disclosure is shown.

[0039] Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present disclosure is shown.

[0040] Figure 3 A flowchart of a topology-aware scheduling method according to an embodiment of the present disclosure is shown.

[0041] Figure 4 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0042] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0043] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0044] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0045] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0046] Figure 1 A framework diagram of a topology-aware scheduling system according to an embodiment of the present disclosure is shown as follows: Figure 1 As shown, the system 10 includes:

[0047] The request receiving module 101 receives a resource application request submitted by a user, wherein the resource application request includes definition information of a container group Pod, and the definition information includes a topology scheduling requirement of the Pod for resources;

[0048] The scheduler 102 obtains the topological scheduling requirements of the Pod for resources and the topological information of the resources of each node;

[0049] The scheduler 102 selects a node that meets the topology scheduling requirement from each node based on the topology information of the resources of each node as a candidate node for the Pod;

[0050] The scheduler 102 allocates resources to the Pod based on the candidate nodes.

[0051] The system includes a request receiving module and a scheduler. These two modules work together to achieve accurate identification and efficient scheduling of the topology scheduling requirements of container groups (Pods).

[0052] In this system, the resource application request received by the request receiving module will include the definition information of the container group Pod, where the container group (Pod) is the smallest deployable unit in container orchestration systems such as Kubernetes, consisting of one or more containers and sharing storage and network resources.

[0053] Resource application requests are used to request the deployment of Pods in the cluster. For example, in the Kubernetes system, users can submit resource application requests through kubectl (Kubernetes command-line tool) or other Kubernetes API clients (such as graphical interfaces, etc.), and the Kubernetes server (API Server) receives the resource application requests submitted by users. The API Server authenticates, authorizes, and performs admission control on the request to ensure that the request is legal and complies with the cluster's policies, and persists it to etcd. The scheduler listens to events issued by the API Server through the List-Watch mechanism. When new Pods or other workload resources are created, the scheduler captures these events and triggers the corresponding processing logic.

[0054] In the resource application request, the user will provide the definition information about the Pod, such as container images, environment variables, resource requirements, etc. The topology scheduling requirements of the Pod for resources are the scheduling strategies that the Pod expects to follow when scheduling topology resources. They are the topology requirements that the Pod needs to consider when allocating and scheduling resources. For example, the Pod may need to be scheduled to a node with specific NUMA topology characteristics, or need to access resources on a specific NUMA node. For example, in the Kubernetes system, the scheduling strategies for topology scheduling requirements mainly include the following four:

[0055] none: The kubelet will not consult the topology manager for its decision. This is the default policy for the kubelet.

[0056] Best-effort: The topology manager will give the preferred allocation based on the topology information. Under this strategy, even if the allocation result is unreasonable, the Pod will be successfully admitted.

[0057] restricted: Unlike best-effort, under this policy, Pods are rejected if the allocation result is unreasonable. Unlike the single-numa-node policy, if it is impossible to satisfy the allocation request on a single NUMA node, some allocations may come from multiple NUMA nodes.

[0058] single-numa-node: The topology manager enforces resource allocation on NUMA nodes, and if the allocation fails, the Pod will be rejected.

[0059] After obtaining the topological scheduling requirements of the Pod for resources, the scheduler also obtains the topological information of the resources of each node. The topological information of resources is the layout and configuration information of the physical resources in the cluster, and the information of the topological structure of the resources inside the node. For example, in a Kubernetes cluster environment, the topological information of resources can be the information of non-uniform memory access nodes (numanode).

[0060] In a Kubernetes cluster, each physical node (usually called a worker node or compute node) contains certain computing resources, such as CPU, memory, disk, and possibly GPU. These resources may be physically organized in the form of numanode. NUMA (Non-Uniform Memory Access) is a computer memory design technology that allows the processor to access its own local memory faster than accessing non-local memory (that is, memory located on other numanodes).

[0061] In a Kubernetes cluster, the Resource Topology Exporter is a component that runs on a Kubernetes node. Its main function is to collect and report information about the resource topology structure inside the node. This information includes, but is not limited to: the number and configuration of NUMA nodes; the number and layout of CPU cores on NUMA nodes; the size and layout of memory on NUMA nodes; other resources that may exist on NUMA nodes (such as GPUs) and their association with CPUs and memory. After this information is collected, it will be stored in Kubernetes' custom resources, such as the NodeResourceTopology CR. By querying these CRs, the Kubernetes scheduler can understand the resource topology structure of each node and make the best scheduling decisions based on the topological scheduling requirements of the Pod.

[0062] According to the topology scheduling requirements of the Pod, nodes that meet the requirements can be screened out from the cluster. There may be more than one node. The scheduler will screen out candidate nodes that meet the Pod topology scheduling requirements from the many nodes in the cluster based on the topology scheduling requirements of the Pod and the resource topology information of the node. This process is achieved by filtering and sorting the nodes to ensure that the final selected node not only meets the topology scheduling requirements of the Pod, but also has sufficient resources to run the Pod.

[0063] Specifically, when determining candidate nodes, the scheduler collects topology hints (TopologyHints) from different resource providers (HintProviders) for each container based on the container's resource request in the resource application request and the node's topology information; then, the scheduler uses a specific policy (Policy) to merge the collected topology hints to form a set of best hints (BestHint), which reflect how to allocate resources to meet the container's topology scheduling requirements; based on the merged best hints, the scheduler selects candidate nodes from the cluster nodes that can meet the Pod topology scheduling requirements.

[0064] The scheduler will select an optimal node from the candidate nodes and assign the Pod to the node. The selection of the optimal node may be based on multiple factors, such as the node's resource utilization, stability, network latency, etc.

[0065] In the disclosed embodiment, the request receiving module receives the resource application request submitted by the user, wherein the resource application request includes the definition information of the container group Pod, and the definition information includes the topological scheduling requirements of the Pod for resources; the scheduler obtains the topological scheduling requirements of the Pod for resources, and the topological information of the resources of each node, and selects the node that meets the topological scheduling requirements from each node based on the topological information of the resources of each node as the candidate node of the Pod, and then allocates resources to the Pod based on the candidate node. Thus, the resource application request submitted by the user is received by the request receiving module, which includes the definition information of the Pod, and these definition information include the topological scheduling requirements of the Pod for resources, that is, the user can customize its unique topological scheduling requirements for each Pod according to the needs of his own business; then, based on the topological scheduling requirements of the Pod and the resource topological information of each node, the scheduler can screen out the candidate nodes that meet the Pod requirements from the many nodes of the cluster, realizing the transition from global unified scheduling to refined scheduling based on the personalized needs of the Pod; since the candidate nodes are selected based on the personalized topological scheduling requirements of the Pod, this resource allocation method is more flexible and efficient than the traditional global unified scheduling method.

[0066] By introducing Pod-level topology scheduling requirements and a scheduling mechanism based on node resource topology information, the problem of poor flexibility in existing technologies, which can only set Pod topology scheduling policies at the cluster level, is solved. The system makes Pod topology scheduling more refined and personalized, which can better meet the differences in requirements between different businesses and different Pods, and improve cluster stability and efficient resource utilization.

[0067] In a possible implementation, the topology scheduling requirement of the Pod for resources is located in the additional information of the Pod, and the additional information includes a label or an annotation; the scheduler obtains the topology scheduling requirement of the Pod for resources from the additional information of the Pod.

[0068] In Kubernetes, the definition of a Pod is usually described through a YAML file. To meet specific scheduling requirements, users can add additional information to the Pod in this YAML file. The additional information can include labels and annotations. Labels are key-value pairs used to identify and select resource objects in the cluster. They are usually used to logically group and select Pods in order to apply policies, route traffic, or perform other logical operations. Annotations: Annotations are also key-value pairs, but they are not used to identify selectors, but to store non-identifying metadata that may be used by external systems or tools. In topology-aware scheduling, annotations can often be used to store the topology scheduling requirements of Pods.

[0069] Users can add specific annotations to the Pod's YAML definition, which contain instructions about Pod scheduling and resource allocation. For example, an annotation may specify that the Pod needs to be scheduled on a node with a specific NUMA configuration, or that the Pod wants to utilize resources on a specific NUMA node.

[0070] The scheduler reads and analyzes these annotations when executing scheduling tasks. By parsing the information in these annotations, the scheduler can understand the specific requirements of the Pod for resource topology and make scheduling decisions that meet these requirements. This approach provides greater flexibility for Pod scheduling, allowing users to customize Pod scheduling strategies based on their business needs and resource optimization considerations.

[0071] In a possible implementation, when there is a unified topology scheduling requirement in the cluster-level default configuration of the node agent, the scheduler uses the topology scheduling requirement obtained from the additional information of the Pod to obtain the topology scheduling requirement configuration at the Pod level.

[0072] In Kubernetes, kubelet is a node agent running on each node, responsible for managing the lifecycle of Pods and containers. Kubelet can configure some cluster-level default settings, which apply to all Pods in the cluster unless these Pods have specific override configurations.

[0073] When a unified topology scheduling requirement is defined in the kubelet's cluster-level default configuration, all Pods will follow these default topology scheduling rules if no other requirements are specified. For example, all Pods are scheduled to nodes with a specific NUMA configuration by default, or follow some default node affinity rules.

[0074] However, when a Pod explicitly specifies its own topology scheduling requirements in its additional information (such as labels or annotations), the scheduler will give priority to the Pod-level requirements instead of simply following the cluster-level default configuration.

[0075] Specifically, the scheduler reads this additional information from the Pod's YAML definition or other related configurations, and parses the Pod's topology scheduling requirements for resources as a specific topology scheduling strategy for the Pod. For example, if the cluster is configured by default to use NUMA topology strategy A for all Pods, but a Pod needs to use strategy B to meet its performance requirements, the user can specify the required topology strategy B in the Pod's annotation. When scheduling the Pod, the scheduler will follow the strategy B specified in the annotation instead of the cluster's default strategy A.

[0076] In the disclosed embodiments, it is allowed to provide greater flexibility and customization for specific Pods while maintaining a unified configuration at the cluster level, which helps to meet the diverse needs in different business scenarios and improve the efficiency and accuracy of resource scheduling.

[0077] In a possible implementation, the scheduler determines a target node from the candidate nodes and allocates the Pod to the target node; the scheduler obtains bandwidth information between GPUs in the target node; the scheduler determines a GPU combination with optimal performance based on the bandwidth information and allocates it to the Pod.

[0078] The scheduler selects the most suitable node from the candidate nodes as the target node. The scheduler has a series of scoring functions that score candidate nodes based on specific criteria (such as the resource idleness of the node, the affinity of the Pod and the node, the density of Pods already running on the node, etc.). Each scoring function calculates a score for each candidate node, reflecting the suitability of the node for running the current Pod. The scheduler weights and sums the scores of each scoring function according to the preset weights to obtain the final score of each candidate node. The candidate nodes are sorted from high to low according to the final score. The node with the highest score is usually selected as the target node, that is, the node to which the Pod will be scheduled. If there are multiple nodes with the same and highest scores, the scheduler can select the final target node based on further strategies (such as random selection or further consideration of other factors).

[0079] The scheduler will further consider the performance information of specific resources within the node, that is, the bandwidth information between GPUs. This is because for some compute-intensive or communication-intensive applications, the communication bandwidth between GPUs may become a key factor affecting application performance.

[0080] Therefore, the scheduler obtains bandwidth information between GPUs in the target node, which can come from the cluster monitoring system, GPU driver, or specialized hardware performance analysis tools. The scheduler evaluates the performance of different GPU combinations in the target node based on the bandwidth information obtained, determines a GPU combination with the best performance based on the bandwidth information, and then assigns this combination to the Pod.

[0081] The selection criteria for the optimal GPU combination can be calculated based on the maximum, minimum, average or other relevant indicators of bandwidth. By selecting the optimal GPU combination, the scheduler can ensure that the Pod can achieve the best performance on the allocated resources.

[0082] In the disclosed embodiment, the node topology information and the communication bandwidth between GPUs within the node are comprehensively considered to schedule the Pod resources, which can not only ensure that the Pod is scheduled to a suitable node, but also ensure that the Pod obtains the optimal resource combination within the node, thereby improving the performance of the application and resource utilization efficiency.

[0083] In a possible implementation, the resource application request includes a target number of requested GPUs, where the target number is M, and the number of GPUs in the target node is N, where both M and N are positive integers;

[0084] The scheduler selects a plurality of GPU combinations including M GPUs from the N GPUs of the target node;

[0085] The scheduler determines the minimum bandwidth of each GPU combination respectively;

[0086] A maximum bandwidth value is determined from each minimum bandwidth value, and a GPU combination corresponding to the maximum bandwidth value is used as a GPU combination with optimal performance.

[0087] The resource request of the Pod contains the target number of GPUs M, which indicates the number of GPUs that the Pod expects to obtain. At the same time, the number of GPUs available in the target node is N. Here, M and N are both positive integers, and M should be less than or equal to N to ensure that the request can be satisfied.

[0088] Then, when determining the optimal GPU combination, the scheduler will select all possible GPU combinations containing M GPUs from the N GPUs of the target node. The "multiple" here can be all possible GPU combinations with M GPUs, that is, the number of GPUs in each GPU combination is M, and all different combinations containing M GPUs.

[0089] Then, the minimum bandwidth of each combination is calculated. For each GPU combination, the scheduler calculates the minimum bandwidth between all GPU pairs in the combination. This is because in a combination, the performance bottleneck usually depends on the pair of GPUs with the lowest bandwidth.

[0090] After calculating the minimum bandwidth values ​​of all combinations, the scheduler compares these minimum values ​​and finds the largest one. The GPU combination corresponding to the maximum value is the combination with the best performance, which indicates the situation with the smallest bandwidth bottleneck among all possible combinations (the maximum value among all bandwidth minimum values).

[0091] After determining the GPU combination with the best performance, the scheduler will assign this combination to the Pod to ensure that the Pod can obtain GPU resources that meet its needs and have the best performance.

[0092] In the disclosed embodiment, by determining the maximum bandwidth from the minimum bandwidth values, the maximum value of the bandwidth bottleneck in all GPU combinations can be determined, ensuring that the GPU pair with the worst bandwidth performance in the selected GPU combination also has a relatively high bandwidth, thereby reducing potential bandwidth bottlenecks, avoiding performance jitter or system crashes caused by insufficient bandwidth of a certain GPU, and effectively utilizing the GPU resources in the cluster, thereby improving the overall utilization of GPU resources.

[0093] In a possible implementation, the system further includes:

[0094] The bandwidth information acquisition module is used to obtain the bandwidth information between GPUs and report the bandwidth information between GPUs to the label of the node;

[0095] The scheduler obtains bandwidth information between GPUs from the labels of the nodes.

[0096] The bandwidth information acquisition module can collect bandwidth information between GPUs. Bandwidth information affects the speed and efficiency of data exchange between GPUs. The bandwidth information acquisition module can obtain actual bandwidth data between GPUs through various means (such as hardware query, system call, etc.). In an example, the bandwidth information acquisition module can actually measure the peer-to-peer (p2p) bandwidth between GPUs by executing tests on each Kubernetes node through a script.

[0097] After obtaining the bandwidth information between GPUs, the bandwidth information acquisition module will add the bandwidth information as labels to the metadata of the Kubernetes node. In one example, the bandwidth information acquisition module can serialize the actual bandwidth value obtained into a string, and then add it as a label to the metadata of the node for subsequent processing.

[0098] The scheduler queries the node labels to obtain bandwidth information between GPUs and performs GPU topology awareness to select the best GPU combination.

[0099] In the disclosed embodiment, by managing the bandwidth information between GPUs as node labels, the system is made more flexible and scalable. With the upgrade of hardware or the addition of new nodes, users can easily update the bandwidth information of nodes without large-scale reconstruction or modification of the entire system. In addition, by combining other node labels (such as GPU model, memory capacity, etc.), the scheduler can make more complex and sophisticated scheduling decisions.

[0100] In a possible implementation, the scheduler adds the identifier of the GPU combination with the best performance to the additional information of the Pod; the system also includes: a resource allocation module, which obtains the identifier of the GPU combination in the additional information of the Pod and allocates the GPU resources corresponding to the identifier to the Pod.

[0101] After selecting the optimal GPU combination, the scheduler can obtain the ID of the GPU in the combination, and then add the ID of the GPU in the combination to the additional information of the Pod. For example, through the Patch operation, the Pod annotation is updated with the ID of the optimal GPU. The Patch operation updates the Pod resources through the Kubernetes API to add or modify the annotation without replacing the entire Pod configuration.

[0102] The updated Pod contains new annotations. The resource allocation module (Device Plugin) monitors these changes and reads the GPU identifier in the annotation. Then, based on the GPU identifier in the annotation, it allocates the corresponding GPU to the Pod from the available GPU resources on the node.

[0103] The GPU resources allocated to the Pod are bound to the Pod, allowing the containers in the Pod to access and use these GPUs. After the resource allocation is completed, the Pod can be started, and the containers in it can use the allocated GPU resources to perform computing tasks.

[0104] An application scenario of an embodiment of the present disclosure is described below.

[0105] Assume that the bandwidth matrix between GPUs is as follows:

[0106]

[0107] After serializing the matrix, the following string is obtained and used as the label of the corresponding node.

[0108] "[[750.48,48.39,48.39,96.46,15.97,16.15,96.41,16.18],[48.39,752.65,96.46,48.38,16.96,16.84,16.67,96.23],[48.38,96.25,749.04,6.02,48.38,17.57,17.54,16.95],[96.44,48.39,96.48,752.65,17.27,48.38,17.33,16.8], [15.99,16.8,48.38,17.27,755.56,96.44,48.38,96.44],[16.14,16.74,17.73,48.38,96.23,755.56,96.25,48.38],[96.43,16.81,17.6,17.35,48.33,96.23,754.83,48.39],[16.28,96.22,17.18,16.88,96.23,48.33,48.33,755.56]]".

[0109] Here, we use a recursive method to traverse the best GPU bandwidth combination that meets the conditions, find a combination containing M GPUs, calculate the minimum bandwidth between each pair as the GPU group bandwidth, and then traverse all possible combinations to find the maximum value of the GPU group bandwidth. The GPU combination corresponding to the maximum value is the optimal GPU combination. The optimal GPU combination is used as the label of pods through the patch operation to allocate GPUs.

[0110] like Figure 2 As shown, Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present disclosure is shown. In this application scenario, Resource Topology Exporter collects resource information of each node in the cluster and organizes the information into NRTCR; Kubernetes API Server receives resource application requests submitted by users, and the resource application requests include definition information of the Pod to be created; the definition information of the Pod includes topology scheduling requirements and the number of GPUs requested;

[0111] The topology aware plugin in the k8s scheduler collects NRT CR to obtain the resource topology information of the node. The topology aware plugin reads the NRT CR and determines the candidate nodes that meet the topology scheduling requirements of the pod based on the NRT CR and the topology scheduling requirements on the pod.

[0112] For candidate nodes, the topology aware plugin will further select the best node as the target node for Pod scheduling;

[0113] During cluster operation, the GPU bandwidth collection component (GPU feature discovery) collects bandwidth information between GPUs on nodes. This bandwidth information is marked on the labels or annotations of the nodes. For the target node, in the PreBind phase, the scheduler reads the labels or annotations of the node to obtain the GPU bandwidth information, and further performs GPU topology awareness based on the GPU bandwidth information. If a pod requests multiple GPUs, the scheduler selects the optimal GPU combination based on the bandwidth information between GPUs.

[0114] The GPU ID of the optimal GPU combination is added to the pod label through the patch operation. The GPU Device Plugin allocates a GPU to the pod according to the GPU ID on the pod label and binds it. After the binding is completed, the kubelet starts to create and run the pod on the node.

[0115] In one possible implementation, the topology-aware scheduling system can be executed by electronic devices such as terminal devices and servers. The terminal devices can be user equipment (UE), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (PDA), handheld devices, computing devices, vehicle-mounted devices, wearable devices, etc. The system can be implemented by a processor calling computer-readable instructions stored in a memory.

[0116] In addition, the present disclosure also provides a topology-aware scheduling method, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any topology-aware scheduling system provided by the present disclosure. The corresponding technical solutions and descriptions are referred to in the corresponding records of the method part and will not be repeated here.

[0117] Figure 3 A flowchart of a topology-aware scheduling method according to an embodiment of the present disclosure is shown as follows: Figure 3 As shown, the method includes:

[0118] In step S21, a resource application request submitted by a user is received, wherein the resource application request includes definition information of a container group Pod, and the definition information includes a topology scheduling requirement of the Pod for resources;

[0119] In step S22, the topology scheduling requirements of the Pod for resources and the topology information of the resources of each node are obtained;

[0120] In step S23, based on the topological information of the resources of each node, a node that meets the topological scheduling requirement is selected from each node as a candidate node for the Pod;

[0121] In step S24, resources are allocated to the Pod based on the candidate nodes.

[0122] In a possible implementation, the additional information of the Pod includes the topology scheduling requirements of the Pod for resources; and obtaining the topology scheduling requirements of the Pod for resources includes: obtaining the topology scheduling requirements of the Pod for resources from the additional information of the Pod.

[0123] In a possible implementation, obtaining the topology scheduling requirements of the Pod for resources includes: when there is a unified topology scheduling requirement in the cluster-level default configuration of the node agent, obtaining the topology scheduling requirement from additional information of the Pod to obtain a Pod-level topology scheduling requirement configuration.

[0124] In a possible implementation, allocating resources to the Pod based on the candidate node includes:

[0125] Determine a target node from the candidate nodes, and allocate the Pod to the target node;

[0126] Obtaining bandwidth information between GPUs in the target node;

[0127] Based on the bandwidth information, a GPU combination with optimal performance is determined and allocated to the Pod.

[0128] In a possible implementation, the resource application request includes a target number of requested GPUs, where the target number is M, and the number of GPUs in the target node is N, where both M and N are positive integers;

[0129] The allocating resources to the Pod based on the candidate node includes:

[0130] Selecting multiple GPU combinations including M GPUs from the N GPUs of the target node;

[0131] Determine the minimum bandwidth for each GPU combination separately;

[0132] A maximum bandwidth value is determined from each minimum bandwidth value, and a GPU combination corresponding to the maximum bandwidth value is used as a GPU combination with optimal performance.

[0133] In a possible implementation, the method further includes:

[0134] Obtain the bandwidth information between GPUs and report the bandwidth information between GPUs to the node label;

[0135] The obtaining bandwidth information between GPUs in the target node includes:

[0136] Get the bandwidth information between GPUs from the labels of the nodes.

[0137] In a possible implementation, allocating resources to the Pod based on the candidate node includes:

[0138] Adding an identifier of the GPU combination with the best performance to the additional information of the Pod;

[0139] An identifier of the GPU combination in the additional information of the Pod is obtained, and a GPU resource corresponding to the identifier is allocated to the Pod.

[0140] This method has a specific technical connection with the internal structure of the computer system, and can solve the technical problem of how to improve the hardware computing efficiency or execution effect (including reducing the amount of data storage, reducing the amount of data transmission, increasing the hardware processing speed, etc.), thereby obtaining the technical effect of improving the internal performance of the computer system in accordance with the laws of nature.

[0141] In some embodiments, the functions or modules included in the method provided in the embodiments of the present disclosure can be used to implement the system described in the above system embodiment. The specific implementation can refer to the description of the above system embodiment, and for the sake of brevity, it will not be repeated here.

[0142] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0143] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0144] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0145] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0146] Figure 4 1 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Figure 4 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0147] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system (MacOSX) launched by Apple TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0148] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0149] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0150] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0151] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0152] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0153] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0154] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0155] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0156] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0157] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0158] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0159] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0160] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0161] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A topology-aware scheduling system, characterized in that: include: A request receiving module receives a resource application request submitted by a user, wherein the resource application request includes definition information of a container group Pod, and the definition information includes a topology scheduling requirement of the Pod for resources; The scheduler obtains the topological scheduling requirements of the Pod for resources and the topological information of the resources of each node; The scheduler selects a node that meets the topology scheduling requirement from each node based on the topology information of the resources of each node as a candidate node for the Pod; The scheduler allocates resources to the Pod based on the candidate nodes.

2. The system according to claim 1, characterized in that The additional information of the Pod includes the topology scheduling requirements of the Pod for resources; The scheduler obtains the topology scheduling requirements of the Pod for resources from the additional information of the Pod.

3. The system according to claim 2, characterized in that In the case where a unified topology scheduling requirement exists in the cluster-level default configuration of the node agent, the scheduler uses the topology scheduling requirement obtained from the additional information of the Pod to obtain the topology scheduling requirement configuration at the Pod level.

4. The system according to claim 1, characterized in that: The scheduler determines a target node from the candidate nodes and allocates the Pod to the target node; The scheduler obtains bandwidth information between GPUs in the target node; The scheduler determines a GPU combination with optimal performance based on the bandwidth information and allocates the GPU combination to the Pod.

5. The system according to claim 4, characterized in that The resource application request includes a target number of requested GPUs, where the target number is M, and the number of GPUs in the target node is N, where both M and N are positive integers; The scheduler selects a plurality of GPU combinations including M GPUs from the N GPUs of the target node; The scheduler determines the minimum bandwidth of each GPU combination respectively; A maximum bandwidth value is determined from each minimum bandwidth value, and a GPU combination corresponding to the maximum bandwidth value is used as a GPU combination with optimal performance.

6. The system according to claim 4, characterized in that The system further comprises: The bandwidth information acquisition module is used to obtain the bandwidth information between GPUs and report the bandwidth information between GPUs to the label of the node; The scheduler obtains bandwidth information between GPUs from the labels of the nodes.

7. The system according to claim 4, characterized in that: The scheduler adds the identifier of the GPU combination with the best performance to the additional information of the Pod; The system further includes: a resource allocation module, which obtains the identification of the GPU combination in the additional information of the Pod, and allocates the GPU resources corresponding to the identification to the Pod.

8. A topology-aware scheduling method, characterized in that: include: Receive a resource application request submitted by a user, wherein the resource application request includes definition information of a container group Pod, and the definition information includes a topology scheduling requirement of the Pod for resources; Obtain the topological scheduling requirements of the Pod for resources and the topological information of the resources of each node; Based on the topological information of the resources of each node, a node that meets the topological scheduling requirements is selected from each node as a candidate node for the Pod; Based on the candidate nodes, resources are allocated to the Pod.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to implement the system described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the system according to any one of claims 1 to 7 is implemented.

Images