Cluster node scheduling method, GPU server scheduling method, and device

Abstract

A cluster node scheduling method, a GPU server scheduling method, and a device. In the method, on the basis of a physical network topology in a cluster and network latency and bandwidths between virtual network interfaces of different physical nodes, communication costs between different physical nodes are determined, and thus, dynamically perceived network latency and network bandwidths between different virtual network interfaces are integrated on the basis of the physical network topology; division is performed on the basis of the communication costs between different physical nodes to obtain node groups, and the communication costs between different node groups are much greater than the communication costs between nodes in a same node group; physical nodes are allocated for a target task on the basis of the node groups obtained by division, and thus, allocation of nodes across groups for the same task can be avoided as far as possible and physical nodes in a same node group are allocated for the same task as far as possible.

Description

Methods and devices for cluster node scheduling and GPU server scheduling

[0001] This disclosure claims priority to Chinese Patent Application No. 202411054225.5, filed with the China Patent Office on August 1, 2024, entitled “Method and Apparatus for Cluster Node Scheduling and GPU Server Scheduling”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to computer technology, and in particular to a method and apparatus for cluster node scheduling and GPU server scheduling. Background Technology

[0003] With the rapid development of Artificial Intelligence (AI), the use cases for large-scale models and big data tasks are constantly increasing, and the complexity of these tasks is also rising. For example, the pre-training task of large language models based on massive amounts of data needs to rely on clusters containing multiple physical nodes to complete the task. The synchronization latency between physical nodes in the cluster needs to be at the millisecond or even microsecond level. This necessitates minimizing the network latency between different physical nodes executing the same task during task deployment, thereby optimizing task deployment at the network performance level.

[0004] For computationally intensive tasks such as big data training, multiple physical nodes need to collaborate. When allocating physical nodes for tasks, the majority of the time, the cluster's physical architecture is observed, and multiple closely spaced physical nodes are assigned based on the distance between them. This approach fails to dynamically assess the actual network performance between physical nodes, resulting in long network latency between nodes executing the same task. Inappropriate physical node allocation negatively impacts task execution efficiency and stability. Summary of the Invention

[0005] This disclosure provides a method and device for cluster node scheduling and GPU server scheduling to solve the problem that unreasonable allocation of physical nodes leads to long network latency between physical nodes executing the same task, thereby affecting the execution efficiency and stability of the task.

[0006] In a first aspect, this disclosure provides a cluster node scheduling method, wherein the cluster includes multiple physical nodes, each physical node having at least one virtual network interface, the method comprising:

[0007] Based on the physical network topology of the physical nodes in the cluster, and the network latency and bandwidth between the virtual network interfaces of different physical nodes, determine the communication cost between any two physical nodes; based on the communication cost between any two physical nodes, divide the multiple physical nodes into multiple node groups, wherein the communication cost between different node groups is greater than the communication cost between different physical nodes within the same node group; based on the multiple node groups, allocate physical nodes for the target task.

[0008] Secondly, this disclosure provides a cluster node scheduling method, including:

[0009] In response to a scheduling request for a target task, information about multiple node groups into which the cluster is divided is obtained. Each node group contains at least one physical node, and the communication cost between different node groups is greater than the communication cost between different physical nodes within the same node group. Based on the multiple node groups, physical nodes are allocated to the target task. The multiple node groups in the cluster are determined through the following steps: determining the communication cost between any two physical nodes based on the physical network topology of the physical nodes in the cluster, and the network latency and bandwidth between the virtual network interfaces of different physical nodes; and dividing the multiple physical nodes into multiple node groups based on the communication cost between any two physical nodes.

[0010] Thirdly, this disclosure provides a GPU server scheduling method applied to a scheduling server in a cluster, the cluster including multiple GPU servers, each GPU server having at least one virtual network interface, including:

[0011] Based on the physical network topology of the GPU servers in the cluster, and the network latency and bandwidth between the virtual network interfaces of different GPU servers, determine the communication cost between any two GPU servers; based on the communication cost between any two GPU servers, divide the multiple GPU servers into multiple server groups, wherein the communication cost between different server groups is greater than the communication cost between different GPU servers within the same server group; based on the multiple server groups, allocate GPU servers for the target task.

[0012] Fourthly, this disclosure provides a GPU server scheduling method, which includes:

[0013] In response to a GPU server scheduling request for a given model training task, information about multiple server groups into which the cluster is divided is obtained, each server group containing at least one GPU server. Based on the information about the multiple server groups, at least one GPU server is allocated to the given model training task, and the at least one GPU server is used to execute the given model training task. The multiple node groups in the cluster are determined through the following steps: based on the physical network topology of the GPU servers in the cluster, and the network latency and bandwidth between the virtual network interfaces of different GPU servers, the communication cost between any two GPU servers is determined; based on the communication cost between any two GPU servers, the multiple GPU servers are divided into multiple server groups, where the communication cost between different server groups is greater than the communication cost between different GPU servers within the same server group.

[0014] Fifthly, this disclosure provides a server, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the server to perform the methods provided in any of the preceding aspects.

[0015] In a sixth aspect, this disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method provided in any of the foregoing aspects.

[0016] In a seventh aspect, this disclosure provides a computer program product including a computer program that, when executed by a processor, implements the methods provided in any of the foregoing aspects.

[0017] The cluster node scheduling and GPU server scheduling methods and devices disclosed herein determine the communication cost between any two physical nodes based on the physical network topology of the physical nodes in the cluster and the network latency and bandwidth between the virtual network interfaces of different physical nodes. By dynamically sensing the network latency and bandwidth between the virtual network interfaces of different physical nodes and integrating the dynamically sensed network latency and bandwidth based on the physical network topology of the physical nodes, the communication cost between different physical nodes can be determined, which can better quantify and characterize the network performance between physical nodes. Furthermore, based on the communication cost between any two physical nodes, multiple physical nodes are divided into multiple node groups, making the communication cost between different node groups much greater than the communication cost between different physical nodes within the same node group. The physical nodes in the cluster are clustered and aggregated based on communication cost to obtain more reasonable node groups. Furthermore, based on multiple node groups, physical nodes are allocated to target tasks. By minimizing the allocation of nodes across groups for the same task and allocating physical nodes within the same node group for the same task as much as possible, the network latency between physical nodes executing the same task can be reduced, improving the efficiency and stability of task execution. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0019] Figure 1 is an example diagram of a cluster physical architecture provided in this disclosure;

[0020] Figure 2 is a schematic diagram of an example system architecture to which this disclosure applies;

[0021] Figure 3 is a flowchart of a cluster node scheduling method provided in an exemplary embodiment of this disclosure;

[0022] Figure 4 is a flowchart of a method for determining the communication cost between physical nodes provided in an exemplary embodiment of this disclosure;

[0023] Figure 5 is a flowchart of physical cluster node grouping provided in an exemplary embodiment of this disclosure;

[0024] Figure 6 is an example diagram of a node group of a physical cluster provided in an exemplary embodiment of this disclosure;

[0025] Figure 7 is a flowchart of a cluster node scheduling method provided in another exemplary embodiment of this disclosure;

[0026] Figure 8 is a flowchart of a GPU server scheduling method provided in an exemplary embodiment of this disclosure;

[0027] Figure 9 is a flowchart of a GPU server scheduling method provided in another exemplary embodiment of this disclosure;

[0028] Figure 10 is a schematic diagram of the structure of a scheduling server provided in an embodiment of this disclosure.

[0029] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0031] It should be noted that the user information (including but not limited to user device information, user attribute information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0032] First, let's explain the terms used in this disclosure:

[0033] Kubernetes: An open-source container orchestration platform for managing physical servers and deploying services according to certain rules.

[0034] Physical cluster: refers to a cluster formed by using software platforms such as Kubernetes to manage multiple physical servers in a unified manner.

[0035] Physical network topology is a type of network topology that focuses on the actual cabling and connection layout (physical signal transmission medium). It describes the actual connection methods of hardware devices in a network, including the physical layout of devices such as computers, servers, switches, and routers, and the physical connections between them.

[0036] A virtual network interface, also known as a bond, combines two or more physical network interfaces (such as network interface cards) to form a virtual network interface. This virtual network interface has its own Internet Protocol Address (IP address) and Media Access Control Address (MAC address) and can distribute data packets among multiple physical network interfaces to increase network bandwidth and improve network reliability. Typically, a host can have one or more bonds.

[0037] Cluster aggregation is a common data aggregation algorithm that divides the nodes in a network into several clusters, each containing one or more nodes.

[0038] Graphics Processing Unit (GPU): Originally designed to accelerate the rendering of graphics displays, GPUs are usually integrated into computer graphics cards or exist as standalone graphics cards. However, with the development of technology, GPUs are now also widely used for non-graphics computationally intensive tasks, such as deep learning, scientific computing, and data analysis.

[0039] Network performance refers to the ability of a network system to transmit data, and it is measured by various factors and indicators. A high-performance network can process and transmit data efficiently, reliably, and quickly, while a low-performance network may cause problems such as latency, packet loss, and unstable connections.

[0040] Connectivity: Indicates whether data can be transmitted between network interfaces and the transmission performance.

[0041] Large models refer to deep learning models with a massive number of parameters, typically containing hundreds of millions, tens of billions, or even trillions of parameters. Large models are also known as foundation models (FM), which are pre-trained on large-scale unlabeled corpora to produce pre-trained models with hundreds of millions of parameters. These models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and Multi-modal Pre-training Models.

[0042] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as natural language processing and computer vision. Specifically, they can be applied to computer vision tasks such as visual question answering (VQA), image captioning (IC), and image generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios of large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0043] In complex scenarios combining Artificial Intelligence (AI) and big data, tasks such as large-scale model training and inference require significant computational resources and necessitate collaboration among multiple GPU servers. In practical applications, for these tasks, multiple GPU servers within a cluster are selected and allocated to each task. Network bandwidth, latency, and jitter among these shared GPU servers can significantly impact task efficiency and stability, potentially causing training instability issues.

[0044] Figure 1 is an example diagram of a cluster physical architecture provided in this disclosure. As shown in Figure 1, the physical cluster is a three-layer physical architecture. The top layer is the aggregation layer switch (PSW), including PSW1 and PSW2. The middle layer is the access layer switch (ASW), including ASW1, ASW2, ASW3, and ASW4. The bottom layer is the physical nodes, including physical node 1, physical node 2, physical node 3, and physical node 4. The physical nodes are different in different scenarios, such as various types of servers or computing nodes such as central processing units (CPUs) and GPUs. Figure 1 illustrates this example with four virtual network interfaces (i.e., bonds) per physical node. In other physical clusters, different physical nodes may have the same or different numbers of virtual network interfaces. The number of virtual network interfaces of each physical node can be configured according to actual application requirements, and no specific limitation is made here.

[0045] In a physical cluster, each physical node can have one or more virtual network interfaces. Ideally, different virtual network interfaces on different physical nodes should be able to communicate with each other. However, due to network load, hardware failures, and other reasons, the latency and bandwidth performance of different virtual network interfaces may vary, and some virtual network interfaces may even be unable to communicate with each other.

[0046] In some complex task scenarios, for tasks with large computational loads that require the collaboration of multiple physical nodes, current solutions typically allocate physical nodes to tasks based on the distance between physical nodes in the cluster's physical architecture. This statically represents the network performance between physical nodes, failing to dynamically perceive the actual network performance. Consequently, the network latency between physical nodes executing the same task becomes long, and the unreasonable allocation of physical nodes affects the efficiency and stability of task execution.

[0047] To address the aforementioned technical issues, this disclosure provides a cluster resource scheduling method. Based on the physical network topology of physical nodes in the cluster, and the network latency and bandwidth between the virtual network interfaces of different physical nodes, the communication cost between any two physical nodes is determined. By dynamically sensing the network latency and bandwidth between the virtual network interfaces of different physical nodes, and integrating the dynamically sensed network latency and bandwidth based on the physical network topology of the physical nodes, the communication cost between different physical nodes is determined, which can better quantify the network performance between physical nodes. Furthermore, based on the communication cost between any two physical nodes, multiple physical nodes are divided into multiple node groups, making the communication cost between different node groups much greater than the communication cost between different physical nodes within the same node group. The physical nodes in the cluster are clustered and aggregated based on communication cost to obtain more reasonable node groups. Furthermore, based on multiple node groups, physical nodes are allocated to target tasks. By minimizing the allocation of nodes across groups for the same task and allocating physical nodes within the same node group as much as possible for the same task, the network latency between physical nodes executing the same task can be reduced, improving the efficiency and stability of task execution.

[0048] Figure 2 is a schematic diagram of an example system architecture to which this disclosure applies. As shown in Figure 2, the system architecture includes a physical cluster, a scheduling server, and end-side devices. The physical cluster includes multiple physical nodes (physical node i and physical node j shown in the figure are two representative nodes in the physical cluster), and each physical node has at least one virtual network interface. Physical nodes can communicate with each other via the virtual network interfaces.

[0049] A scheduling server is a computing-capable device deployed in the cloud, such as a cloud cluster, responsible for managing and scheduling physical nodes within a physical cluster. Specifically, the scheduling server acquires the physical network topology of the physical nodes in the physical cluster, as well as network information such as network latency and bandwidth between the virtual network interfaces of different physical nodes. Based on the acquired network information, it calculates the communication cost between physical nodes to dynamically determine the network performance between them. Furthermore, the scheduling server is also responsible for dividing the physical nodes in the physical cluster into multiple node groups based on the communication cost. This ensures that the communication cost between physical nodes within the same node group is low, the communication cost between physical nodes across node groups is high, and the communication cost between different node groups is significantly higher than the communication cost between different physical nodes within the same node group, thus achieving clustering and aggregation of physical nodes in the cluster based on communication cost.

[0050] Edge devices can be electronic devices that request the allocation of physical nodes for a target task, such as electronic devices requesting big data training or large model inference. Specifically, they can be hardware devices with network communication, computing, and information display capabilities, including but not limited to smartphones, tablets, desktop computers, local servers, and cloud servers. When an edge device needs to run a target task, it sends a scheduling request to the scheduling server. This request contains the task's requirements information, indicating the task's data volume, computational load, and computing power requirements. Based on the task requirements information, the number of physical nodes to be allocated for the task, or the computing power of those physical nodes, can be determined.

[0051] Upon receiving a scheduling request for a target task, the scheduling server allocates physical nodes to the target task based on the multiple node groups divided into physical clusters, minimizing the number of node groups involved in the allocated physical nodes. Furthermore, the scheduling server returns information about the physical nodes allocated to the target task to the end-device.

[0052] The edge device can deploy the target task to the assigned physical node based on the information of the physical node allocated to the target task, and execute the target task through the assigned physical node.

[0053] The technical solutions of this disclosure and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0054] Figure 3 is a flowchart of a cluster node scheduling method provided in an exemplary embodiment of this disclosure. The execution entity in this embodiment is the scheduling server in the aforementioned system architecture. As shown in Figure 3, the specific steps of the method are as follows:

[0055] Step S301: Determine the communication cost between any two physical nodes based on the physical network topology of the physical nodes in the cluster, as well as the network latency and bandwidth between the virtual network interfaces of different physical nodes.

[0056] Virtual network interfaces refer to the bonds of physical nodes, which are logical interfaces formed by combining two or more physical network interfaces (such as network cards). Virtual network interfaces have their own IP address and MAC address, which can improve network bandwidth and reliability.

[0057] In this embodiment, the scheduling server can obtain the physical network topology of the physical nodes in the cluster, and obtain the network latency and bandwidth between the virtual network interfaces of different physical nodes.

[0058] The physical network topology of the physical nodes in the cluster can be pre-stored on the scheduling server. The network latency and bandwidth between the virtual network interfaces of different physical nodes can be collected in real time by the physical nodes and provided to the scheduling server.

[0059] Optionally, each physical node in the cluster can periodically collect the network latency and bandwidth between its own virtual network interface and the virtual network interfaces of other physical nodes, and report the network latency between its own virtual network interface and the virtual network interfaces of other physical nodes to the scheduling server.

[0060] Optionally, each physical node in the cluster can periodically collect network latency and bandwidth between its own virtual network interface and the virtual network interfaces of other physical nodes, and store / update the newly collected data in the database. The scheduling server can retrieve network information for all physical nodes from the database, including network latency and bandwidth between the virtual network interfaces of the physical nodes.

[0061] Furthermore, the scheduling server calculates the communication cost between any two physical nodes based on the physical network topology of the physical nodes in the cluster, as well as the network latency and bandwidth between the virtual network interfaces of different physical nodes.

[0062] For example, the scheduling server can determine the number of switch hops traversed by the virtual network interfaces of different physical nodes based on the physical network topology of the physical nodes, map the number of switch hops traversed by the virtual network interfaces of different physical nodes, network latency, and bandwidth to corresponding performance indicators, and then calculate a comprehensive indicator based on the performance indicators corresponding to the number of switch hops, network latency, and bandwidth, as the communication cost between the two physical nodes.

[0063] For example, the scheduling server can perform a weighted summation of the number of switch hops, network latency, and bandwidth traversed for communication between virtual network interfaces of different physical nodes based on pre-configured weighting coefficients corresponding to the number of switch hops, network latency, and bandwidth, to obtain the communication cost between two physical nodes.

[0064] Step S302: Based on the communication cost between any two physical nodes, divide the multiple physical nodes into multiple node groups, wherein the communication cost between different node groups is greater than the communication cost between different physical nodes within the same node group.

[0065] After determining the communication cost between any two physical nodes, the multiple physical nodes are grouped according to the communication cost between any two physical nodes to obtain multiple node groups, wherein the communication cost between different node groups is greater than the communication cost between different physical nodes within the same node group.

[0066] For example, in this step, the scheduling server can use a clustering aggregation algorithm to cluster the physical nodes in the cluster according to the communication cost between any two physical nodes, resulting in multiple clusters, each containing at least one physical node. Each cluster is then treated as a node group.

[0067] Optionally, after aggregating the physical nodes in the physical cluster into multiple node groups according to communication cost through the aforementioned steps, the server can also output information about each node group, including but not limited to: the physical nodes contained in each node group. In addition, it can also output information such as communication cost information between physical nodes within a node group and the average communication cost between different node groups to provide reference information for resource scheduling.

[0068] Step S303: Assign physical nodes to the target task based on multiple node groups.

[0069] In this embodiment, a node scheduling policy based on node groups can be pre-configured in the scheduling server. In this step, the scheduling server, based on information about the multiple node groups into which the cluster is divided, uses the configured node scheduling policy to allocate physical nodes to the target task, minimizing the number of node groups involved in the physical nodes allocated to the target task. By minimizing the number of node groups involved in the physical nodes allocated to the target task, the situation of allocating physical nodes across groups for the same task can be reduced as much as possible. This ensures that the physical nodes allocated to the target task belong to the same node group or span a small number of node groups, thereby minimizing network latency between physical nodes executing the same task and improving task execution efficiency and stability.

[0070] For example, in a node group-based node scheduling strategy, the scheduling server searches the cluster for node groups that can meet the requirements of the target task (such as the number of physical nodes required, computing power, etc.). If a node group that meets the requirements is found, the scheduling server allocates physical nodes to the target task from that node group (such as the node group with the fewest physical nodes). If no node group that meets the requirements is found, the minimum number of node groups in the cluster is used to meet the requirements.

[0071] For example, if no node group that meets the requirements of the current task is found, the scheduling server can first select all physical nodes from the node group containing the largest number of physical nodes and allocate them to the current task. The number of physical nodes required by the current task is subtracted from the already allocated physical nodes to obtain the remaining allocation requirement of the current task (the number of physical nodes still to be allocated). Based on the remaining allocation requirement of the current task and the number of available physical nodes in the remaining node groups, the scheduling server searches for a node group that can meet the remaining allocation requirement of the current task. If a node group that meets the remaining allocation requirement of the current task is found, the scheduling server allocates physical nodes to the target task from one of the node groups that meets the remaining allocation requirement (such as the node group containing the fewest physical nodes). If no node group that meets the remaining allocation requirement of the current task is found, the scheduling server can first select the node group containing the largest number of available physical nodes, allocate the available physical nodes in the selected node group to the current task, calculate the remaining allocation requirement of the current task, and continue to allocate other physical nodes to the current task until the task requirement is met.

[0072] In addition, when it is necessary to allocate some physical nodes in a node group to the current task, it is preferable to choose a node group with a small number of physical nodes so that the number of remaining physical nodes in the node group is small after the allocation is completed.

[0073] Optionally, the scheduling server may employ an optimization algorithm to allocate physical nodes to the target task, with the optimization objective being to minimize the number of node groups involved in allocating physical nodes to the target task.

[0074] In this embodiment, the node scheduling strategy based on node groups used by the scheduling server can be configured and adjusted according to actual application requirements, and no specific limitations are made here.

[0075] In practical applications, the scheduling server can obtain the network latency and bandwidth between the virtual network interfaces of different physical nodes in the cluster at regular intervals. Based on the physical network topology of the physical nodes in the cluster and the network latency and bandwidth between the virtual network interfaces of different physical nodes obtained in real time, it can recalculate the communication cost between any two physical nodes and adjust existing node groups or update the average communication cost between existing node groups based on the recalculated communication cost between any two physical nodes.

[0076] The method in this embodiment determines the communication cost between any two physical nodes based on the physical network topology of the physical nodes in the cluster, as well as the network latency and bandwidth between the virtual network interfaces of different physical nodes. By dynamically sensing the network latency and bandwidth between the virtual network interfaces of different physical nodes, and integrating the dynamically sensed network latency and bandwidth based on the physical network topology of the physical nodes, the communication cost between different physical nodes can be determined, which can better quantify and characterize the network performance between physical nodes. Furthermore, based on the communication cost between any two physical nodes, multiple physical nodes are divided into multiple node groups, so that the communication cost between different node groups is much greater than the communication cost between different physical nodes within the same node group. The physical nodes in the cluster are clustered and aggregated based on communication cost to obtain more reasonable node groups. Furthermore, based on multiple node groups, physical nodes are allocated to the target task. By minimizing the allocation of nodes across groups for the same task and allocating physical nodes within the same node group for the same task as much as possible, the network latency between physical nodes executing the same task can be reduced, and the efficiency and stability of task execution can be improved.

[0077] Figure 4 is a flowchart of a method for determining the communication cost between physical nodes according to an exemplary embodiment of this disclosure. In an optional embodiment, as shown in Figure 4, the communication cost between any two physical nodes is determined in step S301 based on the physical network topology of the physical nodes in the cluster, as well as the network latency and bandwidth between the virtual network interfaces of different physical nodes. The specific implementation steps are as follows:

[0078] Step S401: Obtain the physical network topology of the physical nodes in the cluster, as well as the network latency and bandwidth between the virtual network interfaces of different physical nodes.

[0079] The scheduling server can obtain the physical network topology of the physical nodes in the cluster, and obtain the network latency and bandwidth between the virtual network interfaces of different physical nodes.

[0080] The physical network topology of the physical nodes in the cluster can be pre-stored on the scheduling server. The network latency and bandwidth between the virtual network interfaces of different physical nodes can be collected in real time by the physical nodes and provided to the scheduling server.

[0081] Optionally, each physical node in the cluster can periodically collect the network latency and bandwidth between its own virtual network interface and the virtual network interfaces of other physical nodes, and report the network latency between its own virtual network interface and the virtual network interfaces of other physical nodes to the scheduling server.

[0082] Optionally, each physical node in the cluster can periodically collect network latency and bandwidth between its own virtual network interface and the virtual network interfaces of other physical nodes, and store / update the newly collected data in the database. The scheduling server can retrieve network information for all physical nodes from the database, including network latency and bandwidth between the virtual network interfaces of the physical nodes.

[0083] Step S402: Determine the communication cost between the virtual network interfaces of different physical nodes based on the physical network topology of the physical nodes in the cluster, as well as the network latency and bandwidth between the virtual network interfaces of different physical nodes.

[0084] In this embodiment, the scheduling server determines the number of switch hops traversed for communication between virtual network interfaces of different physical nodes based on the physical network topology of the physical nodes. The number of switch hops traversed for communication between virtual network interfaces of different physical nodes, network latency, and bandwidth are mapped to corresponding performance indicators. Then, a comprehensive indicator is calculated by combining the performance indicators corresponding to the number of switch hops, network latency, and bandwidth, which serves as the communication cost between two physical nodes. This allows for a more accurate measurement of the communication cost between different physical nodes.

[0085] For example, using H i and H j This represents two different physical nodes, for those belonging to different physical nodes H i and H j Any two virtual network interfaces (referred to as B respectively) s and B t (This indicates that), based on the physical network topology of the physical nodes in the cluster, two virtual network interfaces B are determined. s and B tThe number of switch hops (denoted by Lv) traversed in the communication between the two virtual network interfaces is used to determine the two virtual network interfaces B. s and B t Hierarchical information representation between (using C) Lv express).

[0086] Based on two virtual network interfaces B s and B t The network latency between the two virtual network interfaces (denoted by l) determines the network latency between them. s and B t Inter-delay information representation (using C) L express).

[0087] Based on two virtual network interfaces B s and B t The bandwidth between the two virtual network interfaces (denoted by b) is used to determine the bandwidth between them. s and B t Inter-bandwidth information representation (using C) B express).

[0088] Furthermore, integrating the two virtual network interfaces B s and B t Hierarchical information representation C Lv Delay information representation C L Bandwidth information is represented by C B Determine the communication cost (denoted by C) between the two virtual network interfaces.

[0089] Considering that the physical topology information between physical nodes represents their physical location relationships, the number of switch hops traversed during communication between two virtual network interfaces is the most critical information affecting network performance between them. Different switch hop counts should clearly distinguish the hierarchical information representation between the two virtual network interfaces. The scheduling server determines the hierarchical information representation between the two virtual network interfaces based on the number of switch hops traversed during communication, which can be implemented in the following way:

[0090] Using the number of switch hops traversed during communication between two virtual network interfaces as the exponent, a value of the exponential function with a specified base is calculated to obtain the hierarchical information representation between the two virtual network interfaces, i.e., C. Lv =N Lv Where N is a specified base, which can be configured and adjusted according to the actual application scenario. For example, N can be 10 or other values. This embodiment does not make specific limitations here.

[0091] Taking a base N=10 as an example, when the number of switch hops traversed for communication between two virtual network interfaces is 1, i.e., Lv=1, the hierarchical information representation C between the two virtual network interfaces is... Lv =10; When the number of switch hops traversed for communication between two virtual network interfaces is 2, i.e., Lv=2, the hierarchical information representation C between the two virtual network interfaces is... Lv =10 2 =100; When the number of switch hops traversed for communication between two virtual network interfaces is 3, i.e., Lv=3, the hierarchical information representation C between the two virtual network interfaces is... Lv =10 3 =1000. Therefore, for virtual network interfaces with different switch hop counts, as the switch hop count increases, the hierarchical information representation between virtual network interfaces grows exponentially, and network performance can be significantly distinguished.

[0092] Optionally, based on the number of switch hops traversed in communication between the two virtual network interfaces, other mapping rules can be used to map the number of switch hops traversed in communication between the two virtual network interfaces to a hierarchical information representation between the two virtual network interfaces. For example, the number of switch hops traversed in communication between the two virtual network interfaces can be multiplied by a large multiple. This implementation does not impose specific limitations here.

[0093] For example, the delay information representation between two virtual network interfaces can be determined based on the network delay between the two virtual network interfaces, which can be implemented in the following way:

[0094] The network latency between the two virtual network interfaces is normalized to a first numerical range to obtain a representation of the latency information between the two virtual network interfaces. The first numerical range can be [0,1] or any other numerical range.

[0095] Optionally, based on the configured network latency threshold L th This can be achieved by calculating the network latency between two virtual network interfaces and the network latency threshold L. th The ratio of the network latency between two virtual network interfaces to the network latency threshold L. th If the ratio is less than 1, then the network latency between the two virtual network interfaces is compared with the network latency threshold L. th The ratio of the network latency between the two virtual network interfaces is used to represent the latency information between them. If the network latency between the two virtual network interfaces is less than or equal to the network latency threshold L... th If the ratio is greater than or equal to 1, then the latency information representation between the two virtual network interfaces is determined to be equal to 1. This process can be expressed as: Where l represents the network latency between the two virtual network interfaces, and min{,} represents taking the minimum value, L thThis represents the configured network latency threshold, which can be set to 100 microseconds. The specific value can be configured and adjusted according to actual application needs, and no specific limitation is made here.

[0096] Optionally, the ratio of the network latency between any two virtual network interfaces to the maximum network latency between different virtual network interfaces in the current physical cluster can be used as the latency information between the two virtual network interfaces.

[0097] For example, the bandwidth information representation between two virtual network interfaces can be determined based on the bandwidth between the two virtual network interfaces. Specifically, this can be achieved by normalizing the bandwidth between the two virtual network interfaces to a second numerical range to obtain the bandwidth information representation between the two virtual network interfaces. The second numerical range can be [0,1] or any other numerical range.

[0098] Optionally, based on the configured network bandwidth threshold B th The network bandwidth threshold B can be calculated by amplifying the bandwidth b between two virtual network interfaces by a factor of M (M > 1) to obtain the amplified bandwidth (M × b). th The ratio of this amplified bandwidth (M×b) to the network bandwidth threshold B is... th If the ratio of the amplified bandwidth (M×b) to the network bandwidth threshold B is less than 1, then the network bandwidth threshold B will be adjusted accordingly. th The ratio of this amplified bandwidth (M×b) is used to represent the bandwidth information between the two virtual network interfaces. If the network bandwidth threshold B... th If the ratio of this amplified bandwidth (M×b) to the given bandwidth is greater than or equal to 1, then the bandwidth information representation between the two virtual network interfaces is determined to be equal to 1. This process can be expressed as: Where b represents the bandwidth between the two virtual network interfaces, and min{,} represents taking the minimum value, B th This indicates the configured network bandwidth threshold, which can be set to 400Gbps. The specific value can be configured and adjusted according to actual application needs, and no specific limit is set here.

[0099] Optionally, the bandwidth between any two virtual network interfaces can be represented by the ratio of the maximum network bandwidth between different virtual network interfaces in the current physical cluster to the maximum network bandwidth.

[0100] Furthermore, the scheduling server represents C based on pre-configured hierarchical information. Lv Delay information representation C L Bandwidth information is represented by C B The corresponding weighting coefficients represent the hierarchical information C between any two virtual network interfaces. Lv Delay information representation C L Bandwidth information is represented by CB We calculate the communication cost C between the two virtual network interfaces by weighted summation, so as to comprehensively consider the impact of physical network topology, network latency and bandwidth of different virtual network interfaces on the communication performance between virtual network interfaces.

[0101] This process can be represented as: C = αC Lv +βC L +γC B Where α, β, and γ represent hierarchical information representation C, respectively. Lv Delay information representation C L Bandwidth information is represented by C B The corresponding weighting coefficients can be configured and adjusted according to the needs of the actual application scenario; no specific limitations are made here.

[0102] In an alternative embodiment, the scheduling server can represent delay information C. L Bandwidth information is represented by C B Configure weighting coefficients δ and (1-δ) respectively. The value of δ ranges from (0,1). Calculate the communication cost between the two virtual network interfaces as follows: Because the latency information between the two virtual network interfaces is represented by C L Bandwidth information is represented by C B The values ​​of are all in the range of [0,1], and the values ​​of δ are in the range of (0,1). Therefore, the delay information between the two virtual network interfaces can be represented as C. L Bandwidth information is represented by C B The impact on communication costs between the two virtual network interfaces is limited to The communication cost between the two virtual network interfaces is limited to Within this range, the communication cost between virtual network interfaces, as determined by this method, is primarily influenced by the physical network topology, while also incorporating real-time network latency and bandwidth, allowing for a more accurate measurement of communication performance between virtual network interfaces.

[0103] Taking a base N=10 as an example, when the number of switch hops traversed for communication between two virtual network interfaces is 1, i.e., Lv=1, the hierarchical information representation C between the two virtual network interfaces is... Lv =10, the communication cost between the two virtual network interfaces is limited to [10,15]. When the number of switch hops traversed for communication between the two virtual network interfaces is 2, i.e., Lv=2, the hierarchical information representation C between the two virtual network interfaces is... Lv =100, the communication cost between the two virtual network interfaces is limited to [100, 150]; when the number of switch hops traversed for communication between the two virtual network interfaces is 3, i.e., Lv = 3, the hierarchical information representation C between the two virtual network interfaces is... Lv=1000, the communication cost between the two virtual network interfaces is limited to [1000, 1500].

[0104] It should be noted that in practical applications, within the same time period, physical node H i Collect its own virtual network interface B s With physical node H j A virtual network interface B t Inter-network latency (using l) st (represented), with physical node H j Collect its own virtual network interface B t With physical node H i Virtual network interface B s Inter-network latency (using l) ts (This may be inconsistent.)

[0105] In an alternative embodiment, the scheduling server determines the scheduling based on the physical node H. i Virtual network interface B collected s With virtual network interface B t Network latency st Calculate virtual network interface B s With virtual network interface B t Delay information representation (using C) L_st (This indicates that), and integrates two virtual network interfaces B. s and B t Hierarchical information representation C Lv Bandwidth information representation C B and virtual network interface B s With B t The delay information represents C L_st Determine virtual network interface B s With B t Communication costs (in C) st express).

[0106] Furthermore, the scheduling server determines the scheduling based on the physical node H. j Virtual network interface B collected t With B s Network latency ts Calculate virtual network interface B t With B s Delay information representation (using C) L_ts (This indicates that), and integrates two virtual network interfaces B. s and B t Hierarchical information representation C Lv Bandwidth information representation C B and virtual network interface B t With Bs The delay information represents C L_ts Determine virtual network interface B t With B s Communication costs (in C) ts express).

[0107] Furthermore, the scheduling server will use virtual network interface B s With B t Communication cost C st and virtual network interface B t With B s Communication cost C ts The average value of the two virtual network interfaces B s and B t The communication cost between them is C, that is, C = (C st +C ts ) / 2.

[0108] In another alternative embodiment, the scheduling server determines the physical node H based on... i Virtual network interface B collected s With virtual network interface B t Network latency st and physical node H j Virtual network interface B collected t With B s Network latency ts Calculate l st and l ts The average value of the two virtual network interfaces B t and B s The network latency l between the two virtual network interfaces B t and B s The network latency between the two virtual network interfaces B determines the network latency between them. s and B t The delay information between C L And integrate two virtual network interfaces B s and B t Hierarchical information representation C Lv Delay information representation C L Bandwidth information is represented by C B Determine the communication cost C between the two virtual network interfaces.

[0109] In an optional embodiment, the latency information between the two virtual network interfaces is represented as greater than a preset maximum network latency (denoted by L). max_th (represented by B), or the bandwidth information between two virtual network interfaces indicates a bandwidth greater than the preset maximum bandwidth (represented by B). max_thIn the case of (indicated by), it can be assumed that the two virtual network interfaces cannot communicate, and the scheduling server can directly set the preset maximum communication cost (indicated by C). max_th (This represents the communication cost between two virtual network interfaces.)

[0110] Among them, the preset maximum network latency, preset maximum bandwidth, and preset maximum communication cost can all be configured and adjusted according to the actual application scenario. For example, the preset maximum network latency L max_th It can be configured to 10 10 microseconds, the preset maximum bandwidth can be configured to 10. -10 Gbps, the preset maximum communication cost can be configured to infinity (∞) or a very large value, without specific limitations here.

[0111] For example, the communication cost C between different virtual network interfaces can be configured as a piecewise function as shown in formula (1):

[0112] The network latency between the two virtual network interfaces is greater than L. max_th Or the bandwidth b between two virtual network interfaces max_th In the event of an anomaly in communication between the two virtual network interfaces, the communication cost between the virtual network interfaces is directly set to the preset maximum communication cost C. max_th .

[0113] By setting segmentation functions, virtual network interfaces with communication anomalies in the physical cluster can be monitored, thereby monitoring the network status between various virtual network interfaces in the physical cluster. Furthermore, based on the virtual network interfaces with communication anomalies, the scheduling server can discover abnormal physical nodes in the physical cluster, thereby removing abnormal physical nodes and ensuring the availability of physical nodes allocated to tasks.

[0114] For example, based on the communication anomaly of the virtual network interface, if any virtual network interface on any physical node A has an communication anomaly with the virtual network interfaces on at least two other different physical nodes (such as physical nodes B and C), then the scheduling server determines that physical node A is an abnormal node.

[0115] For example, based on the communication anomaly of the virtual network interface, if all virtual network interfaces on any physical node A have communication anomalies, that is, if any virtual network interface on physical node A has communication anomalies with at least one virtual network interface on another physical node, then the scheduling server determines that physical node A is an abnormal node.

[0116] ​In addition, the scheduling server can also discover abnormal nodes in the physical cluster based on pre-configured abnormal node judgment rules. The abnormal node judgment rules can be configured and adjusted according to the needs of actual application scenarios, and no specific limitations are made here.

[0117] Step S403: Determine the communication cost between any two physical nodes based on the communication cost between the virtual network interfaces of different physical nodes.

[0118] In this step, for any two physical nodes, the average communication cost between any two virtual network interfaces of the two physical nodes is taken as the communication cost between the two physical nodes.

[0119] For example, based on the communication cost between the virtual network interfaces of different physical nodes determined in the aforementioned steps, the cost of any two physical nodes H is calculated using the following formula (2). i and H j Communication costs between them:

[0120] Among them, CH ij Represents any two physical nodes H i and H j The communication costs between them. B s ∈H i Represents physical node H i Any virtual network interface. B t ∈H j Represents physical node H j Any virtual network interface. |H i | Represents physical node H i The number of virtual network interfaces. |H j | Represents physical node H j The number of virtual network interfaces. Indicates two virtual network interfaces B s and B t Communication costs between them.

[0121] In this embodiment, the scheduling server obtains the physical network topology of the physical nodes in the cluster, as well as the network latency and bandwidth between the virtual network interfaces of different physical nodes. Based on the physical network topology of the physical nodes in the cluster, and the network latency and bandwidth between the virtual network interfaces of different physical nodes, the communication cost between the virtual network interfaces of different physical nodes is determined. Based on the communication cost between the virtual network interfaces of different physical nodes, the communication cost between any two physical nodes is determined. By dynamically sensing the network latency and bandwidth between the virtual network interfaces of different physical nodes, and integrating the dynamically sensed network latency and bandwidth on the basis of the physical network topology of the physical nodes, the communication cost between different physical nodes can be determined, which can better quantify and characterize the communication performance between physical nodes.

[0122] Figure 5 is a flowchart of physical cluster node grouping provided by an exemplary embodiment of this disclosure. In an optional embodiment, the aforementioned step S302, which divides multiple physical nodes into multiple node groups based on the communication cost between any two physical nodes, can be implemented as follows: traverse multiple physical nodes and perform the following grouping process on the current physical node: calculate the communication cost between the current physical node and existing node groups respectively; if there is at least one node group whose communication cost with the current physical node is less than the communication cost threshold, then add the current physical node to the node group with the lowest communication cost with the current physical node; if there is no node group whose communication cost with the current physical node is less than the communication cost threshold, then create a new node group containing the current physical node.

[0123] As shown in Figure 5, the specific steps of this method are as follows:

[0124] Step S501: Traverse the physical nodes in the cluster.

[0125] In this embodiment, the physical nodes in the cluster are traversed sequentially. When traversing the physical nodes in the cluster, the traversal can be performed in any order without affecting the final grouping result.

[0126] Perform grouping processing steps S502-S505 on the current physical node that is traversed until all physical nodes in the physical cluster have been grouped, and then complete the clustering and aggregation of the physical cluster to obtain multiple node groups.

[0127] Step S502: For the current physical node that has been traversed, calculate the communication cost between the current physical node and the existing node group.

[0128] In this step, when calculating the communication cost between the current physical node and any node group, the scheduling server calculates the average of the usual costs between the current physical node and each existing physical node in the node group, which is used as the communication cost between the current physical node and the node group.

[0129] In another optional embodiment, when calculating the communication cost between the current physical node and any node group, the scheduling server may also use the maximum value among the existing communication costs between the current physical node and each physical node in the node group as the communication cost between the current physical node and the node group.

[0130] Step S503: Determine whether there is a node group whose communication cost with the current physical node is less than the communication cost threshold.

[0131] Among them, the communication cost threshold is an empirical threshold that can be configured and adjusted according to the actual situation of different clusters, so that the communication between physical nodes in the same group is restricted to the domain corresponding to the same access layer switch node ASW as much as possible.

[0132] Optionally, the scheduling server can also determine the communication cost threshold based on the maximum value among the range of communication costs between different physical nodes connected to the same switch node (i.e., the number of hops through which communication passes is 1).

[0133] For example, based on the implementation principle of calculating the communication cost between different physical nodes in the foregoing embodiments, the communication cost between different physical nodes is C = αC. Lv +βC L +γC B , where C L The maximum value is 1, C B The maximum value is 1, and the maximum communication cost between different physical nodes is (αC). Lv＝1 +β+γ), the communication cost threshold can be configured as (αC) Lv＝1 +β+γ), or in (αC Lv＝1 Add a fault tolerance value to (+β+γ). The fault tolerance value is a small positive number that can be configured according to the actual application scenario; no specific limitation is made here.

[0134] Optionally, the historical maximum communication cost between physical nodes within the same ASW domain in the cluster can be calculated based on historical network data over a certain period of time, and used as the communication cost threshold.

[0135] Optionally, the communication cost threshold can be dynamically adjusted according to the actual situation of the network in the cluster. Specifically, it can be determined as follows: based on the communication cost between any two virtual network interfaces currently calculated, these communication costs are divided into multiple buckets according to their size. Each bucket corresponds to a communication cost range, and the number of communication costs falling into each bucket in the communication cost between any two virtual network interfaces currently calculated is determined.

[0136] For example, suppose communication costs are divided into buckets corresponding to the following communication cost ranges: (0,10], (10,100], (100,1000], and (1000,10000). The number of communication costs falling into each bucket is 32, 18, 25, and 39, respectively. t buckets are determined sequentially such that the total number of communication costs falling into the first t buckets exceeds a preset proportion (e.g., 40%) of the total communication costs. The maximum value of the communication cost range in the first t buckets or the minimum value of the communication cost range in the (t+1)th bucket is used as the communication cost threshold. Taking a preset proportion of 40% as an example, the calculated total communication cost is 114 (the sum of 32, 18, 25, and 39), and 40% of the total communication cost is 45.6. It can be calculated that the total number of communication costs falling into the first two buckets (32+18) exceeds 45.6. The maximum value of the communication cost range in the first two buckets, 100, is used as the communication cost threshold.

[0137] In this step, the scheduling server determines whether there is a node group whose communication cost with the current physical node is less than the communication cost threshold, based on the configured communication cost threshold and the communication cost between the current physical node and the existing node group.

[0138] If there exists at least one node group whose communication cost with the current physical node is less than the communication cost threshold, then step S504 is executed to add the current physical node to one of the node groups.

[0139] If no node group has a communication cost less than the communication cost threshold with the current physical node, then proceed to step S505 to create a new node group and add the current physical node to the new node group.

[0140] Step S504: If there exists at least one node group whose communication cost with the current physical node is less than the communication cost threshold, then add the current physical node to the node group with the lowest communication cost with the current physical node.

[0141] If there exists at least one node group whose communication cost with the current physical node is less than the communication cost threshold, the node group with the lowest communication cost with the current physical node can be selected from these at least one node group, and the current physical node can be added to the node group with the lowest communication cost with the current physical node.

[0142] Step S505: If there is no node group whose communication cost with the current physical node is less than the communication cost threshold, then create a new node group containing the current physical node.

[0143] Step S506: Determine if there are any untraversed physical nodes.

[0144] After grouping the current physical nodes, determine if there are any unvisited physical nodes, i.e., if there are any physical nodes that have not yet been grouped. If there are unvisited physical nodes, proceed to step S501 to continue traversing the next physical node in the cluster. After traversing all physical nodes, multiple node groups of the cluster are obtained.

[0145] For example, Figure 6 is an example diagram of node groups in a physical cluster provided in this embodiment. As shown in Figure 6, the physical cluster is divided into three node groups: Group1, Group2, and Group3. Node group Group1 contains the following four physical nodes: H1, H2, H3, and H4. Node group Group2 contains the following three physical nodes: H5, H6, and H7. Node group Group3 contains the following two physical nodes: H8 and H9. It should be noted that the subscripts of different physical nodes in Figure 6 are only used to distinguish different physical nodes and are not related to the position of the physical nodes.

[0146] In this embodiment, the scheduling server traverses multiple physical nodes and performs the following grouping process on the current physical node: It calculates the communication cost between the current physical node and existing node groups; if at least one node group has a communication cost less than a communication cost threshold with the current physical node, the current physical node is added to the node group with the lowest communication cost; if no node group has a communication cost less than the communication cost threshold with the current physical node, a new node group containing the current physical node is created to group the physical nodes in the cluster based on communication cost. This grouping ensures that physical nodes with lower communication costs are grouped into the same group, resulting in lower communication costs between physical nodes within the same group, higher communication costs between physical nodes belonging to different node groups, and significantly higher communication costs between different node groups than between different physical nodes within the same node group, thus improving the rationality of physical node grouping.

[0147] Based on any of the aforementioned embodiments, after obtaining multiple node groups of the physical cluster, the scheduling server can also calculate the average communication cost between different node groups based on the communication cost between physical nodes within different node groups, and store information on multiple node groups and the average communication cost between different node groups, providing a data foundation for physical node scheduling.

[0148] Specifically, for any two node groups G m and G n Two node groups G m and G n The average communication cost between them can be calculated using the following formula (3):

[0149] Among them, CG mnG represents two node groups m and G n The average communication cost between them. H i ∈G m Represents node group G m For any physical node in |G m | Represents node group G m The number of physical nodes in H. j ∈G n Represents node group G n For any physical node in |G n | Represents node group G n The number of physical nodes in CH. ij Represents physical node H i and H j Communication costs between them.

[0150] Furthermore, in response to a scheduling request for a target task, the scheduling server allocates at least one physical node to the target task based on information from multiple node groups and the average communication cost between different node groups. At least one physical node collaborates to complete the target task.

[0151] In this embodiment, a node scheduling strategy based on node groups is pre-configured in the scheduling server. Based on the information of the multiple node groups into which the cluster is divided and the average communication cost between different node groups, the scheduling server uses the configured node scheduling strategy to allocate physical nodes to the target task, so as to minimize the number of node groups involved in the physical nodes allocated to the target task.

[0152] For example, in a node group-based scheduling strategy, the scheduling server searches the cluster for node groups that can meet the requirements of the target task (such as the number of physical nodes required, computing power, etc.). If a node group that meets the requirements is found, the scheduling server allocates physical nodes to the target task from that node group (such as the node group with the fewest physical nodes). If no node group that meets the requirements is found, the server uses the fewest possible node groups with the lowest average communication cost among different node groups to meet the current task requirements.

[0153] For example, if no node group is found that meets the requirements of the current task, the scheduling server can first select the node group with the largest number of physical nodes (i.e., all physical nodes in the group) and allocate it to the current task. The number of physical nodes required by the current task is subtracted from the number of allocated physical nodes to obtain the remaining allocation requirements of the current task (the number of physical nodes still needing allocation). Based on the remaining allocation requirements of the current task and the number of available physical nodes in the remaining node groups, the scheduling server searches for node groups that can meet the remaining allocation requirements of the current task. If a node group that meets the remaining allocation requirements of the current task is found, the scheduling server selects the node group with the lower average communication cost with the already allocated node groups and allocates it to the current task. If no node group that meets the remaining allocation requirements of the current task is found, the scheduling server can first select the node group with the largest number of available physical nodes, or select the node group with the lowest average communication cost with the already allocated node groups, allocate the selected node group to the current task, calculate the remaining allocation requirements of the current task, and continue to allocate other physical nodes to the current task until the task requirements are met.

[0154] In addition, when it is necessary to allocate some physical nodes in a node group to the current task, it is preferable to choose a node group with a small number of physical nodes so that the number of remaining physical nodes in the node group is small after the allocation is completed.

[0155] Optionally, the scheduling server may employ an optimization algorithm to allocate physical nodes to the target task, with the optimization objective being to minimize the number of node groups involved in allocating physical nodes to the target task.

[0156] In this embodiment, the node scheduling strategy based on node groups used by the scheduling server can be configured and adjusted according to actual application requirements, and no specific limitations are made here.

[0157] Figure 7 is a flowchart of a cluster node scheduling method provided in an exemplary embodiment of this disclosure. As shown in Figure 7, the specific steps of the method are as follows:

[0158] Step S701: In response to the scheduling request for the target task, obtain information about the multiple node groups into which the cluster is divided. Each node group contains at least one physical node. The communication cost between different node groups is greater than the communication cost between different physical nodes within the same node group.

[0159] In this embodiment, when the edge device needs to run a target task, it sends a scheduling request for the target task to the scheduling server. This scheduling request contains the task requirement information of the target task. The task requirement information indicates the data volume, computational load, computing power requirements, etc., of the task. Based on the task requirement information, the number of physical nodes to be allocated for the task, or the computing power of physical nodes to be allocated, can be determined.

[0160] In response to receiving a scheduling request for the target task, the scheduling server obtains information about the multiple node groups into which the physical cluster is divided.

[0161] In this embodiment, the cluster is divided into multiple node groups through the following steps: Based on the physical network topology of the physical nodes in the cluster, and the network latency and bandwidth between the virtual network interfaces of different physical nodes, the communication cost between any two physical nodes is determined; based on the communication cost between any two physical nodes, the multiple physical nodes are divided into multiple node groups. For specific implementation principles and technical effects, please refer to the aforementioned method embodiment; this embodiment will not repeat them here.

[0162] Step S702: Assign physical nodes to the target task based on multiple node groups.

[0163] For the specific implementation principle and technical effects of this step, please refer to the relevant content in the previous embodiment. This embodiment will not repeat it here.

[0164] The method in this embodiment determines the communication cost between any two physical nodes based on the physical network topology of the physical nodes in the cluster, as well as the network latency and bandwidth between the virtual network interfaces of different physical nodes. By dynamically sensing the network latency and bandwidth between the virtual network interfaces of different physical nodes, and integrating the dynamically sensed network latency and bandwidth based on the physical network topology of the physical nodes, the communication cost between different physical nodes can be determined, which can better quantify and characterize the network performance between physical nodes. Furthermore, based on the communication cost between any two physical nodes, multiple physical nodes are divided into multiple node groups, so that the communication cost between different node groups is much greater than the communication cost between different physical nodes within the same node group. The physical nodes in the cluster are clustered and aggregated based on the communication cost to obtain more reasonable node groups. Furthermore, in response to the scheduling request for the target task, physical nodes are allocated to the target task according to the multiple node groups into which the cluster is divided. By minimizing the allocation of nodes across groups for the same task and allocating physical nodes within the same node group for the same task as much as possible, the network latency between physical nodes executing the same task can be reduced, and the efficiency and stability of task execution can be improved.

[0165] Figure 8 is a flowchart of a GPU server scheduling method provided in an exemplary embodiment of this disclosure. As shown in Figure 8, the specific steps of the method are as follows:

[0166] Step S801: Determine the communication cost between any two GPU servers based on the physical network topology of the GPU servers in the cluster, as well as the network latency and bandwidth between the virtual network interfaces of different GPU servers.

[0167] Step S802: Based on the communication cost between any two GPU servers, divide the multiple GPU servers into multiple server groups. The communication cost between different server groups is greater than the communication cost between different GPU servers within the same server group.

[0168] Step S803: Based on the information from multiple server groups, allocate at least one GPU server for a given model training task. The at least one GPU server is used to execute the given model training task.

[0169] In this embodiment, the cluster is a server cluster containing multiple GPU servers. The GPU servers in the cluster are physical nodes of the cluster. The node groups into which the cluster is divided are groups containing at least one GPU server, which are referred to as server groups in this embodiment.

[0170] The specific implementation principle and technical effects of this embodiment are described in the foregoing method embodiment, and will not be repeated here.

[0171] The method in this embodiment determines the communication cost between any two GPU servers based on the physical network topology of the GPU servers in the cluster and the network latency and bandwidth between the virtual network interfaces of different GPU servers. By dynamically sensing the network latency and bandwidth between the virtual network interfaces of different GPU servers and integrating the dynamically sensed network latency and bandwidth based on the physical network topology of the GPU servers, the method can better quantify and characterize the network performance between GPU servers. Furthermore, based on the communication cost between any two GPU servers, multiple GPU servers are divided into multiple server groups, making the communication cost between different server groups much greater than the communication cost between different GPU servers within the same server group. The GPU servers in the cluster are clustered and aggregated based on communication cost to obtain more reasonable server groups. Furthermore, based on multiple server groups, GPU servers are allocated to the target task. This can minimize the allocation of nodes across groups for the same task and allocate GPU servers within the same server group as much as possible for the same task, thereby reducing the network latency between GPU servers executing the same task and improving the efficiency and stability of task execution.

[0172] Figure 9 is a flowchart of a GPU server scheduling method provided in an exemplary embodiment of this disclosure. As shown in Figure 9, the specific steps of the method are as follows:

[0173] Step S901: In response to a GPU server scheduling request for a given model training task, obtain information about multiple server groups in the cluster. Each server group contains at least one GPU server, and the communication cost between different server groups is greater than the communication cost between different GPU servers within the same server group.

[0174] In this embodiment, when the edge device needs to run a model training task, it sends a scheduling request for the given model training task to the scheduling server. This scheduling request contains task requirement information for the model training task. The task requirement information indicates the data volume, computational load, and computing power requirements of the model training task. Based on the task requirement information, the number of physical nodes to be allocated for the model training task, or the computing power of the allocated physical nodes, can be determined.

[0175] When the scheduling server receives a GPU server scheduling request for a given model training task, it obtains information about the multiple server groups into which the physical cluster is divided.

[0176] In this embodiment, the multiple server groups into which the cluster is divided are determined by the following steps: based on the physical network topology of the GPU servers in the cluster, and the network latency and bandwidth between the virtual network interfaces of different GPU servers, the communication cost between any two GPU servers is determined; based on the communication cost between any two GPU servers, the multiple GPU servers are divided into multiple server groups, and the communication cost between different server groups is greater than the communication cost between different GPU servers within the same server group.

[0177] Step S902: Based on the information of multiple node groups, allocate at least one GPU server for a given model training task. The at least one GPU server is used to execute the given model training task.

[0178] For the specific implementation principle and technical effects of this step, please refer to the relevant content in the previous embodiment. This embodiment will not repeat it here.

[0179] The method in this embodiment determines the communication cost between any two GPU servers based on the physical network topology of the GPU servers in the cluster and the network latency and bandwidth between the virtual network interfaces of different GPU servers. By dynamically sensing the network latency and bandwidth between the virtual network interfaces of different GPU servers and integrating the dynamically sensed network latency and bandwidth based on the physical network topology of the GPU servers, the communication cost between different GPU servers can be determined, which can better quantify the network performance between GPU servers. Furthermore, based on the communication cost between any two GPU servers, multiple GPU servers are divided into multiple server groups, so that the communication cost between different server groups is much greater than the communication cost between different GPU servers within the same server group. The GPU servers in the cluster are clustered and aggregated based on communication cost to obtain more reasonable server groups. Furthermore, in response to a GPU server scheduling request for a given model training task, GPU servers are allocated to the target task according to the multiple server groups divided in the cluster. By minimizing the allocation of nodes across groups for the same task and allocating GPU servers within the same server group for the same task as much as possible, the network latency between GPU servers executing the same task can be reduced, and the efficiency and stability of task execution can be improved.

[0180] The disclosed scheme can dynamically collect data from each physical node in a physical cluster, calculate the communication cost between physical nodes, and simulate the network distance between physical nodes. Based on the communication cost (i.e., network distance) between physical nodes, the physical cluster is clustered and aggregated to obtain multiple node groups; through the grouping information of physical nodes, resource scheduling between physical nodes can be optimized.

[0181] Figure 10 is a schematic diagram of a scheduling server provided in an embodiment of this disclosure. As shown in Figure 10, the scheduling server includes a memory 1001 and a processor 1002. The memory 1001 is used to store computer execution instructions and can be configured to store various other data to support operations on the scheduling server. The processor 1002 is communicatively connected to the memory 1001 and is used to execute the computer execution instructions stored in the memory 1001 to implement the technical solutions provided in any of the above method embodiments. Their specific functions and the technical effects they can achieve are similar and will not be repeated here.

[0182] Optionally, as shown in Figure 10, the scheduling server may also include other components such as a firewall 1003, a load balancer 1004, a communication component 1005, and a power supply component 1006. Figure 10 only schematically illustrates some components and does not imply that the scheduling server only includes the components shown in Figure 10. Figure 10 only uses a cloud scheduling server deployed in the cloud as an example for illustrative purposes; the scheduling server can also be deployed locally, and this embodiment does not impose specific limitations.

[0183] This disclosure also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the method of any of the foregoing embodiments. The specific functions and technical effects to be achieved are not described here.

[0184] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the method of any of the foregoing embodiments. The computer program is stored in a readable storage medium, and at least one processor of the scheduling server can read the computer program from the readable storage medium. The execution of the computer program by the at least one processor causes the scheduling server to perform the technical solutions provided in any of the above method embodiments. Specific functions and achievable technical effects are not elaborated here.

[0185] This disclosure provides a chip, including a processing module and a communication interface. The processing module is capable of executing the technical solution of the scheduling server in the aforementioned method embodiments. Optionally, the chip further includes a storage module (e.g., a memory), which stores instructions. The processing module executes the instructions stored in the storage module, and the execution of the instructions stored in the storage module causes the processing module to execute the technical solution provided in any of the aforementioned method embodiments.

[0186] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this disclosure.

[0187] It should be understood that the aforementioned processor can be a CPU, GPU, or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules from at least one processor.

[0188] The memory may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0189] The aforementioned storage device can be object storage service (OSS).

[0190] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0191] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as mobile hotspots (WiFi), second-generation (2G), third-generation (3G), fourth-generation (4G) / Long Term Evolution (LTE), fifth-generation (5G), or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), infrared, Ultra Wide Band (UWB), Bluetooth, and other technologies.

[0192] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0193] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0194] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside within an application-specific integrated circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components within an electronic device or host device.

[0195] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0196] The order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order; however, it should be clearly understood that these operations may not be executed in the order they appear herein or may be executed in parallel. The sequence numbers are merely used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types. "Multiple" means two or more, unless otherwise explicitly specified.

[0197] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solutions of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this disclosure.

[0198] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The above are merely preferred embodiments of this disclosure and do not limit the patent scope of this disclosure. Any equivalent structural or procedural transformations made using the content of this specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this disclosure.

Claims

1. A cluster node scheduling method, wherein, The cluster comprises a plurality of physical nodes, each of the physical nodes having at least one virtual network interface, and the method comprises: determining a communication cost between any two physical nodes according to a physical network topology of the physical nodes in the cluster and network delays and bandwidths between the virtual network interfaces of different physical nodes; dividing the plurality of physical nodes into a plurality of node groups according to the communication cost between any two physical nodes, wherein the communication cost between different node groups is greater than the communication cost between different physical nodes in the same node group; allocating physical nodes for a target task according to the plurality of node groups.

2. The method of claim 1, wherein, The determining of the communication cost between any two physical nodes according to the physical network topology of the physical nodes in the cluster and the network delays and bandwidths between the virtual network interfaces of different physical nodes comprises: obtaining the physical network topology of the physical nodes in the cluster and the network delays and bandwidths between the virtual network interfaces of different physical nodes; determining a communication cost between virtual network interfaces of different physical nodes according to the physical network topology of the physical nodes in the cluster and the network delays and bandwidths between the virtual network interfaces of different physical nodes; determining the communication cost between any two physical nodes according to the communication cost between the virtual network interfaces of different physical nodes.

3. The method of claim 2, wherein, The determining of the communication cost between virtual network interfaces of different physical nodes according to the physical network topology of the physical nodes in the cluster and the network delays and bandwidths between the virtual network interfaces of different physical nodes comprises: for any two virtual network interfaces belonging to different physical nodes, determining a switch hop number between the two virtual network interfaces according to the physical network topology of the physical nodes in the cluster, and determining a hierarchical information representation between the two virtual network interfaces according to the switch hop number between the two virtual network interfaces; determining a delay information representation between the two virtual network interfaces according to the network delay between the two virtual network interfaces; determining a bandwidth information representation between the two virtual network interfaces according to the bandwidth between the two virtual network interfaces; integrating the hierarchical information representation, the delay information representation and the bandwidth information representation between the two virtual network interfaces to determine a communication cost between the two virtual network interfaces.

4. The method of claim 3, wherein, The determining of the hierarchical information representation between the two virtual network interfaces according to the switch hop number between the two virtual network interfaces comprises: taking the switch hop number between the two virtual network interfaces as an index, calculating an exponential function value of a specified base to obtain the hierarchical information representation between the two virtual network interfaces.

5. The method of claim 3, wherein, The determining of the delay information representation between the two virtual network interfaces according to the network delay between the two virtual network interfaces comprises: normalizing the network delay between the two virtual network interfaces to a first numerical range to obtain the delay information representation between the two virtual network interfaces.

6. The method of claim 3, wherein, The determining of the bandwidth information representation between the two virtual network interfaces according to the bandwidth between the two virtual network interfaces comprises: normalizing bandwidth between the two virtual network interfaces to a second numerical range to obtain a bandwidth information representation between the two virtual network interfaces.

7. The method of claim 3, wherein, Further comprising: in the case that the delay information representation between the two virtual network interfaces is greater than a preset maximum network delay, or the bandwidth information representation between the two virtual network interfaces is greater than a preset maximum bandwidth, setting the communication cost between the two virtual network interfaces as a preset maximum communication cost.

8. The method of claim 2, wherein, The determining of the communication cost between any two physical nodes according to the communication cost between the virtual network interfaces of the different physical nodes comprises: For any two physical nodes, taking the average of the communication cost between any two virtual network interfaces of the two physical nodes as the communication cost between the two physical nodes.

9. The method of claim 1, wherein, The dividing of the plurality of physical nodes into a plurality of node groups according to the communication cost between any two physical nodes comprises: traversing the plurality of physical nodes, and performing the following grouping processing on a current physical node traversed: respectively calculating the communication cost between the current physical node and the existing node groups; if there is at least one node group whose communication cost with the current physical node is less than a communication cost threshold, adding the current physical node to the node group with the minimum communication cost with the current physical node; if there is no node group whose communication cost with the current physical node is less than the communication cost threshold, newly creating a node group containing the current physical node.

10. The method of claim 9, wherein, Further comprising: determining the communication cost threshold according to the communication cost between different physical nodes connected to the same switch node.

11. The method of any one of claims 1-10, wherein, Further comprising: calculating the average communication cost between different node groups according to the communication cost between physical nodes in different node groups; storing the information of the plurality of node groups and the average communication cost between different node groups.

12. The method of claim 11, wherein, The allocating of physical nodes for a target task according to the plurality of node groups comprises: in response to a scheduling request for a target task, allocating at least one physical node for the target task according to the information of the plurality of node groups and the average communication cost between different node groups, the at least one physical node completing the target task.

13. A cluster node scheduling method, wherein, Comprise: in response to a scheduling request for a target task, obtaining information of a plurality of node groups into which a cluster is divided, the node groups containing at least one physical node, the communication cost between different node groups being greater than the communication cost between different physical nodes in the same node group; allocating physical nodes for a target task according to the plurality of node groups; wherein the plurality of node groups in the cluster are determined by the following steps: determining the communication cost between any two physical nodes according to the physical network topology of the physical nodes in the cluster and the network delay and bandwidth between the virtual network interfaces of different physical nodes; and dividing the plurality of physical nodes into a plurality of node groups according to the communication cost between any two physical nodes.

14. A method of Graphical Processing Unit (GPU) server scheduling, wherein, A scheduling server applied to a cluster, the cluster comprising a plurality of GPU servers, the GPU servers having at least one virtual network interface, comprising: determine a communication cost between any two GPU servers according to a physical network topology of the GPU servers in the cluster and network delays and bandwidths between virtual network interfaces of different GPU servers; divide the plurality of GPU servers into a plurality of server groups according to the communication cost between any two GPU servers, wherein the communication cost between different server groups is greater than the communication cost between different GPU servers in a same server group; allocate a GPU server for a target task according to the plurality of server groups.

15. A GPU server scheduling method, wherein, The method comprises: in response to a GPU server scheduling request for a given model training task, obtaining information of a plurality of server groups into which a cluster is divided, the server groups containing at least one GPU server, and the communication cost between different server groups being greater than the communication cost between different GPU servers in a same server group; allocate at least one GPU server for the given model training task according to the information of the plurality of server groups, the at least one GPU server being used to execute the given model training task. The plurality of node groups in the cluster are determined by the following steps: determine a communication cost between any two GPU servers according to a physical network topology of the GPU servers in the cluster and network delays and bandwidths between virtual network interfaces of different GPU servers; and divide the plurality of GPU servers into a plurality of server groups according to the communication cost between any two GPU servers.

16. A dispatch server, wherein, The method comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to cause the server to perform the method of any one of claims 1-15.

17. A computer readable storage medium, wherein, The computer readable storage medium stores computer executable instructions, and when the processor executes the computer executable instructions, the method of any one of claims 1-15 is implemented.

18. A computer program product comprising a computer program, wherein, The computer program is executed by the processor to implement the method of any one of claims 1-15.

Images