Communication method, switching node, cluster and related devices

Abstract

The application discloses a communication method, a switching node, a cluster and related devices, and relates to the technical field of artificial intelligence. The switching node determines traffic state information corresponding to a plurality of computing nodes in the cluster for executing an AI task, and the traffic state information is used to indicate the load condition of a communication link between each computing node and the switching node. The switching node reads data from a first computing node with a relatively low load of an uplink communication link between the first computing node and the switching node according to the traffic state information, and forwards the data to a third computing node with a relatively low load of a downlink communication link between the third computing node and the switching node. In this way, in the case that the amount of data required for communication between the plurality of computing nodes is large, the load of part of the communication link can be effectively avoided from being too high to cause traffic congestion, so that the efficiency of data interaction between the plurality of computing nodes can be improved, thereby improving the overall efficiency of the plurality of computing nodes for executing the AI task.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a communication method, switching node, cluster, and related equipment. Background Technology

[0002] With the development of artificial intelligence (AI) technology, it is often necessary to use multiple computing nodes to execute the same collective communication operator in order to collaboratively perform the same AI task. For example, in model training scenarios, multiple computing nodes can be used to perform distributed training of AI models. During this process, multiple computing nodes can interact with their respective gradient data obtained from training by executing the same collective communication operator, and update the parameter values ​​in the AI ​​model based on the fused gradient.

[0003] Currently, when performing AI tasks using multiple computing nodes, computing node A may need to obtain data computed by the other computing nodes. In this case, the multiple computing nodes typically execute the same aggregation communication operator, such as the Reduce operator or the AllGather operator, and the other computing nodes will send data to computing node A separately.

[0004] However, the communication performance of a single computing node is limited. When the amount of data sent to computing node A by the other computing nodes is large, computing node A has difficulty receiving the data sent by the other computing nodes at the same time, which results in many-to-one traffic congestion. This increases the data interaction latency between multiple computing nodes, thereby reducing the execution efficiency of AI tasks. Summary of the Invention

[0005] This application provides a communication method to reduce data interaction latency between multiple computing nodes and improve the execution efficiency of AI tasks. Furthermore, this application also provides corresponding switching nodes, clusters, computer-readable storage media, and computer program products.

[0006] Firstly, this application provides a communication method applied to a cluster comprising multiple computing nodes, each implemented using hardware such as a CPU, accelerator, or server. These computing nodes execute AI (artificial intelligence) tasks, such as distributed training of an AI model or tasks supporting AI model operation to provide inference services. Traffic generated by these computing nodes executing AI tasks is forwarded through a switching node. During the forwarding of traffic generated by the multiple computing nodes, the switching node determines traffic status information corresponding to each computing node. This traffic status information indicates the load of the communication link between each computing node and the switching node. Then, based on the traffic status information, the switching node reads data from a first computing node and forwards the data to a third computing node. The load of the uplink communication link between the first computing node and the switching node is lower than the load of the uplink communication link between the second computing node and the switching node, and the load of the downlink communication link between the third computing node and the switching node is lower than the load of the downlink communication link between the fourth computing node and the switching node. That is, the switching node selects to read data from the computing node with a lower load on the uplink communication link and selects to forward data to the computing node with a lower load on the downlink communication link.

[0007] In this way, by selecting computing nodes with lower loads on communication links for reading and writing data, the switching node can effectively avoid traffic congestion caused by excessive load on some communication links when the amount of data to be communicated between multiple computing nodes is large. This improves the efficiency of data exchange between multiple computing nodes, reducing data exchange latency and thus improving the overall efficiency of multiple computing nodes in executing AI tasks. Conversely, when the amount of data to be communicated between multiple computing nodes is small, each computing node only needs to perform a few data transmission processes with the switching node to achieve data exchange between multiple computing nodes. This eliminates the need to divide the data into multiple parts and perform iterative data transmission, thereby improving the efficiency of data exchange between multiple computing nodes and further enhancing the overall efficiency of multiple computing nodes in executing AI tasks.

[0008] In one possible implementation, the traffic status information includes the amount of data buffered in the send queue and receive queue corresponding to each of the multiple compute nodes in the switching node; alternatively, the traffic status information includes the values ​​of the send counter and receive counter corresponding to the ports used by the switching node to connect to the multiple compute nodes, where the send counter represents the remaining number of data packets that the switching node can continue to send through the port, and the receive counter represents the remaining number of data packets that the switching node can continue to receive through the port. In this way, the switching node can perceive the load of the communication link between each compute node and the switching node based on the amount of data buffered in the queue or the remaining number of send / receive data packets corresponding to the port. This allows the switching node to avoid selecting compute nodes with high communication link loads for data reading or writing, thereby effectively preventing traffic congestion caused by excessive communication link load in the cluster, and thus improving the overall efficiency of multiple compute nodes executing AI tasks.

[0009] In one possible implementation, when a switching node reads data from a first computing node among multiple computing nodes based on traffic status information, it can specifically send a read request to the first computing node based on the traffic status information and receive a response from the first computing node. The response includes data of the MPS (Maximum Load Size), where the amount of data that the first computing node needs to send during the execution of the AI ​​task is greater than the MPS. In this way, the switching node can obtain the data that the first computing node needs to send to other computing nodes by actively sending read requests to the first computing node. This allows the switching node on the network side to control the data interaction between multiple computing nodes, avoiding traffic congestion caused by excessive load on the communication links in the cluster.

[0010] In one possible implementation, before the switching node reads data from the first computing node among multiple computing nodes, the switching node can also receive task requests sent by the multiple computing nodes respectively. These task requests instruct the switching node to forward the traffic generated by the multiple computing nodes performing AI tasks. Furthermore, the switching node can determine whether its available resources are sufficient to forward the traffic generated by the multiple computing nodes performing AI tasks. Accordingly, when the switching node reads data from the first computing node among multiple computing nodes based on traffic status information, specifically, it can do so only if the switching node's available resources are sufficient to forward the traffic generated by the multiple computing nodes performing AI tasks, and the switching node has received task requests from all the computing nodes among the multiple computing nodes. In this way, when the switching node receives task requests from all computing nodes and has sufficient available resources, it actively controls the data transmission between multiple computing nodes, thereby improving the stability and reliability of the switching node's control over data transmission on the network side.

[0011] In one possible implementation, after the switching node has forwarded the traffic generated by multiple computing nodes executing AI tasks, it can also notify each of the multiple computing nodes that the task request has been completed. For example, the switching node can notify each computing node of the end of the communication process by updating the value in the status notification address of each computing node, or it can send a notification message to each computing node separately to notify them of the end of the communication process. In this way, each computing node can continue to execute the subsequent processes of the AI ​​task after determining that the communication process has ended, thereby improving the reliability of multiple computing nodes executing AI tasks.

[0012] In one possible implementation, the switching node includes a first switching node and a second switching node. The first switching node forwards a portion of the traffic generated by multiple computing nodes performing AI tasks, and the second switching node forwards another portion of the traffic generated by multiple computing nodes performing AI tasks. Thus, each switching node is responsible for forwarding a portion of the data between multiple computing nodes, and multiple switching nodes can perform the data forwarding process in parallel, thereby further improving the efficiency of data communication between multiple computing nodes.

[0013] In one possible implementation, the first switching node is directly connected to multiple computing nodes, and the second switching node is also directly connected to multiple computing nodes; alternatively, both the first and second switching nodes are connected to multiple computing nodes through at least one other switching node, and both the first and second switching nodes are directly connected to other switching nodes. In this way, having a Layer 1 or Layer 2 switching node on the network side control the data transmission process between multiple computing nodes can effectively avoid excessive load on some communication links leading to traffic congestion, thereby improving the overall efficiency of multiple computing nodes performing AI tasks.

[0014] Secondly, this application provides a switching node, which includes a determining module for determining traffic status information corresponding to multiple computing nodes. The traffic status information is used to indicate the load of the communication link between each computing node and the switching node. The multiple computing nodes are used to execute artificial intelligence (AI) tasks, and the traffic generated by the multiple computing nodes executing AI tasks is forwarded through the switching node. A communication module is used to read data from a first computing node among the multiple computing nodes according to the traffic status information and forward the data to a third computing node among the multiple computing nodes. The load of the uplink communication link between the first computing node and the switching node is lower than the load of the uplink communication link between the second computing node and the switching node, and the load of the downlink communication link between the third computing node and the switching node is lower than the load of the downlink communication link between the fourth computing node and the switching node.

[0015] In one possible implementation, the traffic status information includes the amount of data buffered in the send queue and receive queue corresponding to the multiple computing nodes in the switching node; or, the traffic status information includes the values ​​of the send counter and receive counter corresponding to the ports used by the switching node to connect to the multiple computing nodes, where the send counter is the remaining number of data packets that the switching node can continue to send through the port, and the receive counter is the remaining number of data packets that the switching node can continue to receive through the port.

[0016] In one possible implementation, the communication module is configured to: send a read request to the first computing node based on traffic status information; and receive a response from the first computing node, the response including data of MPS (maximum payload size), wherein the amount of data that the first computing node needs to send during the execution of the AI ​​task is greater than MPS.

[0017] In one possible implementation, the communication module is further configured to: receive task requests sent by the multiple computing nodes respectively before the switching node reads data from the first computing node among the multiple computing nodes, the task requests being used to instruct the switching node to forward the traffic generated by the multiple computing nodes performing AI tasks; determine whether the available resources are sufficient to forward the traffic generated by the multiple computing nodes performing AI tasks; and the communication module is configured to read data from the first computing node among the multiple computing nodes based on traffic status information, provided that the available resources of the switching node are sufficient to forward the traffic generated by the multiple computing nodes performing AI tasks and the switching node has received task requests sent by all the computing nodes among the multiple computing nodes.

[0018] In one possible implementation, the communication module is further configured to: notify each of the multiple computing nodes that the task request has been completed when the switching node has forwarded the traffic generated by the multiple computing nodes executing AI tasks.

[0019] In one possible implementation, the switching node includes a first switching node and a second switching node. The first switching node is used to forward part of the traffic generated by multiple computing nodes performing AI tasks, and the second switching node is used to forward another part of the traffic generated by multiple computing nodes performing AI tasks.

[0020] In one possible implementation, the first switching node is directly connected to multiple computing nodes, and the second switching node is directly connected to multiple computing nodes; or, both the first and second switching nodes are connected to multiple computing nodes through at least one other switching node, and both the first and second switching nodes are directly connected to other switching nodes.

[0021] The exchange node provided in the second aspect corresponds to the communication method provided in the first aspect. Therefore, the technical effects of any implementation in the second aspect can be found in the relevant descriptions of the technical effects of the corresponding implementation in the first aspect, and will not be repeated here.

[0022] Thirdly, this application provides a switching node, which includes a processor, a memory, and a communication interface. The communication interface is used to receive and send data; the processor is used to execute instructions stored in the memory, causing the switching node to perform the communication method as described in the first aspect or any implementation thereof. It should be noted that the memory can be integrated into the processor or can be independent of the processor. The switching node may also include a bus. The processor is connected to the memory via the bus. The memory may include readable storage and random access memory.

[0023] Fourthly, this application provides a cluster including a switching node and multiple computing nodes, wherein the multiple computing nodes are used to perform AI (artificial intelligence) tasks, the switching node is used to forward the traffic generated by the multiple computing nodes performing AI tasks, and the switching node is also used to perform the operation steps of the communication method of the first aspect or any implementation thereof.

[0024] Fifthly, this application provides a computer-readable storage medium storing instructions that, when executed on a switching node, cause the switching node to perform the operation steps of the communication method described in the first aspect or any implementation thereof.

[0025] In a sixth aspect, this application provides a computer program product containing instructions that, when run on a switching node, causes the switching node to perform the operational steps of the communication method described in the first aspect or any implementation thereof.

[0026] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0027] Figure 1 A schematic diagram of the structure of an exemplary cluster provided in this application;

[0028] Figure 2 A schematic diagram of the structure of the computing node and the switching node provided in this application;

[0029] Figure 3 This application provides another exemplary cluster structure diagram;

[0030] Figure 4 A schematic diagram of the structure of yet another exemplary cluster provided in this application;

[0031] Figure 5 A flowchart illustrating a communication method provided in this application;

[0032] Figure 6 This is a flowchart illustrating the initialization and compilation / loading processes performed in cluster 10.

[0033] Figure 7 A schematic diagram of the structure of a switching node is provided in this application;

[0034] Figure 8 This is a schematic diagram of the hardware structure of a switching node provided in this application. Detailed Implementation

[0035] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a method of distinction used in describing objects with the same attributes in the embodiments of this application.

[0036] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0037] See Figure 1 This is a schematic diagram of the structure of an exemplary cluster 10. Figure 1 As shown, cluster 10 includes multiple compute nodes and multiple switching nodes. For ease of understanding and explanation, Figure 1 The following description uses cluster 10, which includes compute nodes 101 to 104, switching node 201, and switching node 202, as an example. Switching node 201 and switching node 202 are connected to compute nodes 101 to 104, respectively, so that they can forward traffic between compute nodes 101 to 104.

[0038] As an implementation example, the switching nodes in cluster 10 can have Figure 2 The architecture shown includes a traffic manager 301, a communication control unit 302, and a switching architecture 303. The traffic manager 301 manages traffic between multiple compute nodes, such as determining the order of traffic forwarding. The communication control unit 302 controls the communication process between the switching nodes and other devices (other switching nodes or compute nodes). The switching architecture 303 controls the opening and closing of the interfaces on the switching nodes used for data forwarding, enabling the switching nodes to forward data to other devices. In practical applications, the switching nodes in cluster 10 can be any device with data forwarding capabilities, such as switches, switching chips, routers, etc., and the architecture used is not limited to these. Figure 2 Example shown.

[0039] The compute nodes in cluster 10 can be devices including processors, such as compute servers. For example, a compute node may have... Figure 2 The architecture shown includes software stack 400 and processor 500.

[0040] Software stack 400 is a collection of independent components that ensure the normal operation of computing nodes and can execute corresponding business logic, such as training AI models. Figure 2As shown, the software stack 400 includes an application layer 401, a framework layer 402, a communication layer 403, and a driver layer 404. The application layer 401 provides one or more services, such as allowing users to configure services on computing nodes. The framework layer 402, for example, can be a deep learning framework like MindSpore or TensorFlow, and can be used to generate communication operators and computation operators required by the computing nodes based on the services in the application layer 401 and the topology information of the cluster 10. The communication layer 403 receives communication operators from the framework layer 402 and transforms them into communication tasks required by the processor 500. This communication layer 403 may include a collective communication library, such as the Huawei Collective Communication Library (HCCL). The driver layer 404 drives the processor 500 to perform corresponding computation or communication operations based on the computation or communication operators issued by the framework layer 402.

[0041] Processor 500 is used to perform corresponding computational operations for the services in application layer 401; and processor 500 may include queue 501, computing unit 502, scheduler 503, and aggregate communication engine 504. Queue 501 is used to cache data, such as caching data sent to other computing nodes or data sent by other computing nodes. Computing unit 502 is used to perform computational operations, such as performing reduce operations on data from multiple computing nodes. Aggregate communication engine 504, under the scheduling of scheduler 601, can communicate with exchange nodes directly connected to the computing nodes, such as sending data from the queue to the exchange node. For example, the processor 500 may be any type of processor or any combination thereof, such as a central processing unit (CPU), an accelerator, an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a system-on-chip (SoC), a software-defined infrastructure (SDI) chip, an AI processing chip, or a data processing unit (DPU). The accelerator may be, for example, a graphics processing unit (GPU), a neural network processing unit (NPU), or a tensor processing unit (TPU).

[0042] exist Figure 1In the cluster 10 shown, computing nodes 101 to 104 can collaboratively execute AI tasks. These AI tasks can specifically include tasks involving AI models, such as training an AI model, performing inference using an AI model, or other tasks requiring the execution of set communication operators. For example, the AI ​​model can be a large language model (LLM), a bidirectional encoder representations from transformers (BERT) model, or a generative pre-trained Transformer 3 (GPT-3) model, or other types of models such as GPT-4, etc., without limitation.

[0043] During the execution of AI tasks, computing nodes 101 to 104 can communicate with each other, such as exchanging their calculated gradient data (used to update parameter values ​​in the AI ​​model). Furthermore, exchange nodes 201 and 202 can forward communication data between computing nodes 101 and 104. In practical applications, multiple computing nodes may execute the same set communication operators, such as the Allreduce and Allgather operators, resulting in multiple computing nodes sending data to the same node. For example, computing nodes 102 to 104 may simultaneously send their calculated gradient data to computing node 101. In this case, if the amount of data sent by multiple computing nodes is large, it can easily lead to excessive load on some communication links, causing traffic congestion (or traffic conflicts). This reduces the efficiency of data communication between multiple computing nodes, thus reducing the performance of multiple computing nodes executing set communication operators, and consequently reducing the overall efficiency of multiple computing nodes executing AI tasks. However, if the data to be sent by each computing node is divided into multiple parts based on the number of computing nodes executing the same set communication operator, and these multiple parts are sent to other computing nodes through multiple rounds of iterative communication, while this can avoid traffic congestion, when the amount of data to be sent by multiple computing nodes is small, the data communication latency between computing nodes is longer compared to the method of computing nodes sending all data to other computing nodes at once. This leads to lower data communication efficiency between multiple computing nodes, and consequently, lower overall efficiency of multiple computing nodes executing AI tasks.

[0044] Based on this, in the cluster 10 provided in this application, the data transmission between multiple computing nodes is controlled by the switching node on the network side, which can avoid excessive load on some communication links and traffic congestion, thereby improving the execution efficiency of AI tasks.

[0045] In a specific implementation, assuming that computing nodes 101 to 104 collaboratively execute the same AI task, and taking the control of data transmission between multiple computing nodes by switching node 201 as an example, switching node 201 can determine the traffic status information corresponding to computing nodes 101 to 104. This traffic status information is used to indicate the load status of the communication link between each computing node and switching node 201. For example, the traffic status information may be the load information of the sending queue and receiving queue corresponding to each computing node in switching node 201. Furthermore, the larger the amount of data buffered in the sending queue (or receiving queue) corresponding to a computing node, the higher the load on the downlink communication link (or uplink communication link) between that computing node and switching node 201. Alternatively, the traffic status information may be, for example, the value of the sending counter or receiving counter corresponding to the port of switching node 201. The sending counter indicates the remaining number of data packets that switching node 201 can continue to send using that port, and the receiving counter indicates the remaining number of data packets that switching node 201 can continue to receive using that port. Furthermore, the larger the value of the transmit counter, the smaller the load on the downlink communication link between the switching node 201 and the computing node; the larger the value of the receive counter, the smaller the load on the downlink communication link between the switching node 201 and the computing node.

[0046] Then, based on the traffic status information, the switching node 201 selects to read data from the computing node with a lower load on the uplink communication link (this data is the data that the computing node needs to send to other computing nodes), and selects to write data to the computing node with a lower load on the downlink communication link (this data is the data that the switching node 201 receives and needs to forward).

[0047] Taking writing data as an example, assuming that the downlink communication link between the switching node 201 and the computing node 101 has a low load based on the traffic status information, and the downlink communication link between the switching node 201 and the other computing nodes has a high load, then the switching node 201 can postpone sending data to computing nodes 102 to 104, and can execute the process of writing data to computing node 101. This can avoid traffic congestion on the downlink communication link between the switching node 201 and the other computing nodes.

[0048] Because switching node 201 selects computing nodes with lower communication link loads to read and write data during the forwarding of traffic generated by computing nodes 101 and 104 executing AI tasks, it can effectively avoid traffic congestion caused by excessive load on some communication links, even when the amount of data to be communicated between computing nodes 101 and 104 is large. This improves the efficiency of data exchange between computing nodes 101 and 104, reducing data exchange latency and thus improving the overall efficiency of computing nodes 101 and 104 in executing AI tasks. In practical applications, by controlling traffic forwarding in the network through switching node 201, 100% traffic congestion-free network operations can be achieved.

[0049] Meanwhile, when the amount of data required for communication between computing nodes 101 and 104 is small, each computing node and the switching node 201 only need to perform a few data transmission processes to achieve data interaction between computing nodes 101 and 104. There is no need to divide the data into multiple parts and perform iterative data transmission processes, which can improve the efficiency of data interaction between computing nodes 101 and 104, thereby improving the overall efficiency of computing nodes 101 and 104 in performing AI tasks.

[0050] Furthermore, in Figure 1 In the cluster 10 shown, computing nodes 101 to 104, which execute the same set of communication operators, are all connected to switching nodes 201 and 202. In this case, switching nodes 201 and 202 can simultaneously participate in forwarding traffic between computing nodes 101 and 104. Specifically, for each computing node, the data it sends to other computing nodes can be divided into two parts, and each switching node is responsible for forwarding one part of the data to that other computing node. For example, assuming that the data size sent by computing node 101 to computing node 102 is 4 megabytes (MB), then this data can be divided into two parts, each 2MB in size. Switching node 201 is responsible for forwarding 2MB of data to computing node 102, and switching node 202 is responsible for forwarding the other 2MB of data to computing node 102. Thus, by using multiple switching nodes to forward the traffic generated by multiple computing nodes executing AI tasks in parallel, the data communication efficiency between multiple computing nodes can be further improved, thereby helping to further improve the overall efficiency of multiple computing nodes executing AI tasks.

[0051] In practical applications, when each of the multiple computing nodes participating in the execution of the same set communication operator is connected to N (N is a positive integer) exchange nodes, the data that each computing node needs to send can be divided into N parts. Furthermore, each exchange node is responsible for forwarding one part of the N parts of data to other computing nodes. Different exchange nodes are responsible for forwarding different parts of the data, thereby improving the data communication efficiency between multiple computing nodes by N times.

[0052] It is worth noting that the above Figure 1 Cluster 10 shown is merely an illustrative example and is not intended to be limiting. Other possible implementations are illustrated below.

[0053] The above Figure 1 The cluster 10 shown is illustrated using an example that includes one layer of switching nodes. Figure 3 The cluster 20 shown can include a larger number of switching nodes and compute nodes, and the switching nodes 201 to 206 in cluster 20 can be deployed in multiple tiers. Figure 3 Taking a two-tier deployment as an example, switch nodes 201 to 204 can act as leaf nodes, deployed in the first tier (L1), and directly connected to each compute node. Switch nodes 205 and 206 can act as spine nodes, deployed in the second tier (L2). Spine nodes can forward data between different leaf nodes, and switch nodes 205 and 206 can connect to each compute node through other switch nodes. Figure 3 In the cluster 20 shown, switching node 205 (and similarly, switching node 206) can connect to compute nodes 101 to 108 participating in the same AI task through other switching nodes. In this case, switching node 205 can forward the traffic generated by compute nodes 101 to 108 executing the AI ​​task in the manner described above, thereby improving the overall efficiency of the AI ​​task execution by compute nodes 101 to 108. Furthermore, switching nodes 205 and 206 can perform the traffic forwarding process in parallel to further improve the execution efficiency of the AI ​​task.

[0054] The above Figure 1 In the cluster 10 shown, compute nodes 101 to 104 are all connected to multiple switching nodes. Figure 4 In the cluster 30 shown, computing nodes 101 to 104 participating in the execution of the same AI task can also be connected to only one switching node 201, and the switching node 201 forwards the traffic generated by computing nodes 101 to 104 in the manner described above.

[0055] Or, in other possible clusters, Figure 1 Based on the cluster 10 shown, other nodes can be included. For example, the cluster can also include a management node and a host. The management node, such as a subnet manager, can generate topology information indicating the cluster's topology and manage the communication network formed by multiple switching nodes, such as configuring routing information for each switching node. The host can be used to control multiple computing nodes in the cluster to collaboratively execute the same AI task. Additionally, Figure 1 In cluster 10 shown, the switching nodes can adopt Figure 2 The architecture shown, in other possible clusters, the architecture of the switching nodes can be... Figure 2 Based on the architecture shown, a routing table 404 can also be included, which records the forwarding interface and the next-hop route so that the next-hop node for data forwarding can be determined based on the information recorded in the routing table.

[0056] For ease of understanding, embodiments of the communication method provided in this application will be described below with reference to the accompanying drawings.

[0057] See Figure 5 , Figure 5 This is a flowchart illustrating a communication method provided in an embodiment of this application. This method can be applied to... Figure 1 , Figure 3 or Figure 4 The cluster shown can be applied to other suitable clusters. For ease of understanding, this embodiment uses an example of an application... Figure 1 The following example illustrates cluster 10, where switching node 201 is responsible for forwarding traffic generated by compute node 101 to compute node 104 for executing AI tasks.

[0058] in, Figure 5 The communication method shown may specifically include the following steps.

[0059] S501: Switching node 201 determines the traffic status information corresponding to computing nodes 101 to 104. This traffic status information is used to indicate the load status of the communication link between each computing node and switching node 201.

[0060] In this embodiment, computing nodes 101 to 104 can collaboratively execute the same AI task. Furthermore, during the execution of the AI ​​task, computing nodes 101 to 104 can execute aggregate communication operators, such as Allreduce, Allgather, Reduce, or Alltoall operators.

[0061] A communication link can exist between each compute node and the switching node 201, and the switching node can receive data from or send data to the compute node through this communication link. For ease of distinction, the communication link used by the compute node to send data to the switching node is referred to as the uplink communication link, and the communication link used by the switching node to send data to the compute node is referred to as the downlink communication link. In practical applications, the uplink and downlink communication links can be implemented using different physical links. In this case, the switching node can connect to the compute node based on multiple ports, where some ports are located on the physical link used for uplink communication and others on the physical link used for downlink communication. Alternatively, the uplink and downlink communication links can be implemented using the same physical link. The switching node can connect to the compute node based on one port. In this case, the switching node can send data to or receive data from the compute node through the physical link of that port.

[0062] In practical applications, the switching node 201 can utilize multiple built-in queues to interact with computing nodes. For example, for computing node 101, the switching node 201 can be configured with a sending queue and a receiving queue corresponding to computing node 101. The sending queue is used to buffer data to be forwarded to computing node 101, and the receiving queue is used to buffer data received by the switching node 201 from computing node 101. Similarly, for other computing nodes, the switching node 201 can be configured with separate sending and receiving queues for each computing node. Furthermore, each computing node can have one or more sending or receiving queues, and the sending and receiving queues corresponding to different computing nodes are different.

[0063] Based on this, in one possible implementation, during the forwarding of traffic for multiple computing nodes, the switching node 201 can traverse the sending and receiving queues corresponding to computing nodes 101 to 104 to determine the amount of data buffered in the sending and receiving queues corresponding to each computing node. Therefore, the switching node 201 can determine the load of the communication link between itself and the computing node based on the amount of data buffered in the sending and receiving queues. That is, the traffic status information determined by the switching node 201 can specifically be the amount of data buffered in the sending and receiving queues. Taking the sending queue as an example, when the amount of data buffered in the sending queue corresponding to computing node 101 is greater than or equal to a threshold, the switching node 201 can determine that the downlink communication link between itself and computing node 101 has a high load. Conversely, when the amount of data buffered in the sending queue is less than the threshold, the switching node 201 can determine that the downlink communication link between itself and computing node 101 has a low load. Alternatively, when the sending queue corresponding to compute node 101 contains cached data (i.e., the data volume is not zero), switching node 201 can determine that the downlink communication link between it and compute node 101 has a high load. Conversely, when the sending queue does not contain cached data (i.e., the data volume is zero), switching node 201 can determine that the downlink communication link between it and compute node 101 has a low load.

[0064] Alternatively, the port used by the switching node 201 to communicate with each computing node can be configured with a send counter and a receive counter. The send counter is used to indicate the remaining number of data packets that the switching node 201 can continue to send to the computing node using the port, and the receive counter is used to indicate the remaining number of data packets that the switching node 201 can continue to receive from the computing node using the port.

[0065] Based on this, in another possible implementation, during the forwarding of traffic for multiple computing nodes, the switching node 201 can traverse the ports used for communication with computing nodes 101 to 104 to determine the transmit and receive counters corresponding to the ports used to connect each computing node. Thus, the switching node 201 can determine the load of the uplink and downlink communication links with the computing node based on the transmit and receive counters. Specifically, the traffic status information determined by the switching node 201 can be the values ​​of the transmit and receive counters corresponding to the ports used by the switching node 201 to connect to each computing node. Taking computing node 101 as an example, when the value of the transmit counter corresponding to the port used by the switching node 201 to connect to computing node 101 is greater than or equal to a threshold, the switching node 201 can determine that the load of its downlink communication link with computing node 101 is low; when the value of the transmit counter corresponding to the port used by the switching node 201 to connect to computing node 101 is less than the threshold, the switching node 201 can determine that the load of its downlink communication link with computing node 101 is high. Similarly, when the value of the receive counter corresponding to the port used by the switching node 201 to connect to the computing node 101 is greater than or equal to the threshold, the switching node 201 can determine that the load on the uplink communication link between it and the computing node 101 is low; when the value of the receive counter corresponding to the port used by the switching node 201 to connect to the computing node 101 is less than the threshold, the switching node 201 can determine that the load on the uplink communication link between it and the computing node 101 is high.

[0066] It should be noted that the traffic status information determined by the switching node 201 can also be other types of information. For example, in other implementations, the switching node 201 can determine the load of the communication link between the switching node 201 and the computing node by checking the usage status of the ports connecting the switching node 201 to each computing node. For instance, when a port is occupied, the switching node 201 can determine that the communication link containing that port has a high load, while when the port is not occupied, the switching node 201 can determine that the communication link containing that port has a low load.

[0067] In practical application scenarios, before forwarding traffic to computing nodes 101 to 104, switching node 201 can first determine whether each computing node is ready for data communication. If some or all computing nodes are ready for data communication, switching node 201 can start the process of forwarding traffic to these computing nodes.

[0068] Specifically, such as Figure 5As shown, during data communication between computing nodes 101 and 104, each node can send a task request to the exchange node 201. This task request instructs the exchange node 201 to forward the traffic generated by the multiple computing nodes performing AI tasks. Each computing node's task request can instruct the exchange node 201 to forward traffic related to that computing node, such as traffic that the exchange node needs to send to other computing nodes, or traffic that other computing nodes need to send to that computing node. After sending a task request, a computing node can enter a waiting state (waiting for the exchange node 201 to read the data that the computing node needs to send or waiting for the exchange node 201 to send data to the computing node).

[0069] After receiving task requests from some or all of the computing nodes, the switching node 201 can determine whether its remaining available resources are sufficient to forward the traffic generated by computing node 101 to computing node 104 for executing AI tasks. For example, the available resources in the switching node 201 could be the number of concurrent AI tasks that the switching node 201 can support; that is, the switching node 201 can simultaneously support forwarding traffic to computing nodes corresponding to multiple different AI tasks. Furthermore, when the number of AI tasks to which the traffic currently being forwarded by the switching node 201 belongs reaches the concurrency limit, the switching node 201 can determine that available resources are insufficient; conversely, when the number of AI tasks to which the traffic currently being forwarded by the switching node 201 belongs does not reach the concurrency limit, the switching node 201 can determine that available resources are sufficient.

[0070] If sufficient resources are available, switching node 201 can begin forwarding traffic from computing node 101 to computing node 104 after receiving task requests from all computing nodes participating in the AI ​​task. Alternatively, switching node 201 can begin forwarding traffic after receiving task requests from only some computing nodes. If the available resources of switching node 201 are insufficient to forward the traffic generated by computing nodes 101 to 104 executing the AI ​​task, switching node 201 can notify each computing node of the resource shortage so that each computing node can proactively send data to other computing nodes.

[0071] For example, switching node 201 can be connected via host ( Figure 1 (Not shown in the image) Pre-provided communication information determines the relevant information of multiple computing nodes participating in the execution of AI tasks.

[0072] Specifically, before switching node 201 forwards traffic, cluster 10 can pre-complete the initialization and compilation / loading processes. During initialization, for example... Figure 6 As shown, the management node in cluster 10 ( Figure 1(Not shown in the image) can scan each compute node and switching node in cluster 10. Then, the management node can configure routing for each compute node and switching node based on the physical links between the compute nodes and switching nodes, and generate corresponding topology information, which is used to indicate the topology between the compute nodes and switching nodes.

[0073] Then, cluster 10 can execute the compilation and loading process. During this process, such as Figure 6 As shown, the host can create AI tasks, such as creating a task to train an AI model based on a user-provided AI model. Then, the host can obtain the topology information of cluster 10 from the management node for this AI task. In practical applications, this topology information can also be provided to the host by the user through a corresponding interactive interface, etc., without limitation. Next, the host can generate corresponding computational operators and communication operators for the AI ​​task based on the obtained topology information. The computational operators can be used to instruct the computational operations to be performed by each computing node participating in the AI ​​task, and the communication operators can be used to instruct the communication operations to be performed by each computing node participating in the AI ​​task. Furthermore, the host can also generate task request information for the switching node 201, such as... Figure 6 As shown. For example, the task request information may include a task identifier (job-id), the number of compute nodes (device-count), a storage region (memory-region), a queue identifier (queue-id), and a status notification address (status-notify-address). The task identifier is used to distinguish different tasks. In practical applications, compute nodes in cluster 10 can execute multiple different AI tasks in parallel, allowing the host to differentiate between them. The number of compute nodes indicates the number of compute nodes connected to exchange node 201 and participating in the execution of the AI ​​task (the AI ​​task indicated by the task identifier). The storage region indicates the storage areas in the compute nodes that exchange node 201 can access, and can be used to verify the access rights of compute nodes to exchange node 201. The queue identifier indicates the queues (including sending and receiving queues) configured in exchange node 201 for each compute node participating in the AI ​​task execution. The status notification address indicates the completion status of the current communication process for each compute node, which can be incomplete or completed. Finally, the host can obtain the available resources in the switching node 201, and if the available resources in the switching node 201 are sufficient to forward the traffic generated by the compute node 101 to the compute node 104 executing AI tasks, it sends the task request information to the switching node 201, such as... Figure 6As shown. In practical applications, when N switching nodes forward traffic between computing nodes 101 and 104 in parallel, the host can divide the data that each computing node needs to send into N parts, and use task request information to instruct each switching node to forward 1 / N of the data included in each computing node.

[0074] It is understood that the initialization and compilation / loading processes performed in cluster 10 described above are merely illustrative examples and are not intended to limit the scope of the application. In practical applications, cluster 10 may also employ other methods to perform the initialization and compilation / loading processes. For instance, the host may not need to acquire available resources from exchange node 201 and may directly send task request information to exchange node 201. In this way, when the available resources in exchange node 201 are insufficient to forward the traffic generated by multiple compute nodes executing AI tasks, exchange node 201 can notify each compute node of insufficient resources, allowing each compute node to proactively send data to other compute nodes.

[0075] Thus, when using computing nodes 101 to 104 to execute AI tasks, the host can distribute the computed computation operators and communication operators to each computing node. Each computing node can then perform corresponding computational operations based on the computation operators configured by the host. When executing communication operators, each computing node sends a task request to the exchange node 201 to request the exchange node 201 to forward data for that computing node.

[0076] S502: Based on the traffic status information, the switching node 201 reads data from the computing node with a lower load on the uplink communication link and forwards the data to the computing node with a lower load on the downlink communication link.

[0077] In practical implementation, switching node 201 can determine the load of the uplink and downlink communication links between switching node 201 and each computing node based on traffic status information, and further identify the uplink and downlink communication links with lower loads. For example, switching node 201 can determine whether there is data cached in the receive queue corresponding to each computing node in switching node 201 based on traffic status information. Assuming that there is no data cached in the receive queue corresponding to computing node 101, but data cached in the receive queues corresponding to computing nodes 102 to 104, then switching node 201 can determine that the load of its uplink communication link with computing node 101 is low. Therefore, switching node 201 can read data from computing node 101 with a low load on the uplink communication link (this data is the data that computing node needs to send to other computing nodes). Similarly, based on traffic status information, switching node 201 can determine whether there is data cached in the sending queue corresponding to each computing node in switching node 201. Assuming that computing node 104 does not cache data, but the sending queues corresponding to computing nodes 101 to 103 do cache data, then switching node 201 can determine that the downlink communication link between it and computing node 104 has a low load. Therefore, switching node 201 can write data (data sent to this computing node by other computing nodes) to computing node 104 with a low downlink communication link load.

[0078] As an example of data reading, assuming the compute node with a low load on the uplink communication link is compute node 101, switching node 201 can send a read request to compute node 101, which may carry a memory region. Furthermore, switching node 201 can record the destination indication information corresponding to the read request (such as the identifiers of compute nodes 102 to 104), so that switching node 201 can subsequently forward the data accessed based on the read request to the correct destination. After receiving the read request, compute node 101 can verify the memory region in the read request, specifically verifying whether the memory region in the read request is consistent with the storage location of the data that compute node 101 needs to send. If they are consistent, compute node 101 can determine that the authentication is successful and return a response to switching node 201, which includes some or all of the data that compute node 101 needs to send to compute nodes 102 to 104. If there is a discrepancy, compute node 101 can determine that the authentication failed and refuse to return data to exchange node 201. In other implementation examples, the read request sent by exchange node 201 to compute node 101 may not include the memory-region, and there is no restriction on this.

[0079] In practical applications, if the amount of data that compute node 101 needs to send to other compute nodes is small, exchange node 201 can obtain all the data that compute node 101 needs to send by sending a single read request. However, when the amount of data that compute node 101 needs to send to other compute nodes is large, since the maximum amount of data that exchange node 201 can obtain from compute node 101 in a single access is limited (i.e., the amount of data that exchange node 201 can access from compute node 101 based on a single read request does not exceed this maximum amount), exchange node 201 can obtain all the data that compute node 101 needs to send by sending multiple read requests to compute node 101. For example, assuming that the total amount of data that compute node 101 needs to send is 1MB, and the amount of data that exchange node 201 can obtain in a single access is 64 kilobytes (KB), then exchange node 201 can obtain the 1MB of data that compute node 101 needs to send by sending 16 read requests to compute node 101 sequentially.

[0080] For example, after receiving the read request, compute node 101 can send a response to exchange node 201. The response includes the data that compute node 101 needs to send, and the amount of data is the maximum payload size (MPS). The amount of data that exchange node 201 can read from the compute node each time (i.e., the MPS size) can be determined in advance by exchange node 201 and each compute node through negotiation.

[0081] For data read from compute node 101, switching node 201 can forward the data to compute nodes 102, 103, or 104. Specifically, switching node 201 can first determine which compute nodes(s) to forward the data to based on the indication information of the destination node corresponding to the read request, assuming that the data in compute node 101 needs to be sent from compute node 102 to compute node 104. Then, switching node 201 can determine the load status of the downlink communication link of the destination compute node based on traffic status information, and forward the data accessed based on the read request to the compute node with a lower load on the downlink communication link.

[0082] In practical applications, the process of switching node 201 reading data from computing nodes and the process of switching node 201 forwarding data to computing nodes can be independent of each other. Therefore, for multiple computing nodes participating in the execution of AI tasks, switching node 201 can perform the processes of reading data and / or forwarding data in parallel, thereby improving the data forwarding performance of switching node 201.

[0083] Assuming that the uplink and downlink communication links between switching node 201 and each computing node are implemented based on different physical links, taking data communication via the Alltoall operator executed by multiple computing nodes as an example, switching node 201 can send read requests to computing nodes 101 to 104 respectively. These read requests request the reading of data of size MPS, allowing switching node 201 to obtain the MPS-sized data returned by each computing node via the uplink communication link. Then, switching node 201 can forward the MPS-sized data obtained from each computing node to the other computing nodes, such as forwarding the MPS-sized data accessed by computing node 101 to computing nodes 102 to 104. Furthermore, during the process of switching node 201 forwarding data to computing nodes 101 to 104 via the downlink communication link, the load on the uplink communication link between switching node 201 and each computing node is 0. At this time, switching node 201 can send read requests to each computing node again to access the next set of MPS-sized data to be sent by each computing node. During the process of receiving MPS-sized data from various computing nodes, switching node 201 can cache the data in its corresponding receive queue. If some computing nodes have zero data cached in their send queues (e.g., computing node 104 has zero data cached in its send queue, indicating low load on its downlink communication link), then switching node 201 can add the data cached in the receive queues of computing nodes 101 through 103 to the send queue of computing node 104. This data can then be sent by switching node 201 to computing node 104 via the downlink communication link. If the amount of data cached in the send queue corresponding to a computing node is currently 0, indicating that the downlink communication links of each computing node are under heavy load, then switching node 201 can wait for switching node 201 to forward all the data in its send queue to the corresponding computing node. Simultaneously, if the amount of data cached in the receive queue corresponding to each computing node is greater than 0, indicating that the uplink communication links of each computing node are under heavy load, then switching node 201 can temporarily postpone sending read requests to computing nodes 101 to 104 (i.e., postpone accessing a new set of data of MPS size from each computing node). In this way, by sensing the downlink communication links and their loads, switching node 201 can avoid excessive traffic congestion caused by overloaded communication links. Repeating the above process, switching node 201 can forward data from each computing node to the other computing nodes by sending multiple rounds of read requests and forwarding data, thereby completing data communication between multiple computing nodes.

[0084] Assuming that the uplink and downlink communication links between switching node 201 and each computing node are implemented based on the same physical link, and taking the example of multiple computing nodes executing the Alltoall operator for data communication, switching node 201 can send read requests to computing nodes 102 to 104 respectively. These read requests request the reading of data of size MPS. Thus, switching node 201 can obtain the MPS-sized data returned by computing nodes 102 to 104 respectively based on the uplink communication link. Then, switching node 201 can forward the MPS-sized data obtained from computing nodes 102 to 104 back to computing node 101. After forwarding the data to computing node 101, switching node 201 can send read requests to computing nodes 101, 103, and 104 respectively. These read requests request the reading of data of size MPS. Thus, switching node 201 can obtain the MPS-sized data returned by computing nodes 101, 103, and 104 respectively based on the uplink communication link. Then, switching node 201 can forward the MPS-sized data obtained from computing nodes 101, 103, and 104 to computing node 102. By repeating the above process, switching node 201 can forward the data of each computing node to the other computing nodes by sending multiple rounds of read requests to each computing node and forwarding the data, thereby completing data communication between multiple computing nodes.

[0085] Thus, by forwarding traffic between computing nodes 101 and 104 in the manner described above, switching node 201 can avoid excessive load on the communication links between switching node 201 and each computing node, which would affect the communication efficiency between multiple computing nodes. Furthermore, since switching node 201 actively reads data from each computing node by sending read requests, compared to the method where computing nodes send read requests to other computing nodes (data communication latency greater than 1 RTT), the transmission path of read requests can be shortened, and it can bypass the software stack within the computing nodes. This reduces the data communication latency between multiple computing nodes to 0.5 RTT (round-trip time, a metric used to evaluate network latency).

[0086] Furthermore, when switching node 201 manages traffic in the network, after completing the traffic forwarding between computing node 101 and computing node 104, switching node 201 can notify each computing node that the communication process has ended. Specifically, as follows... Figure 5 As shown, this embodiment may also include the following steps.

[0087] S503: When the switching node 201 has finished forwarding the traffic generated by the AI ​​task executed by the computing node 101 to the computing node 104, the switching node 201 notifies each computing node that the communication process has ended.

[0088] In practice, before data interaction occurs between multiple computing nodes, each computing node can send a task request to the exchange node 201 to instruct the exchange node 201 to forward traffic. Then, after the exchange node 201 has forwarded all traffic, the exchange node 201 can notify each computing node that the task request has been completed.

[0089] In this embodiment, the following two implementation examples are provided to notify each computing node that the communication process has ended.

[0090] In the first implementation example, the switching node 201 can access the corresponding storage address in each computing node according to the status notification address in the aforementioned task request information, and update the value in that storage address, that is, update the completion status of the communication process. Taking notifying computing node 101 that the process has ended as an example, the status notification address for computing node 101 in the task request information can specifically be the access address of a register (or memory, etc.) in computing node 101. After computing node 101 sends the task request to switching node 201, the value in that register of computing node 101 can be "1", indicating that the task request is not completed. After switching node 201 has forwarded all traffic, switching node 201 can access the register in computing node 101 according to the status notification address and update the value in that register from "1" to "0", indicating that the task request has been completed. At this time, computing node 101 can determine that the communication process with other computing nodes has ended according to the value in the register, and can execute subsequent program logic, such as updating the parameters in the AI ​​model according to the gradient data calculated by each computing node.

[0091] In the second implementation example, the switching node 201 can send a notification message to each computing node. This notification message includes a target field, and the value of the target field can be used to indicate that the task request has been completed, such as the target field being "0". Thus, each computing node can parse the received notification message and, upon determining that the target field is "0", determine that the communication process with other computing nodes has ended, and can then execute subsequent program logic.

[0092] It should be noted that the above Figure 5In the illustrated embodiment, the example of switching node 201 forwarding communication traffic between computing node 101 and computing node 104 is used for illustration. In other embodiments, switching node 201 and switching node 202 may also perform the forwarding of communication traffic between computing node 101 and computing node 104 in parallel. Each switching node may forward a portion of the traffic between computing node 101 and computing node 104 in the manner described above, such as switching node 201 and switching node 202 forwarding 1 / 2 of the traffic between computing node 101 and computing node 104 respectively.

[0093] In addition, the above Figure 5 The illustrated embodiment uses a Layer 1 switching node as an example to illustrate the forwarding process of communication traffic between computing nodes 101 and 104. In other embodiments, a Layer 2 switching node can also forward communication traffic between multiple computing nodes participating in the same AI task. For example, in Figure 3 In the cluster 20 shown, when the same AI task is executed using computing nodes 101 to 108, the communication traffic between computing nodes 101 and 108 can be forwarded by switching nodes 205 and / or 206 in the manner described above, so as to improve the communication efficiency between multiple computing nodes and thus improve the overall efficiency of the cluster 20 in executing AI tasks.

[0094] It is worth noting that other reasonable combinations of steps that can be conceived by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to this application.

[0095] The above combination Figures 1 to 6 The communication method provided in the embodiments of this application will be introduced, and the structure of the switching node provided in the embodiments of this application will be described next with reference to the accompanying drawings.

[0096] See Figure 7 The diagram illustrates the structure of a switching node. Figure 7 As shown, the switching node 700 includes:

[0097] The determination module 701 is used to determine the traffic status information corresponding to multiple computing nodes. The traffic status information is used to indicate the load status of the communication link between each computing node and the switching node 700. The multiple computing nodes are used to execute AI tasks, and the traffic generated by the multiple computing nodes executing AI tasks is forwarded through the switching node 700.

[0098] The communication module 702 is used to read data from a first computing node among multiple computing nodes according to traffic status information, and forward the data to a third computing node among multiple computing nodes. The load of the uplink communication link between the first computing node and the switching node 700 is lower than the load of the uplink communication link between the second computing node and the switching node 700 among multiple computing nodes, and the load of the downlink communication link between the third computing node and the switching node 700 is lower than the load of the downlink communication link between the fourth computing node and the switching node 700 among multiple computing nodes.

[0099] In one possible implementation, the traffic status information includes the amount of data buffered in the sending queue and receiving queue of the switching node 700, which correspond to the multiple computing nodes respectively.

[0100] Alternatively, the traffic status information includes the values ​​of the send counter and receive counter corresponding to the ports used by the switching node 700 to connect to multiple computing nodes. The send counter is the remaining number of data packets that the switching node 700 can continue to send through the port, and the receive counter is the remaining number of data packets that the switching node 700 can continue to receive through the port.

[0101] In one possible implementation, the communication module 702 is used for:

[0102] Based on the traffic status information, a read request is sent to the first computing node;

[0103] Receive the response sent by the first computing node. The response includes data of size MPS. The amount of data that the first computing node needs to send during the execution of the AI ​​task is greater than MPS.

[0104] In one possible implementation, the communication module 702 is further configured to:

[0105] Before the switching node 700 reads data from the first computing node among multiple computing nodes, it receives task requests sent by multiple computing nodes respectively. The task requests are used to instruct the switching node 700 to forward the traffic generated by the multiple computing nodes performing AI tasks.

[0106] Determine if available resources are sufficient to forward the traffic generated by multiple computing nodes executing AI tasks;

[0107] The communication module 702 is used to read data from the first computing node among the multiple computing nodes based on traffic status information, provided that the available resources of the switching node 700 are sufficient to forward the traffic generated by multiple computing nodes executing AI tasks, and the switching node 700 receives task requests sent by all computing nodes among the multiple computing nodes.

[0108] In one possible implementation, the communication module 702 is further configured to:

[0109] When the switching node 700 forwards the traffic generated by multiple computing nodes executing AI tasks, it notifies each computing node in the multiple computing nodes that the task request has been completed.

[0110] In one possible implementation, the switching node 700 includes a first switching node and a second switching node. The first switching node is used to forward part of the traffic generated by multiple computing nodes performing AI tasks, and the second switching node is used to forward another part of the traffic generated by multiple computing nodes performing AI tasks.

[0111] In one possible implementation, the first switching node is directly connected to multiple computing nodes, and the second switching node is directly connected to multiple computing nodes; or, both the first and second switching nodes are connected to multiple computing nodes through at least one other switching node, and both the first and second switching nodes are directly connected to other switching nodes.

[0112] because Figure 7 The switching node 700 shown corresponds to the above. Figure 5 The switching node 201 in the illustrated embodiment, therefore Figure 7 For the specific implementation of the switching node 700 shown and its technical effects, please refer to the above. Figure 5 The relevant details in the illustrated embodiments are described in detail here, and will not be repeated here.

[0113] Figure 8 This is a schematic diagram of the structure of a switching node provided in this application. Figure 8 As shown, the switching node 800 includes a processor 801, a memory 802, a communication interface 803, and a bus 804. The processor 801, memory 802, and communication interface 803 communicate via the bus 804. The bus 804 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface 803 is used for communication with external systems, such as receiving data sent by computing nodes.

[0114] It should be understood that in the embodiments of this application, processor 801 may be a CPU, or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete device assemblies, etc. General-purpose processors may be microprocessors or any conventional processors, etc.

[0115] The memory 802 may include read-only memory and random access memory, and provides instructions and data to the processor 801. The memory 802 may also include non-volatile random access memory.

[0116] The memory 802 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0117] The memory 802 stores executable code, and the processor 801 executes the executable code to perform the aforementioned actions. Figure 5 The method performed by the exchange node 201 in the illustrated embodiment.

[0118] It should be noted that the switching node 800 in this application embodiment can correspond to the switching node 201 in this application embodiment, and can correspond to the execution of the embodiments in this application embodiment. Figure 5The method executed by the switching node 201 in the illustrated method, and the above and other operations and / or functions implemented by the switching node 800, are respectively for the purpose of implementing... Figure 5 The process of the corresponding methods in [the document] will not be elaborated here for the sake of brevity.

[0119] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a switching node can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the switching node to perform the aforementioned communication method.

[0120] This application also provides a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a switching node, all or part of the processes or functions described in this application are generated.

[0121] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0122] The computer program product can be a software installation package. When any of the aforementioned communication methods is required, the computer program product can be downloaded and executed on the exchange node.

[0123] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0124] The terminology used in the above embodiments is for the purpose of describing specific embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the embodiments of this application, “one or more” refers to one, two, or more; the character “ / ” generally indicates that the preceding and following objects are in an “or” relationship. In the embodiments of this application, “simultaneously” means within the same time period, including situations where they are at the same moment.

[0125] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0126] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A communication method, characterized in that, The method is applied to a cluster comprising multiple computing nodes for executing artificial intelligence (AI) tasks. Traffic generated by the computing nodes executing these AI tasks is forwarded through a switching node. The method includes: The switching node determines the traffic status information corresponding to the plurality of computing nodes, and the traffic status information is used to indicate the load status of the communication link between each computing node and the switching node. The switching node reads data from a first computing node among the plurality of computing nodes according to the traffic status information, and forwards the data to a third computing node among the plurality of computing nodes. The load of the uplink communication link between the first computing node and the switching node is lower than the load of the uplink communication link between the second computing node and the switching node among the plurality of computing nodes, and the load of the downlink communication link between the third computing node and the switching node is lower than the load of the downlink communication link between the fourth computing node and the switching node among the plurality of computing nodes.

2. The method according to claim 1, characterized in that, The traffic status information includes the amount of data cached in the sending queue and receiving queue that correspond to the plurality of computing nodes in the switching node; Alternatively, the traffic status information includes the values ​​of the send counter and receive counter corresponding to the ports used by the switching node to connect to the plurality of computing nodes, wherein the send counter is the remaining number of data packets that the switching node can continue to send through the port, and the receive counter is the remaining number of data packets that the switching node can continue to receive through the port.

3. The method according to claim 1 or 2, characterized in that, The switching node reads data from a first computing node among the plurality of computing nodes based on the traffic status information, including: The switching node sends a read request to the first computing node based on the traffic status information; The switching node receives a response from the first computing node, the response including data of the maximum payload size (MPS), and the amount of data that the first computing node needs to send during the execution of the AI ​​task is greater than the MPS.

4. The method according to any one of claims 1 to 3, characterized in that, Before the switching node reads data from the first computing node among the plurality of computing nodes, the method further includes: The switching node receives task requests sent by the plurality of computing nodes respectively, and the task requests are used to instruct the switching node to forward the traffic generated by the plurality of computing nodes in executing the AI ​​task; The switching node determines whether there are sufficient available resources to forward the traffic generated by the multiple computing nodes executing the AI ​​task; The switching node reads data from a first computing node among the plurality of computing nodes based on the traffic status information, including: When the available resources of the switching node are sufficient to forward the traffic generated by the multiple computing nodes executing the AI ​​task, and the switching node receives task requests sent by all the multiple computing nodes, the switching node reads data from the first computing node among the multiple computing nodes according to the traffic status information.

5. The method according to claim 4, characterized in that, The method further includes: When the switching node has completed forwarding the traffic generated by the multiple computing nodes executing the AI ​​task, the switching node notifies each of the multiple computing nodes that the task request has been completed.

6. The method according to any one of claims 1 to 5, characterized in that, The switching node includes a first switching node and a second switching node. The first switching node is used to forward part of the traffic generated by the multiple computing nodes executing the AI ​​task, and the second switching node is used to forward another part of the traffic generated by the multiple computing nodes executing the AI ​​task.

7. A switching node, characterized in that, The switching nodes include: A determination module is used to determine traffic status information corresponding to multiple computing nodes. The traffic status information is used to indicate the load status of the communication link between each of the multiple computing nodes and the switching node. The multiple computing nodes are used to execute artificial intelligence (AI) tasks. The traffic generated by the multiple computing nodes executing the AI ​​tasks is forwarded through the switching node. A communication module is configured to read data from a first computing node among the plurality of computing nodes based on the traffic status information, and forward the data to a third computing node among the plurality of computing nodes, wherein the load of the uplink communication link between the first computing node and the switching node is lower than the load of the uplink communication link between the second computing node and the switching node among the plurality of computing nodes, and the load of the downlink communication link between the third computing node and the switching node is lower than the load of the downlink communication link between the fourth computing node and the switching node among the plurality of computing nodes.

8. The switching node according to claim 7, characterized in that, The traffic status information includes the amount of data cached in the sending queue and receiving queue that correspond to the plurality of computing nodes in the switching node; Alternatively, the traffic status information includes the values ​​of the send counter and receive counter corresponding to the ports used by the switching node to connect to the plurality of computing nodes, wherein the send counter is the remaining number of data packets that the switching node can continue to send through the port, and the receive counter is the remaining number of data packets that the switching node can continue to receive through the port.

9. The switching node according to claim 7 or 8, characterized in that, The communication module is used for: Based on the traffic status information, a read request is sent to the first computing node; The system receives a response from the first computing node, the response including data of the maximum payload size (MPS), and the amount of data that the first computing node needs to send during the execution of the AI ​​task is greater than the MPS.

10. The switching node according to any one of claims 7 to 9, characterized in that, The communication module is also used for: Before the switching node reads data from the first computing node among the plurality of computing nodes, it receives task requests sent by the plurality of computing nodes respectively. The task requests are used to instruct the switching node to forward the traffic generated by the plurality of computing nodes in executing the AI ​​task. Determine whether the available resources are sufficient to forward the traffic generated by the multiple computing nodes executing the AI ​​task; The communication module is configured to read data from the first computing node among the plurality of computing nodes based on the traffic status information, provided that the available resources of the exchange node are sufficient to forward the traffic generated by the plurality of computing nodes executing the AI ​​task, and the exchange node receives task requests sent by all computing nodes among the plurality of computing nodes.

11. The switching node according to claim 10, characterized in that, The communication module is also used for: Once the switching node has completed forwarding the traffic generated by the multiple computing nodes executing the AI ​​task, it notifies each of the multiple computing nodes that the task request has been completed.

12. The switching node according to any one of claims 7 to 11, characterized in that, The switching node includes a first switching node and a second switching node. The first switching node is used to forward part of the traffic generated by the multiple computing nodes executing the AI ​​task, and the second switching node is used to forward another part of the traffic generated by the multiple computing nodes executing the AI ​​task.

13. A switching node, characterized in that, The management node includes a processor, memory, and a communication interface; The communication interface is used to receive and send data. The processor is configured to execute instructions stored in the memory to cause the switching node to perform the method as described in any one of claims 1 to 6.

14. A cluster, characterized in that, The cluster includes a switching node and multiple computing nodes, the multiple computing nodes being used to execute artificial intelligence (AI) tasks, the switching node being used to forward traffic generated by the multiple computing nodes executing the AI ​​tasks, and the switching node being used to execute the method as described in any one of claims 1 to 6.

15. A computer-readable storage medium, characterized in that, Includes instructions that, when run on a switching node, cause the switching node to perform the method as described in any one of claims 1 to 6.

16. A computer program product containing instructions, characterized in that, When it is running on a switching node, it causes the switching node to perform the method as described in any one of claims 1 to 6.

Images