Ensemble communication method, device, controller and system

Abstract

The invention discloses a set communication method, device, controller and system, and relates to the field of distributed systems. In the process of the set communication method, firstly, a controller determines a plurality of sub-topologies of a physical topology of a cluster network, and determines at least one feasible sub-topology comprising all edges of a logical topology according to the plurality of sub-topologies and the logical topology of a set communication algorithm. And then performing combination according to the at least one feasible sub-topology, and determining at least one combination meeting the constraint condition of the link number of the physical topology and the communication bandwidth requirement of the cluster network. And finally, reconstructing the physical topology based on the target combination in the at least one combination to obtain the reconstructed physical topology. Therefore, the link provided by the reconstructed physical topology is fully utilized, the feasible bandwidth during set communication is increased, and the time required by set communication is reduced, so that the resource utilization rate and the communication performance of the cluster network are improved.

Description

Technical Field

[0001] This application relates to the field of distributed systems, and more particularly to a collective communication method, apparatus, controller, and system. Background Technology

[0002] Set communication is a common method for synchronizing data between multiple processes in distributed applications such as scientific computing and model training. Set communication algorithms refer to the methods used to implement operations in set communication; each set communication operation can be implemented using multiple set communication algorithms. Set communication operations include broadcast, scatter, gather, all gather, reduce, and all reduce.

[0003] Aggregate communication algorithms are applied to cluster networks with diverse physical topologies. These networks can reconstruct the physical topology based on the characteristics and traffic demands of distributed applications, thereby improving network performance. However, because the physical nodes in the reconstructed topology are limited by physical constraints such as actual port connection methods, the reconstructed topology cannot perfectly match the traffic demands of distributed applications, resulting in low overall resource utilization and communication performance of the cluster network. Summary of the Invention

[0004] This application provides a cluster communication method, apparatus, controller, and system that can solve the problems of low overall resource utilization and communication performance in cluster networks.

[0005] To achieve the above objectives, this application adopts the following technical solution.

[0006] Firstly, a set communication method for a controller is provided. This method includes: firstly, the controller determines multiple sub-topologies within the physical topology of a cluster network, where each sub-topology is a generated subgraph of the physical topology. Then, the controller determines at least one feasible sub-topology among the multiple sub-topologies, including all edges of the logical topology containing the set communication algorithm. Each edge represents a communication connection between two physical nodes in the sub-topology or physical topology. Next, based on constraints related to the number of links in the physical topology, the controller determines at least one combination of at least one feasible sub-topology that satisfies the communication bandwidth requirements of the cluster network. Each of these at least one combination includes at least one feasible sub-topology. Finally, the controller reconstructs the physical topology based on a target combination from the at least one combination, obtaining the reconstructed physical topology.

[0007] Based on the above implementation, the controller can determine that at least one feasible sub-topic of the physical topology of the cluster network includes all edges of the logical topology of the collective communication algorithm, thus ensuring a complete traffic fit between the feasible sub-topology and the collective communication algorithm. This improves the communication performance of the cluster network. When the feasible sub-topology is perfectly matched with the collective communication algorithm, the controller determines at least one combination within the feasible sub-topology that satisfies the communication bandwidth requirements of the cluster network. Since the constraint is the communication bandwidth requirements of the cluster network, congested links can be avoided in any combination, and any combination can meet the communication bandwidth requirements. Thus, under the constraint of the number of links in the physical topology, the available bandwidth of the combination is maximized, thereby fully utilizing network link resources. Simultaneously, the controller reconstructs the physical topology based on the target combination from at least one combination, obtaining a reconstructed physical topology. Implementing collective communication based on the reconstructed physical topology can further improve the overall resource utilization of the cluster network and enhance communication performance.

[0008] As one possible implementation, the controller can determine at least one feasible subtopic that includes all edges of the logical topology among multiple subtopics. For example, if multiple subtopics include a first subtopic, the controller first maps each logical node of the logical topology to the first subtopic in different arrangements, where the physical nodes of the first subtopic have different arrangements. By arranging the physical nodes of the first subtopic in different ways, the controller can determine all mapping relationships between the logical topology and the first subtopic. Furthermore, the controller maps each logical node of the logical topology to the first subtopic in different arrangements and determines the various mapping relationships between the logical topology and the first subtopic. Then, the controller determines that any arrangement of the first subtopic after mapping all edges of the logical topology is a feasible subtopic; that is, under the experiment of multiple mapping relationships, a feasible subtopic including all edges of the logical topology under any mapping relationship is determined. In this way, all edges of the logical topology exist in the feasible subtopics, which can perfectly adapt to the traffic of the set communication algorithm, avoiding the problem that physical nodes cannot communicate through direct links, thereby improving the communication performance of the cluster network.

[0009] As one possible implementation, the controller can map each logical node of the logical topology to a first sub-topology with different arrangement orders. For example, the controller determines the order of each physical node in the first sub-topology with different arrangement orders, as well as the order of each logical node in the logical topology. The order indicates the sequential position of nodes; therefore, based on the order of each node, the controller can determine the position of the physical node in the first sub-topology with different arrangement orders, and the position of the logical node in the logical topology. The controller maps each logical node to a physical node with the same order in the first sub-topology with different arrangement orders, based on the arrangement order of the logical nodes in the logical topology and the arrangement order of the physical nodes in the first sub-topology. Thus, the controller can accurately map logical nodes with the same order to physical nodes based on the node's order, thereby accurately determining each feasible sub-topology.

[0010] Optionally, the controller can implement node mapping by generating a node mapping table. The node mapping table includes logical entries and at least one physical entry. The logical entries include logical nodes arranged in the logical order of the logical nodes in the logical topology, and the at least one physical entry includes physical nodes of the first sub-topology arranged in different orders. The logical entries and at least one physical entry in the node mapping table clearly indicate each mapping relationship between the logical topology and the first sub-topology. The controller maps each logical node in the logical entries to a physical node with the same position in the at least one physical entry. The controller performs mapping for each physical entry separately, thereby clearly and accurately completing the mapping from logical nodes to physical nodes under each mapping relationship, avoiding the inability to determine all feasible sub-topologies due to the omission of any mapping relationship.

[0011] Optionally, the controller can determine feasible sub-topologies by determining an adjacency matrix of the logical topology, which represents the connection relationships between logical nodes in the logical topology. The controller uses a matrix to represent the connection relationships of each logical node, which clearly and completely displays these relationships. However, due to the inherent characteristics of the matrix, the connection relationships between logical nodes may be repeatedly displayed. The controller determines a isomorphism detection table based on the adjacency matrix. The isomorphism detection table includes at least one detection entry, and each of these entries includes the logical node to be detected and the logical nodes connected to it. The isomorphism detection table is determined based on the content of the lower left corner of the diagonal from the top left to the bottom right of the adjacency matrix; therefore, each first detection entry only includes the connection relationships between the logical node to be detected and the logical nodes preceding it in the entry.

[0012] The isomorphism detection table does not include duplicate connections between logical nodes, thus avoiding the inefficiency in determining feasible subtopologies caused by repeated detections. When the first subtopology mapped to the first physical table entry includes all connections represented by all detection entries, the controller determines the first subtopology mapped to the first physical table entry as a feasible subtopology. The controller quickly and accurately determines feasible subtopologies by detecting the first subtopology mapped to the first physical table entry.

[0013] Optionally, if at least one detection entry includes a first detection entry, and the controller does not include the connection relationship of the logical node to be detected in the first detection entry in the first sub-topology after mapping corresponding to the first physical entry, then at least one physical entry containing a non-feasible prefix is ​​deleted. The non-feasible prefix is ​​composed of the physical node mapped to the logical node to be detected in the first physical entry and the first physical node of the first physical entry. The mapping relationship between the physical node and the corresponding logical node in the non-feasible prefix is ​​fixed, therefore the connection relationship between every two physical nodes from the first physical node to the last physical node in the non-feasible prefix is ​​fixed.

[0014] When the controller performs detection based on the first detection entry, it detects the fixed connection relationships between the logical node to be detected in the first detection entry and the logical nodes preceding it in the logical entry. Therefore, the determined connection relationships between the last physical node and the previous physical nodes in the infeasible prefix of the physical entry do not include the connection relationships between the logical nodes to be detected in the first detection entry mapped from the last physical node. The controller determines that the first sub-topology mapped to physical entries containing infeasible prefixes is not a feasible sub-topology. Therefore, by deleting at least one physical entry containing an infeasible prefix, pruning optimization of physical entries can be achieved, further improving the efficiency of determining feasible sub-topologies.

[0015] As one possible implementation, the controller can determine the target combination among at least one combination based on the physical link attributes corresponding to at least one combination. For example, physical link attributes include the number of links; the controller determines the combination with the fewest links as the target combination based on the number of links corresponding to at least one combination. Since fewer links corresponding to feasible sub-topologies in at least one combination require fewer resources to execute the ensemble communication algorithm in the cluster network, the resource utilization and communication performance of the cluster network are improved. For example, physical link attributes include bandwidth; the controller determines the combination with the largest bandwidth and value as the target combination based on the bandwidth and value corresponding to at least one combination. Since a larger bandwidth and value corresponding to at least one combination indicates a larger number of feasible sub-topologies in at least one combination, the ensemble communication algorithm has more communication channels available, further improving the communication efficiency and performance of executing the ensemble communication algorithm in the cluster network.

[0016] As one possible implementation, the controller can determine at least one combination of feasible sub-topologies by including a first combination, which is determined by the controller using depth-first search (DFS). The controller then determines whether the number of links in the first combination satisfies the physical topology's link count constraint and whether the total bandwidth of the first combination meets the communication bandwidth requirements of the cluster network; if so, then at least one combination is determined to be the first combination.

[0017] The constraint on the number of links in the physical topology includes the limitation on the actual number of physical links between any two physical nodes in the physical topology that are in communication connection.

[0018] Therefore, the controller determines the first combination by considering the constraints of the number of links in the physical topology and the communication bandwidth requirements of the cluster network. While meeting the communication bandwidth requirements of the ensemble communication algorithm, it avoids situations where the number of uses between two physical nodes in the reconstructed physical topology exceeds the actual number of links, thus preventing bottleneck links. Furthermore, the controller uses the Depth-First Search (DFS) method to enumerate combinations of feasible sub-topologies, stopping the determination when the first combination is identified. This avoids traversing and determining all possibilities, thereby reducing the time required to determine at least one combination when the number of combinations is large.

[0019] As one possible implementation, if the physical nodes of the physical topology include multiple processors, and the physical topology includes a first physical node and a second physical node, and the number of links between the first and second physical nodes is less than a preset number, it will result in a insufficient number of feasible sub-topologies being searched in the node mapping table. Therefore, after the controller generates the node mapping table, before mapping each logical node in the logical table entry to a physical node with the same sequence position in at least one physical table entry, at least one physical table entry is referred to as a second physical table entry. If the first and second physical nodes are adjacent in the second physical table entry, the controller adds at least one physical node to the second physical table entry, resulting in at least one third physical table entry. The number of identical physical nodes in any third physical table entry is no greater than the number of processors included in the same physical node, and the same physical nodes are not contiguous.

[0020] Therefore, the controller can utilize the links between other physical nodes by adding other physical nodes, that is, it realizes the communication between the first physical node and the second physical node by taking a detour. Compared with the scheme of deploying tasks after the link resources are sufficient and the scheme of occupying only one physical link, the embodiment of this application can make full use of the link resources between physical nodes and improve the resource utilization of the cluster network.

[0021] In a second aspect, a collective communication apparatus is provided, the apparatus comprising modules for performing the collective communication method in the first aspect or any possible implementation thereof.

[0022] The second aspect describes a collective communication device that can be a terminal device or a network device, or a chip (system), network card, or other component or assembly that can be disposed in a terminal device or a network device, or a device that includes a terminal device or a network device. This application does not limit the scope of the application.

[0023] Furthermore, the technical effects of the collective communication device described in the second aspect can be referred to the technical effects of the collective communication method described in the first aspect, and will not be repeated here.

[0024] Thirdly, a collective communication system is provided, comprising at least one switch, at least one cluster server, and a controller, the controller being configured to perform operational steps of the collective communication method in any possible design of the first aspect when performing a reconfiguration of the physical topology of the cluster network.

[0025] Furthermore, the technical effects of the third aspect can be referenced from the technical effects of the collective communication method described in the first aspect, and will not be repeated here.

[0026] Fourthly, a computer-readable storage medium is provided, comprising: computer software instructions; when the computer software instructions are executed in a computer, causing the computer to perform operational steps of the method as described in any possible implementation of the first aspect.

[0027] Fifthly, embodiments of this application provide a chip system. The chip system includes a memory and at least one processor. The memory stores a set of computer instructions, which, when executed by the processor, perform operational steps of the method of any optional embodiment in the first aspect.

[0028] Sixthly, a computer program product is provided that, when run on a computer, causes the computer to perform the operational steps of the method as described in any possible implementation of the first aspect.

[0029] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of an all reduce operation using sum;

[0031] Figure 2 A schematic diagram illustrating the implementation of the all reduce operation using a recursive binary search and doubling algorithm;

[0032] Figure 3 A schematic diagram illustrating the implementation of the all reduce operation using a butterfly all reduce algorithm;

[0033] Figure 4 A schematic diagram illustrating the implementation of the all reduce operation using a ring all reduce algorithm;

[0034] Figure 5 A schematic diagram illustrating the traffic characteristics of an AI application;

[0035] Figure 6 This is a schematic diagram of a reconstructed physical topology;

[0036] Figure 7 This is an architecture diagram of an optical AI cluster network;

[0037] Figure 8 This is a schematic diagram of server communication for an existing mapping scheme;

[0038] Figure 9 This is a schematic diagram of the physical topology of a server (DGX-1);

[0039] Figure 10This is a schematic diagram of a loop.

[0040] Figure 11 This is a schematic diagram of another type of loop;

[0041] Figure 12 This is a schematic diagram of a structural tree constructed after a topological rearrangement of the dragonfly topology;

[0042] Figure 13 An architecture diagram of an optical AI cluster network 13 provided in this application embodiment;

[0043] Figure 14 A schematic diagram of a specific optical AI cluster network 14 provided for an embodiment of this application;

[0044] Figure 15 A schematic diagram of another specific optical AI cluster network 15 provided in this application embodiment;

[0045] Figure 16 A flowchart illustrating a collection communication method provided in an embodiment of this application;

[0046] Figure 17 A schematic diagram of a physical topology provided for an embodiment of this application;

[0047] Figure 18 This is a schematic diagram of the communication rules of an HD algorithm provided in an embodiment of this application;

[0048] Figure 19 A schematic diagram of the logical topology of an HD algorithm provided in an embodiment of this application;

[0049] Figure 20 This is a schematic diagram of the communication rules of a ring algorithm provided in an embodiment of this application;

[0050] Figure 21 A schematic diagram of the logical topology of a ring algorithm provided in an embodiment of this application;

[0051] Figure 22 A flowchart illustrating a set communication algorithm provided in an embodiment of this application;

[0052] Figure 23 A flowchart illustrating another set communication algorithm provided in an embodiment of this application;

[0053] Figure 24 This is a schematic diagram of server communication after adopting the aggregated communication method provided in the embodiments of this application;

[0054] Figure 25 A schematic diagram illustrating a process for determining a feasible subtopology, provided as an embodiment of this application;

[0055] Figure 26 A schematic diagram of a physical topology provided for an embodiment of this application;

[0056] Figure 27 This is a schematic diagram of the mapped subtopology corresponding to physical entry 01234;

[0057] Figure 28 This is a schematic diagram of the mapped subtopology corresponding to physical entry 03142;

[0058] Figure 29 A schematic diagram of a physical topology provided for an embodiment of this application;

[0059] Figure 30 A schematic diagram of a feasible subtopology mapped to physical entry 01012 provided in an embodiment of this application;

[0060] Figure 31 A schematic diagram of a feasible subtopology mapped to physical entry 0102 provided in an embodiment of this application;

[0061] Figure 32 A schematic diagram of a feasible subtopology mapped to physical entry 012 provided in an embodiment of this application;

[0062] Figure 33 A schematic diagram of a possible collection communication device provided for embodiments of this application;

[0063] Figure 34 This is a schematic diagram of the structure of a controller provided in an embodiment of this application. Detailed Implementation

[0064] This application provides a method for aggregated communication, particularly a method for reconstructing the physical topology of a cluster network. First, the controller determines at least one feasible sub-topology based on multiple sub-topologies of the logical topology and the physical topology of the cluster network. A feasible sub-topology includes all edges of the logical topology, meaning it can realize the actual physical network connections of all physical nodes in the logical topology. Therefore, each sub-topology in the at least one feasible sub-topology is perfectly adapted to the traffic of the aggregated communication algorithm. Further, the controller determines at least one combination of feasible sub-topologies that satisfies the constraints on the number of links in the physical topology and the communication bandwidth requirements of the cluster network. Since any combination in the at least one set satisfies the constraints on the number of links in the physical topology, congested links can be avoided in any combination of feasible sub-topologies, and any combination can meet the communication bandwidth requirements. Furthermore, the more feasible sub-topologies in the combination, the greater the communication bandwidth that the aggregated communication algorithm can obtain; the fewer the link requirements of the feasible sub-topologies in the combination, the less network resources the aggregated communication algorithm needs; thus, the resource utilization and communication performance of the feasible sub-topologies in the target combination of at least one set are higher when performing aggregated communication. Finally, the controller reconstructs the physical topology based on the target combination in at least one combination, resulting in a reconstructed physical topology that includes all feasible sub-topologies of the target combination.

[0065] Compared to existing solutions that are only applicable to fixed communication topologies and algorithms, the solution in this application does not limit the specific physical topology and aggregation communication algorithm, nor does it limit whether it is within a cluster server or between cluster servers. Therefore, the solution in this application can be used regardless of the physical topology and aggregation communication algorithm used in the cluster network. In this way, the aggregation communication algorithm can make full use of the links provided by the reconstructed physical topology, increase the feasible bandwidth during aggregation communication, alleviate link congestion, reduce the time required for aggregation communication, improve the resource utilization and communication performance of the cluster network, and avoid wasting link bandwidth.

[0066] This application can be applied not only to existing cluster networks and aggregated communication scenarios employing distributed systems, but also to future cluster networks and aggregated communication scenarios employing distributed systems. The terminology used in the implementation section of this application is only used to explain specific embodiments of this application and is not intended to limit this application. For ease of understanding, the relevant terminology involved in the embodiments of this application will be introduced below.

[0067] (1) broadcast

[0068] broadcast is used to broadcast data from one process to other processes.

[0069] (2) scatter

[0070] Scatter is used to distribute data evenly across different processes from the root process.

[0071] (3) gather

[0072] `gather` is used to aggregate data from different processes into the root process.

[0073] (4) all gather

[0074] All gather is used to aggregate data from different processes into all processes, ensuring that all processes have access to the data from all other processes.

[0075] (5) reduce

[0076] The reduce function is used to retrieve a set of data on each process, retrieve the data from all processes, and perform corresponding operations such as summation, maximum value extraction, or average variance extraction, and then aggregate the results to the root process.

[0077] (6)all reduce

[0078] The all reduce operation is used to distribute the results of the reduce operation to other processes.

[0079] Figure 1 This is a diagram illustrating an all-reduce operation using sum. For example... Figure 1 As shown, before data synchronization using the all reduce operation, one process runs on each of the neural processing units (NPUs) 0, NPU1, NPU2, and NPU3. NPU0 process acquires data A. NPU1 process acquires data B. NPU2 process acquires data C. NPU3 process acquires data D. After data synchronization using the all reduce operation with the SUM operation on each process, each process contains the result of the operation A+B+C+D.

[0080] (7) Collective Communication Library

[0081] Collective communication libraries serve as the underlying support libraries for distributed deep learning frameworks. Existing collective communication libraries include the high-performance messaging library (Open MPI), the collective communication library (Gloo), and the NVIDIA Collective Multi-GPU Communication Library (NCCL), among others.

[0082] (8) Implementation of the set communication algorithm

[0083] Existing collective communication algorithms are implemented using their own methods based on the message passing interface (MPI).

[0084] Taking the recursive binary search and doubling algorithm to implement the all reduce operation on four logical nodes as an example. Figure 2 As shown, Figure 2 This is a schematic diagram illustrating the implementation of the all reduce operation using a recursive binary search and doubling algorithm. Figure 2 The CCP consists of four logical nodes: rank0, rank1, rank2, and rank3. rank0 sends data to rank1. rank1 sends its own data and rank0's data to rank3. rank2 sends data to rank3. rank3, after receiving all the data, performs an all-reduce operation to obtain the result, and then distributes the result to rank1 and rank2. rank1 then distributes the result to rank0.

[0085] Let's take the butterfly algorithm (butterfly all reduce) as an example, implementing the all reduce operation on four logical nodes. Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the implementation of the all reduce operation using a butterfly all reduce algorithm. Figure 3 The CCP consists of four logical nodes: rank0, rank1, rank2, and rank3. The first step is to group adjacent nodes of rank0 and rank1 together, and adjacent nodes of rank2 and rank3 together. Within each group, adjacent logical nodes communicate with each other to obtain the sum of data acquired from the data nodes of their respective adjacent nodes. The second step is to group rank0, rank1, rank2, and rank3 together again. rank0 and rank2 nodes communicate with each other to obtain the sum of data acquired from the data nodes of their respective adjacent nodes. Similarly, rank1 and rank3 nodes communicate with each other to obtain the sum of data acquired from the data nodes of their respective adjacent nodes. This ensures that each logical node contains the sum of data acquired from all four logical nodes.

[0086] Let's take the ring all reduce algorithm as an example, implementing the all reduce operation on four logical nodes. Figure 4 As shown, Figure 4This diagram illustrates the implementation of the all-reduce operation in a ring all-reduce algorithm. The first step involves rank0 sending data 0 to rank1, rank1 sending data 1 to rank2, rank2 sending data 2 to rank3, and rank3 sending data 3 to rank0. After the first step, rank0 contains 0+3, rank1 contains 1+0, rank2 contains 2+1, and rank3 contains 3+2.

[0087] The second step is that rank0 sends the obtained data 3 to rank1, rank1 sends the obtained data 0 to rank2, rank2 sends the obtained data 1 to rank3, and rank3 sends the obtained data 2 to rank0. After the second step, the data in rank0 is 0+3+2, the data in rank1 is 1+0+3, the data in rank2 is 2+1+0, and the data in rank3 is 3+2+1.

[0088] The third step is that rank0 sends the obtained data 2 to rank1, rank1 sends the obtained data 3 to rank2, rank2 sends the obtained data 0 to rank3, and rank3 sends the obtained data 1 to rank0. After the second step, the data in rank0 is 0+3+2+1, the data in rank1 is 1+0+3+2, the data in rank2 is 2+1+0+3, and the data in rank3 is 3+2+1+0, thus ensuring that each logical node includes the sum of the data from the four logical nodes.

[0089] As applications expand and underlying network equipment is updated, cluster networks employing distributed systems can have diverse physical topologies. These diverse physical topologies can be determined based on application characteristics, identifying every two physical nodes in the communicating physical topology and reconstructing the topology to improve network performance. However, in existing reconstructed physical topologies, when physical nodes execute the set communication algorithm, the mapping between logical and physical nodes is done in ascending order of node numbers, and communication follows the logical node communication order within the set communication algorithm.

[0090] Due to physical limitations such as actual port connection methods, there are situations where the reconstructed physical topology cannot perfectly match the traffic requirements of the aggregated communication algorithm. These situations include: (1) the available links in the reconstructed physical topology are not used or the bandwidth of the available links is not fully utilized, resulting in a waste of link resources or link bandwidth resources; (2) the links in the reconstructed physical topology do not meet the communication requirements of the aggregated communication algorithm, leading to a decrease in communication performance; therefore, the overall resource utilization and communication performance of the cluster network in the existing methods are low.

[0091] The controller reconstructs the physical topology of the optical artificial intelligence (AI) cluster network based on application characteristics and traffic requirements. For example... Figure 5 and Figure 6 As shown, Figure 5 This is a schematic diagram illustrating the traffic characteristics of an AI application. Figure 5 The numbers 0-15 on the horizontal and vertical axes represent the logical node numbers. Figure 5 In this context, each large grayscale cell represents a logical node on the horizontal axis that communicates with a logical node on the vertical axis; the larger the grayscale value, the more bandwidth is consumed in this communication. Therefore, according to... Figure 5 After physical topology reconstruction of the flow characteristics, the following is obtained Figure 6 . Figure 6 This is a schematic diagram of a reconstructed physical topology. Figure 6 An arrow in the diagram represents a link in a reconstructed physical topology. Figure 6 Each number in the table is the number of the physical node. Figure 6 It includes three interconnected loops. Loop 1 is composed of 0-1-2-3-4-5-6-7-8-9-10-11-12-13-14-15, loop 2 is composed of 0-3-6-9-12-15-2-5-8-11-14-1-4-7-10-13, and loop 3 is composed of 0-7-14-5-12-3-10-1-8-15-6-13-4-11-2-9.

[0092] Although communication between two physical nodes in the reconstructed physical topology can obtain sufficient bandwidth and does not conflict with communication between other physical nodes, the reconstructed physical topology cannot perfectly match the traffic requirements of the aggregated communication algorithm due to physical limitations such as the actual port connection method.

[0093] The above problem is described below with a specific case. In a large-scale optical AI cluster network based on micro-electro-mechanical systems optical cross-connect (MEMS OXC), electrical switches are used for communication within the cluster servers, and optical switches are used for communication between cluster servers. Specifically, the optical switches are MEMS OXCs. Figure 7 This is an architecture diagram of an optical AI cluster network. (For example...) Figure 7 As shown, the optical AI cluster network comprises three layers: the server layer, the electrical switch layer, and the optical switch layer. Server 1 and Server 2 in cluster server (POD) 1 communicate via electrical switch T1; Server 3 and Server 4 in cluster server POD2 communicate via electrical switch T2; Server 5 and Server 6 in cluster server POD3 communicate via electrical switch T3; Server 7 and Server 8 in cluster server POD4 communicate via electrical switch T4; and Server 9 and Server 10 in cluster server POD5 communicate via electrical switch T5. Electrical switches T1, T2, T3, T4, and T5 communicate with each other via MEMS OXC1 and MEMS OXC2.

[0094] Upon receiving Task 5, Task 1 is using Servers 2 and 10, Task 2 is using Servers 3 and 9, Task 3 is using Server 6, and Task 4 is using Server 8. At this time, all 8 remaining ports (X8) of switch T1 are connected to MEMS OXC1. All 8 remaining ports (X8) of switch T2 are connected to MEMS OXC2. Of the 8 remaining ports (X4) of switch T3, 4 are connected to MEMS OXC1, and the other 4 are connected to MEMS OXC2. Of the 8 remaining ports of switch T4, 4 are connected to MEMS OXC1, and the other 4 are connected to MEMS OXC2.

[0095] Task 5 requires 32 graphics processing units (GPUs) to run the ring allreduce algorithm. The controller has assigned servers 1, 4, 5, and 7. If the existing method of mapping logical nodes to physical nodes in the reconstructed physical topology according to their node numbers in ascending order (i.e., mapping logical node rank1 to server 1 on switch T1, logical node rank2 to server 4 on switch T2, logical node rank3 to server 5 on switch T3, and logical node rank4 to server 7 on switch T4), then based on the communication order of logical node rank1-rank2-rank3-rank4 within the ring allreduce algorithm, communication can only occur in the order of server 1-server 4-server 5-server 7. At this time, although the electrical switch T1, which is connected to server 1, and the electrical switch T2, which is connected to server 4, have idle ports, the idle ports of electrical switch T1 and electrical switch T2 are not connected to the same optical switch. Therefore, a ring of T1-T2-T3-T4 cannot be constructed through a direct link.

[0096] Figure 8 This is a schematic diagram of server communication for an existing mapping scheme. For example... Figure 8 As shown, logical node rank1 is mapped to server 1 of switch T1, logical node rank2 is mapped to server 3 of switch T2, logical node rank3 is mapped to server 5 of switch T3, and logical node rank4 is mapped to server 7 of switch T4. A direct link cannot be established between switch T1 and switch T2; therefore, communication between them can only be achieved through multi-hop relay. Specifically, communication between switch T1 and switch T2 is achieved through relay via switch T3 and MEMSOXC2, increasing the transmission path during communication. Furthermore, the relay communication process between switch T1 and switch T2 conflicts with the direct communication between switch T2 and switch T3, making the communication link between switch T1 and switch T2 a bottleneck, resulting in increased communication latency and reduced communication efficiency.

[0097] The controller may also relinquish the scheduling server, waiting for resources to become available before scheduling it. However, this results in current computing resources being idle and wasted, creating resource fragmentation, and increasing task waiting time, leading to a decrease in the overall resource utilization and communication performance of the optical AI cluster network.

[0098] To address the aforementioned technical issues, NCCL can adapt the mapping between the physical topology of the cluster servers and the logical topology of the ring all reduce algorithm when using it within a cluster server. A specific example is described below.

[0099] Figure 9 This is a schematic diagram of the physical topology of a server (DGX-1). Figure 9 As shown, GPU0 is connected to GPU3, GPU2, GPU1, and GPU4. GPU1 is connected to GPU0, GPU3, GPU2, and GPU5. GPU2 is connected to GPU0, GPU3, GPU1, and GPU6. GPU3 is connected to GPU0, GPU1, GPU2, and GPU7. GPU4 is connected to GPU0, GPU5, GPU8, and GPU7. GPU5 is connected to GPU4, GPU7, GPU6, and GPU1. GPU6 is connected to GPU4, GPU5, GPU7, and GPU2. GPU7 is connected to GPU4, GPU5, GPU6, and GPU3.

[0100] NCCL in Figure 9 When the communication within the server shown uses the ring all reduce algorithm, it creates four types of one-way loops that are connected end-to-end:

[0101] Loop 1: GPU0-GPU1-GPU5-GPU4-GPU7-GPU6-GPU2-GPU3.

[0102] Loop 2: GPU3-GPU2-GPU6-GPU7-GPU4-GPU5-GPU1-GPU0.

[0103] Loop 3: GPU0-GPU2-GPU1-GPU3-GPU7-GPU5-GPU6-GPU4.

[0104] Loop 4: GPU4-GPU6-GPU5-GPU7-GPU3-GPU1-GPU2-GPU0.

[0105] Figure 10 This is a schematic diagram of a loop. For example... Figure 10 As shown, Figure 10 In the diagram, the clockwise loop starting from GPU0 represents loop 1, and the counterclockwise loop starting from GPU3 represents loop 2.

[0106] Figure 11 This is a schematic diagram of another type of loop. For example... Figure 11 As shown, Figure 11In the diagram, the clockwise loop starting from GPU0 represents loop 3, and the counterclockwise loop starting from GPU4 represents loop 4.

[0107] Figure 10 and Figure 11 The four types of rings do not affect each other and have independent link bandwidth. Each ring is only responsible for the sliced ​​data of its own communication portion. These four rings can also be viewed as a mapping between the physical topology within the cluster server and the logical topology of the ring all reduce algorithm. The logical topology of the ring all reduce algorithm communicates in the order of rank0-rank1-rank2-rank3-rank4-rank5-rank6-rank7. Therefore, the logical nodes in the logical topology of the ring all reduce algorithm are sequentially mapped to the physical nodes of each ring in the physical topology.

[0108] The above solution can only be applied within a cluster server, and the cluster server only supports the ring all reduce algorithm, so it cannot solve the technical problem of this invention.

[0109] In a scheme for rearranging cluster servers based on the dragonfly topology ring algorithm, the main steps involve randomly shuffling the order of all child nodes in the dragonfly topology's structural tree, rearranging the target switch nodes within a group that are physically closest to adjacent group nodes to the edge positions, and then traversing the central node of the structural tree to obtain a new node order list. Child nodes then perform ring communication according to this new node order list.

[0110] Figure 12 This is a schematic diagram of a structural tree constructed after a topological rearrangement using the dragonfly topology. (Example) Figure 12 As shown, Figure 12 The topology consists of four groups of nodes, each containing at least one switch node, and each switch node connects to at least one child node. The shuffled order of the group nodes is random, and the switch node within a group node that is physically closest to its neighbor is located at the edge of the group node. Because the shuffled order is random, the link load between and within group nodes is uniform. When performing ring communication based on the new node order, traffic can be evenly distributed throughout the Dragonfly topology, reducing interference from simultaneous ring communication between different jobs and alleviating link congestion in the Dragonfly topology.

[0111] The above-mentioned scheme is an arrangement scheme for specific physical topologies and aggregated communication algorithms, and cannot be adapted to optical AI cluster networks and other aggregated communication algorithms. Therefore, it cannot solve the technical problem of this invention.

[0112] The aggregated communication method provided in this application can be applied to various networks, including optical AI cluster networks, electrical cluster architecture networks, and other cluster networks employing distributed systems. The following explanation uses a controller in an optical AI cluster network as an example.

[0113] like Figure 13 As shown, Figure 13 This is an architecture diagram of an optical AI cluster network 13 provided in an embodiment of this application. The optical AI cluster network 13 includes a controller 131, at least one switch 132, and at least one cluster server (such as cluster server 133, cluster server 134, and cluster server 135). The controller 131 is connected to at least one switch 132, cluster server 133, cluster server 134, and cluster server 135, respectively. For example, the controller 131 can be connected to at least one switch 132, cluster server 133, cluster server 134, and cluster server 135 through an aggregation port or a hybrid port.

[0114] Controller 131 is used to schedule network resources of the optical AI cluster network 13. For example, controller 131 stores information such as the physical topology and resource occupancy status of the optical AI cluster network 13. Based on the logical and physical topologies of the AI ​​tasks, controller 131 determines at least one target combination of feasible sub-topologies. Based on the target combination, controller 131 schedules at least one switch 132 to reconstruct the physical topology.

[0115] At least one switch 132 is used to provide data exchange services for communication between cluster servers. The at least one switch 132 can be an optical cross-connect switch, a fast optical switch, or other optical switch. This embodiment does not limit the specific technology or device form used in the switch.

[0116] At least one server in the cluster will deploy multiple GPUs, which are the actual main components for AI computation in the optical AI cluster network. Each server can have one or more GPUs. When a server has multiple GPUs, communication between the GPUs can be achieved using either an optical switch or an electrical switch; this embodiment does not impose any limitations on this.

[0117] Figure 13This is merely a schematic diagram provided in this embodiment and does not limit the specific structure of the optical AI cluster network 13. For example, the optical AI cluster network 13 may also include a passive optical network (PON).

[0118] In the aggregated communication method, controller 131 determines a target combination of feasible sub-topologies that are fully adapted to the traffic of the aggregated communication algorithm, based on the physical topology of the cluster network and the logical topology of the aggregated communication algorithm. Controller 131 then reconstructs the physical topology based on the links used in the target combination to obtain the reconstructed physical topology.

[0119] As one possible implementation, controller 131 determines multiple sub-topologies of the physical topology of the cluster network. Based on these sub-topologies and the logical topology of the ensemble communication algorithm, it determines at least one feasible sub-topology that includes all edges of the logical topology. Controller 131 combines the at least one feasible sub-topology to determine at least one combination that satisfies the constraints on the number of links in the physical topology and the communication bandwidth requirements of the cluster network. Controller 131 reconstructs the physical topology based on the links used by the target combination in the at least one combination to obtain the reconstructed physical topology.

[0120] Optionally, the physical topology of a cluster network refers to the physical structure between devices in the cluster network. Sub-topology can also be called a generated subgraph or subnetwork, and multiple sub-topologies form a generated subgraph of the physical topology.

[0121] Optionally, the logical topology of the collective communication algorithm refers to the actual path during data transmission.

[0122] Optionally, the controller 131 combines at least one feasible sub-topology to determine at least one combination in which the number of links satisfies the constraint of the number of links in the physical topology and the total bandwidth satisfies the communication bandwidth requirements of the cluster network.

[0123] Optionally, controller 131 may randomly select a combination from at least one combination as the target combination; or controller 131 may determine the target combination with the fewest links among at least one combination based on the number of links in at least one combination; or controller 131 may determine the target combination with the largest bandwidth among at least one combination based on the bandwidth sum of at least one combination.

[0124] To illustrate the optical AI cluster network of the embodiments of this application, two specific optical AI cluster networks are provided in the embodiments of this application.

[0125] Figure 14 This is a schematic diagram of a specific optical AI cluster network 14 provided in an embodiment of this application. Figure 14As shown, the optical AI cluster network 14 includes a controller 141, at least one optical circuit switching (OSC) (such as optical switch OSC142, optical switch OSC143, and optical switch OSC144), and at least one cluster server (such as cluster server 145, cluster server 146, cluster server 147, and cluster server 148). Each of the at least one cluster server contains one terabit physical layer (TeraPHY), and the chip contains one GPU. Figure 14 The optical switches OSC142, OSC143, and OSC144 in the text are equivalent to Figure 13 At least one switch 132 in the system. Figure 14 One photonic chip is equivalent to Figure 13 One of the cluster servers.

[0126] Figure 15 This is a schematic diagram of another specific optical AI cluster network 15 provided in an embodiment of this application. (See diagram below.) Figure 15 As shown, the optical AI cluster network 15 includes a controller 151, at least one optical switch (two optical switches are shown in the figure, namely optical switch 152 and optical switch 153, but not limited to two optical switches), and at least one cluster server (three cluster servers are shown in the figure, namely cluster server 154, cluster server 155 and cluster server 156, but not limited to three cluster servers). Any of the at least one cluster server includes an electrical switching network group and multiple servers. Figure 15 Optical switches 152 and 153 are equivalent to Figure 13 At least one switch 132 in the system.

[0127] Combination Figure 14 and Figure 15 For any specific optical AI cluster network shown, this application embodiment provides an implementation of a cluster communication method. The following, in conjunction with... Figures 16 to 24 The collection communication method provided in the embodiments of this application will be described in detail.

[0128] like Figure 16 As shown, Figure 16 This is a flowchart illustrating a set communication method provided in an embodiment of this application. This set communication method is applied to... Figure 14 or Figure 15 The optical AI cluster network shown is illustrated. This cluster communication method can be achieved by… Figure 14 The controller 141 in the optical AI cluster network shown executes, or is performed by, Figure 15The controller 151 in the illustrated optical AI cluster network performs the operation. Here, we will use the controller 151 executing the aggregated communication method provided in this embodiment as an example for explanation. Figure 16 As shown, the collection communication method provided in this embodiment includes steps 1601 to 1604.

[0129] Step 1601: Controller 151 determines multiple sub-topologies in the physical topology of the cluster network, and the multiple sub-topologies are generated subgraphs of the physical topology.

[0130] The controller 151 determines multiple sub-topologies of the physical topology based on each physical node in the physical topology of the cluster network and the connection relationships between the physical nodes.

[0131] As one possible implementation, physical topology can be used to graphically represent the physical connections between devices.

[0132] For example, Figure 17 This is a schematic diagram of a physical topology provided in an embodiment of this application. For example... Figure 17 As shown, physical node 0 is connected to physical nodes 1, 2, 3, 4, 5, 6, and 7. Physical node 1 is connected to physical nodes 0, 2, and 5. Physical node 2 is connected to physical nodes 0, 1, 3, and 7. Physical node 3 is connected to physical nodes 0, 2, 4, 5, 6, and 7. Physical node 4 is connected to physical nodes 0, 3, and 5. Physical node 5 is connected to physical nodes 0, 1, 3, 4, and 6. Physical node 6 is connected to physical nodes 0, 3, 5, and 7. Physical node 7 is connected to physical nodes 0, 2, 3, and 6.

[0133] Figure 17 Each physical node in the cluster network represents a device, and the connection between any two physical nodes represents the device connection between the two physical nodes.

[0134] One possible implementation is to use multiple sub-topologies as generating subgraphs of the physical topology. According to the definition of a generating subgraph, the number of vertices in the generating subgraph must be the same as the number of vertices in the original graph, and the set of all edges in the original graph must contain the set of all edges in the generating subgraph. Therefore, the physical topology is used as the original graph, and multiple sub-topologies are used as multiple generating subgraphs of the original graph. Any one of the multiple sub-topologies includes every physical node of the physical topology of the cluster network, but does not necessarily include every edge of the physical topology.

[0135] Step 1602: Controller 151 determines at least one feasible subtopic among the multiple subtopics that include all edges of the logical topology of the set communication algorithm.

[0136] After determining the logical topology of the set communication algorithm, the controller 151 determines at least one feasible subtopic whose edges include all edges of the logical topology of the set communication algorithm, based on all edges of the logical topology and all edges of the multiple subtopologies.

[0137] Here, edges are used to represent communication connections between two physical nodes in a sub-topology or physical topology.

[0138] As one possible implementation, when all edges of the feasible subtopology include all edges of the logical topology, it means that the feasible subtopology is isomorphic to the logical topology, and the flow of the feasible subtopology matches the set communication algorithm.

[0139] As one possible implementation, controller 151 can determine the logical topology of the collective communication algorithm based on the communication rules of each logical node communicating with other logical nodes.

[0140] For example, consider the halving and doubling (HD) algorithm as an example of a set communication algorithm. Figure 18 As shown, Figure 18 This is a schematic diagram of the communication rules of an HD algorithm provided in an embodiment of this application. Figure 18 There are a total of 8 logical nodes, and the communication process is divided into three steps.

[0141] In the first step, logical node a0 communicates with logical node a1. Logical node a1 communicates with logical node a0. Logical node a2 communicates with logical node a3. Logical node a3 communicates with logical node a2. Logical node a4 communicates with logical node a5. Logical node a5 communicates with logical node a4. Logical node a6 communicates with logical node a7. Logical node a7 communicates with logical node a6.

[0142] In the second step, logical node a0 communicates with logical node a2. Logical node a1 communicates with logical node a3. Logical node a2 communicates with logical node a0. Logical node a3 communicates with logical node a1. Logical node a4 communicates with logical node a6. Logical node a5 communicates with logical node a7. Logical node a6 communicates with logical node a4. Logical node a7 communicates with logical node a5.

[0143] In the third step, logical node a0 communicates with logical node a4. Logical node a1 communicates with logical node a5. Logical node a2 communicates with logical node a6. Logical node a3 communicates with logical node a7. Logical node a4 communicates with logical node a0. Logical node a5 communicates with logical node a1. Logical node a6 communicates with logical node a2. Logical node a7 communicates with logical node a3.

[0144] Controller 151 according to Figure 18 The logical topology of the HD algorithm is determined by each pair of logical nodes that communicate with each other. Figure 19 This is a schematic diagram of the logical topology of an HD algorithm provided in an embodiment of this application. Figure 19 As shown, Figure 19 The vertices of the cube represent logical nodes in the logical topology, that is, vertex a0 corresponds to... Figure 18 Logical node a0 in the [context]. Vertex a1 corresponds to [the context]. Figure 18 Logical node a1 in the diagram. Vertex a2 corresponds to... Figure 18 Logical node a2 in the diagram. Vertex a3 corresponds to... Figure 18 Logical node a3 in the diagram. Vertex a4 corresponds to... Figure 18 Logical node a4 in the diagram. Vertex a5 corresponds to... Figure 18 Logical node a5 in the diagram. Vertex a6 corresponds to... Figure 18 Logical node a6 in the diagram. Vertex a7 corresponds to... Figure 18 In the cube, the edges represent the connections between two logical nodes on the logical topology corresponding to the two vertices on the edge. That is, the edge a0a1 indicates that logical node a0 and logical node a1 are connected, and logical node a0 and logical node a1 communicate and transmit data. Figure 19 The remaining edges of the cube in the diagram have the same meaning as edges a0 and a1, and will not be elaborated upon here.

[0145] Controller 151 according to Figure 19 The logical topology of the HD algorithm shown includes each edge, as well as all edges of multiple feasible subtopologies. Any subtopology in which all edges of multiple feasible subtopologies contain each edge of the logical topology of the HD algorithm is determined as a feasible subtopology.

[0146] For example, let's take the ring algorithm as an example of a set communication algorithm. Figure 20 As shown, Figure 20 This is a schematic diagram of the communication rules of a ring algorithm provided in an embodiment of this application. Figure 20There are n logical nodes in total, and the communication process consists of n-1 steps. Logical node a0 communicates only with logical nodes a1 and a3 in each step. Logical node a1 communicates only with logical nodes a0 and a2 in each step. Logical node a2 communicates only with logical nodes a1 and a3 in each step. Logical node an-1 communicates only with logical nodes an-2 and a0 in each step.

[0147] Controller 151 according to Figure 20 The logical topology of the ring algorithm is determined by each pair of logical nodes that communicate with each other. Figure 21 This is a schematic diagram of the logical topology of a ring algorithm provided in an embodiment of this application. Figure 21 As shown, Figure 21 Each vertex of the polygon represents a logical node in the logical topology, that is, vertex a0 corresponds to... Figure 20 In the polygon, logical node a0 corresponds to vertex a1. Vertex a2 corresponds to vertex a2. Vertex an-2 corresponds to vertex an-2. Vertex an-1 corresponds to vertex an-1. Each edge of the polygon represents a connection between two logical nodes on the logical topology corresponding to the two vertices on that edge. That is, edge a0a1 represents a connection between logical node a0 and logical node a1, and logical nodes a0 and a1 communicate and transmit data. Figure 21 The remaining sides of the polygon in the diagram have the same meaning as sides a0 and a1, and will not be elaborated upon here.

[0148] Controller 151 according to Figure 21 Given each edge in the logical topology of the ring algorithm and all edges in multiple feasible subtopologies, determine any subtopology in which all edges in multiple feasible subtopologies contain each edge in the logical topology of the ring algorithm as a feasible subtopology.

[0149] The specific method by which this application determines feasible subtopologies can be found in the following reference. Figure 25 Steps 2501-2502 shown in the diagram will not be repeated here.

[0150] Step 1603: Based on the constraint of the number of links in the physical topology, controller 151 determines at least one combination of at least one feasible subtopology that meets the communication bandwidth requirements of the cluster network, wherein any combination of at least one combination includes at least one feasible subtopology.

[0151] The connection between any two physical nodes in a physical topology is achieved through physical links, which can be cables, optical fibers, telephone lines, etc. The number of links between any two physical nodes is subject to constraints; that is, the constraints on the number of links in the physical topology include limitations on the actual number of physical links between all two physical nodes in the physical topology that are in a communication connection. When determining at least one combination of feasible subtopologies based on at least one feasible subtopology, controller 151 needs to make a selection based on the constraints on the number of links in the physical topology.

[0152] Furthermore, when the collective communication algorithm is executed in a cluster network, there is a minimum communication bandwidth requirement. If the communication bandwidth requirement cannot be met, the collective communication algorithm will fail to execute.

[0153] Therefore, controller 151 determines at least one combination of at least one feasible subtopic that simultaneously satisfies the constraints on the number of links in the physical topology and the communication bandwidth requirements of the cluster network. Any combination of at least one combination includes at least one feasible subtopic.

[0154] The specific method by which this application determines at least one combination can be found in the following text. Figures 26-28 The relevant descriptions and the content of the first optimization function will not be repeated here.

[0155] Step 1604: Controller 151 reconstructs the physical topology of the cluster network based on the target combination in at least one combination, and obtains the reconstructed physical topology.

[0156] Controller 151 determines a target combination among at least one combination of at least one feasible sub-topology. Controller 151 reconstructs the physical topology of the cluster network based on the links used by at least one feasible sub-topology in the target combination, thereby obtaining the reconstructed physical topology of the cluster network.

[0157] As one possible implementation, controller 151 reconstructs the physical topology of the cluster network based on the usage links determined by the mapping relationship between the physical and logical topologies of at least one feasible sub-topology in the target combination.

[0158] As one possible implementation, when the cluster network is an optical AI cluster network, the controller 151 reconstructs the physical topology of the optical AI cluster network based on a target combination in at least one combination of physical topologies and the links used by at least one feasible sub-topology in the target combination, thereby obtaining the reconstructed physical topology of the optical AI cluster network.

[0159] Optionally, after reconstructing the physical topology of the optical AI cluster network, the controller 151 can also delete the links that are not used by the target combination in the physical topology of the optical AI cluster network to obtain the reconstructed physical topology of the optical AI cluster network.

[0160] As another possible implementation, when the trunking network is an electric trunking architecture network, taking the controller in the electric trunking architecture network as an example, the controller in the electric trunking architecture network reconstructs the physical topology of the electric trunking architecture network based on the target combination in at least one combination of physical topologies, according to the links used by at least one feasible sub-topology in the target combination, to obtain the reconstructed physical topology of the electric trunking architecture network.

[0161] As one possible implementation, when at least one combination includes only one combination, the controller 151 determines that combination as the target combination.

[0162] As one possible implementation, when multiple combinations are included in at least one combination, the controller 151 can reconstruct the physical topology based on any combination, so the controller 151 first determines the target combination from at least one combination.

[0163] However, the embodiments of this application do not limit the method of determining the target combination in at least one combination.

[0164] For example, controller 151 randomly selects one combination from multiple combinations as the target combination.

[0165] For example, controller 151 uses a greedy algorithm to determine a given combination as the target combination, as described later. Figures 26-28 .

[0166] For example, controller 151 uses linear programming to determine the target combination from multiple combinations, which can be determined based on the number of links or bandwidth of at least one combination. Refer to the second and third optimization functions described later.

[0167] As one possible implementation, controller 151 can determine the target combination in at least one combination based on the physical link attributes corresponding to at least one combination.

[0168] At least one combination corresponds to physical link attributes including at least one of the number of links and bandwidth.

[0169] When physical link attributes include bandwidth, a larger bandwidth and value corresponding to at least one combination indicates a greater number of feasible sub-topologies within that combination. This results in a greater number of communication channels available for the ensemble communication algorithm, leading to higher communication efficiency and performance when executing the ensemble communication algorithm in a cluster network. Therefore, in this embodiment, based on the bandwidth corresponding to at least one combination, the combination with the largest bandwidth and value is determined as the target combination.

[0170] As one possible implementation method, Figure 22 This is a flowchart illustrating a set communication algorithm provided in an embodiment of this application. Figure 22 As shown, the set communication algorithm includes steps 2201 to 2205.

[0171] Step 2201: Controller 151 determines multiple sub-topologies in the physical topology of the cluster network.

[0172] Step 2202: Controller 151 determines at least one feasible subtopic among the multiple subtopics that include all edges of the logical topology of the set communication algorithm.

[0173] Step 2203: Based on the constraint of the number of links in the physical topology, controller 151 determines at least one combination of at least one feasible sub-topology that meets the communication bandwidth requirements of the cluster network.

[0174] Step 2204: Controller 151 determines the combination with the largest bandwidth and value as the target combination based on the bandwidth corresponding to at least one combination.

[0175] Step 2205: Controller 151 reconstructs the physical topology based on the target combination to obtain the reconstructed physical topology.

[0176] When the number of physical links includes the number of links, since fewer links correspond to feasible sub-topologies in at least one combination, fewer resources are required to execute the ensemble communication algorithm in the cluster network, thereby improving the resource utilization and communication performance of the cluster network. Therefore, in this embodiment, based on the number of links corresponding to at least one combination, the combination with the fewest links is determined as the target combination.

[0177] As one possible implementation, the combination with the minimum number of links between every two physical nodes in all feasible sub-topologies of at least one combination is determined as the target combination.

[0178] As one possible implementation method, Figure 23 This is a flowchart illustrating another set communication algorithm provided in an embodiment of this application. Figure 23 As shown, the set communication algorithm includes steps 2301 to 2305.

[0179] Step 2301: Controller 151 determines multiple sub-topologies in the physical topology of the cluster network.

[0180] Step 2302: Controller 151 determines at least one feasible subtopic among the multiple subtopics that include all edges of the logical topology of the set communication algorithm.

[0181] Step 2303: Based on the constraint of the number of links in the physical topology, controller 151 determines at least one combination of at least one feasible sub-topology that meets the communication bandwidth requirements of the cluster network.

[0182] Step 2304: Controller 151 determines the combination with the fewest links as the target combination based on the number of links corresponding to at least one combination.

[0183] Step 2305: Controller 151 reconstructs the physical topology based on the target combination to obtain the reconstructed physical topology.

[0184] In this embodiment, the controller 151 reconstructs the physical topology based on the algorithmic traffic characteristics of the logical topology representation of the collective communication algorithm, determining the reconstructed physical topology using links according to at least one feasible sub-topology in the target combination. The reconstructed physical topology includes all edges of the logical topology of the collective communication algorithm, thus perfectly matching the traffic of the collective communication algorithm. The reconstructed physical topology can provide the collective communication algorithm with the maximum bandwidth to meet its communication requirements, thereby fully utilizing the links provided by the reconstructed physical topology when executing the collective communication algorithm, alleviating link congestion, reducing the time required for collective communication, and improving the resource utilization and communication performance of the cluster network. Alternatively, the reconstructed physical topology can provide the collective communication algorithm with the minimum number of links to meet its communication requirements, saving network resources required to execute the collective communication algorithm, allowing the cluster network to also provide network resources for executing other tasks, thereby improving the overall resource utilization and communication performance of the cluster network.

[0185] against Figure 8 The problem shown can be solved by using the collection communication method provided in the embodiments of this application. Figure 24 This is a schematic diagram of server communication after adopting the aggregated communication method provided in the embodiments of this application. Figure 24As shown, controller 151 maps logical node rank1 to server 1 of electrical switch T1, logical node rank2 to server 5 of electrical switch T3, logical node rank3 to server 4 of electrical switch T2, and logical node rank4 to server 7 of electrical switch T4. Based on the communication order of logical node rank1-rank2-rank3-rank4 within the logical topology of the ring all reduce algorithm, controller 151 controls the servers to communicate in the order of server 1-server 5-server 4-server 7. At this time, server 1 and server 5 can establish a direct link through MEMS OXC1, server 5 and server 4 can establish a direct link through MEMS OXC2, server 4 and server 7 can establish a direct link through MEMS OXC2, and server 7 and server 1 can establish a direct link through MEMS OXC1, thus solving the aforementioned problem. Figure 8 The issues shown aim to improve the communication performance of the cluster network.

[0186] The above text combined Figures 16-24 The collection communication method provided in this application has been described in general. The following section will combine... Figure 25 Tables 1-6 explain the specific methods for determining feasible subtopologies.

[0187] Figure 25 This is a flowchart illustrating a step for determining a feasible subtopology, provided as an embodiment of this application. Figure 25 As shown, this step may include sub-steps 2501 to 2502. The step of determining feasible sub-topologies can be applied to... Figure 14 or Figure 15 The optical AI cluster network shown. Figure 14 The controller 141 in the optical AI cluster network shown, or Figure 15 The controller 151 in the illustrated optical AI cluster network can perform the step of determining a feasible subtopology. The following description uses the controller 151 performing the step of determining a feasible subtopology in this embodiment as an example.

[0188] Step 2501: Controller 151 maps each logical node of the logical topology to a first sub-topology with different arrangement orders. The arrangement order of each physical node in the first sub-topology is different in the first sub-topology with different arrangement orders.

[0189] For each first subtopic with a different arrangement order, controller 151 maps each logical node to a physical node of the first subtopic with that arrangement order, based on each logical node in the logical topology and each physical node of the first subtopic with that arrangement order. The controller then performs the mapping again for the first subtopic with the next arrangement order, until all first subtopics with different arrangements are mapped.

[0190] The physical topology of a clustered network includes a first subtopology among multiple subtopologies. The first subtopology is simply a name for any one of the multiple subtopologies, without any other restrictions.

[0191] As one possible implementation, the controller sorts the logical nodes in the first sub-topology in different ways to obtain the first sub-topology with different arrangement orders.

[0192] For example, controller 151 will Figure 17 When the provided physical topology is used as the first sub-topology, the first sub-topologies with different arrangements include 01234567, 01234576, 01234657, 01234675, 01235467, 01235476, 01235647, etc.

[0193] As one possible implementation, the controller 151 maps the logical nodes sequentially to each physical node of the first sub-topology with different arrangement orders, according to the arrangement order of each logical node in the logical topology.

[0194] For example, when the logical nodes in the logical topology of the HD algorithm are arranged in the order of a0a1a2a3a4a5a6a7, and the physical nodes in the first sub-topology are arranged in the order of 01234567, the controller 151 sequentially maps logical node a0 to physical node 0, logical node a1 to physical node 1, logical node a2 to physical node 2, logical node a3 to physical node 3, logical node a4 to physical node 4, logical node a5 to physical node 5, logical node a6 to physical node 6, and logical node a7 to physical node 7.

[0195] As one possible implementation, controller 151 determines the order of each physical node in the first sub-topology with different arrangement orders, and the order of each logical node in the logical topology. Based on the arrangement order of each logical node in the logical topology and the arrangement order of each physical node in the first sub-topology, controller 151 maps each logical node to a physical node with the same order in the first sub-topology with different arrangement orders.

[0196] Optionally, for each first sub-topology of a given arrangement, the controller 151 determines the order of each physical node in the first sub-topology of that arrangement based on the position of each physical node in the arrangement order.

[0197] For example, when the first sub-topology of any arrangement order is 01235647, the controller 151 determines that the order of physical node 0 is 1, the order of physical node 1 is 2, the order of physical node 2 is 3, the order of physical node 3 is 4, the order of physical node 5 is 5, the order of physical node 6 is 6, the order of physical node 4 is 7, and the order of physical node 7 is 8.

[0198] Optionally, the controller 151 determines the order of each logical node in the logical topology according to any arrangement order of each logical node in the logical topology of the set communication algorithm.

[0199] For example, when the order of each logical node in the logical topology of the HD algorithm is a0a1a2a3a4a5a6a7, the controller 151 determines that the order of logical node a0 is 1, the order of logical node a1 is 2, the order of logical node a2 is 3, the order of logical node a3 is 4, the order of logical node a4 is 5, the order of logical node a5 is 6, the order of logical node a6 is 7, and the order of logical node a7 is 8.

[0200] The arrangement of logical nodes in the logical topology, a0a1a2a3a4a5a6a7, is merely an example and does not constitute a limitation on the arrangement of logical nodes in the logical topology. Understandably, the arrangement of logical nodes in the HD algorithm's logical topology could also be a7a3a2a5a6a0a4a1, in which case logical node a7 is in position 1, logical node a3 is in position 2, logical node a2 is in position 3, logical node a5 is in position 4, logical node a6 is in position 5, logical node a0 is in position 6, logical node a4 is in position 7, and logical node a1 is in position 8.

[0201] Optionally, for each arrangement order of the first sub-topology, the controller 151 determines the logical nodes and physical nodes with the same order based on the arrangement order of each logical node in the logical topology and the arrangement order of each physical node in the first sub-topology of the arrangement order, and maps each logical node to each physical node with the same order in the first sub-topology of the arrangement order.

[0202] For example, when the logical nodes of the logical topology are arranged in the order a7a3a2a5a6a0a4a1, and the physical nodes of the first sub-topology are arranged in the order 01235647, the controller 151 determines the logical node a7 with a sequence bit of 1 and the physical node 0, and maps the logical node a7 to the physical node 0. The controller 151 determines the logical node a3 with a sequence bit of 2 and the physical node 1, and maps the logical node a3 to the physical node 1. The controller 151 determines the logical node a2 with a sequence bit of 3 and the physical node 2, and maps the logical node a2 to the physical node 2. The controller 151 determines the logical node a5 with a sequence bit of 4 and the physical node 3, and maps the logical node a5 to the physical node 3. The controller 151 determines the logical node a6 with a sequence bit of 5 and the physical node 5, and maps the logical node a6 to the physical node 4. The controller 151 determines the logical node a0 with a sequence bit of 6 and the physical node 6, and maps the logical node a0 to the physical node 6. Controller 151 determines logic a4, where both sequence bits are 7, and physical node 4, and maps logic node a4 to physical node 4. Controller 151 determines logic a1, where both sequence bits are 8, and physical node 7, and maps logic node a1 to physical node 7.

[0203] Optionally, the controller 151 generates a node mapping table. The node mapping table includes logical entries and at least one physical entry. Each logical entry includes logical nodes arranged in the order of their arrangement in the logical topology, and each physical entry includes physical nodes of a first sub-topology arranged in a different order. The controller 151 maps each logical node in a logical entry to a physical node with the same position in the at least one physical entry.

[0204] The controller 151 arranges the logical nodes in the logical topology according to their order to obtain logical entries in the node mapping table. For example, the logical entries in the node mapping table include a0, a1, a2, a3, a4, a5, a6, and a7.

[0205] The controller 151 arranges the physical nodes in the first sub-topology according to their arrangement order to obtain at least one physical entry in the node mapping table. For example, at least one physical entry in the node mapping table includes 01234567, 01234576, 01234657, etc. Table 1 is a node mapping table provided in an embodiment of this application.

[0206] Table 1

[0207] Map (a0, a1, a2, a3, a4, a5, a6, a7) Is it valid? 01234567 01234576 01234657 01234675 01235467 01235476 01235647 …… ……

[0208] As shown in Table 1, the logical entries in Table 1 include a0, a1, a2, a3, a4, a5, a6, and a7. At least one physical entry in Table 1 includes 01234567, 01234576, 01234657, 01234675, 01235467, 01235476, 01235647, and so on. The next column in the same row for each physical entry in Table 1 contains whether the entry is valid. This valid entry records whether the first sub-topology corresponding to the mapped logical node of the logical entry is a feasible sub-topology after mapping to the physical node of the physical entry in the same row.

[0209] Specifically, the controller 151 maps each logical node to a physical node with the same position in the physical table entry for any physical table entry in at least one physical table entry, based on the position of each logical node in the logical table entry and the position of each physical node in the physical table entry.

[0210] For example, controller 151 maps logical node a0 in logical table entry to physical node 0 in the same order in the first row of physical table entry in table 1, maps logical node a1 to physical node 1 in the same order in the first row of physical table entry in table 1, maps logical node a2 to physical node 2 in the same order in the first row of physical table entry in table 1, maps logical node a3 to physical node 3 in the same order in the first row of physical table entry in table 1, maps logical node a4 to physical node 4 in the same order in the first row of physical table entry in table 1, maps logical node a5 to physical node 5 in the same order in the first row of physical table entry in table 1, maps logical node a6 to physical node 7 in the same order in the first row of physical table entry in table 1, and maps logical node a7 to physical node 7 in the same order in the first row of physical table entry in table 1.

[0211] Step 2502: Controller 151 determines the first subtopology of any permutation order after mapping all edges of the logical topology as a feasible subtopology.

[0212] For each mapped first subtopic in a given arrangement, controller 151 determines, based on the logical topology and the mapped first subtopic of that arrangement, whether every two logical nodes mapped to each pair of connected physical nodes in the mapped first subtopic also have a connection in the logical topology. If yes, controller 151 determines that the mapped first subtopic of that arrangement is a feasible subtopic. Otherwise, controller 151 checks the first subtopic of the next arrangement until it has checked the first subtopic of each arrangement and determined all feasible subtopics.

[0213] As one possible implementation, at least one physical entry includes a first physical entry. Controller 151 determines an adjacency matrix of the logical topology, which represents the connection relationships between logical nodes in the logical topology. Controller 151 determines a homogeneity detection table based on the adjacency matrix. The homogeneity detection table includes at least one detection entry, and any one of the at least one detection entry includes a logical node to be detected and logical nodes connected to the logical node to be detected. If the first sub-topology mapped to the first physical entry includes all connection relationships represented by all detection entries, controller 151 determines the first sub-topology mapped to the first physical entry as a feasible sub-topology.

[0214] Optionally, controller 151 determines the two logical nodes connected by each edge in the logical topology. Based on each pair of connected logical nodes, controller 151 performs a standard representation of the logical topology and determines an adjacency matrix, which represents the connection relationships between the logical nodes in the logical topology.

[0215] For example, consider the logical topology of the HD algorithm. Controller 151 will... Figure 19 The logical topology of the HD algorithm shown is represented in a standard way to obtain the adjacency matrix of the logical topology of the HD algorithm. Table 2 is an adjacency matrix of the logical topology of an HD algorithm provided in an embodiment of this application.

[0216] Table 2

[0217] a0 a1 a2 a3 a4 a5 a6 a7 a0 1 1 1 a1 1 1 1 a2 1 1 1 a3 1 1 1 a4 1 1 1 a5 1 1 1 a6 1 1 1 a7 1 1 1

[0218] As shown in Table 2, each 1 in Table 2 indicates that the logical node in its row is connected to the logical node in its column. For example, the 1 in the 3rd row and 2nd column of the adjacency matrix in Table 2 indicates that the logical node a1 in the 3rd row is connected to the logical node a0 in the 2nd column.

[0219] Taking the logical topology of the ring algorithm as an example, controller 151 will... Figure 22 The logical topology of the ring algorithm shown is represented in a standard way to obtain the adjacency matrix of the logical topology of the ring algorithm. Table 3 is an adjacency matrix of the logical topology of the ring algorithm provided in an embodiment of this application.

[0220] Table 3

[0221] a0 a1 a2 a3 … an-3 an-2 an-1 a0 1 1 a1 1 1 a2 1 1 a3 1 … an-3 1 an-2 1 1 an-1 1 1

[0222] As shown in Table 3, each 1 in Table 3 indicates that the logical node in its row is connected to the logical node in its column. For example, the 1 in the 3rd row and 4th column of the adjacency matrix in Table 3 indicates that the logical node a1 in the 3rd row is connected to the logical node a2 in the 4th column.

[0223] Optionally, due to the diagonal symmetry of the adjacency matrix, the connection relationships between logical nodes in the adjacency matrix will be repeatedly displayed. Therefore, the controller 151 can determine the isomorphism detection table of the logical topology based on the adjacency matrix of the logical topology.

[0224] Optionally, the controller 151 uses the lower left half of the diagonal of the adjacency matrix as the logical node to be tested for each detection entry in the isomorphism detection table. The controller 151 determines the logical nodes in each detection entry that are connected to each logical node to be tested, based on the lower left half of the diagonal of the adjacency matrix. Each first detection entry only includes the connection relationship between the logical node to be tested and the logical nodes preceding it in the logical table entry.

[0225] For example, the adjacency matrix of the logical topology of the HD algorithm is used as an example. The logical nodes in the lower left half of the diagonal of the adjacency matrix of the HD algorithm include logical node a0, logical node a1, logical node a2, logical node a3, logical node a4, logical node a5, logical node a6, and logical node a7. The controller 151 uses the aforementioned logical nodes as the logical nodes to be detected for each detection entry in the isomorphism detection table. Based on the lower left half of the adjacency matrix, controller 151 determines the logical nodes that are not connected to the logical node a0 to be detected, the logical nodes a0 connected to the logical node a1 to be detected, the logical nodes a0 connected to the logical node a2 to be detected, the logical nodes a1 and a2 connected to the logical node a3 to be detected, the logical nodes a0 connected to the logical node a4 to be detected, the logical nodes a1 and a4 connected to the logical node a5 to be detected, the logical nodes a2 and a4 connected to the logical node a6 to be detected, and the logical nodes a3, a5, and a6 connected to the logical node a7 to be detected. Table 4 is an isomorphism detection table of an HD algorithm provided in an embodiment of this application.

[0226] Table 4

[0227] Logical node to be detected Logical nodes to be verified a0 a1 a0 a2 a0 a3 a1,a2 a4 a0 a5 a1,a4 a6 a2,a4 a7 a3, a5, a6

[0228] As shown in Table 4, the isomorphism detection table includes 8 detection entries. The first row of detection entries only includes the logical node a0 to be detected.

[0229] The second row of the detection table includes the logical node to be detected, a1, and the logical node a0 that is connected to the logical node to be detected, a1.

[0230] The third row of the test table includes the logical node a2 to be tested and the logical node a0 that is connected to the logical node a2 to be tested.

[0231] The fourth row of the detection table includes the logical node to be detected, a3, and logical nodes a1 and a2 that are connected to the logical node to be detected, a3.

[0232] The fifth row of the detection table includes the logical node to be detected, a4, and the logical node a0 that is connected to the logical node to be detected, a4.

[0233] The sixth row of the test table includes the logical node to be tested, a5, and logical nodes a1 and a4 that are connected to the logical node to be tested, respectively.

[0234] The seventh row of the test table includes the logical node to be tested, a6, and logical nodes a2 and a4 that are connected to the logical node to be tested, respectively.

[0235] The eighth row of the test table includes the logical node to be tested a7, as well as logical nodes a3, a5 and a6 that are connected to the logical node to be tested a7.

[0236] Taking the adjacency matrix of the logical topology of the Ring algorithm as an example, the logical nodes in the lower left diagonal of the adjacency matrix of the Ring algorithm include logical nodes a0, a1, a2, a3, ..., an-3, an-2, and an-1. The controller 151 uses the aforementioned logical nodes as the logical nodes to be tested in each detection entry of the isomorphism detection table. Based on the lower left diagonal of the adjacency matrix of the Ring algorithm, the controller 151 determines the logical nodes that the logical node a0 to be tested has no connection relationship with, the logical nodes a0 that are connected to the logical node a1 to be tested, the logical nodes a1 that are connected to the logical node a2 to be tested, the logical nodes a2 that are connected to the logical node a3 to be tested, ..., the logical nodes an-4 that are connected to the logical node an-3 to be tested, the logical nodes an-3 that are connected to the logical node an-2 to be tested, and the logical nodes an-2 and a0 that are connected to the logical node an-1 to be tested. Table 5 is a homogeneity detection table for a ring algorithm provided in an embodiment of this application.

[0237] Table 5

[0238] Logical node to be detected Logical nodes to be verified a0 a1 a0 a2 a1 a3 a2 … … an-3 an-4 an-2 an-3 an-1 an-2,a0

[0239] As shown in Table 5, the isomorphism detection table includes n rows of detection entries. The first row of detection entries only includes the logical node a0 to be detected.

[0240] The second row of the detection table includes the logical node to be detected, a1, and the logical node a0 that is connected to the logical node to be detected, a1.

[0241] The third row of the test table includes the logical node a2 to be tested and the logical node a1 that is connected to the logical node a2 to be tested.

[0242] The fourth row of the detection table includes the logical node to be detected, a3, and the logical node a2 that is connected to the logical node to be detected, a3.

[0243] The (n-2)th row of the detection table includes the logical node to be detected, an-3, and the logical node an-4 that is connected to the logical node to be detected, an-3.

[0244] The (n-1)th row of the detection table includes the logical node to be detected, an-2, and the logical node an-3 that is connected to the logical node to be detected, an-2.

[0245] The nth row of the detection table includes the logical node to be detected, an-1, and logical nodes an-2 and a0 that are connected to the logical node to be detected, an-1.

[0246] Optionally, in this embodiment, the controller 151 refers to any physical entry in the node mapping table as the first physical entry. The first physical entry is only for the convenience of describing the naming of any physical entry and has no other limitations.

[0247] Optionally, after determining the first sub-topology after mapping corresponding to the first physical table entry, the controller 151 judges the connection relationship represented by each detection table entry in the isomorphism detection table of the logical topology based on the connection relationship of each logical node mapped by each physical node in the first sub-topology after mapping corresponding to the first physical table entry. If the controller 151 determines that it includes all connection relationships represented by all detection table entries in the isomorphism detection table of the logical topology, then the first sub-topology after mapping corresponding to the first physical table entry is determined to be a feasible sub-topology.

[0248] For example, the isomorphism detection table of the logical topology of the HD algorithm shown in Table 4, the node mapping table shown in Table 1, and... Figure 17 The first topology shown is an example.

[0249] Controller 151 maps logical node a0 to the node mapping table shown in Table 1 based on the logical table entries and the first row of physical table entries. Figure 17On the physical node 0 shown, map logical node a1 to Figure 17 On the physical node 1 shown, the logical node a2 is mapped to Figure 17 On the physical node 2 shown, map the logical node a3 to Figure 17 On the physical node 3 shown, map the logical node a4 to Figure 17 On the physical node 4 shown, map the logical node a5 to Figure 17 On the physical node 5 shown, the logical node a6 is mapped to... Figure 17 On the physical node 6 shown, map the logical node a7 to Figure 17 On physical node 7 shown.

[0250] As shown in the second row of the isomorphism detection table in Table 4, logical node a1 and logical node a0 are connected. Based on the first sub-topology mapped from the first row of physical entries, controller 151 can determine that physical node 1 mapped to logical node a1 and physical node 0 mapped to logical node a0 in the first sub-topology are also connected.

[0251] As shown in the third row of the isomorphism detection table in Table 4, logical node a2 and logical node a0 are connected. Based on the first sub-topology after mapping corresponding to the first row of physical table entries, controller 151 can determine that physical node 2 mapped by logical node a2 and physical node 0 mapped by logical node a0 in the first sub-topology after mapping are also connected.

[0252] As shown in the fourth row of the isomorphism detection table in Table 4, logical node a3 is connected to logical nodes a1 and a2. According to the first sub-topology mapped to the first row of physical entries, controller 151 can determine that the physical node 3 mapped to logical node a3 and the physical node 1 mapped to logical node a1 in the first sub-topology are not connected. Therefore, it can be determined that the first sub-topology mapped to the first row of physical entries is not a feasible sub-topology, and the validity entry in the first row is false.

[0253] The isomorphism detection table does not include duplicate connections between logical nodes, thus avoiding the inefficiency in determining feasible subtopologies caused by repeated detections. When the first subtopology mapped to the first physical table entry includes all connections represented by all detection entries, the controller 151 determines the first subtopology mapped to the first physical table entry as a feasible subtopology. The controller 151 quickly and accurately determines feasible subtopologies by detecting the first subtopology mapped to the first physical table entry.

[0254] As one possible implementation, in this embodiment, the controller 151 can also determine the first subtopic with the same adjacency matrix as the logical topology as any of the adjacency matrices of the first subtopics with different adjacency matrices, based on the adjacency matrix of the logical topology and the adjacency matrices of each first subtopic with different adjacency matrices.

[0255] As one possible implementation, at least one detection entry includes a first detection entry. If the first sub-topology after mapping corresponding to the first physical entry does not include the connection relationship of the logical node to be detected in the first detection entry, the controller 151 deletes at least one physical entry containing a non-feasible prefix. The non-feasible prefix is ​​composed of the physical node mapped by the logical node to be detected in the first physical entry to the first physical node of the first physical entry.

[0256] Optionally, in the embodiments of this application, any one of the at least one test entries in the isomorphism test table is referred to as the first test entry. The first test entry is only for the convenience of describing the naming of any test entry and has no other limitations.

[0257] Optionally, if the first subtopology after mapping corresponding to the first physical entry does not include the connection relationship of the logical node to be detected in the first detection entry, the controller 151 determines that the first subtopology after mapping corresponding to the first physical entry is not a feasible subtopology.

[0258] Among them, the logical node to be detected in the first detection table entry is the first logical node to be detected in the first sub-topology after the mapping corresponding to the first physical table entry, whose connection relationship is not included in the first physical table entry.

[0259] Optionally, the controller 151 determines the infeasible prefix based on the physical node mapped to the logical node to be detected in the first detection entry in the first physical entry, and the first physical node of the first physical entry. The controller 151 deletes at least one physical entry containing an infeasible prefix.

[0260] For example, based on the aforementioned isomorphism detection table of the logical topology of the HD algorithm shown in Table 4, the node mapping table shown in Table 1, and... Figure 17 In the example of the first sub-topology shown, if the first sub-topology after mapping the first row physical table entry 01234567 does not include the connection relationship of the logical node a3 to be detected in the fourth row detection table entry, the controller 151 determines the fourth row detection table entry as the first detection table entry. Based on the physical node 3 mapped to the logical node a3 to be detected in the first row physical table entry and the first physical node 0 of the first row physical table entry, the controller 151 determines the infeasible prefix as 0123. The controller 151 deletes the infeasible prefix 0123 from the node mapping table shown in Table 2.

[0261] The mapping relationship between physical nodes and their corresponding logical nodes in the infeasible prefix is ​​fixed. Therefore, the connection relationship between any two physical nodes from the first physical node to the last physical node in the infeasible prefix is ​​fixed.

[0262] When controller 151 performs detection based on the first detection entry, it detects the fixed connection relationship between the logical node to be detected in the first detection entry and the logical nodes preceding the logical node in the logical entry. Therefore, the determined connection relationship between the last physical node in the infeasible prefix of the physical entry and the previous physical nodes does not include the connection relationship between the logical node to be detected in the first detection entry mapped from the last physical node. The controller determines that the first sub-topology after mapping corresponding to physical entries containing infeasible prefixes is not a feasible sub-topology. Therefore, by deleting at least one physical entry containing an infeasible prefix, pruning optimization of physical entries can be achieved, further improving the efficiency of determining feasible sub-topologies.

[0263] As one possible implementation, controller 151 searches for the next feasible prefix in at least one physical entry.

[0264] Optionally, the last physical node is the physical node mapped in the first physical table entry of the first detection table entry that has no connection relationship to the logical node to be detected. Therefore, the controller 151 performs pruning optimization on the physical table entries in the node mapping table. That is, the controller 151 determines at least one prefix when the last physical node in the infeasible prefix is ​​in turn other physical nodes, and makes a judgment based on the first sub-topology after mapping any of the at least one prefix and the isomorphism detection table.

[0265] For example, Table 6 is a node mapping table after deleting infeasible prefixes provided in an embodiment of this application.

[0266] Table 6

[0267] Map (a0, a1, a2, a3, a4, a5, a6, a7) Is it valid? 0123 false 0124 false 0125 false 0126 false 0127 false 0132456 false …… ……

[0268] As shown in the second row of the physical table entries in Table 6, controller 151 maps logical node a3 to... Figure 17 On physical node 4 of the first sub-topology shown. According to the isomorphism detection table shown in Table 4, controller 151 knows that logical node a3 is connected to logical nodes a1 and a2. In the mapped first sub-topology, physical node 4, mapped from logical node a3, is not connected to physical node 1, mapped from logical node a1. Therefore, prefix 0124 is determined to be an infeasible prefix. Understandably, prefixes 0125, 0126, 0127, and 0132456 are also infeasible prefixes.

[0269] In the collection communication method provided in this application, after the controller determines a feasible sub-topology from the logical topology, it then determines at least one combination from at least one feasible sub-topology. The following is in conjunction with... Figures 26-28 The specific method for determining at least one combination shall be explained.

[0270] The controller can use a greedy algorithm to determine a combination from at least one feasible subtopology.

[0271] In this embodiment, at least one combination includes a first combination, which is determined by the controller using depth-first search (DFS). The controller determines whether the number of links in the first combination meets the constraints of the number of links in the physical topology, and whether the total bandwidth of the first combination meets the communication bandwidth requirements of the cluster network; if so, the controller determines that at least one combination is the first combination.

[0272] The controller uses Depth-First Search (DFS) to enumerate multiple combinations of at least one feasible sub-topology, and any combination among these is referred to as the first combination. The term "first combination" is used merely for ease of description and has no other limitations. The controller determines whether the number of links in the first combination satisfies the physical topology's link count constraint, based on the number of links in the first combination and the physical topology's link count constraint. The controller also determines whether the total bandwidth of the first combination meets the cluster network's bandwidth requirements, based on the total bandwidth of the first combination and the cluster network's communication bandwidth requirements. If both conditions are met, the controller determines at least one combination as the first combination. Once a first combination is determined, the controller stops determining the next combination; it does not need to iterate through and determine all first combinations.

[0273] As one possible implementation, the constraint on the number of links in the physical topology includes the limitation on the actual number of physical links between all two physical nodes in the physical topology that are in communication connection.

[0274] The controller determines the sum of the number of links between two physical nodes in any feasible subtopic of the first combination and the number of links between two physical nodes with the same node number as other feasible subtopics in the first combination, based on the number of links between all two physical nodes in each feasible subtopic with communication connections in the first combination. The controller then determines whether the sum of the number of links between two physical nodes with the same node number in each feasible subtopic of the first combination satisfies the constraint on the actual number of physical links between two physical nodes with the same node number in the physical topology. If so, the controller checks the number of links between the next two physical nodes in communication connections until the number of links between all two physical nodes in communication connections satisfies the constraint on the actual number of physical links between the corresponding two physical nodes in the physical topology. Finally, the controller determines the first combination based on the total bandwidth of the first combination and the communication bandwidth requirements of the cluster network. If the total bandwidth of the first combination satisfies the communication bandwidth requirements of the cluster network, then at least one combination is designated as the first combination.

[0275] The greedy algorithm will be illustrated with an example below. Figure 26 This is a schematic diagram of a physical topology provided in an embodiment of this application. For example... Figure 26 As shown, the physical topology includes 5 physical nodes. Physical node 0 is connected to physical nodes 1, 2, 3, and 4; physical node 1 is connected to physical nodes 0, 2, 3, and 4; physical node 2 is connected to physical nodes 0, 1, 3, and 4; physical node 3 is connected to physical nodes 0, 1, 2, and 4; and physical node 4 is connected to physical nodes 0, 1, 2, and 3.

[0276] exist Figure 26 In the physical topology shown, the communication domain of the data parallelism (DP) model overlaps with the communication domain of the mixture of experts (MoE) model. The connection between the five physical nodes is a wireless mesh network connection, so the actual number of physical links between any two physical nodes is 1.

[0277] exist Figure 25 When the dynamic programming (DP) algorithm in the physical topology shown is applied using the ring all reduce algorithm, at least one feasible subtopology is determined. The specific method for determining feasible subtopologies is described above. Figure 25 The operational steps of steps 2601 to 2602 are not described in detail here in the embodiments of this application. Table 7 is a schematic table of a feasible subtopology provided in the embodiments of this application.

[0278] Table 7

[0279]

[0280] As shown in Table 7, feasible subtopologies include the mapped subtopologies corresponding to physical table entry 01234, physical table entry 01243, and physical table entry 03142.

[0281] Figure 27 This is a schematic diagram of the mapped sub-topology corresponding to physical entry 01234. For example... Figure 27 As shown, logical node a0 is mapped to physical node 0, logical node a1 is mapped to physical node 1, logical node a2 is mapped to physical node 2, logical node a3 is mapped to physical node 3, and logical node a4 is mapped to physical node 3. Physical node 0 connects to physical node 1 and physical node 4, physical node 1 connects to physical node 0 and physical node 2, physical node 2 connects to physical node 1 and physical node 3, physical node 3 connects to physical node 2 and physical node 4, and physical node 4 connects to physical node 3 and physical node 0.

[0282] Figure 28 This is a schematic diagram of the mapped sub-topology corresponding to physical entry 03142. For example... Figure 28 As shown, logical node a0 is mapped to physical node 0, logical node a1 is mapped to physical node 1, logical node a2 is mapped to physical node 2, logical node a3 is mapped to physical node 3, and logical node a4 is mapped to physical node 3. Physical node 0 connects to physical nodes 2 and 3, physical node 1 connects to physical nodes 3 and 4, physical node 2 connects to physical nodes 0 and 3, physical node 3 connects to physical nodes 0 and 1, and physical node 4 connects to physical nodes 1 and 2.

[0283] Since the ring all reduce algorithm is a one-way communication algorithm, there are two different one-way rings: logical node a0->logical node a1->logical node a2->logical node a3->logical node a4 and logical node a4->logical node a3->logical node a2->logical node a1->logical node a0. To simplify the solution, each feasible subtopology in Table 7 represents two one-way rings. Figure 27 The feasible subtopology in the diagram represents bidirectional links, including two different unidirectional rings: physical node 0->physical node 1->physical node 2->physical node 3->physical node 4 and physical node 4->physical node 3->physical node 2->physical node 1->physical node 0.

[0284] Taking a cluster network communication requirement of bandwidth 2 as an example, based on the determined bandwidth of each feasible sub-topology, the number of links between two physical nodes in each sub-topology that are in communication connection, the actual number of physical links between all two physical nodes in the physical topology that are in communication connection, and the cluster network communication requirement bandwidth 2, the first combination is determined to include... Figure 27 and Figure 28 The feasible subtopology is shown.

[0285] Figure 27 and Figure 28 The feasible sub-topologies shown all involve two distinct unidirectional rings, resulting in four rings in the first combination, allowing four data streams to flow simultaneously. Assuming each physical node has eight GPU cards, and each physical node sends eight data streams to other physical nodes at any given time, there are four rings, each with two data streams. However, using only... Figure 27 When two unidirectional rings are shown, there are four data streams on each ring. The bandwidth is halved compared to the bandwidth of the cluster network using this application. Furthermore, when the data volume is large, backpressure will be triggered, and the communication time will become more than twice that of the cluster network using this application.

[0286] As another possible implementation, the controller can also use linear programming to determine all possible combinations of at least one feasible subtopology.

[0287] The number of times each feasible subtopology is selected is X = (X1, X2, X3…X…). n ), where n is the number of feasible subtopologies.

[0288] Link usage matrix n represents the number of feasible subtopologies, l represents the total number of links in the physical topology, and each element in the matrix represents the number of times that link is used in the current feasible subtopology; for example, element L 1l This indicates the number of times the l-th link in the first feasible subtopology is used.

[0289] Bandwidth vector B = (B1, B2, B3…B) n ), where n represents the number of feasible subtopologies, and each element in the vector represents the available bandwidth of each feasible subtopology. When there is 1 link, the bandwidth can be normalized to 1.

[0290] Available link vector C = (C1, C2, C3…C n ), where l represents the total number of links in the physical topology, and each element in the vector represents the total number of links in each physical topology.

[0291] Optionally, the controller sets the communication bandwidth requirement of the cluster network to N, and uses a first optimization function u=XB≥Ns.t XL≤C to solve for all combinations of at least one feasible sub-topology. The first optimization function means that the bandwidth of the combination is not less than N, and XL≤C means that the number of times each link of each feasible sub-topology is used does not exceed the total number of the same link.

[0292] The above text combined Figures 26-28 The specific method for determining at least one combination provided in this application is described. After the controller determines at least one combination of at least one feasible sub-topology, it then determines the target combination from the at least one combination. The specific method for determining the target combination is explained below.

[0293] The above Figures 26-28 The document describes a combination of at least one feasible subtopology determined using a greedy algorithm, and the controller can use one of the combinations determined by the greedy algorithm as the target combination.

[0294] The above describes the use of linear programming to determine all possible combinations of at least one feasible subtopology. As another possible implementation, the controller can also use linear programming to determine the optimal combination from at least one combination of at least one feasible subtopology.

[0295] Optionally, the controller determines the combination with the maximum bandwidth and maximum bandwidth by setting constraints that ensure the combined bandwidth and the maximum optimization objective, as well as that the number of links in the combination does not exceed the actual number of physical links in the physical topology.

[0296] Combining the number of times each feasible sub-topology is selected (X), the link usage matrix (L), the bandwidth vector (B), and the available link vector (C) described above, the controller uses a second optimization function, maxu = XB st XL≤C, to determine the optimal combination among at least one combination of feasible sub-topologies. This second optimization function represents the maximum sum of bandwidth in the combination, XL≤C indicates that the number of times each link in each feasible sub-topology is used does not exceed the total number of links of the same type, and XB represents the sum of bandwidth in the combination.

[0297] Optionally, the controller determines the combination with the fewest links that satisfies the constraints by setting an optimization objective that minimizes the number of links in the combination and a constraint that the number of links in the combination does not exceed the actual number of physical links in the physical topology.

[0298] Combining the number of times each feasible subtopic is selected (X), the link usage matrix (L), and the available link vector (C) described above, the controller uses a third optimization function, minv = XLst, where XL ≤ C, to determine the optimal combination among at least one combination of feasible subtopics. This third optimization function represents the combination with the minimum number of links, and XL ≤ C means that the number of times each link in each feasible subtopic is used does not exceed the total number of times the same link is used.

[0299] The above text combined Figures 26-29 The application describes the process of determining at least one combination of at least one feasible subtopic and the process of determining the target combination, as provided by the optimization function. However, the above is a scheme implemented when there are many feasible subtopics. The above scheme cannot be used when a sufficient number of feasible subtopics cannot be determined, and it is necessary to determine a larger number of feasible subtopics first.

[0300] For example, in Figure 15 In the optical AI cluster network shown, optical switches are deployed between cluster servers, and each cluster server contains multiple GPUs. Since the bandwidth within each cluster server is relatively ample, the cluster network typically utilizes the bandwidth within the cluster server first when executing aggregated communication algorithms, and then considers cross-cluster communication. However, when cross-cluster communication is needed and the number of links between cluster servers is small, and the controller cannot find a sufficient number of feasible sub-topologies after generating the node mapping table, the controller can adopt the detour mode described in this embodiment.

[0301] The following is combined Figures 29 to 32 The detour mode for determining feasible subtopologies in the embodiments of this application will be described.

[0302] The physical topology includes multiple processors in its physical nodes. The physical topology includes a first physical node and a second physical node. The number of links between the first physical node and the second physical node is less than a preset number. At least one physical entry includes a second physical entry. If the first physical node and the second physical node are adjacent in the second physical entry, the controller adds at least one physical node to the second physical entry to obtain at least one third physical entry. The number of the same physical node in any third physical entry of at least one third physical entry is not greater than the number of processors included in the same physical node, and the same physical nodes are not contiguous.

[0303] As one possible implementation, the physical nodes of the physical topology include multiple processors, and the physical topology includes a first physical node and a second physical node. The first physical node and the second physical node are simply different names for two physical nodes among the multiple physical nodes in the physical topology, without any other limitations.

[0304] Optionally, the number of links between the first physical node and the second physical node is less than a preset number. The preset number can be a number pre-set in this embodiment of the application, or it can be the minimum number of other links in the physical topology besides the link between the first physical node and the second physical node. This embodiment of the application does not limit this.

[0305] As one possible implementation, in this embodiment of the application, after generating the node mapping table, the controller refers to any physical entry in the node mapping table as a second physical entry. The term "second physical entry" is merely for ease of description and does not imply any other limitations. In the case where the first physical node and the second physical node are adjacent in the second physical entry, the number of links between the first physical node and the second physical node is less than a preset number. It is understood that the link between the first physical node and the second physical node is prone to congestion, therefore a detour is performed between the first physical node and the second physical node.

[0306] Optionally, the controller adds at least one physical node to the second physical entry; the at least one physical node can be at least one of the first physical node, the second physical node, or other physical nodes. Based on the physical entry after adding at least one physical node, the controller determines whether the number of identical physical nodes is not greater than the number of processors included in the same physical node, and whether the same physical nodes are not contiguous; if so, it is determined as the third physical entry.

[0307] The controller can be an additional physical node between the first and second physical nodes of the second physical entry, or it can be at least one physical node before or after the first and second physical nodes of the second physical entry, or it can be at least one physical node estimated between any two physical nodes of the second physical entry.

[0308] The following specific embodiment illustrates the detour mode for determining feasible subtopologies in this application.

[0309] Figure 29 This is a schematic diagram of a physical topology provided in an embodiment of this application. For example... Figure 29 As shown, Figure 29 It includes 3 physical nodes, each representing a POD. For example, physical node 0 represents POD0, physical node 1 represents POD1, and physical node 2 represents POD2. Each POD contains two cards. Figure 29In this context, 2r represents two cards within a POD. Network resources are limited between POD1 and POD2, meaning only one link can be established between them. However, link resources are ample between POD0 and POD1, and between POD0 and POD2; three links can be established between POD0 and POD1, and between POD0 and POD2. For example, if the ring algorithm is used for the aggregated communication, the link between POD1 and POD2 becomes the bottleneck link, resulting in only 1 unit of bandwidth usable in the physical topology, and wasted link resources between POD0 and POD1, and between POD0 and POD2.

[0310] Therefore, in this embodiment, the controller utilizes link searching between POD0 and POD1, and between POD0 and POD2, to determine other feasible channels. Specifically, the controller determines other feasible channels based on... Figure 29 The physical topology and the logical topology of the collective communication algorithm are shown to generate a node mapping table. For the second physical entry in the node mapping table, where physical node 1 and physical node 2 are adjacent, the controller adds at least one physical node between physical node 1 and physical node 2 to obtain at least one third physical entry. The number of identical physical nodes in the third physical entry is no greater than two, and identical physical nodes are not contiguous. Table 8 is a node mapping table for a bypass mode provided in an embodiment of this application.

[0311] Table 8

[0312] map(a0, a1, a2, a3, a4, a5, a6, a7) Valid Bandwidth (bw) 01012 true 1 0102 true 1 01021 true 1 012 true 1 0201 true 1 02012 true 1 02021 true 1 …… …… ……

[0313] As shown in Table 8, the controller adds physical nodes 0 and 1 between physical nodes 1 and 2 of the second physical entry 012, resulting in the third physical entry 01012. The bandwidth of the feasible sub-topology corresponding to the third physical entry 01012 is 1. The controller adds physical node 0 between physical nodes 1 and 2 of the second physical entry 012, resulting in the third physical entry 0102. The bandwidth of the feasible sub-topology corresponding to the third physical entry 0102 is 1. The controller adds physical node 0 between physical nodes 1 and 2 of the second physical entry 012, and adds physical node 1 after physical node 2, resulting in the third physical entry 01021. The bandwidth of the feasible sub-topology corresponding to the third physical entry 01021 is 1.

[0314] The controller adds physical node 0 between physical nodes 2 and 1 of the second physical table entry 021, resulting in the third physical table entry 0201. The bandwidth of the feasible subtopology corresponding to the third physical table entry 0201 is 1. The controller adds physical node 0 between physical nodes 2 and 1 of the second physical table entry 021, and then adds physical node 2 after physical node 1, resulting in the third physical table entry 02012. The bandwidth of the feasible subtopology corresponding to the third physical table entry 02012 is 1. The controller directly adds physical nodes 0 and 2 between physical nodes 2 and 1 of the second physical table entry 021, resulting in the third physical table entry 02021. The bandwidth of the feasible subtopology corresponding to the third physical table entry 02021 is 1.

[0315] In the embodiments of this application, POD0 represented by physical node 0 contains card 0 and card 1, POD1 represented by physical node 0 contains card 2 and card 3, and POD2 represented by physical node 0 contains card 4 and card 5.

[0316] Figure 30 This is a schematic diagram of a feasible sub-topology mapped to physical entry 01012 provided in an embodiment of this application. For example... Figure 30 As shown, the ring sequence corresponding to physical entry 01012 is Card 0 (POD0) - Card 2 (POD1) - Card 1 (POD0) - Card 3 (POD1) - Card 4 (POD2) - Card 5 (POD2).

[0317] Figure 31 This is a schematic diagram of a feasible sub-topology mapped to physical entry 0102 provided in an embodiment of this application. For example... Figure 31 As shown, the ring sequence corresponding to physical entry 0102 is Card 0 (POD0) - Card 2 (POD1) - Card 3 (POD1) - Card 1 (POD0) - Card 5 (POD2) - Card 4 (POD2).

[0318] Figure 32 This is a schematic diagram of a feasible subtopology mapped to physical entry 012, provided in an embodiment of this application. For example... Figure 32 As shown, the ring sequence corresponding to physical entry 012 is card 0 (POD0) - card 1 (POD0) - card 2 (POD1) - card 3 (POD1) - card 4 (POD2) - card 5 (POD2).

[0319] Based on the node mapping table shown in Table 8, at least one feasible subtopology of the physical topology is determined. The specific method for determining the feasible subtopology is as described above. Figure 25 The operational steps of steps 2601 to 2602 are not described in detail here in the embodiments of this application. Table 9 is a schematic table of a feasible sub-topology determination combination provided in the embodiments of this application.

[0320] Table 9

[0321] map(a0, a1, a2, a3, a4, a5, a6, a7) L0：01 L1：12 L2：02 bandwidth map1: 01012 3 1 1 1 map2: 0102 2 0 2 1 map3: 01021 3 1 1 1 map4: 012 1 1 1 1 map5: 0201 2 0 2 1 map6: 02012 1 1 3 1 map7: 02021 1 1 3 1 Physical actual number of links 3 1 3

[0322] As shown in Table 9, in the feasible subtopology after mapping corresponding to map1, the link between physical node 0 and physical node 1 is used 3 times, the link between physical node 1 and physical node 2 is used 1 time, the link between physical node 0 and physical node 2 is used 1 time, and the bandwidth of the feasible subtopology corresponding to map1 is 1.

[0323] In the feasible subtopology mapped to map2, the link between physical node 0 and physical node 1 is used 2 times, the link between physical node 1 and physical node 2 is used 0 times, and the bandwidth of the feasible subtopology corresponding to map1 is 1.

[0324] In the feasible subtopology mapped to map3, the link between physical node 0 and physical node 1 is used 3 times, the link between physical node 1 and physical node 2 is used 1 time, the link between physical node 0 and physical node 2 is used 1 time, and the bandwidth of the feasible subtopology corresponding to map1 is 1.

[0325] In the feasible subtopology mapped to map4, the link between physical node 0 and physical node 1 is used 1 time, the link between physical node 1 and physical node 2 is used 1 time, the link between physical node 0 and physical node 2 is used 1 time, and the bandwidth of the feasible subtopology corresponding to map1 is 1.

[0326] In the feasible subtopology mapped to mMap5, the link between physical node 0 and physical node 1 is used 2 times, the link between physical node 1 and physical node 2 is used 0 times, the link between physical node 0 and physical node 2 is used 2 times, and the bandwidth of the feasible subtopology corresponding to map1 is 1.

[0327] In the feasible subtopology mapped to map6, the link between physical node 0 and physical node 1 is used 1 time, the link between physical node 1 and physical node 2 is used 1 time, the link between physical node 0 and physical node 2 is used 3 times, and the bandwidth of the feasible subtopology corresponding to map1 is 1.

[0328] In the feasible subtopology mapped to map7, the link between physical node 0 and physical node 1 is used 1 time, the link between physical node 1 and physical node 2 is used 1 time, the link between physical node 0 and physical node 2 is used 3 times, and the bandwidth of the feasible subtopology corresponding to map1 is 1.

[0329] Figure 29 In the physical topology shown, the number of physical links between physical node 0 and physical node 1 is 3, the number of physical links between physical node 1 and physical node 2 is 1, and the number of physical links between physical node 0 and physical node 2 is 3.

[0330] If the communication bandwidth requirement of the cluster network is 2, the controller determines at least one combination of the constraints on the number of links satisfying the physical topology and the communication bandwidth requirement. The target combination with the largest bandwidth value among these at least one combinations includes... Figure 31 The feasible subtopologies shown are, and Figure 30 The feasible subtopology is shown.

[0331] By adopting a detour mode in this embodiment, the link resources between cluster servers can be fully utilized. Compared with the scheme of abandoning the current task scheduling and deploying the task after the link resources are sufficient, and the scheme of occupying only one physical link, this embodiment can improve the resource utilization of the cluster network.

[0332] Next, combine Figure 33 The flow of the above-described collection communication method provided in the embodiments of this application will be described.

[0333] The above text describes in detail the collection communication method provided according to this embodiment. The following will combine... Figure 33 This describes the aggregate communication device provided in this embodiment.

[0334] Figure 33 This is a schematic diagram of a possible collective communication device provided in an embodiment of this application. The collective communication device can be used to implement the function of the controller in the above method embodiments, and therefore can also achieve the beneficial effects of the above method embodiments. In this embodiment, the collective communication device may be... Figure 13 The controller 131 used to implement the control function can also be a module (such as a chip) applied to the controller 131.

[0335] The communication device 3300 includes a determination module 3301 and a reconstruction module 3302.

[0336] The determination module 3301 is used to determine multiple sub-topologies in the physical topology of the cluster network, wherein the multiple sub-topologies are generated subgraphs of the physical topology.

[0337] The determining module 3301 is further configured to determine at least one feasible subtopic of all edges of the logical topology including the set communication algorithm in the plurality of subtopics, wherein the edge is used to represent a communication connection between two physical nodes in the subtopic or the physical topology.

[0338] The determining module 3301 is further configured to determine at least one combination of the at least one feasible sub-topology that satisfies the communication bandwidth requirements of the cluster network based on the constraint of the number of links in the physical topology, wherein any combination of the at least one feasible sub-topology includes at least one feasible sub-topology.

[0339] The reconstruction module 3302 is used to reconstruct the physical topology based on the target combination in the at least one combination, so as to obtain the reconstructed physical topology.

[0340] As one possible implementation, the plurality of sub-topologies includes a first sub-topology, and the determining module 3301 is specifically used to: map each logical node of the logical topology to the first sub-topology in different arrangement orders, wherein the arrangement order of each physical node of the first sub-topology in the first sub-topology in different arrangement orders is different; and determine the first sub-topology in any arrangement order after mapping all edges of the logical topology as a feasible sub-topology.

[0341] As one possible implementation, the determining module 3301 is specifically used to: determine the order of each physical node in the first sub-topology with different arrangement orders, and the order of each logical node in the logical topology; and map each logical node to a physical node with the same order in the first sub-topology with different arrangement orders according to the arrangement order of each logical node in the logical topology and the arrangement order of each physical node in the first sub-topology.

[0342] As one possible implementation, the determining module 3301 is specifically used to: generate a node mapping table; the node mapping table includes logical entries and at least one physical entry, the logical entries include logical nodes arranged in the order of logical nodes in the logical topology, and the at least one physical entry includes physical nodes of the first sub-topology arranged in different orders; and map each logical node in the logical entries to a physical node with the same position in the at least one physical entry.

[0343] As one possible implementation, the at least one physical table entry includes a first physical table entry, and the determining module 3301 is specifically used to: determine the adjacency matrix of the logical topology, the adjacency matrix being used to represent the connection relationships between each logical node in the logical topology; determine an isomorphism detection table based on the adjacency matrix, the isomorphism detection table including at least one detection table entry, each of the at least one detection table entry including a logical node to be detected, and a logical node having a connection relationship with the logical node to be detected; and determine the first sub-topology after mapping corresponding to the first physical table entry as a feasible sub-topology if the first sub-topology after mapping corresponding to the first physical table entry includes all connection relationships represented by all detection table entries.

[0344] As one possible implementation, the at least one detection item includes a first detection item, and the device further includes:

[0345] The deletion module 3303 is used to delete physical entries containing infeasible prefixes in the at least one physical entry when the first sub-topology after mapping corresponding to the first physical entry does not include the connection relationship of the logical node to be detected in the first detection entry. The infeasible prefix is ​​composed of the physical node mapped by the logical node to be detected in the first detection entry to the first physical node of the first physical entry.

[0346] As one possible implementation, the determining module 3301 is specifically used for:

[0347] Based on the physical link attributes corresponding to the at least one combination, determine the target combination among the at least one combination.

[0348] As one possible implementation, the physical link attribute includes the number of links, and the determining module 3301 is specifically used to: determine the combination with the fewest links as the target combination based on the number of links corresponding to the at least one combination.

[0349] As one possible implementation, the physical link attribute includes bandwidth, and the determining module 3301 is specifically used to: determine the combination with the largest bandwidth and value as the target combination based on the bandwidth and value corresponding to the at least one combination.

[0350] As one possible implementation, the at least one combination includes a first combination, and the determining module 3301 is specifically used to: determine the first combination using depth-first search (DFS); determine whether the number of links in the first combination meets the constraint condition of the number of links in the physical topology, and whether the total bandwidth of the first combination meets the communication bandwidth requirements of the cluster network; if so, determine that the at least one combination is the first combination.

[0351] As one possible implementation, the constraint on the number of links in the physical topology includes a limitation on the actual number of physical links between all two physical nodes in the physical topology that are in communication connection.

[0352] As one possible implementation, the physical nodes of the physical topology include multiple processors, the physical topology includes a first physical node and a second physical node, the number of links between the first physical node and the second physical node is less than a preset number, and the at least one physical table entry includes a second physical table entry;

[0353] The determining module 3301 is further configured to, after generating the node mapping table and before mapping each logical node in the logical table entry to a physical node with the same position in the at least one physical table entry, add at least one physical node between the first physical node and the second physical node in the second physical table entry, to obtain at least one third physical table entry, provided that the first physical node and the second physical node are adjacent in the second physical table entry; the number of the same physical node in any third physical table entry of the at least one third physical table entry is not greater than the number of processors included in the same physical node, and the same physical nodes are not contiguous.

[0354] It should be understood that the aggregated communication device 3300 in this application embodiment can be implemented using a GPU, NPU, ASIC, or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Software implementation is also possible. Figure 33 In the method shown, the communication device 3300 and its various modules can also be software modules.

[0355] This application also provides a controller, please refer to... Figure 34 , Figure 34 This is a schematic diagram of a controller provided in an embodiment of this application. The controller 3400 includes a memory 3401, a processor 3402, a communication interface 3403, and a bus 3404. The memory 3401, processor 3402, and communication interface 3403 are interconnected via the bus 3404.

[0356] Memory 3401 can be a read-only memory, a static storage device, a dynamic storage device, or a random access memory. Memory 3401 can store computer instructions. When the computer instructions stored in memory 3401 are executed by processor 3402, processor 3402 and communication interface 3403 are used to execute the steps in the collective communication method. For example, processor 3402 is used to execute the above-mentioned... Figure 16 Steps 1601 to 1604 in the illustrated collection communication method, and the above-mentioned Figure 33 The functions of the aforementioned collective communication device 3300.

[0357] Processor 3402 may be a general-purpose CPU, an application-specific integrated circuit (ASIC), a GPU, or any combination thereof. Processor 3402 may include one or more chips. Processor 3402 may include an AI accelerator, such as an NPU.

[0358] The communication interface 3403 uses a transceiver module, such as, but not limited to, a transceiver, to enable communication between the controller 3400 and other devices or communication networks.

[0359] Bus 3404 may include a pathway for transmitting information between various components of controller 3400 (e.g., memory 3401, processor 3402, communication interface 3403).

[0360] The controller 3400 can be a computer (e.g., a server) in a cloud data center, or a computer or terminal in an edge data center.

[0361] The method steps in this embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a terminal device. Of course, the processor and storage medium can also exist as discrete components in a network device or terminal device.

[0362] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD). The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A collective communication method, characterized in that, Applied to a controller, the method includes: Determine multiple sub-topologies in the physical topology of the cluster network, wherein the multiple sub-topologies are generated subgraphs of the physical topology; Determine at least one feasible subtopic from the plurality of subtopics that includes all edges of the logical topology of the set communication algorithm, wherein the edges are used to represent a communication connection between two physical nodes in the subtopic or the physical topology; Based on the constraint of the number of links in the physical topology, at least one combination of the at least one feasible sub-topology that satisfies the communication bandwidth requirements of the cluster network is determined, and any combination of the at least one combination includes at least one feasible sub-topology. The physical topology is reconstructed based on the target combination in at least one of the combinations to obtain the reconstructed physical topology.

2. The method according to claim 1, characterized in that, The plurality of subtopologies includes a first subtopology, and determining at least one feasible subtopology that includes all edges of the logical topology among the plurality of subtopologies includes: Each logical node of the logical topology is mapped to a first sub-topology with different arrangement orders, and the arrangement order of each physical node of the first sub-topology is different in the first sub-topology with different arrangement orders. The first subtopology, which is determined by any permutation order of the mappings of all edges of the logical topology, is a feasible subtopology.

3. The method according to claim 2, characterized in that, The step of mapping each logical node of the logical topology to the first sub-topology with different arrangement orders includes: Determine the order of each physical node in the first sub-topology with different arrangement orders, and the order of each logical node in the logical topology; Based on the arrangement order of each logical node in the logical topology and the arrangement order of each physical node in the first sub-topology, each logical node is mapped to a physical node with the same position in the first sub-topology with different arrangement orders.

4. The method according to claim 3, characterized in that, The step of mapping each logical node to a physical node with the same position in the first sub-topology with different arrangement orders, based on the arrangement order of each logical node in the logical topology and the arrangement order of each physical node in the first sub-topology, includes: Generate a node mapping table; the node mapping table includes logical entries and at least one physical entry, the logical entries include logical nodes arranged in the order of logical nodes in the logical topology, and the at least one physical entry includes physical nodes of the first sub-topology arranged in different orders. Each logical node in the logical table entry is mapped to a physical node with the same position in the at least one physical table entry.

5. The method according to claim 4, characterized in that, The at least one physical entry includes a first physical entry, and determining the first subtopology, which includes any permutation order of the mapped edges of the logical topology, as a feasible subtopology includes: Determine the adjacency matrix of the logical topology, the adjacency matrix being used to represent the connection relationships between each logical node in the logical topology; A homogeneity detection table is determined based on the adjacency matrix. The homogeneity detection table includes at least one detection table entry. Each of the at least one detection table entry includes a logical node to be detected and a logical node that has a connection relationship with the logical node to be detected. If the first subtopology after mapping the first physical table entry includes all connection relationships represented by all detection table entries, then the first subtopology after mapping the first physical table entry is determined to be a feasible subtopology.

6. The method according to claim 5, characterized in that, The at least one detection item includes a first detection item, and the method further includes: If the first sub-topology after mapping corresponding to the first physical table entry does not include the connection relationship of the logical node to be detected in the first detection table entry, delete the physical table entry containing the infeasible prefix in the at least one physical table entry. The infeasible prefix is ​​composed of the physical node mapped by the logical node to be detected in the first detection table entry to the first physical node of the first physical table entry.

7. The method according to claim 1, characterized in that, The process of determining the target combination includes: Based on the physical link attributes corresponding to the at least one combination, determine the target combination among the at least one combination.

8. The method according to claim 7, characterized in that, The physical link attributes include the number of links. Determining the target combination among the at least one combination based on the physical link attributes corresponding to the at least one combination includes: Based on the number of links corresponding to at least one combination, the combination with the fewest links is determined as the target combination.

9. The method according to claim 7, characterized in that, The physical link attributes include bandwidth, and determining the target combination among the at least one combination based on the physical link attributes corresponding to the at least one combination includes: Based on the bandwidth and value corresponding to at least one of the combinations, the combination with the largest bandwidth and value is determined as the target combination.

10. The method according to claim 1, characterized in that, The at least one combination includes a first combination, wherein the constraints of the links based on the physical topology, determining at least one combination of the at least one feasible sub-topology that satisfies the communication bandwidth requirements of the cluster network, includes: The first combination is determined using depth-first search (DFS); Determine whether the number of links in the first combination meets the constraint of the number of links in the physical topology, and whether the total bandwidth of the first combination meets the communication bandwidth requirements of the cluster network. If so, then the at least one combination is determined to be the first combination.

11. The method according to claim 1, characterized in that, The constraints on the number of links in the physical topology include the limitation on the actual number of physical links between any two physical nodes in the physical topology that are in communication connection.

12. The method according to claim 4, characterized in that, The physical nodes of the physical topology include multiple processors, the physical topology includes a first physical node and a second physical node, the number of links between the first physical node and the second physical node is less than a preset number, and the at least one physical table entry includes a second physical table entry; after generating the node mapping table and before mapping each logical node in the logical table entry to the physical node with the same position in the at least one physical table entry, the method further includes: In the case where the first physical node and the second physical node are adjacent in the second physical table entry, at least one physical node is added to the second physical table entry to obtain at least one third physical table entry; the number of the same physical node in any third physical table entry of the at least one third physical table entry is not greater than the number of processors included in the same physical node, and the same physical nodes are not contiguous.

13. A collective communication device, characterized in that, The device includes: A determination module is used to determine multiple sub-topologies in the physical topology of the cluster network, wherein the multiple sub-topologies are generated subgraphs of the physical topology; The determining module is further configured to determine at least one feasible subtopic of all edges of the logical topology including the set communication algorithm in the plurality of subtopics, wherein the edge is used to represent a communication connection between two physical nodes in the subtopic or the physical topology; The determining module is further configured to determine at least one combination of the at least one feasible sub-topology that satisfies the communication bandwidth requirements of the cluster network based on the constraint of the number of links in the physical topology, wherein any combination of the at least one feasible sub-topology includes at least one feasible sub-topology. A reconstruction module is used to reconstruct the physical topology based on a target combination in at least one combination, to obtain a reconstructed physical topology.

14. The apparatus according to claim 13, characterized in that, The plurality of sub-topologies includes a first sub-topology, and the determining module is specifically used for: Each logical node of the logical topology is mapped to a first sub-topology with different arrangement orders, and the arrangement order of each physical node of the first sub-topology is different in the first sub-topology with different arrangement orders. The first subtopology, which is determined by any permutation order of the mappings of all edges of the logical topology, is a feasible subtopology.

15. The apparatus according to claim 14, characterized in that, The determining module is specifically used for: Determine the order of each physical node in the first sub-topology with different arrangement orders, and the order of each logical node in the logical topology; Based on the arrangement order of each logical node in the logical topology and the arrangement order of each physical node in the first sub-topology, each logical node is mapped to a physical node with the same position in the first sub-topology with different arrangement orders.

16. The apparatus according to claim 15, characterized in that, The determining module is specifically used for: Generate a node mapping table; the node mapping table includes logical entries and at least one physical entry, the logical entries include logical nodes arranged in the order of logical nodes in the logical topology, and the at least one physical entry includes physical nodes of the first sub-topology arranged in different orders. Each logical node in the logical table entry is mapped to a physical node with the same position in the at least one physical table entry.

17. The apparatus according to claim 16, characterized in that, The at least one physical entry includes a first physical entry, and the determining module is specifically used for: Determine the adjacency matrix of the logical topology, the adjacency matrix being used to represent the connection relationships between each logical node in the logical topology; A homogeneity detection table is determined based on the adjacency matrix. The homogeneity detection table includes at least one detection table entry. Each of the at least one detection table entry includes a logical node to be detected and a logical node that has a connection relationship with the logical node to be detected. If the first subtopology after mapping the first physical table entry includes all connection relationships represented by all detection table entries, then the first subtopology after mapping the first physical table entry is determined to be a feasible subtopology.

18. The apparatus according to claim 17, characterized in that, The at least one detection item includes a first detection item, and the device further includes: The deletion module is used to delete physical entries containing infeasible prefixes in the at least one physical entry when the first sub-topology after mapping corresponding to the first physical entry does not include the connection relationship of the logical node to be detected in the first detection entry. The infeasible prefix is ​​composed of the physical node mapped by the logical node to be detected in the first detection entry to the first physical node of the first physical entry.

19. The apparatus according to claim 13, characterized in that, The determining module is specifically used for: Based on the physical link attributes corresponding to the at least one combination, determine the target combination among the at least one combination.

20. The apparatus according to claim 19, characterized in that, The physical link attributes include the number of links, and the determining module is specifically used for: Based on the number of links corresponding to at least one combination, the combination with the fewest links is determined as the target combination.

21. The apparatus according to claim 19, characterized in that, The physical link attributes include bandwidth, and the determining module is specifically used for: Based on the bandwidth and value corresponding to at least one of the combinations, the combination with the largest bandwidth and value is determined as the target combination.

22. The apparatus according to claim 13, characterized in that, The at least one combination includes a first combination. When the determining module determines at least one combination of the at least one feasible sub-topology that satisfies the communication bandwidth requirements of the cluster network based on the constraints of the links in the physical topology, it is specifically used for: The first combination is determined using depth-first search (DFS); Determine whether the number of links in the first combination meets the constraint of the number of links in the physical topology, and whether the total bandwidth of the first combination meets the communication bandwidth requirements of the cluster network. If so, then the at least one combination is determined to be the first combination.

23. The apparatus according to claim 13, characterized in that, The constraints on the number of links in the physical topology include the limitation on the actual number of physical links between any two physical nodes in the physical topology that are in communication connection.

24. The apparatus according to claim 16, characterized in that, The physical nodes of the physical topology include multiple processors, the physical topology includes a first physical node and a second physical node, the number of links between the first physical node and the second physical node is less than a preset number, and the at least one physical table entry includes a second physical table entry; The determining module is further configured to, after generating the node mapping table and before mapping each logical node in the logical table entry to a physical node with the same position in the at least one physical table entry, add at least one physical node in the second physical table entry when the first physical node and the second physical node are adjacent in the second physical table entry, to obtain at least one third physical table entry; the number of the same physical node in any third physical table entry of the at least one third physical table entry is not greater than the number of processors included in the same physical node, and the same physical nodes are not contiguous.

25. A controller, characterized in that, The method includes a memory and a processor, wherein the memory is used to store at least one set of computer instructions; when the processor executes the at least one set of computer instructions, it performs the operation steps of the method according to any one of claims 1-12.

26. A collective communication system, characterized in that, The aggregated communication system includes at least one switch, at least one cluster server, and a controller as described in claim 25, wherein the controller is connected to the at least one switch and the at least one cluster server respectively, and the controller performs the operation steps of the method as described in any one of claims 1-12.

27. A computer-readable storage medium, characterized in that, include: Computer software instructions; when the computer software instructions are executed in a controller, the controller performs the method of any one of claims 1-12.

28. A chip system, characterized in that, The chip system includes a memory and at least one processor, the memory being used to store a set of computer instructions; when the processor executes the set of computer instructions, it performs the method of any one of claims 1-12.

Images