A cross-node set communication multi-link optimization method

Abstract

The application discloses a cross-node set communication multi-link optimization method, which comprises the following steps: a double half-ring algorithm is designed for an AllGather type intensive operator, two-way transmission is adopted, and data blocking is introduced in the final communication step, and sub-blocks are distributed according to the bandwidth proportion of heterogeneous links, so that the link utilization rate in the whole process is maximized; a PairwisePro algorithm is designed for an AlltoAll operator, the transmission time slot definition and data subdivision strategy are adjusted, and the multi-link parallel efficiency is improved; finally, the optimized algorithm is combined according to the node scale by using a hierarchical strategy, the optimal communication group size is dynamically determined, a cross-node multi-link concurrent communication scheduling scheme is generated, and the overall algorithm bandwidth is maximized. The application is used for efficient set communication under a complex cross-node heterogeneous network, can fully utilize the bandwidth of heterogeneous links, and can significantly improve the execution efficiency of the set communication operator in large-scale distributed training.

Description

Technical Field

[0001] This invention relates to the fields of high-performance computing, distributed computing, and large-scale artificial intelligence model training technology, and in particular to a multi-link optimization method for cross-node aggregate communication. Background Technology

[0002] In large-scale distributed training, especially in scenarios involving large models, ensemble communication is a crucial operation for synchronizing and exchanging model parameters and gradients between computing nodes (such as GPUs or dies). Common ensemble communication operators include AllReduce, AllGather, ReduceScatter, and AlltoAll. The execution efficiency of these operators directly impacts the overall throughput and training time of distributed training. Modern computing clusters typically have complex, multi-layered physical topologies. For example, computing nodes are connected via heterogeneous links such as high-speed interconnects, PCIe switches, NVLink, and RDMA networks. These links differ significantly in bandwidth and latency. Traditional ensemble communication algorithms (such as Ring AllReduce, HD AllReduce, and Pairwise) are often designed for homogeneous networks or do not fully consider the concurrent utilization of multiple links, often resulting in some links being idle during certain communication steps. In heterogeneous topologies, this limitation is easily restricted to the lowest bandwidth link, leading to low overall communication bandwidth utilization and failing to fully realize the hardware's potential.

[0003] Therefore, there is an urgent need for an efficient aggregation communication optimization method that enables cross-node, multi-link concurrency, in order to adapt to complex heterogeneous cluster topologies and maximize overall communication performance. Summary of the Invention

[0004] The purpose of this invention is to address the problems of link idleness and low bandwidth utilization in certain communication steps of existing collective communication algorithms in complex heterogeneous multi-link cluster environments, and to provide an efficient multi-link concurrent scheduling method that significantly improves the performance of collective communication operators.

[0005] The objective of this invention is achieved through the following technical solution: a multi-link optimization method for cross-node aggregate communication, comprising the following steps: Obtain the physical topology of the cluster, the bandwidth and latency information of each link, the target set communication operator type, and the amount of data to be synchronized; use the bandwidth and latency information of each link to estimate the transmission time on the link; If the target set communication operator is a condensation operator for the AllGather class, then the double semi-ring algorithm is executed: the nodes participating in the communication are organized into a logical ring for bidirectional transmission, bidirectional data transmission is carried out during the communication process, and data block division is introduced in the final communication step, and the allocated sub-blocks are divided and transmitted according to the bandwidth ratio of the heterogeneous links. If the target set communication operator is AlltoAll, then the PairwisePro algorithm is executed: the definition of the transmission time slot is adjusted based on the transmission time of the data block on the superimposed link, and the data subdivision strategy is set according to the bandwidth ratio of each link, so that different links transmit different data sub-blocks in parallel in different time slots; For cross-node scale scenarios, based on the amount of data to be synchronized, a hierarchical strategy is adopted to combine the dual semi-ring algorithm or the PairwisePro algorithm to dynamically determine the optimal communication group and generate a cross-node multi-link concurrent communication scheduling scheme.

[0006] Furthermore, estimating the transmission time on a link using the bandwidth and delay information of each link specifically involves: transmitting a single point-to-point communication... m Temporal modeling of byte data in Indicates static delay; For link bandwidth; data segmentation, time slot definition, and communication group selection enable parallel links. Alignment, and in scenarios with small amounts of data, prioritize avoiding candidate paths with excessive startup delays.

[0007] Furthermore, the implementation of the double semi-ring algorithm in step S2 is specifically as follows: Organize the n nodes participating in the communication into two logical rings in opposite directions; In the first n / 2-1 communication steps, the original data block is not divided into blocks, and the data transfer of the regular Ring algorithm is performed independently on the two rings respectively. In the final communication step, the data block to be transmitted is divided into blocks and transmitted on two logical half-rings respectively. If there are high-bandwidth links and low-bandwidth auxiliary links between nodes at the same time, the data block is subdivided into multiple sub-blocks according to the theoretical bandwidth ratio of each link type, and each sub-block is assigned to the corresponding heterogeneous link for parallel transmission.

[0008] Furthermore, the implementation of the PairwisePro algorithm in step S3 is as follows: the duration of the transmission time slot is defined as the combined transmission time of a single data block on multiple links with superimposed bandwidth; In the first transmission time slot, a node simultaneously sends data blocks to its neighboring nodes and specific cross-nodes through the highest bandwidth link and the bandwidth superimposed link. The data sent through the bandwidth superimposed link is subdivided according to the bandwidth ratio of each link. In subsequent transmission time slots, nodes send data sub-blocks to different peer nodes only through bandwidth superposition links in a cyclic shift manner; By merging the transmission operations of all time slots, an equivalent fully connected communication mode between all nodes is achieved, making full use of the bidirectional bandwidth of all links.

[0009] Furthermore, the hierarchical strategy for combining the double-half-ring algorithm or the PairwisePro algorithm specifically involves: when nodes are distributed across a multi-level physical structure, the entire communication process is divided into multiple physical levels, including inter-cluster communication, inter-server communication, and intra-server communication; based on the current amount of data to be synchronized, a performance analysis program is run in advance to compare the theoretical or measured communication time under different node group sizes; the node size with the shortest communication time is selected as the optimal communication group size under the current level; nodes are grouped according to this optimal communication group size; and the double-half-ring algorithm or the PairwisePro algorithm is recursively or iteratively applied at different physical levels to complete global aggregate communication.

[0010] According to another aspect of the specification, a cross-node aggregate communication multi-link optimization device is also provided, including a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it implements the cross-node aggregate communication multi-link optimization method.

[0011] According to another aspect of the specification, a computer-readable storage medium is also provided, on which a program is stored, which, when executed by a processor, implements the aforementioned method for optimizing multi-link cross-node aggregate communication.

[0012] The beneficial effects of this invention are: 1. This invention proposes a double-half-ring algorithm for AllGather-type operators and a PairwisePro algorithm for AlltoAll operators. Compared with the existing Ring algorithm, which only propagates along a single logical ring, and the Pairwise algorithm, which occupies a single path according to a fixed pair-end order, this invention further transforms the inefficient waiting time of idle links and heterogeneous links at the end of traditional algorithms into parallel transmission windows allocated according to bandwidth ratios. By introducing sub-block partitioning, link mapping, and completion time alignment in the final communication step or transmission time slot, links with different bandwidths such as SIO, HCCS, and RDMA can all participate in effective data transmission, achieving continuous high link utilization throughout the communication process and breaking through the link idle limitation of traditional algorithms in specific steps.

[0013] 2. By introducing a hierarchical strategy and performance analysis mechanism, the optimal communication granularity is selected based on the actual data volume and topology, effectively balancing bandwidth utilization and communication latency, and perfectly adapting to large-scale expansion scenarios from single servers to cross-clusters.

[0014] 3. The solution directly focuses on maximizing the utilization of the underlying physical link characteristics in the communication process. Under various data scales and number of nodes, it can bring significant performance improvements to various ensemble communication operators, fully tapping the hardware potential and meeting the high concurrency and high throughput requirements of ensemble communication in large model training. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the execution flow of the cross-node aggregate communication multi-link optimization method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the 8-node physical topology used in the embodiment; Figure 3 This is a schematic diagram of the 12-node physical topology used in the embodiment; Figure 4 This is a schematic diagram of the 32-node physical topology used in the embodiment; Figure 5 This is a schematic diagram of the 48-node physical topology used in the embodiment; Figure 6 This is a schematic diagram of the 64-node physical topology used in the embodiment; Figure 7 A schematic diagram illustrating the specific process of the double semi-loop algorithm for the Allgather operator on a single machine with 8 nodes; Figure 8 A schematic diagram illustrating the specific process of the PairwisePro algorithm from Epoch 1 to Epoch 2 for the AlltoAll operator on a single machine with 8 nodes; Figure 9 A schematic diagram illustrating the specific process of the PairwisePro algorithm from Epoch 13 to Epoch 4 for the AlltoAll operator on a single 8-node machine. Figure 10 A schematic diagram illustrating the specific process of the PairwisePro algorithm from Epoch 5 to Epoch 6 for the AlltoAll operator on a single 8-node machine. Figure 11 This is a schematic diagram of a cross-node aggregate communication multi-link optimization device provided in an embodiment of the present invention. Detailed Implementation

[0016] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0018] like Figure 1 As shown, the specific content of the embodiments of the present invention is as follows: S1: Obtain the physical topology of the cluster, the bandwidth and latency information of each link, the target set communication operator type, and the amount of data to be synchronized.

[0019] The physical topology includes: The cluster topology describes the underlying physical connections between computing nodes and can be given by a set of graphs. The cluster topology contains multiple levels of heterogeneous links, and the link information includes characteristics such as bandwidth and latency for each link. Computing nodes (such as GPUs or dies) and switch nodes are arranged in a hierarchical structure, connected by heterogeneous links. Different types of links have different theoretical bandwidths; for example, in a specific topology, the SIO link bandwidth is 160G / s, the HCCS link bandwidth is 120G / s, and the RDMA link bandwidth is 20G / s, etc.

[0020] Latency and bandwidth information are used to estimate the transmission time of a data block on a candidate link, specifically a single point-to-point communication transmission. m Temporal modeling of byte data in This represents static latency, which is independent of the amount of data and includes fixed overhead such as protocol processing and queuing. This refers to the link bandwidth. When performing data segmentation, time slot definition, and communication group selection, the parallel link bandwidth is optimized. Alignment should be prioritized, and candidate paths with excessive startup latency should be avoided in scenarios with small data volumes. For a specific set of communication operators under the same environment, the optimal communication strategy will differ depending on the data volume, and a corresponding concurrent scheduling scheme needs to be generated based on the physical topology characteristics.

[0021] The physical topology diagrams for 8, 12, 32, 48, and 64 nodes used in the embodiments are shown below. Figure 2-6 As shown, the 8-node topology uses server0 as the basic server unit. Within the server, there are GPU0, GPU1, GPU2, and GPU3. Each GPU includes Die0 and Die1. Die0 and Die1 within a GPU are connected via SIO links. Dies between GPUs communicate via L0 switches using HCCS. An L1 switch (connecting each die) is used for upper-level aggregation of dies within the server. The 12-node, 32-node, 48-node, and 64-node topologies expand upon this, adding multiple server units such as server0, server1, server2, and server3. These are further connected across servers or clusters via L1, L2, and L3 switches. Cross-node communication is carried out via RDMA links.

[0022] S2: For AllGather-type aggregation operators (including AllGather, ReduceScatter, and AllReduce, which mainly complete data aggregation or data redistribution within a node set), a dual-half-ring algorithm optimization process is executed. This algorithm organizes nodes into a bidirectional logical ring, and through bidirectional transmission and the introduction of data chunking in the final communication step, it allocates sub-blocks according to the bandwidth ratio of heterogeneous links to ensure that all links are fully utilized throughout the communication process.

[0023] The specific execution process of the double semi-ring algorithm includes: For AllGather-type intensive operators, traditional ring algorithms often result in half of the link being idle in the last communication step, failing to achieve full bandwidth utilization. To address this issue, this invention employs a dual-half-ring algorithm for scheduling. Assuming the number of nodes is n, the specific execution process is as follows: Step 1: Initialize the device order of two ring links in opposite directions, and organize the n nodes participating in communication into two logical half-rings for bidirectional data transmission.

[0024] Step 2: In the first n / 2-1 communication steps, the original data block is not additionally divided. Each node independently executes the data transmission of the conventional ring algorithm on the two logical half-rings.

[0025] Step 3: In the n / 2th communication step (i.e., the final communication step), the data block to be transmitted is split into two equal parts, which are then transmitted on two logical half-rings respectively. If both high-bandwidth links and low-bandwidth auxiliary links exist between nodes at this point, the data block is further subdivided into corresponding sub-blocks strictly according to the theoretical bandwidth ratio of these heterogeneous links. High-proportion sub-blocks are transmitted through high-bandwidth links, while low-proportion sub-blocks are transmitted in parallel through low-bandwidth links.

[0026] Figure 2 Taking a single machine with 8 nodes as an example, the specific process of the dual-half-ring algorithm is illustrated. Nodes 0 to 7 form two logical half-rings in opposite directions. The red arrows represent SIO link transmission, and the blue arrows represent HCCS+RDMA superimposed link transmission. In Steps 1 to 3, each node transmits complete data blocks along both directions. The circles labeled 0 to 7 indicate the data block numbers transmitted along the corresponding links in the current step. Step 4 is the final communication step. In the diagram, sub-blocks are represented by numbers such as 0.1 and 0.2, indicating that the original data blocks have been further divided. Some sub-blocks are transmitted via the SIO link, and others via the HCCS+RDMA superimposed link, thus maintaining multi-link parallelism in the final step. Through the above proportional block division strategy in the final step, the link gap period at the end of the traditional algorithm is filled, achieving full link utilization throughout the entire process.

[0027] For large-scale node scenarios across servers (such as 32, 48, or 64 nodes), this invention extends the concept by incorporating a hierarchical strategy, determining the optimal number of nodes in the bidirectional ring based on theoretical estimation. For the 32-node, two-level algorithm, a dual-half-ring algorithm is used within each server, and the HD algorithm is used between servers. The specific theoretical bandwidth evaluation formula is as follows, where... = 8 / 16 / 32, = 4 / 2 / 1, For the amount of data transmitted: For larger-scale scenarios with 48 and 64 nodes, a three-level algorithm is used: a dual-half-ring algorithm within a server, and a Ring algorithm between servers and between pods. The specific theoretical bandwidth evaluation formula is as follows, where for 48 nodes... = 8 / 16, =2 / 1, =3,64 nodes = 8 / 16 / 32, = 4 / 2 / 1, = 2: Based on the optimal result, the nodes are grouped, and the inter-group set communication operation is performed first. Then, the double semi-ring algorithm is performed within each group, thereby smoothly extending the local optimum to the global optimum.

[0028] S3: For the AlltoAll operator, execute the PairwisePro algorithm optimization process. This algorithm adjusts the definition of the transmission time slot as the transmission time of the data block on the superimposed link, and formulates a data subdivision strategy based on the link bandwidth ratio to achieve efficient parallel transmission of multiple links in different time slots.

[0029] The specific execution process of the PairwisePro algorithm includes: This invention optimizes the AlltoAll operator. Since each node in the AlltoAll operator receives a different set of data, the traditional Pairwise algorithm cannot fully utilize all parallel links in multi-link heterogeneous networks and is susceptible to the bottleneck effect. This invention redesigns the algorithm: Step 1: Define a transmission time slot. The duration of a transmission time slot is defined as the total time required for a single data block to be transmitted across multiple links with superimposed bandwidth, rather than the transmission time of a single link.

[0030] Step 2: In the first transmission time slot, the node simultaneously initiates multiple links to send data to different targets. That is, the node simultaneously performs transmissions on specific direct links and bandwidth-overlay links, sending data to adjacent nodes connected by direct links and specific cross-link nodes. During this process, the data sent through the bandwidth-overlay links is further finely divided according to the actual bandwidth ratio of each link to ensure that no link is in a waiting state.

[0031] Step 3: In the subsequent transmission slots, the node uses only the bandwidth superposition link to send different data sub-blocks to the remaining peer nodes sequentially using a cyclic offset logic. In the last transmission slot, the peer node is an adjacent node that is not connected by a specific direct link.

[0032] Step 4: Perform operation merging after all transmission time slots have ended. Since each node outputs data block fragments at full capacity according to the link ratio within each calculated time slice, the overall effect is equivalent to each node completing full interconnection communication with all other nodes in the same period. This method concurrently uses multiple links to read or write data from the other end, allowing the bidirectional bandwidth of the full-duplex interconnection links to be utilized simultaneously, maximizing the overall algorithm bandwidth for multi-link concurrency, and simplifying implementation and deployment. Figures 8-10 The PairwisePro algorithm's time slot arrangement is illustrated using a single-machine, 8-node AlltoAll model as an example. Ellipses 0 to 7 represent communication nodes, and the squares next to each node (0 to 7) indicate the data block numbers that node needs to send or receive. Black arrows represent SIO direct links, and orange arrows represent HCCS+RDMA overlay links. The labels "RDMA: 1 / 7 chunk" and "HCCS: 6 / 7 chunk" in the diagram indicate that the data on the overlay link is divided into 1 / 7 and 6 / 7 according to the bandwidth ratios of RDMA and HCCS, respectively. Epochs 1 to 6 correspond to consecutive transmission time slots. In the first time slot, both the SIO link and the HCCS+RDMA overlay link are used simultaneously. In subsequent time slots, the remaining target node's data sub-blocks are sent according to a cyclic shift relationship, ultimately ensuring that nodes 0 to 7 all obtain their required complete AlltoAll data sets.

[0033] For cross-node scale scenarios, a hierarchical strategy is adopted to combine the dual semi-ring algorithm or the PairwisePro algorithm to dynamically determine the optimal communication group and generate a cross-node multi-link concurrent communication scheduling scheme.

[0034] When nodes are distributed across a multi-level physical structure, the entire communication process is divided into multiple physical levels, including inter-cluster communication, inter-server communication, and intra-server communication. Based on the current amount of data to be synchronized, a performance analysis program is run in advance to compare the theoretical or measured communication time under different node group sizes. The node size with the shortest communication time is selected as the optimal communication group size under the current level. Nodes are grouped according to this optimal communication group size, and the double semi-ring algorithm or PairwisePro algorithm is recursively or iteratively applied at different physical levels to complete global set communication.

[0035] To verify the effectiveness of the multi-link optimization method described in this invention, extensive real-world testing was conducted in a real cluster environment. The test scenarios covered different node topologies (including 16 nodes on a single machine, 32 nodes across servers, and 64 nodes across pods, etc.), and comprehensively evaluated mainstream aggregate communication operators such as AllGather, ReduceScatter, AllReduce, and AlltoAll under different data volumes (64M, 256M, 1G, and 2G). The Alltoall operator did not involve cross-pod communication (only 16-node and 32-node scenarios were tested). By comparing the real-world data of the baseline algorithm and the improved algorithm of this invention, the actual performance gains of the proposed solution were verified.

[0036] Example 1: This experiment tested the optimization effect of the AllGather, ReduceScatter and AllReduce operators using the double semi-ring algorithm compared with the traditional Ring algorithm. The results are shown in Table 1.

[0037] Table 1 shows that, compared to the traditional baseline Ring method, the dual-half-ring algorithm combined with a hierarchical strategy described in this invention can significantly improve operator execution bandwidth. Under a joint scheduling mechanism using the dual-half-ring algorithm within a single machine and the hierarchical algorithm across machines, the performance improvement is particularly significant in small data volume (64M) scenarios, with the actual execution bandwidth improvement for each operator ranging from 11% to 33%. In large data volume scenarios (256M and above), this optimization method maintains a stable performance advantage. Proportional data subdivision effectively eliminates communication bottlenecks in heterogeneous topologies, proving the practical feasibility of full-load link utilization throughout the entire communication process.

[0038] Table 1. Baseline and Real-world Comparison of AllGather, ReduceScatter, and AllReduce Operators with Improved Algorithms Example 2: This experiment tested the optimization effect of the AlltoAll operator using the PairwisePro algorithm, and the results are shown in Table 2.

[0039] Table 2 shows that, for the AlltoAll operator, the PairwisePro algorithm (i.e., the equivalent fully interconnected algorithm) described in this invention exhibits superior bandwidth performance compared to the traditional Staged Pairwise baseline algorithm across various node sizes and data volume gradients. Considering the statistical percentage improvement across all real-world test data, the method of this invention achieves an average performance improvement of 24.33% across all operators in scenarios with small data volumes (64M); and an average performance improvement of 10.51% across all operators in scenarios with large data volumes (256M and 1G / 2G). These experimental results fully demonstrate that the method of this invention can effectively overcome hardware limitations, significantly reduce communication time, and significantly improve the overall execution efficiency of distributed training in complex, multi-level heterogeneous network environments.

[0040] Table 2. Comparison of baseline and improved algorithm performance of AlltoAll operator. Corresponding to the aforementioned embodiment of a cross-node aggregated communication multi-link optimization method, the present invention also provides an embodiment of a cross-node aggregated communication multi-link optimization device.

[0041] See Figure 11 The present invention provides a cross-node aggregate communication multi-link optimization device, including a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement a cross-node aggregate communication multi-link optimization method in the above embodiment.

[0042] The embodiment of the cross-node aggregated communication multi-link optimization device provided by this invention can be applied to any device with data processing capabilities, such as a computer or other similar device. The device embodiment can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 11 The diagram shown is a hardware structure diagram of any device with data processing capabilities, where the cross-node aggregated communication multi-link optimization device provided by the present invention is located. (Except for...) Figure 11 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0043] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0044] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0045] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements a cross-node aggregate communication multi-link optimization method as described in the above embodiments.

[0046] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0047] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method for optimizing multi-link cross-node aggregate communication.

[0048] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0049] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. This application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for optimizing multi-link cross-node aggregate communication, characterized in that, Includes the following steps: Obtain the physical topology of the cluster, the bandwidth and latency information of each link, the target set communication operator type, and the amount of data to be synchronized; use the bandwidth and latency information of each link to estimate the transmission time on the link; If the target set communication operator is an AllGather type intensive operator, then the double semi-ring algorithm is executed: the nodes participating in the communication are organized into a logical ring for bidirectional transmission, bidirectional data transmission is carried out during the communication process, and data block division is introduced in the final communication step, and the allocated sub-blocks are divided and transmitted according to the bandwidth ratio of the heterogeneous links. If the target set communication operator is AlltoAll, then the PairwisePro algorithm is executed: the definition of the transmission time slot is adjusted based on the transmission time of the data block on the superimposed link, and the data subdivision strategy is set according to the bandwidth ratio of each link, so that different links transmit different data sub-blocks in parallel in different time slots; For cross-node scale scenarios, based on the amount of data to be synchronized, a hierarchical strategy is adopted to combine the dual semi-ring algorithm or the PairwisePro algorithm to dynamically determine the optimal communication group and generate a cross-node multi-link concurrent communication scheduling scheme.

2. The method for optimizing multi-link cross-node aggregate communication according to claim 1, characterized in that, The method of estimating the transmission time on a link using the bandwidth and delay information of each link specifically involves: transmitting a single point-to-point communication... m Temporal modeling of byte data in Indicates static delay; For link bandwidth; data segmentation, time slot definition, and communication group selection enable parallel links. Alignment, and in scenarios with small amounts of data, priority should be given to avoiding candidate paths with excessive startup delays.

3. The method for optimizing multi-link cross-node aggregate communication according to claim 1, characterized in that, The AllGather-type aggregation operators are operators that complete the aggregation or redistribution of data within a set of nodes, specifically including AllGather, ReduceScatter, and AllReduce.

4. The method for optimizing multi-link cross-node aggregate communication according to claim 1, characterized in that, The specific implementation of the double semi-ring algorithm in step S2 is as follows: Organize the n nodes participating in the communication into two logical rings in opposite directions; In the first n / 2-1 communication steps, the original data block is not divided into blocks, and the data transfer of the regular Ring algorithm is performed independently on the two rings respectively. In the final communication step, the data block to be transmitted is divided into blocks and transmitted on two logical half-rings respectively. If there are high-bandwidth links and low-bandwidth auxiliary links between nodes at the same time, the data block is subdivided into multiple sub-blocks according to the theoretical bandwidth ratio of each link type, and each sub-block is assigned to the corresponding heterogeneous link for parallel transmission.

5. The method for optimizing multi-link cross-node aggregate communication according to claim 1, characterized in that, The implementation of the PairwisePro algorithm in step S3 is as follows: the duration of the transmission time slot is defined as the combined transmission time of a single data block on multiple links with superimposed bandwidth. In the first transmission time slot, a node simultaneously sends data blocks to its neighboring nodes and specific cross-nodes through the highest bandwidth link and the bandwidth superimposed link. The data sent through the bandwidth superimposed link is subdivided according to the bandwidth ratio of each link. In subsequent transmission time slots, nodes send data sub-blocks to different peer nodes only through bandwidth superposition links in a cyclic shift manner; By merging the transmission operations of all time slots, an equivalent fully connected communication mode between all nodes is achieved, making full use of the bidirectional bandwidth of all links.

6. The method for optimizing multi-link cross-node aggregate communication according to claim 1, characterized in that, The specific implementation of the hierarchical strategy combining the double-half-ring algorithm or the PairwisePro algorithm is as follows: When nodes are distributed in a multi-level physical structure, the entire communication process is divided into multiple physical levels, including inter-cluster communication, inter-server communication, and intra-server communication; based on the current amount of data to be synchronized, a performance analysis program is run in advance to compare the theoretical or measured communication time under different node group sizes; the node size with the shortest communication time is selected as the optimal communication group size under the current level, and the nodes are grouped according to the optimal communication group size, and the double-half-ring algorithm or the PairwisePro algorithm is applied recursively or iteratively at different physical levels to complete global set communication.

7. A multi-link optimization device for cross-node aggregated communication, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the processor executes the executable code, it implements a cross-node aggregate communication multi-link optimization method as described in any one of claims 1-6.

8. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements a cross-node aggregate communication multi-link optimization method as described in any one of claims 1-6.

Images