A collection communication method suitable for a core grain device cluster of a unified bus interconnection

Abstract

The present application relates to the technical field of computer cluster communication, and discloses a collection communication method suitable for a core grain device cluster interconnected by a unified bus, which comprises the following steps: collecting hierarchical topology and link information of the core grain device cluster, and constructing a global topology information database according to the collected hierarchical topology and link information; optimizing core grain level primitives, node level primitives and supernode level primitives; deploying a link monitoring unit to monitor link state indicators in real time, triggering an alarm when a link fault is detected, screening an optimal backup link based on the global topology information database and updating a communication path, and allocating computing power and communication resources by using a dynamic resource scheduling algorithm; and dynamically adjusting the communication path and bandwidth allocation according to link dynamic parameters according to the collection communication task demand. The present application is suitable for a three-level link architecture of a core grain device cluster, can improve communication efficiency and bandwidth utilization, enhance cluster reliability and resource utilization, and effectively adapt to the high computing power and high communication intensity demand of large model training.

Description

Technical Field

[0001] This invention relates to the field of computer cluster communication technology, and more specifically to a collective communication method for a cluster of chip devices interconnected by a unified bus. Background Technology

[0002] With the rapid development of artificial intelligence technology, the field of large-scale models has entered a stage of large-scale evolution with hundreds of billions to trillions of parameters. New model forms such as Mixture of Experts (MoE) models and sparse activation architectures are constantly emerging, driving an exponential increase in computing power demand. Traditional single-node architectures are no longer sufficient to meet the core requirements of efficient training and inference for large models. During large-scale model training, the density and real-time requirements of data interaction increase dramatically. In particular, the Expert Parallelism (EP) processing mode for MoE model training requires distributing the weights of different expert models to different computing units for parallel computation, resulting in a geometric increase in communication volume between nodes and computing units. Communication efficiency has become a core bottleneck restricting the performance improvement of large-scale model training. Current mainstream cluster architectures based on stacked independent servers typically experience inter-node communication latency exceeding 2 ms when facing the TB-level single data transmission requirements of large-scale model training. Such communication latency has reached a level that cannot be masked by the computation process, causing a large amount of computing power to be consumed by communication time, and the computing power efficiency cannot be fully released. At the same time, due to the dynamic imbalance of model computing tasks, the problem of uneven allocation of node resources is common. Popular nodes are overloaded, leading to performance bottlenecks, while unpopular nodes are idle, resulting in waste. Ultimately, the overall training efficiency drops by more than 30%, significantly prolonging the iteration cycle of large models and further exacerbating the problem of low model iteration efficiency.

[0003] Currently, the field of computing cluster interconnection is accelerating its transformation from the traditional "block-style server stacking" to a high-density, highly collaborative hierarchical integrated architecture, forming a clear evolutionary trend of a three-level interconnection cluster of "chiplet devices-nodes-supernodes". However, existing technical solutions still have significant drawbacks: First, the multi-protocol hybrid architecture in the three-level system suffers from poor protocol compatibility, multiple data forwarding layers, and low resource pooling efficiency; Second, in the unified bus interconnection scenario, high-density interconnection links are prone to parasitic capacitance, signal crosstalk, and other problems. Furthermore, the bandwidth allocation between chiplet devices, nodes, and supernodes lacks a dynamic adaptation mechanism, leading to bandwidth mismatch and difficulty in simultaneously adapting to the mixed traffic patterns of locally intensive data interaction and long-distance sparse data transmission during large model training, which severely restricts the improvement of the overall cluster performance. Collective Communication (CPC) operations, acting as the "nervous system" for collaborative computing clusters, directly determine the extent to which cluster computing power is released through their transmission efficiency and reliability. It is a key supporting technology for ensuring efficient training of large models, especially in three-tier interconnected clusters of "chip-node-supernode," where the hierarchical adaptability of CPC is even more crucial. Current mainstream CPC solutions are represented by libraries such as HCCL, NCCL, and OpenMPI. Through optimization and even step reconstruction of basic communication primitives like AllReduce, AllGather, and All-to-All, they have achieved certain performance improvements in traditional server cluster scenarios. However, in the unified bus scenario of three-tier interconnected chip-node-supernode computing devices, many adaptability shortcomings still exist. On the one hand, existing aggregated communication solutions are not fully adapted to the topology characteristics of a unified bus, nor have they been specifically optimized for the three-tier link architecture of "independent high-speed links within core devices - intra-node links - inter-supernode links." This makes it impossible to dynamically schedule communication paths and accurately allocate bandwidth resources according to data transmission requirements. Especially in the frequent All-to-All communication operations during large model training, bandwidth utilization is generally below 60%, severely impacting data transmission efficiency. On the other hand, in ultra-large-scale three-tier interconnected cluster scenarios, the probability of link failures increases significantly. Traditional aggregated communication solutions often employ retransmission or backup link switching mechanisms, resulting in fault recovery times of up to hours, which can easily lead to frequent interruptions of training tasks, causing a significant waste of computing power and time. Furthermore, the resource contention between communication and computing tasks has not been effectively resolved. Communication processes can easily consume excessive Stream Multiprocessor (SM) resources, leading to extended waiting times for computing tasks and further reducing the overall computing efficiency of the cluster.

[0004] In summary, the contradiction between the current high computing power and high communication density requirements for large-scale model training and the insufficient adaptability of computing cluster interconnection technology and the lack of scenario-based optimization of aggregated communication solutions is becoming increasingly prominent. Especially in the three-level system scenario of "chip device-node-super node" interconnection of unified bus, existing technical solutions are unable to simultaneously meet the core requirements of high bandwidth, low latency, high reliability and efficient resource utilization between each level. There is an urgent need for a targeted aggregated communication method that can improve the efficiency of cluster collaborative computing and adapt to the actual needs of large-scale model training. Summary of the Invention

[0005] To address the technical problems existing in the prior art, this invention provides a collective communication method for chip device clusters with unified bus interconnection. It can adapt to a three-level link architecture, improve communication efficiency and bandwidth utilization, enhance cluster reliability and resource utilization, and effectively meet the high computing power and high communication density requirements of large-scale large model training.

[0006] The objective of this invention can be achieved by adopting the following technical solutions:

[0007] A method for aggregated communication of a cluster of chip devices using a unified bus interconnect, the method comprising:

[0008] S1. Collect the hierarchical topology and link information of the chip device cluster, and construct a global topology information database based on the collected hierarchical topology and link information. The chip device cluster is a three-level interconnected cluster of chip device-node-supernode.

[0009] S2. Optimize the aggregate communication primitives by optimizing the core-level primitives, node-level primitives, and supernode-level primitives to adapt to the three-level link architecture of the core-level device cluster.

[0010] S3. Construct a dynamic fault tolerance and resource scheduling mechanism, deploy link monitoring units at each level of link interface to monitor link status indicators in real time, trigger alarms when link failure is detected, select the optimal backup link based on the global topology information database and update the communication path, and use dynamic resource scheduling algorithm to allocate computing power and communication resources.

[0011] S4. Based on the requirements of the aggregated communication task, query the corresponding level topology and link information from the global topology information database, call the optimized aggregated communication primitives, and dynamically adjust the communication path and bandwidth allocation according to the link dynamic parameters.

[0012] Specifically, step S1 includes:

[0013] S11. Collect core-level information, obtain static topology information of core devices through the core device management unit, detect dynamic parameters of the link in real time, and report static topology information and dynamic parameters to the node management controller.

[0014] S12. Collect node-level information, initiate intra-node topology probing based on the node management controller, send probe messages to all core devices through the unified bus, collect parameters of core devices within the node, and construct a node-level topology graph.

[0015] S13. Collect supernode-level information, initiate cross-node / supernode topology probing based on the supernode management node, send probe messages through the inter-node interconnection links, collect parameters of all nodes, and construct a supernode-level global topology graph.

[0016] S14. Collect core-level information, node-level information, and supernode-level information through the cluster management plane, verify and integrate them to build a global topology information database.

[0017] Specifically, the core device management unit uses an MCU chip, the node management controller uses an FPGA chip, the super node management node uses an ARM server, and the cluster management plane uses a distributed database to store the global topology information.

[0018] Specifically, step S2 includes:

[0019] S21. Core-level primitive optimization: Core-level primitives adopt a direct memory mapping mechanism and a lock-free queue mode. Data inside the core is directly transmitted through a hybrid bonded link, and the communication throughput is improved through the lock-free queue mode.

[0020] S22. Node-level primitive optimization: The intra-node aggregated communication adopts a weight-based bus bandwidth dynamic allocation algorithm, which allocates bandwidth weights according to the communication data volume of each core device, adopts an aggregation transmission strategy for messages less than the aggregation threshold, and plans communication paths based on the shortest path algorithm using the node-level topology graph.

[0021] S23. Supernode-level primitive optimization: Combining the dynamic attributes of links in the global topology information database, low-latency link transmission is used for dense traffic, high-bandwidth link transmission is used for sparse traffic, and multi-path parallel transmission mode is used for primitives between supernodes.

[0022] Specifically, the intra-node aggregated communication employs a weighted dynamic bus bandwidth allocation algorithm, allocating bandwidth weights based on the communication data volume of each core device, and using an aggregation transmission strategy for messages smaller than the aggregation threshold, including:

[0023] Bandwidth weights are allocated based on the communication data volume of each chip device, with a weight range of 0.1-0.8. The weights are dynamically adjusted according to real-time communication needs. An aggregation threshold of 8 MB is set to aggregate small packets smaller than 8 MB.

[0024] Specifically, step S3 includes:

[0025] S31. Deploy link monitoring units on the high-speed link interface, node bus interface, and supernode interconnection interface of the core device to monitor the link bit error rate and latency jitter in real time. Trigger an alarm when the bit error rate and latency jitter exceed the preset values.

[0026] S32. After receiving the alarm, the system selects the optimal backup link in real time based on the global topology information database, updates the communication path, and pushes the new path information to the relevant computing units.

[0027] S33. Real-time acquisition of computing load and communication requirements of computing units, and allocation of computing and communication resources using dynamic resource scheduling algorithms, adjusting the resource allocation ratio of computing units in real time.

[0028] Specifically, step S4 includes:

[0029] S41. Based on the requirements of the aggregated communication task, the communication task is decomposed, and a query request is initiated to the global topology information database of the cluster management plane through the RDMA interface. The request queries the topology and link information of the corresponding level. The topology and link information of the corresponding level includes: the real-time bandwidth of the dual GPU link within the core, the current load of the unified bus within the node, and the latency and utilization of each optical communication link between supernodes.

[0030] S42. Based on the retrieved topology and link information and communication task type, call the optimized set of communication primitives at the core level, node level, and supernode level.

[0031] S43. During the execution of the aggregated communication, the link monitoring unit collects the dynamic parameters of each level of the link at a preset time interval. The dynamic parameters of each level of the link include: bandwidth utilization, latency and packet loss rate. The communication path and bandwidth allocation are dynamically adjusted according to the changes in the parameters of each level of the link.

[0032] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0033] This invention discloses a cluster communication method for interconnecting chip-level devices using a unified bus interconnect. A global topology information database is constructed based on collected hierarchical topology and link information. Chip-level, node-level, and supernode-level primitives are optimized. An alarm is triggered when a link failure is detected. The optimal backup link is selected based on the global topology information database, and the communication path is updated. A dynamic resource scheduling algorithm is used to allocate computing and communication resources. Through hierarchical primitive optimization and path planning, intra-chip communication latency is reduced to the microsecond level, improving intra-node communication bandwidth utilization and all-to-all primitive bandwidth utilization between supernodes, thereby significantly improving the overall cluster communication efficiency. A real-time link monitoring and rapid fault tolerance mechanism enables millisecond-level fault recovery, avoiding frequent interruptions to training tasks and improving cluster operational stability. Through collaborative scheduling of communication and computing resources, resource contention is effectively alleviated, improving the overall utilization of computing and communication resources and adapting to the high computing power and high communication density requirements of large-scale model training. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0035] Figure 1 This is a flowchart of a cluster communication method for a chip device cluster applicable to unified bus interconnection, as described in an embodiment of the present invention.

[0036] Figure 2 This is a schematic diagram of the training cluster topology of the unified bus interconnected "chip device-node-supernode" three-level system network in an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram illustrating the hierarchical invocation implementation of the optimized All-to-All set communication primitive in the "chip device-node-supernode" three-level system network in this embodiment of the invention. Detailed Implementation

[0038] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments, and the implementation of the present invention is not limited thereto. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Example 1:

[0040] This invention discloses a cluster interconnection and communication method for chip devices applicable to unified bus interconnection, applied to a three-level interconnection cluster of "chip device-node-supernode". Chip computing is the underlying core of the three-level system. Multiple chip devices are aggregated through a high-speed bus to form computing nodes, realizing the initial integration of computing resources within the node. Supernodes, on the other hand, integrate multiple computing nodes with high density through a scale-up architecture, achieving full interconnection and collaboration between nodes. A single supernode can support tens to hundreds of xPUs to form a large-scale computing pool to match the ultra-high computing power requirements of training and pushing trillion-level large models. Currently, three-level interconnection clusters mainly adopt a layered technical path: within chip devices, advanced packaging technologies such as silicon interposers and hybrid bonding are used to enhance high-speed link performance, achieving micron-level low-latency interconnection between xPUs; high-speed interconnection networks are built within and between nodes using protocols such as PCIe 5.0 and NVLink 4.0; within supernodes, non-blocking, full-to-full-topology dedicated extended networks (such as unified buses) are used to achieve non-converging interconnection between nodes, and optical communication technology is introduced across supernode scenarios to ensure high-speed and low-power transmission.

[0041] Specifically, the unified bus includes one or more of the following: PCIe 5.0 and above, NVLink bus, and UB 2.0 bus. A chip computing device refers to a computing unit that integrates two parallel processing units (xPUs) in a single device, with a high-speed interconnect link between the two xPUs that is independent of the communication link between the chip computing devices. Specifically, the xPU of a chip device is one or more heterogeneous combinations of GPU, TPU, or NPU.

[0042] like Figure 1 The diagram shows a flowchart of a clustered communication method for chip device clusters applicable to unified bus interconnect. The present invention provides a clustered communication method for chip device clusters applicable to unified bus interconnect, comprising the following steps:

[0043] S1. Collect the hierarchical topology and link information of the chip device cluster, and construct a global topology information database based on the collected hierarchical topology and link information. The chip device cluster is a three-level interconnected cluster of chip device-node-supernode.

[0044] This embodiment of the chip device cluster interconnection and aggregation communication method applicable to unified bus interconnection can be applied to large model training clusters consisting of two supernodes. Each supernode contains eight computing nodes, each computing node integrates eight chip devices, and each chip device integrates two NPUs (a specific implementation of xPU). High-speed interconnection between NPUs within a chip is achieved through independent high-speed links. Interconnection within nodes uses the UB 2.0 unified bus, while non-convergent interconnection between nodes within supernodes is achieved using the UB plane. Interconnection across supernodes uses optical communication links. Figure 2The diagram shows a training cluster topology of a three-tiered network architecture consisting of "chip device-node-supernode" interconnected by a unified bus.

[0045] Since the scale and network topology of the chip computing devices participating in each collective communication may differ, and there may be non-logically adjacent devices involved, such as chip computing devices 0-7 and chip computing devices 128-135, a total of 16 chip computing devices participating in the collective communication operation, it is necessary to perceive the dynamic topology changes in real time in order to adapt to the flexible expansion scenarios of nodes and supernodes.

[0046] S11. Collect chip device information, obtain static topology information of chip devices through chip device management unit, detect dynamic parameters of link in real time, and report static topology information and dynamic parameters to node management controller.

[0047] In this embodiment, the core device management unit uses an MCU chip. The core device management unit loads a Local Topology Description Table (LTDT) from the factory-installed firmware. The LTDT contains parameters such as the link identifiers, default bandwidth (200 GB / s), and latency (50 ns) of the two NPUs within the core device. Based on the LTDT, the unit obtains static parameters such as the link identifiers, bandwidth, and latency of the dual xPUs within the core device. Through dedicated probe frames for the internal high-speed links, dynamic parameters such as link connectivity and current bandwidth utilization are detected in real time and reported to the node management controller. The MCU reads real-time data from the link monitoring unit via the I2C interface and reports the dynamic parameters of the links within the core device to the node management controller every 10 ms.

[0048] S12. Collect node information, initiate intra-node topology probing based on the node management controller, send probe messages to all core devices through the unified bus, collect parameters such as the interconnection relationship of core devices within the node, link real-time latency, packet loss rate, etc., and construct a node-level topology graph.

[0049] In this embodiment, the node management controller uses an FPGA chip to send UB probe messages to the eight core devices in the node every 50ms. It collects parameters such as the interconnection relationship of each core device, the real-time latency of the link (e.g., 30-80 ns), and the packet loss rate (less than 0.1%), constructs a node topology map, and stores it in local DDR memory.

[0050] S13. Collect supernode information, initiate cross-node / supernode topology probe based on the supernode management node, send probe messages through the inter-node interconnection links, collect parameters of all nodes, and construct a supernode-level global topology graph.

[0051] In this embodiment, the supernode management node adopts an ARM server and runs an improved LLDP protocol. Every 100ms, it sends probe messages to the 8 compute nodes within the supernode to collect node location information, link bandwidth (UB plane link bandwidth 4 TB / s), latency (inter-node latency 200 ns), hop count and other parameters to construct a global topology map at the supernode level. When probing across supernodes, probe messages are sent through optical communication links to collect link parameters between supernodes (bandwidth 10 TB / s, latency 1 us).

[0052] S14. Collect core-level information, node-level information, and supernode-level information through the cluster management plane, verify and integrate them to build a global topology information database.

[0053] In this embodiment, the cluster management plane uses a distributed database to store the global topology information database, storing the complete interconnection relationship of the three-level system, the static and dynamic attributes of the links, supporting on-demand queries by computing units, supporting queries at the level of 100,000 QPS, and computing nodes can quickly query topology and link information through the RDMA interface with a query latency of less than 10 µs.

[0054] S2. Optimize the collection communication primitives, including the core-level primitives, node-level primitives, and supernode-level primitives, to adapt to the three-level link architecture of the core-level device cluster.

[0055] Following step S1, the number of independent high-speed links is equal to the number of core devices, satisfying both non-blocking, full-to-full interconnection between core devices via a unified bus and high-speed direct connection of xPUs within each core device. To this end, the optimal communication algorithm can be adaptively selected based on factors such as the amount of communication data and the types of communication aggregated, and multi-link load balancing strategies can be applied to further improve the communication throughput of both independent high-speed links and the unified bus. Specifically, step S2 can be further subdivided into the following steps:

[0056] S21. Core-level primitive optimization: Core-level primitives adopt the direct memory mapping mechanism (DMM) and lock-free queue mode. Data inside the core is directly transmitted through the hybrid bonded link, which directly maps the specific memory region of the source core to the address space of the target core. The communication throughput is improved by using the lock-free queue mode.

[0057] Inter-core communication efficiency has become a key bottleneck restricting overall system performance, especially in multi-core heterogeneous computing scenarios. The data exchange efficiency between xPUs (such as NPUs and GPUs) within a core directly affects the performance of critical applications such as AI training and inference. Traditional inter-core communication often relies on CPU relays or standard interconnect protocols, resulting in numerous data copies and high communication latency, making it difficult to meet the low latency and high bandwidth requirements of high-performance computing. For the independent high-speed link between dual xPUs within a core, lightweight AllReduce and AllGather primitives are customized, employing a "direct memory mapping + lock-free queue" transmission mode. The Direct Memory Mapping (DMM) mechanism bypasses traditional memory copy paths, achieving direct mapping of memory spaces between cores. A high-speed interconnect link built through hybrid bonding technology directly maps specific memory regions of the source core to the address space of the target core. Data does not need to pass through CPU memory relays, achieving "zero-copy" or "single-copy" transmission, reducing the number of data copies and lowering intra-core communication latency to the microsecond level. Lock-free queue transmission eliminates latency jitter caused by lock contention, ensuring deterministic and low-latency communication. It employs techniques such as circular buffers and atomic operations to achieve efficient data exchange, avoiding context switching and waiting latency caused by lock contention, and improving communication throughput.

[0058] Specifically, the chip-level AllReduce primitive adopts a direct memory mapping (DMM) and lock-free queue mode. Data between NPUs is directly transmitted through a hybrid bonded link without the need for CPU relay, reducing the number of data copies from 3 to 1. The measured communication latency of AllReduce within the chip is 80 ns.

[0059] S22. Node-level primitive optimization: The intra-node aggregated communication adopts a weighted bus bandwidth dynamic allocation algorithm. The bandwidth weight is allocated according to the communication data volume of each core device. For messages less than the aggregation threshold, an aggregation transmission strategy is adopted. The communication path is planned based on the node-level topology graph using the Dijkstra algorithm or the A* algorithm.

[0060] Intra-node aggregated communication employs a "dynamic bus bandwidth allocation + message aggregation" strategy. The bus bandwidth allocation is dynamically adjusted based on the communication needs of each core device, and small-sized messages are aggregated for transmission, reducing bus transmission overhead. Simultaneously, based on a node-level topology graph, a shortest path algorithm is used to plan communication paths. In the graph, nodes represent core devices, edges represent communication links, and edge weights reflect link latency or bandwidth characteristics. Topology graph information includes: physical connections: the actual interconnection structure between core devices (e.g., Mesh, Ring, Crossbar, Halving-Doubling, etc.); link attributes: parameters such as bandwidth, latency, and error rate for each link; and dynamic status information: real-time link load, congestion status, etc., with communication paths planned using a shortest path algorithm based on the node-level topology graph.

[0061] In this embodiment, intra-node aggregated communication employs a weighted dynamic bus bandwidth allocation algorithm. Bandwidth weights are allocated based on the communication data volume of each core device, with a weight range of 0.1-0.8. These weights are dynamically adjusted according to real-time communication requirements to adapt to different application scenarios. An aggregation threshold of 8 MB is set to balance aggregation benefits with latency overhead for small packets smaller than 8 MB. Depending on network load and communication mode, the aggregation threshold can be dynamically adjusted within a certain range (e.g., 4-16 MB), reducing intra-node bus transmission overhead by 40% and significantly improving communication efficiency.

[0062] Specifically, the Dijkstra algorithm or A* algorithm is used to calculate the shortest path from the source core to the target core. Improved Dijkstra and A* algorithms are used to calculate the shortest path from a node to all other nodes in a weighted directed or undirected graph. In this embodiment, the weights represent the time taken for a single communication between core computing devices. The specific steps are as follows: 1. Set the distance from the source node to itself to 0, and the distance to all other nodes to infinity; 2. Maintain two sets: S (nodes with determined shortest paths) and U (nodes with undetermined shortest paths); 3. Select a vertex u from set U that is closest to the source node; 4. Add vertex u to set S, traverse all neighbors v of u, and if "the distance from the source node to u + the edge weight from u to v" is less than "the current distance from the source node to v", update the distance of v; 5. Repeat steps 3 and 4 until all nodes are added to set S.

[0063] S23. Supernode-level primitive optimization: Combining the dynamic attributes of links in the global topology information database, low-latency link transmission is used for dense traffic, high-bandwidth link transmission is used for sparse traffic, and multi-path parallel transmission mode is used for primitives between supernodes.

[0064] For communication links, the time taken for a single communication operation typically follows the classic Hockney model:

[0065] T = alpha + n / beta;

[0066] Where alpha is the physical latency of the communication link itself, determined by factors such as link materials, communication module hardware, and communication protocol, and is independent of the amount of data transmitted; n is the number of bytes involved in the transmission; and beta is the link bandwidth value. For any communication link, low latency and high bandwidth are difficult to achieve simultaneously. Generally, a link with a physical latency alpha of less than 100 microseconds is called a low-latency link, suitable for frequent transmission of small data packets (KB level); a link with a physical latency beta of greater than 100 GB / s is called a high-bandwidth link, suitable for sparse transmission of large data volumes (MB, GB level).

[0067] Specifically, the supernode-to-supernode communication adopts a "segmented transmission + traffic scheduling" strategy. Combining the dynamic attributes of links in the global topology information database, it uses low-latency links for dense traffic and high-bandwidth links for sparse traffic. The supernode-to-supernode All-to-All primitive adopts a "4-path parallel transmission" mode, selecting 4 optimal links to transmit data in parallel based on the global topology information database. The bandwidth utilization of supernode-to-supernode All-to-All is increased from 60% in the existing technology to 78%.

[0068] S3. Construct a dynamic fault tolerance and resource scheduling mechanism. Deploy link monitoring units at each level of link interface to monitor link status indicators in real time. When a link failure is detected, trigger an alarm. Select the optimal backup link based on the global topology information database and update the communication path. Use a dynamic resource scheduling algorithm to allocate computing power and communication resources to avoid resource contention.

[0069] In ultra-large-scale three-level interconnected cluster scenarios, the probability of link failures is inevitably on the rise. Traditional aggregated communication solutions often use retransmission or backup link switching as their fault tolerance mechanism. Fault recovery time can be as long as hours, which can easily lead to frequent interruptions of training tasks and a large waste of computing power and time. Therefore, it is necessary to build a dynamic fault tolerance and resource scheduling mechanism.

[0070] S31. Real-time link monitoring: Deploy link monitoring units on the high-speed link interface, node bus interface, and supernode interconnection interface of the chip device to monitor the link bit error rate and latency jitter in real time. When the bit error rate and latency jitter exceed the preset values, an alarm is triggered and reported to the cluster management plane.

[0071] In this embodiment, the link monitoring unit uses a dedicated ASIC chip to monitor parameters such as link bit error rate and latency jitter in real time. When the bit error rate exceeds 10% or the latency jitter exceeds 50ns, an alarm is triggered, and the fault information is reported to the global topology information database through a dedicated control channel. An abnormal state can be detected using a sliding window algorithm, and an alarm is triggered when the indicators exceed the threshold for three consecutive periods.

[0072] S32. Rapid fault recovery: After receiving an alarm in the cluster management plane, the optimal backup link is selected in real time based on the global topology information database, the communication path is updated, and the new path information is pushed to the relevant computing units to achieve millisecond-level fault recovery.

[0073] In this embodiment, after receiving an alarm, the cluster management plane filters backup links using the shortest path algorithm, updates the communication path, and pushes it to the computing unit to achieve millisecond-level fault recovery.

[0074] S33. Communication-computing resource collaborative scheduling: Real-time collection of computing load and communication requirements of computing units, and the use of dynamic resource scheduling algorithms to allocate computing and communication resources, adjusting the allocation ratio of computing and communication resources in the NPU computing core in real time. This avoids excessive consumption of computing resources by communication and achieves efficient collaborative operation between communication and computing.

[0075] In this implementation, a reinforcement learning algorithm is used to achieve dynamic resource scheduling, with the optimization goals of minimizing communication latency and maximizing computing power utilization. The allocation ratio of NPU computing core resources is adjusted in real time. The computing load (CPU utilization, memory bandwidth, cache hit rate, etc.) and communication requirements (data packet queue length, communication waiting time) of each computing unit are collected periodically. When the communication task load is high and exceeds a threshold L, resource reallocation is triggered, adjusting the computing core resource allocation ratio from the default 8:2 (computation:communication) to 6:4, prioritizing the completion of communication tasks and avoiding excessive computing resource consumption by communication. After the communication load decreases, the ratio is gradually restored to the default ratio to avoid frequent switching.

[0076] S4. Collective communication execution and dynamic optimization: Based on the requirements of the collective communication task, query the corresponding level topology and link information from the global topology information database, call the optimized collective communication primitives in a hierarchical manner, and dynamically adjust the communication path and bandwidth allocation according to the link dynamic parameters to ensure optimal communication performance.

[0077] In this embodiment, a large-scale MoE model training scenario with hundreds of billions of parameters (1.2T model parameters, 128 experts, batch size of 1024, and input sequence length of 512) is used to demonstrate the improvement in communication efficiency, bandwidth utilization, cluster reliability, and resource utilization through specific operation examples and measured data. The specific steps are as follows:

[0078] S41. Based on the requirements of the aggregated communication task, the communication task is decomposed, and a query request is initiated to the global topology information database of the cluster management plane through the RDMA interface. The request queries the topology and link information of the corresponding level. The topology and link information of the corresponding level includes: the real-time bandwidth of the dual GPU link within the core, the current load of the unified bus within the node, and the latency and utilization of each optical communication link between supernodes.

[0079] During the large model training process, when executing the All-to-All aggregate communication task of MoE expert parallel training, distributing the intermediate calculation results of 128 expert modules to the xPU of the corresponding chip device for aggregation calculation, the NPU of the chip device first initiates a query request to the global topology information database of the cluster management plane through the RDMA interface. The query content includes: the real-time bandwidth of the dual GPU link within the chip, the current load of the unified bus within the node, and the latency and utilization of each optical communication link between supernodes.

[0080] S42. Based on the retrieved topology and link information and communication task type (All-to-All high-frequency communication), call the optimized set communication primitives at the core level, node level, and supernode level.

[0081] like Figure 3 As shown, this invention's optimized All-to-All set communication primitive is implemented in a hierarchical network of "chip device-node-supernode". A specific example is as follows:

[0082] The optimized aggregation communication primitives at the core level are invoked. NPU 1 in core device 8 needs to transmit the local 2GB expert calculation results to NPU 2 in the same core for initial aggregation. The lightweight AllReduce primitive customized by S21 is invoked, and the "Direct Memory Mapping (DMM) + Lock-Free Queue" mode is adopted. The data is transmitted directly through an independent high-speed link without the need for CPU relay. The latency of a single 2GB data transmission is 80ns, which is 75% lower than the existing HCCL primitive (traditional transmission mode, latency 320ns) and the communication efficiency is improved by 3 times. The measured bandwidth utilization of the link within the core is 98%, which is 33 percentage points higher than the existing technology (bandwidth utilization 65%), giving full play to the low latency advantage of the independent high-speed link within the core.

[0083] The optimized node-level aggregation communication primitive is invoked. Eight core devices within node 1 need to transmit their aggregated 16GB data (totaling 128GB) to other core devices within the node. The optimized node-level primitive S22 is invoked, employing a "dynamic bus bandwidth allocation + message aggregation" strategy. Bandwidth weights are allocated based on the communication data volume of each core device (core device 8 has a weight of 0.7, other core devices have weights of 0.05-0.1). Small messages below 64KB (32,000 in total) are aggregated into large 128KB messages for transmission. Actual test data shows that the total latency for 128GB data transmission within the node is 1.2ms, a 73.3% reduction compared to existing technologies (4.5ms latency). The measured unified bus bandwidth utilization rate within the node is 87%, a 29 percentage point improvement compared to existing technologies (58% bandwidth utilization), significantly enhancing the efficiency and bandwidth utilization of centralized communication within the node.

[0084] By calling the optimized supernode-level aggregation communication primitive, Node 1 needs to transmit the aggregated 128GB data within the node to the 8 compute nodes of Supernode 3. The optimized supernode-level All-to-All primitive S23 is called, employing a "segmented transmission + multi-path parallel" strategy to divide the 128GB data into 4 segments (32GB each), which are transmitted in parallel through 4 optimal links (2 UB plane links and 2 optical links). Actual test data shows that the latency for a single 128GB data transmission across supernodes is 8.5ms, a 73.4% reduction compared to existing technologies (32ms latency); the measured bandwidth utilization of the links between supernodes is 79%, a 19 percentage point improvement compared to existing technologies (60% bandwidth utilization), thus solving the bandwidth bottleneck problem of high-frequency All-to-All communication in large models.

[0085] S43. During the execution of the aggregated communication, the link monitoring unit collects the dynamic parameters of each level of the link at a preset time interval. The dynamic parameters of each level of the link include: bandwidth utilization, latency and packet loss rate. The communication path and bandwidth allocation are dynamically adjusted according to the changes in the parameters of each level of the link.

[0086] During the execution of aggregated communication, the link monitoring unit collects dynamic parameters (bandwidth utilization, latency, packet loss rate) of each level of link every 1ms and feeds them back to the aggregated communication strategy module in real time. The strategy module dynamically adjusts the communication path and bandwidth allocation based on parameter changes. Specific examples of dynamic optimization for bandwidth utilization and resource utilization are given below: For dynamic optimization of bandwidth utilization, when the utilization of one of the optical links from supernode 1 to supernode 3 rises to 85%, the strategy module switches the 32GB segmented data of that link to a backup optical link with a utilization of 48% in real time. After the adjustment, the utilization rates of the two optical links are 62% and 65% respectively, both within the optimal load range (40%-70%), avoiding increased transmission latency caused by link saturation and ensuring that the bandwidth utilization between supernodes remains stable between 75%-80%, continuously improving bandwidth utilization efficiency.

[0087] Dynamic optimization of resource utilization: During communication, the cluster management plane collects the NPU streaming processor resource usage every 5ms. When it detects that the proportion of streaming processor resources occupied by NPU 1 in chip device 8 due to communication tasks exceeds the threshold of 50%, the S33 communication-computing resource collaborative scheduling algorithm immediately adjusts the resource allocation ratio, changing the streaming processor resource allocation ratio of NPU 1 from 6:4 (computing:communication) to 7:3. After the adjustment, the waiting time of NPU 1's computing tasks decreases from 180ns to 60ns, and the computing resource utilization rate increases from 62% to 83%. At the same time, the communication task latency only increases by a negligible 5ns, achieving efficient collaborative utilization of communication and computing resources and avoiding the overall performance decline caused by a single task occupying too many resources.

[0088] S44. After the cluster communication task is completed, the computing unit verifies the received data to confirm its integrity. At the same time, it feeds back the key parameters of this communication (such as latency, bandwidth utilization, resource utilization, etc.) to the cluster management plane for subsequent primitive optimization and strategy adjustment.

[0089] In this embodiment, the model training throughput reached 185 TFLOPS, a 35% improvement over the existing HCCL solution's 137 TFLOPS; the single-round training cycle was shortened from 28 hours to 18 hours, improving training efficiency by 35.7%; the overall cluster computing resource utilization rate was measured at 84%, a 26 percentage point improvement over existing technologies, and the communication resource utilization rate remained stable at around 80%. This fully demonstrates that this method can effectively adapt to the high computing power and high communication density requirements of large-scale models, breaking through the performance bottleneck of existing technologies.

[0090] In this embodiment, during the communication process, the system collects dynamic link parameters in real time, such as end-to-end latency (RTT), available bandwidth, packet loss rate, and jitter. Each computing unit reports the link status to the aggregated communication strategy module. The communication strategy module aggregates global information and updates the topology information database. Based on the latest link status, it dynamically adjusts the communication strategy. When the performance of a certain path degrades, it switches to a backup path and adjusts the data stream bandwidth allocation according to the link quality to ensure optimal communication performance.

[0091] In summary, this embodiment provides a method for aggregated communication of chip-level device clusters with unified bus interconnection. By constructing a hierarchical topology and link information acquisition module, and through firmware pre-configuration, hierarchical detection, and centralized management, it accurately collects topology and link information of the three-level architecture. A hierarchical aggregated communication primitive optimization strategy is designed, customizing optimization primitives for the characteristics of chip-level, node-level, and supernode-level links to plan the optimal communication path. A dynamic fault-tolerance and resource scheduling mechanism is constructed to achieve real-time link monitoring, millisecond-level fault recovery, and collaborative scheduling of communication and computing resources. This invention adapts to three-level link architectures, significantly improves communication efficiency and bandwidth utilization, enhances cluster reliability and resource utilization, and can effectively meet the high computing power and high communication density requirements of large-scale model training.

[0092] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for aggregated communication of chip device clusters applicable to unified bus interconnection, characterized in that, Includes the following steps: S1. Collect the hierarchical topology and link information of the chip device cluster, and construct a global topology information database based on the collected hierarchical topology and link information. The chip device cluster is a three-level interconnected cluster of chip device-node-supernode. S11. Collect core-level information, obtain static topology information of core devices through the core device management unit, detect dynamic parameters of the link in real time, and report static topology information and dynamic parameters to the node management controller. S12. Collect node-level information, initiate intra-node topology probing based on the node management controller, send probe messages to all core devices through the unified bus, collect parameters of core devices within the node, and construct a node-level topology graph. S13. Collect supernode-level information, initiate cross-node / supernode topology probing based on the supernode management node, send probe messages through the inter-node interconnection links, collect parameters of all nodes, and construct a supernode-level global topology graph. S14. Collect core-level information, node-level information, and supernode-level information through the cluster management plane, verify and integrate them to build a global topology information database; S2. Optimize the aggregate communication primitives by optimizing the core-level primitives, node-level primitives, and supernode-level primitives to adapt to the three-level link architecture of the core-level device cluster. S21. Core-level primitive optimization: Core-level primitives adopt a direct memory mapping mechanism and a lock-free queue mode. Data inside the core is directly transmitted through a hybrid bonded link, and the communication throughput is improved through the lock-free queue mode. S22. Node-level primitive optimization: The intra-node aggregated communication adopts a weight-based bus bandwidth dynamic allocation algorithm. The bandwidth weight is allocated according to the communication data volume of each core device. For messages less than the aggregation threshold, an aggregation transmission strategy is adopted. The Dijkstra algorithm or A* algorithm is used to plan the communication path. S23. Supernode-level primitive optimization: Combining the dynamic attributes of links in the global topology information database, low-latency link transmission is used for dense traffic, high-bandwidth link transmission is used for sparse traffic, and multi-path parallel transmission mode is used for primitives between supernodes. S3. Construct a dynamic fault tolerance and resource scheduling mechanism, deploy link monitoring units at each level of link interface to monitor link status indicators in real time, trigger alarms when link failure is detected, select the optimal backup link based on the global topology information database and update the communication path, and use dynamic resource scheduling algorithm to allocate computing power and communication resources. S4. Based on the requirements of the aggregated communication task, query the corresponding level topology and link information from the global topology information database, call the optimized aggregated communication primitives in a hierarchical manner, and dynamically adjust the communication path and bandwidth allocation according to the dynamic parameters of the link.

2. The aggregated communication method for a chip device cluster applicable to unified bus interconnection as described in claim 1, characterized in that, The core device management unit uses an MCU chip, the node management controller uses an FPGA chip, the super node management node uses an ARM server, and the cluster management plane uses a distributed database to store the global topology information.

3. The aggregated communication method for a chip device cluster applicable to unified bus interconnection as described in claim 1, characterized in that, The intra-node aggregated communication employs a weighted dynamic bus bandwidth allocation algorithm. Bandwidth weights are allocated based on the communication data volume of each core device. For messages smaller than the aggregation threshold, an aggregation transmission strategy is used, including: Bandwidth weights are allocated based on the communication data volume of each chip device, with a weight range of 0.1-0.

8. The weights are dynamically adjusted according to real-time communication needs. An aggregation threshold of 8 MB is set, and packets smaller than 8 MB are aggregated.

4. The aggregated communication method for a chip device cluster with unified bus interconnection according to claim 1, characterized in that, Step S3 specifically includes: S31. Deploy link monitoring units on the high-speed link interface, node bus interface, and supernode interconnection interface of the core device to monitor the link bit error rate and latency jitter in real time. Trigger an alarm when the bit error rate and latency jitter exceed the preset values. S32. After receiving the alarm, the system selects the optimal backup link in real time based on the global topology information database, updates the communication path, and pushes the new path information to the relevant computing units. S33. Real-time acquisition of computing load and communication requirements of computing units, and allocation of computing and communication resources using dynamic resource scheduling algorithms, adjusting the resource allocation ratio of computing units in real time.

5. The aggregated communication method for a chip device cluster with unified bus interconnection according to claim 4, characterized in that, Step S4 specifically includes: S41. Based on the requirements of the aggregated communication task, the communication task is decomposed, and a query request is initiated to the global topology information database of the cluster management plane through the RDMA interface. The request queries the topology and link information of the corresponding level. The topology and link information of the corresponding level includes: the real-time bandwidth of the dual GPU link within the core, the current load of the unified bus within the node, and the latency and utilization of each optical communication link between supernodes. S42. Based on the retrieved topology and link information, and in conjunction with the communication task type, call the optimized set of communication primitives at the core level, node level, and supernode level. S43. During the execution of the aggregated communication, the link monitoring unit collects the dynamic parameters of each level of the link at a preset time interval. The dynamic parameters of each level of the link include: bandwidth utilization, latency and packet loss rate. The communication path and bandwidth allocation are dynamically adjusted according to the changes in the parameters of each level of the link.

Images