Interconnection architecture applied to accelerometer cluster

Abstract

The invention discloses an interconnection architecture applied to an accelerator cluster, which can be applied to any interconnection architecture capable of prolonging a transmission distance, reducing signal loss, reducing transmission cost and power consumption and having exchange capacity, namely any architecture capable of connecting two ends in the accelerator cluster to serve as interconnection components. The characteristics of data routing and forwarding at the two connected ends are utilized, the expandability and dynamic reconfigurability of the accelerator cluster are enhanced, the resource utilization rate is improved, and the requirement of the accelerator cluster for a high-capacity switch can be directly reduced. In addition, an interconnection network deployment mode of independent networking can be achieved without using a switch, and collaborative / hybrid deployment with a traditional switch can also be achieved. In addition, the invention also has the following advantages: 1, networking is allowed under the condition that a traditional switch is not used, and the reliability and fault tolerance of the system are improved through a mode of connecting a plurality of active switching connectors in parallel; 2, on the basis of mixed use of a novel high-performance interconnection architecture and a traditional switch, the requirement of networking on high-capacity traditional switching equipment can be reduced; and 3, the expandability of the accelerator cluster is enhanced, and the system can be expanded more conveniently.

Description

Technical Field

[0001] This invention belongs to the field of communication technology, and more specifically, relates to an interconnection architecture for accelerator clusters. Background Technology

[0002] In current large-scale AI model training and inference systems, computing resources typically consist of accelerator clusters comprised of multiple accelerators, such as graphics processing units (GPUs), used for computation. These accelerators, acting as computing nodes, require a high-bandwidth, low-latency interconnect architecture to meet the high-frequency communication demands for model parameter synchronization, gradient transfer, and inference task scheduling. Currently, interconnection between computing nodes primarily relies on onboard high-speed switching chips, independent switching devices, or point-to-point connections constructed via passive / active cables (interconnect devices). For computing nodes located at greater distances, to ensure communication quality and efficiency, the current industry solution involves adding connectors at both ends to amplify, retime, and enhance signal integrity.

[0003] Figure 1 This is the mainstream connection solution for interconnecting GPUs and switching devices. It involves adding an interconnect device to both ends of the connection between the GPU and the switching device. Figure 1 The interconnect device, constructed using a DSP, together with the connecting cables, forms an Active Electrical Cable (AEC). This interconnect device amplifies, retims, and enhances signal integrity. However, the interconnect device in this implementation lacks any data forwarding or path scheduling capabilities and cannot actively participate in the reconstruction of the system interconnect architecture or traffic scheduling.

[0004] As accelerator clusters continue to expand, traditional switching architectures and point-to-point connections face a series of challenges. On the one hand, centralized switching chips occupy a large amount of motherboard space and power consumption budget, limiting system scalability. On the other hand, while point-to-point connections can improve bandwidth utilization, their fixed topology makes dynamic reconfiguration or flexible expansion difficult. Furthermore, when establishing complex topologies among multiple accelerators, the lack of intelligent forwarding capabilities from intermediate nodes leads to redundancy or low resource utilization in some connection paths. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an interconnect architecture for accelerator clusters to enhance the scalability and dynamic reconfigurability of accelerator clusters, improve resource utilization, and reduce the demand for high-capacity switches in accelerator clusters.

[0006] To achieve the above-mentioned objectives, this invention is applied to the interconnect architecture of accelerator clusters, characterized by comprising:

[0007] Multiple active switching connectors are used to amplify, retime, and enhance the integrity of signals. They also have data routing and forwarding functions. They are directly connected to the accelerators in this group for data routing and forwarding. At the same time, they are connected to active switching connectors at other ends of a set distance via cables or backplanes to route and forward data. After amplifying, retime, and enhancing the integrity of the analog signals obtained from the data conversion, they are sent to active switching connectors at other ends of a set distance for amplification, retime, and integrity enhancement, and then converted back into data for routing and forwarding.

[0008] The objective of this invention is achieved as follows:

[0009] This invention relates to an interconnect architecture for accelerator clusters that can be applied to any interconnect architecture that extends transmission distance, reduces signal loss, lowers transmission costs and power consumption, and includes switching capabilities. In other words, it applies to any architecture that can connect both ends of an accelerator cluster as interconnect components. By leveraging the data routing and forwarding capabilities at both ends, it enhances the scalability and dynamic reconfigurability of the accelerator cluster, improves resource utilization, and directly reduces the need for high-capacity switches. Furthermore, it allows for standalone interconnect network deployment without switches, as well as collaborative / hybrid deployment with traditional switches.

[0010] Furthermore, the present invention also has the following advantages:

[0011] 1. Allows networking without using traditional switches. By connecting multiple active switching connectors (with their switching chips) in parallel, the system's reliability and fault tolerance are improved.

[0012] 2. The hybrid use of a new high-performance interconnect architecture and traditional switches can reduce the need for large-capacity traditional switching equipment in network deployment;

[0013] 3. The scalability of the accelerator cluster is enhanced. The interconnection method of the accelerator cluster is no longer limited by the performance of the switch, and the system can be expanded more conveniently. Attached Figure Description

[0014] Figure 1 This is a connection diagram of a traditional accelerator (GPU) interconnected with a switching device;

[0015] Figure 2 This is a typical use case diagram of the present invention applied to the interconnection architecture of accelerator clusters, using small-capacity switches for data routing and forwarding;

[0016] Figure 3This is a typical application scenario diagram of the interconnection architecture of the present invention applied to accelerator clusters, which realizes large-scale networking without switches through active switching connectors. Detailed Implementation

[0017] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0018] Figure 2 This invention relates to a network topology diagram for the interconnection architecture of accelerator clusters, which uses small-capacity switches for data routing and forwarding.

[0019] In this embodiment, as Figure 2 As shown, the interconnect architecture of the present invention applied to the accelerator cluster includes multiple active switching connectors 1,2,...,n-1,n. The active switching connectors 1,2,...,n-1,n have the functions of amplifying, retiming, and enhancing the integrity of signals, as well as data routing and forwarding functions. They are directly connected to the accelerators in the same group and perform data routing and forwarding. In this embodiment, the accelerators are GPUs. Active switching connectors 1 and 2 are directly connected to the accelerators GPU1, GPU2, GPU3, and GPU4 in the same group, respectively. Active switching connectors n-1 and n are directly connected to the accelerators GPUm-3, GPUm-2, GPUm-1, and GPUm in the same group, respectively.

[0020] Simultaneously, active switching connectors 1, 2, ..., n-1, n are connected via cables or backplanes to active switching connectors J1, J2, ..., Jn-1, Jn at the other end, which are at a distance exceeding a set limit. This allows for data routing and forwarding. The analog signals obtained from the data conversion are amplified, retimed, and have their integrity enhanced before being sent to the active switching connectors J1, J2, ..., Jn-1, Jn at the other end, which are at a distance exceeding the set limit. After further amplification, retiming, and integrity enhancement, the signals are converted back to data and then routed and forwarded. In this embodiment, active switching connectors 1, 2, ..., n-1, n and active switching connectors J1, J2, ..., Jn-1, Jn are connected one-to-one. In this embodiment, active switching connector J1 is connected to two switches S1 and 2 in this group, and active switching connector J2 is connected to two switches Sk-1 and Sk in another group. Active switching connector Jn connects to two switches Sk-1 and Sk in this group, and active switching connector Jn-1 connects to two switches S1 and 2 in another group, thus enabling interconnection between different accelerator groups. This offloads the switching load of the core switch to the active switching connectors, reducing the core switch's load and consequently decreasing the accelerator cluster's need for high-capacity switches. In practice, active switching connectors can be connected using methods other than cables or backplanes, but with equivalent methods.

[0021] Specifically, in this embodiment, accelerators (such as GPUs) and switches in the accelerator cluster are grouped according to the performance of the active switching connectors. For example... Figure 2 As shown, accelerators GPU1, GPU2, GPU3, GPU4, active switch connectors 1 and 2, active switch connectors J1 and J2 belong to the same group as switches S1 and S2. Accelerators GPUm-3, GPUm-2, GPUm-1, GPUm, active switch connectors n-1 and n, active switch connectors Jn-1 and Jn belong to the same group as switches Sk-1 and Sk.

[0022] For accelerators within the same group, the accelerator cluster is fully interconnected via an active switching connector located at one end of the connection. In this embodiment, as... Figure 2As shown, accelerators GPU1, GPU2, GPU3, and GPU4 are all directly connected to active switching connectors 1 and 2. In this configuration, accelerators belonging to the same group can communicate directly through active switching connectors 1 and 2. For example, when GPU1 sends data to GPU2, GPU1 simultaneously sends data to active switching connectors 1 and 2. Upon receiving data, active switching connectors 1 and 2 simultaneously send communication data back to GPU2. In this case, there is no loss in the total communication bandwidth of the accelerators. However, because the switching domain of the active switching connector is smaller, and its switching chip is smaller than that of a switch, the communication latency is lower. Therefore, communication via cable has lower latency than propagation using a switch, offering an advantage in communication.

[0023] For a physically connected link, both ends have active switching connectors. As mentioned above, one end of the link's active switching connector interconnects with an accelerator within the group, and the other end interconnects with a switch. However, unlike the interconnection method with accelerators, when an active switching connector is connected to a switch, for multiple active switching connectors directly connected to a group of accelerators, each active switching connector interconnects with a different group of switches. For example... Figure 2 As shown, active switching connector J1 will be interconnected with switches S1 and S2, and active switching connector J2 will be interconnected with switches Sk-1 and Sk.

[0024] For accelerators in different groups to communicate, they can communicate through multiple paths. For example... Figure 2 As shown, when accelerator GPU1 communicates with GPUm-3, accelerator GPU1 communicates simultaneously through two links: active switching connectors 1 and 2. One communication path involves GPU1 sending data to switches S1 and 2 via active switching connectors 1 and J1. After receiving the data, switches S1 and 2 send it to active switching connectors Jn-1 and Jn-1, ultimately reaching the target accelerator GPUm-3. The other communication path involves GPU1 sending data to switches Sk-1 and Sk via active switching connectors 2 and J2. After receiving the data, switches Sk-1 and Sk send it to the target accelerator GPUm-3 via active switching connectors Jn-1 and Jn-1. During this communication phase, GPU1 does not reduce its communication bandwidth.

[0025] Figure 3 This is a typical application scenario diagram of the interconnection architecture of the present invention applied to accelerator clusters, which realizes large-scale networking without switches through active switching connectors.

[0026] In this embodiment, as Figure 3As shown, in a large-scale network scheme without switches, it is difficult to cover the connection of all accelerators using a single layer of active switching connectors. Therefore, a multi-layer network approach is needed to connect a larger communication domain. Active switching connectors at the other end, with distances exceeding a set threshold, are directly connected to different groups of accelerators. Thus, from the perspective of the objects at both ends of the connection, the system has two types of connections: active switching connector-active switching connector and active switching connector-accelerator.

[0027] In this embodiment, as Figure 3 As shown, the eight accelerators are grouped together, and the accelerators within a group communicate through an active switching connector within the group. Figure 3 Accelerator GPU0 sends data to GPU1, and GPU0 simultaneously sends data to active switch connectors 0 through 15. Multiple active switch connectors, upon receiving data, forward it to the target accelerator GPU1. When an accelerator communicates with all other accelerators in its group, all accelerators are interconnected peer-to-peer and do not block each other.

[0028] For accelerators outside the group, this can be achieved through the interconnection of multiple active switching connectors. Figure 3 Accelerator GPU0 sends data to GPU64. Simultaneously, GPU0 sends data to the target accelerator GPU64 via active switch connectors 0 through 15. Upon receiving the data, active switch connectors 0 through 15 transmit the data to the group containing the target accelerator GPU64 via the active switch connector-to-active switch connector connection, and then transmit the data to the target accelerator GPU64 via intra-group communication.

[0029] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. An interconnect architecture for accelerator clusters, characterized in that, include: Multiple active switching connectors are used to amplify, retime, and enhance the integrity of signals. They also have data routing and forwarding functions. They are directly connected to the accelerators in this group for data routing and forwarding. At the same time, they are connected to active switching connectors at other ends of a set distance via cables or backplanes to route and forward data. After amplifying, retime, and enhancing the integrity of the analog signals obtained from the data conversion, they are sent to active switching connectors at other ends of a set distance for amplification, retime, and integrity enhancement, and then converted back into data for routing and forwarding.

2. The interconnect architecture for accelerator clusters according to claim 1, characterized in that, The other end of the active switching connector is connected to the switch at a distance exceeding the set distance, and each active switching connector is connected to a different group of switches.

3. The interconnect architecture for accelerator clusters according to claim 1, characterized in that, The active switching connectors at the other end, which are more than a set distance away, are directly connected to different groups of accelerators.

Images