Interconnect system for multiple computing modules and computing system

Abstract

This invention relates to an interconnection system and a computing system for multiple computing modules. A first-level computing module is constructed from multiple computing modules, then a second-level computing group is constructed from multiple first-level computing modules, and so on, allowing for the construction of two or more layers of computing modules. For any given layer of computing modules, the number of input / output interfaces used for internal communication interconnection within that layer is greater than the number of input / output interfaces used for inter-layer communication interconnection between computing modules at that level.

Description

Technical Field

[0001] This invention relates to the field of computers, and more specifically, to an interconnection system and a computing system for multiple computing modules. Background Technology

[0002] According to data from the US-based AI research company OpenAI, the computational demands of artificial intelligence (AI) models are growing far faster than the computing power of computing hardware. As AI accelerators continue to achieve increased computing power through process technology iterations and chip architecture innovations, the interconnect bandwidth between AI accelerators is also constantly increasing. AI accelerator interconnect networks have become crucial for enhancing overall computing power.

[0003] However, since the response of electrical channels decays with increasing signal rate, higher-speed interfaces often involve more complex architectures and circuit designs, introducing latency costs, consuming more power, and occupying a larger chip area, thus limiting the chip's input / output bandwidth. Furthermore, longer metal wiring distances further deteriorate the circuit's loss characteristics, limiting the interconnect distance between AI accelerators.

[0004] To overcome the limitations of long-distance, high-frequency electrical link losses, optical interconnects have become a promising technological approach. Although optical interconnects overcome distance limitations, the number of interfaces per computing node is limited. There is an urgent need to develop an effective topology using these limited interfaces to enable computing clusters to simultaneously meet the demands of high computing power and high bandwidth. Summary of the Invention

[0005] On one hand, embodiments of the present invention provide an interconnection system for multiple computing modules. This interconnection system includes multiple computing modules, each having K input / output interfaces, where K is an integer greater than 0. The multiple computing modules constitute a j-layer computing module, where j is an integer greater than or equal to 2. Every N1 computing modules form a first-level computing module. The M1 input / output interfaces of each computing module are used for communication interconnection between the various computing modules within the first-level computing module. The various computing modules within the first-level computing module are interconnected via a first link, where N1 is an even number greater than or equal to 2. i Each (i-1)th level computing module is constructed into an (i-1)th level computing module, and the M of each (i-1)th level computing module is... i There are 1 input / output interface for communication interconnection between the (i-1)th level computing modules within the i-th level computing module. The (i-1)th level computing modules within the i-th level computing module are interconnected via the i-th link, where i is an integer greater than 1 and less than or equal to j, and N... iIt is an integer greater than or equal to 2. For any level of computing module from level 1 to level j, the number of input / output interfaces used for internal communication interconnection of computing modules at that level is greater than the number of input / output interfaces used for communication interconnection between computing modules at that level.

[0006] In some implementations, the number of input / output interfaces for communication interconnection between the various computing modules within a first-level computing module is greater than the number of input / output interfaces for communication interconnection between the various first-level computing modules within a second-level computing module.

[0007] In some implementations, the number of input / output interfaces for communication interconnection between each of the (i-1)th level computing modules within the i-th level computing module is greater than the number of input / output interfaces for communication interconnection between each of the i-th level computing modules within the (i+1)th level computing module.

[0008] In some implementations, the number of input / output interfaces for each computing module in the first to j-th level computing modules is equal to K.

[0009] In some implementations, at least one of the layer-1 to layer-j computing modules uses an optical link for internal communication interconnection.

[0010] In some implementations, the plurality of computing modules constitute a two-layer computing module, i.e., i = j = 2. Every four or six of the computing modules form a first-level computing module, i.e., N1 = 4 or 6. The i-th link is the second link, and the second link is an optical link.

[0011] In some embodiments, the computing modules in the first-level computing module are arranged along a first plane, and the first-level computing modules in the second-level computing module are arranged along the first plane or along a second plane perpendicular to the first plane.

[0012] In some implementations, N1 = 4, meaning that every 4 of the computing modules form a first-level computing module. In the second-level computing module, N2 of the first-level computing modules form a ring topology, where N2 ≥ 4.

[0013] In some implementations, K=8, M1=6, the four computing modules in the first-level computing module are directly connected to each other via the first link, and the computing modules communicate with each other through two input / output interfaces.

[0014] In some implementations, K=5, M1=3, the four computing modules in the first-level computing module are directly connected to each other via the first link, and the computing modules communicate with each other through an input / output interface.

[0015] In some implementations, the plurality of computing modules constitute a three-layer computing module, i.e., j=3. Every four or six of the computing modules form a first-level computing module, i.e., N1=4 or 6. The links used for communication interconnection within the second-level computing module are called second links, and the links used for communication interconnection within the third-level computing module are called third links, wherein the second link is an electrical link or an optical link, and the third link is an optical link.

[0016] In some implementations, the first-level computing module, the second-level computing module, and the third-level computing module are arranged along the same plane within their respective levels, forming a three-dimensional structure between different levels.

[0017] In some embodiments, the computing modules in the first layer have direct communication channels connected to each other. In some embodiments, the computing modules in the second layer have communication channels with a maximum communication hop count of 2 connected to each other. In some embodiments, the computing modules in the third layer have communication channels with a maximum communication hop count of 3 connected to each other.

[0018] In some embodiments, the computing module includes a computing chip and multiple optical modules. The computing chip has multiple electrical input / output interfaces, and each optical module includes an electrical input / output interface and an optical input / output interface. At least some of the electrical input / output interfaces on the computing chip are directly electrically connected to the electrical input / output interfaces on the optical modules, wherein the number of optical modules is less than or equal to K.

[0019] In some embodiments, the computing module further includes a PCB board, and at least some electrical input / output interfaces on the computing chip are electrically connected to the electrical input / output interfaces on the optical module via wiring on the PCB board.

[0020] In some implementations, the electrical input / output interface on the computing chip is a SerDes interface, which includes a serializer and a deserializer.

[0021] In some implementations, the electrical input / output interface on the computing chip is a PCIe physical interface, and the computing chip provides PCIe electrical signals to the optical module, which are used to drive the optical module.

[0022] In some embodiments, the PCB board is provided with a slot in which the optical module is pluggably disposed.

[0023] In some embodiments, the computing module includes a computing chip and an HBM memory chip.

[0024] In some embodiments, the optical link is an active optical cable, with optical modules at both ends. The computing module includes a PCB board and a computing chip mounted on the PCB board. The active optical cable and the computing chip's electrical input / output interface are pluggably connected. The electrical input / output interface on the computing chip is a PCIe physical interface or a SerDes interface.

[0025] On the other hand, embodiments of the present invention provide a computing system that includes the interconnection system described in any embodiment of the present invention.

[0026] As described above, in the interconnection system proposed in the embodiments of the present invention, a first-level computing module is constructed by grouping multiple computing modules, and then a second-level computing group is constructed by multiple first-level computing modules, and so on, to construct computing modules with two or more layers. The inventors have found that increasing the communication bandwidth can reduce latency. Increasing the interconnection bandwidth within a certain level of computing module significantly reduces latency, but increasing the interconnection bandwidth between computing modules at the same level does not significantly reduce latency. The size of the communication bandwidth is limited by the number of input / output interfaces. Therefore, in this interconnection system, the number of input / output interfaces used for internal communication interconnection within a certain level of computing module is greater than the number of input / output interfaces used between computing modules at the same level (i.e., within another level of computing module constructed from that level of computing module). For example, the number of input / output interfaces used within a first-level computing module is greater than the number of input / output interfaces used between first-level computing modules (second level), the number of interfaces used within a second-level computing module is greater than the number of input / output interfaces used between second-level computing modules (third level), and so on. This breaks through the limitation on the number of interfaces of a single computing node (computing module), allowing the computing scale to be flexibly adjusted according to needs to meet the requirements of high computing power and high bandwidth.

[0027] Furthermore, as can be seen from the above embodiments, at least some links are interconnected using optical links, which saves power consumption and reduces latency, facilitating large-scale expansion of computing modules and greatly improving communication capabilities. Of course, computing modules in close proximity can still communicate via electrical links, thus the interconnection system of the present invention offers flexible configuration and allows for flexible control over cost and efficiency.

[0028] The various aspects, features, advantages, etc. of the embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0029] Figure 1 This is a schematic diagram illustrating the topology of an interconnection system according to Embodiment 1 of the present invention.

[0030] Figure 2 This is a schematic diagram illustrating the topology of an interconnection system according to Embodiment 2 of the present invention.

[0031] Figure 3 This is a schematic diagram illustrating the topology of an interconnection system according to Embodiment 3 of the present invention.

[0032] Figure 4 This is a schematic diagram illustrating the topology of an interconnection system according to Embodiment 4 of the present invention.

[0033] Figure 5A and Figure 5B An example of a computing module according to an embodiment of the present invention is shown, wherein, Figure 5A This is a top view showing an example configuration of the computing module, and Figure 5B This is a cross-sectional view showing an example configuration of the computing module.

[0034] Figure 6 This is a side view showing an example of the structure of a computing module with a PCIe interface. Detailed Implementation

[0035] Exemplary embodiments will now be described in more detail with reference to the accompanying drawings. It should be understood that the invention may be embodied in various different forms and should not be construed as limited to the embodiments shown herein. Rather, these embodiments are provided by way of example so that the disclosure of the invention is thorough and complete, and will fully convey to those skilled in the art various aspects and features of the invention. Processes, elements, and techniques not necessarily necessary for those skilled in the art to fully understand the aspects and features of the invention may not be described.

[0036] Unless otherwise stated, similar reference numerals denote similar elements throughout the accompanying drawings and text description, and therefore their description may not be repeated. Furthermore, features or aspects within each exemplary embodiment should generally be considered as other similar features or aspects applicable to other exemplary embodiments.

[0037] The terminology used herein is for the purpose of describing particular embodiments and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” and “having” as used in this specification designate the presence of the stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0038] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that, unless expressly defined herein, terms (such as those defined in common dictionaries) should be interpreted as having the same meaning as they have in the relevant field and / or the context of this specification, and should not be interpreted in an idealized or overly rigid sense.

[0039] The embodiments of the present invention generally relate to an interconnection system and a computing system for multiple computing modules. A first-level computing module is constructed by grouping multiple computing modules, then a second-level computing group is constructed from multiple first-level computing modules, and so on, allowing for the construction of two or more layers of computing modules. For any given layer of computing modules, the number of input / output interfaces used for internal communication interconnection within that layer of computing modules is greater than the number of input / output interfaces used for communication interconnection between computing modules at that layer.

[0040] In some embodiments, the interconnection system includes multiple computing modules, each having K input / output interfaces, where K is an integer greater than 0. The multiple computing modules constitute a j-layer computing module, where j is an integer greater than or equal to 2. Every N1 computing modules form a first-level computing module. The M1 input / output interfaces of each computing module are used for communication interconnection between the various computing modules within the first-level computing module. The various computing modules within the first-level computing module are interconnected via a first link, where N1 is an even number greater than or equal to 2. i Each (i-1)th level computing module is constructed into an (i-1)th level computing module, and the M of each (i-1)th level computing module is... i There are 1 input / output interface for communication interconnection between the (i-1)th level computing modules within the i-th level computing module. The (i-1)th level computing modules within the i-th level computing module are interconnected via the i-th link, where i is an integer greater than 1 and less than or equal to j, and N... i It is an integer greater than or equal to 2. For any level of computing module from level 1 to level j, the number of input / output interfaces used for internal communication interconnection of computing modules at that level is greater than the number of input / output interfaces used for communication interconnection between computing modules at that level.

[0041]

Example 1

[0042] Figure 1The topology of an interconnect system according to Embodiment 1 of the present invention is shown. The interconnect system includes multiple computing modules (e.g., XPU: a processing unit), exemplarily each computing module having eight input / output interfaces, i.e., K=8. The multiple computing modules are configured into a two-layer computing module, namely a first-layer computing module 1001 and a second-layer computing module 1002.

[0043] In this interconnected system, every four computing modules form a first-level computing module 1001, i.e., N1 = 4. For example... Figure 1 As shown, the first-level computing module contains computing module 0, computing module 1, computing module 2, and computing module 3. Each computing module has six input / output interfaces (M1=6) for communication interconnection between the computing modules within the first-level computing module. Specifically, computing modules 0, 1, 2, and 3 are interconnected in pairs via a first link (shown as a red line in the figure). The interconnection between computing module 0 and computing module 1 uses two input / output interfaces from both sides; the interconnection between computing module 0 and computing module 2 uses two input / output interfaces from both sides; and the interconnection between computing module 0 and computing module 3 uses two input / output interfaces from both sides. This allows the computing modules within the first-level computing module to form a bidirectional 64Gb / s interconnection channel. The first link can be an optical link or an electrical link. In this exemplary embodiment, since the communication distance between the computing modules within the first level is relatively short (usually less than 1 meter), the choice between an electrical link and an optical link for the first link can be made as needed.

[0044] Furthermore, in this interconnection system, N2 first-level computing modules are connected via a second link (shown as blue lines in the diagram) to form a ring structure (each computing module reserves 2 input / output interfaces), thereby constructing a second-level computing module 1002. Each computing module's 2 input / output interfaces are used for communication interconnection between first-level computing modules; that is, each first-level computing module 1001's 8 input / output interfaces are used for communication interconnection between computing modules within the second-level computing module, M2 = 8. For example, one input / output interface of computing module 0 in a first-level computing module is connected to one input / output interface of computing module 0 in an upstream adjacent first-level computing module via a second link, and another input / output interface is connected to one input / output interface of computing module 0 in a downstream adjacent first-level computing module via a second link. Similarly, other computing modules 1, 2, and 3 in adjacent first-level computing modules are interconnected, forming a bidirectional 32Gb / s interconnection channel between computing modules in different first-level computing modules. In other words, the various first-level computing modules within the second-level computing module are connected via a bidirectional 32Gb / s interconnection channel. The second link can be an electrical link or an optical link. Since N2 can be any integer greater than 3, preferably 4 ≤ N2 ≤ 16. When the distance between interconnected computing modules is greater than 1 meter, signal stability deteriorates and latency becomes severe. Optical interconnection can overcome these defects; therefore, the second link is preferably an optical link.

[0045] In Example 1, each computing module has 6 input / output interfaces for interconnection within the first-level module and 2 input / output interfaces for interconnection between first-level modules. That is, the number of input / output interfaces within the first-level computing module is greater than the number of input / output interfaces between first-level computing modules (i.e., within the second-level computing module). Increasing communication bandwidth can reduce latency. The inventors found that increasing the interconnection bandwidth within the first-level computing module significantly reduces latency, but increasing the interconnection bandwidth between first-level computing modules does not significantly reduce latency. Therefore, by having more input / output interfaces for communication interconnection within the computing module than for communication interconnection between computing modules, more computing modules can be expanded and computing power increased with a limited number of input / output interfaces, while simultaneously increasing bandwidth and reducing latency.

[0046] Although in Embodiment 1, every 4 of the computing modules are constructed into a first-level computing module, i.e., N1 = 4, it should be understood that in some embodiments, every 6 of the computing modules can be constructed into a first-level computing module, i.e., N1 = 6.

[0047] In some embodiments, the computing modules in the first-level computing module are arranged along a first plane, and the first-level computing modules in the second-level computing module are arranged along the first plane or along a second plane perpendicular to the first plane.

[0048] In some implementations, at least one of the first link and the second link is an optical link that uses an optical input / output interface.

[0049] In some implementations, each computing module uses eight input / output interfaces for the first-level to second-level computing modules. This means maximizing the use of each input / output interface to achieve the largest possible communication bandwidth, while simultaneously expanding the scale of the computing modules and increasing computing power.

[0050] In Embodiment 1, K = 8, M1 = 6, and the four computing modules within the first-level computing module are directly connected to each other via the first link, and the computing modules communicate with each other through two input / output interfaces. The invention is not limited to this; in an optional embodiment, K = 5, M1 = 3, and the four computing modules within the first-level computing module are directly connected to each other via the first link, and the computing modules communicate with each other through one input / output interface.

[0051]

Example 2

[0052] Figure 2 The topology of an interconnect system according to Embodiment 2 of the present invention is shown. Embodiment 1 exemplarily illustrates a two-tier interconnect system in which each computing module has eight input / output interfaces, but the present invention is not limited thereto; computing modules can typically have four, five, six, seven, eight, or even more input / output interfaces. Here, Embodiment 2 uses K=4 (i.e., each computing module has four input / output interfaces) as an example to illustrate a two-tier interconnect system.

[0053] In Embodiment 2, the interconnect system includes 16 computing modules G (GPUs), each G having 4 input / output interfaces. At the first level, every 4 computing modules G (i.e., N1 = 4) form a first-level computing module, resulting in 4 first-level computing modules. Within each first-level computing module, the 4 computing modules G are interconnected pairwise via a first link (shown as a red line in the diagram). Thus, each computing module uses 3 input / output interfaces, giving each first-level computing module 12 input / output interfaces for intra-module communication interconnection and 4 input / output interfaces for inter-module communication interconnection. At the second level, the 4 first-level computing modules are interconnected pairwise. Specifically, one first-level computing module is connected to the other 3 first-level computing modules via a second link (shown as a blue line in the diagram) through 3 input / output interfaces. For each computing module G, the number of input / output interfaces used for internal interconnection within the first-level computing module is 3, and the number of input / output interfaces used for inter-module communication is 1. That is, the number of input / output interfaces used for internal communication interconnection within the computing module is greater than the number of input / output interfaces used for inter-module communication interconnection.

[0054] At least one of the first link and the second link is an optical link. Preferably, the first link is an electrical link and the second link is an optical link. In an optional embodiment, both the first link and the second link are optical links.

[0055] It should be understood that, according to the principles of this invention, multiple computing modules can construct computing modules with two or more layers. For example, based on a two-layer computing module, a third-layer computing module can be constructed using the second-layer computing module as the unit. Furthermore, a fourth-layer computing module can be constructed using the third-layer computing module as the unit. And so on, more layers of computing modules can be constructed. For each layer of computing module, the number of input / output interfaces used for communication interconnection within the computing module is greater than the number of input / output interfaces used for communication interconnection between computing modules.

[0056]

Example 3

[0057] Figure 3 The topology of an interconnection system according to Embodiment 3 of the present invention is shown. Figure 3 As shown, in Embodiment 3, the second-level computing module of Embodiment 2 is treated as a unit, and two second-level computing modules are interconnected through a third link to form a third-level computing module. In other words, Embodiment 3 uses a 3-layer computing module constructed with 32 computing modules.

[0058] Similar to Example 2, every four computing modules G form a first-level computing module, resulting in eight first-level computing modules. The first-level computing modules are interconnected internally via first-level links. Every four first-level computing modules construct a second-level computing module, resulting in two second-level computing modules. The first-level computing modules within each second-level computing module are interconnected via second-level links. The two second-level computing modules are interconnected via a third link (shown as a red line in the diagram), thus forming a third-level computing module.

[0059] Each computing module G has four input / output interfaces: three for communication interconnection within the first-level computing module and one for communication interconnection outside the first-level computing module. Thus, each first-level computing module can have four input / output interfaces for external communication interconnection. Specifically, three input / output interfaces are used for communication interconnection between the first-level computing modules within the second-level computing module, and one is used for communication interconnection outside the second-level computing module. Since the second-level computing module comprises four first-level computing modules, each second-level computing module has four input / output interfaces for external communication interconnection. Therefore, the four input / output interfaces of each of the two second-level computing modules are interconnected via four corresponding third links. According to the interconnection system of this embodiment, each first-level computing module can have 12 input / output interfaces for communication interconnection within the module, and each first-level computing module can have 3 input / output interfaces for communication interconnection between the four first-level computing modules within the second-level computing module; each second-level computing module can have 12 input / output interfaces for communication interconnection between the four first-level computing modules within the module, and each second-level computing module can have 4 input / output interfaces for communication interconnection between the two second-level computing modules within the third-level computing module. Therefore, compared to Embodiment 2 where the second-level computing module has 4 unused input / output interfaces, in Embodiment 3 all input / output interfaces of all computing modules are used, and through the construction of multiple levels of computing modules, the computing cluster is built to be large enough to obtain the largest possible communication bandwidth and computing power.

[0060] In some embodiments, at least one of the first link, second link, and third link is an optical link. Preferably, the third link is an optical link. More preferably, the second link and the third link are optical links.

[0061] In some implementations, the first-level computing module, the second-level computing module, and the third-level computing module are arranged along the same plane within their respective levels, forming a three-dimensional structure between different levels.

[0062]

Example 4

[0063] Figure 4 The topology of an interconnection system according to Embodiment 4 of the present invention is shown. In Embodiment 4, the interconnection system includes 16 computing modules, each with 6 input / output interfaces. Every 4 computing modules constitute a first-level computing module, resulting in 4 first-level computing modules: computing modules 0, 1, 2, and 3 (green boxes) constitute one first-level computing module, and computing modules 4, 5, 6, and 7 constitute another first-level computing module; computing modules 0, 1, 2, and 3 (blue boxes) constitute one first-level computing module, and computing modules 4, 5, 6, and 7 constitute another first-level computing module. The 3 input / output interfaces of each computing module are used for internal communication interconnection within the first-level computing module via a first link, and these 3 input / output interfaces can also be used for external communication interconnection of the first-level computing module. Thus, each first-level computing module has 12 input / output interfaces for external communication interconnection.

[0064] Two Level 1 computing modules constitute one Level 2 computing module. For example, a Level 1 computing module containing green modules 0, 1, 2, and 3, and a Level 1 computing module containing blue modules 0, 1, 2, and 3, interconnected via a second link, constitute one Level 2 computing module. Similarly, a Level 1 computing module containing green modules 4, 5, 6, and 7, and a Level 1 computing module containing blue modules 4, 5, 6, and 7, interconnected via another second link, constitute another Level 2 computing module. These two Level 2 computing modules are then interconnected via a third link to form a Level 3 computing module.

[0065] As described above, each first-level computing module has 12 input / output interfaces for external communication interconnection. Eight of these interfaces are used for interconnection between two first-level computing modules within a second-level computing module via a second link (shown as red lines in the diagram). The remaining four interfaces are used for interconnection between two second-level computing modules within a third-level computing module via a third link. According to the interconnection system of Embodiment 4, each first-level computing module can have 12 input / output interfaces for communication interconnection within the module; each first-level computing module can have 8 input / output interfaces for communication interconnection between two first-level computing modules within a second-level computing module; each second-level computing module can have 16 input / output interfaces for communication interconnection between two first-level computing modules within the module; and each second-level computing module can have 8 input / output interfaces for communication interconnection between two second-level computing modules within a third-level computing module. According to the interconnection system of Embodiment 4, the computing modules in the first-level computing module have a communication path between each other through one hop, the computing modules in the second-level computing module have a communication path between each other through up to two hops, and the computing modules in the third-level computing module have a communication path between each other through up to three hops.

[0066] In some embodiments, at least one of the first, second, and third links is an optical link. Preferably, the third link is an optical link. More preferably, the second and third links are optical links. The computing system topology of this embodiment has a large communication bandwidth and low latency.

[0067] Depending on actual needs, the interconnection system of the present invention may also have a fourth layer, a fifth layer, or even more layers of computing modules.

[0068] [Calculation Module]

[0069] In some embodiments, the computing module in the interconnect system of the present invention includes a computing chip and multiple optical modules. Figure 5A and Figure 5B An example of a computing module according to an embodiment of the present invention is shown, wherein, Figure 5A A top view of an example configuration of the computing module is shown, and Figure 5B A cross-sectional view of an example configuration of this computing module is shown.

[0070] like Figure 5A As shown in Figure 5B, the computing module includes one or more optical modules 100 and a computing chip 300. The one or more optical modules 100 and the computing chip 300 are arranged on the same PCB board 500.

[0071] Each optical module 100 includes multiple electrical input / output interfaces and optical input / output interfaces. The number of optical modules is less than or equal to the number of electrical input / output interfaces.

[0072] The computing chip 300 and each optical module 300 are arranged on the same PCB board 500, and the computing chip 300 includes one or more high-speed input / output interfaces (e.g., long-distance SerDes interfaces). For simplicity, Figure 5A The diagram only shows four high-speed input / output interfaces of the computing chip 300 connected to the optical module (e.g., interfaces connected to wiring 304). When there are electrical links between computing modules in the interconnection system of the present invention, the computing chip may also have one or more high-speed input / output interfaces not connected to the optical module for interconnection of electrical links. Optionally, the computing chip 300 may be mounted on a PCB board 500 via a substrate 400.

[0073] In some implementations, the computing chip 300 may be an artificial intelligence chip, such as, but not limited to, a graphics processing unit (GPU), a neural network processor (NPU), a tensor processor (TPU), an intelligent processor (IPU), a deep learning processor (DPU), etc.

[0074] It should be noted that the four high-speed input / output interfaces of the computing chip 300 shown above, corresponding to four optical modules, are merely for ease of description. In practical applications, any number of high-speed input / output interfaces can be configured for the computing chip 300 according to specific requirements. In this embodiment, the output of each high-speed input / output interface of the computing chip 300 performs parallel-to-serial conversion on the parallel signals within the computing chip 300 to output a first electrical signal (not shown in the figure). This first electrical signal is provided to the corresponding electrical input port of the corresponding optical module 100 for photoelectric conversion. The parallel-to-serial conversion operation can be performed, for example, by a serializer (the SERializer circuit in the SerDes interface), thereby enabling the conversion of multiple low-speed parallel signals within the computing chip 300 into high-speed serial signals.

[0075] Furthermore, the input terminal of each input / output interface of the computing chip 300 performs a serial-to-parallel conversion (not shown) on the second electrical signal received from the optical module 100 as a high-speed serial signal, thereby converting the high-speed serial signal back into a low-speed parallel signal for use by the computing chip 300. The serial-to-parallel conversion operation can be performed, for example, by a deserializer (the DESerializer circuit in the SerDes interface), thereby enabling the high-speed serial signal within the computing chip 300 to be converted into a low-speed parallel signal.

[0076] It should be noted that wiring 304 in PCB board 500 is a general term for the electrical connection wiring between optical module 100 and computing chip 300, and is not intended to represent any specific wiring.

[0077] Functionally, each optical module 100 is used for electro-optical conversion and optical-electrical conversion.

[0078] The computing module 3000 also includes at least one high-bandwidth memory (HBM) unit 302. For example, the HBM unit can work with the computing chip 300 to perform various caching or storage functions.

[0079] In some embodiments, multiple HBM units 302 may be arranged together with the computing chip 300 on the substrate 400. Optionally, multiple HBM units are arranged on the substrate 400 and located on both sides of the computing chip 300.

[0080] In some implementations, the electrical input / output interface on the computing chip may be a PCIe physical interface, and the computing chip provides PCIe electrical signals to the optical module, which are used to drive the optical module. Figure 6 This is a side view illustrating a structural example of a computing module with a PCIe interface. In this embodiment, the computing chip is configured to have a PCIe physical interface, also known as a PCIe PHY interface. The PCIe PHY interface is used to perform protocol conversion on the input / output I / O signals of the computing chip, thereby making them compliant with the PCIe standard.

[0081] exist Figure 6 In the illustrated embodiment, the PCIe PHY interface on the computing chip 2020 can convert the chip's internal digital signals into PCIe electrical signals. These PCIe electrical signals can be directly fed to the electrical input interface of the PCIe optical module 2010 for electro-optical (EO) conversion.

[0082] In addition, the PCIe electrical signal output from the electrical output interface after photoelectric (OE) conversion by the PCIe optical module 2010 can also be directly fed to the PCIe PHY interface on the computing chip 2020. The PCIe PHY interface then converts it into a digital signal for use by the computing chip 2020.

[0083] In some embodiments, a slot (not shown) may be arranged in the middle or at the edge of the PCB board 2040, and the optical module 2010 is pluggably arranged in the slot.

[0084] The computing chip 2020 is disposed on the packaging substrate 2030, and the packaging substrate 2030 is disposed on the PCB board 2040.

[0085] The electrical interconnection between the computing chip 2020 and the optical module 2010 is achieved through wiring 2060 in the packaging substrate 2030 and the PCB board 2040.

[0086] For example, the edge of the PCB board 2040 can correspond to the front panel of the server device. In this case, the optical module 2010 can be easily plugged in and out of the front panel of the server, thus avoiding the impact on other components when operating the optical module.

[0087] In addition, in such Figure 6 In the case of the packaging example shown, the computing chip 2020 can be placed as close as possible to the optical module 2010, thereby reducing the length of the wiring 2060 between the computing chip 2020 and the optical module 2010, which is beneficial to improving signal integrity.

[0088] In addition, corresponding wiring 2060 can be arranged for various data signals and control signals as described above for the optical module, so that the optical module 2010 can perform EO conversion on various electrical signals from the computing chip 2020, and then the fiber array 2050 as shown in the figure can transmit the converted optical signals.

[0089] In another embodiment, the optical module 2010 is fixedly arranged in the first region of the PCB board 2040, instead of as... Figure 6 They are thus pluggably arranged in the slots of the PCB board 2040. At the same time, the computing chip 2020 is arranged on the packaging substrate 2030, and the packaging substrate 2030 is arranged in a second region on the PCB board 2040 that is different from the first region mentioned above, that is, the packaging substrate 2030 and the optical module 2010 are not arranged overlappingly.

[0090] Furthermore, in some embodiments, the optical link of the present invention is an active optical cable, with optical modules at both ends. The computing module includes a PCB board and a computing chip mounted on the PCB board. The active optical cable and the computing chip's electrical input / output interface are pluggably connected. The electrical input / output interface on the computing chip is a PCIe physical interface or a SerDes interface.

[0091] Based on the above, those skilled in the art should understand that embodiments of the computing system of the present invention include any of the above-described implementation methods or the interconnection system described in the embodiments.

[0092] Those skilled in the art should understand that the above-disclosed embodiments are merely implementations of the present invention and should not be construed as limiting the scope of the patent protection claimed in this invention. Equivalent variations made according to the embodiments of the present invention are still within the scope of the claims of the present invention.

Claims

1. An interconnection system for multiple computing modules, comprising multiple computing modules, each computing module having K input / output interfaces, where K is an integer greater than 0; The multiple computing modules constitute a computing module of layer j, where j is an integer greater than or equal to 2; in, Every N1 computing modules constitute a first-level computing module. M1 input / output interfaces of each computing module are used for communication interconnection between the computing modules within the first-level computing module. The computing modules within the first-level computing module are interconnected through a first link. N1 is an even number greater than or equal to 2. per N i Each (i-1)th level computing module is constructed into an (i-1)th level computing module, and the M of each (i-1)th level computing module is... i There are 1 input / output interface for communication interconnection between the (i-1)th level computing modules within the i-th level computing module. The (i-1)th level computing modules within the i-th level computing module are interconnected via the i-th link, where i is an integer greater than 1 and less than or equal to j, and N... i It is an integer greater than or equal to 2; For any one level of computing module from level 1 to level j, the number of input / output interfaces used for internal communication interconnection within that level of computing module is greater than the number of input / output interfaces used for communication interconnection between computing modules at that level.

2. The interconnection system as described in claim 1, characterized in that, The number of input / output interfaces used for communication interconnection between computing modules within a Level 1 computing module is greater than the number of input / output interfaces used for communication interconnection between Level 1 computing modules within a Level 2 computing module.

3. The interconnection system as described in claim 1, characterized in that, The number of input / output interfaces used for communication interconnection between each of the (i-1)th level computing modules within the i-th level computing module is greater than the number of input / output interfaces used for communication interconnection between each of the i-th level computing modules within the (i+1)th level computing module.

4. The interconnection system as claimed in claim 1, characterized in that, The number of input / output interfaces for each computing module used in the computing modules from level 1 to level j is equal to K.

5. The interconnection system as claimed in claim 1, characterized in that, In at least one of the computing modules at levels 1 to j, the internal communication interconnection link is an optical link.

6. The interconnection system according to any one of claims 1 to 5, characterized in that, The multiple computing modules constitute a two-layer computing module, i.e., i = j = 2. Every four or six of the aforementioned computing modules constitute a first-level computing module, i.e., N1 = 4 or 6. The i-th link is the 2nd link, and the 2nd link is an optical link.

7. The interconnection system as claimed in claim 6, characterized in that, The computing modules within the first-level computing module are arranged along a first plane. Each of the first-level computing modules within the second-level computing module is arranged along a first plane or along a second plane perpendicular to the first plane.

8. The interconnection system as claimed in claim 6, characterized in that, N1 = 4, meaning that every 4 of the aforementioned computing modules constitute a first-level computing module. In the second-level computing module, N2 of the first-level computing modules form a ring topology, where N2 ≥ 4.

9. The interconnection system as claimed in claim 8, characterized in that, K=8, M1=6, the four computing modules in the first-level computing module are directly connected to each other via the first link, and the computing modules communicate with each other through two input / output interfaces.

10. The interconnection system as claimed in claim 8, characterized in that, K=5, M1=3, the four computing modules in the first-level computing module are directly connected to each other via the first link, and the computing modules communicate with each other through an input / output interface.

11. The interconnection system according to any one of claims 1 to 5, characterized in that, The multiple computing modules constitute a 3-layer computing module, i.e., j=3. Every four or six of the aforementioned computing modules constitute a first-level computing module, i.e., N1 = 4 or 6. The link used for communication interconnection within the second-level computing module is called the second link, and the link used for communication interconnection within the third-level computing module is called the third link. The second link is an electrical link or an optical link, and the third link is an optical link.

12. The interconnection system as claimed in claim 11, characterized in that, The first-level computing module, the second-level computing module, and the third-level computing module are arranged along the same plane within their respective levels, forming a three-dimensional structure between different levels.

13. The interconnection system as claimed in claim 1, characterized in that, The computing modules within the first level have direct communication channels connected to each other.

14. The interconnection system as claimed in claim 13, characterized in that, The computing modules in the second level have a communication channel with a maximum communication hop count of 2 between each other.

15. The interconnection system as claimed in claim 14, characterized in that, The computing modules in the third level have communication channels with a maximum communication hop count of 3 between each other.

16. The interconnection system as claimed in claim 5, characterized in that, The computing module includes a computing chip and multiple optical modules. The computing chip has multiple electrical input / output interfaces, and each optical module includes both electrical input / output interfaces and optical input / output interfaces. At least some of the electrical input / output interfaces on the computing chip are directly electrically connected to the electrical input / output interfaces on the optical module, wherein the number of optical modules is less than or equal to K.

17. The interconnection system as claimed in claim 16, characterized in that, The computing module also includes a PCB board, and at least some of the electrical input / output interfaces on the computing chip are electrically connected to the electrical input / output interfaces on the optical module through wiring on the PCB board.

18. The interconnection system as claimed in claim 17, characterized in that, The electrical input / output interface on the computing chip is a SerDes interface.

19. The interconnection system as claimed in claim 17, characterized in that, The electrical input / output interface on the computing chip is a PCIe physical interface. The computing chip provides PCIe electrical signals to the optical module, and the PCIe electrical signals are used to drive the optical module.

20. The interconnection system as claimed in claim 19, characterized in that, The PCB board has a slot, and the optical module is pluggably arranged in the slot.

21. The interconnection system as claimed in claim 1, characterized in that, The computing module includes a computing chip and an HBM memory chip.

22. The interconnection system as claimed in claim 5, characterized in that, The optical link is an active optical cable, and optical modules are installed at both ends of the active optical cable. The computing module includes a PCB board and a computing chip mounted on the PCB board. The active optical cable can be plugged into the electrical input / output interface of the computing chip.

23. A computing system, characterized in that, Includes the interconnection system according to any one of claims 1 to 22.

Images