Interconnection system for plurality of computing modules and computing system

Abstract

The invention relates to an interconnection system for a plurality of computing modules and a computing system. A first-level calculation module is constructed by a plurality of calculation modules, then a second-level calculation group is constructed by a plurality of first-level calculation modules, and by parity of reasoning, two or more layers of calculation modules can be constructed. For any hierarchy of calculation modules, the number of the input and output interfaces used for internal communication interconnection of the hierarchy of calculation modules is larger than that of the input and output interfaces used for communication interconnection between the hierarchy of calculation modules.

Description

Technical Field

[0001] The present invention relates to the field of computers, and more particularly, to an interconnection system for multiple computing modules and a computing system. Background Art

[0002] According to data from OpenAI, an American artificial intelligence research company, the growth rate of artificial intelligence (AI) model computing volume far exceeds the growth rate of computing hardware computing power. As AI accelerators continue to gain computing power through process technology iteration and chip architecture innovation, the interconnection bandwidth between AI accelerators is also growing. The AI ​​accelerator interconnection network has become the key to improving overall computing power.

[0003] However, since the response of the electrical channel will decay as the signal rate increases, higher-speed interfaces often involve more complex architectures and circuit designs, introduce latency costs, consume more power, and occupy a larger chip area, thus limiting the chip's input and output bandwidth. In addition, longer metal wiring distances will further deteriorate the circuit's loss characteristics, limiting the interconnection distance between AI accelerators.

[0004] In order to break the limitation of long-distance high-frequency electrical link loss, optical interconnection is favored as a potential technology route. Although optical interconnection breaks the distance limitation, the number of interfaces of each computing node is limited. There is an urgent need to use limited interfaces to develop an effective topology so that the computing cluster can meet the needs of high computing power and high bandwidth at the same time. Summary of the invention

[0005] On the one hand, an embodiment of the present invention provides an interconnection system for multiple computing modules. The interconnection system includes multiple computing modules, each computing module has K input and output interfaces, K is an integer greater than 0. The multiple computing modules constitute a j-layer computing module, j is an integer greater than or equal to 2, wherein every N1 computing modules are constructed into a first-level computing module, and the M1 input and output interfaces of each computing module are used for communication and interconnection between the computing modules inside the first-level computing module, and the computing modules inside the first-level computing module are communicated and interconnected through the first link, N1 is an even number greater than or equal to 2; every N i The i-1th level computing modules are constructed into an i-th level computing module, and the M of each of the i-1th level computing modules i The input and output interfaces are used for communication and interconnection between the i-1th level computing modules in the i-th level computing module, and the i-1th level computing modules in the i-th level computing module are communicated and interconnected through the i-th link, where i is an integer greater than 1 and less than or equal to j, and N iis an integer greater than or equal to 2. For any level of computing modules from the 1st level to the jth level, the number of input and output interfaces used for internal communication and interconnection of the computing modules at that level is greater than the number of input and output interfaces used for communication and interconnection between computing modules at that level.

[0006] In some embodiments, the number of input and output interfaces used for communication interconnection between computing modules within the first-level computing module is greater than the number of input and output interfaces used for communication interconnection between first-level computing modules within the second-level computing module.

[0007] In some embodiments, the number of input and output interfaces used for communication interconnection between the i-1th level computing modules within the i-th level computing module is greater than the number of input and output interfaces used for communication interconnection between the i-th level computing modules within the i+1th level computing module.

[0008] In some embodiments, the number of input and output interfaces of each computing module for the 1st to jth level computing modules is equal to K.

[0009] In some embodiments, the link used for communication interconnection within at least one level computing module among the 1st level to the jth level computing modules is an optical link.

[0010] In some embodiments, the plurality of computing modules form a 2-layer computing module, i.e., i=j=2. Every 4 or 6 computing modules form a 1st-layer computing module, i.e., N1=4 or 6. The i-th link is the 2nd link, and the 2nd link is an optical link.

[0011] In some embodiments, the computing modules within the first-level computing module are arranged along a first plane, and the first-level computing modules within the second-level computing module are arranged along the first plane or along a second plane perpendicular to the first plane.

[0012] In some embodiments, N1=4, that is, every 4 computing modules form a first-level computing module. In the second-level computing module, N2 first-level computing modules form a ring topology structure, and N2≥4.

[0013] In some embodiments, K=8, M1=6, the four computing modules in the first-level computing module are directly connected to each other by the first link, and the computing modules communicate with each other through two input and output interfaces.

[0014] In some embodiments, K=5, M1=3, the four computing modules in the first-level computing module are directly connected to each other by the first link, and the computing modules communicate with each other through an input-output interface.

[0015] In some embodiments, the plurality of computing modules constitute a 3-layer computing module, i.e., j = 3. Every 4 or 6 computing modules constitute a 1st-level computing module, i.e., N1 = 4 or 6. The link used for communication interconnection inside the 2nd-level computing module is the 2nd link, and the link used for communication interconnection inside the 3rd-level computing module is the 3rd link, wherein the 2nd link is an electrical link or an optical link, and the 3rd link is an optical link.

[0016] In some embodiments, the first-level computing modules, the second-level computing modules, and the third-level computing modules are arranged along the same plane within their respective levels, and are arranged into a three-dimensional structure between different levels.

[0017] In some embodiments, the computing modules in the first level have a directly connected communication channel between each other. In some embodiments, the computing modules in the second level have a communication channel with a maximum communication hop number of 2 between each other. In some embodiments, the computing modules in the third level have a communication channel with a maximum communication hop number of 3 between each other.

[0018] In some embodiments, the computing module includes a computing chip and a plurality of optical modules. The computing chip has a plurality of electrical input and output interfaces, and each of the optical modules includes an electrical input and output interface and an optical input and output interface. At least some of the electrical input and output interfaces on the computing chip are directly electrically connected to the electrical input and output interfaces on the optical modules, wherein the number of the optical modules is less than or equal to K.

[0019] In some implementations, the computing module further includes a PCB board, and at least part of the electrical input / output interface on the computing chip is electrically connected to the electrical input / output interface on the optical module via wiring on the PCB board.

[0020] In some implementations, the electrical input-output interface on the computing chip is a SerDes interface, which includes a serializer and a deserializer.

[0021] In some implementations, the electrical input and output interface on the computing chip is a PCIe physical interface, and the computing chip provides a PCIe electrical signal to the optical module, and the PCIe electrical signal is used to drive the optical module.

[0022] In some implementations, a card slot is arranged on the PCB board, and the optical module is pluggably arranged in the card slot.

[0023] In some embodiments, the computing module includes a computing chip and an HBM memory chip.

[0024] In some embodiments, the optical link is an active optical cable, and optical modules are provided at both ends of the active optical cable. The computing module includes a PCB board and a computing chip arranged on the PCB board. The active optical cable is pluggably connected to the electrical input and output interface of the computing chip. The electrical input and output interface on the computing chip is a PCIe physical interface or a SerDes interface.

[0025] On the other hand, an embodiment of the present invention provides a computing system, which includes the interconnection system described in any one embodiment of the present invention.

[0026] According to the above, in the interconnection system proposed in the embodiment of the present invention, a plurality of computing modules are grouped to construct a first-level computing module, and then a plurality of first-level computing modules are used to construct a second-level computing group, and so on, a computing module of 2 layers or more can be constructed. The inventor found that the increase in communication bandwidth can reduce delay, and the increase in interconnection bandwidth within a certain level of computing modules significantly reduces delay, but the increase in interconnection bandwidth between computing modules at this level does not significantly reduce delay. The size of the communication bandwidth is limited by the number of input and output interfaces. Therefore, in the interconnection system, the number of input and output interfaces used for internal communication interconnection of a certain level of computing modules is greater than the number of input and output interfaces used between computing modules at this level (i.e., another level of computing modules constructed by computing modules at this level). For example, the number of input and output interfaces used inside the first-level computing module is greater than the number of input and output interfaces used between computing modules at the first level (the second level), and the number of interfaces used inside the second-level computing module is greater than the number of input and output interfaces used between computing modules at the second level (the third level), and so on. This breaks through the limitation on the number of interfaces of a single computing node (computing module), allowing the computing scale to be flexibly adjusted according to demand to meet the requirements of high computing power and high bandwidth.

[0027] Furthermore, according to the above implementation, at least part of the links are interconnected using optical links, which saves power consumption and reduces latency, is conducive to large-scale expansion of computing modules, and greatly improves communication capabilities. Of course, computing modules in close proximity can still choose electrical links for communication, so the interconnection system of the present invention is flexible in configuration and cost efficiency can be flexibly controlled.

[0028] The various aspects, features, advantages, etc. of the embodiments of the present invention are described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 : is a schematic diagram showing the topological structure of the interconnection system according to Embodiment 1 of the present invention.

[0030] Figure 2 : is a schematic diagram showing the topological structure of the interconnection system according to Embodiment 2 of the present invention.

[0031] Figure 3 : is a schematic diagram showing the topological structure of the interconnection system according to Embodiment 3 of the present invention.

[0032] Figure 4 : is a schematic diagram showing the topological structure of an interconnection system according to Embodiment 4 of the present invention.

[0033] Figure 5A and Figure 5B An example of a calculation module according to an embodiment of the present invention is shown, wherein: Figure 5A is a top view showing an example configuration of the computing module, and Figure 5B is a cross-sectional view showing an example configuration of the computing module.

[0034] Figure 6 is a side view showing a structural example of a computing module having a PCIe interface. DETAILED DESCRIPTION

[0035] Next, exemplary embodiments will be described in more detail with reference to the accompanying drawings. It should be understood that the present invention can be embodied in a variety of different forms and should not be construed as being limited to the embodiments shown herein. Instead, these embodiments are provided as examples so that the disclosure of the present invention is thorough and comprehensive, and will fully convey the various aspects and features of the present invention to those skilled in the art. Processes, elements, and techniques that are not necessary for those of ordinary skill in the art to fully understand the various aspects and features of the present invention may not be described.

[0036] Unless otherwise specified, similar reference numerals denote similar elements throughout the drawings and text description, and therefore, description thereof may not be repeated. In addition, features or aspects within each exemplary embodiment should generally be considered applicable to other similar features or aspects in other exemplary embodiments.

[0037] The terms used herein are for the purpose of describing specific embodiments and are not intended to limit the present invention. As used herein, the singular forms "a", "the" are intended to include plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the terms "comprise", "include" and "have" when used in this specification specify the presence of stated features, wholes, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, parts and / or their collections.

[0038] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the invention belongs. It should also be understood that, unless explicitly defined as such herein, terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and / or this specification, and should not be interpreted in an idealized or overly rigid sense.

[0039] Embodiments of the present invention generally relate to an interconnection system and a computing system for multiple computing modules. A first-level computing module is constructed by grouping multiple computing modules, and then a second-level computing group is constructed by multiple first-level computing modules. By analogy, computing modules of 2 layers or more can be constructed. For any one-level computing module, the number of input and output interfaces used for internal communication interconnection of the computing module of the level is greater than the number of input and output interfaces used for communication interconnection between computing modules of the level.

[0040] In some embodiments, the interconnection system includes a plurality of computing modules, each computing module having K input and output interfaces, where K is an integer greater than 0. The plurality of computing modules constitute a j-layer computing module, where j is an integer greater than or equal to 2, wherein each N1 computing modules constitute a first-level computing module, and the M1 input and output interfaces of each computing module are used for communication and interconnection between the computing modules within the first-level computing module, and the computing modules within the first-level computing module are communicated and interconnected via the first link, where N1 is an even number greater than or equal to 2; i The i-1th level computing modules are constructed into an i-th level computing module, and the M of each of the i-1th level computing modules i The input and output interfaces are used for communication and interconnection between the i-1th level computing modules in the i-th level computing module, and the i-1th level computing modules in the i-th level computing module are communicated and interconnected through the i-th link, where i is an integer greater than 1 and less than or equal to j, and N i is an integer greater than or equal to 2. For any level of computing modules from the 1st level to the jth level, the number of input and output interfaces used for internal communication and interconnection of the computing modules at that level is greater than the number of input and output interfaces used for communication and interconnection between computing modules at that level.

[0041] [Example 1]

[0042] Figure 1The topology of the interconnection system according to the first embodiment of the present invention is shown. The interconnection system includes multiple computing modules (e.g., XPU: a certain processing unit), and each computing module has 8 input and output interfaces, i.e., K=8. The multiple computing modules are constructed into 2-layer computing modules, i.e., the first-level computing module 1001 and the second-level computing module 1002.

[0043] In this interconnection system, every 4 computing modules form a first-level computing module 1001, that is, N1=4. Figure 1 As shown, there are computing modules 0, 1, 2, and 3 inside the first-level computing module. The 6 input and output interfaces of each computing module (i.e., M1=6) are used for communication and interconnection between the computing modules inside the first-level computing module. Specifically, computing modules 0, 1, 2, and 3 are interconnected in pairs using the first link (shown as a red line in the figure). The interconnection between computing module 0 and computing module 1 uses two input and output interfaces of both parties, the interconnection between computing module 0 and computing module 2 uses two input and output interfaces of both parties, and the interconnection between computing module 0 and computing module 3 uses two input and output interfaces of both parties, so that a bidirectional 64Gb / s interconnection channel can be formed between the computing modules in the first-level computing module. The first link can be an optical link or an electrical link. In an exemplary manner, since the communication distance of the computing modules in the first level is relatively close (usually less than 1 meter), whether the first link uses an electrical link or an optical link can be selected as needed.

[0044] Further, in the interconnection system, N2 first-level computing modules are connected through the second link (shown as a blue line in the figure) to form a ring structure (two input and output interfaces are reserved for each computing module), thereby constructing the second-level computing module 1002. Among them, the two input and output interfaces of each computing module are used for communication interconnection between the first-level computing modules, that is, the eight input and output interfaces of each first-level computing module 1001 are used for communication interconnection between the computing modules in the second-level computing modules, M2=8. Exemplarily, an input and output interface of computing module No. 0 in a certain first-level computing module is connected to an input and output interface of computing module No. 0 in the upstream adjacent first-level computing module through the second link, and another input and output interface is connected to an input and output interface of computing module No. 0 in the downstream adjacent first-level computing module through the second link. By analogy, other computing modules 1, 2, and 3 of adjacent first-level computing modules are interconnected with each other, thereby forming a bidirectional 32Gb / s interconnection channel between computing modules of different first-level computing modules. In other words, each first-level computing module in the second-level computing module is connected by a bidirectional 32Gb / s interconnection channel. The second link can be an electrical link or an optical link. Because N2 can be any integer greater than 3, preferably, 4≤N2≤16. When the distance between the interconnected computing modules is greater than 1 meter, the signal stability deteriorates and the delay is serious. Optical interconnection can overcome the above defects. Therefore, the second link is preferably an optical link.

[0045] In Example 1, the number of input and output interfaces of each computing module for interconnection within the first-level modules is 6, and the number of input and output interfaces for interconnection between the first-level modules is 2. That is, the number of input and output interfaces used within the first-level computing module is greater than the number of input and output interfaces used between the first-level computing modules (i.e., used within the second-level computing module). The increase in communication bandwidth can reduce latency. The inventors found that the increase in interconnection bandwidth within the first-level computing module significantly reduced latency, but the increase in interconnection bandwidth between the first-level computing modules did not significantly reduce latency. Therefore, by making the number of input and output interfaces used for communication interconnection within the computing module greater than the number of input and output interfaces used for communication interconnection between computing modules, more computing modules can be expanded while increasing bandwidth and reducing latency with a limited number of input and output interfaces, thereby improving computing power.

[0046] Although in Example 1, every 4 computing modules are constructed into a first-level computing module, that is, N1=4, it should be understood that in some embodiments, every 6 computing modules can be constructed into a first-level computing module, that is, N1=6.

[0047] In some embodiments, the computing modules within the first-level computing module are arranged along a first plane, and the first-level computing modules within the second-level computing module are arranged along the first plane or along a second plane perpendicular to the first plane.

[0048] In some embodiments, at least one of the first link and the second link is an optical link using an optical input and output interface.

[0049] In some embodiments, the number of input and output interfaces of each computing module for the first-level to second-level computing modules is equal to 8. That is, each input and output interface is used to the maximum extent to obtain the largest possible communication bandwidth, while expanding the scale of the computing module and improving the computing power.

[0050] In Example 1, K=8, M1=6, the four computing modules in the first-level computing module have the first link directly connected to each other, and the computing modules communicate with each other through two input and output interfaces. The present invention is not limited to this. In an optional implementation, K=5, M1=3, the four computing modules in the first-level computing module have the first link directly connected to each other, and the computing modules communicate with each other through one input and output interface.

[0051] [Example 2]

[0052] Figure 2 The topology of the interconnection system according to the second embodiment of the present invention is shown. The first embodiment exemplarily illustrates a two-layer interconnection system in which each computing module has eight input and output interfaces, but the present invention is not limited thereto. The computing module can generally have four, five, six, seven, eight or even more input and output interfaces. Here, the second embodiment takes K=4 (i.e., each computing module has four input and output interfaces) as an example to illustrate the two-layer interconnection system.

[0053] In Example 2, the interconnected system includes 16 computing modules G (GPUs), and each computing module G has 4 input and output interfaces. At the first level, every 4 computing modules G (i.e., N1=4) form a first-level computing module, and 4 first-level computing modules are obtained. In the first-level computing module, the 4 computing modules G are interconnected with each other using the first link (shown as red lines in the figure), so that each computing module uses 3 input and output interfaces, then each first-level computing module can have 12 input and output interfaces for communication interconnection within the module, and each first-level computing module can have 4 input and output interfaces for communication interconnection outside the module. At the second level, the four first-level computing modules communicate and interconnect with each other in pairs. Specifically, a first-level computing module is connected to the other three first-level computing modules through the second link (shown as blue lines in the figure) through 3 input and output interfaces. For each computing module G, the number of input and output interfaces used for internal interconnection of the first-level computing modules is 3, and the number of input and output interfaces that can be used between the first-level computing modules is 1, that is, the number of input and output interfaces used for communication interconnection within the computing module is greater than the number of input and output interfaces used for communication interconnection between computing modules.

[0054] At least one of the first link and the second link is an optical link. Preferably, the first link is an electrical link and the second link is an optical link. In an optional embodiment, the first link is an optical link and the second link is an optical link.

[0055] It should be understood that according to the principles of the present invention, multiple computing modules can construct computing modules of more than 2 layers. For example, on the basis of 2 layers of computing modules, the 3rd layer of computing modules can be constructed with the 2nd layer of computing modules as units, and further, the 4th layer of computing modules can be constructed with the 3rd layer of computing modules as units. By analogy, more layers of computing modules can be constructed. For each layer of computing modules, the number of input and output interfaces used for communication interconnection within the computing module is greater than the number of input and output interfaces used for communication interconnection between computing modules.

[0056] [Example 3]

[0057] Figure 3 FIG. 4 shows the topological structure of the interconnection system according to Embodiment 3 of the present invention. Figure 3 As shown, in Example 3, the second-level computing module of Example 2 is used as a unit, and two second-level computing modules are interconnected through a third link to construct a third-level computing module. In other words, Example 3 uses a three-layer computing module constructed by 32 computing modules.

[0058] Similar to Example 2, every 4 computing modules G form a first-level computing module, and 8 first-level computing modules are obtained. The first-level computing modules are interconnected by the first link. Every 4 first-level computing modules construct a second-level computing module, and 2 second-level computing modules are obtained. The first-level computing modules in the second-level computing module are interconnected by the second link. The two second-level computing modules are interconnected by the third link (shown as a red line in the figure), thereby forming a third-level computing module.

[0059] Among them, each computing module G has 4 input and output interfaces, 3 of which are used for communication interconnection within the first-level computing module, and 1 is used for communication interconnection outside the first-level computing module. Therefore, each first-level computing module can have 4 input and output interfaces for external communication interconnection, of which 3 input and output interfaces are used for communication interconnection between the first-level computing modules in the second-level computing module, and 1 is used for communication interconnection outside the second-level computing module. Since the second-level computing module includes 4 first-level computing modules, each second-level computing module has 4 input and output interfaces for external communication interconnection, so that the 4 input and output interfaces of the two second-level computing modules are respectively interconnected through 4 third links. According to the interconnection system of this embodiment, each first-level computing module can have 12 input and output interfaces for communication interconnection within the module, and each first-level computing module can have 3 input and output interfaces for communication interconnection between 4 first-level computing modules within the second-level computing module; each second-level computing module can have 12 input and output interfaces for communication interconnection between 4 first-level computing modules within the module, and each second-level computing module can have 4 input and output interfaces for communication interconnection between 2 second-level computing modules within the third-level computing module. Thus, compared with the second-level computing module in Example 2 having 4 unused input and output interfaces, all input and output interfaces of all computing modules in Example 3 have been used, and through the construction of computing modules at multiple levels, the computing cluster is constructed to be large enough to obtain the largest possible communication bandwidth and computing power.

[0060] In some embodiments, at least one of the first link, the second link, and the third link is an optical link. Preferably, the third link is an optical link. More preferably, the second link and the third link are optical links.

[0061] In some embodiments, the first-level computing modules, the second-level computing modules, and the third-level computing modules are arranged along the same plane within their respective levels, and are arranged into a three-dimensional structure between different levels.

[0062] [Example 4]

[0063] Figure 4 The topological structure of the interconnection system according to Example 4 of the present invention is shown. In Example 4, the interconnection system includes 16 computing modules, each of which has 6 input and output interfaces, and every 4 computing modules constitute a first-level computing module, so that 4 first-level computing modules are obtained, which are: computing modules No. 0, 1, 2, and 3 in the green box constitute a first-level computing module, and computing modules No. 4, 5, 6, and 7 constitute another first-level computing module; computing modules No. 0, 1, 2, and 3 in the blue box constitute a first-level computing module, and computing modules No. 4, 5, 6, and 7 constitute another first-level computing module. The 3 input and output interfaces of each computing module are used for communication interconnection within the first-level computing module through the first link, and the 3 input and output interfaces can be used for external communication interconnection of the first-level computing module. Thus, each first-level computing module has 12 input and output interfaces for external communication interconnection.

[0064] Every two first-level computing modules constitute a second-level computing module. For example, the first-level computing module including green computing modules No. 0, 1, 2, and 3 and the first-level computing module including blue computing modules No. 0, 1, 2, and 3 are interconnected through the second link to form a second-level computing module. The first-level computing module including green computing modules No. 4, 5, 6, and 7 and the first-level computing module including blue computing modules No. 4, 5, 6, and 7 are interconnected through another second link to form another second-level computing module. These two second-level computing modules are interconnected through the third link to form a third-level computing module.

[0065] As can be seen from the above, each first-level computing module has 12 input and output interfaces for external communication and interconnection, of which 8 input and output interfaces are used for the interconnection between the two first-level computing modules inside the second-level computing module through the second link (shown as red lines in the figure), and the other 4 input and output interfaces are used for the interconnection between the two second-level computing modules inside the third-level computing module through the third link. According to the interconnection system of Example 4, each first-level computing module can have 12 input and output interfaces for communication and interconnection within the module, and each first-level computing module can have 8 input and output interfaces for communication and interconnection between the two first-level computing modules inside the second-level computing module; each second-level computing module can have 16 input and output interfaces for communication and interconnection between the two first-level computing modules within the module, and each second-level computing module can have 8 input and output interfaces for communication and interconnection between the two second-level computing modules inside the third-level computing module. According to the interconnection system of Example 4, computing modules within the first-level computing modules have communication paths through one hop direct communication between each other, computing modules within the second-level computing modules have communication paths through at most two hops direct communication between each other, and computing modules within the third-level computing modules have communication paths through at most three hops direct communication between each other.

[0066] In some implementations, at least one of the first link, the second link, and the third link is an optical link. Preferably, the third link is an optical link. More preferably, the second link and the third link are optical links. The topological structure of the computing system of this embodiment has a large communication bandwidth and low latency.

[0067] According to actual needs, the interconnection system of the present invention can also have layer 4, layer 5 or even more layers of computing modules.

[0068]

Calculation module

[0069] In some embodiments, the computing module in the interconnection system of the present invention includes a computing chip and a plurality of optical modules. Figure 5A and Figure 5B An example of a calculation module according to an embodiment of the present invention is shown, wherein: Figure 5A A top view of an example configuration of the computing module is shown, and Figure 5B A cross-sectional view of an example configuration of the computing module is shown.

[0070] like Figure 5A As shown in FIG. 5B , the computing module includes one or more optical modules 100 and a computing chip 300 . The one or more optical modules 100 and the computing chip 300 are arranged in the same PCB board 500 .

[0071] Each optical module 100 includes a plurality of electrical input / output interfaces and optical input / output interfaces. The number of optical modules is less than or equal to the number of electrical input / output interfaces.

[0072] The computing chip 300 and each optical module 300 are arranged on the same PCB board 500, and the computing chip 300 includes one or more input and output high-speed interfaces (for example, long-distance SerDes interfaces). Figure 5A Only four input and output high-speed interfaces (e.g., interfaces connected to wiring 304) of the computing chip 300 connected to the optical module are shown by way of example. When there is an electrical link between the computing modules in the interconnection system of the present invention, the computing chip also has one or more input and output high-speed interfaces that are not connected to the optical module, which are used for interconnection of the electrical link. Optionally, the computing chip 300 can be mounted on the PCB board 500 through the substrate 400.

[0073] In some embodiments, the computing chip 300 can be an artificial intelligence chip, for example, including but not limited to a graphics processing unit (GPU), a neural network processor (NPU), a tensor processing unit (TPU), an intelligent processing unit (IPU), a deep learning processor (DPU), etc.

[0074] It should be noted that the four input and output high-speed interfaces of the computing chip 300 shown above correspond to the four optical modules only for the convenience of description. In practical applications, any number of input and output high-speed interfaces can be configured for the computing chip 300 according to specific needs. In an embodiment, the output end of each input and output high-speed interface of the computing chip 300 performs parallel-to-serial conversion on the parallel signal in the computing chip 300 to output a first electrical signal (not shown in the figure), and the first electrical signal is provided to the corresponding electrical input port of the corresponding optical module 100 for photoelectric conversion. The parallel-to-serial conversion operation can be performed, for example, by a serializer (SERializer circuit in a SerDes interface), so that multiple low-speed parallel signals in the computing chip 300 can be converted into high-speed serial signals.

[0075] In addition, the input end of each input / output interface of the computing chip 300 performs serial-to-parallel conversion (not shown in the figure) on the second electrical signal received from the optical module 100 as a high-speed serial signal, thereby reconverting the high-speed serial signal into a low-speed parallel signal for use by the computing chip 300. The serial-to-parallel conversion operation can be performed, for example, by a deserializer (DESerializer circuit in a SerDes interface), so that the high-speed serial signal in the computing chip 300 can be converted into a low-speed parallel signal.

[0076] It should be noted that the wiring 304 in the PCB board 500 is a general term for the electrical connection wiring between the optical module 100 and the computing chip 300, and is not intended to represent a specific wiring.

[0077] Functionally, each optical module 100 is used to perform electrical-optical conversion and optical-electrical conversion.

[0078] The computing module 3000 also includes at least one high bandwidth memory (HBM) unit 302. For example, the HBM unit can cooperate with the computing chip 300 to perform various cache or storage functions.

[0079] In some embodiments, the plurality of HBM units 302 may be disposed on the substrate 400 together with the computing chip 300. Optionally, the plurality of HBM units are disposed on the substrate 400 and located on both sides of the computing chip 300.

[0080] In some implementations, the electrical input and output interface on the computing chip may be a PCIe physical interface, and the computing chip provides a PCIe electrical signal to the optical module, and the PCIe electrical signal is used to drive the optical module. Figure 6 1 is a side view showing a structural example of a computing module with a PCIe interface. In an embodiment, the computing chip is configured to have a PCIe physical interface, also known as a PCIe PHY interface. The PCIe PHY interface is used to perform protocol conversion on the input and output I / O signals of the computing chip so that they comply with the PCIe standard.

[0081] exist Figure 6 In the illustrated embodiment, the PCIe PHY interface on the computing chip 2020 can convert the digital signal inside the chip into a PCIe electrical signal. The PCIe electrical signal can be directly fed to the electrical input interface of the PCIe optical module 2010 for electrical-to-optical (EO) conversion.

[0082] In addition, the PCIe electrical signal output from the electrical output interface after optical-to-electrical (OE) conversion by the PCIe optical module 2010 can also be directly fed to the PCIe PHY interface on the computing chip 2020, which then converts it into a digital signal for use by the computing chip 2020.

[0083] In some implementations, a card slot (not shown) may be arranged in the middle or at an edge of the PCB board 2040 , and the optical module 2010 is pluggably arranged in the card slot.

[0084] The computing chip 2020 is arranged on a package substrate 2030 , and the package substrate 2030 is arranged on a PCB board 2040 .

[0085] The electrical interconnection between the computing chip 2020 and the optical module 2010 is achieved through the wiring 2060 in the packaging substrate 2030 and the PCB board 2040 .

[0086] For example, the edge of the PCB board 2040 may correspond to the front panel of the server device. In this case, the optical module 2010 can be easily plugged in and out on the front panel of the server, thus avoiding the influence on other components when operating the optical module.

[0087] In addition, in Figure 6 In the packaging example shown, the computing chip 2020 can be arranged as close to the optical module 2010 as possible, thereby reducing the length of the wiring 2060 between the computing chip 2020 and the optical module 2010, which is beneficial to improving signal integrity.

[0088] In addition, corresponding wiring 2060 can be arranged for the various data signals and control signals described above for the optical module, so that the optical module 2010 can perform EO conversion on various electrical signals from the computing chip 2020, and then the optical fiber array 2050 as shown in the figure transmits the converted optical signals.

[0089] In another embodiment, the optical module 2010 is fixedly arranged in the first area of ​​the PCB board 2040, instead of being Figure 6 The computing chip 2020 is arranged in a pluggable manner in a card slot of the PCB board 2040. At the same time, the computing chip 2020 is arranged on the packaging substrate 2030, and the packaging substrate 2030 is arranged in a second area on the PCB board 2040 that is different from the above-mentioned first area, that is, the packaging substrate 2030 and the optical module 2010 are not arranged in an overlapping manner.

[0090] In addition, in some embodiments, the optical link of the present invention is an active optical cable, and optical modules are provided at both ends of the active optical cable. The computing module includes a PCB board and a computing chip arranged on the PCB board. The active optical cable is pluggably connected to the electrical input and output interface of the computing chip. The electrical input and output interface on the computing chip is a PCIe physical interface or a SerDes interface.

[0091] Based on the above, those skilled in the art should understand that embodiments of the computing system of the present invention include the interconnection system described in any one of the above implementations or embodiments.

[0092] Those skilled in the art should understand that what is disclosed above is only an implementation mode of the present invention, and it certainly cannot be used to limit the scope of rights for patent protection requested by the present invention. Equivalent changes made according to the implementation mode of the present invention are still within the scope covered by the claims of the present invention.

Claims

1. An interconnection system for multiple computing modules, comprising multiple computing modules, each computing module having K input and output interfaces, where K is an integer greater than 0; The multiple computing modules constitute a computing module of layer j, where j is an integer greater than or equal to 2; in, Every N1 computing modules form a first-level computing module, and the M1 input and output interfaces of each computing module are used for communication and interconnection between the computing modules in the first-level computing module, and the computing modules in the first-level computing module are interconnected through the first link, and N1 is an even number greater than or equal to 2; Each N i The i-1th level computing modules are constructed into an i-th level computing module, and the M of each of the i-1th level computing modules i The input and output interfaces are used for communication and interconnection between the i-1th level computing modules in the i-th level computing module, and the i-1th level computing modules in the i-th level computing module are communicated and interconnected through the i-th link, where i is an integer greater than 1 and less than or equal to j, and N i is an integer greater than or equal to 2; For any level of computing modules from the 1st level to the jth level, the number of input and output interfaces used for internal communication and interconnection of the computing modules at that level is greater than the number of input and output interfaces used for communication and interconnection between computing modules at that level.

2. The interconnection system according to claim 1, characterized in that The number of input and output interfaces used for communication interconnection between various computing modules within the first-level computing module is greater than the number of input and output interfaces used for communication interconnection between various first-level computing modules within the second-level computing module.

3. The interconnection system according to claim 1, characterized in that The number of input and output interfaces used for communication interconnection between the i-1th level computing modules within the i-th level computing module is greater than the number of input and output interfaces used for communication interconnection between the i-th level computing modules within the i+1th level computing module.

4. The interconnection system according to claim 1, wherein: The number of input and output interfaces of each computing module used for the 1st to jth level computing modules is equal to K.

5. The interconnection system according to claim 1, wherein: The link used for communication interconnection inside at least one level computing module among the 1st level to the jth level computing modules is an optical link.

6. The interconnection system according to any one of claims 1 to 5, characterized in that: The multiple computing modules constitute a 2-layer computing module, i.e., i=j=2. Every 4 or 6 computing modules are constructed into a first-level computing module, that is, N1=4 or 6, The i-th link is the second link, and the second link is an optical link.

7. The interconnection system according to claim 6, characterized in that The computing modules in the first-level computing module are arranged along a first plane. The first-level computing modules in the second-level computing modules are arranged along a first plane or along a second plane perpendicular to the first plane.

8. The interconnection system according to claim 6, wherein: N1=4, that is, every 4 computing modules are constructed into a first-level computing module, In the second-level computing module, N2 of the first-level computing modules form a ring topology structure, and N2≥4.

9. The interconnection system according to claim 8, characterized in that K=8, M1=6, the four computing modules in the first-level computing module have the first link directly connected to each other, and the computing modules communicate with each other through two input and output interfaces.

10. The interconnection system according to claim 8, characterized in that K=5, M1=3, the four computing modules in the first-level computing module have the first link directly connected to each other, and the computing modules communicate with each other through an input-output interface.

11. The interconnection system according to any one of claims 1 to 5, characterized in that: The multiple computing modules constitute a three-layer computing module, i.e., j=3. Every 4 or 6 computing modules are constructed into a first-level computing module, that is, N1=4 or 6, The link used for communication interconnection inside the second-level computing module is the second link, and the link used for communication interconnection inside the third-level computing module is the third link, wherein the second link is an electrical link or an optical link, and the third link is an optical link.

12. The interconnection system according to claim 11, characterized in that The first-level computing modules, the second-level computing modules, and the third-level computing modules are arranged along the same plane in their respective corresponding levels, and are arranged into a three-dimensional structure between different levels.

13. The interconnection system according to claim 1, wherein: There are direct communication channels between the computing modules in the first level.

14. The interconnection system according to claim 13, wherein: There is a communication channel with a maximum communication hop number of 2 between each of the computing modules in the second level.

15. The interconnection system according to claim 14, characterized in that There is a communication channel with a maximum communication hop number of 3 between each of the computing modules in the third level.

16. The interconnection system according to claim 5, wherein: The computing module includes a computing chip and a plurality of optical modules. The computing chip has a plurality of electrical input and output interfaces, and each of the optical modules includes an electrical input and output interface and an optical input and output interface. At least part of the electrical input and output interfaces on the computing chip are directly electrically connected to the electrical input and output interfaces on the optical modules, wherein the number of the optical modules is less than or equal to K.

17. The interconnection system according to claim 16, wherein: The computing module further includes a PCB board, and at least part of the electrical input and output interfaces on the computing chip are electrically connected to the electrical input and output interfaces on the optical module through wiring on the PCB board.

18. The interconnection system of claim 17, wherein: The electrical input and output interface on the computing chip is a SerDes interface.

19. The interconnection system of claim 17, wherein: The electrical input and output interface on the computing chip is a PCIe physical interface. The computing chip provides a PCIe electrical signal to the optical module, and the PCIe electrical signal is used to drive the optical module.

20. The interconnection system of claim 19, wherein: A card slot is arranged on the PCB board, and the optical module is pluggably arranged in the card slot.

21. The interconnection system of claim 1, wherein: The computing module includes a computing chip and an HBM memory chip.

22. The interconnection system of claim 5, wherein: The optical link is an active optical cable, and optical modules are provided at both ends of the active optical cable. The computing module includes a PCB board and a computing chip arranged on the PCB board. The active optical cable is pluggably connected to the electrical input and output interface of the computing chip.

23. A computing system, characterized in that: An interconnection system comprising any one of claims 1 to 22.

Images