Interconnection system for multiple computing modules, and computing system

Abstract

An interconnection system for multiple computing modules, and a computing system. Multiple computing modules constitute a first-level computing module group, then multiple first-level computing module groups constitute a second-level computing module group, and so on, computing module groups at two or more levels are constructed. For a computing module group at any level, the number of input / output interfaces for communication interconnection within said computing module group is greater than the number of input / output interfaces for communication interconnections between other computing module groups at the same level.

Description

Interconnection system and computing system for multiple computing modules

[0001] This application claims priority to and the benefit of Chinese Patent Application No. 2024105300400, filed April 28, 2024, entitled “Interconnection system and computing system for multiple computing modules,” the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the field of computers, and more specifically, to an interconnection system and computing system for multiple computing modules. BACKGROUND

[0003] According to OpenAI, an artificial intelligence (AI) research company in the United States, the computing amount of AI models grows at a rate much faster than the computing power of computing hardware. As AI accelerators continuously improve in computing power through process iteration and chip architecture innovation, the interconnection bandwidth between AI accelerators also continues to grow. The interconnection network of AI accelerators has become a key to improving overall computing power.

[0004] However, as the response of electrical channels decays with the increase in signal rate, higher-rate interfaces often involve more complex architecture and circuit design, introducing delay at a cost and consuming more power and occupying more chip area, thereby limiting the chip input / output bandwidth. In addition, longer metal wiring distances also further worsen the loss characteristics of the circuit, limiting the interconnection distance between AI accelerators.

[0005] In order to break through the limitation of long-distance high-frequency electrical link loss, optical interconnection is favored as a potential technology route. Although optical interconnection breaks the distance limit, the number of interfaces of each computing node is limited, and there is an urgent need to develop an effective topology structure using limited interfaces to meet the demand for large computing power and high bandwidth at the same time. SUMMARY

[0006] In one aspect, embodiments of the present disclosure provide an interconnection system for multiple computing modules. The interconnection system includes multiple computing modules, each computing module having K input / output interfaces, K being an integer greater than 0. The multiple computing modules form j layers of computing modules, j being an integer greater than or equal to 2, wherein every N1 computing modules form a first-level computing module, M1 input / output interfaces of each computing module are used for communication interconnection between each computing module in the first-level computing module, each computing module in the first-level computing module is communicatively interconnected by a first link, and N1 is an even integer greater than or equal to 2; every N2 computing modules form a second-level computing module, M2 input / output interfaces of each computing module are used for communication interconnection between each computing module in the second-level computing module, each computing module in the second-level computing module is communicatively interconnected by a second link, and N2 is an even integer greater than or equal to 2; and each computing module in the second-level computing module is communicatively interconnected by a third link. ieach of the i-1 level computing modules is configured to form an i level computing module, and the number of input / output interfaces for communication interconnection between each of the i-1 level computing modules inside the i level computing module is M i each of the i-1 level computing modules inside the i level computing module is configured to form an i level computing module, and the number of input / output interfaces for communication interconnection between each of the i-1 level computing modules inside the i level computing module is M i N is an integer greater than or equal to 2. For any one of the first level to the j level computing modules, the number of input / output interfaces for communication interconnection inside the computing module is greater than the number of input / output interfaces for communication interconnection between the computing modules.

[0007] In some embodiments, the number of input / output interfaces for communication interconnection between each of the computing modules inside the first level computing module is greater than the number of input / output interfaces for communication interconnection between each of the first level computing modules inside the second level computing module.

[0008] In some embodiments, the number of input / output interfaces for communication interconnection between each of the i-1 level computing modules inside the i level computing module is greater than the number of input / output interfaces for communication interconnection between each of the i level computing modules inside the i+1 level computing module.

[0009] In some embodiments, the number of input / output interfaces for each of the computing modules for the first level to the j level computing modules is equal to K.

[0010] In some embodiments, the link for communication interconnection inside at least one of the first level to the j level computing modules is an optical link.

[0011] In some embodiments, the plurality of computing modules form a 2 level computing module, i.e. i=j=2. Each 4 or 6 of the computing modules is configured to form a first level computing module, i.e. N1=4 or 6. The i link is a second link, and the second link is an optical link.

[0012] In some embodiments, each of the computing modules inside the first level computing module is arranged along a first plane, and each of the first level computing modules inside the second level computing module is arranged along a first plane or a second plane perpendicular to the first plane.

[0013] In some embodiments, N1=4, i.e. each 4 of the computing modules is configured to form a first level computing module. In the second level computing module, N2 of the first level computing modules form a ring topology, and N2≥4.

[0014] In some embodiments, K=8, M1=6, the first link between each two of the four computing modules in the first level computing module has a direct connection, and each two of the computing modules communicates through two input-output interfaces.

[0015] In some embodiments, K=5, M1=3, the first link between each two of the four computing modules in the first level computing module has a direct connection, and each two of the computing modules communicates through one input-output interface.

[0016] In some embodiments, the plurality of computing modules form a three-level computing module, i.e., j=3. Every four or six of the computing modules form a first level computing module, i.e., N1=4 or 6. The link for communication interconnection within the second level computing module is a second link, and the link for communication interconnection within the third level computing module is a third link, wherein the second link is an electrical link or an optical link, and the third link is an optical link.

[0017] In some embodiments, the first level computing module, the second level computing module, and the third level computing module are arranged in the same plane in the respective corresponding level, and arranged in a three-dimensional structure between different levels.

[0018] In some embodiments, the computing modules in the first level have a direct connection between each two of the computing modules. In some embodiments, the computing modules in the second level have a communication channel with a maximum communication hop count of 2 between each two of the computing modules. In some embodiments, the computing modules in the third level have a communication channel with a maximum communication hop count of 3 between each two of the computing modules.

[0019] In some embodiments, the computing module includes a computing chip and a plurality of optical modules. The computing chip has a plurality of electrical input-output interfaces, and each of the optical modules includes an electrical input-output interface and an optical input-output interface. At least part of the electrical input-output interfaces on the computing chip are directly electrically connected with the electrical input-output interfaces on the optical modules, wherein the number of the optical modules is less than or equal to K.

[0020] In some embodiments, the computing module further includes a PCB board, and at least part of the electrical input-output interfaces on the computing chip are electrically connected with the electrical input-output interfaces on the optical modules through wiring on the PCB board.

[0021] In some embodiments, the electrical input-output interfaces on the computing chip are SerDes interfaces, which include a serializer and a deserializer.

[0022] In some embodiments, the electrical input / output interface on the computing chip is a PCIe physical interface, and the computing chip provides a PCIe electrical signal to the optical module, and the PCIe electrical signal is used to drive the optical module.

[0023] In some embodiments, a card slot is arranged on the PCB board, and the optical module is plug-in arranged in the card slot.

[0024] In some embodiments, the computing module includes a computing chip and an HBM memory chip

[0025] In some embodiments, the optical link is an active optical cable, and the active optical cable is provided with optical modules at both ends. The computing module includes a PCB board and a computing chip arranged on the PCB board. The active optical cable is plug-in connected with the electrical input / output interface of the computing chip. The electrical input / output interface on the computing chip is a PCIe physical interface or a SerDes interface.

[0026] In another aspect, embodiments of the present application provide a computing system including the interconnection system of any one of the embodiments of the present application.

[0027] According to the above, in the interconnection system proposed in the embodiments of the present application, a plurality of computing modules are grouped to form a first-level computing module, and then a plurality of first-level computing modules are grouped to form a second-level computing module, and so on, so that a computing module of 2 layers or more can be constructed. The inventor found that the increase of communication bandwidth can reduce the delay, and the increase of interconnection bandwidth in a certain level computing module can significantly reduce the delay, but the increase of interconnection bandwidth between the computing modules of the level does not significantly reduce the delay. The size of the communication bandwidth is limited by the number of input / output interfaces. Therefore, in the interconnection system, the number of input / output interfaces used for internal communication interconnection of a certain level computing module is greater than the number of input / output interfaces used between the computing modules of the level (i.e., in another level computing module formed by the computing modules of the level). For example, the number of input / output interfaces used in the first-level computing module is greater than the number of input / output interfaces used between the first-level computing modules (second level), the number of interfaces used in the second-level computing module is greater than the number of input / output interfaces used between the second-level computing modules (third level), and so on. Thus, the limitation of the number of interfaces of a single computing node (computing module) is broken, and the computing scale can be flexibly adjusted according to the demand to meet the demand of large computing power and high bandwidth.

[0028] And, according to the above-mentioned embodiments, at least part of the links are interconnected using optical links, which saves power consumption and reduces latency, and is conducive to large-scale expansion of the computing modules and greatly improves the communication capability. Of course, the computing modules in close proximity can still choose to communicate through electrical links, so the configuration of the interconnection system of the present application is flexible, and the cost efficiency can be flexibly controlled.

[0029] Various aspects, features, and advantages of the present embodiments will be described in connection with the following figures. BRIEF DESCRIPTION OF DRAWINGS

[0030] Fig. 1 is a schematic diagram showing the topology of an interconnection system according to Embodiment 1 of the present application.

[0031] Fig. 2 is a schematic diagram showing the topology of an interconnection system according to Embodiment 2 of the present application.

[0032] Fig. 3 is a schematic diagram showing the topology of an interconnection system according to Embodiment 3 of the present application.

[0033] Fig. 4 is a schematic diagram showing the topology of an interconnection system according to Embodiment 4 of the present application.

[0034] Figs. 5A and 5B show an example of a computing module according to an embodiment of the present application, wherein Fig. 5A is a top view showing an example configuration of the computing module, and Fig. 5B is a cross-sectional view showing an example configuration of the computing module.

[0035] Fig. 6 is a side view showing an example of the structure of a computing module having a PCIe interface. DETAILED DESCRIPTION

[0036] Next, the example embodiments will be described in more detail with reference to the accompanying drawings. It should be understood that the application can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided as examples so that the disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art. Processes, elements, and techniques that are not necessary for understanding the aspects and features of the present application can not be described.

[0037] Unless otherwise defined, like reference numerals in the drawings and the text indicate like elements throughout the various drawings and the text, and thus, the description thereof can not be repeated. In addition, features or aspects within each example embodiment should generally be considered to be applicable to other similar features or aspects in other example embodiments.

[0038] The terminology used herein is for the purpose of describing particular embodiments and is not intended to be limiting of the application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises", "comprising", "includes" and / or "including" when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0039] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and / or the

[0040] Embodiments of the present application generally relate to an interconnection system and a computing system for a plurality of computing modules. A first level computing module is built from a plurality of computing module groupings, and then a second level computing module is built from a plurality of first level computing modules, and so on, so that a 2-level and above computing module can be built. For any level computing module, the number of input / output interfaces used for intra-communication interconnection of the level computing module is greater than the number of input / output interfaces used for inter-communication interconnection between the level computing modules.

[0041] In some embodiments, the interconnection system includes a plurality of computing modules, each computing module having K input / output interfaces, K being an integer greater than 0. The plurality of computing modules form a j-level computing module, j being an integer greater than or equal to 2, wherein every N1 computing modules build a first level computing module, M1 input / output interfaces of each computing module are used for communication interconnection between each of the computing modules within the first level computing module, each of the computing modules within the first level computing module are communicatively interconnected via first links, N1 being an even integer greater than or equal to 2; every N i i-1 level computing modules build an i-level computing module, M i i-1 level computing modules within the i-level computing module are communicatively interconnected via i links, i being an integer greater than 1 and less than or equal to j, N iis an integer greater than or equal to 2. For any one of the level-1 to level-j computing modules, the number of input / output interfaces for internal communication interconnection of the level computing module is greater than the number of input / output interfaces for communication interconnection between the level computing modules.

[0042] [Example 1]

[0043] Fig. 1 shows the topology of an interconnection system according to Example 1 of the present application. The interconnection system includes a plurality of computing modules (e.g., XPU: certain processing unit), each of which has 8 input / output interfaces, i.e., K = 8, exemplarily. The plurality of computing modules are constructed into 2-level computing modules, i.e., a level-1 computing module 1001 and a level-2 computing module 1002.

[0044] In the interconnection system, every 4 computing modules are constructed into a level-1 computing module 1001, i.e., N1= 4. As shown in Fig. 1, within the level-1 computing module, there are computing module 0, computing module 1, computing module 2, and computing module 3, each of which has 6 input / output interfaces (i.e., M1= 6) for communication interconnection between the computing modules within the level-1 computing module. Specifically, the computing module 0, the computing module 1, the computing module 2, and the computing module 3 are interconnected in pairs by using the first link (shown as a dashed line in the figure), the interconnection between the computing module 0 and the computing module 1 uses two input / output interfaces of both sides, the interconnection between the computing module 0 and the computing module 2 uses two input / output interfaces of both sides, and the interconnection between the computing module 0 and the computing module 3 uses two input / output interfaces of both sides, so that the computing modules within the level-1 computing module can form a bidirectional 64 Gb / s interconnection channel. The first link can be an optical link or an electrical link. In an exemplary manner, since the computing modules within the level-1 computing module have a relatively short communication distance (usually less than 1 meter), whether the first link is an electrical link or an optical link can be selected as needed.

[0045] Further, in the interconnection system, the N2 first-level computing modules are connected by the second links (shown as solid lines in the figure) to form a ring structure (each computing module reserves two input / output interfaces), thereby constructing the second-level computing module 1002. Among them, the two input / output interfaces of each computing module are used for communication interconnection between the first-level computing modules, that is, the eight input / output interfaces of each first-level computing module 1001 are used for communication interconnection between the computing modules in the second-level computing module, and M2=8. For example, an input / output interface of the No. 0 computing module in a certain first-level computing module is connected to an input / output interface of the No. 0 computing module in the upstream adjacent first-level computing module through the second link, and the other input / output interface is connected to an input / output interface of the No. 0 computing module in the downstream adjacent first-level computing module through the second link. In this way, the other computing modules 1, 2, and 3 of the adjacent first-level computing modules are interconnected with each other, thereby forming a bidirectional 32 Gb / s interconnection channel between the computing modules of different first-level computing modules. In other words, the first-level computing modules in the second-level computing module are connected by bidirectional 32 Gb / s interconnection channels. The second link can be an electrical link or an optical link. Because N2 can be any integer greater than 3, preferably, 4≤N2≤16. When the distance between the interconnected computing modules is greater than 1 meter, the signal stability becomes poor and the delay is serious, and the optical interconnection can overcome the above defects, therefore, the second link is preferably an optical link.

[0046] In embodiment 1, the number of input / output interfaces of each computing module used for internal interconnection of the first-level module is 6, and the number of input / output interfaces used for interconnection between the first-level modules is 2. That is, the number of input / output interfaces used for internal interconnection of the first-level computing module is greater than the number of input / output interfaces used for interconnection between the first-level computing modules (i.e., for internal interconnection of the second-level computing module). The increase of communication bandwidth can reduce the delay, and the inventors found that the increase of interconnection bandwidth in the first-level computing module significantly reduces the delay, but the increase of interconnection bandwidth between the first-level computing modules does not significantly reduce the delay. Therefore, by increasing the number of input / output interfaces used for internal communication interconnection of the computing module than the number of input / output interfaces used for communication interconnection between the computing modules, more computing modules are expanded with limited input / output interfaces, and the computing power is improved while increasing the bandwidth and reducing the delay.

[0047] Although in embodiment 1, each of the four computing modules is constructed into a first-level computing module, that is, N1=4, it should be understood that in some embodiments, each of the six computing modules can be constructed into a first-level computing module, that is, N1=6.

[0048] In some embodiments, each of the computing modules in the first-level computing module is arranged along a first plane, and each of the first-level computing modules in the second-level computing module is arranged along a first plane or a second plane perpendicular to the first plane.

[0049] In some embodiments, at least one of the first link and the second link is an optical link using an optical input / output interface.

[0050] In some embodiments, the number of input / output interfaces for the input / output interface of the first-level to second-level computing module of each computing module is equal to 8. That is, each input / output interface is used to the maximum extent to obtain the largest possible communication bandwidth, while expanding the scale of the computing module and improving the computing power.

[0051] In Embodiment 1, K=8, M1=6, the first link with direct connection between each of the four computing modules in the first-level computing module, and the computing modules communicate through two input / output interfaces. The present application is not limited to this, in an alternative embodiment, K=5, M1=3, the first link with direct connection between each of the four computing modules in the first-level computing module, and the computing modules communicate through one input / output interface.

[0052]

Embodiment 2

[0053] Figure 2 shows the topology of the interconnection system according to Embodiment 2 of the present application. Embodiment 1 exemplarily illustrates a 2-level interconnection system with each computing module having 8 input / output interfaces, but the present application is not limited to this, and the computing module can generally have 4, 5, 6, 7, 8 or even more input / output interfaces. Here, Embodiment 2 illustrates a 2-level interconnection system with K=4 (i.e. each computing module has 4 input / output interfaces).

[0054] In embodiment 2, the interconnection system includes 16 computing modules G (GPUs), each of which has 4 input / output interfaces. At the first level, each 4 computing modules G (i.e., N1=4) forms a first level computing module, resulting in 4 first level computing modules. Within the first level computing module, the 4 computing modules G are interconnected with each other by the first link (shown as dashed lines in the figure), so that each computing module uses 3 input / output interfaces. Each first level computing module can have 12 input / output interfaces for intra-module communication interconnection, and each first level computing module can have 4 input / output interfaces for inter-module communication interconnection. At the second level, the 4 first level computing modules are interconnected with each other, specifically, one first level computing module is connected to the other three first level computing modules by the second link (shown as solid lines in the figure) through 3 input / output interfaces. For each computing module G, the number of input / output interfaces used for intra-module interconnection is 3, and the number of input / output interfaces used for inter-module interconnection is 1, i.e., the number of input / output interfaces used for intra-module communication interconnection is greater than the number of input / output interfaces used for inter-module communication interconnection.

[0055] At least one of the first link and the second link is an optical link. Preferably, the first link is an electrical link and the second link is an optical link. In an alternative embodiment, the first link is an optical link and the second link is an optical link.

[0056] It should be understood that, according to the principles of the present application, a plurality of computing modules can be constructed into a computing module of more than 2 levels. For example, on the basis of a 2-level computing module, a 3-level computing module is constructed by taking the 2-level computing module as a unit, and further, a 4-level computing module can be constructed by taking the 3-level computing module as a unit. By analogy, a computing module of more levels can be constructed. For each level of the computing module, the number of input / output interfaces used for intra-module communication interconnection is greater than the number of input / output interfaces used for inter-module communication interconnection.

[0057]

Embodiment 3

[0058] Figure 3 shows the topology of the interconnection system according to embodiment 3 of the present application. As shown in Figure 3, embodiment 3 takes the 2-level computing module of embodiment 2 as a unit, and 2 2-level computing modules are interconnected by a third link to construct a 3-level computing module. That is, embodiment 3 constructs a 3-level computing module by using 32 computing modules.

[0059] Similar to embodiment 2, each 4 computing modules G form a first level computing module, and 8 first level computing modules are obtained, and the first level computing modules are internally connected and communicated by using the first link. Each 4 first level computing modules form a second level computing module, and 2 second level computing modules are obtained, and the first level computing modules in the second level computing module are connected and communicated by using the second link. The 2 second level computing modules are connected and communicated by using the third link (shown as a dotted line in the figure), thereby forming a third level computing module.

[0060] Each computing module G has 4 input and output interfaces, 3 of which are used for communication and interconnection within the first level computing module, and 1 is used for communication and interconnection outside the first level computing module. Thus, each first level computing module can have 4 input and output interfaces for external communication and interconnection, of which 3 input and output interfaces are used for communication and interconnection between the first level computing modules in the second level computing module, and 1 is used for communication and interconnection outside the second level computing module. Since the second level computing module includes 4 first level computing modules, each second level computing module has 4 input and output interfaces for external communication and interconnection, and thus the 4 input and output interfaces of the 2 second level computing modules are respectively connected and communicated by using 4 third links. According to the interconnection system of the present embodiment, each first level computing module can have 12 input and output interfaces for communication and interconnection within the module, and each first level computing module can have 3 input and output interfaces for communication and interconnection between the 4 first level computing modules in the second level computing module; each second level computing module can have 12 input and output interfaces for communication and interconnection between the 4 first level computing modules in the module, and each second level computing module can have 4 input and output interfaces for communication and interconnection between the 2 second level computing modules in the third level computing module. Thus, compared with the 4 input and output interfaces of the second level computing module not used in embodiment 2, all input and output interfaces of all computing modules in embodiment 3 are used, and through the construction of multiple levels of computing modules, the computing cluster is constructed to be large enough, and the largest possible communication bandwidth and computing power are obtained.

[0061] In some embodiments, at least one of the first link, the second link, and the third link is an optical link. Preferably, the third link is an optical link. More preferably, the second link and the third link are optical links.

[0062] In some embodiments, the first level computing module, the second level computing module, and the third level computing module are arranged along the same plane in the respective corresponding level, and arranged into a three-dimensional structure between different levels.

[0063]

Embodiment 4

[0064] Figure 4 shows the topology of the interconnection system according to embodiment 4 of the present application. In embodiment 4, the interconnection system comprises 16 computing modules, each of which has 6 input / output interfaces, and each 4 computing modules form a first level computing module group, resulting in 4 first level computing module groups, which are: the 0, 1, 2, 3 computing modules in solid-line boxes form a first level computing module group, and the 4, 5, 6, 7 computing modules form another first level computing module group; the 0, 1, 2, 3 computing modules in dashed-line boxes form a first level computing module group, and the 4, 5, 6, 7 computing modules form another first level computing module group. Three input / output interfaces of each computing module are used for communication interconnection within the first level computing module group through first links, and the other three input / output interfaces can be used for external communication interconnection of the first level computing module group. Thus, each first level computing module group has 12 input / output interfaces for external communication interconnection.

[0065] Each 2 first level computing module groups form a second level computing module group, for example, the first level computing module group comprising the 0, 1, 2, 3 computing modules in solid-line boxes and the first level computing module group comprising the 0, 1, 2, 3 computing modules in dashed-line boxes are interconnected by a second link to form a second level computing module group, and the first level computing module group comprising the 4, 5, 6, 7 computing modules in solid-line boxes and the first level computing module group comprising the 4, 5, 6, 7 computing modules in dashed-line boxes are interconnected by another second link to form another second level computing module group. The two second level computing module groups are interconnected by a third link to form a third level computing module group.

[0066] As known from the above, each first-level computing module has 12 input / output interfaces for external communication interconnection, of which 8 input / output interfaces are used for interconnection between 2 first-level computing modules within the second-level computing module through the second link (shown as a dot-dash line in the figure), and the other 4 input / output interfaces are used for interconnection between 2 second-level computing modules within the third-level computing module through the third link. According to the interconnection system of Embodiment 4, each first-level computing module can have 12 input / output interfaces for communication interconnection within the module, and each first-level computing module can have 8 input / output interfaces for communication interconnection between 2 first-level computing modules within the second-level computing module; each second-level computing module can have 16 input / output interfaces for communication interconnection between 2 first-level computing modules within the module, and each second-level computing module can have 8 input / output interfaces for communication interconnection between 2 second-level computing modules within the third-level computing module. According to the interconnection system of Embodiment 4, each pair of computing modules within the first-level computing module has a communication path for direct communication through one hop, each pair of computing modules within the second-level computing module has a communication path for direct communication through at most two hops, and each pair of computing modules within the third-level computing module has a communication path for direct communication through at most three hops.

[0067] In some embodiments, at least one of the first link, the second link, and the third link is an optical link. Preferably, the third link is an optical link. More preferably, the second link and the third link are optical links. The topology of the computing system of the present embodiment has large communication bandwidth and low delay.

[0068] According to actual needs, the interconnection system of the present application can further have fourth-level, fifth-level, or even more levels of computing modules.

[0069]

Computing module

[0070] In some embodiments, the computing module in the interconnection system of the present application comprises a computing chip and a plurality of optical modules. FIG. 5A and FIG. 5B show an example of a computing module according to an embodiment of the present application, wherein FIG. 5A shows a top view of an example configuration of the computing module, and FIG. 5B shows a cross-sectional view of the example configuration of the computing module.

[0071] As shown in FIG. 5A or 5B, the computing module comprises one or more optical modules 100 and a computing chip 300. The one or more optical modules 100 and the computing chip 300 are arranged in the same PCB board 500.

[0072] Each optical module 100 comprises a plurality of electrical input / output interfaces and optical input / output interfaces. The number of optical modules is less than or equal to the number of electrical input / output interfaces.

[0073] The computing chip 300 and the individual optical modules 300 are arranged on the same PCB board 500, and the computing chip 300 includes one or more input / output high-speed interfaces (for example, long-distance SerDes interfaces). For simplicity, only 4 input / output high-speed interfaces of the computing chip 300 (for example, interfaces connected with the wirings 304) are exemplarily shown in FIG. 5A. When there is an electrical link between the connections of the computing modules in the interconnection system of the present application, the computing chip also has one or more input / output high-speed interfaces that are not connected with the optical modules, for the interconnection of the electrical link. Alternatively, the computing chip 300 can be mounted on the PCB board 500 through the substrate 400.

[0074] In some embodiments, the computing chip 300 can be an artificial intelligence chip, for example, including but not limited to a graphics processing unit (GPU), a neural network processing unit (NPU), a tensor processing unit (TPU), an intelligent processing unit (IPU), a deep learning processing unit (DPU), etc.

[0075] It should be noted that the 4 input / output high-speed interfaces of the computing chip 300 shown above correspond to the 4 optical modules only for the convenience of description. In actual applications, any number of input / output high-speed interfaces can be configured for the computing chip 300 according to specific needs. In embodiments, the output end of each input / output high-speed interface of the computing chip 300 performs parallel-to-serial conversion on the parallel signals within the computing chip 300 to output a first electrical signal (not shown in the figure), which is provided to the corresponding electrical input port of the corresponding optical module 100 for optoelectronic conversion. The parallel-to-serial conversion operation can be performed, for example, by a serializer (SERializer circuit in a SerDes interface), so that the multiple low-speed parallel signals within the computing chip 300 can be converted into a high-speed serial signal.

[0076] In addition, the input end of each input / output interface of the computing chip 300 performs serial-to-parallel conversion (not shown in the figure) on the second electrical signal received from the optical module 100 as a high-speed serial signal, so as to reconvert the high-speed serial signal into a low-speed parallel signal for use by the computing chip 300. The serial-to-parallel conversion operation can be performed, for example, by a deserializer (DESerializer circuit in a SerDes interface), so that the high-speed serial signal within the computing chip 300 can be converted into a low-speed parallel signal.

[0077] It should be noted that the wirings 304 in the PCB board 500 are a general term for the electrical connection wirings between the optical modules 100 and the computing chip 300, and are not intended to represent a certain specific wiring.

[0078] Functionally, each optical module 100 is used for electrical-optical conversion and optical-electrical conversion.

[0079] The computing module 3000 also includes at least one high bandwidth memory (HBM) unit 302. For example, the HBM unit can cooperate with the computing chip 300 to perform various cache or storage functions.

[0080] In some embodiments, multiple HBM units 302 can be co-located with the computing chip 300 on the substrate 400. Optionally, multiple HBM units are co-located on the substrate 400 and on both sides of the computing chip 300.

[0081] In some embodiments, the electrical input / output interface on the computing chip can be a PCIe physical interface, and the computing chip provides a PCIe electrical signal to the optical module, which is used to drive the optical module. FIG. 6 is a side view showing an example of the structure of a computing module with a PCIe interface. In an embodiment, the computing chip is configured to have a PCIe physical interface, also referred to as a PCIe PHY interface. The PCIe PHY interface is used to protocol convert the input / output (I / O) signals of the computing chip so that they conform to the PCIe standard.

[0082] In the embodiment shown in FIG. 6, the PCIe PHY interface on the computing chip 2020 can convert digital signals inside the chip into PCIe electrical signals. The PCIe electrical signals can be directly fed to the electrical input interface of the PCIe optical module 2010 for electrical-optical (E-O) conversion.

[0083] In addition, the PCIe electrical signals output from the electrical output interface after optical-electrical (O-E) conversion by the PCIe optical module 2010 can also be directly fed to the PCIe PHY interface on the computing chip 2020, which in turn converts them into digital signals for use by the computing chip 2020.

[0084] In some embodiments, a card slot (not shown) can be arranged at the middle or edge of the PCB board 2040, and the optical module 2010 is arranged pluggably in the card slot.

[0085] The computing chip 2020 is arranged on the packaging substrate 2030, and the packaging substrate 2030 is arranged on the PCB board 2040.

[0086] In which the electrical interconnection between the computing chip 2020 and the optical module 2010 is realized by the wiring 2060 in the packaging substrate 2030 and the PCB board 2040.

[0087] For example, the edge of the PCB board 2040 can correspond to the front panel of the server device. In this case, the optical module 2010 can be easily plugged and unplugged on the front panel of the server, so that the influence on other components during the operation of the optical module can be avoided.

[0088] In addition, in the case of the packaging example shown in FIG. 6, the computing chip 2020 can be arranged as close to the optical module 2010 as possible, so as to reduce the length of the wiring 2060 between the computing chip 2020 and the optical module 2010, and facilitate the improvement of the integrity of the signal.

[0089] In addition, the respective wiring 2060 can be arranged for various data signals and control signals and the like described above for the optical module, so that the optical module 2010 can perform E-O conversion on various electrical signals from the computing chip 2020, and then transmit the converted optical signals by the fiber array 2050 as shown.

[0090] In another embodiment, the optical module 2010 is fixedly arranged in the first area of the PCB board 2040, instead of being plug-in arranged in the card slot of the PCB board 2040 as shown in FIG. 6. Meanwhile, the computing chip 2020 is arranged on the packaging substrate 2030, and the packaging substrate 2030 is arranged in a second area of the PCB board 2040 different from the above-mentioned first area, i.e., the packaging substrate 2030 and the optical module 2010 are not arranged in an overlapping manner.

[0091] In addition, in some embodiments, the optical link of the present application is an active optical cable, two ends of which are provided with optical modules. The computing module includes a PCB board and a computing chip arranged on the PCB board. The active optical cable is plug-in connected with the electrical input and output interface of the computing chip. The electrical input and output interface on the computing chip is a PCIe physical interface or a SerDes interface.

[0092] Based on the above, it should be understood by those skilled in the art that the embodiments of the computing system of the present application include the interconnection system described in any one of the above embodiments or examples.

[0093] It should be understood by those skilled in the art that the above disclosure is only an embodiment of the present application, and of course cannot limit the scope of the patent protection right claimed by the present application. The equivalent changes made according to the embodiments of the present application still fall within the scope of the claims of the present application.

Claims

1. An interconnection system for a plurality of computing modules, comprising a plurality of computing modules, each computing module having K input / output interfaces, K being an integer greater than 0; said plurality of computing modules form j layers of computing modules, j being an integer greater than or equal to 2; wherein every N1 computing modules form a first level computing module, M1 input / output interfaces of each of said computing modules are used for communication interconnection between each of said computing modules within said first level computing module, each of said computing modules within said first level computing module are communication interconnected by first links, N1 being an even integer greater than or equal to 2; Every N i The i-1 level computing module is constructed into an i level computing module, M i Input and output interfaces of each of the i-1 level computing module are used for communication interconnection between each of the i-1 level computing module inside the i level computing module, each of the i-1 level computing module inside the i level computing module is communicated and interconnected through the i link, i is an integer greater than 1 and less than or equal to j, N i Is an integer greater than or equal to 2; for any one of the first level to the jth level computing modules, the number of input / output interfaces used for communication interconnection within the level is greater than the number of input / output interfaces used for communication interconnection between the level computing modules.

2. The interconnection system of claim 1, wherein, the number of input / output interfaces used for communication interconnection between each of the computing modules within the first level computing module is greater than the number of input / output interfaces used for communication interconnection between each of the first level computing modules within the second level computing module.

3. The interconnection system of claim 1, wherein, the number of input / output interfaces used for communication interconnection between each of the i-1th level computing modules within the ith level computing module is greater than the number of input / output interfaces used for communication interconnection between each of the ith level computing modules within the i+1th level computing module.

4. The interconnection system of claim 1, wherein, the number of input / output interfaces of each of said computing modules used for the first level to the jth level computing modules is equal to K.

5. The interconnection system of claim 1, wherein, the links used for communication interconnection within at least one of the first level to the jth level computing modules are optical links.

6. The interconnection system of any one of claims 1 to 5, wherein, said plurality of computing modules form 2 layers of computing modules, i.e. i = j = 2, every 4 or 6 of said computing modules form a first level computing module, i.e. N1 = 4 or 6, said ith link is a second link, said second link is an optical link.

7. The interconnection system of claim 6, wherein, each of said computing modules within the first level computing module is arranged along a first plane, each of said first level computing modules within the second level computing module is arranged along a first plane or along a second plane perpendicular to the first plane.

8. The interconnection system of claim 6, wherein, N1 = 4, i.e. every 4 of said computing modules form a first level computing module, in the second level computing module, N2 of said first level computing modules form a ring topology, N2 ≥ 4.

9. The interconnection system of claim 8, wherein, K = 8, M1 = 6, each of said computing modules within the first level computing module has a direct connection of said first link with another of said computing modules, and said computing modules communicate with each other through two input / output interfaces.

10. The interconnection system of claim 8, wherein, K = 5, M1 = 3, each of said computing modules within the first level computing module has a direct connection of said first link with another of said computing modules, and said computing modules communicate with each other through one input / output interface.

11. The interconnection system of any one of claims 1 to 5, wherein, said plurality of computing modules form 3 layers of computing modules, i.e. j = 3, Each 4 or 6 of the computing modules is constructed into a first-level computing module, i.e. N1 = 4 or 6, The link for communication interconnection inside the second-level computing module is a second link, and the link for communication interconnection inside the third-level computing module is a third link, wherein the second link is an electrical link or an optical link, and the third link is an optical link.

12. The interconnection system of claim 11, wherein, The first-level computing module, the second-level computing module, and the third-level computing module are arranged in the same plane in the respective corresponding level, and are arranged into a three-dimensional structure between different levels.

13. The interconnection system of claim 1, wherein, The computing modules in the first level have direct connection communication channels between each other.

14. The interconnection system of claim 13, wherein, The computing modules in the second level have communication channels with a maximum communication hop count of 2 between each other.

15. The interconnection system of claim 14, wherein, The computing modules in the third level have communication channels with a maximum communication hop count of 3 between each other.

16. The interconnection system of claim 5, wherein, The computing module includes a computing chip and a plurality of optical modules, The computing chip has a plurality of electrical input and output interfaces, and each optical module includes an electrical input and output interface and an optical input and output interface, At least part of the electrical input and output interfaces on the computing chip are directly electrically connected with the electrical input and output interfaces on the optical modules, and the number of the optical modules is less than or equal to K.

17. The interconnection system of claim 16, wherein, The computing module further includes a PCB board, and at least part of the electrical input and output interfaces on the computing chip are electrically connected with the electrical input and output interfaces on the optical modules through wiring on the PCB board.

18. The interconnection system of claim 17, wherein, The electrical input and output interfaces on the computing chip are SerDes interfaces.

19. The interconnection system of claim 17, wherein, The electrical input and output interfaces on the computing chip are PCIe physical interfaces, and the computing chip provides a PCIe electrical signal to the optical modules, and the PCIe electrical signal is used to drive the optical modules.

20. The interconnection system of claim 19, wherein, The PCB board is provided with a card slot, and the optical modules are plug-in arranged in the card slot.

21. The interconnection system of claim 1, wherein, The computing module includes a computing chip and an HBM memory chip.

22. The interconnection system of claim 5, wherein, The optical link is an active optical cable, and both ends of the active optical cable are provided with optical modules, The computing module includes a PCB board and a computing chip arranged on the PCB board, The active optical cable is plug-in connected with the electrical input and output interfaces of the computing chip.

23. A computing system, comprising: The interconnection system of any one of claims 1 to 22.

Images