3D interconnection method based on PCB and cable heterogeneity and high-performance computing system

Abstract

The invention provides a 3D interconnection method based on PCB and cable heterogeneity and a high-performance computing system, and the system comprises a plurality of racks, each rack is internally provided with a plurality of machine frames, and each machine frame is internally provided with a plurality of computing blades; each calculation blade is internally provided with a plurality of calculation nodes and is provided with a front panel and a rear panel, one of the front panel and the rear panel is provided with a cable connector, and the other one is provided with a cable connector and a backboard connector. The first part of the total link number is completed through an in-board interconnection mode based on PCB connection, the second part of the total link number is completed through a backboard connection mode based on PCB connection, and the remaining part of the total link number is completed through a cable connection mode. The advantages of high density and low cost of PCB interconnection and the characteristic of low loss of the cable are combined, and advantage complementation of the PCB interconnection and the cable is achieved. Through the innovative combination, not only is the interconnection density improved and the cost reduced, but also the low loss of signal transmission is ensured, so that the optimal balance between the performance and the cost is achieved.

Description

Technical Field

[0001] The present application relates to the field of high-performance computing technology, and in particular to a 3D interconnection method and a high-performance computing system based on heterogeneous implementation of PCBs and cables. Background Art

[0002] The application of 3D interconnect structures in supercomputing systems is an important development direction in the field of high-performance computing. This structure can significantly improve computing density, reduce data transmission distance, reduce power consumption and improve overall computing efficiency by stacking and interconnecting processors, memory and other electronic components in three-dimensional space. In the field of 3D scientific computing, 3D interconnect structures can better simulate three-dimensional problems in the real world, such as the dynamic simulation of biological molecules. Compared with traditional two-dimensional computing architectures, 3D architectures can reduce the distance and complexity of communication between computing nodes, thereby improving computing efficiency. For example, the Itanium supercomputer (Anton) uses a 3D scientific computing architecture to achieve efficient simulation of the dynamic movement of proteins through dedicated ASIC chips and high-speed three-dimensional ring networks, greatly accelerating the process of biological computing research and drug development.

[0003] In summary, supercomputing systems based on 3D interconnect structures have the following advantages: ① Efficient parallel computing capabilities: 3D interconnect structures can support large-scale parallel computing, which enables supercomputing systems to process large-scale data and complex computing tasks faster. ② High scalability: 3D interconnect structures can be easily expanded to adapt to increasing computing needs. This enables supercomputing systems to be upgraded and expanded over time to meet changing computing needs. ③ Fault tolerance: 3D interconnect structures have good fault tolerance and can tolerate the failure of certain computing nodes. This makes supercomputing systems more reliable and stable, and can ensure the successful completion of critical tasks. ④ Efficient communication performance: 3D interconnect structures can provide high-bandwidth and low-latency communication performance, making data transmission between computing nodes faster, reducing communication delays and data transmission bottlenecks. ⑤ High energy efficiency: 3D interconnect structures have very high energy efficiency. Due to the use of P2P interconnection, huge data exchange equipment is removed. It can effectively use energy and reduce energy consumption and carbon emissions. This makes supercomputing systems more environmentally friendly and sustainable. However, due to the high complexity of the 3D interconnect structure, system design also faces great challenges. ⑥ Interconnection complexity: 3D interconnection is a complex interconnection structure that requires connections between computing nodes in multiple dimensions. This makes system design more complex and requires processing more connection relationships and communication paths. ⑦ Interconnection reliability: The number of connections in 3D interconnection is much higher than that in 2D interconnection structure, and connection reliability will face huge challenges. ⑧ Interconnection cost: The increase in the number of connections in 3D interconnection also pushes up the cost of interconnection. How to optimize the cost of interconnection becomes the key to system design.

[0004] However, when designing a supercomputer system with a 3D interconnect structure with multiple computing nodes, there are also some challenges, namely how to effectively manage and reduce the number of physical cables required while maintaining sufficient I / O performance for each computing node. Take a 3D interconnect structure supercomputer system consisting of 512 computing nodes as an example. In order to ensure that each computing node has sufficient input and output (I / O) bandwidth, each computing node is designed with 96 links. This results in a very large total number of links in the entire system, which is 24,576 (512 computing nodes multiplied by 96 links divided by 2). If the QSFP28 cable commonly used in traditional supercomputer systems is used, a total of 6,144 cables are required (24,576 divided by 4). Even without considering the cost issue, the computer room space required for the cables alone may exceed the available range, making wiring a huge challenge. Summary of the invention

[0005] In view of the shortcomings of the prior art described above, the purpose of the present application is to provide a 3D interconnection method and a high-performance computing system based on PCB and cable heterogeneity, which are used to solve the technical problems that the links of the existing high-performance computing system use cable connectors, resulting in a large number of cables and a large space occupied.

[0006] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a 3D interconnection method based on PCB and cable heterogeneity, which is applied to a high-performance computing system; the high-performance computing system includes a plurality of racks, each of which is provided with a plurality of machine frames; each of which is provided with a plurality of computing blades, each of which is provided with a plurality of computing nodes; the 3D interconnection method includes:

[0007] The first part of the total number of links is completed by an intra-board interconnection method based on PCB connection, which includes: configuring N computing nodes in three dimensions of each computing blade in the high-performance computing system to form a (N×N×N) 3D interconnection structure; the high-performance computing system is configured with S computing blades; each computing node in the computing blade is configured with M connecting lines bidirectionally connected to adjacent computing nodes; the number of links completed in each computing blade is N 3 ×M×L / 2 / 2, the high performance computing system completes a total link number of S×N through the intra-board interconnection method based on PCB connection 3 ×M×L / 2 / 2;

[0008] The second part of the total number of links is completed by a backplane connection method based on PCB connection, which includes: the high-performance computing system is configured with B1 backplanes in total, each backplane is connected with B2 computing blades, and the number of lines connected to the corresponding backplane by each computing blade is configured as P; the total number of links completed by the high-performance computing system through the backplane connection method based on PCB connection is B1×B2×P×L / 2;

[0009] The remaining part of the total number of links is completed by cable connection, which includes: the number of links completed by cable connection is N 3 ×2×L / 2×64.

[0010] In some embodiments of the first aspect of the present application, the high-performance computing system is a high-performance computing system with 512 computing nodes, which includes 2 racks, each rack includes 4 frames, each frame includes 8 computing blades, and each computing blade includes 8 computing nodes.

[0011] In some embodiments of the first aspect of the present application, the 3D interconnect structure includes: configuring 2 computing nodes respectively in three dimensions of the computing blade to form a 2×2×2 3D interconnect structure.

[0012] In some embodiments of the first aspect of the present application, the first part of the total number of links is completed through the intra-board interconnection method based on PCB connection, which includes: let N be 2, M be 6, L be 16, and S be 64; the number of links completed in each computing blade is 192, and the total number of links completed in the high-performance computing system through the intra-board interconnection method based on PCB connection is 12288.

[0013] In some embodiments of the first aspect of the present application, the second part of the total number of links is completed by the backplane connection method based on PCB connection, which includes: setting B1 to 8, B2 to 8, P to 8, and L to 16. The total number of links completed in the high-performance computing system by the backplane connection method based on PCB connection is 4096.

[0014] In some embodiments of the first aspect of the present application, the remaining part of the total number of links is completed by cable connection, including: the number of links completed by cable connection for each computing blade is 128, and the total number of links completed by cable connection in the high-performance computing system is 8192.

[0015] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a high-performance computing system for implementing 3D interconnection based on PCB and cable heterogeneity, comprising: a plurality of racks, each rack having a plurality of machine frames, each machine frame having a plurality of computing blades; each computing blade having a plurality of computing nodes and a front and rear panel, one of the front and rear panels having a cable connector, and the other having a cable connector and a backplane connector; each of the computing nodes is connected to an adjacent computing node, and is connected to the cable connector and the backplane connector;

[0016] The total number of links completed in the high performance computing system includes: the first part of the total number of links completed by the intra-board interconnection method based on PCB connection, which is S×N 3 ×M×L / 2 / 2; the second part of the total number of links is completed by the backplane connection based on PCB connection, which is B1×B2×P×L / 2; the remaining part of the total number of links is completed by cable connection, which is N 3 ×2×L / 2×64; wherein S represents the number of computing blades in the high-performance computing system, N represents the number of computing nodes configured by the computing blades in each dimension, M represents the connection lines bidirectionally connecting each computing node with adjacent computing nodes, and L represents the number of links of each connection line; B1 represents the number of backplanes configured in the high-performance computing system, B2 represents the number of computing blades connected to each backplane, and P represents the number of connection lines connecting each computing blade to the corresponding backplane.

[0017] In some embodiments of the second aspect of the present application, the type of the cable connector includes a high-density high-speed connector.

[0018] In some embodiments of the second aspect of the present application, the high-performance computing system is a high-performance computing system with 512 computing nodes, which includes 2 racks, each rack includes 4 frames, each frame includes 8 computing blades, and each computing blade includes 8 computing nodes.

[0019] In some embodiments of the second aspect of the present application, the 3D interconnect structure includes: configuring 2 computing nodes respectively in three dimensions of the computing blade to form a 2×2×2 3D interconnect structure.

[0020] As described above, the 3D interconnection method and high-performance computing system based on heterogeneous PCB and cable of the present application have the following beneficial effects: the present application combines the high-density and low-cost advantages of PCB interconnection and the low-loss characteristics of cable, and realizes the complementary advantages of the two. Through this innovative combination, not only the density of interconnection is improved and the cost is reduced, but also the low loss of signal transmission is ensured, thereby achieving the best balance between performance and cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Shown is a structural schematic diagram of a high performance computing system in one embodiment of the present application.

[0022] Figure 2 Shown is a flow chart of a 3D interconnection method based on PCB and cable heterogeneity implementation in one embodiment of the present application.

[0023] Figure 3 Shown is a schematic diagram of a 3D interconnection structure in an embodiment of the present application.

[0024] Figure 4A Shown is a schematic diagram of the structure of a computing blade server and a connector in one embodiment of the present application.

[0025] Figure 4B Shown is a partial structural diagram of a high performance computing system in one embodiment of the present application.

[0026] Figure 5 Shown is a schematic diagram of the framework structure of a computing blade server in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0028] Before further describing the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are applicable to the following interpretations:

[0029] <1> HPC (High-Performance Computing): refers to the use of the most advanced computer technology to solve complex computing problems in science, engineering or business through large-scale parallel processing and high-speed network communication. HPC systems are usually composed of a large number of processors, storage devices and high-speed networks, capable of performing large-scale numerical simulations and data analysis.

[0030] <2> 3D interconnect: This is an electronic interconnect technology that allows electronic components to be interconnected in three dimensions, rather than just on the traditional two-dimensional plane. This technology can improve the integration and performance of electronic devices, reduce power consumption and volume, and is commonly used in high-performance computing and integrated circuit design.

[0031] <3> Compute blade: An XPU is a programmable processor that can perform many types of computing tasks, not just traditional CPUs or GPUs. A compute blade may refer to a modular hardware unit that can be plugged into a server or other computing system to provide additional computing power or specific functions.

[0032] <4> Backplane: In electronic devices or computer systems, the backplane usually refers to the back of the motherboard, which carries the connectors and other components on the circuit board. The backplane can provide electrical connections and physical support so that various circuit boards and modules can be fixed and connected with other system components.

[0033] <5> Backplane connector: This is a connector mounted on a backplane that is used to connect circuit boards or other electronic components. Backplane connectors can be various types of interfaces, such as PCIe slots, SATA ports, or other customized interfaces that allow electrical and physical connections between devices.

[0034] <6> Cable Connector: This is an interface device used to connect cables to the device end. Cable connectors can be of various types, such as HDMI, USB, Ethernet RJ45, etc. They allow cables to physically and electrically connect to electronic devices to transfer data, power or other signals.

[0035] This application combines the high density and low cost advantages of PCB interconnection with the low loss characteristics of cables, achieving complementary advantages of the two. Through this innovative combination, not only the density of interconnection is improved and the cost is reduced, but also the low loss of signal transmission is ensured, thus achieving the best balance between performance and cost.

[0036] It should be understood that the characteristics of PCB interconnection include high density, low cost and high loss. High density means that PCB can achieve a large number of interconnections in a small space and can accommodate more circuits and connection points in a limited space. Low cost means that the manufacturing technology of PCB is mature and the cost is low when it is mass-produced. High loss means that when high-frequency signals are transmitted, PCB interconnection may introduce more signal loss, affecting the quality and speed of data transmission. The characteristics of cable interconnection include low density, high cost and low loss. Low density means that compared with PCB interconnection, cable interconnection provides fewer connection points in the same space. High cost means that the production cost of cables is usually higher than that of PCBs, especially when cables of high quality or special materials are required. Low loss means that cables usually provide lower signal transmission loss, which is beneficial for maintaining the integrity and speed of data transmission. Therefore, the purpose of combining these two technologies is to use the high density and low cost advantages of PCB to increase the number of interconnections in the system when designing a supercomputer system, while using the low loss characteristics of cables to ensure the quality and speed of data transmission.

[0037] To facilitate understanding by those skilled in the art, the following takes a high-performance computing system with 512 computing nodes as an example to explain the 3D interconnection method and high-performance computing system based on PCB and cable heterogeneous implementation provided by the present application. The structure of the high-performance computing system with 512 computing nodes is as follows: Figure 1 As shown, it includes 2 racks, each rack includes 4 frames, each frame includes 8 computing blades, and each computing blade includes 8 computing nodes (not shown), so there are 512 computing nodes in total. It should be noted that although the technical solution of the present invention is described by taking 512 computing nodes as an example, its application scope is not limited to this, and can be expanded to a wider range of applicable scenarios, which are not listed here one by one.

[0038] It should be understood that high-performance computing systems (HPC) refer to computer systems that can perform complex computing tasks and can be used to handle large-scale scientific computing, data analysis, simulations, and other tasks that require a lot of computing resources. A rack is a frame structure used to install and organize computer hardware. A chassis is a large computer hardware housing that is used to accommodate multiple computing blades. A computing blade is a modular computing unit designed to be inserted into a chassis. Each blade typically contains one or more processors, memory, storage, and other necessary hardware components, and can run independently or as part of a larger system.

[0039] Figure 2 A flow chart of a 3D interconnection method based on PCB and cable heterogeneity implementation in an embodiment of the present invention is shown, and the method includes the following steps:

[0040] Step S21: completing the first part of the total number of links through an intra-board interconnection method based on PCB connection, which includes: configuring N computing nodes in three dimensions of each computing blade in the high-performance computing system to form a (N×N×N) 3D interconnection structure; the high-performance computing system is configured with S computing blades; each computing node in the computing blade is configured with M connecting lines bidirectionally connected to adjacent computing nodes; the number of links completed in each computing blade is N 3 ×M×L / 2 / 2, the high performance computing system completes a total link number of S×N through the intra-board interconnection method based on PCB connection 3 ×M×L / 2 / 2.

[0041] The "computing node" in the computing blade refers to the XPU module (eXtensible Processing Unit), which represents any kind of scalable processor unit, including but not limited to CPU processors, GPU processors or NPU processors. The focus of the XPU processor is scalability and adaptability, and different types of processors can be selected to perform tasks according to different computing needs. In addition, the XPU processor also includes processors specially optimized for specific computing tasks, such as AI reasoning, graphics processing, video processing, etc. In actual applications, the XPU can be a single chip that integrates the CPU and GPU, or it can be a dedicated chip designed for specific tasks such as AI computing or graphics processing, which is not limited in this embodiment.

[0042] For ease of understanding, combined Figure 3 The 3D interconnection structure diagram shown in the figure is used for explanation. For a 512-node high-performance computing system, N is 2, M is 6, L is 16, and S is 64. Each computing blade is configured with 2 computing nodes (XPU modules) in three dimensions (X, Y, and Z), and there are a total of 8 computing nodes (2×2×2=8). Each XPU module is connected to a cable connector (CABLE CONN) in the X and Y directions, and to a backplane connector (BP CONN) in the Z direction. It should be understood that Figure 3 The diagram shown is a simplified diagram. Each double-arrow line represents 16 links. For a single XPU module, there are 6 double-arrow lines, so the number of links for a single XPU module is 6×16=96. There are 8 XPU modules in each computing blade, half of which are interconnected within the board and the overall number of links is halved after the two computing node links are connected. Therefore, the number of links completed in a single computing blade is 8×6×16 / 2 / 2=192. The " / 2 / 2" here means that each link is calculated once in both directions, and only half are interconnected within the board, so the actual number of links is obtained by dividing by 2 twice. Therefore, for a 512 computing node system, 64 computing blades are required, so the total number of links that can be completed within the board is 64*192=12288.

[0043] Step S22: The second part of the total number of links is completed by a backplane connection method based on PCB connection, which includes: assuming that the number of connection lines connecting each computing blade to the backplane is P, each connection line represents L links, the high-performance computing system is configured with B1 backplanes in total, each backplane is connected to B2 computing blades, and the total number of links completed by the backplane connection is B1×B2×P×L / 2.

[0044] Combination Figure 3The 3D interconnection structure diagram shown is used to illustrate that for a 512-node high-performance computing system, B1 is 8, B2 is 8, P is 8, and L is 16. In a high-performance computing system, the chassis backplane is used to achieve 8×Z-direction interconnection. A single backplane can complete 512 link interconnections. The backplane of a 512-computing node supercomputing system can complete a total of 8×512=4096 link interconnections.

[0045] Specifically, "8×Z-direction interconnection" refers to the multi-layer vertical interconnection technology implemented using the chassis backplane, which means that 8 times the interconnection density can be achieved in the vertical direction (Z axis). Figure 3 It can be seen that there are 8 connection lines connecting each computing blade to the backplane connector (BPCONN), and the corresponding number of links that can be completed is 8×16 / 2=64; the " / 2" here means that each link is calculated once in both directions, so the actual number of links is obtained by dividing by 2. According to the calculation of 8 computing blades connected to one chassis, the number of links that can be completed by each backplane is 8×64=512. For a high-performance computing system with 512 computing nodes, there are 8 backplanes in total, so the total number of links that can be completed by all backplanes is 8×512=4096.

[0046] The backplane referred to here mainly provides electrical connection, signal transmission, power distribution and physical support for various modules within the system. It usually contains multiple connectors that can be inserted into different modules or circuit boards and connected through high-speed signal lines or optical fibers.

[0047] Step S13: completing the remaining part of the total number of links by cable connection, which includes: the number of links completed by cable connection is N 3 ×2×L / 2×64.

[0048] from Figure 4A It can be seen that each computing knife includes a front and a back panel, with the lower part of the picture being the front panel and the upper part of the picture being the back panel. The front panel is provided with a cable connector, and the back panel is provided with a cable connector and a back panel connector. Figure 3 It can be seen that Figure 3 The upper position is used as the front panel, and the upper position is used as the rear panel. Each XPU module in the computing blade is connected to the cable connectors on the front and rear panels, and also connected to the cable connector on the rear panel.

[0049] Still taking the 512-node high-performance computing system as an example, N is 2 and L is 16. The number of links completed by each computing blade through cable connection is 8×2×16 / 2=128. Among them, "8×2" refers to the number of connections between the computing blade and the cable connectors on the front and rear panels, and " / 2" means that each link is calculated once in both directions, so divide by 2 to get the actual number of links. For a high-performance computing system with 512 computing nodes, the total number of links completed through cable connectors is 64×128=8192.

[0050] It should be understood that connectors refer to physical components used to achieve electrical connections, including cable connectors and backplane connectors, such as Figure 4A and 4B As shown. Cable connectors are used to link cables and devices, and backplane connectors are used to connect individual computing nodes to the backplane or motherboard of the system. Cable connectors are connectors used at the ends of cables, and they can be inserted into corresponding sockets on devices or backplanes to achieve electrical connections. In supercomputing systems, if QSFP28 cables are used, there will be QSFP28 cable connectors at both ends of each cable, one end connected to the computing node and the other end connected to a switch, router or other device. The backplane connector is installed on the backplane of the system to receive the insertion of cable connectors, thereby connecting each computing node to the communication network of the entire system.

[0051] The high performance computing system comprises: a computing blade and a backplane; the backplane is provided with a backplane connector and a cable connector. The backplane establishes a communication connection with the computing blade through the backplane connector provided thereon, and establishes a communication connection with an external device through the cable connector provided thereon.

[0052] It should be understood that the computing blade of this embodiment is preferably a computing blade, and the computing blade is usually connected to the backplane through a high-speed connector. This connection method allows data transmission between the blade server and the backplane, and can support hot-swap function, that is, the blade can be replaced or upgraded without restarting the system. The backplane acts as a central hub and manages the connection and data exchange of all blade servers. The backplane is connected to external devices or networks through cable connectors. These cable connectors can be copper cables or optical cables, which provide physical connections from the backplane to external switches, routers or other servers. This connection method allows the server to communicate with a wider network environment. The backplane connector is part of the backplane, and they provide a direct connection point with the computing blade. These connectors support high-speed data transmission and are designed to meet performance requirements such as signal integrity, low insertion loss and low crosstalk. For example, TE Connectivity's STRADA Whisper backplane connector can support data transmission rates up to 112Gbps PAM-4 while maintaining low noise and low insertion loss to ensure signal integrity. The other end of the cable connector is connected to an external device such as a switch, storage device or network interface. These connectors can be standard RJ45 Ethernet ports, fiber optic ports, or other specialized ports, and they ensure the server's connection to the outside world.

[0053] by Figure 5 The framework diagram of the computing blade server is used as an example for explanation. The detailed technical indicators of the computing blade server are as follows:

[0054] (1) Vertically inserted blade, containing 8 XPU submodules: The blade server is designed to be inserted vertically into the rack, which can save space compared to other horizontally inserted blade servers. Each computing blade contains 8 independent XPU processing units, which can be different types of processors, such as CPU, GPU or other special processors, for performing different computing tasks.

[0055] (2) Maximum power consumption ≤ 2000W: The maximum power consumption of the computing blade server when running at full load shall not exceed 2000W. The maximum power consumption is an important indicator to measure the energy efficiency and heat dissipation requirements of the blade.

[0056] (3) MaPU board size ≤ 430mm*400mm: The MaPU (Motherboard Processing Unit) board is the main circuit board on the blade, and its size is limited to 430mm by 400mm, which helps keep the blade compact.

[0057] (4) Input power 48V DC (Power Block): The blade server accepts 48V DC power.

[0058] (5) Support loop heat pipe water cooling: The blade adopts a water cooling system, which transfers heat from the heat source to the coolant through the heat pipe, and then discharges the heat through the water cooling system to keep the server running stably under high load.

[0059] (6) Support IPMC, IPMI intelligent management, support hot plug: IPMC (Intelligent Platform Management Controller) and IPMI (Intelligent Platform Management Interface) are intelligent management interfaces that allow remote monitoring and management of server hardware status, such as temperature, voltage, fan speed, etc.

[0060] (7) Front panel: 1*RJ45 Gigabit Ethernet port, 1*USB2.0 port, 1*clock interface (Clock Block), 4*Superlink (16 Gbps) interfaces: The front panel provides an RJ45 Gigabit Ethernet port, a USB2.0 port, a clock interface, and four Superlink (16 Gbps) interfaces for data transmission and device connection.

[0061] It should be understood that the Superlink (16Gbps) interface usually refers to the 16G FC SFP+ optical module, which is a high-performance optical fiber transmission device, mainly used in high-speed data transmission scenarios such as data centers, storage networks, virtualization and cloud computing. The 16GFC SFP+ optical module supports a data rate of 14.025Gbps and can be used with OM2 and OM3 fiber optic patch cords to provide a stable and reliable solution. This optical module realizes signal transmission based on multi-mode optical fiber and uses multiplexing technology to merge and transmit data from multiple channels to achieve a higher transmission rate.

[0062] (8) 4*Superlink (16Gbps) interfaces on the rear panel: The rear panel also provides four Superlink interfaces, which are high-speed interfaces used to connect to other devices or networks.

[0063] (9) Internal Superlink interconnection forms a 2x2x2 cube, and its Z-direction Superlink is connected to the backplane: The Superlink interconnection inside the blade forms a cube structure, which may help optimize the internal data transmission path and improve communication efficiency. In the cube structure, the Z-direction Superlink is connected to the backplane, which is the central board that connects all blades and provides power, data, and cooling interfaces.

[0064] (10) 8*XPU chips: Each blade contains 8 XPU chips, which are the core of the blade's computing power and can be processors specifically optimized for specific tasks.

[0065] In the embodiment of the present application, each computing blade is respectively configured with N computing nodes in three dimensions to form a (N×N×N) 3D interconnection structure; the high-performance computing system is configured with S computing blades; each computing node in the computing blade is configured with M connecting lines bidirectionally connected to adjacent computing nodes; the number of links completed in each computing blade is N 3 ×M×L / 2 / 2, the high performance computing system completes a total link number of S×N through the intra-board interconnection method based on PCB connection 3 ×M×L / 2 / 2; the second part of the total number of links is completed by a backplane connection method based on PCB connection, which includes: the high-performance computing system is configured with B1 backplanes, each backplane is connected to B2 computing blades, and the number of connections between each computing blade and the corresponding backplane is configured as P; the high-performance computing system completes a total number of links of B1×B2×P×L / 2 by a backplane connection method based on PCB connection; the remaining part of the total number of links is completed by cable connection, which includes: the number of links completed by cable connection is N 3 ×2×L / 2×64.

[0066] Take a high-performance supercomputer system with 512 computing nodes as an example: it includes 2 racks, each rack includes 4 frames, each frame includes 8 computing blades, and each computing blade includes 8 computing nodes.

[0067] The 3D interconnection structure includes: configuring 2 computing nodes respectively on three dimensions of the computing blade to form a 2×2×2 3D interconnection structure.

[0068] The first part of the total number of links is completed through the intra-board interconnection method based on PCB connection, which includes: let N be 2, M be 6, L be 16, and S be 64; the number of links completed in each computing blade is 192, and the total number of links completed in the high-performance computing system through the intra-board interconnection method based on PCB connection is 12288.

[0069] The second part of the total number of links is completed through a backplane connection method based on PCB connection, which includes: let B1 be 8, B2 be 8, P be 8, and L be 16. The total number of links completed in the high-performance computing system through a backplane connection method based on PCB connection is 4096.

[0070] The remaining part of the total number of links is completed by cable connection, including: the number of links completed by each computing blade by cable connection is 128, and the total number of links completed by cable connection in the high-performance computing system is 8192.

[0071] Exemplarily, each computing blade is provided with several computing nodes and front and rear panels, one of which is provided with a cable connector, and the other is provided with a cable connector and a backplane connector; each computing node is connected to an adjacent computing node and is connected to the cable connector and the backplane connector.

[0072] Exemplarily, the type of the cable connector includes a high-density high-speed connector.

[0073] Exemplarily, the computing blade includes a computing blade, on which 8 XPU modules are configured as 8 computing nodes; the XPU module is an expandable processor unit, and its type includes a CPU module, a GPU module or an NPU module.

[0074] Exemplarily, the Superlinks inside the computing blades are interconnected to form a cube structure; in the cube structure, the vertical Superlinks are connected to the backplane.

[0075] It should be noted that the high-performance computing system for implementing 3D interconnection based on PCB and cable heterogeneity provided in the embodiment of the present invention has similar implementation methods, implementation methods and principles as the above-mentioned 3D interconnection method based on PCB and cable heterogeneity, and will not be repeated here.

[0076] In addition, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same items or similar items with substantially the same functions and effects. For example, the first XX and the second XX are only used to distinguish different XXs, and do not limit their order. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0077] In the embodiments of the present application, the words "exemplary" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0078] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0079] In the embodiment of the present application, the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here. It should also be understood that the division of modules in the embodiment of the present application is schematic, which is only a logical function division, and there may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application can be integrated in a processor, or it can exist physically separately, or two or more modules can be integrated in one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0080] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0081] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0082] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0083] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0084] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0085] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0086] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0087] In summary, the present application provides a 3D interconnection method and a high-performance computing system based on heterogeneous implementation of PCB and cables. The present application combines the high-density, low-cost advantages of PCB interconnection and the low-loss characteristics of cables to achieve complementary advantages of the two. Through this innovative combination, not only the density of interconnection is improved and the cost is reduced, but also the low loss of signal transmission is ensured, thereby achieving the best balance between performance and cost. Therefore, the present application effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.

[0088] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A 3D interconnection method based on PCB and cable heterogeneity, characterized in that: Applicable to a high-performance computing system; the high-performance computing system comprises a plurality of racks, each rack having a plurality of machine frames; each machine frame having a plurality of computing blades, each computing blade having a plurality of computing nodes; The 3D interconnection method comprises: The first part of the total number of links is completed by an intra-board interconnection method based on PCB connection, which includes: configuring N computing nodes in three dimensions of each computing blade in the high-performance computing system to form a (N×N×N) 3D interconnection structure; the high-performance computing system is configured with S computing blades; each computing node in the computing blade is configured with M connecting lines bidirectionally connected to adjacent computing nodes; the number of links completed in each computing blade is N 3 ×M×L / 2 / 2, the high performance computing system completes a total link number of S×N through the intra-board interconnection method based on PCB connection 3 ×M×L / 2 / 2; The second part of the total number of links is completed by a backplane connection method based on PCB connection, which includes: the high-performance computing system is configured with B1 backplanes in total, each backplane is connected with B2 computing blades, and the number of lines connected to the corresponding backplane by each computing blade is configured as P; the total number of links completed by the high-performance computing system through the backplane connection method based on PCB connection is B1×B2×P×L / 2; The remaining part of the total number of links is completed by cable connection, which includes: the number of links completed by cable connection is N 3 ×2×L / 2×64.

2. The 3D interconnection method based on PCB and cable heterogeneity according to claim 1, characterized in that: The high-performance computing system is a high-performance computing system with 512 computing nodes, which includes 2 racks, each rack includes 4 frames, each frame includes 8 computing blades, and each computing blade includes 8 computing nodes.

3. The 3D interconnection method based on PCB and cable heterogeneity according to claim 2, characterized in that: The 3D interconnection structure includes: configuring 2 computing nodes respectively on three dimensions of the computing blade to form a 2×2×2 3D interconnection structure.

4. The 3D interconnection method based on PCB and cable heterogeneity according to claim 2, characterized in that: The first part of the total number of links is completed by the intra-board interconnection method based on PCB connection, which includes: Let N be 2, M be 6, L be 16, and S be 64; the number of links completed in each computing blade is 192, and the total number of links completed in the high-performance computing system through the intra-board interconnection method based on PCB connection is 12288.

5. The 3D interconnection method based on PCB and cable heterogeneity according to claim 2, characterized in that: The second part of the total number of links is completed by a backplane connection method based on PCB connection, which includes: Let B1 be 8, B2 be 8, P be 8, and L be 16. The total number of links completed in the high-performance computing system through the backplane connection based on PCB connection is 4096.

6. The 3D interconnection method based on PCB and cable heterogeneity according to claim 2, characterized in that: The remaining part of the total number of links is completed by cable connection, which includes: The number of links completed by each computing blade through cable connection is 128, and the total number of links completed through cable connection in the high-performance computing system is 8192.

7. A high-performance computing system based on PCB and cable heterogeneity to achieve 3D interconnection, characterized in that: include: A plurality of racks, each rack having a plurality of machine frames, each machine frame having a plurality of computing blades; each computing blade having a plurality of computing nodes and a front and rear panel, one of the front and rear panels having a cable connector, and the other having a cable connector and a backplane connector; each computing node is connected to an adjacent computing node and is connected to the cable connector and the backplane connector; The total number of links completed in the high performance computing system includes: The first part of the total number of links is completed by the intra-board interconnection based on PCB connection, which is S×N 3 ×M×L / 2 / 2; The second part of the total number of links is completed by a backplane connection based on a PCB connection, which is B1×B2×P×L / 2; The remaining part of the total number of links is completed by cable connection, which is N 3 ×2×L / 2×64; Among them, S represents the number of computing blades in the high-performance computing system, N represents the number of computing nodes configured by the computing blades in each dimension, M represents the connection lines that bidirectionally connect each computing node to the adjacent computing nodes, and L represents the number of links of each connection line; B1 represents the number of backplanes configured in the high-performance computing system, B2 represents the number of computing blades connected to each backplane, and P represents the number of connection lines connecting each computing blade to the corresponding backplane.

8. The high performance computing system for implementing 3D interconnection based on PCB and cable heterogeneity according to claim 7, characterized in that: The types of cable connectors include high-density and high-speed connectors.

9. The high performance computing system for implementing 3D interconnection based on PCB and cable heterogeneity according to claim 7, characterized in that: The high-performance computing system is a high-performance computing system with 512 computing nodes, which includes 2 racks, each rack includes 4 frames, each frame includes 8 computing blades, and each computing blade includes 8 computing nodes.

10. The high performance computing system for implementing 3D interconnection based on PCB and cable heterogeneity according to claim 9, characterized in that: The 3D interconnection structure includes: configuring 2 computing nodes respectively on three dimensions of the computing blade to form a 2×2×2 3D interconnection structure.

Images