Interconnection apparatus, high-performance switching apparatus, and large-model all-in-one machine

Abstract

The present application relates to the field of communications. Disclosed are an interconnection apparatus, a high-performance switching apparatus, and a large-model all-in-one machine, which aim to solve the problem of both the interconnection rate and the communication bandwidth between multiple cards in a large-model all-in-one machine being relatively low. The interconnection apparatus comprises a plurality of switching devices, wherein each switching device comprises a plurality of external interfaces and a plurality of interconnection interfaces. An interconnection interface of a first switching device can be selectively connected to an interconnection interface of a second switching device, and the first switching device further comprises an embedded controller, wherein the embedded controller is configured to receive topology configuration information, and, when it is determined on the basis of the topology configuration information that there are target interfaces among the plurality of interconnection interfaces of the first switching device where the embedded controller is located, control the target interfaces to establish connections with the interconnection interfaces of the second switching device in a manner of corresponding thereto on a one-to-one basis, such that all external devices connected to the switching devices are interconnected. The present application can flexibly adjust an interconnection topology architecture, such that all devices on the interconnection topology architecture achieve full interconnection, thereby improving the interconnection rate and the communication bandwidth.

Description

An interconnection device, a high-performance switching device, and a large-model all-in-one machine

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on March 29, 2024, with application number 202410382097.0, and entitled “An interconnection device, a high-performance switching device and a large-model all-in-one machine”, all contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of communications, and in particular to an interconnection device, a high-performance switching device, and a large-scale all-in-one machine. Background Art

[0004] With the rapid development of large AI (Artificial Intelligence) models, the customer concentration for basic large model pre-training will increase, and the vast majority of customer business scenarios will be based on fine-tuning and inference applications based on these large models. To address the computing power requirements of large model pre-training and fine-tuning applications, when designing all-in-one AI large model machines, a large-scale multi-card model model was adopted to improve computing power. This model includes multiple GPU (Graphics Processing Unit) cards, which are divided into multiple groups. Any two GPUs within a group can be interconnected, but any two GPUs between groups cannot. This results in low interconnection rates and communication bandwidth, resulting in poor communication performance.

[0005] Therefore, how to provide a solution to the above technical problems is a problem that those skilled in the art need to solve at present. Summary of the Invention

[0006] The purpose of this application is to provide an interconnection device, a high-performance switching device and a large-model all-in-one machine, which can flexibly adjust the interconnection topology architecture so that all devices on the interconnection topology architecture can be fully interconnected, thereby improving the interconnection rate and communication bandwidth.

[0007] To solve the above technical problems, a first aspect of an embodiment of the present application provides an interconnection device, comprising a plurality of switching devices, the switching devices comprising a plurality of external interfaces and a plurality of interconnection interfaces, the external interfaces of the switching devices being configured to connect to external devices, the interconnection interface of a first switching device being selectively connected to the interconnection interface of a second switching device; the first switching device being any one of the plurality of switching devices, and the second switching device being any one of the plurality of switching devices except the first switching device;

[0008] The first switching device also includes an embedded controller, which is configured to receive topology configuration information. When it is confirmed based on the topology configuration information that a target interface exists among the multiple interconnection interfaces of the first switching device where it is located, the embedded controller controls the target interface to establish a one-to-one connection with the interconnection interface of the second switching device, so that any two external devices among all external devices connected to each switching device are interconnected.

[0009] Among them, the embedded controller is also configured to enumerate external devices connected to the first switching device where it is located and external devices connected to the second switching device to which the target interface is connected, determine the global number of each external device, and establish an internal routing table based on the global number so as to use the internal routing table to forward the received data; the global number of the external device is composed of the device number of the external device and the device number of the switching device to which the external device is connected.

[0010] The embedded controller is further configured to receive interface configuration information and adjust the interface parameters of the external interface of the first switching device based on the interface configuration information so that the interface parameters of the external interface match the communication parameters of the external device connected to the external interface.

[0011] Wherein, when it is confirmed based on the topology configuration information that a target interface exists among the multiple interconnected interfaces of the first switching device, a process of controlling the target interface to establish a connection with the interconnected interface of the second switching device in a one-to-one correspondence includes:

[0012] When it is confirmed based on the topology configuration information that a target interface exists among the multiple interconnected interfaces of the first switching device where the target interface is located, controlling the target interface to be enabled so that the target interface is connected to the enabled target interface in the second switching device in a one-to-one correspondence;

[0013] Control the interconnection interfaces other than the target interface among the multiple interconnection interfaces of the first switching device where the first switching device is located to be disabled.

[0014] The number of switching devices is n, the number of interconnection interfaces of each switching device is n-1, n is an integer greater than 1, the j-th interconnection interface of the i-th switching device is selectively connected to the i-th interconnection interface of the j+1-th switching device, i=1, 2, ..., n-1, j=i, ..., n-1.

[0015] To solve the above technical problems, a second aspect of the embodiments of the present application further provides a high-performance switching device, including:

[0016] An interconnection device as described in any one of the above, wherein the interconnection device comprises a plurality of switching devices, each switching device comprises an embedded controller, and the embedded controller comprises a plurality of configuration registers;

[0017] a management device configured to receive a user configuration instruction, determine topology configuration information and / or interface configuration information based on the user configuration instruction, determine a configuration parameter group for each switching device based on the topology configuration information and / or the interface configuration information, and write each configuration parameter value in each configuration parameter group into each configuration register in the embedded controller of the switching device;

[0018] The power supply device is configured to supply power to the interconnection device and the management device.

[0019] The management device includes a plurality of management controllers, and the plurality of management controllers are connected to the interconnection device;

[0020] The current interaction controller is configured to perform data interaction with the interconnected device; the current interaction controller is any one of the multiple management controllers that is in a normal working state.

[0021] Among them, high-performance switching devices also include complex programmable logic devices;

[0022] Each management controller is further configured to send a heartbeat signal to the complex programmable logic device at a preset period;

[0023] A complex programmable logic device is configured to determine whether there is a faulty controller in an abnormal working state among multiple management controllers. If it is determined that there is a faulty controller in an abnormal working state among the multiple management controllers, it is determined whether the faulty controller is the current interactive controller. If it is determined that the faulty controller is the current interactive controller, the faulty controller is controlled to stop data interaction with the interconnection device, and any one of the multiple management controllers in a normal working state is selected as the new current interactive controller; the abnormal working state is a working state in which a heartbeat signal is not sent according to a preset period.

[0024] The multiple management controllers include a master management controller and a slave management controller, and the slave management controller is a management controller other than the master management controller among the multiple management controllers;

[0025] The complex programmable logic device is also configured to control the current interactive controller to stop data interaction with the interconnection device when it is determined that the current interactive controller is a slave management controller and it is determined that there is a master management controller in normal working condition, and select a master management controller in normal working condition as the new current interactive controller.

[0026] The high-performance switching device further comprises a first switching device, wherein the plurality of first ends of the first switching device are connected to the plurality of management controllers in a one-to-one correspondence, the plurality of second ends of the first switching device are connected to the plurality of switching devices in a one-to-one correspondence, and the control end of the first switching device is connected to the complex programmable logic device;

[0027] a complex programmable logic device configured to determine whether there is a faulty controller in an abnormal working state among the multiple management controllers; if it is determined that there is a faulty controller in an abnormal working state among the multiple management controllers, determine whether the faulty controller is a current interactive controller; if it is determined that the faulty controller is the current interactive controller, generate a first control instruction corresponding to the faulty controller; select any one of the multiple management controllers in a normal working state as a new current interactive controller; and generate a second control instruction corresponding to the new current interactive controller;

[0028] The first switching device is configured to, upon receiving a first control instruction, determine a first target end based on the first control instruction, and control the communication link between the management controller connected to the first target end and each switching device to be disconnected; and upon receiving a second control instruction, determine a second target end based on the second control instruction, and control the communication link between the management controller connected to the second target end and each switching device to be connected.

[0029] The high-performance switching device also includes:

[0030] The first port expansion device has a first end connected to the second end of the first switching device, and multiple second ends of the first port expansion device are connected to multiple switching devices in a one-to-one correspondence.

[0031] The process of data interaction with interconnected devices includes:

[0032] Writing each configuration parameter in each configuration parameter group into each configuration register in the embedded controller of the switching device;

[0033] Obtain the status parameters of each switching device and the device parameters of the external devices connected to each switching device;

[0034] The high-performance switching device also includes:

[0035] prompting device;

[0036] A first baseboard controller connected to each management controller is configured to determine a current topology state of the interconnected device through state parameters of each switching device and generate a current state prompt instruction based on the current topology state;

[0037] The complex programmable logic device is also configured to respond to a current state prompt instruction and control the prompt device to prompt current topology state information.

[0038] Wherein, the high-performance switching device further comprises a heat dissipation device, and the heat dissipation device comprises at least one fan;

[0039] The first baseboard controller is further configured to generate a current heat dissipation control instruction based on a current topology state and device parameters;

[0040] The complex programmable logic device is further configured to adjust the speed of the fan in response to the current heat dissipation control instruction.

[0041] Wherein, the first baseboard controller, the management controller, the complex programmable logic device and the interconnection device are connected via Ethernet;

[0042] The first baseboard controller, the management controller, the complex programmable logic device and the interconnection device are connected via a universal serial bus or an integrated circuit bus.

[0043] To solve the above technical problems, the third aspect of the embodiments of the present application further provides a large model all-in-one machine, comprising:

[0044] A high performance switching device as described in any one of the above;

[0045] A plurality of general-purpose computing devices, wherein the plurality of general-purpose computing devices are connected to an interconnection device of a high-performance switching device;

[0046] A plurality of heterogeneous computing devices, wherein the plurality of heterogeneous computing devices are connected to an interconnection device;

[0047] Any two computing devices connected to the interconnection device are interconnected, and the computing devices are general computing devices or heterogeneous computing devices.

[0048] Wherein, the large model all-in-one machine further includes a second substrate controller and a plurality of second switching devices;

[0049] The first ends of the plurality of second switching devices are connected to the plurality of heterogeneous computing devices in a one-to-one correspondence, the second end of each second switching device is connected to the second baseboard controller, and the third end of each second switching device is connected to the high-performance switching device.

[0050] Among them, the large model all-in-one machine also includes multiple network cards, the interconnection device includes multiple switching devices, and each switching device is connected to at least one network card, so that the heterogeneous computing devices connected to the interconnection device can be networked with other large model all-in-one machines through the network cards.

[0051] Among them, the interconnection device includes multiple switching devices, and the large model all-in-one machine also includes multiple network cards. Heterogeneous computing devices are connected to the switching devices through network cards, and heterogeneous computing devices are networked with other large model all-in-one machines through network cards.

[0052] Among them, the large model all-in-one machine also includes:

[0053] at least one memory expansion card, configured to store data to be processed;

[0054] A general computing device is configured to obtain local memory and write data to be processed into a memory expansion card when the local memory is less than a preset value.

[0055] The memory expansion card includes a third baseboard controller and a plurality of memory controllers, and each memory controller is mounted with at least one memory stick;

[0056] a third backplane controller configured to receive and forward memory access instructions sent by the general-purpose computing device, obtain state parameters of each memory controller, obtain target data converted into serial data, and send the data to the general-purpose computing device;

[0057] The memory controller is configured to parse the memory access data in the memory access instruction. When the memory access data is data to be written, the data to be written is converted from serial data to parallel data and then written to the corresponding memory stick mounted on itself. When the memory access data is data to be read, the target data is read from the memory stick mounted on itself according to the data to be read, the target data is converted from parallel data to serial data and then forwarded to the third baseboard controller.

[0058] Among them, the large model all-in-one machine also includes a pooling management engine;

[0059] The general computing device is further configured to initiate a request for dynamic adjustment of heterogeneous resources;

[0060] The pooling management engine is configured to, upon receiving a request for dynamic adjustment of heterogeneous resources, determine a heterogeneous computing device to be adjusted based on the request, perform a hot removal operation on the heterogeneous computing device to be adjusted, and send a location acquisition request corresponding to the heterogeneous computing device to be adjusted to a high-performance switching device, send a reset execution instruction to the heterogeneous computing device to be adjusted based on the received physical location information, determine the heterogeneous computing device to be adjusted as the heterogeneous computing device to be allocated upon receiving the reset completion instruction, and allocate the heterogeneous computing device to be allocated to the general-purpose computing device corresponding to the reallocation instruction upon receiving a reallocation instruction;

[0061] The heterogeneous computing device is configured to perform a reset operation upon receiving a reset execution instruction and generate a reset completion instruction after the reset operation is successfully completed;

[0062] The high-performance switching device is configured to obtain physical location information of the heterogeneous computing device to be adjusted and return the information to the pooling management engine upon receiving a location acquisition request.

[0063] The present application provides an interconnection device, including multiple switching devices, each switching device including multiple external interfaces configured to connect to external devices, and multiple interconnection interfaces configured to interconnect between switching devices. The interconnection interfaces between any two switching devices can be selectively connected, and each switching device also includes an embedded controller. The embedded controller controls the target interface on its own switching device to establish a communication connection with the interconnection interface on the corresponding switching device based on the received topology configuration information. On the one hand, the topology structure of the interconnection device can be flexibly adjusted to suit different working scenarios. On the other hand, the interconnection device can achieve full interconnection of all external devices connected to it under the current topology structure, thereby improving the interconnection rate and communication bandwidth between each external device. When the interconnection device is applied to an AI large model all-in-one machine, it can improve the communication performance of the AI ​​large model all-in-one machine. The present application also provides a high-performance switching device and a large model all-in-one machine, which have the same beneficial effects as the above-mentioned interconnection device. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0065] FIG1 is a schematic structural diagram of a first interconnection device provided in this application;

[0066] FIG2 is a schematic structural diagram of a second interconnection device provided in this application;

[0067] FIG3 is a schematic diagram of an interconnection device of a first 2×2 fully interconnected topology architecture provided by the present application;

[0068] FIG4 is a schematic diagram of an interconnection device of a first 2×1 fully interconnected topology architecture provided by the present application;

[0069] FIG5 is a schematic structural diagram of a third interconnection device provided in this application;

[0070] FIG6 is a schematic structural diagram of a fourth interconnection device provided in this application;

[0071] FIG7 is a schematic diagram of a global number allocation provided by the present application;

[0072] FIG8 is a schematic diagram of an embedded firmware provided by this application;

[0073] FIG9 is a schematic diagram of an interconnection device of a fifth fully interconnected topology architecture provided by the present application;

[0074] FIG10 is a schematic diagram of an interconnection device of a sixth fully interconnected topology architecture provided by the present application;

[0075] FIG11 is a schematic diagram of an interconnection device of a second 2×1 fully interconnected topology architecture provided in the present application;

[0076] FIG12 is a schematic diagram of an interconnection device of a second 2×2 fully interconnected topology architecture provided by the present application;

[0077] FIG13 is a schematic diagram of an interconnection device of a 2×3 fully interconnected topology architecture provided by the present application;

[0078] FIG14 is a schematic diagram of an interconnection device of a 2×4 fully interconnected topology architecture provided by the present application;

[0079] FIG15 is a schematic structural diagram of a first high-performance switching device provided by the present application;

[0080] FIG16 is a schematic structural diagram of a second high-performance switching device provided by this application;

[0081] FIG17 is a schematic structural diagram of a third high-performance switching device provided by this application;

[0082] FIG18 is a schematic structural diagram of a fourth high-performance switching device provided by the present application;

[0083] FIG19 is a schematic structural diagram of a fifth high-performance switching device provided in this application;

[0084] FIG20 is a schematic structural diagram of a sixth high-performance switching device provided by this application;

[0085] FIG21 is a schematic structural diagram of the first large-scale all-in-one machine provided in this application;

[0086] FIG22 is a schematic structural diagram of a second large-scale all-in-one machine provided by this application;

[0087] FIG23 is a schematic structural diagram of the third large-scale all-in-one machine provided in this application;

[0088] FIG24 is a schematic structural diagram of the fourth large-scale all-in-one machine provided by this application;

[0089] FIG25 is a schematic structural diagram of the fifth large-scale all-in-one machine provided in this application;

[0090] FIG26 is a schematic structural diagram of the sixth large-scale all-in-one machine provided in this application;

[0091] FIG27 is a schematic diagram of the first large-scale all-in-one machine network provided in this application;

[0092] FIG28 is a schematic diagram of a second large-scale all-in-one machine network provided in this application;

[0093] FIG29 is a schematic diagram of a third large-scale all-in-one network provided in this application;

[0094] FIG30 is a schematic structural diagram of the seventh large-scale all-in-one machine provided in this application;

[0095] FIG31 is a schematic structural diagram of a memory expansion board provided by the present application;

[0096] FIG32 is a resource category and topology display provided by this application;

[0097] FIG33 is a schematic diagram of dynamic resource allocation provided by this application;

[0098] FIG34 is a schematic structural diagram of the eighth large-scale all-in-one machine provided by the present application;

[0099] FIG35 is a schematic structural diagram of a multi-core system provided in this application. DETAILED DESCRIPTION

[0100] The core of this application is to provide an interconnection device, a high-performance switching device and a large-model all-in-one machine, which can flexibly adjust the interconnection topology architecture so that all devices on the interconnection topology architecture can be fully interconnected, thereby improving the interconnection rate and communication bandwidth.

[0101] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0102] In a first aspect, referring to FIG1 , the present application provides an interconnection device, comprising a plurality of switching devices 1, the switching devices 1 comprising a plurality of external interfaces 11 and a plurality of interconnection interfaces 12, the external interfaces 11 of the switching devices 1 being configured to connect to an external device, the interconnection interface 12 of the first switching device being selectively connected to the interconnection interface 12 of the second switching device; the first switching device being any one of the plurality of switching devices 1, and the second switching device being any one of the plurality of switching devices 1 except the first switching device;

[0103] The first switching device also includes an embedded controller 13, which is configured to receive topology configuration information. When it is confirmed based on the topology configuration information that there is a target interface among the multiple interconnection interfaces 12 of the first switching device where it is located, the embedded controller 13 controls the target interface to establish a one-to-one connection with the interconnection interface 12 of the second switching device, so that any two external devices among all external devices connected to each switching device 1 are interconnected.

[0104] In this embodiment, the interconnection device includes multiple switching devices 1, the number of which is determined according to the actual interconnection topology requirements. Each switching device 1 includes multiple external interfaces 11, which are configured to connect to external devices such as heterogeneous computing devices, general-purpose computing devices, network card devices, storage devices, etc., wherein heterogeneous computing devices include but are not limited to FPGAs (Field Programmable Gate Arrays), GPUs, etc., general-purpose computing devices include but are not limited to CPUs (Central Processing Units), etc., network card devices include but are not limited to NICs (Network Interface Controllers), Smart NICs CX7, etc., and storage devices include but are not limited to DDR (Double Data Rate) memories, etc. Among them, the external interface configured to connect to the general-purpose computing device can be set to an uplink interface, and the external interface configured to connect to the heterogeneous computing device, network card device, and storage device can be set to a downlink interface.

[0105] Each switching device 1 includes multiple interconnection interfaces 12, which are configured to achieve connection between any two switching devices 1. Taking one switching device 1 as an example, the number of interconnection interfaces 12 of other switching devices 1 connected to any interconnection interface 12 on the switching device 1 is less than or equal to 1. In order to ensure that any two switching devices 1 can be connected, the number of interconnection interfaces 12 on each device should be greater than or equal to n-1, where n is the number of switching devices 1. In this embodiment, each switching device 1 can be a first switching device. When different switching devices 1 are first switching devices, their corresponding second switching devices are different. For example, referring to Figure 2, Figure 2 shows that the interconnection device includes four switching devices 1, namely the first device, the second device, the third device and the fourth device. When the first device is the first switching device, the second device, the third device and the fourth device are all the second switching devices corresponding to the first device. When the second device is the first switching device, the first device, the third device and the fourth device are all the second switching devices corresponding to the second device. When the third device is the first switching device, the first device, the second device and the fourth device are all the second switching devices corresponding to the third device. When the fourth device is the first switching device, the first device, the second device and the third device are all the second switching devices corresponding to the fourth device.

[0106] Taking the first device as the first switching device as an example, the first device includes multiple interconnection interfaces 12. In Figure 2, in order to realize the fully interconnected architecture of four switching devices 1, three interconnection interfaces 12 are set on the i-th device, namely the first interconnection interface Pi1, the second interconnection interface Pi2 and the third interconnection interface Pi3, i = 1, 2, 3, 4, and the dotted lines in Figure 2 indicate optional connection. The first interconnection interface P11 on the first device is optionally connected to the first interconnection interface P21 on the second device, and the second interconnection interface P12 on the first device is optionally connected to the first interconnection interface P31 on the third device. The third interconnection interface P13 on the device is selectively connected to the first interconnection interface P41 on the fourth device, the second interconnection interface P22 on the second device is selectively connected to the second interconnection interface P32 on the third device, the third interconnection interface P23 on the second device is selectively connected to the second interconnection interface P42 on the fourth device, and the third interconnection interface P33 on the third device is selectively connected to the third interconnection interface P43 on the fourth device. The selective connection in this embodiment means that the interconnection interfaces 12 on the two switching devices 1 can be connected or not, thereby realizing different interconnection topology architectures. For example, referring to FIG3 , a solid line is used to indicate establishment of a connection. When all the interconnection interfaces 12 mentioned above are connected to each other, the interconnection topology is a 2×2 topology. Referring to FIG4 , a solid line is used to indicate establishment of a connection, and a dotted line indicates that no connection is established. The interconnection topology is a 2×1 topology. Of course, when selecting the interconnection interfaces 12 to be set to cooperate with each other on any two switching devices 1, their positions, layouts, wiring, etc. can be comprehensively considered. The cooperation methods of the interconnection interfaces 12 in FIG2 to FIG4 are only for illustration.

[0107] Since each switching device 1 in this embodiment is a first switching device, each switching device 1 includes an embedded controller 13. The embedded controller 13 in one switching device 1 is used as an example for description. The embedded controllers 13 in other switching devices 1 are similarly configured to receive topology configuration information. This topology configuration information is written to the embedded controller 13 by an external control device via a register write, and the embedded controller 13 responds by reading the register. The topology configuration information can be sent by a user or automatically switched based on the current application scenario. This topology configuration information represents the currently required interconnection topology. Interconnection topologies include, but are not limited to, a 2×1 interconnection topology, a 2×2 interconnection topology, a 2×3 interconnection topology, a 2×4 interconnection topology, a p×q interconnection topology, and so on, where p and q are both positive integers. Under different interconnection topologies, the communication bandwidth between external devices connected to each switching device 1 also varies.

[0108] Assume that the AI ​​large model all-in-one machine needs to be configured with 16 GPUs, namely the first GPU G0, the second GPU G1, ..., the sixteenth GPU G15. The 2×2 interconnection topology architecture is used as an example for illustration. Referring to Figure 5, Figure 5 includes four switching devices 1, namely the first device, the second device, the third device and the fourth device. Each switching device 1 is connected to four GPUs, thereby realizing full interconnection of 16 GPUs. Under this interconnection topology architecture, each switching device 1 can realize three interconnection links for simultaneous P2P (Peer-to-Peer) communication. It can be understood that the 2×2 fully interconnected topology architecture can support 16 PCIE (Peripheral Component Interconnect Express, high-speed serial computer extension bus standard) 5.0 GPUs, and each 4 GPUs can be grouped into a group, for a total of 4 groups, which are interconnected through an internal Fabric (full interconnection line) bus. The bidirectional bandwidth of the interconnection bus is 192GB / s (PCIE GEN4) and 384GB / s (PCIE GEN5). Taking a 2×3 interconnect topology as an example, as shown in FIG6 , FIG6 includes six switching devices 1, namely, a first device, a second device, a third device, a fourth device, a fifth device, and a sixth device. The first, second, third, and fourth devices are each connected to two GPUs, and the fifth and sixth devices are each connected to four GPUs, thereby achieving full interconnection of 16 GPUs. In this interconnect topology, each switching device 1 can implement five interconnect links for simultaneous P2P communication. It can be understood that a 2×3 fully interconnected topology can support 16 PCIE 5.0 GPUs, with each GPU providing five channels to connect to other GPUs. These are interconnected via an internal fabric bus with a bidirectional bandwidth of 320 GB / s (PCIE GEN4 (PCIE Generation 4, fourth generation PCIE)) and 640 GB / s (PCIE GEN5). Different interconnect topologies have different communication bandwidths, and the choice can be made based on actual project needs.

[0109] It can be understood that under different interconnection topology architectures, the number of interconnection interfaces 12 on each first switching device that needs to establish a connection with its corresponding second switching device is also different, and can be 0, 1, or more. The embedded controller 13 can determine whether there is a target interface among the multiple interconnection interfaces 12 of the first switching device where it is located that needs to establish a connection with the interconnection interface 12 of the second switching device based on the received topology configuration information. The number of target interfaces is 0, 1, or more. When the number of target interfaces is 0, that is, the current interconnection topology architecture does not include the first switching device where the embedded controller 13 is located. When the number of target interfaces is 1 or more, that is, the current interconnection topology architecture includes the first switching device where the embedded controller 13 is located, for each target interface on the first switching device where it is located, the target interface is controlled to establish a connection with the interconnection interface 12 on the second switching device where it is located. In this embodiment, for each interconnection interface 12 on the first switching device, a one-to-one correspondence relationship between each interconnection interface 12 and each interconnection interface 12 on the second switching device can be predetermined. As shown in FIG2 , the first interconnection interface P11 of the first device can be pre-set to cooperate with the first interconnection interface P21 of the second device, and the two can be selectively connected. The second interconnection interface P12 of the first device can be pre-set to cooperate with the first interconnection interface P31 of the third device, and the two can be selectively connected. The third interconnection interface P13 of the first device can be pre-set to cooperate with the first interconnection interface P41 of the fourth device, and the two can be selectively connected. Of course, the first interconnection interface P11 of the first device can also be connected to the second interconnection interface P22 of the second device. The one-to-one correspondence relationship of other interfaces can be adaptively modified to ensure that any two switching devices 1 establish interconnection through the corresponding interconnection interfaces 12.

[0110] For example, referring to FIG3 and FIG4, the first device and the third device are respectively used as the first switching device for explanation, and the second device and the fourth device are similarly used. The first device is the first switching device, and the second device, the third device and the fourth device are the second switching devices corresponding to the first device. When the interconnection topology architecture corresponding to the topology configuration information received by the embedded controller 13 of the first device is the 2×2 interconnection topology architecture as shown in FIG3, the embedded controller 13 in the first device confirms that the first interconnection interface P11, the second interconnection interface P12, and the third interconnection interface P13 on the first device are target interfaces, and the embedded controller 13 in the first device controls the first interconnection interface P11 in the first device to establish a connection with the first interconnection interface P21 of the second device, controls the second interconnection interface P12 in the first device to establish a connection with the first interconnection interface P31 of the third device, and controls the first interconnection interface P13 in the first device to establish a connection with the first interconnection interface P21 of the third device. The third interconnection interface P13 establishes a connection with the first interconnection interface P41 of the fourth device. When the interconnection topology architecture corresponding to the topology configuration information received by the embedded controller 13 of the first device is the 2×1 interconnection topology architecture as shown in Figure 4, the embedded controller 13 of the first device confirms that the first interconnection interface P11 on the first device is the target interface. The embedded controller 13 in the first device controls the first interconnection interface P11 in the first device to establish a connection with the first interconnection interface P21 of the second device, and controls the second interconnection interface P12 in the first device not to establish a connection with the first interconnection interface P31 in the third device, and controls the third interconnection interface P13 in the first device not to establish a connection with the first interconnection interface P41 in the fourth device.The third device is a first switching device, and the first device, the second device, and the fourth device are all second switching devices corresponding to the third device. When the interconnection topology architecture corresponding to the topology configuration information received by the embedded controller 13 of the third device is a 2×2 topology architecture as shown in FIG3 , the embedded controller 13 in the third device confirms that the first interconnection interface P31, the second interconnection interface P32, and the third interconnection interface P33 on the third device are target interfaces, and the embedded controller 13 in the third device controls the first interconnection interface P31 in the third device to establish a connection with the second interconnection interface P12 in the first device, controls the second interconnection interface P32 in the third device to establish a connection with the second interconnection interface P22 in the second device, and controls the third interconnection interface P33 in the third device to establish a connection with the third interconnection interface P33 in the fourth device. P43 establishes a connection. When the interconnection topology architecture corresponding to the topology configuration information received by the embedded controller 13 of the third device is the 2×1 topology architecture as shown in Figure 4, the embedded controller 13 on the third device confirms that the first interconnection interface P31, the second interconnection interface P32 and the third interconnection interface P33 on the third device are not target interfaces, that is, the third device is not in the current interconnection topology architecture. At this time, the embedded controller 13 on the third device controls the first interconnection interface P31 on the third device to not establish a connection with the second interconnection interface P12 on the first device, controls the second interconnection interface P32 on the third device to not establish a connection with the second interconnection interface P22 on the second device, and controls the third interconnection interface P33 on the third device to not establish a connection with the third interconnection interface P43 on the fourth device.

[0111] As an exemplary embodiment, when it is confirmed based on the topology configuration information that a target interface exists among the multiple interconnection interfaces 12 of the first switching device, a process of controlling the target interface to establish a one-to-one connection with the interconnection interface 12 of the second switching device includes:

[0112] When it is confirmed based on the topology configuration information that a target interface exists in the multiple interconnection interfaces 12 of the first switching device, the target interface is controlled to be enabled so that the target interface is connected to the enabled target interface in the second switching device in a one-to-one correspondence;

[0113] The interconnection interfaces 12 other than the target interface among the multiple interconnection interfaces 12 of the first switching device where the first switching device is located are controlled to be disabled.

[0114] In this embodiment, to improve control efficiency, the interconnection interfaces 12 of each switching device 1 can be connected in pairs with cables. As shown in Figure 2, the first interconnection interface P11 of the first device is connected to the first interconnection interface P21 of the second device with a cable. The embedded controller 13 on the first device controls whether the first interconnection interface P11 of the first device is enabled or disabled to control whether the first interconnection interface P11 of the first device is connected to the first interconnection interface P21 of the second device. The embedded controller 13 on the second device controls whether the first interconnection interface P21 of the second device is enabled or disabled to control whether the first interconnection interface P21 of the second device is connected to the first interconnection interface P11 of the first device. When the two interconnection interfaces 12 that cooperate with each other are both enabled, the two are connected. When one of the two interconnection interfaces 12 that cooperate with each other is not enabled, the two are not connected.

[0115] As an optional embodiment, the correspondence between each interconnection topology architecture, the switching device 1 that establishes the interconnection topology architecture, and the target interface on the switching device 1 can be pre-determined, and the correspondence can be pre-stored in the embedded controller 13 of each switching device 1. When the embedded controller 13 receives the topology configuration information, it determines whether the first switching device where it is located is the switching device 1 that establishes the interconnection topology architecture based on the interconnection topology architecture determined by the topology configuration information and the pre-stored correspondence. If it is determined based on the interconnection topology architecture determined by the topology configuration information and the pre-stored correspondence that the first switching device where it is located is not the switching device 1 that establishes the interconnection topology architecture, all interconnection interfaces 12 on the first switching device where it is located are controlled to be disabled. If it is determined based on the interconnection topology architecture determined by the topology configuration information and the pre-stored correspondence that the first switching device where it is located is the switching device 1 that establishes the interconnection topology architecture, the corresponding target interface is determined and enabled.

[0116] As shown in Figure 3, when the embedded controller 13 of the first device determines that the current interconnection topology architecture is a 2×2 topology architecture based on the received topology configuration information, it determines that the first device is a switching device 1 for establishing the interconnection topology architecture, and determines that all interconnection interfaces 12 on the first device are target interfaces based on the pre-stored correspondence and enable the target interface; when the embedded controller 13 of the first device determines that the current interconnection topology architecture is a 2×1 topology architecture based on the received topology configuration information, it determines that the first device is a switching device 1 for establishing the interconnection topology architecture, and determines that the first interconnection interface P11 on the first device is the target interface and enables the target interface based on the pre-stored correspondence, and at the same time controls the second interconnection interface P12 and the third interconnection interface P13 on the first device to be disabled.

[0117] As shown in Figure 4, when the embedded controller 13 of the third device determines that the current interconnection topology architecture is a 2×2 topology architecture based on the received topology configuration information, it is determined that the third device is a switching device 1 that establishes the interconnection topology architecture, and based on the pre-stored correspondence, it is determined that all interconnection interfaces 12 on the third device are target interfaces and the target interfaces are enabled. When the embedded controller 13 of the third device determines that the current interconnection topology architecture is a 2×1 topology architecture based on the received topology configuration information, it is determined that the third device is not a switching device 1 that establishes the interconnection topology architecture, and all interconnection interfaces 12 on the third device are controlled to be disabled.

[0118] As an optional embodiment, multiple switch boards may be designed, with at least two switch devices 1 being provided on each switch board.

[0119] As an optional embodiment, each switching device 1 is interconnected with the other end via a PCIE x16 link. Taking 16 GPUs as an example, the GPU interconnection bandwidth can reach 384GB / s (Gigabits per second), and the fully interconnected topology has better aggregation bandwidth.

[0120] It can be seen that in this embodiment, the interconnection device includes multiple switching devices 1, each switching device 1 includes multiple external interfaces 11 configured to connect external devices, and also includes multiple interconnection interfaces 12 configured to interconnect between switching devices 1. The interconnection interfaces 12 between any two switching devices 1 can be selectively connected, and each switching device 1 also includes an embedded controller 13. The embedded controller 13 controls the target interface on its own switching device 1 to establish a communication connection with the interconnection interface 12 on the corresponding switching device 1 according to the received topology configuration information. On the one hand, the topology structure of the interconnection device can be flexibly adjusted to suit different working scenarios. On the other hand, the interconnection device can achieve full interconnection of all external devices connected to it under the current topology structure, thereby improving the interconnection rate and communication bandwidth between the external devices. When the interconnection device is applied to the AI ​​large model all-in-one machine, it can improve the communication performance of the AI ​​large model all-in-one machine.

[0121] Based on the above embodiment:

[0122] As an exemplary embodiment, the embedded controller 13 is also configured to enumerate external devices connected to the first switching device where it is located and external devices connected to the second switching device to which the target interface is connected, determine the global number of each external device, and establish an internal routing table based on the global number so as to use the internal routing table to forward the received data; the global number of the external device is composed of the device number of the external device and the device number of the switching device 1 to which the external device is connected.

[0123] In this embodiment, since the external device connected to the external interface 11 on the switching device 1 can be interconnected with other external devices through the switching device 1 and the interconnection interface 12 of the switching device 1, as shown in Figure 5, the first GPU G0 connected to the first device can be interconnected with the second GPU G1, the third GPU G2, and the fourth GPU G3 connected to the first device, and can also be interconnected with the ninth GPU G8 on the third device through the interconnection interface 12 on the first device, and can be interconnected with the fifth GPU G4 on the second device through the interconnection interface 12 on the first device. Therefore, the embedded controller 13 on the switching device 1 in this application is configured to enumerate the external devices connected to the switching device 1 where it is located, as well as the external devices connected to the second switching device connected to the target interface. That is, the embedded controller 13 of the first device can enumerate the external devices G0 to G3 connected to its own external interface 11, and can also enumerate the external devices G4 to G7 connected to the external interface 11 of the second device, the external devices G8 to G11 connected to the external interface 11 of the third device, and the external devices G12 to G15 connected to the external interface 11 of the fourth device. It can be understood that each switching device 1 has its own device number, and each external device connected to the device has its own device number. Since the embedded controller 13 of each switching device 1 in this application can enumerate all external devices, for a system with multiple switching devices 1 to achieve large-scale interconnection, in order to facilitate distinction, this embodiment uses the device number and the device's internal ID (Identification) number to form a segmented unique number, thereby achieving the number uniqueness of all external devices in the entire system. After determining the global number of each external device, the corresponding internal routing table can be established based on the global numbers of all enumerated external devices to achieve mapping between internal interfaces. The internal routing table performs data forwarding within the switching device 1 to improve data forwarding efficiency, thereby improving communication efficiency.

[0124] For example, as shown in FIG7 , FIG7 illustrates two switching devices 1, namely a first device and a second device, and both switching devices 1 include an embedded controller 13. The external interface of the first device is connected to the first host device, the first external device, and the second external device, respectively. The external interface of the second device is connected to the second host device, the third external device, and the fourth external device, respectively. It can be understood that the first host device and the second host device are also external devices. The embedded controller 13 in the first device and the embedded controller 13 in the second device are connected through a fabric (fully interconnected line), wherein the global number assigned to the first host device is 0001:03:00.0, the global number assigned to the first external device is 0001:01:00.0, and the global number assigned to the second external device is 00 01:02:00.0, where the first segment 0001 in the global number is the device number of the first device, the second segment in the global number is the device number of each device, 01 is the device number of the first external device, 02 is the device number of the second external device, and 03 is the device number of the first host device; the global number assigned to the second host device is 0002:03:00.0, the global number assigned to the third external device is 0002:01:00.0, and the global number assigned to the fourth external device is 0002:02:00.0, where the first segment 0002 in the global number is the device number of the second device, the second segment in the global number is the device number of each device, 01 is the device number of the third external device, 02 is the device number of the fourth external device, and 03 is the device number of the second host device. As shown in FIG7 , there are two communication links between the host device, the switching device 1, and the external device. One is a configuration TLP (Transaction Layer Packet) based on ID (global number) routing, refer to the dotted line in FIG7 , and the other is a data TLP based on address (global address) routing, refer to the short dashed line in FIG7 .

[0125] As an exemplary embodiment, the embedded controller 13 is also configured to receive interface configuration information, and adjust the interface parameters of the external interface 11 of the first switching device in which it is located based on the interface configuration information, so that the interface parameters of the external interface 11 match the communication parameters of the external device connected to the external interface 11.

[0126] In this embodiment, for the heterogeneous resource pool system, each switching device 1 is interconnected with the external device using a unified interface. For each external interface on the switching device 1, the external device connected to the external interface is determined, and the interface parameters of the external interface are configured based on the communication parameters of the external device. Optionally, corresponding interface configuration information is generated according to the communication parameters of each external device. The embedded controller 13 adjusts the interface parameters of the external interface 11 of the switching device 1 where it is located based on the received interface configuration information. The interface parameters include but are not limited to basic attributes such as interface type, bandwidth, rate and clock, and also include interface Bifurcation (splitting) settings, interface clock settings, and interface resource reservation settings, which greatly improves the utilization efficiency of the switching device 1 and makes the heterogeneous resource pool system suitable for different application scenarios.

[0127] In an exemplary embodiment, the overall technical solution of embedded firmware is described. As shown in Figure 8, through the embedded controller 13, a complex resource logical structure based on the Virtual Bridge can be implemented, supporting the configuration of basic attributes such as port type, bandwidth, rate and clock, supporting complex topology interconnection, and realizing resource pooling for large-scale networking: configuration TLP (configuration TLP is a data packet used to configure PCIE devices, such as modifying the value of device registers) forwarding based on global address ID routing can be implemented, and DLUT (Destination Look Up Table) is established to implement TLP routing for complex topologies. Each switching device 1 also provides a rich set of peripheral low-speed protocols, such as UART (Universal Asynchronous Receiver / Transmitter) and GPIO (General-purpose input / output), supports secure boot to ensure device security, supports AER (Advanced Error Reporting), and supports fault monitoring mechanisms such as real-time monitoring of the health status of I / O resources. The embedded controller 13 of the switching device 1 implements the I / O resource logical structure design based on the Virtual Bridge, independently enumerates the Virtual Bridge inside the switching device 1 through the management port, and is configured to uniformly manage the device resources connected to the switching device 1.

[0128] As an exemplary embodiment, the number of switching devices 1 is n, the number of interconnection interfaces 12 of each switching device 1 is n-1, n is an integer greater than 1, and the j-th interconnection interface 12 of the i-th switching device 1 is selectively connected to the i-th interconnection interface 12 of the j+1-th switching device 1, i=1, 2,…, n-1, j=i,…, n-1.

[0129] In this embodiment, in order to achieve full interconnection between the switching devices 1, the number of interconnection interfaces 12 of each switching device 1 is limited. When the number of switching devices 1 is n, the number of interconnection interfaces 12 on each switching device 1 is n-1.

[0130] Exemplarily, when there are four switching devices 1, the number of interconnection interfaces 12 of each switching device 1 is three, and the four switching devices 1 are respectively the first device, the second device, the third device and the fourth device. The first interconnection interface on the first device is P11, the second interconnection interface on the first device is P12, and the third interconnection interface on the first device is P13. The first interconnection interface on the second device is P21, the second interconnection interface on the second device is P22, and the third interconnection interface on the second device is P23. The first interconnection interface on the third device is P31, the second interconnection interface on the third device is P32, and the third interconnection interface on the third device is P33. The first interconnection interface on the fourth device is P41, the second interconnection interface on the fourth-third device is P42, and the third interconnection interface on the fourth device is P43. The corresponding connection relationship can be shown in Figure 2. The four switching devices 1 can support switching between a 2×1 full interconnection topology and a 2×2 full interconnection topology. The switching method can be shown in Figures 3 and 4.

[0131] For example, referring to Figure 9, when there are six switching devices 1, each switching device 1 includes five interconnection interfaces 12, and the six switching devices 1 are respectively the first device, the second device, the third device, the fourth device, the fifth device and the sixth device. The first interconnection interface on the first device is P11, the second interconnection interface on the first device is P12, the third interconnection interface on the first device is P13, the fourth interconnection interface on the first device is P14, and the fifth interconnection interface on the first device is P15. The first interconnection interface on the second device is P21, the second interconnection interface on the second device is P22, the third interconnection interface on the second device is P23, the fourth interconnection interface on the second device is P24, and the fifth interconnection interface on the second device is P25. The first interconnection interface on the third device is P31, the second interconnection interface on the third device is P32, the third interconnection interface on the third device is P33, the fourth interconnection interface on the third device is P34, and the fifth interconnection interface on the third device is P35. 5. The first interconnection interface on the fourth device is P41, the second interconnection interface on the fourth device is P42, the third interconnection interface on the fourth device is P43, the fourth interconnection interface on the fourth device is P44, and the fifth interconnection interface on the fourth device is P45. The first interconnection interface on the fifth device is P51, the second interconnection interface on the fifth device is P52, the third interconnection interface on the fifth device is P53, the fourth interconnection interface on the fifth device is P54, and the fifth interconnection interface on the fifth device is P55. The first interconnection interface on the sixth device is P61, the second interconnection interface on the sixth device is P62, the third interconnection interface on the sixth device is P63, the fourth interconnection interface on the sixth device is P64, and the fifth interconnection interface on the sixth device is P65. The corresponding connection relationship can be shown in Figure 9. The six switching devices 1 can support switching between a 2×1 full interconnection topology, a 2×2 full interconnection topology, and a 2×3 full interconnection topology. The dotted lines in Figure 9 indicate optional connections.

[0132] For example, referring to Figure 10, when there are eight switching devices 1, each switching device 1 includes seven interconnection interfaces 12, and the eight switching devices 1 are respectively the first device, the second device, the third device, the fourth device, the fifth device, the sixth device, the seventh device and the eighth device. The first interconnection interface on the first device is P11, the second interconnection interface on the first device is P12, the third interconnection interface on the first device is P13, the fourth interconnection interface on the first device is P14, the fifth interconnection interface on the first device is P15, the sixth interconnection interface on the first device is P16, the seventh interconnection interface on the first device is P17, the first interconnection interface on the second device is P21, the second interconnection interface on the second device is P22, and the second interconnection interface on the second device is P23. The third interconnection interface on the second device is P23, the fourth interconnection interface on the second device is P24, the fifth interconnection interface on the second device is P25, the sixth interconnection interface on the second device is P26, the seventh interconnection interface on the second device is P27, the first interconnection interface on the third device is P31, the second interconnection interface on the third device is P32, the third interconnection interface on the third device is P33, the fourth interconnection interface on the third device is P34, the fifth interconnection interface on the third device is P35, the sixth interconnection interface on the third device is P36, the seventh interconnection interface on the third device is P37, the first interconnection interface on the fourth device is P41, the second interconnection interface on the fourth device is P42, the third interconnection interface on the fourth device is P 43, the fourth interconnection interface on the fourth device is P44, the fifth interconnection interface on the fourth device is P45, the sixth interconnection interface on the fourth device is P46, the seventh interconnection interface on the fourth device is P47, the first interconnection interface on the fifth device is P51, the second interconnection interface on the fifth device is P52, the third interconnection interface on the fifth device is P53, the fourth interconnection interface on the fifth device is P54, the fifth interconnection interface on the fifth device is P55, the sixth interconnection interface on the fifth device is P56, the seventh interconnection interface on the fifth device is P57, the first interconnection interface on the sixth device is P61, the second interconnection interface on the sixth device is P62, the third interconnection interface on the sixth device is P63, and the fourth interconnection interface on the sixth device is P64. The fourth interconnection interface is P64, the fifth interconnection interface on the sixth device is P65, the sixth interconnection interface on the sixth device is P66, the seventh interconnection interface on the sixth device is P67, the first interconnection interface on the seventh device is P71, the second interconnection interface on the seventh device is P72, the third interconnection interface on the seventh device is P73, the fourth interconnection interface on the seventh device is P74, the fifth interconnection interface on the seventh device is P75, the sixth interconnection interface on the seventh device is P76, the seventh interconnection interface on the seventh device is P77, the first interconnection interface on the eighth device is P81, the second interconnection interface on the eighth device is P82, the third interconnection interface on the eighth device is P83, and the fourth interconnection interface on the eighth device is P84.The fifth interconnection interface on the eighth device is P85, the sixth interconnection interface on the eighth device is P86, and the seventh interconnection interface on the eighth device is P87. The corresponding connection relationship can be referred to as shown in Figure 10. The eight switching devices 1 can support switching between 2×1 full interconnection topology, 2×2 full interconnection topology, 2×3 full interconnection topology, and 2×4 full interconnection topology.

[0133] 11 , 12 , 13 , and 14 , the process of switching eight switching devices 1 to implement a 2×1 fully interconnected topology, a 2×2 fully interconnected topology, a 2×3 fully interconnected topology, and a 2×4 fully interconnected topology is described.

[0134] FIG11 shows a schematic diagram of an interconnection device of a second 2×1 fully interconnected topology architecture provided by the present application, wherein a first interconnection interface P11 of a first device is connected to a first interconnection interface P21 of a second device;

[0135] Figure 12 is a schematic diagram of an interconnection device of a second 2×2 fully interconnected topology architecture provided by the present application, wherein a first interconnection interface P11 of a first device is connected to a first interconnection interface P21 of a second device, a second interconnection interface P12 of the first device is connected to a first interconnection interface P31 of a third device, a third interconnection interface P13 of the first device is connected to a first interconnection interface P41 of a fourth device, a second interconnection interface P22 of the second device is connected to a second interconnection interface P32 of the third interconnection interface 12, a third interconnection interface P23 of the second device is connected to a second interconnection interface P42 of the fourth device, and a third interconnection interface 12 of the third device is connected to a third interconnection interface 12 of the fourth device;

[0136] Figure 13 is a schematic diagram of an interconnection device of a 2×3 fully interconnected topology architecture provided by the present application, wherein the first interconnection interface P11 of the first device is connected to the first interconnection interface P21 of the second device, the second interconnection interface P12 of the first device is connected to the first interconnection interface P31 of the third device, the third interconnection interface P13 of the first device is connected to the first interconnection interface P41 of the fourth device, the second interconnection interface P22 of the second device is connected to the second interconnection interface P32 of the third interconnection interface 12, the third interconnection interface P23 of the second device is connected to the second interconnection interface P42 of the fourth device, the third interconnection interface 12 of the third device is connected to the third interconnection interface 12 of the fourth device, and the fourth interconnection interface P14 of the first device is connected to the first interconnection interface P41 of the fifth device. Interface P51 establishes a connection, the fifth interconnection interface P15 of the first device establishes a connection with the first interconnection interface P61 of the sixth device, the fourth interconnection interface P24 of the second device is connected with the second interconnection interface P52 of the fifth device, the fifth interconnection interface P25 of the second device is connected with the second interconnection interface P62 of the sixth device, the fourth interconnection interface P34 of the third device is connected with the third interconnection interface P53 of the fifth device, the fifth interconnection interface P35 of the third device is connected with the third interconnection interface P63 of the sixth device, the fifth interconnection interface 12 of the fourth device is connected with the fourth interconnection interface P54 of the fifth device, the sixth interconnection interface 12 of the fourth device is connected with the fourth interconnection interface P64 of the sixth device, and the fifth interconnection interface P55 of the fifth device is connected with the fifth interconnection interface P65 of the sixth device.

[0137] Figure 14 is a schematic diagram of an interconnection device of a 2×4 fully interconnected topology architecture provided by the present application, wherein the j-th interconnection interface Pij of the i-th device establishes a connection with the i-th interconnection interface P(j+1)i of the j+1-th switching device 1.

[0138] In a second aspect, referring to FIG. 15 , the present application further provides a high-performance switching device, including:

[0139] The interconnection device 21 as in any one of the above items, wherein the interconnection device 21 comprises a plurality of switching devices, each switching device comprises an embedded controller, and the embedded controller comprises a plurality of configuration registers;

[0140] The management device 22 is configured to receive a user configuration instruction, determine topology configuration information and / or interface configuration information based on the user configuration instruction, determine a configuration parameter group for each switching device based on the topology configuration information and / or the interface configuration information, and write each configuration parameter value in each configuration parameter group into each configuration register in the embedded controller of the switching device;

[0141] The power supply device 23 is configured to supply power to the interconnection device 21 and the management device 22 .

[0142] In this embodiment, the high-performance switching device includes a management device 22, a power supply device 23 and an interconnection device 21 as described in the above embodiment, wherein the interconnection device 21 includes multiple switching devices, and different interconnection topology architectures are composed of multiple switching devices. As an optional embodiment, the multiple switching devices in the above-mentioned interconnection device 21 can be set on a data exchange board, and the management device 22 interacts with each switching device in the interconnection device 21 for data. The power supply device 23 includes a PSU (Power Supply Unit) and multiple VR (Voltage Regulated) modules. The PSU supplies power to the management device 22, the interconnection device 21 or other devices in the high-performance switching device through the corresponding VR.

[0143] The management device 22 is interconnected with the data exchange board via a connector to implement links such as a network, USB (Universal Serial Bus), UART, and IIC (Inter-Integrated Circuit Bus), thereby realizing data communication and control functions for each link. Optionally, the management device 22 is configured to receive user configuration instructions sent by the user, wherein the user configuration instructions include topology configuration instructions and / or interface configuration instructions. After receiving the topology configuration instructions, the management device 22 determines topology configuration information. Based on the topology configuration information, topology configuration parameters corresponding to each switching device can be determined. After receiving the interface configuration instructions, the management device 22 determines interface configuration information. Based on the interface configuration information, interface configuration parameters corresponding to each switching device can be determined. When the management device 22 only receives the interface configuration instructions, the configuration parameter group corresponding to each switching device includes the interface configuration parameters. When the management device 22 only receives the topology configuration instructions, the configuration parameter group corresponding to each switching device includes the topology configuration parameters. When the management device 22 receives both the interface configuration instructions and the topology configuration instructions, the configuration parameter group corresponding to each switching device includes the interface configuration parameters and the topology configuration parameters.

[0144] In this embodiment, the embedded controller in each switching device includes multiple configuration registers, and the management device 22 writes each configuration parameter in the configuration parameter group into the configuration register corresponding to the embedded controller, so that the embedded controller can determine the interconnection status of the switching device in which it is located and configure the parameters of the external interface by reading the configuration register, so that the interconnection device 21 meets the topology requirements of the current application scenario and improves the flexibility of adjusting the interconnection topology structure.

[0145] As an exemplary embodiment, referring to FIG16 , the management device 22 includes a plurality of management controllers 221 , and the plurality of management controllers 221 are all connected to the interconnection device 21 ;

[0146] The current interaction controller 222 is configured to perform data interaction with the interconnection device 21 ; the current interaction controller 222 is any one of the multiple management controllers 221 that is in a normal working state.

[0147] In this embodiment, the management device 22 includes multiple management controllers 221 that serve as hot standbys for each other. The management controllers 221 are primarily responsible for topology switching management, data transmission management, ID identification, and other functions within the interconnection device 21. They are core modules of a high-performance switching device, and their redundancy is designed to meet the needs of switching control signals between the management device 22 and the baseboard controller, and between the management device 22 and the switching device, in the event of a failure. Each management controller 221 is connected to each switching device within the interconnection device 21. In this embodiment, a management controller 221 in normal operation is selected from the multiple management controllers 221 as the current interaction controller 222. The current interaction controller 222 is configured to implement data exchange with each switching device. This data exchange includes, but is not limited to, sending configuration parameter groups to each switching device within the interconnection device 21, obtaining status parameters of each switching device, and obtaining device parameters of external devices connected to each switching device.

[0148] In an exemplary embodiment, communication efficiency and / or proximity may be used as selection principles for the current interaction controller 222 . Of course, random selection or other selection schemes may also be used, which is not limited in this embodiment.

[0149] As an exemplary embodiment, referring to FIG17 , the high-performance switching device further includes a complex programmable logic device 24 ;

[0150] Each management controller 221 is also configured to send a heartbeat signal to the complex programmable logic device 24 at a preset period;

[0151] The complex programmable logic device 24 is configured to determine whether there is a faulty controller in an abnormal working state among the multiple management controllers 221. If it is determined that there is a faulty controller in an abnormal working state among the multiple management controllers 221, it is determined whether the faulty controller is the current interaction controller 222. If it is determined that the faulty controller is the current interaction controller 222, the faulty controller is controlled to stop data interaction with the interconnection device 21, and any one of the multiple management controllers 221 that is in a normal working state is selected as the new current interaction controller 222; the abnormal working state is a working state in which the heartbeat signal is not sent according to the preset period.

[0152] In this embodiment, the high-performance switching device also includes a complex programmable logic device (CPLD) 24. The heartbeat port of each management controller 221 is connected to the CPLD. Each management controller 221 sends a heartbeat signal to the CPLD through the heartbeat port according to a preset period. The CPLD is configured to determine whether the heartbeat signal sent by each management controller 221 is received in the preset period corresponding to the management controller 221. If there is a management controller 221 that does not send a heartbeat message according to the preset period, it is determined that the management controller 221 is currently in an abnormal working state, and the management controller 221 currently in the abnormal working state is determined as a faulty controller. The CPLD determines whether the faulty controller is the current interaction controller 222 configured to perform data interaction with each switching device. If so, the CPLD controls the current interaction controller 222 to stop performing data interaction with each switching device to ensure normal system operation. The CPLD then determines a management controller 221 from all other management controllers 221 in normal operation other than the faulty controller as the new current interaction controller 222, and performs data interaction with each switching device through the new current interaction controller 222. If the faulty controller is not the current interaction controller 222, a prompt message corresponding to the faulty controller is generated to remind staff to promptly perform maintenance operations on the faulty controller.

[0153] For example, it is assumed that the high-performance switching device includes two management controllers 221, namely the first management controller and the second management controller. It is assumed that the two management controllers 221 are currently in normal working state, and the first management controller is determined as the current interactive controller 222, and the first management controller interacts with each switching device. If the CPLD detects that the first management controller does not send a heartbeat signal according to the preset period, it is determined that the first management controller is in an abnormal working state, and the first management controller is determined as a faulty controller. The first management controller is controlled to stop interacting with each switching device. At this time, if the second management controller is in a normal working state, the second management controller is determined as the current interactive controller 222, and the second management controller interacts with each switching device, thereby improving the operation safety of the high-performance switching device.

[0154] As an exemplary embodiment, the multiple management controllers 221 include a master management controller and a slave management controller, and the slave management controller is a management controller 221 other than the master management controller among the multiple management controllers 221;

[0155] The complex programmable logic device 24 is also configured to control the current interaction controller 222 to stop data interaction with the interconnection device 21 when it is determined that the current interaction controller 222 is a slave management controller and it is determined that there is a master management controller in normal working condition, and select a master management controller in normal working condition as the new current interaction controller 222.

[0156] In this embodiment, communication efficiency and / or proximity can be used as the selection criteria for the master management controller. That is, the communication efficiency between the master management controller and the switching device or baseboard controller is higher than the communication efficiency between the slave management controller and the switching device or baseboard controller. Therefore, when determining the current interaction controller 222, the current interaction controller 222 can be selected based on the principle of master management controller priority. If no master management controller is currently in normal operation, a slave management controller in normal operation is selected as the current interaction controller 222. If, while a slave management controller is acting as the current interaction controller 222 and interacting with various switching devices, the CPLD detects that a master management controller is in normal operation, management authority is returned to the master management controller. Specifically, the slave management controller acting as the current interaction controller 222 is controlled to stop interacting with various switching devices, and a master management controller in normal operation is controlled to serve as the new current interaction controller 222, thereby improving communication efficiency.

[0157] As an exemplary embodiment, as shown in FIG18 , the high-performance switching device further includes:

[0158] The first port expansion device 25 has a first end connected to the second end of the first switching device 26 , and multiple second ends of the first port expansion device 25 are connected to multiple switching devices in a one-to-one correspondence.

[0159] In this embodiment, port control is performed through the first port expansion device 25, which, on the one hand, reduces the occupancy of the port of the first switching device, and on the other hand, enables the interconnection device 21 to mount more switching devices, thereby establishing a larger system topology and further improving the system computing power.

[0160] As an exemplary embodiment, referring to FIG. 18 , the high-performance switching device further includes a first switching device 26 , wherein multiple first terminals of the first switching device 26 are connected to multiple management controllers 221 in a one-to-one correspondence, multiple second terminals of the first switching device 26 are connected to multiple switching devices in a one-to-one correspondence, and a control terminal of the first switching device 26 is connected to the complex programmable logic device 24 ;

[0161] The complex programmable logic device 24 is configured to determine whether there is a faulty controller in an abnormal working state among the multiple management controllers 221, and if it is determined that there is a faulty controller in an abnormal working state among the multiple management controllers 221, determine whether the faulty controller is the current interaction controller 222, and if it is determined that the faulty controller is the current interaction controller 222, generate a first control instruction corresponding to the faulty controller, select any one of the multiple management controllers 221 in a normal working state as a new current interaction controller 222, and generate a second control instruction corresponding to the new current interaction controller 222;

[0162] The first switching device 26 is configured to, upon receiving a first control instruction, determine a first target end based on the first control instruction, and control the communication link between the management controller 221 connected to the first target end and each switching device to be disconnected; and upon receiving a second control instruction, determine a second target end based on the second control instruction, and control the communication link between the management controller 221 connected to the second target end and each switching device to be connected.

[0163] In this embodiment, the high-performance switching device also includes a first switching device 26, each management controller 221 is connected to the first end of the first switching device 26, and multiple second ends of the first switching device 26 are respectively connected one-to-one with multiple switching devices. The CPLD adjusts the status of multiple communication links in the first switching device 26 by sending corresponding control instructions to the first switching device 26. The communication link status includes a connected state or a disconnected state.

[0164] Assume that the high-performance switching device includes a first management controller and a second management controller, the first management controller is connected to a first end (d1) of the first switching device 26, the second management controller is connected to the other first end (d2) of the first switching device 26, and a second end (d3) of the first switching device 26 is connected to a port of the first port expansion device. When the first management controller is in normal working state and the first management controller is the main management controller, the first switching device 26 controls the communication link between d1 and d3 to be in a connected state, and controls the communication link between d2 and d3 to be in a disconnected state. When the first management controller is in an abnormal working state and the second management controller is in a normal working state, the first switching device 26 controls the communication link between d1 and d3 to be in a disconnected state, and controls the communication link between d2 and d3 to be in a connected state, under the control of the first control instruction and the second control instruction sent by the CPLD.

[0165] As an exemplary embodiment, referring to FIG. 19 , the process of interacting with the interconnected device 21 includes:

[0166] Writing each configuration parameter in each configuration parameter group into each configuration register in the embedded controller of the switching device;

[0167] Obtain the status parameters of each switching device and the device parameters of the external devices connected to each switching device;

[0168] The high-performance switching device also includes:

[0169] The first baseboard controller 27 connected to each management controller 221 is configured to determine the current topology state of the interconnection device 21 through the state parameters of each switching device, and generate a current state prompt instruction based on the current topology state;

[0170] prompting device 28;

[0171] The complex programmable logic device 24 is further configured to respond to a current state prompt instruction and control the prompt device 28 to prompt current topology state information.

[0172] In this embodiment, the high-performance switching device also includes a first baseboard controller 27 and a prompt device 28. The current interactive controller 222 obtains the status parameters of each switching device and the device parameters of the external devices connected to each switching device. The first baseboard controller 27 can determine the current topology state of the interconnection device 21 based on the status parameters of each switching device and the device parameters of the external devices connected to each switching device, thereby controlling the prompt device 28 to prompt the corresponding topology state information so that the staff can check it. If the current topology state does not match the preset, it can be checked and adjusted in time.

[0173] As an exemplary embodiment, referring to FIG20 , the high-performance switching device further includes a heat dissipation device 29 , and the heat dissipation device 29 includes at least one fan;

[0174] The first baseboard controller 27 is further configured to generate a current heat dissipation control instruction based on the current topology state and device parameters;

[0175] The complex programmable logic device 24 is further configured to adjust the speed of the fan in response to the current heat dissipation control instruction.

[0176] In this embodiment, the high-performance switching device further includes a heat dissipation device 29 , which can dynamically adjust the heat dissipation level of the heat dissipation device 29 according to the current interconnection topology architecture and the number of devices under the architecture, thereby improving the operational safety of the high-performance switching device.

[0177] As an exemplary embodiment, the first baseboard controller 27, the management controller 221, the complex programmable logic device 24 and the interconnection device 21 are connected via Ethernet, and the first baseboard controller 27, the management controller 221, the complex programmable logic device 24 and the interconnection device 21 are connected via a universal serial bus or an integrated circuit bus.

[0178] In a third aspect, referring to FIG. 21 , the present application further provides a large-scale all-in-one machine, which includes:

[0179] A high performance switching device 31 as described above;

[0180] A plurality of general-purpose computing devices 33, wherein the plurality of general-purpose computing devices 33 are connected to the interconnection device of the high-performance switching device 31;

[0181] A plurality of heterogeneous computing devices 32, wherein the plurality of heterogeneous computing devices 32 are connected to an interconnection device;

[0182] Any two computing devices connected to the interconnection device are interconnected, and the computing devices are general computing devices 33 or heterogeneous computing devices 32.

[0183] In this embodiment, multiple general-purpose computing devices 33 and multiple heterogeneous computing devices 32 are all connected to the corresponding switching devices in the interconnection device, and the general-purpose computing devices 33 and the heterogeneous computing devices 32 are connected to the external interface of the switching device. As shown in Figure 22, the interconnection device includes four switching devices (respectively, a first device, a second device, a third device, and a fourth device). The four switching devices constitute a 2×2 interconnection topology architecture. The large model all-in-one machine includes two general-purpose computing devices 33 (respectively, a first general-purpose computing device 331 and a second general-purpose computing device 332) and sixteen heterogeneous computing devices 32. The heterogeneous computing devices 32 can be full-height and full-length GPUs, represented by G0 to G15 in Figure 22, and Figure 2 2, every four GPUs form a group, with a total of four groups. As shown in Figure 23, the interconnection device includes six switching devices, and the six switching devices (respectively, the first device, the second device, the third device, the fourth device, the fifth device and the sixth device) constitute a 2×3 interconnection topology architecture. The large model all-in-one machine includes two general-purpose computing devices 33 (respectively, the first general-purpose computing device 331 and the second general-purpose computing device 332) and sixteen heterogeneous computing devices 32. The heterogeneous computing devices 32 can be full-height and full-length GPUs, represented by G0 to G15 in Figure 23. The large model all-in-one machine provided in this embodiment supports multiple cards in a single machine to improve computing power density. The two general-purpose computing devices 33 have a total of four standard PCIE x16s, which are connected to four switching devices respectively, satisfying the upstream and downstream matching relationship between the general-purpose computing devices 33 and the heterogeneous computing devices 32.

[0184] As an exemplary embodiment, referring to FIG. 24 , the large model integrated machine further includes a second substrate controller 34 and a plurality of second switching devices 35 ;

[0185] The first ends of the plurality of second switching devices 35 are connected to the plurality of heterogeneous computing devices 32 in a one-to-one correspondence, the second end of each second switching device 35 is connected to the second baseboard controller 34 , and the third end of each second switching device 35 is connected to the high-performance switching device 31 .

[0186] In this embodiment, the heterogeneous computing acceleration pooling board composed of multiple heterogeneous computing devices 32 also includes a second baseboard controller 34 and multiple second switching devices 35. The first ends of the multiple second switching devices 35 are connected one-to-one with the multiple heterogeneous computing devices 32, the second end of each second switching device 35 is connected to the second baseboard controller 34, and the third end of each second switching device 35 is connected to the high-performance switching device 31, so that the heterogeneous computing device 32 can interact with the high-performance switching device 31 and the second baseboard controller 34 through two links respectively, providing more data paths for software development. As shown in Figure 24, in order to perform port expansion, the high-performance switching device 31 also includes multiple second port expansion devices, and the third end of the second switching device 35 is connected to the second port expansion device in the high-performance switching device 31. The large model all-in-one machine also includes a third port expansion device. This embodiment is implemented by a management hardware architecture that supports a dual-redundancy system. Its management core is a set of management software running on a management controller with high-performance data link topology identification, high-performance data link topology switching, and a visual operation interface. Through software definition, a business-aware resource reconstruction decision system is established to achieve functions such as dynamic allocation of uplink and downlink links, link topology switching, and dynamic reconstruction of data links. It completes intelligent reconstruction of hardware resources and realizes pooling and centralized management of resources, thereby achieving dynamic adjustment, flexible combination, and intelligent allocation, improving the computing efficiency and response rate of the entire system, realizing intelligent and efficient heterogeneous computing, and covering the application requirements of mainstream models and business scenarios.

[0187] As an optional embodiment, the high-performance switching device 31 can use IT968GSE+RTF3 low-loss PCB board material. Each switching device is interconnected through the PCIE GEN5x16 bus, and a single-board physical connection of up to four switching devices can be achieved through MCIO (Memory Channel I / O) high-speed cables. In addition, each processor supports flexible bifurcation operations. In addition to supporting traditional PCIE, UART, and IIC links, the heterogeneous computing acceleration pooling board also supports network data interaction links, providing more efficient and fast data transmission. Two homologous clock CLK BUFFERs (Clock BUFFERs) and non-homologous CLK lines can be added to the high-performance switching device 31 to provide more reliable input clock source redundancy. All key signals such as management, control, and status monitoring of the high-performance switching device 31 are connected to the backplane COME (Computer On Module Express) module and RunBMC (Run Baseboard Management Controller) module for real-time status detection. Once an abnormal situation occurs in the machine (abnormal operation of the switching equipment, error alarm of the switching link), the RunBMC module will transmit the data to the upper level and alarm, and at the same time detect the detailed time scale of the signal, and save it in the form of a log to the TF (TransFlash) card, which is convenient for users to retrieve and analyze the cause of the abnormality.

[0188] As an exemplary embodiment, the large model all-in-one machine also includes multiple network cards, and the interconnection device includes multiple switching devices, each of which is connected to at least one network card, so that the heterogeneous computing devices 32 connected to the interconnection device can be networked with other large model all-in-one machines through the network cards.

[0189] In this embodiment, a network card is added to the topology, and the network card is connected to the external interface on the switching device. Based on the large model all-in-one machine provided in FIG22, as shown in FIG25, sixteen heterogeneous computing devices 32 (respectively, a first heterogeneous computing device G0, a second heterogeneous computing device G1, ..., a sixteenth heterogeneous computing device G15), two general-purpose computing devices 33 (respectively, a first general-purpose computing device 331 and a second general-purpose computing device 332), an interconnection device including four switching devices (respectively, a first device, a second device, a third device, and a fourth device) and four network cards (a first network card, a second network card, and a fourth network card) are used as an example for illustration. Among them, G0, G1, G2, and G3 are interconnected to the external network through the first network card to expand the interconnection bandwidth of the large model all-in-one machine, which can support eight 800G high-speed network RDMA (Remote Direct Memory Access) interconnections. It can be understood that x16 in FIG25 is used to illustrate bandwidth.

[0190] As an exemplary embodiment, the interconnection device includes multiple switching devices, the large model all-in-one machine also includes multiple network cards, the heterogeneous computing devices are connected to the switching devices through the network cards, and the heterogeneous computing devices 32 are networked with other large model all-in-one machines through the network cards.

[0191] In this embodiment, an intelligent network card CX7 can also be selected to realize external network interconnection. In this embodiment, each heterogeneous computing device 32 is not directly connected to the switching device. Each heterogeneous computing device 32 is connected to the switching device through an intelligent network card CX7, which can reduce the excessive occupation of the external interface of the switching device. Based on the large-scale all-in-one machine provided in Figure 22, as shown in Figure 26, in the large-scale all-in-one machine provided in this embodiment, sixteen heterogeneous computing devices 32 (respectively, a first heterogeneous computing device G0, a second heterogeneous computing device G1, ..., a sixteenth heterogeneous computing device G15), two general-purpose computing devices 33 (respectively, a first general-purpose computing device 331 and a second general-purpose computing device 332), an interconnection device including four switching devices (respectively, a first device, a second device, a third device, and a fourth device) and sixteen network cards (a first network card, a second network card, ..., a sixteenth network card) are used as an example for illustration.

[0192] It can be understood that in terms of cluster network interconnection, the large-model all-in-one system is designed with sufficient and flexible I / O resources, which can support a variety of large-scale cluster expansion methods and flexibly meet application needs, such as the cluster expansion methods shown in Figures 27-29. Figure 27 is a schematic diagram of the first large-model all-in-one network provided by this application, which shows a network card expansion method based on a switching device. Figure 28 is a schematic diagram of the second large-model all-in-one network provided by this application, which shows a heterogeneous computing device with an integrated network card and a direct network port expansion method. Figure 29 is a schematic diagram of the third large-model all-in-one network provided by this application, which shows a high-speed network expansion method based on a Cedar7 fusion device.

[0193] As an exemplary embodiment, referring to FIG30 , the large model all-in-one machine further includes:

[0194] at least one memory expansion card 36, configured to store data to be processed;

[0195] The general computing device 33 is configured to obtain local memory, and when the local memory is less than a preset value, writes the data to be processed into the memory expansion card 36.

[0196] In order to improve the reliability of the system, the large model all-in-one machine provided in this embodiment also includes at least one memory expansion card 36. When the local memory is insufficient, data is accessed through the memory expansion card 36. The memory expansion card 36 can be assembled in the double-width slot on the rear window of the general computing device 33. A single machine can expand up to 4 memory expansion cards 36. A single memory expansion card 36 can expand 2T (Terabyte) memory (256GB / memory stick, 8 memory sticks per card), a total of 8T memory can be expanded. Together with 8T of local memory, a single machine can achieve up to 16TB of system memory. A dual-card module can be adapted to be assembled in the standard double-width slot. A dual-card module can support two CXL (Compute Express Link) expansion cards by staggering them up and down, realizing high-density memory expansion to meet the high memory bandwidth and capacity requirements of the AI ​​large model all-in-one machine. The CXL Add-in Card is a CXL Memory Expander in a PCIe standard AIC (Add-in Card) form factor (full height, 3 / 4 length, double width). It implements serial memory expansion via the CXL cache coherence bus. In principle, it enables flexible CXL serial memory expansion for general-purpose servers supporting the CXL protocol. Structurally, it can replace a GPU slot. The card introduces 4x8 CXL 2.0 signals via a 1x16 PCIe gold finger and a 1x16 MCIO connector. It supports eight DDR5-4800 RDIMM expansion slots, with a maximum capacity of 2TB per card.

[0197] As an exemplary embodiment, referring to FIG31 , the memory expansion card 36 includes a third baseboard controller and a plurality of memory controllers, each memory controller mounting at least one memory stick;

[0198] The third backplane controller is configured to receive and forward memory access instructions sent by the general computing device 33, obtain state parameters of each memory controller, obtain target data converted into serial data, and send the target data to the general computing device 33;

[0199] The memory controller is configured to parse the memory access data in the memory access instruction. When the memory access data is data to be written, the data to be written is converted from serial data to parallel data and then written to the corresponding memory stick mounted on itself. When the memory access data is data to be read, the target data is read from the memory stick mounted on itself according to the data to be read, the target data is converted from parallel data to serial data and then forwarded to the third baseboard controller.

[0200] In this embodiment, each memory expansion board supports four x8 memory controllers. Each memory controller can be connected to at least one RDIMM (Registered Dual In-line Memory Module) for memory expansion, supporting a total of eight RDIMMs per board. The CXL x8 link of the first memory controller and the CXL x8 link of the second memory controller form a 2x8 CXL link, interconnected externally via the high-speed PCIE x16 interface. The CXL x8 link of the third memory controller and the CXL x8 link of the fourth memory controller form a 2x8 CXL link, interconnected externally via the high-speed MCIO x16 interface, supporting serial memory expansion via the host via the CXL link. The I2C (Inter-Integrated Circuit) signals of the memory controller communicate with the third backplane controller via an I2C switch, supporting debug debugging. Communication between the I2C switch and the third backplane controller is connected via the high-speed interface gold finger from the backplane. The connection between the memory controller and the third backplane controller in Figure 31 is for illustrative purposes only. The third and fourth memory controllers can also be connected to the third backplane controller via high-speed PCIE gold finger interfaces. The first and second memory controllers can also be connected to the third backplane controller via high-speed connectors. The bus switch can also be connected to one end of the high-speed connector. Considering the large number of pins on the gold finger, a low-speed port can be selected from the multiple pins on the gold finger to connect to the I2C switch, or a separate low-speed connector can be provided to connect to the I2C switch. The end of the memory controller connected to the high-speed connector or high-speed PCIE gold finger interface transmits serial data, and the end of the memory controller connected to the memory module transmits parallel data. The memory controller is configured to convert between serial and parallel data.

[0201] It is understood that each memory controller is configured to parse serial data packets sent by the third backplane controller, convert them into parallel DDR format, and then send them. Each memory controller also includes multiple status registers configured to store status parameters of the memory controller and the status parameters of the memory modules mounted on the memory controller. The third backplane controller is configured to perform management operations, such as detecting the presence of memory modules and memory controllers, controlling temperature, and sending instructions to the memory controllers by reading the memory controller status registers. Status parameters include information about the operating status of the memory modules or memory faults.

[0202] For AI large-scale all-in-one machines, thanks to the powerful resource expansion capability of the high-performance switching device 31, key devices such as CPU and heterogeneous computing resources can be decoupled to achieve modular design. A unified interface is designed between each module, and they are uniformly connected to the high-performance switching unit. The resource management module calls the CLI interface to obtain heterogeneous device resource pool information from the embedded controller side of the I / O switching device, and provides a network API interface for the visualization WEB unit. Through the visualization interface, the I / O resource list and topology display can be completed more flexibly and conveniently, as shown in Figure 32. It mainly includes list display and management and graphical display and management, among which the list display and management include heterogeneous resource list, computing unit list, and I / O port list. The graphical display and management include interconnection topology display, real-time link monitoring, and visual dynamic adjustment. Among them, the heterogeneous computing resource list: the manufacturer information of the device (VID (Vendor ID, manufacturer identifier), DID (Device The pooling management engine also obtains and displays information about the host in the heterogeneous computing system, including: the status of the device resources allocated to the host: whether devices are allocated to the host and their location information; the location of the host devices; and the health status of the device links used by the host. The I / O port list mainly includes: basic I / O port properties, I / O port resource reservation status, and I / O port configuration interface. A graphical user interface (GUI) is used to display and manage heterogeneous computing resources. Users can dynamically allocate resources in the interconnection topology view by dragging and dropping, and monitor device links in real time. In Figure 32, DP represents a downlink port, HP represents an uplink port, and FP represents an interconnection port.

[0203] As an exemplary embodiment, referring to FIG33 , the large model all-in-one machine further includes a pooling management engine;

[0204] The general computing device 33 is further configured to initiate a request for dynamic adjustment of heterogeneous resources;

[0205] The pooling management engine is configured to, upon receiving a request for dynamic adjustment of heterogeneous resources, determine the heterogeneous computing device 32 to be adjusted according to the request, perform a hot removal operation on the heterogeneous computing device 32 to be adjusted, and send a location acquisition request corresponding to the heterogeneous computing device 32 to be adjusted to the high-performance switching device 31, send a reset execution instruction to the heterogeneous computing device 32 to be adjusted based on the received physical location information, determine the heterogeneous computing device 32 to be adjusted as the heterogeneous computing device 32 to be allocated after receiving the reset completion instruction, and allocate the heterogeneous computing device 32 to the general-purpose computing device 33 corresponding to the reallocation instruction when a reallocation instruction is received;

[0206] The heterogeneous computing device 32 is configured to perform a reset operation upon receiving the reset execution instruction and generate a reset completion instruction after the reset operation is successfully completed;

[0207] The high-performance switching device 31 is configured to obtain the physical location information of the heterogeneous computing device 32 to be adjusted and return it to the pooling management engine after receiving the location acquisition request.

[0208] Through the interfaces provided by the pooling management engine, users can implement on-demand allocation, dynamic scaling, and resource release for general-purpose and heterogeneous computing devices and system resources. Users can match resources based on load requirements, and when applications stop, users can release resources back to the resource pool for efficient resource flow and full utilization.

[0209] The optional dynamic resource switching process is illustrated using Figure 33 as an example:

[0210] Taking the release of a heterogeneous computing device 32 and its allocation to a general-purpose computing device 33 as an example, before the user initiates a request for dynamic adjustment of heterogeneous computing resources through the pooling management engine, it is necessary to ensure that the application layer process related to the heterogeneous computing device has ended to avoid abnormal access to the device by the application layer, resulting in program abnormalities; the pooling management engine completes the hot removal of the device, and the pooling management engine sends a request to the high-performance switching device 31 to obtain the physical location of the device. The pooling management engine confirms the physical location of the device, and the pooling management engine resets the heterogeneous computing device resources and retrains them to restore the operating status to the default value. The pooling management engine confirms that the device has been reset, and the pooling management engine reallocates the device to another general-purpose computing device 33. The computing device will see the newly added device without the business being aware of it, and the dynamic switching of heterogeneous computing resources can be completed. On the basis of the above embodiment, in order to release the binding relationship between the heterogeneous computing device 32 and the CPU at the physical link level and realize the on-demand elastic allocation of heterogeneous computing resources by the general computing unit, different device allocation relationship expert templates can be established in the pooling management engine for different application scenarios. When the application scenario switches, the topology is automatically switched according to different templates, simplifying the application scenario switching process. Based on the network API interface provided by the pooling management engine, the system interface standard is studied, and the system framework standard is established to realize the expert template description and dynamic switching interface of heterogeneous computing resources, so that applications can apply for resources based on the expert template; at the same time, a callable API is provided to realize the visual application of the expert template and provide a logical topology display of the allocated resources.

[0211] Through hardware decoupling and standardized interface definitions, a modular design is achieved, combining heterogeneous computing units with storage, I / O, and memory resources. The host system is composed of unified management, networking, power supply, and cooling. The software-defined system design flexibly provides diverse computing power. Its standard-defined top-level architecture is shown in Figure 34. Multi-core computing modules are coherently interconnected with storage device modules, I / O (Input / Output) device modules, and global memory modules via standard interfaces. Resource sharing and dynamic allocation are achieved through switching devices. A unified control module is responsible for scheduling and coordinating the diverse computing cores. Global cache coherence is maintained between multi-core computing modules via an open internal cache coherence bus. The entire system is integrated at the system level through basic modules such as a unified network module, a unified management module, and a unified power supply and cooling module, achieving a comprehensive, single-machine, multi-core architecture.

[0212] In order to give full play to the performance of heterogeneous multi-cores and adapt to various application scenarios, it is very important that applications can be freely migrated between multi-cores. However, migration between heterogeneous multi-cores with different instruction sets has always been a difficult problem in the industry. Load migration and resource scheduling are applied in the following scenarios: the operating system performs load balancing, and processes are migrated to static or lightly loaded cores; according to different load types, they are migrated to cores with different instruction set architectures; when the power consumption state changes, it is necessary to handle the migration of some processes to achieve power consumption control; when a core is overheated, the process is migrated to other cores, etc. In this embodiment, an AI large model all-in-one machine is established. The operating system contains multiple kernels. Each kernel is compiled and runs under a specific ISA (Instruction Set Architecture) instruction set. The entire system uses unified system software to achieve on-demand balanced scheduling of computing power for applications. At the operating system level, inter-kernel communication and coordination are implemented so that the entire system still has a global state under different ISA core running instances; on this basis, through a unified scheduling core, advanced scheduling technology is applied to achieve application programs without perception of the underlying hardware. Through the collaboration of the operating system and the scheduling system, applications can automatically select the optimal mapping, realizing the performance improvement of multi-instruction set heterogeneous multi-core systems, as shown in Figure 35.

[0213] In summary, the large-model all-in-one machine provided by this application is system-centric, innovative in artificial intelligence server design, breaking the number of heterogeneous acceleration cards in the traditional 8-card server, and realizing a single-machine 16-card server design through hardware decoupling pooling and high-performance switching of the server's internal bus to meet the high computing power requirements of artificial intelligence. At the same time, when key core devices and components are limited, the system integrates multiple cards to break through technical blockades and improve the overall performance of the system. In an era of rapid development of large-model applications, multi-card server clusters can better realize the parallelization of models and data to meet the computing power requirements of artificial intelligence. The all-in-one design can adapt to various scenarios of pre-training, fine-tuning, and reasoning of large AI models, greatly saving the design and production costs of AI servers, and realizing multi-purpose use of one machine and flexible adaptation.

[0214] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0215] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An interconnection device, characterized in that: The invention comprises a plurality of switching devices, each of the switching devices comprising a plurality of external interfaces and a plurality of interconnection interfaces, wherein the external interfaces of the switching devices are configured to connect to external devices, and the interconnection interface of a first switching device is selectively connectable to the interconnection interface of a second switching device; the first switching device is any one of the plurality of switching devices, and the second switching device is any one of the plurality of switching devices except the first switching device; The first switching device also includes an embedded controller, which is configured to receive topology configuration information. When it is confirmed based on the topology configuration information that there is a target interface among the multiple interconnection interfaces of the first switching device where it is located, the embedded controller controls the target interface to establish a one-to-one connection with the interconnection interface of the second switching device, so that any two of all the external devices connected to each of the switching devices are interconnected.

2. The interconnection device according to claim 1, characterized in that The embedded controller is also configured to enumerate the external devices connected to the first switching device where the embedded controller is located and the external devices connected to the second switching device to which the target interface is connected, determine the global number of each external device, and establish an internal routing table based on the global number so as to forward the received data using the internal routing table; the global number of the external device is composed of the device number of the external device and the device number of the switching device to which the external device is connected.

3. The interconnection device according to claim 2, characterized in that: The embedded controller is further configured to receive interface configuration information and adjust interface parameters of the external interface of the first switching device based on the interface configuration information so that the interface parameters of the external interface match communication parameters of an external device connected to the external interface.

4. The interconnection device according to claim 1, characterized in that When it is confirmed based on the topology configuration information that a target interface exists among the plurality of interconnected interfaces of the first switching device, the process of controlling the target interface to establish a connection with the interconnected interfaces of the second switching device in a one-to-one correspondence includes: When it is confirmed based on the topology configuration information that a target interface exists among the plurality of interconnected interfaces of the first switching device, controlling the target interface to be enabled so that the target interface is connected to the enabled target interface of the second switching device in a one-to-one correspondence; The interconnection interfaces other than the target interface among the plurality of interconnection interfaces of the first switching device where the first switching device is located are controlled to be disabled.

5. The interconnection device according to any one of claims 1 to 4, characterized in that: The number of the switching devices is n, and the number of the interconnection interfaces of each switching device is n-1, where n is an integer greater than 1. The j-th interconnection interface of the i-th switching device and the i-th interconnection interface of the j+1-th switching device can be selectively connected, i=1, 2, ..., n-1, j=i, ..., n-1.

6. A high-performance switching device, characterized in that: include: The interconnection device according to any one of claims 1 to 5, comprising a plurality of switching devices, each of the switching devices comprising an embedded controller, and the embedded controller comprising a plurality of configuration registers; a management device configured to receive a user configuration instruction, determine topology configuration information and / or interface configuration information based on the user configuration instruction, determine a configuration parameter group for each of the switching devices based on the topology configuration information and / or the interface configuration information, and write each configuration parameter value in each of the configuration parameter groups into each of the configuration registers in the embedded controller of the switching device; A power supply device is configured to supply power to the interconnection device and the management device.

7. The high-performance switching device according to claim 6, characterized in that: The management device includes a plurality of management controllers, and the plurality of management controllers are connected to the interconnection device; The current interaction controller is configured to perform data interaction with the interconnected device; the current interaction controller is any one of the plurality of management controllers that is in a normal working state.

8. The high-performance switching device according to claim 7, characterized in that: The high-performance switching device also includes a complex programmable logic device; Each of the management controllers is further configured to send a heartbeat signal to the complex programmable logic device at a preset period; The complex programmable logic device is configured to determine whether there is a faulty controller in an abnormal working state among the multiple management controllers. If it is determined that there is a faulty controller in an abnormal working state among the multiple management controllers, it is determined whether the faulty controller is the current interactive controller. If it is determined that the faulty controller is the current interactive controller, the faulty controller is controlled to stop data interaction with the interconnection device, and any one of the multiple management controllers in the normal working state is selected as the new current interactive controller; the abnormal working state is a working state in which the heartbeat signal is not sent according to the preset period.

9. The high-performance switching device according to claim 8, characterized in that: The multiple management controllers include a master management controller and a slave management controller, and the slave management controller is a management controller other than the master management controller among the multiple management controllers; The complex programmable logic device is also configured to control the current interactive controller to stop data interaction with the interconnection device when it is determined that the current interactive controller is the slave management controller and it is determined that there is a master management controller in normal working condition, and select a master management controller in normal working condition as the new current interactive controller.

10. The high-performance switching device according to claim 8, characterized in that: The high-performance switching device further includes a first switching device, wherein a plurality of first terminals of the first switching device are connected to a plurality of the management controllers in a one-to-one correspondence, a plurality of second terminals of the first switching device are connected to a plurality of the switching devices in a one-to-one correspondence, and a control terminal of the first switching device is connected to the complex programmable logic device; The complex programmable logic device is configured to determine whether there is a faulty controller in an abnormal working state among the multiple management controllers; if it is determined that there is a faulty controller in an abnormal working state among the multiple management controllers, determine whether the faulty controller is a current interactive controller; if it is determined that the faulty controller is the current interactive controller, generate a first control instruction corresponding to the faulty controller; and select any one of the multiple management controllers in the normal working state as a new current interactive controller, and generate a second control instruction corresponding to the new current interactive controller. The first switching device is configured to, upon receiving the first control instruction, determine the first target end based on the first control instruction, and control the communication link between the management controller connected to the first target end and each of the switching devices to be disconnected; and upon receiving the second control instruction, determine the second target end based on the second control instruction, and control the communication link between the management controller connected to the second target end and each of the switching devices to be connected.

11. The high-performance switching device according to claim 10, characterized in that: The high-performance switching device further includes: A first port expansion device, wherein a first end of the first port expansion device is connected to a second end of the first switching device, and a plurality of second ends of the first port expansion device are connected to a plurality of the switching devices in a one-to-one correspondence.

12. The high-performance switching device according to claim 8, characterized in that: The process of data interaction with the interconnected device includes: Writing each configuration parameter in each configuration parameter group into each configuration register in the embedded controller of the switching device; Acquiring status parameters of each of the switching devices and device parameters of external devices connected to each of the switching devices; The high-performance switching device further includes: prompting device; a first baseboard controller connected to each of the management controllers, configured to determine a current topology state of the interconnection device through state parameters of each of the switching devices, and generate a current state prompt instruction based on the current topology state; The complex programmable logic device is further configured to respond to a current state prompt instruction and control the prompt device to prompt current topology state information.

13. The high-performance switching device according to claim 12, characterized in that: The high-performance exchange device further includes a heat dissipation device, wherein the heat dissipation device includes at least one fan; The first baseboard controller is further configured to generate a current heat dissipation control instruction based on the current topology state and the device parameters; The complex programmable logic device is further configured to adjust the rotation speed of the fan in response to the current heat dissipation control instruction.

14. The high-performance switching device according to any one of claims 12 to 13, characterized in that: The first baseboard controller, the management controller, the complex programmable logic device and the interconnection device are connected via Ethernet; The first baseboard controller, the management controller, the complex programmable logic device and the interconnection device are connected via a universal serial bus or an integrated circuit bus.

15. A large model all-in-one machine, characterized in that: include: The high-performance switching device according to any one of claims 6 to 14; a plurality of general-purpose computing devices, wherein the plurality of general-purpose computing devices are connected to the interconnection device of the high-performance switching device; a plurality of heterogeneous computing devices, wherein the plurality of heterogeneous computing devices are connected to the interconnection device; Any two computing devices connected to the interconnection device are interconnected, and the computing devices are the general computing devices or the heterogeneous computing devices.

16. The large model all-in-one machine according to claim 15, characterized in that: The large model all-in-one machine further includes a second substrate controller and a plurality of second switching devices; The first ends of the plurality of second switching devices are connected one-to-one with the plurality of heterogeneous computing devices, the second end of each second switching device is connected with the second baseboard controller, and the third end of each second switching device is connected with the high-performance switching device.

17. The large model all-in-one machine according to claim 15, characterized in that: The large model all-in-one machine also includes multiple network cards, and the interconnection device includes multiple switching devices. Each of the switching devices is connected to at least one network card, so that the heterogeneous computing devices connected to the interconnection device can be networked with other large model all-in-one machines through the network cards.

18. The large model all-in-one machine according to claim 15, characterized in that: The interconnection device includes multiple switching devices, and the large model all-in-one machine also includes multiple network cards. The heterogeneous computing devices are connected to the switching devices through the network cards, and the heterogeneous computing devices are networked with other large model all-in-one machines through the network cards.

19. The large model all-in-one machine according to claim 15, characterized in that: The large model all-in-one machine also includes: at least one memory expansion card, configured to store data to be processed; The general computing device is configured to obtain local memory, and when the local memory is less than a preset value, write the data to be processed into the memory expansion card.

20. The large model all-in-one machine according to claim 19, characterized in that: The memory expansion card includes a third baseboard controller and a plurality of memory controllers, each of the memory controllers being mounted with at least one memory stick; The third baseboard controller is configured to receive and forward memory access instructions sent by the general-purpose computing device, obtain state parameters of each of the memory controllers, obtain target data converted into serial data, and send the target data to the general-purpose computing device; The memory controller is configured to parse the memory access data in the memory access instruction. When the memory access data is data to be written, the data to be written is converted from serial data to parallel data and then written into the corresponding memory stick mounted on itself. When the memory access data is data to be read, the target data is read from the memory stick mounted on itself according to the data to be read, and the target data is converted from parallel data to serial data and then forwarded to the third baseboard controller.

21. The large model all-in-one machine according to any one of claims 15 to 20, characterized in that: The large model all-in-one machine also includes a pooling management engine; The general computing device is further configured to initiate a request for dynamic adjustment of heterogeneous resources; The pooling management engine is configured to, upon receiving the heterogeneous resource dynamic adjustment request, determine the heterogeneous computing device to be adjusted according to the heterogeneous resource dynamic adjustment request, perform a hot removal operation on the heterogeneous computing device to be adjusted, and send a location acquisition request corresponding to the heterogeneous computing device to be adjusted to the high-performance switching device, send a reset execution instruction to the heterogeneous computing device to be adjusted based on the received physical location information, determine the heterogeneous computing device to be adjusted as the heterogeneous computing device to be allocated after receiving the reset completion instruction, and allocate the heterogeneous computing device to be allocated to the general-purpose computing device corresponding to the reallocation instruction when receiving the reallocation instruction; The heterogeneous computing device is configured to perform a reset operation upon receiving the reset execution instruction, and generate a reset completion instruction after the reset operation is successfully completed; The high-performance switching device is configured to obtain the physical location information of the heterogeneous computing device to be adjusted and return the physical location information to the pooling management engine after receiving the location acquisition request.

Images