Computing node, service node and server cabinet
By designing computing nodes in the server cabinet and using optical modules to transmit PCIe high-speed signals and switching circuits to transmit PCIe low-speed signals, the transmission loss problem caused by the increase in cable length between the general-purpose server and the GPU BOX is solved, and a higher GPU BOX is achieved Quantity and system performance while reducing power costs.
Patent Information
- Application Number
- CN202411528822.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-02-11
- Estimated Expiration
- 2044-10-30
AI Technical Summary
In the same server cabinet, the height distance between the general-purpose server and the GPU BOX increases, resulting in an increase in the length of the connection cable, which in turn increases the cable transmission loss, limiting the number and performance of the GPU BOX.
A computing node is designed, including a processor, switching circuit, compensation circuit and optical module, to transmit PCIe high-speed signals through optical modules, reduce cable length and loss, and transmit PCIe low-speed signals through switching circuits.
Effectively reduces cable length and loss, supports more GPU BOX quantity, improves system performance and efficiency, and reduces power costs.
Smart Images

Figure CN119415464A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of servers, and in particular to a computing node, a service node and a server cabinet. Background Art
[0002] The form of an artificial intelligence AI server is to place a graphics processing unit (GPU) card in a general-purpose server, or to design multiple GPU cards into a separate GPU box (BOX), which is connected to a general-purpose server through cables for business management. The GPU BOX can accommodate a large number of GPU cards or GPU modules and supports the remote memory access function of the GPU. When performing large-scale model training and inference business, a high-performance GPU BOX is often used to complete it. In addition, when a general-purpose server and a GPU BOX are deployed in the same cabinet, the two are interconnected by DAC (Direct Attach Copper Cable).
[0003] The number of GPU BOX chassis is subject to multiple restrictions such as SI (system integration), power supply, heat dissipation, and structure. According to actual evaluation, the maximum length of the copper cable is about 500mm. If the cable length is increased, the loss of the cable will cause the PCIe signal quality to deteriorate rapidly, resulting in transmission errors, bit errors, and even connection failures.
[0004] In addition, when multiple GPU BOXs are stacked up and down with general-purpose servers, the increased height increases the length of the interconnection cables and increases the loss. Due to the natural disadvantage, each general-purpose server cannot mount more GPU BOXs. In the same cabinet, if you want to support more GPU resources, you can only increase the number of general-purpose servers, resulting in a waste of resources. Summary of the invention
[0005] In view of this, the present invention provides a computing node, a business node and a server cabinet to solve the problem that when a general server is connected to a GPU BOX in the same server cabinet, as the number of servers increases, due to the height structure limitation of the chassis, the height distance between the general server and the GPU BOX becomes larger and larger, which increases the length of the connecting cable and further increases the cable transmission loss.
[0006] In a first aspect, the present invention provides a computing node, comprising: a processor, N switch circuits, N compensation circuits, and N optical modules, where N≥1 and N is a positive integer;
[0007] The processor is connected to N switch circuits and N charging circuits respectively, and the N charging circuits are also connected to N optical modules; the N switch circuits are also connected to an external backplane, and a connector is installed on the backplane for connecting at least one service node, and each service node includes an optical module and an internal control circuit;
[0008] A processor, configured to generate a PCIe high-speed signal and a PCIe low-speed signal, and transmit the PCIe high-speed signal to the N optical modules through the N compensation circuits, and transmit the PCIe low-speed signal to the backplane through the N switch circuits;
[0009] N optical modules, used to receive and convert the PCIe high-speed signal into an optical signal, and transmit the optical signal to at least one optical module of the at least one service node;
[0010] A backplane, used for transmitting the PCIe low-speed signal to the internal control circuit of the at least one service node through a connector.
[0011] In combination with the first aspect, in a possible implementation manner, the processor includes N transmitting modules, each transmitting module includes a detection module; each of the compensation circuits includes: a coupling capacitor and a compensation resistor, which are used to form a compensation path to charge the coupling capacitor;
[0012] The N detection modules are connected to the N optical modules via coupling capacitors and compensation resistors;
[0013] Each of the transmitting modules is used for the processor to generate at least one pulse level using the detection module before generating the PCIe high-speed signal and the PCIe low-speed signal, and send the at least one pulse level to the coupling capacitor and the compensation resistor;
[0014] The processor is further configured to detect whether all of the N optical modules are connected to a service node according to the at least one pulse level;
[0015] If it is detected that at least one pulse level disappears within a preset time, it is determined that at least one optical module transmitting at least one pulse level is connected to a service node, and the steps of generating the PCIe high-speed signal and the PCIe low-speed signal are performed;
[0016] If it is detected that at least one pulse level does not disappear within the preset time, it is determined that no service node is connected to at least one optical module transmitting the at least one pulse level.
[0017] In combination with the first aspect, in another possible implementation manner, the PCIe low-speed signal includes: a sideband signal and an ID signal; each of the internal control circuits includes: a service module and an ID circuit;
[0018] The processor is specifically used to send the sideband signal and the ID signal to the connector of the backplane when controlling the N switch circuits to be turned on;
[0019] The connector of the backplane is specifically used to transmit the sideband signal to the service module and transmit the ID signal to the ID circuit.
[0020] In combination with the first aspect, in another possible implementation, the processor is also used to determine at least one received service node based on the at least one ID signal after sending at least one ID signal, establish a binding relationship between each ID signal and at least one service node, and use the binding relationship to control the conduction of some or all of the N switching circuits.
[0021] In combination with the first aspect, in yet another possible implementation, the computing node further includes at least one clock circuit, and each of the clock circuits is connected to one of the N optical modules;
[0022] Each of the clock circuits is used to generate a clock signal, and send the clock signal to the N optical modules synchronously according to the PCIe high-speed signal clock;
[0023] The N optical modules are also used to receive a clock signal, and synchronously convert the clock signal and the PCIe low-speed signal into the optical signal, and transmit it to at least one optical module of the at least one service node.
[0024] In a second aspect, the present invention further provides a service node, comprising: an internal control circuit and a second optical module; wherein the internal control circuit is connected to the second optical module;
[0025] The internal control circuit is also connected to an external backplane, and a connector is installed on the backplane for connecting a computing node, wherein the computing node includes a processor and a first optical module, and the first optical module is connected to the second optical module;
[0026] The internal control circuit is used to receive the PCIe low-speed signal transmitted by the processor through the connector of the backplane;
[0027] The second optical module is used to receive the optical signal transmitted by the processor through the first optical module, convert the optical signal into a PCIe high-speed signal, and send it to the internal control circuit.
[0028] In conjunction with the second aspect, in a possible implementation, the PCIe low-speed signal includes: a sideband signal and an ID signal;
[0029] The internal control circuit includes: a service module and an ID circuit, and the service module is connected to the second optical module;
[0030] The ID circuit is used to receive an ID signal from the connector of the backplane;
[0031] The service module is used to receive the sideband signal from the connector of the backplane, and to receive the PCIe high-speed signal transmitted from the second optical module.
[0032] In combination with the second aspect, in another possible implementation, the second optical module is further used to convert the optical signal into a clock signal, and send the clock signal and the PCIe high-speed signal to the service module.
[0033] In a third aspect, the present invention further provides a server cabinet, the cabinet comprising: a backplane, a plurality of hierarchical structures and a power supply copper busbar; wherein the plurality of hierarchical structures are respectively connected to the backplane and the power supply copper busbar;
[0034] At least one computing node and at least one service node are installed on the multiple hierarchical structures, and the at least one computing node is connected to at least one service node; a connector is installed on the backplane for connecting at least one computing node and at least one computing node;
[0035] Among them, each of the computing nodes is the computing node described in the first aspect or any implementation scheme of the first aspect; each of the service nodes is the service node described in the second aspect or any implementation scheme of the second aspect.
[0036] Optionally, the service node is a graphics processing unit (GPU) box or a memory pool; the GPU box includes at least one GPU card.
[0037] In a fourth aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method steps of the computing node of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0038] In addition, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method steps of the computing node of the above-mentioned first aspect or any corresponding embodiment thereof.
[0039] The present invention provides a computing node, a service node and a server cabinet, so that the cable length between the computing node and the service node, such as GPU BOX, no longer becomes a bottleneck. By designing a switch circuit, a compensation circuit and an optical module, a low-speed link is established between the processor and the connector of the switch circuit and the backplane. At the same time, a high-speed link with other service nodes is established through the compensation circuit and the optical module, so that the PCIe high-speed signal generated by the processor can be transmitted through the optical signal converted by the optical module, and other PCIe low-speed signals can be transmitted through the low-speed link. Compared with the use of traditional copper cable transmission, this solution uses optical modules to transmit high-speed signals to save the number of PSU power supplies, reduce costs, and also reduce the loss of copper cable transmission, that is, when using copper cable connection, the PCIe signal has an increasing loss as the cable length increases, which will cause the signal to be unable to be transmitted correctly. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 is a structural schematic diagram of a server cabinet according to an embodiment of the present invention;
[0042] Figure 2 is a schematic diagram of the internal structure of a server cabinet according to an embodiment of the present invention;
[0043] Figure 3 is a flow chart of a signal transmission method according to an embodiment of the present invention;
[0044] Figure 4 is a circuit diagram of a computing node and a service node according to an embodiment of the present invention;
[0045] Figure 5 It is a schematic diagram of the hardware structure of a computing node according to an embodiment of the present invention. DETAILED DESCRIPTION
[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0047] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0048] Furthermore, the terms “first” and “second” are used for descriptive purposes only and should not be understood as indicating or implying relative importance.
[0049] The technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0050] The technical solution of the present invention is applied to the field of server technology, especially to artificial intelligence AI server cabinets. In an AI server cabinet, a general server and at least one GPU box (BOX) are generally included. As the number of general servers increases, the height distance between the general server and the GPU BOX increases due to the height structure limitation of the cabinet chassis. This increases the length of the copper cable connecting the general server and each GPU BOX.
[0051] In order to solve the problem that when using copper cables for connection, the loss of PCIe signals increases with the increase of cable length, and even causes the signal to fail to be transmitted correctly. An embodiment of the present invention provides a high-density AI server cabinet with all-fiber interconnection, which not only solves the problem of increased cable length and loss between general servers and GPU BOX. At the same time, it also supports fiber optic interconnection of CXL (Compute Express Link) memory pools to increase memory capacity. It also supports centralized power supply, saving the number of PSU (Power Supply Unit) power supplies in the cabinet and reducing costs.
[0052] See also Figure 1 , is a schematic diagram of the structure of a server cabinet according to an embodiment of the present invention, namely the aforementioned high-density AI server cabinet, which includes multiple hierarchical structures. For example, in a 42U cabinet, up to 8 4U GPU BOX chassis, 2 2U general servers, 1 2U CXL memory pool and 1 3U centralized power supply module (power shelf) can be placed.
[0053] Among them, U is the abbreviation of Unit, which is used to represent the height unit of the server or related equipment chassis. Based on the EIA-310 standard, the height of each U is usually about 4.445 cm (or 1.75 inches). This standard ensures that servers and other equipment can fit into the size of standard racks. For example, a 4U GPU BOX chassis means that the height of the GPU BOX chassis is 4 U, which is about 17.78 cm (or 7 inches).
[0054] like Figure 1 As shown in the figure, each general-purpose server is responsible for managing a total of 4 GPU BOXs, and the CXL memory pool is responsible for supporting the memory expansion of two general-purpose servers. The general-purpose server is interconnected with the GPU BOX and the CXL memory pool using PCIe signals.
[0055] It should be understood that other server cabinets may include more or fewer levels for carrying multiple general servers, GPU BOX, CXL memory pool, etc. This embodiment does not limit the number of these devices / apparatuses.
[0056] The following describes the internal structure of the AI server cabinet, the connection relationship between each layer, and the communication process.
[0057] See also Figure 2 FIG. 1 is a schematic diagram of the internal structure of a server cabinet according to an embodiment of the present invention, wherein the server cabinet includes at least one general-purpose server, or a computing node. The computing node includes: a processor, N switch circuits, N compensation circuits, and N optical modules, where N≥1 and N is a positive integer.
[0058] The processor is connected to N switch circuits and N charging circuits respectively, and the N charging circuits are also connected to N optical modules. Figure 2 The system includes a first switch circuit, a second switch circuit, ..., an Nth switch circuit. The processor is connected to the N switch circuits respectively. The N switch circuits are also connected to an external backplane, on which a connector is installed for connecting at least one service node, and each service node includes an optical module and an internal control circuit.
[0059] Optionally, the processor may be a central processing unit CPU.
[0060] Optionally, the above-mentioned 1 to N switch circuits may be a 245 buffer chip or a chip circuit.
[0061] In addition, the internal control circuit of each service node includes: a service module and an ID circuit. The service module and the ID circuit may be connected or not connected, and this embodiment does not limit this. The service module is used to process actual services, such as video data stream transmission, etc. The ID circuit is used to identify the current service node and provide a basis for the computing node to identify and establish the binding relationship between each service node.
[0062] Optionally, in this embodiment, the service node may be a GPU BOX or a memory pool, such as an XCL memory pool.
[0063] In addition, the processor is also connected to the optical module through a compensation circuit. If the processor needs to be connected to N optical modules, N compensation circuits are required, and each compensation circuit is used to connect to one optical module.
[0064] based on Figure 2 The internal structure of the cabinet shown in the figure, this embodiment also provides a signal transmission method, such as Figure 3 As shown, the method includes:
[0065] Step S101: the processor is used to generate a PCIe high-speed signal and a PCIe low-speed signal.
[0066] Among them, the PCIe low-speed signal may include a sideband signal and an ID signal; further, the sideband signal is some control signals, such as WAKE (wake-up), RESET (reset), SMBUS (System Management Bus) signal, etc. And because the rate of these key signals is low, they are not suitable for transmission in the optical fiber together with the high-speed signal of PCIe data, so they are separated and transmitted separately.
[0067] PCIe high-speed signals are used to transmit high-speed video data streams.
[0068] Step S102: the processor transmits the PCIe high-speed signal to N optical modules through N compensation circuits.
[0069] Each of the N optical modules is used to receive and convert a PCIe high-speed signal into an optical signal, and transmit the optical signal to at least one optical module of at least one service node.
[0070] In this embodiment, each optical module is a photoelectric conversion device. For example, CDFP (Cabled Distributed Fanout Pluggable) is a standard type of optical module.
[0071] Step S103: the processor transmits the PCIe low-speed signal to the backplane through N switch circuits.
[0072] The backplane is used to transmit the PCIe low-speed signal to the internal control circuit of at least one service node through the connector. The number of connectors on the backplane can be set according to the number of service nodes.
[0073] Specifically, Figure 2 As shown, the processor is specifically used to send the sideband signal and the ID signal to the connector of the backplane when controlling N switch circuits to be turned on. The connector of the backplane is specifically used to transmit the sideband signal to the service module and transmit the ID signal to the ID circuit.
[0074] The above-mentioned ID circuit includes pull-up and pull-down resistors, such as a first resistor R1 and a second resistor R2, and R1 and R2 are connected in parallel, and R2 is grounded. When the ID signal is transmitted to the ID circuit, a level signal value is generated according to the pull-up and pull-down resistors inside the service node, and the service node ID of the receiving end is sensed by the computing node, so that each computing node establishes a corresponding relationship between the ID signal and each service node, such as GPU BOX. This corresponding relationship can be used to control 1 to N switch circuits to be turned on or off in the subsequent processor.
[0075] Optional, in Figure 2 In the circuit, the service node side may also include a BMC (Baseboard Management Controller). The PCIe low-speed signal generated by the processor in the computing node also includes an IPMB out-of-band management signal (I2C), and the processor transmits the I2C signal to the BMC of the service node using the connector of the backplane.
[0076] In this embodiment, the processor includes N transmitting modules, each transmitting module includes a detection module (Detect module). In addition, each compensation circuit includes: a coupling capacitor Cn and a compensation resistor Rc, which are used to form a compensation path to charge the coupling capacitor Cn. The N detection modules are connected to the N optical modules through the coupling capacitor and the compensation resistor.
[0077] Because the PCIe bus needs to undergo a link training process before formally transmitting data, link training is completely implemented by the physical layer and consists of three stages: Detect, Polling, and Configuration. In the Detect stage, the transmitter needs to send a detection pulse and determine whether the other end is connected based on the time when the pulse level drops. If the detection pulse disappears within the time when the pulse level drops, it is determined that the other end is connected to a circuit device; otherwise, there is no connection.
[0078] This detection mechanism is an analog circuit. When using optical fiber interconnection, it is impossible to transmit the detection pulse. Therefore, it is necessary to train the PCIe link in advance. The training process is as follows: Figure 2As shown, a virtual resistor network is added. A pulse level current path composed of a resistor network is added to the board circuit before the photoelectric conversion device (ie, the optical module).
[0079] Specifically, each transmitting module is used to generate at least one pulse level using the detection module before the processor generates the PCIe high-speed signal and the PCIe low-speed signal, and send the at least one pulse level to the coupling capacitor and the compensation resistor.
[0080] The processor is also used to detect whether N optical modules are all connected to business nodes based on at least one pulse level; if it is detected that at least one pulse level disappears within a preset time, it is determined that at least one optical module transmitting at least one pulse level is connected to a business node, and the steps of generating a PCIe high-speed signal and a PCIe low-speed signal are executed; if it is detected that at least one pulse level does not disappear within the preset time, it is determined that at least one optical module transmitting at least one pulse level is not connected to a business node.
[0081] The processor is also used to determine at least one received service node based on at least one ID signal after sending at least one ID signal, establish a binding relationship between each ID signal and at least one service node, and use the binding relationship to control the conduction of some or all of the N switch circuits.
[0082] When the transmitter is preparing to perform link training, the pulse level sent by the Detect detection module forms a charging path for TX_Cn after passing through the TX_Cn coupling capacitor and the compensation resistor Rc network. Due to the charging of the capacitor, the amplitude of the pulse level will be greatly reduced.
[0083] For the detection module, it is considered that the other end is connected and the next link training can be carried out. Since the training is initiated by the host HOST as the transmitter when the PCIe link is initialized, the analog resistance circuit is only set at the TX end of the transmitter module of the general server (host HOST). The detection module (Detect module) sends a detection level and observes whether the detection level disappears. If the detection level disappears in a very short time, it is determined that other circuits are connected to the photoelectric conversion device.
[0084] In a specific embodiment, Figure 4 FIG. 1 is a circuit diagram of a computing node and a service node according to an embodiment of the present invention. The circuit diagram includes a first computing node and a first service node.
[0085] The first computing node may be a general server, and includes a transmitting module TX, a first compensation circuit and a first optical module, the first optical module is connected to an optical module of the first service node, such as a second optical module, and the second optical module is connected to a receiving module RX.
[0086] The transmitting module TX in the first computing node is connected to the first optical module through the first compensation circuit. The first optical module converts the PCIe high-speed signal from the first compensation circuit into an optical signal and transmits it to the second optical module through the optical fiber.
[0087] In another possible implementation, the computing node also includes at least one clock circuit (such as ClockBuffer), each clock circuit is connected to one of the N optical modules; each clock circuit is used to generate a clock signal and send the clock signal to the N optical modules in synchronization with the PCIe high-speed signal clock.
[0088] The N optical modules are also used to receive a clock signal, and synchronously convert the clock signal and the PCIe low-speed signal into an optical signal, and transmit it to at least one optical module of at least one service node. For example, in this embodiment, the first optical module receives a PCIe clock signal sent from the Clock Buffer, converts the clock signal and the PCIe low-speed signal into an optical signal, and then transmits the optical signal to the second optical module via an optical fiber. The first optical module and the second optical module are connected via an optical fiber cable.
[0089] Optical modules are set on the computing node at the sending end and the business node at the receiving end. These optical modules can be used to converge the PCIe high-speed signal and clock signal generated by the CPU and transmit them to the GPU card or CXL memory pool, thereby realizing the complete transmission of the PCIe full signal.
[0090] The computing node provided in this embodiment has a low-speed signal dedicated backplane designed on the left side of the rear of the entire cabinet, which is used to transmit PCIe low-speed signals and other sideband signals from the computing node. First, the PCIe low-speed signal passes through at least one switch circuit and is transmitted to the low-speed signal dedicated backplane through the connector of the general server motherboard and the low-speed signal dedicated backplane.
[0091] Through the backplane, it is distributed to each business node, such as GPU BOX and CXL memory pool. In addition, the 100M reference clock of PCIe can use the same channel as the PCIe high-speed signal for optical transmission. When each processor transmits data to a GPU BOX or CXL memory pool, the computing node can generate corresponding low-speed signals, including ID signals, and transmit the ID signals at low speed through the backplane connector to the ID circuit of the business node.
[0092] When the CPU intends to communicate with a GPU BOX or CXL memory pool, it first enables a group of low-speed signals corresponding to the GPUBOX with the corresponding GPU ID. Then, the switch circuit corresponding to the group of low-speed signals is turned on, and the low-speed signal path from the computing node CPU to the corresponding GPU BOX is opened. Then, the CPU starts normal communication, and both high-speed and low-speed signals are automatically used and allocated by the CPU.
[0093] Corresponding to the aforementioned computing node, this embodiment further provides a service node, including: an internal control circuit and a second optical module; wherein the internal control circuit is connected to the second optical module. The internal control circuit is also connected to an external backplane, and a connector is installed on the backplane for connecting the computing node, and the computing node includes a processor and a first optical module, and the first optical module is connected to the second optical module.
[0094] The internal control circuit is used to receive the PCIe low-speed signal transmitted by the processor through the connector of the backplane. The second optical module is used to receive the optical signal transmitted by the processor through the first optical module, and convert the optical signal into a PCIe high-speed signal and send it to the internal control circuit.
[0095] Furthermore, the sideband signal includes: a control signal and an ID signal. The internal control circuit includes: a service module and an ID circuit, the service module is connected to the second optical module; the ID circuit is used to receive an ID signal from a connector of the backplane; the service module is used to receive a control signal from the connector of the backplane, and receive a PCIe high-speed signal transmitted from the second optical module.
[0096] In addition, in any service node, the second optical module is also used to convert the optical signal into a clock signal, and send the clock signal and the PCIe high-speed signal to the service module.
[0097] Any of the above service nodes includes a GPU box and a CXL memory pool, and the GPU box includes at least one GPU card.
[0098] To expand the memory capacity, a 2U CXL memory pool is set up in the server cabinet. The CXL memory pool and the general server are interconnected by PCIe bus, and the memory of the general server is expanded by PCIe optical interconnection.
[0099] Optionally, in each of the above nodes, the optical module can use a CDFP optical module, which meets the OIF CEI28GVSR specification, supports optical transmission, and the bandwidth can support up to 16x25G, which can perfectly support the PCIe link with x16 bandwidth. The low-speed part also uses a dedicated backplane for low-speed signals to transmit the PCIe low-speed signal between the CXL memory pool and the general server.
[0100] This embodiment also provides a server cabinet, the cabinet structure can be as described above Figure 1 As shown, the cabinet includes: a backplane, multiple hierarchical structures and a power supply copper busbar; wherein the multiple hierarchical structures are respectively connected to the backplane and the power supply copper busbar.
[0101] Among them, at least one computing node and at least one business node are installed on the multiple hierarchical structures, and at least one computing node is connected to at least one business node; a connector is installed on the backplane for connecting at least one computing node and at least one computing node.
[0102] Specifically, each computing node is the aforementioned Figure 2 The computing node shown. Optionally, the computing node may be a server, such as a general-purpose server.
[0103] Each business node is the aforementioned Figure 2 Optionally, the service node is a graphics processing unit (GPU) box or a CXL memory pool. The GPU box includes at least one GPU card.
[0104] The connection relationship and functions between the various nodes are described in the previous embodiment and will not be repeated here.
[0105] The server cabinet provided in this embodiment adopts optical fiber interconnection, and the GPU BOX connected to each general server in the cabinet eliminates the limitation that the increase in the number of GPU BOXes leads to excessive signal loss due to the increase in cable length, so the number of GPU BOXes can be greatly increased.
[0106] In addition, when the number of GPU BOX increases, if the original method of using an independent PSU to power each GPU BOX is adopted, each GPU BOX needs to achieve N+N PSU power redundancy, resulting in a waste of PSU and a significant increase in cost. To solve this problem, the server cabinet provided in this embodiment adopts a centralized power supply method, which is implemented by a centralized power supply module (power shelf) equipped with multiple PSUs. The AC power supply is input from the outside to the centralized power supply module, and DC transmission is achieved through the copper busbar on the rear side of the cabinet on the centralized power supply module. General servers, GPU BOXs and CXL memory pools also draw power through the copper busbars on the rear sides of their respective chassis. After adopting centralized power supply, the number of PSUs inside the centralized power supply module can adopt N+1 redundancy, which greatly saves PSU costs and is more efficient.
[0107] The server cabinet provided in this embodiment solves the problem that when using traditional copper cables for connection, the loss of PCIe high-speed signals increases with the increase of cable length, which eventually leads to the problem that the signals cannot be transmitted correctly. The embodiment of the present invention provides a high-density AI server cabinet with all-optical interconnection, so that the cable length between the general server and the GPU BOX is no longer a bottleneck problem. At the same time, it supports CXL memory pool optical interconnection to expand the memory capacity.
[0108] In addition, the server cabinet also realizes centralized power supply by installing a centralized power supply module, saving the number of PSU power supplies in the cabinet and reducing costs.
[0109] It should be noted that the server cabinet provided in this embodiment can be used on all server products.
[0110] In the present embodiment, a signal transmission device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.
[0111] This embodiment provides a signal sending device, including a generating module, a first sending module and a second sending module.
[0112] The generating module is used to generate PCIe high-speed signals and PCIe low-speed signals. The first transmitting module is used to transmit the PCIe high-speed signals to N optical modules through N compensation circuits. The second transmitting module is used to transmit the PCIe low-speed signals to the backplane through N switch circuits.
[0113] Furthermore, the first sending module may be as follows Figure 4 The second transmitting module may be as shown in FIG. Figure 2 The switching circuit shown.
[0114] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0115] The signal sending device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and a memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0116] The embodiment of the present invention also provides a computer device having the above Figure 2 and Figure 4 The circuit structure shown.
[0117] See also Figure 5 , is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common motherboard or in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface).
[0118] In some optional embodiments, if desired, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.
[0119] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0120] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the signal sending method shown in the above embodiment.
[0121] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0122] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0123] The computer device also includes an input device and an output device. The processor 10, the memory 20, and the input device / output device can be connected via a bus or other means. The input device can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device may include a display device, an auxiliary lighting device (e.g., an LED), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display.
[0124] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.
[0125] An embodiment of the present invention also provides a computer-readable storage medium, and the above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented by downloading through a network and originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware.
[0126] The storage medium may be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid state drive, etc.; further, the storage medium may also include a combination of the above-mentioned types of memories. It is understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, processor or hardware, the method shown in the above embodiment is implemented.
[0127] Embodiments of the present application may also provide a computer program product, including computer program instructions, which, when executed by a processor, cause the processor to perform the steps in the above method. Wherein, the computer program product may be written in any combination of one or more programming languages to perform program codes for performing the operations of the disclosed embodiments, wherein the programming languages include object-oriented programming languages, such as Java, C++, etc., and also include conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device, partially on a remote computing device, or entirely on a remote computing device or server.
[0128] The above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the embodiments of the present invention are described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A computing node, characterized in that: include: A processor, N switch circuits, N compensation circuits and N optical modules, where N ≥ 1 and N is a positive integer; The processor is connected to N switch circuits and N charging circuits respectively, and the N charging circuits are also connected to N optical modules; The N switch circuits are also connected to an external backplane, on which a connector is installed for connecting at least one service node, each of which includes an optical module and an internal control circuit; The processor is used to generate a PCIe high-speed signal and a PCIe low-speed signal, and transmit the PCIe high-speed signal to the N optical modules through the N compensation circuits, and transmit the PCIe low-speed signal to the backplane through the N switch circuits; The N optical modules are used to receive and convert the PCIe high-speed signal into an optical signal, and transmit the optical signal to at least one optical module of the at least one service node; The backplane is used to transmit the PCIe low-speed signal to the internal control circuit of the at least one service node through a connector.
2. The computing node according to claim 1, characterized in that: The processor includes N transmitting modules, each of which includes a detection module; Each of the compensation circuits comprises: a coupling capacitor and a compensation resistor, which are used to form a compensation path and charge the coupling capacitor; The N detection modules are connected to the N optical modules through the coupling capacitors and the compensation resistors; Each of the transmitting modules is used for the processor to generate at least one pulse level using the detection module before generating the PCIe high-speed signal and the PCIe low-speed signal, and send the at least one pulse level to the coupling capacitor and the compensation resistor; The processor is further configured to detect whether all of the N optical modules are connected to a service node according to the at least one pulse level; If it is detected that the at least one pulse level disappears within a preset time, it is determined that a service node is connected to at least one optical module transmitting the at least one pulse level, and the steps of generating the PCIe high-speed signal and the PCIe low-speed signal are performed; If it is detected that the at least one pulse level does not disappear within the preset time, it is determined that no service node is connected to at least one optical module transmitting the at least one pulse level.
3. The computing node according to claim 1, characterized in that: The PCIe low-speed signal includes: a sideband signal and an ID signal; each of the internal control circuits includes: a service module and an ID circuit; The processor is specifically configured to send the sideband signal and the ID signal to the connector of the backplane when controlling the N switch circuits to be turned on; The connector of the backplane is specifically used to transmit the sideband signal to the service module and transmit the ID signal to the ID circuit.
4. The computing node according to claim 3, characterized in that: The processor is also used to determine at least one received service node based on the at least one ID signal after sending at least one ID signal, establish a binding relationship between each ID signal and at least one service node, and use the binding relationship to control the conduction of part or all of the N switch circuits.
5. The computing node according to any one of claims 1 to 4, characterized in that: The computing node further includes at least one clock circuit, each of the clock circuits being connected to one of the N optical modules; Each of the clock circuits is used to generate a clock signal, and send the clock signal to the N optical modules synchronously according to the PCIe high-speed signal clock; The N optical modules are also used to receive the clock signal, and synchronously convert the clock signal and the PCIe low-speed signal into the optical signal, and transmit it to at least one optical module of the at least one service node.
6. A service node, characterized in that: include: An internal control circuit and a second optical module; wherein the internal control circuit is connected to the second optical module; The internal control circuit is also connected to an external backplane, and a connector is installed on the backplane for connecting a computing node, wherein the computing node includes a processor and a first optical module, and the first optical module is connected to the second optical module; The internal control circuit is used to receive the PCIe low-speed signal transmitted by the processor through the connector of the backplane; The second optical module is used to receive the optical signal transmitted by the processor through the first optical module, convert the optical signal into a PCIe high-speed signal, and send it to the internal control circuit.
7. The service node according to claim 6, characterized in that: The PCIe low-speed signal includes: a sideband signal and an ID signal; The internal control circuit includes: a service module and an ID circuit, and the service module is connected to the second optical module; The ID circuit is used to receive the ID signal from the connector of the backplane; The service module is used to receive the sideband signal from the connector of the backplane, and to receive the PCIe high-speed signal transmitted from the second optical module.
8. The service node according to claim 7, characterized in that: The second optical module is further used to convert the optical signal into a clock signal, and send the clock signal and the PCIe high-speed signal to the service module.
9. A server cabinet, characterized in that: The cabinet comprises: a backplane, a plurality of hierarchical structures and a power supply copper busbar; wherein the plurality of hierarchical structures are respectively connected to the backplane and the power supply copper busbar; At least one computing node and at least one service node are installed on the multiple hierarchical structures, and the at least one computing node is connected to the at least one service node; The backplane is provided with a connector for connecting at least one computing node and at least one computing node; Each of the computing nodes is a computing node as claimed in any one of claims 1 to 5; Each of the service nodes is a service node as claimed in any one of claims 6 to 8.
10. The server cabinet according to claim 9, characterized in that: The service node is a graphics processing unit GPU box or a memory pool; The GPU box includes at least one GPU card.
Citation Information
Patent Citations
System with external expansion GPU (graphics processing unit) cards
CN103605404A
Voltage compensation system, method of compensating voltage drop and multi-node system
CN111030434A
RECONFIGURABLE PERIPHERAL COMPONENT INTERCONNECT EXPRESS (PCIe) DATA PATH TRANSPORT TO REMOTE COMPUTING ASSETS
US20230144056A1
Multi-host networking systems and methods
US20230350833A1