Server, fault recovery method, computing node equipment and service switch

Abstract

The invention provides a server, a fault recovery method, computing node equipment and a service switch, the server comprises the computing node equipment, the service switch and a cable bridge, and the computing node equipment comprises a first controller, a plurality of graphics processors and a plurality of first connectors; the service switch comprises a second controller, a second connector and an Ethernet switching module; the first controller is connected with the graphics processor through a first communication bus, and the first controller is connected with the second controller through a second communication bus; the graphics processor is connected with the matched first connector through the first connecting channel; the Ethernet switching module is connected with the second controller through a first management bus, and the Ethernet switching module is connected with the second connector through a second connection channel; the first connector and the second connector are interconnected through a cable bridge. The problem that the fault recovery of the Ethernet link between the graphics processor and the service switch is difficult is solved.

Description

Technical Field

[0001] The present application relates to the technical field of servers, and in particular to a server, a fault recovery method, a computing node device, and a service switch. Background Art

[0002] With the development of AI cluster technology, upper-layer services require that the GPUs in the AI ​​cluster be interconnected, and the interconnection bandwidth requirements are also increasing. In order to increase the interconnection bandwidth rate, Ethernet switches are widely used to achieve full interconnection of GPUs in servers.

[0003] As switches become more mature and widely used in servers, it is particularly important to implement the channel fault recovery function between Ethernet switches and graphics processors in the overall system design. Therefore, a solution that can quickly locate faults and perform recovery is urgently needed to improve the reliability and competitiveness of the entire server. Summary of the invention

[0004] Based on this, it is necessary to provide a server, a fault recovery method, a computing node device and a business switch to address the above technical problems.

[0005] In a first aspect, the present application provides a server, the server comprising a computing node device, a service switch and a cable tray; The computing node device includes a first controller, a plurality of graphics processors, and a plurality of first connectors; The service switch includes a second controller, a second connector, and an Ethernet switching module; The first controller is connected to the graphics processor via a first communication bus, and the first controller is connected to the second controller via a second communication bus; The graphics processor is connected to a matching first connector through a first connection channel; The Ethernet switch module is connected to the second controller via the first management bus, and the Ethernet switch module is connected to the second connector via the second connection channel; The first connector and the second connector are interconnected via a cable tray.

[0006] In some embodiments, the computing node device includes a first storage medium; The first storage medium is used to store the fault recovery firmware; The first controller is connected to the first storage medium through the first interface channel to read and execute the fault recovery firmware.

[0007] In some embodiments, the service switch includes a second storage medium; The second storage medium is used to store the fault recovery firmware; The second controller is connected to the second storage medium through the second interface channel to read and execute the fault recovery firmware.

[0008] In some embodiments, the computing node device further includes a physical layer enhancement device; The graphics processor is connected to a matching first connector through a first connection channel, including: The graphics processor is connected to the input end of the physical layer enhancement device through the first connection channel, and the output end of the physical layer timer is connected to the first connector through the first connection channel.

[0009] In some embodiments, the computing node device further includes a multiplexer, and there is a one-to-one correspondence between the multiplexer and the graphics processor; The graphics processor is connected to the first input channel of the multiplexer through the second management bus; The output end of the multiplexer is connected to the physical layer enhancement device through the third management bus.

[0010] In some embodiments, the first controller accesses the second input channels of the plurality of multiplexers through a fourth management bus.

[0011] In some embodiments, the first controller accesses a designated pin of the multiplexer through a designated channel to transmit a selection signal to the multiplexer; The multiplexer determines whether the data signal outputted from the output end comes from the first input channel or the second input channel based on the selection signal.

[0012] In some embodiments, the computing node device further includes a first microprocessor, an expansion switch module, and a third storage medium; The first microprocessor is connected to the expansion switch module via an expansion bus; The expansion switch module is connected to the graphics processor via the expansion bus; The microprocessor is connected to the third storage medium through a third interface channel to obtain the system firmware stored in the third storage medium.

[0013] In some embodiments, the service switch includes a second microprocessor and a fourth storage medium; The second microprocessor is connected to the fourth storage medium through a fourth interface channel to obtain the second microprocessor firmware and the operating system stored in the fourth storage medium; The second microprocessor is connected to the Ethernet switching module through the expansion bus.

[0014] In some embodiments, the server includes a management switch; The computing node device includes a first Ethernet interface, the first controller is connected to the first Ethernet interface through a first channel, and the first Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable; The service switch includes a second Ethernet interface, the second controller is connected to the second Ethernet interface through a second channel, and the second Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable.

[0015] In some embodiments, the server includes a system maintenance terminal; The system maintenance end is connected to any idle Ethernet interface in the management switch through a network cable; The system maintenance end stores a mapping relationship table between the physical ports of the graphics processor and the physical ports of the Ethernet module.

[0016] In a second aspect, the present application provides a fault recovery method, which is applied to the server disclosed in the second aspect, and the method includes: When it is detected that a first fault port exists in a target graphics processor in a target computing node device, a first controller is triggered to perform a first recovery operation, the first recovery operation comprising: Sending a fault recovery instruction to the target graphics processor via the first communication bus to perform fault recovery on the first fault port; First fault information is generated and sent to the second controller via the second communication bus to trigger the second controller to perform a second recovery operation, wherein the first fault information includes a target computing node device node number, a target graphics processor serial number, and a first fault port number.

[0017] In some embodiments, the second recovery operation includes: Determine a first target port according to the first fault information and the mapping relationship table; A fault recovery instruction is sent to the first target port via the first management bus to perform fault recovery on the first target port.

[0018] In some embodiments, the method comprises: When it is detected that the Ethernet switch module in the target service switch has a second fault port, the second controller is triggered to perform a third recovery operation, and the third recovery operation includes: Sending a fault recovery instruction to the second fault port via the first management bus to perform fault recovery on the second fault port; Second fault information is generated and sent to the first controller through the second communication bus to trigger the first controller to perform a fourth recovery operation, wherein the second fault information includes a target service switch node number and a second fault port number.

[0019] In some embodiments, the fourth recovery operation includes: Determine a second target port according to the second fault information and the mapping relationship table; A fault recovery instruction is sent to the second target port via the first communication bus to perform fault recovery on the second target port.

[0020] In some embodiments, the detection of a first failed port of a target graphics processor within a target computing node device includes: Triggering a first controller in the computing node device to obtain a port status of one or more physical ports of the graphics processor through a first communication bus, where the port status includes a disconnection status and a connection status; A physical port whose port status is disconnected is determined as a first fault port, a graphics processor including the first fault port is determined as a target graphics processor, and a computing node device including the target graphics processor is determined as a target computing node device.

[0021] In some embodiments, the detection of the first faulty port of the Ethernet switch module in the target service switch includes: Triggering the second controller in the service switch to obtain the port status of multiple physical ports in the Ethernet switch module through the first management bus, where the port status includes a disconnection status and a connection status; The physical port in the Ethernet switching module whose port status is disconnected is determined to be the second fault port, and the service switch including the second fault port is determined to be the target computing service switch.

[0022] In a third aspect, the present application provides a computing node device, the computing node device comprising a first controller, a plurality of graphics processors, and a plurality of first connectors; The first controller is connected to the graphics processor via a first communication bus; The graphics processor is connected to a matching first connector through a first connection channel; The first connector is externally connected to a cable tray to achieve interconnection with a service switch; The first controller is externally connected to the second controller in the service switch through the second communication bus.

[0023] In some embodiments, the computing node device includes a first storage medium; The first storage medium is used to store the fault recovery firmware; The first controller is connected to the first storage medium through the first interface channel to read and execute the fault recovery firmware.

[0024] In some embodiments, the computing node device further includes a physical layer enhancement device; The graphics processor is connected to a matching first connector through a first connection channel, including: The graphics processor is connected to the input end of the physical layer enhancement device through the first connection channel, and the output end of the physical layer timer is connected to the first connector through the first connection channel.

[0025] In some embodiments, the computing node device further includes a multiplexer, and there is a one-to-one correspondence between the multiplexer and the graphics processor; The graphics processor is connected to the first input channel of the multiplexer through the second management bus; The output end of the multiplexer is connected to the physical layer enhancement device through the third management bus.

[0026] In some embodiments, the first controller accesses the second input channels of the plurality of multiplexers through a fourth management bus.

[0027] In some embodiments, the first controller accesses a designated pin of the multiplexer through a designated channel to transmit a selection signal to the multiplexer; The multiplexer determines whether the data signal outputted from the output end comes from the first input channel or the second input channel based on the selection signal.

[0028] In some embodiments, the computing node device further includes a first microprocessor, an expansion switch module, and a third storage medium; The microprocessor is connected to the expansion switch module via an expansion bus; The expansion switch module is connected to the graphics processor via the expansion bus; The microprocessor is connected to the third storage medium through a third interface channel to obtain the system firmware stored in the third storage medium.

[0029] In some embodiments, the computing node device includes a first Ethernet interface, the first controller is connected to the first Ethernet interface through a first channel, and the first Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable.

[0030] In a fourth aspect, the present application provides a service switch, the service switch comprising a second controller, a second connector, and an Ethernet switching module; The Ethernet switch module is connected to the second controller via the first management bus, and the Ethernet switch module is connected to the second connector via the second connection channel; The second connector is externally connected to a cable bridge to achieve interconnection with a computing node device; The second controller is externally connected to the first controller in the computing node device through a second communication bus.

[0031] In some embodiments, the service switch includes a second storage medium; The second storage medium is used to store the fault recovery firmware; The second controller is connected to the second storage medium through the second interface channel to read and execute the fault recovery firmware.

[0032] In some embodiments, the service switch includes a second microprocessor and a fourth storage medium; The second microprocessor is connected to the fourth storage medium through a fourth interface channel to obtain the second microprocessor firmware and the operating system stored in the fourth storage medium; The second microprocessor is connected to the Ethernet switching module through the expansion bus.

[0033] In some embodiments, the service switch includes a second Ethernet interface, and the second Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable.

[0034] In a fifth aspect, the present application provides a fault recovery firmware, including a computer program, which implements the following steps when executed by a processor: When it is detected that a first fault port exists in a target graphics processor in a target computing node device, a first controller is triggered to perform a first recovery operation, the first recovery operation comprising: Sending a fault recovery instruction to the target graphics processor via the first communication bus to perform fault recovery on the first fault port; First fault information is generated and sent to the second controller via the second communication bus to trigger the second controller to perform a second recovery operation, wherein the first fault information includes a target computing node device node number, a target graphics processor serial number, and a first fault port number.

[0035] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: When it is detected that the Ethernet switch module in the target service switch has a second fault port, the second controller is triggered to perform a third recovery operation, and the third recovery operation includes: Sending a fault recovery instruction to the second fault port via the first management bus to perform fault recovery on the second fault port; Second fault information is generated and sent to the first controller through the second communication bus to trigger the first controller to perform a fourth recovery operation, wherein the second fault information includes a target service switch node number and a second fault port number.

[0036] In a sixth aspect, the present application provides an electronic device, the electronic device comprising: one or more processors; and a memory associated with one or more processors, the memory being used to store program instructions, which when read and executed by the one or more processors, perform the following steps: When it is detected that a first fault port exists in a target graphics processor in a target computing node device, a first controller is triggered to perform a first recovery operation, the first recovery operation comprising: Sending a fault recovery instruction to the target graphics processor via the first communication bus to perform fault recovery on the first fault port; First fault information is generated and sent to the second controller via the second communication bus to trigger the second controller to perform a second recovery operation, wherein the first fault information includes a target computing node device node number, a target graphics processor serial number, and a first fault port number.

[0037] In some embodiments, when the program instructions are read and executed by one or more processors, the program instructions further perform the following steps: When it is detected that the Ethernet switch module in the target service switch has a second fault port, the second controller is triggered to perform a third recovery operation, and the third recovery operation includes: Sending a fault recovery instruction to the second fault port via the first management bus to perform fault recovery on the second fault port; Second fault information is generated and sent to the first controller through the second communication bus to trigger the first controller to perform a fourth recovery operation, wherein the second fault information includes a target service switch node number and a second fault port number.

[0038] In a seventh aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program enables a computer to perform the following operations: When it is detected that a first fault port exists in a target graphics processor in a target computing node device, a first controller is triggered to perform a first recovery operation, the first recovery operation comprising: Sending a fault recovery instruction to the target graphics processor via the first communication bus to perform fault recovery on the first fault port; First fault information is generated and sent to the second controller via the second communication bus to trigger the second controller to perform a second recovery operation, wherein the first fault information includes a target computing node device node number, a target graphics processor serial number, and a first fault port number.

[0039] In some embodiments, the computer program causes the computer to perform the following operations: When it is detected that the Ethernet switch module in the target service switch has a second fault port, the second controller is triggered to perform a third recovery operation, and the third recovery operation includes: Sending a fault recovery instruction to the second fault port via the first management bus to perform fault recovery on the second fault port; Second fault information is generated and sent to the first controller through the second communication bus to trigger the first controller to perform a fourth recovery operation, wherein the second fault information includes a target service switch node number and a second fault port number.

[0040] The beneficial effects achieved by this application are: The present application provides a server, including a computing node device, a service switch and a cable tray, wherein the computing node device includes a first controller, a plurality of graphics processors and a plurality of first connectors; the service switch includes a second controller, a second connector and an Ethernet switching module; the first controller is connected to the graphics processor via a first communication bus, and the first controller is connected to the second controller via a second communication bus; the graphics processor is connected to a matching first connector via a first connection channel; the Ethernet switching module is connected to the second controller via a first management bus, and the Ethernet switching module is connected to the second connector via a second connection channel; the first connector and the second connector are interconnected via a cable tray.

[0041] This architecture solves the problem of difficult channel fault recovery between the graphics processor and the service switch on the rack server. When a fault occurs in the connection channel (i.e., Ethernet link) between the computing node device and the service switch (including the first connection channel and the second connection channel), the computing node device or the service switch on the faulty side can transmit the fault information to the corresponding controller via the second communication bus so that the fault information can be used to locate the faulty port and recover the fault later.

[0042] Furthermore, the present application discloses a fault recovery method, which is based on a simple and flexible architecture, combined with the interconnection scheme of each node inside the server and the fault recovery method, greatly simplifies the system design, improves the system reliability, and solves the existing problems. At the same time, the architecture has strong versatility and can be transplanted to the connection channel fault rapid location and recovery of switches and router cabinets, and has high market value. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which: Figure 1 This is a schematic diagram of a server partial architecture provided by an embodiment of the present application; Figure 2 This is another schematic diagram of a server partial architecture provided in an embodiment of the present application; Figure 3 This is another schematic diagram of a server partial architecture provided in an embodiment of the present application; Figure 4 It is a schematic diagram of a computing node device architecture provided by an embodiment of the present application; FIG5 (a) is a schematic diagram of another computing node device architecture provided in an embodiment of the present application; FIG5( b ) is a schematic diagram of a service switch architecture provided in an embodiment of the present application; Figure 6 This is a schematic diagram of a server architecture provided by an embodiment of the present application; Figure 7 This is a schematic diagram of a fault recovery method in a scenario provided by an embodiment of the present application; Figure 8 This is a schematic diagram of a fault recovery method in another scenario provided by an embodiment of the present application; Fig. 9 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] It should be understood that in the description of the present application, unless the context clearly requires otherwise, words such as "include", "comprises", and the like in the entire specification and claims should be interpreted as inclusive rather than exclusive or exhaustive; that is, the meaning of "including but not limited to".

[0046] It should also be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0047] It should be noted that the terms "S1", "S2", etc. are only used for the purpose of describing the steps, and do not specifically refer to the order or sequence, nor are they used to limit the present application. They are only for the convenience of describing the method of the present application, and cannot be understood as indicating the order of the steps. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0048] As mentioned in the background technology, switches are increasingly used in servers, such as rack servers (RACK), which are specifically reflected in the ability to achieve the highest bandwidth and speed requirements, deep integration with servers, support for intelligent network management, impact on data center network architecture, and enhancement of data security. Therefore, due to the increasing maturity and large-scale application of switches on servers, the channel fault location function between the graphics processor on the server based on the Ethernet switch and the switch is extremely important in the design of the entire server system.

[0049] In the traditional server architecture design, there are the following problems: when there is a problem with the differential pair sent by the GPU to the switch, causing the physical port link down on the switch side, according to the Ethernet protocol, the switch will send a remote fault sequence to the GPU through the same physical port. After receiving the sequence, the GPU will also set the physical port of the GPU side to link down; then the GPU side and the switch side will do port link up (Link up) and link down (Link down) several times in the two MAC layers (media access control layer, media access control layer) of the same physical port to perform fault recovery; if the fault recovery cannot be performed, it is necessary to define hardware or firmware problems. In the traditional hardware architecture design, due to the hardware channel fault problem sent by the switch to the graphics processor, the switch cannot send the remote fault sequence to the graphics processor side of the computing node device, which ultimately results in the same physical port of the graphics processor and the switch, the switch side displays link down, and the graphics processor side displays link up; further, when performing fault recovery, the graphics processor side cannot obtain which physical port channel of the graphics processor is closed, resulting in the graphics processor side being unable to perform fault recovery actions. That is, the traditional hardware architecture design has the problems of difficulty in fault location and long fault location time, which to a certain extent increases the cost of fault location and recovery of the Ethernet channel between the graphics processor in the server and the switch, and brings certain challenges to the fault location and fault recovery of the Ethernet channel between the graphics processor in the server and the switch.

[0050] Therefore, the present application proposes a new type of server, which can solve the problem of difficult fault location and recovery of the Ethernet channel between the graphics processor and the switch on the mainstream servers in the industry, and greatly improve the reliability of the server system.

[0051] Embodiment 1 The embodiment of the present application provides a server, specifically, the server includes a computing node device, a service switch and a cable tray. It can be understood that the computing node device is interconnected with the computing node device through the cable tray, there are multiple computing node devices and multiple service switches in a server, a computing node device can be connected to multiple service switches, and a service switch can also be connected to multiple computing node devices.

[0052] Specifically, the computing node device includes a first controller, multiple graphics processors, and multiple first connectors; the service switch includes a second controller, a second connector, and an Ethernet switching module; the first controller is connected to the graphics processor through a first communication bus, and the first controller is connected to the second controller through a second communication bus; the graphics processor is connected to a matching first connector through a first connection channel; the Ethernet switching module is connected to the second controller through a first management bus, and the Ethernet switching module is connected to the second connector through a second connection channel; the first connector and the second connector are interconnected through the cable tray. There is a corresponding relationship between the graphics processor and the first connector, that is, one graphics processor is correspondingly provided with one first connector.

[0053] In a specific implementation scenario, the first connection channel and the second connection channel are both high-speed connection channels, such as 112 Gbps Serders x16 and other high-speed channels, or other specifications such as 112 Gbps Serders x8, which is not limited in this application. The first connector and the second connector can select 112G high-speed connectors, and of course, connectors with faster transmission rates can also be selected, which is not limited in this application. The first communication bus and the second communication bus are set as synchronous serial buses, including but not limited to I2C (Inter-Integrated Circuit). The first management bus is set as a communication interface channel for managing and controlling Ethernet physical layer devices, including but not limited to MDIO (Management Data Input / Output) bus.

[0054] Specifically, the above-mentioned server also includes a management switch; the computing node device includes a first Ethernet interface, the first controller is connected to the first Ethernet interface through a first channel, and the first Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable. The service switch includes a second Ethernet interface, the second controller is connected to the second Ethernet interface through a second channel, and the second Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable. Among them, the first channel and the second channel are any Ethernet interface channels used to connect the MAC layer and the PHY (Physical Layer) layer, including but not limited to Gigabit channel (RGMII, Reduced Gigabit Media Independent Interface) and GMII (Gigabit Media Independent Interface, Gigabit Media Independent Interface) channels, etc.

[0055] by Figure 1 Taking the specific implementation scenario shown as an example, the computing node device includes four graphics processors, which are connected to a corresponding first connector through a first connection channel; the Ethernet switching module is connected to the second controller through a first management bus, and the Ethernet switching module is connected to the second connector through a second connection channel; the first connector and the second connector are interconnected through a cable tray. That is, the graphics processors 0, 1, 2 and 3 of the computing node device respectively output a 112Gbps first connection channel to the 112Gbps first connector in the board, and the first connector is connected to the Cable Tray, and the Cable Tray then outputs a second connection channel to the Ethernet switching module on the service switch through the second connector, so as to realize the interconnection between the computing node device and the service switch. It can be understood that a service switch can be interconnected with multiple computing node devices, and a computing node device can be interconnected with multiple service switches.

[0056] Based on the above architecture, when the first connection channel and the second connection channel between the computing node device and the service switch fail, the computing node device or the service switch on the failed side can transmit the fault information to the corresponding controller through the second communication bus so that the fault information can be used to locate the fault port and recover the fault later.

[0057] In some embodiments, Figure 2Taking the architecture diagram shown as an example, the computing node device includes a first storage medium; the first storage medium is used to store fault recovery firmware; the first controller is connected to the first storage medium through a first interface channel to read and execute the fault recovery firmware. The service switch includes a second storage medium; the second storage medium is used to store fault recovery firmware; the second controller is connected to the second storage medium through a second interface channel to read and execute the fault recovery firmware. In this way, when an Ethernet link fails, the first controller in the computing node device and the second controller in the service switch can realize fault recovery of the fault port on the graphics processor side and the fault port on the Ethernet switch module side by reading and executing the fault recovery firmware stored in the corresponding first storage medium or second storage medium. Among them, the first storage medium and the second storage medium can be a serial peripheral interface flash memory, or any other non-volatile storage cutoff, which is not limited in this application; the first interface channel and the second interface channel are set as channels that match the storage medium. For example, when the storage medium is set to a serial peripheral interface flash memory, the first interface channel and the second interface channel are set to a serial interface channel (SPI, Serial Peripheral Interface).

[0058] In some embodiments, Figure 3 Taking the architecture diagram shown as an example, the computing node device also includes a physical layer enhancement device, wherein the physical layer enhancement device is usually configured as a physical layer signal conditioning enhancement chip for signal equalization and enhancement to reduce the attenuation that may occur when the signal is transmitted over long distances, such as a PHY Retimer; the graphics processor is connected to the matching first connector via a first connection channel, including: the graphics processor is connected to the input end of the physical layer timer via the first connection channel, and the output end of the physical layer timer is connected to the first connector via the first connection channel. That is, the graphics processor is connected to the first connection channel, and then the first connection channel is connected to the physical layer enhancement device, and then connected from the physical layer enhancement device to the cable tray. That is, Figure 2A physical layer enhancement device is added on the basis of the disclosed architecture, and the graphics processor can send high-speed signals (including TXDP / TXDN differential pairs) to the physical layer enhancement device, and the physical layer enhancement device sends the TXDP / TXDN differential pairs to the Ethernet switching chip. Specifically, the graphics processor transmits the high-speed signal to the physical layer enhancement device through the first connection channel, and then transmits it from the physical layer enhancement device to the first connector through the first connection channel; then the cable tray transmits it to the second connector, and finally transmits it to the Ethernet switching module in the switch through the second connector. The Ethernet switching module can also transmit the high-speed signal (including the TXDP / TXDN differential pair) to the second connector through the second connection channel, and then transmits it to the first connector through the cable tray, and then transmits it to the first connection channel through the first connector, and then transmits it to the physical layer enhancement device through the first connection channel, and finally transmits it to the graphics processor through the physical layer enhancement device. It can be understood that the present application avoids the influence of signal attenuation, jitter, reflection and other factors that may occur in high-speed signals that need to be transmitted over long distances through the physical layer enhancement device; the physical layer enhancement device cleans up these degraded signals to ensure that they reach the best state before being transmitted to the target node, further improves the stability of signal transmission, and reduces the probability of failure during signal transmission.

[0059] In some embodiments, Figure 4 Taking the architecture diagram shown as an example, the computing node device also includes a multiplexer, and there is a one-to-one correspondence between the multiplexer and the graphics processor; that is, one multiplexer is connected to one graphics processor; the graphics processor is connected to the first input channel of the multiplexer through the second management bus; the output end of the multiplexer is connected to the physical layer enhancement device through the third management bus. The first controller accesses the second input channels of multiple multiplexers through the fourth management bus. The first controller accesses the designated pin of the multiplexer through the designated channel to transmit the selection signal to the multiplexer. The above-mentioned designated channel and the above-mentioned designated pin can be set by a technician in this field, such as channel 1 in the eight-way channel, etc., and this application does not limit this; the multiplexer determines that the source of the data signal output by the output end is the first input channel or the second input channel based on the selection signal.

[0060] That is, the first controller and the graphics processor module each output a management bus connected to the multiplexer, the management bus output by the first controller is also connected to the multiplexer 0-3, and the management bus output by the graphics processor 0-3 is also connected to the multiplexer 0-3; the output management bus of the multiplexer is connected to the physical layer enhancement device, and the connection channel status of the physical layer enhancement device and the Ethernet switch chip of the opposite service switch node is acquired and managed. The first controller controls whether the management data signal at the output end of the multiplexer comes from the first controller or the graphics processor module by sending a Select0-3 signal; for example, it is defined that when Select0=1, the management data signal at the output end of the multiplexer comes from the first controller, and when Select0=0, the management data signal at the output end of the multiplexer comes from the graphics processor module. Therefore, the first controller controls whether the management data signal at the output end of multiplexer 0-3 comes from the first controller or the graphics processor by sending a Select[3:0] signal. Through the above settings, the management data bus can be used to obtain the LINK DOWN (link disconnection) and LINK UP (link connection) states of each physical port on each graphics processor side, as well as the LINK DOWN (link disconnection) and LINK UP (link connection) states of each physical port of the Ethernet switch module, so as to timely discover whether the corresponding MAC layer is faulty, improve the fault discovery speed, and further improve the fault recovery speed.

[0061] In some embodiments, taking the partial architecture diagram of the computing node device shown in FIG5 (a) as an example, the computing node device also includes a first microprocessor, an extended switching module, and a third storage medium; the first microprocessor is connected to the extended switching module through an extended bus; the extended switching module is connected to the graphics processor through the extended bus; the microprocessor is connected to the third storage medium through a third interface channel to obtain the system firmware stored in the third storage medium, such as BIOS firmware. That is, the high-speed link in the computing node device is designed as the first microprocessor outputs a PCIE x16 high-speed link to connect to two extended switching modules, the extended switching module 0 is connected to the graphics processor 0-1 through two PCIE x16 high-speed links downstream, and the extended switching module 1 is connected to the graphics processor 2-3 through two PCIE x16 high-speed links downstream; then the graphics processor 0-3 outputs the first connection channel to the physical enhancement device, and finally connects to the Ethernet switching module in the service switch, which has been explained in detail in the previous text and will not be repeated here.

[0062] In some embodiments, taking the partial architecture diagram of the service switch shown in FIG5 (b) as an example, the service switch includes a second microprocessor and a fourth storage medium; the second microprocessor is connected to the fourth storage medium through a fourth interface channel; the second microprocessor is connected to the Ethernet switch module through an expansion bus. The second microprocessor is connected to the Ethernet switch module through a high-speed link of PCIE x2, and the second microprocessor hangs the fourth storage medium through the fourth interface channel to store the second microprocessor firmware and the operating system; the second microprocessor is connected to the Ethernet switch module through the expansion bus. The settings of the third storage medium and the fourth storage medium and the third interface channel and the fourth interface channel are the same as the settings of the first storage medium and the second storage medium and the first interface channel and the second interface channel, and the present application will not repeat them here.

[0063] Through the above architecture, the interconnection between the first controller in the computing node device and the graphics processor is increased, and the interconnection between the first controller of the computing node device and the second controller in the service switch is also increased. When the TXDP / DN_GPUx signal sent by the graphics processor to the Ethernet switching chip has a problem, causing the MAC layer of the Ethernet switching chip to be in linkdown, but the connection channel from the Ethernet switching module to the graphics processor is also in a fault, causing the physical port of the switching chip in linkdown to be unable to send a remote fault sequence to the graphics processor through the connection channel, the second controller of the service switch first obtains which physical port has failed from the Ethernet switching module through the management bus, and then the first controller of the switching node device sends its own node ID number and the corresponding physical port number of the Ethernet switching module in linkdown to the first controller of the computing node device through the second communication bus, and at the same time, the second controller of the service switch sends instructions to the Ethernet switching module through the management bus to control the corresponding physical port to perform link up and link down fault recovery actions. After the first controller of the computing node device receives the fault information sent by the second controller of the switch, the first controller of the computing node device performs a table lookup operation, which is a mapping relationship table of the physical port of the graphics processor and the physical port of the Ethernet module maintained by the system maintenance end, wherein the physical port correspondence between the physical port of the graphics processor of the computing node and the physical port of the Ethernet switching module of the service switch during the hardware design is stored. The first controller of the computing node device knows which physical port of the graphics processor of the computing node device corresponds to the link down physical port of the service switch through the table lookup operation, and then the first controller of the computing node device sends an instruction to the graphics processor through the first communication bus to perform the link up and link down fault recovery actions on the physical port.

[0064] Similarly, when there is a problem with the second connection channel sent by the business switch to the computing node, causing the MAC layer on the graphics processor side to be link down, but the graphics processor cannot send the remote fault sequence to the Ethernet switching module of the switch through the first connection channel, the first controller of the computing node device sends its own node number and the corresponding graphics processor sequence number that is link down and the physical port number of the graphics processor to the second controller of the business switch through the second communication bus. At the same time, the first controller of the computing node device sends instructions to the graphics processor through the first communication bus to control the corresponding physical port to perform link up and link down fault recovery actions.

[0065] In a specific embodiment, Figure 6 Taking the architecture diagram shown as an example, the first Ethernet interface in the computing node device and the second Ethernet interface in the service switch in the server disclosed in this application are both connected to any idle Ethernet interface in the management switch through a network cable; it can be understood that the idle Ethernet interface is an interface that has not yet been connected to a device; the system maintenance end is connected to any idle Ethernet interface in the management switch through a network cable; the system maintenance end stores a mapping relationship table between the physical port of the graphics processor and the physical port of the Ethernet module. It can be understood that the above mapping relationship table can also be transferred to the first storage medium in the computing node device and the second storage medium in the service switch for easy call.

[0066] A server is usually equipped with multiple service switches and computing node devices. In this application, the service switches and computing node devices are interconnected through cable bridges. The first controllers of all computing node devices have Ethernet interfaces and are connected to the management switch through network cables. The second controllers of all service switches have Ethernet interfaces and are connected to the management switch through network cables. Any computing node device and service switch can be interconnected. A Power Shelf is also configured in the server to supply power to the entire cabinet, which is not shown in the figure.

[0067] Embodiment 2 Corresponding to the server disclosed in the above-mentioned embodiment 1, the present application provides a fault recovery method, which is applied to the server disclosed in the above-mentioned embodiment 2. Specifically, it includes two specific fault recovery scenarios, wherein the first fault recovery scenario is that when there is a problem with the differential pair link in the second connection channel sent by the service switch to the computing node device, the MAC layer of the graphics processor is in link down, and the first connection channel sent by the graphics processor to the switch is also faulty. At this time, the remote fault sequence cannot be sent to the Ethernet switching chip of the switching node through the connection channel.

[0068] Therefore, when it is detected that there is a first fault port on the target graphics processor in the target computing node device, the first controller is triggered to perform a first recovery operation, such as Figure 7 As shown, the first recovery operation includes: S1. Sending a fault recovery instruction to a target graphics processor via a first communication bus to perform fault recovery on a first faulty port.

[0069] S2. Generate first fault information and send it to the second controller through the second communication bus to trigger the second controller to perform a second recovery operation, wherein the first fault information includes a target computing node device node number, a target graphics processor serial number, and a first fault port number.

[0070] The second recovery operation includes: determining the first target port according to the first fault information and the mapping relationship table; sending a fault recovery instruction to the first target port through the first management bus to perform fault recovery on the first target port. The mapping relationship table is maintained at the system maintenance end, and the first controller obtains the mapping relationship table through the management switch through the first Ethernet interface. Similarly, the second controller can also obtain the mapping relationship table stored in the system maintenance end through the second Ethernet interface and then through the management switch. Of course, in some embodiments, the above-mentioned mapping relationship table can also be transferred to the first storage medium and the second storage medium for the first controller and the second controller to call.

[0071] In some embodiments, the detection of the first fault port of the target graphics processor in the target computing node device includes: Trigger the first controller in the computing node device to obtain the port status of one or more physical ports of the graphics processor through the first communication bus, the port status including the disconnected state and the connected state; that is, the first controller of the computing node device obtains the status of each physical port of each GPU through the communication bus; determine that the physical port with a port status of the disconnected state is the first fault port and determine that the graphics processor containing the first fault port is the target graphics processor and determine that the computing node device containing the target graphics processor is the target computing node device.

[0072] The second fault recovery scenario is that when there is a problem with the first connection channel from the graphics processor of the computing node device to the service switch, the MAC layer of the Ethernet switch module is in link down, and the second connection channel sent by the Ethernet switch module to the graphics processor is also faulty. At this time, the remote fault sequence cannot be sent to the graphics processor of the computing node device through the connection channel. Therefore, when it is detected that the Ethernet switch module in the target service switch has a second fault port, the second controller is triggered to perform the third recovery operation, such as Figure 8 As shown, the third recovery operation includes: X1. Sending a fault recovery instruction to the second fault port through the first management bus to perform fault recovery on the second fault port; X2. Generate second fault information and send it to the first controller through the second communication bus to trigger the first controller to perform a fourth recovery operation, wherein the second fault information includes a target service switch node number and a second fault port number.

[0073] The fourth recovery operation includes: determining the second target port according to the second fault information and the mapping relationship table; and sending a fault recovery instruction to the second target port through the first communication bus to perform fault recovery on the second target port.

[0074] In some embodiments, the detection of the first fault port of the Ethernet switching module in the target service switch includes: triggering the second controller in the service switch to obtain the port status of multiple physical ports in the Ethernet switching module through the first management bus, the port status including the disconnected state and the connected state; that is, the second controller of the service switch can obtain which physical port has failed from the Ethernet switching module through the first management bus; determine that the physical port in the Ethernet switching module whose port status is the disconnected state is the second fault port and determine that the service switch containing the second fault port is the target computing service switch.

[0075] The fault recovery method disclosed in the present application does not need to use the system maintenance end to send instructions to the first controller to trigger the first controller to send a fault recovery instruction to the graphics processor. Similarly, there is no need to use the system maintenance end to send instructions to the second controller to trigger the second controller to send a fault recovery instruction to the Ethernet switching module. The entire fault recovery process is integrated in the first controller and the second controller, and there is no need to exchange business with the upper-level system maintenance end, which reduces the complexity of the upper-level business software architecture design and greatly improves the fault recovery speed.

[0076] In a third embodiment, the present application provides a computing node device, the computing node device comprising a first controller, a plurality of graphics processors, and a plurality of first connectors; The first controller is connected to the graphics processor via a first communication bus; The graphics processor is connected to a matching first connector through a first connection channel; The first connector is externally connected to a cable tray to achieve interconnection with a service switch; The first controller is externally connected to the second controller in the service switch through the second communication bus.

[0077] In some embodiments, the computing node device includes a first storage medium; The first storage medium is used to store the fault recovery firmware; The first controller is connected to the first storage medium through the first interface channel to read and execute the fault recovery firmware.

[0078] In some embodiments, the computing node device further includes a physical layer enhancement device; The graphics processor is connected to a matching first connector through a first connection channel, including: The graphics processor is connected to the input end of the physical layer enhancement device through the first connection channel, and the output end of the physical layer timer is connected to the first connector through the first connection channel.

[0079] In some embodiments, the computing node device further includes a multiplexer, and there is a one-to-one correspondence between the multiplexer and the graphics processor; The graphics processor is connected to the first input channel of the multiplexer through the second management bus; The output end of the multiplexer is connected to the physical layer enhancement device through the third management bus.

[0080] In some embodiments, the first controller accesses the second input channels of the plurality of multiplexers through a fourth management bus.

[0081] In some embodiments, the first controller accesses a designated pin of the multiplexer through a designated channel to transmit a selection signal to the multiplexer; The multiplexer determines whether the data signal outputted from the output end comes from the first input channel or the second input channel based on the selection signal.

[0082] In some embodiments, the computing node device further includes a first microprocessor, an expansion switch module, and a third storage medium; The microprocessor is connected to the expansion switch module via an expansion bus; The expansion switch module is connected to the graphics processor via the expansion bus; The microprocessor is connected to the third storage medium through a third interface channel to obtain the system firmware stored in the third storage medium.

[0083] In some embodiments, the computing node device includes a first Ethernet interface, the first controller is connected to the first Ethernet interface through a first channel, and the first Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable.

[0084] In a fourth embodiment, the present application provides a service switch, the service switch comprising a second controller, a second connector, and an Ethernet switching module; The Ethernet switch module is connected to the second controller via the first management bus, and the Ethernet switch module is connected to the second connector via the second connection channel; The second connector is externally connected to a cable bridge to achieve interconnection with a computing node device; The second controller is externally connected to the first controller in the computing node device through a second communication bus.

[0085] In some embodiments, the service switch includes a second storage medium; The second storage medium is used to store the fault recovery firmware; The second controller is connected to the second storage medium through the second interface channel to read and execute the fault recovery firmware.

[0086] In some embodiments, the service switch includes a second microprocessor and a fourth storage medium; The second microprocessor is connected to the fourth storage medium through a fourth interface channel to obtain the second microprocessor firmware and the operating system stored in the fourth storage medium; The second microprocessor is connected to the Ethernet switching module through the expansion bus.

[0087] In some embodiments, the service switch includes a second Ethernet interface, and the second Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable.

[0088] Embodiment 5 Corresponding to all the above embodiments, the embodiments of the present application further provide a fault recovery firmware, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the following method are implemented: When it is detected that a first fault port exists in a target graphics processor in a target computing node device, a first controller is triggered to perform a first recovery operation, the first recovery operation comprising: Sending a fault recovery instruction to the target graphics processor via the first communication bus to perform fault recovery on the first fault port; First fault information is generated and sent to the second controller via the second communication bus to trigger the second controller to perform a second recovery operation, wherein the first fault information includes a target computing node device node number, a target graphics processor serial number, and a first fault port number.

[0089] In some embodiments, when the computer program is executed by the processor, the second recovery operation further includes: Determine a first target port according to the first fault information and the mapping relationship table; A fault recovery instruction is sent to the first target port via the first management bus to perform fault recovery on the first target port.

[0090] In some embodiments, when the computer program is executed by a processor, the steps of the following method are implemented: When it is detected that the Ethernet switch module in the target service switch has a second fault port, the second controller is triggered to perform a third recovery operation, and the third recovery operation includes: Sending a fault recovery instruction to the second fault port via the first management bus to perform fault recovery on the second fault port; Second fault information is generated and sent to the first controller through the second communication bus to trigger the first controller to perform a fourth recovery operation, wherein the second fault information includes a target service switch node number and a second fault port number.

[0091] In some embodiments, when the computer program is executed by the processor, the fourth recovery operation further includes: Determine a second target port according to the second fault information and the mapping relationship table; A fault recovery instruction is sent to the second target port via the first communication bus to perform fault recovery on the second target port.

[0092] In some embodiments, when the computer program is executed by a processor, the steps of the following method are implemented: Triggering a first controller in the computing node device to obtain a port status of one or more physical ports of the graphics processor through a first communication bus, where the port status includes a disconnection status and a connection status; A physical port whose port status is disconnected is determined as a first fault port, a graphics processor including the first fault port is determined as a target graphics processor, and a computing node device including the target graphics processor is determined as a target computing node device.

[0093] In some embodiments, when the computer program is executed by a processor, the steps of the following method are implemented: Triggering the second controller in the service switch to obtain the port status of multiple physical ports in the Ethernet switch module through the first management bus, where the port status includes a disconnection status and a connection status; The physical port in the Ethernet switching module whose port status is disconnected is determined to be the second fault port, and the service switch including the second fault port is determined to be the target computing service switch.

[0094] Embodiment 7 Corresponding to all the above embodiments, an embodiment of the present application provides an electronic device, including: one or more processors; and a memory associated with the one or more processors, the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the following operations are performed: When it is detected that a first fault port exists in a target graphics processor in a target computing node device, a first controller is triggered to perform a first recovery operation, the first recovery operation comprising: Sending a fault recovery instruction to the target graphics processor via the first communication bus to perform fault recovery on the first fault port; First fault information is generated and sent to the second controller via the second communication bus to trigger the second controller to perform a second recovery operation, wherein the first fault information includes a target computing node device node number, a target graphics processor serial number, and a first fault port number.

[0095] In some embodiments, when the program instructions are read and executed by one or more processors, further performing a second recovery operation includes: Determine a first target port according to the first fault information and the mapping relationship table; A fault recovery instruction is sent to the first target port via the first management bus to perform fault recovery on the first target port.

[0096] In some embodiments, when the program instructions are read and executed by one or more processors, the following operations are performed: When it is detected that the Ethernet switch module in the target service switch has a second fault port, the second controller is triggered to perform a third recovery operation, and the third recovery operation includes: Sending a fault recovery instruction to the second fault port via the first management bus to perform fault recovery on the second fault port; Second fault information is generated and sent to the first controller through the second communication bus to trigger the first controller to perform a fourth recovery operation, wherein the second fault information includes a target service switch node number and a second fault port number.

[0097] In some embodiments, when the program instructions are read and executed by one or more processors, further performing a fourth recovery operation includes: Determine a second target port according to the second fault information and the mapping relationship table; A fault recovery instruction is sent to the second target port via the first communication bus to perform fault recovery on the second target port.

[0098] In some embodiments, when the program instructions are read and executed by one or more processors, the following operations are performed: Triggering a first controller in the computing node device to obtain a port status of one or more physical ports of the graphics processor through a first communication bus, where the port status includes a disconnection status and a connection status; A physical port whose port status is disconnected is determined as a first fault port, a graphics processor including the first fault port is determined as a target graphics processor, and a computing node device including the target graphics processor is determined as a target computing node device.

[0099] In some embodiments, when the program instructions are read and executed by one or more processors, the following operations are performed: Triggering the second controller in the service switch to obtain the port status of multiple physical ports in the Ethernet switch module through the first management bus, where the port status includes a disconnection status and a connection status; The physical port in the Ethernet switching module whose port status is disconnected is determined to be the second fault port, and the service switch including the second fault port is determined to be the target computing service switch.

[0100] in, Fig. 9 The electronic device architecture is shown as an example, which may include a processor 910, a video display adapter 911, a disk drive 912, an input / output interface 913, a network interface 914, and a memory 920. The processor 910, the video display adapter 911, the disk drive 912, the input / output interface 913, the network interface 914, and the memory 920 may be communicatively connected via a bus 930.

[0101] Among them, the processor 910 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more circuits, etc., to execute relevant programs to implement the technical solution provided in this application.

[0102] The memory 920 can be implemented in the form of ROM (Read Only Memory, writable memory), RAM (Random Access Memory, random access memory), static storage device, dynamic storage device, etc. The memory 920 can store an operating system 921 for controlling the execution of the electronic device 900, and a basic input and output system (BIOS) 922 for controlling the low-level operations of the electronic device 900. In addition, a web browser 923, a data storage management system 924, and an icon font processing system 925, etc. can also be stored. The above-mentioned icon font processing system 925 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided in the present application is implemented by software or firmware, the relevant program code is stored in the memory 920 and is called and executed by the processor 910.

[0103] The input / output interface 913 is used to connect the input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0104] The network interface 914 is used to connect to a communication module (not shown) to achieve communication interaction between the device and other devices. The communication module can achieve communication through a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

[0105] The bus 930 comprises a pathway for transmitting information between the various components of the device (eg, the processor 910 , the video display adapter 911 , the disk drive 912 , the input / output interface 913 , the network interface 914 , and the memory 920 ).

[0106] In addition, the electronic device 900 can also obtain information on specific collection conditions from the virtual resource object collection condition information database for use in condition judgment, etc.

[0107] It should be noted that, although the above device only shows a processor 910, a video display adapter 911, a disk drive 912, an input / output interface 913, a network interface 914, a memory 920, a bus 930, etc., in the specific implementation process, the device may also include other components necessary for normal execution. In addition, it can be understood by those skilled in the art that the above device may also only include components necessary for implementing the solution of the present application, and does not necessarily include all the components shown in the figure.

[0108] It can be seen from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a cloud service end, or a network device, etc.) to execute the methods of various embodiments of the present application or certain parts of the embodiments.

[0109] Embodiment 8 Corresponding to all the above embodiments, the embodiments of the present application further provide a computer-readable storage medium storing a computer program, which enables a computer to perform the following operations: When it is detected that a first fault port exists in a target graphics processor in a target computing node device, a first controller is triggered to perform a first recovery operation, the first recovery operation comprising: Sending a fault recovery instruction to the target graphics processor via the first communication bus to perform fault recovery on the first fault port; First fault information is generated and sent to the second controller via the second communication bus to trigger the second controller to perform a second recovery operation, wherein the first fault information includes a target computing node device node number, a target graphics processor serial number, and a first fault port number.

[0110] In some embodiments, the computer program causes the computer to further perform a second operation, including: Determine a first target port according to the first fault information and the mapping relationship table; A fault recovery instruction is sent to the first target port via the first management bus to perform fault recovery on the first target port.

[0111] In some embodiments, the computer program causes the computer to further perform the following operations: When it is detected that the Ethernet switch module in the target service switch has a second fault port, the second controller is triggered to perform a third recovery operation, and the third recovery operation includes: Sending a fault recovery instruction to the second fault port via the first management bus to perform fault recovery on the second fault port; Second fault information is generated and sent to the first controller through the second communication bus to trigger the first controller to perform a fourth recovery operation, wherein the second fault information includes a target service switch node number and a second fault port number.

[0112] In some embodiments, the computer program causes the computer to further perform a fourth operation, including: Determine a second target port according to the second fault information and the mapping relationship table; A fault recovery instruction is sent to the second target port via the first communication bus to perform fault recovery on the second target port.

[0113] In some embodiments, the computer program causes the computer to further perform the following operations: Triggering a first controller in the computing node device to obtain a port status of one or more physical ports of the graphics processor through a first communication bus, where the port status includes a disconnection status and a connection status; A physical port whose port status is disconnected is determined as a first fault port, a graphics processor including the first fault port is determined as a target graphics processor, and a computing node device including the target graphics processor is determined as a target computing node device.

[0114] In some embodiments, the computer program causes the computer to further perform the following operations: Triggering the second controller in the service switch to obtain the port status of multiple physical ports in the Ethernet switch module through the first management bus, where the port status includes a disconnection status and a connection status; The physical port in the Ethernet switching module whose port status is disconnected is determined to be the second fault port, and the service switch including the second fault port is determined to be the target computing service switch.

[0115] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0116] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A server, comprising a computing node device, a service switch and a cable tray, characterized in that: The computing node device includes a first controller, a plurality of graphics processors, and a plurality of first connectors; The service switch includes a second controller, a second connector, and an Ethernet switching module; The first controller is connected to the graphics processor via a first communication bus, and the first controller is connected to the second controller via a second communication bus; The graphics processor is connected to the matching first connector through a first connection channel; The Ethernet switch module is connected to the second controller via a first management bus, and the Ethernet switch module is connected to the second connector via a second connection channel; The first connector and the second connector are interconnected via the cable tray.

2. The server according to claim 1, characterized in that: The computing node device includes a first storage medium; The first storage medium is used to store fault recovery firmware; The first controller is connected to the first storage medium through a first interface channel to read and execute the fault recovery firmware.

3. The server according to claim 2, characterized in that: The service switch includes a second storage medium; The second storage medium is used to store the fault recovery firmware; The second controller is connected to the second storage medium through a second interface channel to read and execute the fault recovery firmware.

4. The server according to claim 1, characterized in that: The computing node device also includes a physical layer enhancement device; The graphics processor is connected to the matching first connector through a first connection channel, including: The graphics processor is connected to the input end of the physical layer enhancement device through the first connection channel, and the output end of the physical layer timer is connected to the first connector through the first connection channel.

5. The server according to claim 4, characterized in that: The computing node device further includes a multiplexer, and there is a one-to-one correspondence between the multiplexer and the graphics processor; The graphics processor is connected to the first input channel of the multiplexer through a second management bus; The output end of the multiplexer is connected to the physical layer enhancement device through a third management bus.

6. The server according to claim 5, characterized in that: The first controller is connected to the second input channels of the multiplexers through a fourth management bus.

7. The server according to claim 6, characterized in that: The first controller accesses a designated pin of the multiplexer through a designated channel to transmit a selection signal to the multiplexer; The multiplexer determines, based on the selection signal, that the data signal outputted from the output end comes from the first input channel or the second input channel.

8. The server according to claim 1, characterized in that: The computing node device also includes a first microprocessor, an expansion switch module and a third storage medium; The first microprocessor is connected to the expansion switch module via an expansion bus; The expansion switch module is connected to the graphics processor via an expansion bus; The microprocessor is connected to the third storage medium through a third interface channel to obtain the system firmware stored in the third storage medium.

9. The server according to claim 8, characterized in that: The service switch includes a second microprocessor and a fourth storage medium; The second microprocessor is connected to the fourth storage medium through a fourth interface channel to obtain the second microprocessor firmware and the operating system stored in the fourth storage medium; The second microprocessor is connected to the Ethernet switching module via an expansion bus.

10. The server according to any one of claims 1 to 9, characterized in that: The server includes a management switch; The computing node device includes a first Ethernet interface, the first controller is connected to the first Ethernet interface through a first channel, and the first Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable; The service switch includes a second Ethernet interface, the second controller is connected to the second Ethernet interface through a second channel, and the second Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable.

11. The server according to claim 10, characterized in that: The server includes a system maintenance terminal; The system maintenance terminal is connected to any idle Ethernet interface in the management switch via a network cable; The system maintenance end stores a mapping relationship table between the physical ports of the graphics processor and the physical ports of the Ethernet module.

12. A fault repair method, applied to the server according to claim 11, characterized in that: The method comprises: When it is detected that a first fault port exists in a target graphics processor in a target computing node device, a first controller is triggered to perform a first recovery operation, wherein the first recovery operation includes: Sending a fault recovery instruction to the target graphics processor via the first communication bus to perform fault recovery on the first fault port; First fault information is generated and sent to the second controller via the second communication bus to trigger the second controller to perform a second recovery operation, wherein the first fault information includes a target computing node device node number, a target graphics processor serial number, and a first fault port number.

13. The method according to claim 12, characterized in that The second recovery operation includes: Determine a first target port according to the first fault information and the mapping relationship table; A fault recovery instruction is sent to the first target port via the first management bus to perform fault recovery on the first target port.

14. The method according to claim 12, characterized in that The method comprises: When it is detected that the Ethernet switch module in the target service switch has a second fault port, the second controller is triggered to perform a third recovery operation, and the third recovery operation includes: Sending a fault recovery instruction to the second fault port through the first management bus to perform fault recovery on the second fault port; Second fault information is generated and sent to the first controller through the second communication bus to trigger the first controller to perform a fourth recovery operation, wherein the second fault information includes a target service switch node number and a second fault port number.

15. The method according to claim 14, characterized in that The fourth recovery operation includes: Determine a second target port according to the second fault information and the mapping relationship table; A fault recovery instruction is sent to the second target port via the first communication bus to perform fault recovery on the second target port.

16. The method according to any one of claims 12 to 13, characterized in that: The detection of a first fault port of a target graphics processor in the target computing node device comprises: Triggering a first controller in the computing node device to obtain a port status of one or more physical ports of the graphics processor through a first communication bus, wherein the port status includes a disconnection status and a connection status; The physical port whose port status is disconnected is determined to be a first fault port, a graphics processor including the first fault port is determined to be a target graphics processor, and a computing node device including the target graphics processor is determined to be a target computing node device.

17. The method according to any one of claims 14-15, characterized in that: The detection of the first fault port of the Ethernet switch module in the target service switch includes: Triggering a second controller in the service switch to obtain port states of multiple physical ports in the Ethernet switch module through the first management bus, wherein the port states include a disconnection state and a connection state; The physical port in the Ethernet switching module whose port status is disconnected is determined to be a second fault port, and the service switch including the second fault port is determined to be a target computing service switch.

18. A computing node device, characterized in that: The computing node device includes a first controller, a plurality of graphics processors, and a plurality of first connectors; The first controller is connected to the graphics processor via a first communication bus; The graphics processor is connected to the matching first connector through a first connection channel; The first connector is externally connected to a cable bridge to achieve interconnection with a service switch; The first controller is externally connected to a second controller in the service switch via a second communication bus.

19. The computing node device according to claim 18, characterized in that: The computing node device includes a first storage medium; The first storage medium is used to store fault recovery firmware; The first controller is connected to the first storage medium through a first interface channel to read and execute the fault recovery firmware.

20. The computing node device according to claim 19, characterized in that: The computing node device also includes a physical layer enhancement device; The graphics processor is connected to the matching first connector through a first connection channel, including: The graphics processor is connected to the input end of the physical layer enhancement device through the first connection channel, and the output end of the physical layer timer is connected to the first connector through the first connection channel.

21. The computing node device according to claim 20, characterized in that: The computing node device further includes a multiplexer, and there is a one-to-one correspondence between the multiplexer and the graphics processor; The graphics processor is connected to the first input channel of the multiplexer through a second management bus; The output end of the multiplexer is connected to the physical layer enhancement device through a third management bus.

22. The computing node device according to claim 21, characterized in that: The first controller is connected to the second input channels of the multiplexers through a fourth management bus.

23. The computing node device according to claim 22, characterized in that: The first controller accesses a designated pin of the multiplexer through a designated channel to transmit a selection signal to the multiplexer; The multiplexer determines, based on the selection signal, that the data signal outputted from the output end comes from the first input channel or the second input channel.

24. The computing node device according to claim 23, characterized in that: The computing node device also includes a first microprocessor, an expansion switch module and a third storage medium; The microprocessor is connected to the expansion switch module via an expansion bus; The expansion switch module is connected to the graphics processor via an expansion bus; The microprocessor is connected to the third storage medium through a third interface channel to obtain the system firmware stored in the third storage medium.

25. The computing node device according to claim 24, characterized in that: The computing node device includes a first Ethernet interface, the first controller is connected to the first Ethernet interface through a first channel, and the first Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable.

26. A service switch, characterized in that: The service switch includes a second controller, a second connector, and an Ethernet switching module; The Ethernet switch module is connected to the second controller via a first management bus, and the Ethernet switch module is connected to the second connector via a second connection channel; The second connector is externally connected to a cable bridge to achieve interconnection with a computing node device; The second controller is externally connected to the first controller in the computing node device through a second communication bus.

27. The service switch according to claim 26, characterized in that: The service switch includes a second storage medium; The second storage medium is used to store fault recovery firmware; The second controller is connected to the second storage medium through a second interface channel to read and execute the fault recovery firmware.

28. The service switch according to claim 27, characterized in that: The service switch includes a second microprocessor and a fourth storage medium; The second microprocessor is connected to the fourth storage medium through a fourth interface channel to obtain the second microprocessor firmware and the operating system stored in the fourth storage medium; The second microprocessor is connected to the Ethernet switching module via an expansion bus.

29. The service switch according to claim 28, characterized in that: The service switch includes a second Ethernet interface, and the second Ethernet interface is connected to any idle Ethernet interface in the management switch through a network cable.

30. A fault recovery firmware, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 12 to 17 are implemented.

31. An electronic device, characterized in that: The electronic device comprises: one or more processors; And a memory associated with the one or more processors, the memory is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the method of any one of claims 12-17 is executed.

32. A non-volatile computer-readable storage medium, characterized in that: The computer program stores a computer program, wherein the computer program enables a computer to execute the method according to any one of claims 12 to 17.

Images