A method and apparatus for fast fault location in an MGX system

Abstract

The application provides a method and device for quickly locating faults in an MGX system, and belongs to the field of server system management and fault diagnosis. The method comprises the following steps: determining an idle universal serial bus interface on a mainboard, and selecting a target USB interface and a fixed IP network segment configuration corresponding to the target USB interface; a BMC establishes a physical connection with a device where an HMC is located through the target USB interface, and constructs a virtual local area network channel; in the virtual local area network channel, the BMC and the HMC perform out-of-band communication; the BMC obtains inventory information of components in the MGX system from the HMC, and classifies the components according to component types; the HMC respectively pre-collects state information of each category and stores the state information according to the classification result; and the BMC periodically reads the state information based on a Redfish protocol, and identifies faults in the MGX system according to the state information. The method and device provided by the application can quickly and accurately locate faults.

Description

Technical Field

[0001] This application relates to the field of server system management and fault diagnosis technology, and in particular to a method and apparatus for rapid fault location in an MGX system. Background Technology

[0002] The MGX (Modular GPU Extensions) system, as a core platform for high-performance computing and AI training, achieves high-speed interconnection of multiple GPUs through NVLinkSwitch. Leveraging bandwidth and latency advantages far exceeding PCIe, it supports large-scale parallel computing tasks (such as deep learning model training and scientific computing simulations). With the surge in AI computing power demands, the number of GPUs and interconnection complexity in the MGX system continue to increase. Hardware failures (such as GPU data transmission errors, NVLink link interruptions, and GPU crashes) have a more significant impact on business continuity, necessitating efficient fault location methods to ensure stable system operation.

[0003] Currently, fault localization in the MGX system mainly relies on two types of technologies. The first is the OS-based in-band diagnostic method, which uses tools under the system operating system (such as Linux) (such as nvidia-smi) to query the GPU status, NVLink link topology and error count, and analyzes the cause of the fault in conjunction with GPU driver logs (such as / var / log / nvidia / ). The second is the traditional out-of-band management method, which in some scenarios uses the IPMI protocol to collect basic hardware status (such as temperature and power supply) from the BMC (Baseboard Management Controller).

[0004] Existing technologies have significant limitations in fault localization for the MGX system. In-band diagnostic methods based on the OS require core fault information (such as NVLink error logs) to be obtained through the OS and drivers. If a GPU crash causes the OS to crash or the driver to fail, critical data will be lost, making it impossible to trace the root cause of the fault. Logging into the OS to query the status requires interrupting or affecting running business tasks (such as AI training), posing a risk of business interruption and failing to meet the requirements of high availability scenarios. Furthermore, fault information is scattered across multiple sources of data, such as tool outputs, driver logs, and hardware registers, requiring manual integration and analysis, which is time-consuming and cannot meet real-time requirements. At the same time, the status query methods for GPUs and NVLink Switches from different manufacturers differ (such as command parameters and log formats), lacking standardized interfaces and increasing the complexity of cross-platform fault localization. Traditional out-of-band management methods can only obtain surface-level status. Core information such as NVLink link status and GPU internal errors still need to be obtained by the OS and drivers, resulting in insufficient depth of fault diagnosis. In addition, the IPMI protocol uses binary data format, which is complex to parse and has poor compatibility with heterogeneous hardware from multiple vendors, making it difficult to support the fine-grained fault location requirements of multi-GPU and high-speed interconnect links in the MGX system. Summary of the Invention

[0005] In view of this, this application provides a method and apparatus for rapid fault location in the MGX system, which solves the problems of vague location, low diagnostic efficiency and unclear maintenance guidance caused by relying on a single data source in traditional fault diagnosis.

[0006] Specifically, this application is implemented through the following technical solution:

[0007] This application provides a method for rapid fault location in an MGX system, the MGX system including a management controller (BMC) and a hardware management controller (HMC) mounted on a motherboard, the method comprising:

[0008] The management controller (BMC) determines the available Universal Serial Bus (USB) interfaces on the motherboard and selects a target USB interface and the corresponding fixed IP network segment configuration from them.

[0009] The BMC establishes a physical connection with the device where the HMC is located through the target USB interface, and constructs a USB virtual LAN channel based on a fixed IP network segment configuration. In the USB virtual LAN channel, the BMC and the HMC communicate out of band using the Redfish protocol.

[0010] The BMC obtains the list information of each component in the MGX system from the HMC based on the Redfish protocol, and classifies them according to the component type;

[0011] The HMC pre-collects the status information of each category according to the classification results and stores it in the MGX system;

[0012] The BMC periodically reads the status information based on the Redfish protocol and identifies faults in the MGX system based on the status information.

[0013] A second aspect of this application provides a fault rapid location device for an MGX system, the device comprising:

[0014] The motherboard contains the management controller (BMC).

[0015] The Hardware Management Controller (HMC) is communicatively connected to the BMC.

[0016] A Universal Serial Bus (USB) interface is used to establish a USB virtual local area network (VLAN) channel between the BMC and the HMC based on a fixed IP network segment configuration.

[0017] The storage module is used to store component status information and fault diagnosis information collected by the HMC;

[0018] The processor module is configured to execute any of the fault location methods in the MGX system.

[0019] The fault rapid location method and apparatus in the MGX system provided in this application achieve stable out-of-band communication between the BMC and HMC, independent of the business network, by constructing a USB virtual LAN channel based on an idle USB interface and using a fixed IP network segment configuration. This communication method does not rely on the host operating system or business network, and can still guarantee the transmission of management data even if the system crashes or the network is interrupted, solving the data loss problem caused by traditional reliance on the OS or business network during faults. On this basis, standardized out-of-band communication is achieved using the Redfish protocol, unifying the interaction interface between the BMC and HMC. In the USB virtual LAN out-of-band channel of this solution, the standardized HTTP / HTTPS request-response mechanism of the Redfish protocol can adapt to the hardware differences between HMCs and BMCs from multiple vendors, ensuring that the status data of core components such as GPUs and NVSwitch are reliably transmitted and parsed in an independent channel, avoiding compatibility barriers caused by proprietary communication protocols of different vendors' devices, and providing a unified interface support for the integrated analysis of fault data of cross-platform hardware. BMC can acquire status data and make anomaly judgments based on component type, replacing the traditional manual query and analysis mode, improving the timeliness and accuracy of fault detection, and laying the foundation for rapid fault location. Attached Figure Description

[0020] Figure 1 A flowchart of an embodiment of the rapid fault location method in the MGX system provided in this application;

[0021] Figure 2 This is a schematic diagram of the structure of Embodiment 2 of the fault rapid location device in the MGX system provided in this application. Detailed Implementation

[0022] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0023] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0024] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0025] The following specific embodiments are given to illustrate the technical solution of this application in detail.

[0026] Example 1

[0027] It should be noted that the MGX system provided in this application includes a management controller (BMC) and a hardware management controller (HMC) located on the motherboard. The MGX system is a hardware system (similar to a server / computing node cluster) designed for high-performance computing and deep learning scenarios. It integrates multiple GPUs, high-speed interconnect links, and other components to provide hardware support for large-scale parallel computing and AI training and inference. It can efficiently handle complex computing tasks, such as training large language models and performing scientific simulations. The motherboard is the basic hardware carrier of the MGX system, carrying components such as the CPU, memory, various interfaces (USB, PCIe, etc.), and management controllers, and is responsible for the physical connection and signal transmission between hardware. The Baseboard Management Controller (BMC) is a dedicated management chip / module on the motherboard that can run independently of the host operating system. It can monitor and manage the motherboard and related hardware (such as temperature, voltage, and device status), and can also perform out-of-band communication (communication methods that do not rely on the host business network). The Hardware Management Controller (HMC) focuses on system hardware-level management. It can deeply collect and store status data of multiple components (GPU, PCIe link, NVSwitch, etc.), and work with the BMC to complete hardware monitoring and fault diagnosis, providing underlying data support for functions such as fault location.

[0028] It's important to note that the BMC and HMC work collaboratively, with the BMC focusing on basic management and communication scheduling at the motherboard and system levels (such as building virtual networks via USB and communicating using the Redfish protocol); and the HMC focusing on in-depth hardware status acquisition and storage (such as fine-grained status data for GPUs and PCIe links). Working together, the BMC can send data queries and log collection commands to the HMC, which responds and provides feedback on the hardware status, creating a data loop for fault location and system health management.

[0029] Based on the setup described in this application, from system power-on initialization to full-load operation, the BMC and HMC work together to monitor hardware temperature, voltage, link status, etc., providing early warnings of potential faults (such as frequency throttling triggered by excessive temperature or alarms due to excessive link error counts). It should also be noted that the HMC pre-stores the status of multiple components, allowing the BMC to quickly retrieve data. Combined with protocol parsing and log analysis, this accurately locates faults in GPUs, PCIe links, etc., shortening troubleshooting time (compared to systems without an independent management controller, this reduces the time cost of manually checking each hardware component). Furthermore, the management channel, independent of the host OS (such as an out-of-band network built by the BMC via USB), allows remote access to the BMC and HMC even if the host OS crashes, enabling hardware status queries and fault diagnosis, ensuring system operational controllability and reducing the risk of business interruption.

[0030] Figure 1 This is a flowchart of an embodiment of the rapid fault location method in the MGX system provided in this application. Please refer to... Figure 1 The method provided in this embodiment may include:

[0031] S101. Determine the available Universal Serial Bus (USB) interfaces on the motherboard, and select the target USB interface and the corresponding fixed IP network segment configuration from them.

[0032] It should be noted that Universal Serial Bus (USB) is a standardized hardware interface used for data transmission and power supply between devices. In the MGX system, the USB interface can be used as a management channel, allowing the BMC to communicate with other hardware (such as the HMC), and it features plug-and-play functionality and strong compatibility. Fixed IP network segment configuration refers to assigning fixed IP addresses and subnet masks to network devices (such as BMCs and HMCs connected via USB), so that the devices are on the same logical network, ensuring stable and predictable communication (as opposed to connection fluctuations that may be caused by dynamic IPs).

[0033] Specifically, the BMC calls the underlying hardware driver to traverse all physical USB interfaces on the motherboard (such as USB 3.0 and USB-C interfaces). By querying the interface status register, it filters out idle interfaces (no device occupation, no fault errors) to form a list of idle USB interfaces. Then, based on pre-configured rules (such as interface bandwidth priority and physical location priority), it selects the optimal interface from the list of idle USB interfaces. For example, it prioritizes USB 3.0 interfaces (higher bandwidth, meeting the needs of large-scale log transmission). If there are multiple interfaces of the same type, it selects the physical interface closer to the motherboard management area (shorter cabling, less signal interference) and marks it as the target USB interface. Finally, the BMC reads the system's pre-stored IP network segment mapping table (such as the IP segment 192.168.10.0 / 24 corresponding to USB interface A and the IP segment 192.168.20.0 / 24 corresponding to USB interface B), and automatically associates and loads the corresponding fixed IP network segment configuration (including IP address, subnet mask, gateway, and other parameters) according to the selected target USB interface. In the IP network segment mapping table, each USB interface corresponds to a unique fixed IP network segment, thereby avoiding the risk of IP conflict when multiple USB interfaces are used at the same time and ensuring the stability of communication between BMC and HMC.

[0034] Based on the above description, in the MGX system, the BMC and HMC require a stable out-of-band communication channel. However, the motherboard's USB ports may be occupied by other devices (such as external storage or debugging tools), and dynamic IP configuration can easily lead to communication interruptions (such as IP conflicts or address changes after a reboot), affecting fault diagnosis data transmission. In this step, the BMC actively scans and filters idle USB ports, selects the target port based on pre-configured rules, and binds it to a fixed IP network segment, building a reliable communication foundation. This avoids dynamic IP conflicts and address drift issues, ensuring long-term stable communication between the BMC and HMC. By prioritizing high-bandwidth, low-interference USB ports, data transmission latency is reduced, accelerating the fault diagnosis process. Furthermore, standardized USB ports are compatible with various hardware scenarios (new and old motherboards, different HMC devices). When adding new devices later, this logic can be reused to quickly establish communication, reducing operational complexity.

[0035] S102. The BMC establishes a physical connection with the device where the HMC is located through the target USB interface, and constructs a USB virtual LAN channel based on a fixed IP network segment configuration. In the USB virtual LAN channel, the BMC and the HMC communicate out-of-band using the Redfish protocol.

[0036] It's important to note that physical connection refers to the physical pins of the target USB interface being connected via a USB cable between the BMC and HMC devices, establishing a hardware-level data transmission path and forming the foundation of the virtual network channel. USB Virtual LAN Channel (LAN over USB) is a technology that virtualizes a USB interface as an Ethernet network interface, allowing a USB interface originally used for peripheral connections to transmit network data like a network cable. Simply put, it uses the USB as a network cable to build a virtual local area network between devices, enabling the BMC and HMC to communicate via the USB interface without the need for additional physical network cables. During this process, the BMC needs to configure its own virtual network interface according to the fixed IP network segment configuration corresponding to the target USB interface; simultaneously, the USB virtual network interface of the HMC device also needs to be configured with an IP address in the same network segment. The two achieve stable network layer communication through a static IP network segment matching mechanism.

[0037] Specifically, maintenance personnel use a USB cable, inserting one end into the target USB port on the motherboard where the BMC is located, and the other end into the USB port on the device where the HMC is located (it must be confirmed beforehand that the HMC device supports LAN over USB and the port is available), completing the physical connection at the hardware level, allowing electrical signals to be transmitted at both ends. Then, the BMC firmware calls the USB controller driver to switch the working mode of the target USB port to USB Ethernet mode (i.e., virtualizing the USB port as an Ethernet network interface through the USB network driver, giving it the data link layer communication capability of the network card). Simultaneously, the USB port on the HMC device also needs to have the corresponding function enabled (either automatically recognized by the HMC or enabled through underlying configuration), allowing both ends to use the USB channel as a network channel. Further, the BMC, based on the fixed IP network segment configuration selected by S101, sets the IP address (e.g., 192.168.10.2) and subnet mask (e.g., 255.255.255.0) for the virtual USB network interface. Simultaneously, the corresponding virtual interface of the device where the HMC resides needs to be configured with an IP address in the same network segment (e.g., 192.168.10.3) to ensure that both ends can communicate within the same virtual LAN. Finally, the BMC sends a simple network request (e.g., a ping command) to the device where the HMC resides through the virtual network interface. If a response packet is received from the HMC, it indicates that the USB virtual LAN channel has been successfully established; if it fails, a retry or error message is triggered (e.g., checking IP configuration or whether the USB cable is loose). Based on this, a physical + virtual network dual channel is established between the BMC and HMC, forming a stable out-of-band communication link, which can be used to transmit Redfish protocol data, fault diagnosis logs, etc.

[0038] In practice, a USB virtual LAN channel is constructed based on a fixed IP network segment configuration, including:

[0039] (1) In the MGX system, the management controller BMC is configured to have a USB Ethernet function module and set to master device mode.

[0040] It's important to note that the USB Ethernet function module refers to enabling the USB interface to function as an Ethernet network adapter within the BMC firmware / driver. Simply put, it gives a USB interface, which could previously only connect to USB flash drives and mice, network communication capabilities similar to a wired network interface, allowing the transmission of IP data packets. Master mode is a working mode of the USB interface where the BMC acts as the host, actively identifying and managing the connected device (such as the device containing the HMC). Similar to connecting a phone to a computer's USB port, the computer is the master device, capable of charging and transferring data to the phone; conversely, when the phone is connected via USB, it's in slave mode. Here, the BMC needs to actively manage the HMC's USB connection, hence the master device designation.

[0041] Specifically, in the BMC's system settings (or underlying driver configuration), enable the USB Ethernet function (the path varies depending on the BMC firmware; for example, the driver may be loaded via the iplink command, or the function may be selected in the Web management interface); then, set the BMC's target USB interface mode to master device, allowing the BMC to actively control the connected HMC device instead of passively waiting for a connection.

[0042] (2) Connect the BMC to the device where the hardware management controller HMC is located through the target USB interface, and configure a static IP address for the USB virtual LAN channel; wherein, after configuration, the BMC and HMC are in the same fixed network segment.

[0043] Referring to the previous description, the maintenance personnel use a USB cable, plugging one end into the target USB port of the BMC (which is already configured as a master device) and the other end into the USB port of the device where the HMC is located (which must support USB slave device mode). On the BMC side, configure a static IP address and subnet mask for the virtual Ethernet adapter; on the HMC side, configure a static IP address in the same network segment for the corresponding USB network interface.

[0044] (3) Start the virtual Ethernet adapter on the BMC, identify and initialize the USB virtual LAN channel.

[0045] It should be noted that the virtual Ethernet adapter is a software-simulated network card within the BMC system. It encapsulates the data stream from the USB interface into Ethernet frames through the driver layer, enabling the BMC to perform network data transmission and reception operations identical to a physical wired network card via the USB interface using USB Ethernet functionality. Specifically, within the BMC system, the startup command for the virtual Ethernet adapter is invoked. After the adapter starts, it automatically identifies the hardware parameters of the USB virtual LAN channel, completes initialization, and allows the BMC to send and receive network data through this adapter.

[0046] (4) After initialization, establish a network communication link between the BMC and HMC based on the static IP address.

[0047] Specifically, the BMC sends a TCPSYN packet (a connection request) to the HMC's IP address via a virtual adapter; upon receiving the packet, the HMC replies with a SYN+ACK packet, and the BMC then replies with an ACK packet, completing the TCP three-way handshake and establishing a stable network connection. Redfish protocol data can then be transmitted based on this link.

[0048] It should be noted that the TCP / IP protocol stack automatically completes the process of addressing → connection → data transmission. Since step (2) has ensured that the network segments are the same, the data packets sent by the BMC will be directly delivered to the corresponding network interface of the HMC through the virtual adapter + USB channel, without the need for complex routing, thereby achieving the purpose of efficient transmission.

[0049] In this step, a USB virtual network is physically isolated from the business network. Even if the host OS crashes or the business network is interrupted, the BMC can still access the HMC through the USB channel to retrieve fault logs (such as EROT information), ensuring that the fault diagnosis process is uninterrupted. Static IP configuration combined with master device mode avoids dynamic IP conflicts, and master device mode allows the BMC to actively manage connections, reducing the risk of connection interruption. It should also be noted that the USB virtual LAN built in this step is an out-of-band communication channel, completely decoupled from the host OS and business network. When the MGX system crashes due to GPU failure, NVLink link interruption, or other faults causing the host OS to crash, or the business network is interrupted due to excessive load or configuration errors, the BMC and HMC can still maintain communication through the USB physical connection and the virtual LAN. This feature completely solves the single point of failure problem of traditional in-band diagnostics (relying on the OS) and some out-of-band solutions (relying on the business network). Even if core system components fail, the hardware status data required for fault diagnosis (such as EROT logs and FPGA register snapshots) can still be fully retrieved by the BMC through the out-of-band channel. In addition, by utilizing the motherboard's idle USB ports, no additional wiring is required, making it suitable for the high-density hardware environment of the MGX system (multiple GPUs, multiple module stacks). When adding new nodes, the management channel can be expanded by reusing the USB connection logic, making operation and maintenance simpler.

[0050] It's important to note that the Redfish protocol is an open standard network management protocol based on HTTP / HTTPS and JSON, developed by the DMTF (Distributed Management Task Force) and specifically designed for managing hardware such as servers and storage devices. Its core value lies in enabling unified management of heterogeneous hardware from multiple vendors (such as BMC and HMC) through a standardized resource model and communication interface. For example, the BMC can initiate standardized hardware status query requests to the HMC via the Redfish protocol, and the HMC returns operational data for components such as GPUs and NVSwitch in JSON format. This eliminates the need to develop customized drivers for different vendors' devices, significantly reducing the compatibility costs of cross-platform management.

[0051] Out-of-band communication refers to an independent communication method that does not rely on the host operating system (OS) and business network. In this application, the BMC and HMC achieve out-of-band communication through a USB virtual LAN (the channel constructed by S102): even if the "business network" on which the MGX system's GPU and CPU depend for running services is interrupted, or the host OS crashes due to a fault, the BMC can still maintain communication with the HMC through the USB physical connection, ensuring that critical operations such as hardware status acquisition and fault log retrieval are not interrupted. Furthermore, the application of the Redfish protocol in this out-of-band channel, through a standardized HTTP / HTTPS request-response mechanism, ensures reliable transmission and parsing of fault data (such as EROT logs and FPGA register information) within the independent channel, providing a unified data interaction interface for multi-dimensional fault analysis. Specifically, in the virtual LAN channel, the BMC and HMC use the Redfish protocol for out-of-band communication, including:

[0052] (1) The BMC sends a Redfish request based on the HTTPS protocol to the HMC through the USB virtual LAN channel, and calls the corresponding resource path to access the running status information of the GPU unit, PCIe interconnect link and NVSwitch module.

[0053] It should be noted that the resource path is the unique address for accessing hardware resources in the Redfish protocol, similar to a web page URL. Each path corresponds to a specific hardware component, and the BMC obtains data precisely through the path.

[0054] Specifically, the BMC sends Redfish requests encrypted with HTTPS to the HMC via the established USB VLAN channel. These requests, through standardized resource paths, precisely target the GPU unit, PCIe interconnect link, or NVSwitch module to obtain the target hardware's operational status information. HTTPS encryption ensures secure communication, while the resource paths allow the BMC to directly locate and access the required data, avoiding redundant queries.

[0055] (2) After receiving the Redfish request, the HMC returns a JSON format response containing the device's operating status, including the device's unique identifier, operating status code, link connection topology, and error count information.

[0056] Upon receiving a request, the HMC first verifies its validity (e.g., identity authentication, permission checks), then extracts the corresponding information from locally cached hardware status data and organizes it into a JSON format response according to the Redfish protocol specification before returning it to the BMC. The response includes key information such as the device's unique identifier, operating status code, link connection topology, and error count (e.g., the number of correctable errors). A single response can provide multi-dimensional status data, significantly improving information retrieval efficiency.

[0057] (3) The BMC dynamically adjusts the polling frequency of each resource path based on the response content according to the preset collection and scheduling strategy, and automatically triggers the retry mechanism when a communication abnormality is detected.

[0058] Specifically, BMC dynamically adjusts the polling frequency of each resource path based on the preset acquisition scheduling strategy and the response content returned by HMC: when the hardware is in normal condition, a low frequency (e.g., once every 30 seconds) is used to reduce resource consumption; when an anomaly occurs (e.g., GPU overheating), a high frequency (e.g., once every 1 second) is switched to track changes in real time. At the same time, if a communication timeout or error is detected, BMC will automatically trigger a retry mechanism (e.g., up to 3 retries, with each retry interval increasing exponentially) to ensure that data can still be stably acquired during occasional network fluctuations, providing reliable information support for subsequent fault location.

[0059] In this step, the BMC can directly obtain the status of the target hardware through resource paths. Compared with traditional methods, this enables more accurate fault monitoring and rapid fault location. Furthermore, regardless of the HMC manufacturer, as long as it supports Redfish, the BMC uses the same set of resource paths for access, eliminating the need to develop dedicated drivers for each manufacturer and reducing integration costs. In addition, by dynamically polling the frequency, the BMC focuses more on the problematic component during faults and relaxes monitoring during normal periods. This adaptive monitoring method improves efficiency, and through out-of-band channels and retry mechanisms, data loss is ensured during faults.

[0060] S103. The BMC obtains the list information of each component in the MGX system from the HMC based on the Redfish protocol, and classifies them according to the component type.

[0061] It should be noted that the component list information refers to the structured data list of MGX system hardware components obtained by BMC from HMC via the Redfish protocol. This list includes basic information such as each component's unique identifier (e.g., UUID), hardware model, physical slot, function type, health status, and firmware version. Categorization by component type means grouping the list information according to hardware functional attributes. For example, GPU units responsible for graphics computing are grouped together, PCIe interconnect links responsible for high-speed data transmission are grouped together, and NVSwitch modules responsible for multi-GPU collaborative scheduling are grouped together. This categorization method provides the data foundation for HMC to implement differentiated status pre-collection strategies for different types of components (e.g., GPUs focus on temperature and error counts, PCIe links focus on speed and topology), and also allows BMC to quickly locate the hardware category to which a fault belongs based on the component type. Specifically, the BMC sends a request to the HMC via the Redfish protocol, calling a dedicated resource path. This path corresponds to a pre-stored system hardware list in the HMC. Upon receiving the request, the HMC returns structured data in JSON format conforming to the Redfish specification. Each hardware component contains Redfish standard attribute fields (such as Type, Id, Name, etc.). The BMC then parses the returned list information, extracts the Type field for each component, and groups components of the same type together based on the field value, forming a classification result. For example, the GPU category includes all GPU units (GPU-0, GPU-1, etc.); the PCIe Link category includes all PCIe interconnect links (PCIeLink-1, PCIeLink-2, etc.); and the NVSwitch category includes all NVSwitch modules (NVSwitch-0, NVSwitch-1, etc.). This classification method allows the BMC to quickly locate the hardware category to which a fault belongs based on component type, avoiding the efficiency loss caused by indiscriminate scanning of the entire system.

[0062] It's important to note that after categorization, BMC can develop differentiated strategies for different types of components, making status acquisition more accurate and efficient. For example, for GPUs, the focus is on monitoring temperature and memory error counts; for PCIe links, the focus is on monitoring speed and link width. Different MGX systems may have different hardware configurations (e.g., some contain 8 GPUs, others 4). By automatically acquiring and categorizing the inventory, BMC doesn't need to pre-hardcode hardware information and can adapt to different system configurations. For example, when a new NVSwitch module is added, BMC will automatically classify it as an NVSwitch and monitor it according to that category's strategy, without requiring manual program modification. Furthermore, when a fault occurs, BMC can directly and quickly locate the fault type based on the category. For example, when an abnormal Health attribute value is detected, the Type field is first used to determine the hardware category of the fault—whether it's a GPU or a PCIe link—before invoking the corresponding diagnostic logic (e.g., checking the GPU's EROT log or PCIe register information), avoiding a full system scan and saving time.

[0063] S104. The HMC pre-collects the status information of each category according to the classification results and stores it in the MGX system.

[0064] It should be noted that HMC actively collects the real-time status (such as temperature, error count, link speed, etc.) of each type of component, categorized by GPU unit, PCIe interconnect link, NVSwitch module, etc., and stores it in a unified format for easy retrieval by BMC when needed.

[0065] Specifically, based on the classification results, HMC pre-collects state information for each category and stores it in the MGX system, including:

[0066] (1) The HMC collects the operating status data of various components through the underlying driver interface between the GPU unit, the PCIe interconnect chip and the NVSwitch module.

[0067] It should be noted that the underlying driver interface is a dedicated communication interface between the HMC and hardware components (such as GPUs and PCIe chips). It is supported by drivers provided by hardware manufacturers and can directly read the underlying hardware status (such as GPU register values ​​and PCIe link signal strength), which is more real-time and lower-level than the data obtained through the operating system.

[0068] Specifically, HMC actively reads the real-time status of the hardware for each type of component categorized by BMC through the corresponding underlying driver interface. For GPU units, it collects data such as temperature, memory usage, and ECC error count through the GPU driver interface provided by NVIDIA; for PCIe interconnect links, it collects data such as link speed, link width, and error packet count through the PCIe driver interface; and for NVSwitch modules, it collects data such as port connection status, bandwidth utilization, and topology changes through the NVSwitch driver interface.

[0069] By communicating directly with the hardware registers or control chip through the low-level driver interface, bypassing the operating system, the most original hardware state can be obtained. This results in high data real-time performance and accuracy, and can capture instantaneous changes in the hardware state.

[0070] (2) The HMC caches the acquired status data locally and organizes and stores the status data in a structured manner based on the resource path format defined in the Redfish protocol.

[0071] It should be noted that the local cache here refers to the storage area built into the HMC (such as memory or local hard drive), which is used to temporarily store the collected status data and has a much faster access speed than remote storage; the resource path format defined in the Redfish protocol is a standardized data storage path, and each path corresponds to a specific status data (such as the health status of GPU-0), which makes it convenient for the BMC to query directly through the path.

[0072] Specifically, a key-value pair storage method can be used, where the key is the Redfish resource path and the value is the corresponding status data. This structure allows BMC to directly retrieve the value (data) by simply providing the key (path) when querying later, which is very efficient.

[0073] When a fault occurs, the BMC needs to immediately obtain the hardware status, while the HMC collects and stores data in advance, which can quickly locate the cause of the fault when it occurs. In addition, the HMC collects and stores data centrally, so the BMC does not need to communicate with each hardware separately, reducing the waste of resources from repeated data collection.

[0074] S105. The BMC periodically reads the status information based on the Redfish protocol and identifies faults in the MGX system based on the status information.

[0075] It should be noted that periodic readings enable the BMC to detect hardware anomalies in a timely manner and issue warnings before a failure occurs.

[0076] Specifically, the BMC periodically reads the status information based on the Redfish protocol and identifies faults in the MGX system based on the status information, including:

[0077] (1) The BMC periodically sends Redfish status read requests to the HMC according to the preset polling period and resource path table to obtain the latest status data of each GPU unit, PCIe interconnect link and NVSwitch module in the MGX system.

[0078] It's important to note that the polling cycle is set by the system based on hardware importance (e.g., a 5-second polling cycle for the GPU as a core component, and a 30-second cycle for NVSwitch), balancing real-time performance and resource consumption. The resource path table is simply a standardized list of addresses; the BMC doesn't need to memorize the physical locations of the hardware, but can directly query by path. This ensures that the BMC can continuously and stably obtain the latest status data, avoiding delays in fault detection due to missed checks.

[0079] (2) BMC performs structured parsing on the acquired state data and extracts the state data fields.

[0080] It should be noted that the status data returned by HMC is in JSON format. BMC uses parsing tools (such as JSON parsing libraries) to extract key fields, such as temperature from GPU data, speed from PCIe link data, and bandwidth utilization from NVSwitch data. This allows for quick filtering of the core data needed for fault diagnosis and elimination of redundant information (such as hardware model and manufacturer information).

[0081] (3) Compare the extracted status data fields with the local preset operating threshold range to determine whether there is an abnormal status.

[0082] It should be noted that the operating threshold range is a range of normal hardware operating parameters preset locally by BMC (preset by hardware manufacturers or maintenance personnel according to the characteristics of the equipment, such as the normal range of GPU temperature 0-85℃, and the normal PCIe link speed 16GT / s). Exceeding the range is considered abnormal.

[0083] This process, from periodically reading data to extracting key information and comparing thresholds to determine anomalies, requires no manual intervention. It transforms fault location from passively waiting for alarms to proactive, real-time monitoring, significantly improving fault identification efficiency. Furthermore, standardized resource paths ensure consistent data sources, structured parsing ensures accurate extracted fields, and threshold comparison ensures objective judgment logic, avoiding misjudgments caused by differing analytical standards among individuals, thus enhancing the accuracy and consistency of fault diagnosis.

[0084] Furthermore, identifying faults in the MGX system based on the status information includes: the BMC parsing the status information returned by the HMC and determining whether an abnormal state exists; when an abnormal state is detected, the BMC automatically triggers a fault log download operation to obtain fault diagnosis data from the HMC, including EROT fault information, FPGA register information, and EROT log information. That is, when the BMC determines an abnormal state based on the comparison results, it automatically triggers the log collection process, specifically including:

[0085] When the BMC determines that any component is in an abnormal state, based on the device identification information corresponding to the abnormal state, the BMC calls the log collection interface via the Redfish protocol to initiate a log download request to the HMC. According to the request, the HMC extracts the EROT fault information, FPGA register information, and EROT log information corresponding to the target component from its local cache. The extracted fault diagnosis data is compressed and packaged, and returned to the BMC via the USB virtual LAN channel. The BMC categorizes and stores the received fault diagnosis data according to device number and timestamp.

[0086] It should be noted that the log collection interface is the Redfish protocol standard interface provided by HMC, used to trigger log collection and download; EROT fault information is recorded during emergency hardware operation, containing key parameters at the moment of failure (such as temperature and voltage when the GPU crashes); FPGA register information is the internal register state of the field-programmable gate array, reflecting the underlying hardware configuration and operating status (such as signal strength and clock frequency of the PCIe link), used to locate hardware-level faults; EROT log information is a sequence of events recorded during hardware operation (such as GPU temperature change curves and error count growth trends), used to analyze the fault development process.

[0087] Specifically, the BMC sends a log download request to the HMC via the standard log acquisition interface of the Redfish protocol, based on the device identification information of the faulty component (such as GPU-0). This request explicitly specifies the required log type (such as EROT fault information, FPGA register values) and time range (usually a few minutes before and after the fault occurred) to ensure that the HMC can accurately extract relevant data. For example, when the BMC detects that the temperature of GPU-0 exceeds 85°C, it will immediately request all operational logs of that GPU from the past 5 minutes.

[0088] Subsequently, upon receiving a request, the HMC quickly extracts three types of key information from its local cache: first, EROT fault information, which includes key parameters recorded by the hardware in an emergency (such as temperature and voltage at the moment of the fault); second, FPGA register information, which reflects the real-time operating status of the hardware (such as signal strength and training status of the PCIe link); and third, EROT log information, which includes hardware events recorded in time series (such as temperature change trends and error count growth). This data has already been pre-collected and cached by the HMC in step S105, thus enabling a rapid response to the BMC's request and preventing data loss due to further hardware damage.

[0089] Furthermore, the HMC compresses and packages the extracted data into multiple formats (such as ZIP or TGZ) and returns it to the BMC via a USB virtual LAN channel. Compression significantly reduces data size (e.g., compressing 100MB of raw data to 20MB) and shortens transmission time, while the out-of-band channel ensures reliable acquisition of fault logs even in the event of a service network outage.

[0090] Finally, after receiving the compressed package, BMC categorizes and stores the data according to device number and timestamp. For example, it creates a directory structure like / logs / GPU-0 / 20250708T103000Z / to store decompressed register dump files, temperature change curves, etc., by type. This structured storage method not only facilitates maintenance personnel in quickly retrieving fault data of specific devices at specific times, but also lays the foundation for subsequent automated fault analysis.

[0091] Furthermore, it should be noted that upon detecting an abnormal state, the BMC automatically triggers a fault log download operation. After obtaining fault diagnostic data from the HMC, including EROT fault information, FPGA register information, and EROT log information, the data includes:

[0092] The BMC simultaneously parses the EROT fault information and EROT log information. Specifically, the BMC extracts Message and Resolution attributes from the EROT fault information. Based on the GPU ID, fault location, and fault type contained in the Message attribute, it determines the faulty GPU unit and fault category, and generates maintenance guidance information by combining the operation suggestions provided by the Resolution attribute. The BMC extracts contextual events and time sequence information of the fault occurrence time from the EROT log information to identify the propagation process of the abnormal state and associated link components. The BMC parses the FPGA register information to obtain the underlying hardware status parameters associated with the faulty GPU and compares them with abnormal events in the EROT log to determine if there are any register abnormal characteristics. Based on the obtained GPU fault location information, the abnormal event sequence extracted from the log, the FPGA register status characteristics, and the generated maintenance guidance information, the BMC integrates the fault diagnosis data to generate a structured standard fault log file.

[0093] It should be noted that the Message attribute is the core field in the EROT fault information, containing a concise description of the fault (e.g., GPU-0 temperature is too high), and the Resolution attribute provides repair suggestions in the EROT fault information (e.g., check the GPU fan). Context events are related events in the EROT log before and after the fault occurred (e.g., the temperature rises from 70℃ to 90℃ within 5 minutes), used to reconstruct the fault development process. Time series information is a sequence of events recorded in chronological order, reflecting the evolution path of the fault (e.g., event A → event B → fault occurs). Register anomaly features are parameters in the FPGA registers that deviate from the normal range (e.g., abnormal values ​​in the PCIe link training register), which are direct evidence of hardware-level faults.

[0094] It's important to note that, firstly, BMC parses both EROT fault information and EROT log information simultaneously. Within the EROT fault information, BMC focuses on extracting the Message and Resolution attributes. The Message attribute directly contains crucial information such as the GPU ID, fault location, and fault type. For example, if GPU-0 shows an overheating issue, BMC can quickly pinpoint the specific GPU unit and fault category by parsing this attribute. The Resolution attribute, on the other hand, is a predefined repair suggestion field from the hardware manufacturer. By parsing this attribute, BMC can obtain structured repair guidance information (such as checking the GPU fan or cleaning the heatsink) and directly output it to maintenance personnel.

[0095] Next, BMC extracts the contextual events and time-series information of the time point when the fault occurred from the EROT log information. This information records the evolution process from normal to abnormal. For example, the log may show that before the GPU temperature became too high, the bandwidth utilization of the NVSwitch port was consistently at 100%, after which the GPU temperature began to rise sharply. By analyzing the sequence and correlation of these events, BMC can identify the propagation path of the abnormal state, such as NVSwitch congestion causing GPU data transmission obstruction, which in turn leads to overheating. This step helps to locate the source of the fault.

[0096] Then, the BMC parses the FPGA register information to obtain the underlying hardware status parameters associated with the faulty GPU. The FPGA registers record the real-time operating status of the hardware. The BMC compares these register parameters with the abnormal events in the EROT log to look for any abnormal register characteristics. For example, if the log shows that the GPU temperature is too high, while the registers show that the GPU's power management module has triggered frequency reduction protection, this forms a complete chain of overheating → protection triggering. This cross-validation of underlying parameters and upper-level events can more accurately pinpoint the cause of the fault and avoid misdiagnosis.

[0097] Finally, BMC integrates all the information obtained in the previous three steps to generate a structured standard fault log file. This file contains fault location information (such as which GPU and what type of fault), anomaly event sequence (how the fault developed), FPGA register status characteristics (underlying hardware evidence), and generated maintenance guidance information. This log uses a standardized JSON format, which is convenient for maintenance personnel to read and understand quickly, and also facilitates subsequent integration with AI systems for automatic analysis.

[0098] Furthermore, the BMC parses the FPGA register information to obtain the underlying hardware status parameters associated with the faulty GPU, and compares them with abnormal events in the EROT log to determine whether there are register abnormal characteristics. This includes: the BMC extracting hardware status data from the FPGA register information; based on the hardware status data, the BMC combining the timestamps and event types of abnormal events recorded in the EROT log to perform time and event sequence comparison; the BMC analyzing the abnormal threshold triggering of register parameters to determine whether there are register status abnormal characteristics; and the BMC determining the severity and scope of the fault based on the correlation between register abnormal characteristics and abnormal events.

[0099] Specifically, the BMC extracts hardware status data from the FPGA register information. The FPGA registers are the low-level hardware monitors, recording key parameters associated with the GPU, such as the signal strength of the PCIe link, the operating status of the GPU power management module, and the operating parameters of the memory controller. This data is stored in binary form, which the BMC converts into readable information using a pre-defined mapping table. Based on this, the BMC compares this hardware status data with the abnormal events in the EROT log, performing time and event sequence comparisons. The EROT log records key events before and after the fault (such as NVSwitch congestion at 10:30 and GPU overheating at 10:32), each event with a precise timestamp. Through time alignment, the BMC can clearly see the correspondence between changes in register parameters and log events: for example, one minute after the NVSwitch congestion, the PCIe link signal strength register value drops from -6dBm to -9dBm, and the log records that the PCIe error count begins to increase.

[0100] Furthermore, the BMC analyzes the abnormal threshold triggering of register parameters. Each register parameter has a preset normal range (e.g., the normal PCIe link rate should be 16GT / s, and the GPU temperature safety threshold is 90℃). The BMC checks whether the parameter exceeds the threshold: for example, if it finds that the PCIe link rate register value drops to 8GT / s (below the standard value), or the GPU temperature register shows 92℃ (above the safety threshold), it will be marked as a register abnormality.

[0101] Finally, the BMC determines the severity and scope of the fault based on the correlation between register anomaly characteristics and abnormal events. If the register anomaly is perfectly synchronized in time with a serious fault in the log (such as GPU computation interruption) and the parameter deviation is large (such as temperature exceeding the threshold by 10°C), it is judged as a serious fault, and the impact may involve the entire GPU unit. If the register parameters only fluctuate slightly (such as the link rate briefly dropping to 14GT / s and then recovering), and the correlation with the warning event in the log is weak, it is judged as a minor anomaly, requiring only continuous monitoring. This classification helps operations and maintenance personnel prioritize handling urgent issues and optimize resource allocation.

[0102] The method provided in this embodiment achieves multi-dimensional technological breakthroughs by constructing an independent and reliable out-of-band communication and automated diagnostic process. First, the out-of-band communication mechanism based on the USB virtual LAN channel and Redfish protocol completely eliminates dependence on the host operating system and business network. Even if the system crashes or the driver fails, the BMC can still stably acquire the hardware status data pre-stored by the HMC, ensuring that fault information is not lost and solving the data interruption problem of traditional in-band diagnostics. Second, by pre-collecting and structured storing status data (such as GPU temperature and PCIe link speed) by component type through the HMC, combined with the dynamically adjusted polling strategy of the BMC, real-time monitoring and accurate screening of hardware status are achieved, avoiding the blindness of manual troubleshooting and reducing fault discovery time from hours to seconds. Furthermore, the multi-source data cross-validation mechanism (EROT fault information extraction for repair guidance, EROT log reconstruction of fault propagation paths, and FPGA registers providing underlying hardware evidence) significantly improves fault location accuracy. For example, through correlation analysis between register anomalies and log events, heat dissipation problems and power module faults can be quickly distinguished, reducing misjudgments. Finally, the standardized integration and categorized storage of fault logs not only provides clear maintenance guidance for operations and maintenance personnel, but also lays a data foundation for subsequent AI diagnostic model training, significantly reducing the complexity of cross-platform fault location and improving the overall reliability and operational efficiency of the MGX system.

[0103] Example 2

[0104] Corresponding to the aforementioned embodiment of a method for rapid fault location in an MGX system, this application also provides an embodiment of a device for rapid fault location in an MGX system.

[0105] Figure 2 This is a schematic diagram of the structure of Embodiment 2 of the fault rapid location device in the MGX system provided in this application. Please refer to... Figure 2 The apparatus provided in this embodiment includes:

[0106] The motherboard contains the management controller (BMC).

[0107] The Hardware Management Controller (HMC) is communicatively connected to the BMC.

[0108] A Universal Serial Bus (USB) interface is used to establish a USB virtual local area network (VLAN) channel between the BMC and the HMC based on a fixed IP network segment configuration.

[0109] The storage module is used to store component status information and fault diagnosis information collected by the HMC;

[0110] The processor module is configured to execute any of the fault location methods in the MGX system.

[0111] The apparatus of this embodiment can be used to perform... Figure 1 The steps of the method embodiment shown are similar in principle and process, and will not be repeated here.

[0112] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for rapid fault location in an MGX system, characterized in that, The MGX system includes a management controller (BMC) and a hardware management controller (HMC) mounted on the motherboard. The method includes: Identify any available Universal Serial Bus (USB) interfaces on the motherboard, and select the target USB interface and the corresponding fixed IP network segment configuration from them; The BMC establishes a physical connection with the HMC via the target USB interface and constructs a USB virtual LAN channel based on a fixed IP network segment configuration. In the USB virtual LAN channel, the BMC and the HMC communicate out-of-band using the Redfish protocol. The physical connection at the hardware level is completed using a USB cable. The BMC firmware calls the USB controller driver to switch the working mode of the target USB interface to USB Ethernet mode, using the USB channel as a network channel. The BMC obtains the list information of each component in the MGX system from the HMC based on the Redfish protocol, and classifies them according to the component type; The HMC pre-collects state information for each category based on the classification results and stores it in the MGX system. Specifically, a differentiated state pre-collection strategy is implemented for different types of components. The BMC extracts the Type field for each component and groups components of the same type together based on the field value to form a classification result. The component types include at least GPU units, PCIe interconnect links, and NVSwitch modules. The BMC periodically reads the status information based on the Redfish protocol and identifies faults in the MGX system based on the status information; The selection of the target USB interface includes: querying the interface status register, filtering out idle interfaces, forming an idle USB interface list, and selecting the interface with the highest priority from the idle USB interface list as the target USB interface based on interface bandwidth priority and physical location priority; if there are multiple USB interfaces of the same type, select the physical interface that is closer to the motherboard management area. The method of constructing a USB virtual LAN channel based on a fixed IP network segment includes: configuring the management controller (BMC) in the MGX system to have a USB Ethernet function module and setting it to master device mode; connecting the BMC to the device where the hardware management controller (HMC) is located through the target USB interface, and configuring a static IP address for the USB virtual LAN channel; wherein, after configuration, the BMC and HMC are in the same fixed network segment; starting the virtual Ethernet adapter on the BMC, identifying and initializing the USB virtual LAN channel; and after initialization, establishing a network communication link between the BMC and HMC based on the static IP address.

2. The method according to claim 1, characterized in that, In the USB virtual local area network channel, the BMC and the HMC communicate out-of-band using the Redfish protocol, including: The BMC sends a Redfish request based on the HTTPS protocol to the HMC via the USB virtual LAN channel, and calls the corresponding resource path to access the running status information of the GPU unit, PCIe interconnect link and NVSwitch module; After receiving the Redfish request, the HMC returns a JSON-formatted response containing the device's operating status. The response includes the device's unique identifier, operating status code, link connection topology, and error count information. The BMC dynamically adjusts the polling frequency of each resource path based on the response content according to the preset acquisition and scheduling strategy, and automatically triggers a retry mechanism when a communication anomaly is detected.

3. The method according to claim 1, characterized in that, The HMC, based on the classification results, pre-collects status information for each category and stores it in the MGX system, including: The HMC collects operational status data of various components through the underlying driver interface between the GPU unit, PCIe interconnect chip and NVSwitch module. The HMC caches the acquired status data locally and organizes and stores the status data in a structured manner based on the resource path format defined in the Redfish protocol.

4. The method according to claim 1, characterized in that, The BMC periodically reads the status information based on the Redfish protocol and identifies faults in the MGX system based on the status information, including: The BMC periodically sends Redfish status read requests to the HMC according to a preset polling cycle and resource path table to obtain the latest status data of each GPU unit, PCIe interconnect link and NVSwitch module in the MGX system. BMC performs structured parsing on the acquired status data and extracts the status data fields; The extracted status data fields are compared with the locally preset operating threshold range to determine whether there is an abnormal status.

5. The method according to claim 1, characterized in that, The step of identifying faults in the MGX system based on the status information includes: The BMC parses the status information returned by the HMC and determines whether there is an abnormal status; When an abnormal state is detected, the BMC automatically triggers a fault log download operation to obtain fault diagnosis data from the HMC, including EROT fault information, FPGA register information, and EROT log information.

6. The method according to claim 5, characterized in that, Upon detecting an abnormal state, the BMC automatically triggers a fault log download operation. After obtaining fault diagnosis data from the HMC, including EROT fault information, FPGA register information, and EROT log information, the data includes: BMC simultaneously parses the EROT fault information and EROT log information; wherein, BMC extracts Message attribute and Resolution attribute from the EROT fault information, determines the GPU unit that has failed and the fault category based on the GPU number, fault location and fault type contained in the Message attribute, and generates maintenance guidance information by combining the operation suggestions provided by the Resolution attribute. The BMC extracts the context events and time series information of the time point of the fault occurrence from the EROT log information, and identifies the propagation process of the abnormal state and the associated link components. The BMC parses the FPGA register information, obtains the underlying hardware status parameters associated with the faulty GPU, and compares them with the abnormal events in the EROT log to determine whether there are any abnormal register characteristics. The BMC integrates the acquired GPU fault location information, the abnormal event sequence extracted from the log, the FPGA register status characteristics, and the generated maintenance guidance information to generate a structured standard fault log file.

7. The method according to claim 6, characterized in that, The BMC parses the FPGA register information to obtain the underlying hardware status parameters associated with the faulty GPU, and compares them with the abnormal events in the EROT log to determine whether there are any register abnormal characteristics, including: The BMC extracts hardware status data from the FPGA register information; Based on the hardware status data, the BMC combines the timestamps and event types of abnormal events recorded in the EROT log to perform time and event sequence comparison; The BMC analyzes the abnormal threshold triggering of register parameters to determine whether there are abnormal register state characteristics. The BMC determines the severity and scope of the fault based on the correlation between register anomaly characteristics and anomaly events.

8. The method according to claim 5, characterized in that, Upon detecting an abnormal state, the BMC automatically triggers a fault log download operation to obtain fault diagnosis data from the HMC, including EROT fault information, FPGA register information, and EROT log information, including: When the BMC determines that any component is in an abnormal state, based on the device identification information corresponding to the abnormal state, the BMC calls the log collection interface through the Redfish protocol to initiate a log download request to the HMC; According to the request, the HMC extracts the EROT fault information, FPGA register information and EROT log information corresponding to the target component from the local cache; The extracted fault diagnosis data is compressed and packaged, and then returned to the BMC via the USB virtual LAN channel; The BMC classifies and stores the received fault diagnosis data according to the device number and timestamp.

9. A rapid fault location device in an MGX system, characterized in that, The apparatus is used to perform the method as described in any one of claims 1-8, the apparatus comprising: The motherboard contains the management controller (BMC). The Hardware Management Controller (HMC) is communicatively connected to the BMC. A Universal Serial Bus (USB) interface is used to establish a USB virtual local area network (VLAN) channel between the BMC and the HMC based on a fixed IP network segment configuration. The storage module is used to store component status information and fault diagnosis information collected by the HMC; A processor module configured to perform the method as described in any one of claims 1 to 8.

Images