Fault detection method

Abstract

The invention provides a fault detection method. The method comprises the following steps: acquiring hard disk information of a target hard disk through a BMC (Baseboard Management Controller) and an out-of-band channel; when the BMC does not acquire the hard disk information through an out-of-band channel, acquiring serial communication bus I2C state information through an in-band channel; and determining a fault state of the I2C according to a data line SDA level state and a clock line SCL level state in the I2C state information. The monitoring continuity is ensured by checking the out-of-band channel and the in-band channel, even if the target hard disk fails to respond to the acquisition command, the BMC can still acquire the working state of the I2C through the in-band channel so as to confirm whether the I2C is in a fault state or not, so that the problem can be found in time, targeted measures can be taken as soon as possible, and the safety of the I2C is improved. And the I2C in the fault state is recovered to the normal operation state as soon as possible, so that the fault lasting time is shortened.

Description

Technical Field

[0001] The present application relates to the field of server technology, and in particular to a fault detection method. Background Art

[0002] As enterprise-class NVMe SSDs continue to evolve, their increased complexity does bring a series of management challenges. Hardware conflicts, software errors, incorrect power management settings, and firmware or driver incompatibilities can all cause NVMe SSDs to hang.

[0003] Currently, the server communicates and manages NVMe SSDs only through the I2C channel. This single channel dependency means that once the I2C channel fails or hangs, the server will lose access to the NVMe SSD. Since the I2C channel may be affected by a variety of factors (such as hardware conflicts, software errors, power management issues, etc.), this dependency greatly increases the instability and risk of the system. Any factor that affects the stability of the I2C channel may cause the server to fail to work properly, thereby affecting the performance and reliability of the entire system. Furthermore, when the I2C channel is in an abnormal state, the server cannot obtain key information such as VPD (Vital Product Data) and SMD (Solid-State Metadata) stored in the NVMe SSD through the I2C channel. Key information such as VPD and SMD is essential for hard disk health status monitoring, performance optimization, and fault prediction. When this key information cannot be obtained, the service will not be able to accurately assess the status of the hard disk, and thus cannot promptly determine whether the I2C is faulty, and cannot take targeted countermeasures for abnormal situations. Summary of the invention

[0004] The embodiment of the present application provides a fault detection method, which aims to solve the problem that I2C fault detection cannot be performed.

[0005] In a first aspect, an embodiment of the present application provides a fault detection method, including:

[0006] The baseboard management controller BMC obtains the hard disk information of the target hard disk through an out-of-band channel;

[0007] When the BMC fails to obtain the hard disk information through the out-band channel, obtaining serial communication bus I2C status information through the in-band channel;

[0008] The fault state of the I2C is determined according to the data line SDA level state and the clock line SCL level state in the I2C state information.

[0009] In the method provided in the embodiment of the present application, the BMC obtains the hard disk information through the out-of-band channel. If the BMC cannot obtain the hard disk information through the out-of-band channel, it indicates that there may be a channel failure or a problem with the hard disk itself. The BMC obtains the status information of the I2C through the in-band channel to confirm whether the I2C is in a faulty state. That is, the present application ensures the continuity of monitoring through the inspection of the out-of-band and in-band channels. Even if the target hard disk fails to respond to the acquisition command, resulting in the server being unable to obtain the relevant hardware information in the VPD and SMD of the hard disk through the out-of-band managed I2C channel, the BMC can still obtain the working status of the I2C through the in-band channel to confirm whether the I2C is in a faulty state, thereby promptly discovering the problem, and then taking targeted measures as soon as possible to restore the I2C in a faulty state to a normal operating state as soon as possible, thereby reducing the duration of the fault.

[0010] In a possible implementation, the baseboard management controller BMC obtains hard disk information of the target hard disk through an out-of-band channel, including:

[0011] The BMC obtains hard disk information of the target hard disk through an out-of-band channel based on an acquisition cycle;

[0012] When the BMC fails to obtain the hard disk information through the out-of-band channel, obtaining the serial communication bus I2C status information through the in-band channel includes:

[0013] When the BMC fails to obtain the hard disk information within a first preset number of acquisition cycles, the BMC obtains the I2C status information through an in-band channel.

[0014] In a possible implementation, the I2C status information is pre-stored in a target storage area, which is an area in the target hard disk used to record hardware internal status and operation information. The I2C status information is determined by:

[0015] The target hardware detects the data line SDA level state and the clock line SCL level state multiple times according to the inspection cycle;

[0016] Feedback the detected SDA level state and SCL level state to the main controller SOC of the target hardware each time;

[0017] The SOC records the SDA level status and the SCL level status detected multiple times into a target storage area in the target hard disk.

[0018] In a possible implementation, determining the fault state of the I2C according to the data line SDA level state and the clock line SCL level state in the I2C state information includes:

[0019] When the SDA level in the I2C status information is continuously at a low level and the SCL level is continuously at a high level within a second preset number of consecutive inspection cycles, the BMC determines that the I2C is in a hung state;

[0020] When the SDA level state and the SCL level state in the I2C state information do not flip within a third preset number of consecutive inspection cycles, the BMC determines that the I2C is in a hung state.

[0021] In a possible implementation, the method further includes:

[0022] The SOC marks the state information of the I2C based on the SDA level state and the SCL level state detected multiple times, and stores the marking result of the I2C state information in a target storage area in the target hard disk;

[0023] The SOC marks the status information of I2C based on the SDA level status and SCL level status detected multiple times:

[0024] When the SDA level state in the I2C status information is continuously at a low level and the SCL level state is continuously at a high level within a second preset number of consecutive inspection cycles, the SOC marks the I2C status information as a hung state;

[0025] When the SDA level state and the SCL level state in the I2C state information do not flip within a third preset number of consecutive inspection cycles, the SOC marks the I2C state information as a hung state.

[0026] In a possible implementation, determining the fault state of the I2C according to the data line SDA level state and the clock line SCL level state in the I2C state information includes:

[0027] The marking results corresponding to the SDA level state and the SCL level state in the I2C state information are read as the fault state of the I2C.

[0028] In a possible implementation, the fault state includes a hung state, and the method further includes:

[0029] When the BMC determines that the I2C is in a hung state, it sends an I2C reset command to the target hard disk through an in-band channel. The I2C reset command is used to instruct the power supply unit of the I2C to power off and then power on again to restore the I2C to a normal working state.

[0030] In a possible implementation, the method further includes:

[0031] After the I2C reset command is executed, the BMC obtains the hard disk information through the out-of-band channel;

[0032] When the BMC obtains the hard disk information through the out-of-band channel, determining that the I2C channel is restored to a normal operating state;

[0033] When the BMC cannot obtain hard disk information through the out-of-band channel, an alarm instruction is generated

[0034] In a second aspect, an embodiment of the present application provides a fault detection method, including:

[0035] The target hard disk receives an acquisition command sent by a baseboard management controller BMC, wherein the acquisition command is a command for the BMC to acquire hard disk information of the target hard disk through an out-of-band channel;

[0036] When the target hard disk fails to return the hard disk information to the BMC through the out-of-band channel, the serial communication bus I2C status information is returned to the BMC through the in-band channel, so that the BMC determines the fault status of the I2C according to the data line SDA level status and the clock line SCL level status in the I2C status information.

[0037] According to the method provided by the embodiment of the present application, when the target hard disk fails to return the hard disk information to the BMC through the out-of-band channel after receiving the acquisition command sent by the baseboard management controller BMC, it indicates that there may be a channel failure or a problem with the hard disk itself. The target hard disk can return the serial communication bus I2C status information to the BMC through the in-band channel, so that the BMC can confirm whether the I2C is in a faulty state according to the data line SDA level state and the clock line SCL level state in the I2C status information. That is, the present application ensures the continuity of monitoring through the inspection of the out-of-band and in-band channels. Even if the target hard disk fails to respond to the acquisition command, resulting in the server being unable to obtain the relevant hardware information in the VPD and SMD of the hard disk through the out-of-band managed I2C channel, the target hard disk can still return the working status of the I2C to the BMC through the in-band channel, so that the BMC can confirm whether the I2C is in a faulty state, thereby promptly discovering the problem, and then taking targeted measures as soon as possible to restore the I2C in a faulty state to a normal operating state as soon as possible, thereby reducing the duration of the fault.

[0038] In a possible implementation, when the target hard disk fails to return the hard disk information to the BMC through the out-of-band channel, returning the serial communication bus I2C status information to the BMC through the in-band channel includes:

[0039] When the target hard disk does not return the hard disk information to the BMC through the out-of-band channel, the target hard disk receives the access command sent by the BMC;

[0040] After receiving the access command, the target hard disk reads the current level state of SDA and the current level state of SCL, and determines whether the current level state of SDA and the current level state of SCL are flipped relative to the level state of SDA and the level state of SCL obtained in the most recent inspection cycle;

[0041] When the current level state of SDA and the current level state of SCL do not flip, the target hard disk marks the I2C state information as a hung state and writes it into the target storage area, and returns the I2C state information marked as a hung state in the target storage area to the BMC through the in-band channel. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic diagram of the interaction between the server and the NVMe SSD provided in the embodiment of the present application;

[0044] Figure 2 A flowchart of a fault detection method provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of a first interaction between an I2C circuit and a controller inside an NVMe SSD provided in an embodiment of the present application;

[0046] Figure 4 This is the I2C dead-time logic diagram provided in the embodiment of the present application;

[0047] Figure 5 A second schematic diagram of interaction between an I2C circuit and a controller inside an NVMe SSD provided in an embodiment of the present application;

[0048] Figure 6 A schematic diagram of the first interaction between the BMC and the NVMe SSD provided in the embodiment of the present application;

[0049] Figure 7 A schematic diagram of the second interaction between BMC and NVMe SSD provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] It should be noted that the embodiments described in this application are only part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0051] In order to make the following embodiments clear, the technical terms involved in this application are first introduced.

[0052] NVMe SSD (Non-Volatile Memory Express Solid State Drive) is a solid state drive that uses the NVMe protocol. It is designed to take full advantage of the high-speed performance of NAND flash memory. The NVMe protocol is optimized for the PCIe (Peripheral Component Interconnect Express) bus and can provide faster data transfer speeds and lower latency than traditional SATA or SAS interface SSDs.

[0053] I2C hangs: I2C is a two-wire (clock SCL and data SDA) multi-master and multi-slave synchronous communication bus. "Hangs" means that a device abnormally and continuously pulls down the SDA or SCL line, making it impossible for other devices to communicate normally. The state of SDA and SCL is always low and always high respectively.

[0054] VPD: The full name is Vital Product Data. It contains key information about the SSD, such as manufacturer information, serial number (SN), model name (MN), etc. This data is usually stored in EEPROM and can be accessed even when auxiliary power (3.3V aux power supply) is present. This means that the baseboard management controller (BMC) can read this data even in the absence of main power, which is critical for remote management and troubleshooting.

[0055] SMD: In the NVMe-MI protocol, SMD refers to Subsystem Management Data. This data structure allows managers to obtain the real-time status of the device, such as temperature, power status, etc., through the SMBus / I2C interface. Compared with VPD, SMD provides more dynamic device operation information, which is essential for monitoring the health and performance of the device.

[0056] Log page is a physical page in the database used to store log information. These log information records all modification operations in the database, including data insertion, update, deletion, etc., as well as transaction management information such as transaction start, commit and rollback.

[0057] In the embodiment of the present application, the BMC checks the status of the NVMe SSD through an out-of-band channel. The out-of-band channel is a communication method independent of the main data channel. It can still work when the system main processor is not working or the system crashes, thereby providing a backup monitoring and control means. If the BMC fails to obtain the hard disk information through the out-of-band channel, it means that there may be a communication problem. At this time, the BMC attempts to obtain the working status of I2C through the in-band channel. The BMC detects the working status of I2C through the in-band channel, and it can analyze this information to determine whether I2C is in a hung state. If the BMC determines that the I2C channel is indeed in a hung state, it can send an I2C reset command to the NVMe SSD through the in-band channel. This reset command is intended to reset the I2C channel and restore it to normal working state.

[0058] For example, due to NVMe SSD hang, physical connection problems, hardware failure or configuration errors, the server cannot obtain key information such as VPD and SMD of the hard disk through the conventional out-of-band management I2C channel. However, in the embodiment of the present application, the BMC bypasses this limitation through the in-band channel and directly checks the working status of the I2C channel. This capability enables the BMC to perform necessary diagnostic and recovery operations even when the NVMe SSD hangs.

[0059] And the existing reset method is just a simple reset or restart, but if it is a problem at the hardware or firmware level, the existing reset method is not enough to solve the problem. Hardware failures are difficult to repair automatically through software means, and usually require manual intervention, such as replacing hardware or reconfiguring the system. In the embodiment of the present application, once the I2C channel is found to be hung, the BMC can try to restore the channel by sending a reset command, reducing the downtime of the NVMe SSD. This automated recovery process reduces the need for manual intervention, thereby further shortening the time interval from problem discovery to resolution, and improving the self-recovery capability and stability of the NVMe SSD.

[0060] In summary, the embodiments of the present application provide a strong guarantee for the stable operation of the target hardware through the inspection mechanism of the out-of-band and in-band channels and the automated recovery process. Even in the face of a fault situation where hardware information cannot be obtained, the method provided by the embodiments of the present application can quickly locate the problem and try to restore the normal operation of the I2C channel, thereby reducing the duration of the fault and improving the stability and reliability of the overall system.

[0061] See also Figure 1 , Figure 1 A schematic diagram of interaction between a server and an NVMe SSD provided in an embodiment of the present application includes a server 110 and an NVMe SSD 113.

[0062] The server 110 includes a baseboard management controller BMC 111 and a PCIe port 112 . The baseboard management controller is responsible for monitoring, managing and restoring the server hardware, including monitoring the SSD. Figure 1 The "PCIe Port" numbered 112 refers to the high-speed serial computer expansion bus standard (Peripheral Component Interconnect Express, PCIe) port. PCIe is a high-speed serial computer expansion bus standard used for connecting hardware components inside a computer. In the context of NVMe SSD, the PCIe port is used to connect the SSD to the server's motherboard.

[0063] The internal communication channel between the server 110 and the NVMe SSD 113 through the interface 119 and the in-band channel 120 is used for high-speed data transmission between the server 110 and the NVMe SSD 113. The external communication channel between the baseboard management controller 111 and the NVMe SSD 113 through the interface 119 and the out-of-band channel 121 is generally used for monitoring and managing the SSD.

[0064] As the management core, BMC111 is responsible for monitoring and managing various hardware components of the system. For example, when BMC accesses NVMe SSD through the out-of-band channel IIC / I2C, if it fails to successfully obtain relevant information or the access to obtain status information times out, BMC can query the logpage storing the I2C status in the NVMe SSD through the in-band channel PCIe. This mechanism ensures that even when the out-of-band channel fails, BMC can still obtain the I2C status information of the NVMe SSD through other channels.

[0065] NVMe SSD113 includes a power supply unit 118, a controller 117, a backup capacitor 115, one or more NAND114 and I2C116. Among them, SOC (System on Chip) is the main control chip in the NVMe SSD, which is responsible for managing data read and write operations, processing commands from the host, and managing the internal NAND flash memory. In the embodiment of the present application, it can also be translated as a controller, that is, the controller 117 in the figure. The full English name of NAND114 is NAND Flash, and the Chinese name is NAND flash memory. It is a non-volatile storage medium based on Flash storage technology. The backup capacitor 115 is used to provide short-term power to the SSD when the power is off to protect data from loss. The power supply is the source of power for the SSD and its internal components.

[0066] In actual application scenarios, the I2C circuit labeled 116 is the circuit part inside the NVMe SSD for implementing I2C communication, including a serial clock line (Serial Clock Line, SCL) and a serial data line (Serial Data Line, SDA). SCL is a clock signal line used to synchronize data transmission. SDA is a signal line used to transmit data on the I2C bus. Both lines can be connected to the power supply VCC through a pull-up resistor to ensure that they remain in a high level state when the bus is idle. The I2C state refers to the state on the I2C bus, that is, the state of the data line SDA and the clock line SCL, which can reflect the communication state of the I2C bus.

[0067] The following describes a fault detection method provided by an embodiment of the present application through an embodiment. Figure 2 ,Should Figure 2 A flowchart of a fault detection method provided in an embodiment of the present application, the method comprising:

[0068] S101. BMC obtains hard disk information through an out-of-band channel.

[0069] The out-of-band channel is a communication method that is independent of the main data communication path and is usually used for management, monitoring, and maintenance purposes. In a server environment, the out-of-band channel allows the BMC to directly access the target hard disk without going through the operating system or the main processor. The target hard disk can be the NVMe SSD to be tested.

[0070] In a possible implementation, the BMC can obtain hard disk information through an out-of-band channel multiple times at a preset acquisition cycle. That is, the BMC can regularly check the status and performance information of the hard disk at a predetermined time interval (such as every hour, every day, etc.), thereby ensuring the health of the hardware and discovering potential problems in a timely manner.

[0071] Exemplarily, the acquisition cycle can be set to a shorter first cycle, such as 2-10 seconds, which is suitable for scenarios that are very sensitive to hardware status changes, such as high-performance computing clusters, real-time data processing systems, etc. These systems need to respond quickly to any hardware failure or performance degradation to avoid data loss or service interruption.

[0072] In another case, the acquisition cycle can be set to a longer second cycle, such as 30-60 seconds or longer, which is suitable for scenarios where real-time monitoring is not required, such as ordinary enterprise servers, file servers, or Web servers. In these systems, the impact of hardware failure or performance degradation may be more moderate, and although the hardware status needs to be monitored, it is usually not necessary to update the information so frequently.

[0073] BMC obtains NVMe SSD related information through out-of-band channels to help administrators discover and solve potential problems in a timely manner and ensure the stable operation of the server. In actual application scenarios, the preset cycle can be adaptively adjusted according to the specific needs of the server, hardware performance, and management strategies.

[0074] S102: When the BMC fails to obtain the hard disk information through the out-band channel, the BMC obtains serial communication bus I2C status information through the in-band channel.

[0075] In-band channels (such as PCIe) are the main data communication paths in the server, used for data transmission and device control. When out-of-band channels (such as IPMI, serial console, etc.) do not work properly or cannot access specific hardware, in-band channels can be used as an alternative communication method for BMC.

[0076] The I2C interface is a widely used low-speed serial communication protocol that allows a processor or microcontroller to communicate with peripheral devices such as sensors, Electrically Erasable Programmable Read-Only Memory (EEPROM), and management chips in NVMe SSDs. In NVMe SSDs, the I2C interface can be used to connect to the controller inside the SSD so that the BMC can monitor and control the status of the SSD.

[0077] In a possible implementation, when the BMC fails to obtain the hard disk information through the out-band channel within the first preset number of times, the BMC obtains the working status of the I2C through the in-band channel. The first preset number of times is the maximum number of times to obtain the hard disk information through the out-band channel. If the BMC still cannot obtain the hard disk information through the out-band channel after multiple attempts, the BMC will switch to the in-band channel and first check the working status of the I2C interface to determine whether the failure to obtain information is caused by an I2C communication problem.

[0078] Exemplarily, the BMC first attempts to obtain the NVMe SSD information through the out-of-band channel according to a preset period. In the first attempt, the NVMe SSD fails to respond to the server's access command and does not provide the relevant information required by the server. This may be caused by hardware failure, firmware problem, or I2C interface failure. The BMC records this failure and prepares for the next attempt.

[0079] Therefore, the BMC tries to obtain the NVMe SSD information through I2C again according to the preset cycle to confirm whether it is caused by a temporary communication problem or a temporary failure of the NVMe SSD. If the SSD fails to respond to the host's access command and still does not feedback the required relevant information as in the first attempt, this further indicates that there may be an I2C communication problem.

[0080] After the first preset number of attempts, if the relevant information of the NVMe SSD is still not obtained through the I2C out-of-band channel, the BMC takes further diagnostic measures. That is, the BMC accesses the logpage reserved in the NVMe SSD through the PCIe in-band channel. This log page stores the I2C status information of the NVMe SSD. The I2C status information may include the current status of the I2C interface, error codes or warning signs, etc.

[0081] The I2C status information is the information that the NVMe SSD stores the current I2C status information in the logpage in advance through the internal detection circuit, waiting for the host to access it. This process allows the server to obtain the I2C status information of the NVMe SSD without relying on the I2C channel that may fail.

[0082] S103 . Determine the fault state of the I2C according to the level state of the data line SDA and the level state of the clock line SCL in the I2C state information.

[0083] When the NVMe SSD IIC is in a hung state, the states of SDA and SCL are always low and always high respectively. For example, when the SDA and SCL of I2C appear in the following situations, it can be detected that I2C is in a hung state.

[0084] The determination method of the fault status of I2C can include the following two situations:

[0085] In the first case, the BMC obtains the SDA level status and SCL level status stored in the logpage, and then determines the fault status of the I2C based on the obtained SDA level status and SCL level status. When the SDA level status in the I2C status information continues to be low and the SCL level status continues to be high for the second consecutive preset number of inspection cycles, it is determined that the I2C is in a dead state; or, when the SDA level status and SCL level status in the I2C status information do not flip within the third consecutive preset number of inspection cycles, it is determined that the I2C is in a dead state.

[0086] In the second case, the SOC of the target hardware pre-marks the I2C status information based on the SDA level status and SCL level status detected multiple times, and stores the marking result of the I2C status information in the target storage area in the target hard disk. The BMC can directly obtain the read marking result stored in the logpage as

[0087] See also Figure 3 , Figure 3 The first interaction diagram between the I2C circuit and the controller inside the NVMe SSD provided in the embodiment of the present application includes an I2C circuit 124 and a SOC 117. SOC 117 is responsible for processing various control logics, including monitoring and control of I2C communication. The I2C circuit 124 is a circuit part inside the NVMe SSD for implementing I2C communication, including a data line SDA 126 and a clock line SDL 125. SDA 126 is a signal line on the I2C bus for transmitting data. SDL 125 is a clock signal line on the I2C bus for synchronous data transmission.

[0088] The controller SOC117 includes an input-output control unit 122. The input-output control unit 122 inside the SOC is responsible for managing the data flow on the SDA and SDL lines to ensure that the data is correctly transmitted between the SSD and the external device. GND127 represents grounding and is a reference level in the circuit. SDA125 and SDL126 can be connected to GND127 through pull-up resistors to ensure that the line remains in a high level state when there is no data transmission. GND123 represents grounding and is a reference level in the circuit. GND123 can be connected to the input-output control unit 122 to ensure the circuit integrity inside the SOC117.

[0089] like Figure 3 As shown, the signals of SDA125 and SDL126 of the I2C circuit are input into SOC117, and SOC117 detects the SDA and SDL signals through the input-output control unit 122. If it is detected that the I2C is in a hung state according to the SDA and SDL signals, SOC117 can write the detected I2C state information into the logpage of NVMeSSD113 through the input-output control unit 122. Thus, in a possible implementation, the system administrator or BMC can diagnose the problem by reading the logpage, take corresponding measures to restore the I2C communication, and / or perform troubleshooting, thereby improving the reliability and maintainability of the NVMe SSD.

[0090] For the judgment logic of whether I2C is in a hung state based on the SDA and SDL signals, please refer to Figure 4 , Figure 4The I2C deadlock level logic diagram provided in the embodiment of the present application. If SDA continues to be at a low level of 0 and SCL continues to be at a high level of 1 within the second preset number of inspection cycles, the SOC or BMC can detect that the I2C is in a deadlock state. The second preset number of times is the maximum number of times to detect whether the SDA level state is continuously at a high level and whether the SCL level state is continuously at a low level. If the SDA level state is still continuously at a high level and the SCL level state is still continuously at a low level after the second preset number of attempts, it is confirmed that the I2C is in a deadlock state.

[0091] When the SDA level state and the SCL level state do not flip within the third preset number of consecutive inspection cycles, the SOC determines that the I2C is in a dead state. The SOC or BMC can detect that the I2C is in a dead state and write the dead state of the I2C into the target storage area. The third preset number of times is the maximum number of times the SDA level state and the SCL level state are detected to see whether they flip. If the SDA level state and the SCL level state still do not flip after the second preset number of attempts, it is confirmed that the I2C is in a dead state.

[0092] In a possible implementation, when the BMC determines that the I2C is in a hung state, an I2C reset command is sent to the target hard disk through an in-band channel.

[0093] The target hard disk can refer to NVMe SSD, and the I2C hang state refers to a device on the I2C bus or the entire bus stops responding or cannot work properly due to some reason (such as hardware failure, software error, communication conflict, etc.). The I2C reset command is used to reset the I2C interface, clear the error state that may cause communication failure, and try to restore the communication function of the NVMe SSD.

[0094] The BMC obtains information from the log page of the NVMe SSD and recognizes that the I2C interface of the NVMe SSD is in a hung state. In order to restore the normal operation of the NVMe SSD, the BMC sends an I2C reset command to the NVMe SSD through the PCIe in-band channel. The NVMe SSD receives and executes the reset command, completes the reset operation, and returns the reset command completion status to the BMC, indicating that the I2C interface has been reset.

[0095] In one possible implementation, during the I2C interface reset operation of the NVMe SSD, if the NVMe SSD I2C is still in a hung state after the reset operation is completed, this usually indicates a more serious hardware or firmware problem, and the BMC can record the relevant logs of this reset failure. The relevant logs of the reset failure contain enough information so that the system administrator can diagnose the problem. Exemplarily, the relevant logs of the reset failure may include the timestamp of the reset operation, the source of the reset command (such as the BMC itself), the detailed steps and results of the reset operation, the status information of the I2C interface (such as whether it responds, error code, etc.), and the status of other related components in the NVMe SSD and the server (such as PCIe channels, SOC, etc.).

[0096] In one possible implementation, during the I2C interface reset operation of the NVMe SSD, if the NVMe SSD I2C is still in a hung state after the reset operation is completed, an alarm can be triggered or the administrator can be notified to intervene. The alarm can be issued in a variety of ways, including but not limited to: system logs (such as Syslog), email notifications, Short Messaging Service (SMS) notifications, Simple Network Management Protocol (SNMP) traps, and displaying warning messages on the management interface of the server. The alarm can include relevant logs of the failed reset operation so that the administrator can quickly identify the severity and urgency of the problem and take appropriate measures.

[0097] In a possible implementation, in the I2C interface reset operation of the NVMe SSD, if after the reset operation is completed, when the BMC confirms that the NVMe SSD I2C status is normal, it switches to using the PCIe VMD interface to obtain information such as the temperature and status of the SSD.

[0098] See also Figure 5 , Figure 5 A second interaction diagram between the I2C circuit and the controller inside the NVMe SSD provided in an embodiment of the present application, wherein the NVMe SSD 113 includes an I2C circuit 132 and a SOC 117 . Figure 5 Output control 129 is used to represent an output control unit inside the SOC, which is responsible for controlling the power supply and reset signals of various hardware components inside the SSD. In the embodiment of the present application, the output control unit is responsible for controlling the power supply of the I2C circuit 132.

[0099] Power 131 refers to a unit that provides power to the I2C circuit 132. This part of the power can be temporarily cut off or restarted during the reset of the I2C circuit. GND 127 represents grounding, which is the reference level in the I2C circuit 132. It can be connected to Power 131, GND 123 in the SOC, and GND 128 inside the NVMe SSD. Multiple GNDs together ensure the stable operation of the entire circuit. During the reset process, GND ensures that the power of the circuit can be correctly disconnected and connected.

[0100] After SOC117 receives the reset command from BMC through PCIe channel, SOC117 triggers the reset mechanism of I2C circuit according to the reset command from BMC, so that the output control unit of SOC realizes reset by controlling the power supply of I2C circuit, such as Figure 3 In the embodiment, the output control unit 129 of the SOC can reset the operation state of the I2C circuit by sending a reset instruction 130 to the power 131 in the I2C circuit. The reset instruction 130 is used to instruct to cut off or restart the power supply unit of the I2C circuit.

[0101] Exemplarily, the reset process may include cutting off the power supply unit of the I2C circuit within a preset time, and then restarting the power supply unit of the I2C circuit to clear abnormal conditions in the circuit that may cause the hang state. In a possible implementation, the power supply unit of the I2C circuit may be cut off for a period of time, and then the power supply unit of the I2C circuit may be restarted, so that the I2C circuit is restored to an initial state.

[0102] See also Figure 6 , Figure 6 The first interaction diagram between the BMC and the NVMe SSD provided in the embodiment of the present application includes the following steps:

[0103] S201. Obtain hard disk information through an out-of-band channel.

[0104] BMC does not directly access server hardware information or control hardware through the server's main processor (Central Processing Unit, CPU) or operating system, but through management networks or interfaces independent of the normal operation of the server, such as the Intelligent Platform Management Interface (Intelligent Platform Management Interface, IPMI) and other out-of-band channels for hardware monitoring, fault diagnosis and remote management. Hard disk information can include hard disk status, model, capacity and other information.

[0105] After receiving the hard disk information acquisition command sent by the BMC, the NVMe SSD fails to return the hard disk information to the BMC, that is, the BMC fails to successfully acquire the hard disk information. In actual application scenarios, this may be due to communication failure, hard disk failure, or BMC configuration problems. In the embodiment of the present application, when the hard disk information cannot be successfully acquired through the out-of-band channel, it can be preliminarily confirmed that the current NVMe SSD status is abnormal.

[0106] S202. Obtain I2C status through an in-band channel.

[0107] If the out-of-band channel fails to obtain hard disk information, the BMC can try to check the status of the I2C bus through the in-band channel.

[0108] S203. Return I2C status information.

[0109] When the NVMe SSD receives the I2C status acquisition command sent by the BMC, the NVMe SSD returns the I2C status information to the BMC. The I2C status information may include the device list on the bus, the communication rate, the error status, etc., which helps to further diagnose the reason why the hard disk cannot be accessed by the out-of-band channel.

[0110] S204: Send a reset command through the in-band channel.

[0111] The BMC sends a reset command to the NVMe SSD through the in-band channel. The reset operation is intended to reset the hardware to its initial state to resolve communication failures or temporary problems.

[0112] S205. Return the reset result.

[0113] After the BMC sends the reset command, the NVMe SSD performs the reset operation. The BMC needs to check whether the reset operation is successfully executed and verify whether the hard disk information can be obtained through the out-of-band channel.

[0114] In a possible implementation, if the reset is successful and the hard disk information can be obtained, the process ends.

[0115] In a possible implementation, if the NVMe SSD still has problems after the reset, further diagnosis or other recovery measures are required, such as replacing hardware or contacting management.

[0116] It should be noted that NVMe SSD has the ability to self-monitor and diagnose during operation, including regular detection of the status of the I2C interface. This regular detection is an important means to ensure the stable operation of the SSD and to promptly discover potential problems. The NVMe SSD has an integrated monitoring circuit to monitor the physical status of the I2C interface, such as the level status of the SCL and SDA signals.

[0117] NVMe includes logpage, which is used to store various status information and statistics of SSD. For I2C status information, NVMe SSD can write the detected I2C status information to the reserved logpage, waiting for BMC to access. BMC can query the log page through the in-band channel and obtain the I2C status information stored in the logpage.

[0118] Different NVMe SSDs may have different hardware designs and firmware implementations, and the detection interval can be adaptively set and adjusted according to actual needs. For example, during high load or critical tasks, the I2C status may need to be detected more frequently to ensure the reliability of communication; while during low load or non-critical tasks, the detection frequency can be reduced to save resources.

[0119] In the embodiment of the present application, the NVMe SSD can periodically and multiple times detect the I2C status, see Figure 7 , NVMe SSD can detect I2C status three times, Figure 7 The second interaction diagram between the BMC and the NVMe SSD provided in the embodiment of the present application includes the following steps:

[0120] S301, feedback the SDA / SCL level status.

[0121] NVMe SSD detects the level status of SDA or SCL according to the inspection cycle, and feeds back the detected level status to SOC to monitor whether the communication is in a normal state and detect whether the levels of SDA and SCL are in the expected state.

[0122] S302, feedback the SDA / SCL level status.

[0123] NVMe SSD performs a second detection of the level status of SDA or SCL according to the inspection cycle, and feeds back the detected level status to SOC to monitor whether the communication is in a normal state and detect whether the levels of SDA and SCL are in the expected state.

[0124] S303, feedback the SDA / SCL level status.

[0125] NVMe SSD performs a third detection of the SDA or SCL level status according to the inspection cycle (at a certain time interval), and feeds back the detected level status to the SOC to monitor whether the communication is in a normal state and whether the SDA and SCL levels are in the expected state. The inspection cycle can be set according to the design requirements and communication requirements of the SSD, aiming to balance the detection frequency and performance consumption.

[0126] S304: If an abnormal SDA / SCL level status is detected within three inspection cycles, the I2C status is written to the log page.

[0127] In normal I2C communication, the levels of SDA and SCL will switch between high and low levels according to the communication protocol. If the level status of SDA or SCL is detected to be abnormal in three consecutive inspection cycles, for example, SDA is continuously at a low level and SCL is continuously at a high level, indicating a possible hang state, this abnormal state will be recorded in the log page of the NVMe SSD. Logpage is a storage area in the SSD for recording various status information and error logs. This is to record abnormal information and wait for the BMC's access command for subsequent analysis and processing.

[0128] S305: Receive the access command sent by the BMC, obtain and determine whether the SDA / SCL level is flipped.

[0129] As a management controller, BMC can send access commands to NVMe SSD through the I2C bus to perform corresponding access tasks, such as querying the SDA / SCL level status.

[0130] When the BMC fails to obtain hardware information through the out-of-band channel, it can further confirm whether there is a problem with the I2C interface. The BMC can send an access command to the NVMe SSD to request the current SDA / SCL level status. After receiving the command, the NVMe SSD can read the SDA / SCL level status again and determine whether these levels have flipped since the most recent detection, that is, whether the SDA / SCL level status has changed.

[0131] S306, feedback that the SDA / SCL level state has not been flipped.

[0132] If the NVMe SSD detects that the level state of SDA / SCL has not flipped since the last detection, that is, the I2C of the NVMe SSD is still in an abnormal state, the monitoring circuit can feed back the abnormal information to the SOC, so that the SOC feeds back the I2C state to the BMC through the interface and the in-band channel.

[0133] S307, BMC accesses NVMe SSD out-of-band, the SDA and SCL level states are not flipped, and the I2C state is written to logpage.

[0134] After confirming that the SDA / SCL level status has not flipped, the BMC accesses the NVMe SSD in an out-of-band manner. The NVMe SSD can write the current I2C status and abnormal level information into the logpage, so that the SDA and SCL level status can be fed back to the BMC through the in-band channel for subsequent troubleshooting, thereby ensuring the reliability of fault information recording and storage.

[0135] The embodiments of the present application have the following beneficial effects:

[0136] NVMe SSD periodically checks the status of the I2C channel. This proactive checking mechanism helps to detect potential communication problems in a timely manner to prevent problems from worsening and leading to larger failures.

[0137] The I2C status monitoring and alarm function is integrated into the firmware of the NVMe SSD, which can regularly monitor the status of the I2C channel and record the monitoring results in the NVMe SSD customized logpage. This not only helps the BMC or other management systems obtain the status information of the I2C channel, but also provides historical data support for troubleshooting.

[0138] An embodiment of the present application further provides a BMC, the BMC comprising a processor and a memory, the memory being used to store a computer program, and the processor being used to execute the computer program to implement the fault detection method as described in the above embodiment.

[0139] An embodiment of the present application further provides a computer program product, characterized in that when the computer program product runs on a computer, the computer executes the fault detection method as described in the above embodiment.

[0140] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0141] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0142] It should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0143] It should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0144] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0145] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0146] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fault detection method, characterized in that: include: The baseboard management controller BMC obtains the hard disk information of the target hard disk through an out-of-band channel; When the BMC fails to obtain the hard disk information through the out-band channel, obtaining serial communication bus I2C status information through the in-band channel; The fault state of the I2C is determined according to the data line SDA level state and the clock line SCL level state in the I2C state information.

2. The method according to claim 1, characterized in that The baseboard management controller BMC obtains hard disk information of the target hard disk through an out-of-band channel, including: The BMC obtains hard disk information of the target hard disk through an out-of-band channel based on an acquisition cycle; When the BMC fails to obtain the hard disk information through the out-of-band channel, obtaining the serial communication bus I2C status information through the in-band channel includes: When the BMC fails to obtain the hard disk information within a first preset number of acquisition cycles, the BMC obtains the I2C status information through an in-band channel.

3. The method according to claim 1, characterized in that The I2C status information is pre-stored in a target storage area, which is an area in the target hard disk used to record hardware internal status and operation information. The I2C status information is determined by: The target hardware detects the data line SDA level state and the clock line SCL level state multiple times according to the inspection cycle; Feedback the SDA level state and SCL level state detected each time to the main controller SOC of the target hardware; The SOC records the SDA level status and the SCL level status detected multiple times into a target storage area in the target hard disk.

4. The method according to claim 3, characterized in that The step of determining the fault state of the I2C according to the data line SDA level state and the clock line SCL level state in the I2C state information includes: When the SDA level in the I2C status information is continuously at a low level and the SCL level is continuously at a high level within a second preset number of consecutive inspection cycles, the BMC determines that the I2C is in a hung state; When the SDA level state and the SCL level state in the I2C state information do not flip within a third preset number of consecutive inspection cycles, the BMC determines that the I2C is in a hung state.

5. The method according to claim 3, characterized in that: The method further comprises: The SOC marks the state information of the I2C based on the SDA level state and the SCL level state detected multiple times, and stores the marking result of the I2C state information in a target storage area in the target hard disk; The SOC marks the status information of I2C based on the SDA level status and SCL level status detected multiple times: When the SDA level in the I2C status information is continuously at a low level and the SCL level is continuously at a high level within a second preset number of consecutive inspection cycles, the SOC marks the I2C status information as a hung state; When the SDA level state and the SCL level state in the I2C state information do not flip within a third preset number of consecutive inspection cycles, the SOC marks the I2C state information as a hung state.

6. The method according to claim 5, characterized in that The step of determining the fault state of the I2C according to the data line SDA level state and the clock line SCL level state in the I2C state information includes: The marking results corresponding to the SDA level state and the SCL level state in the I2C state information are read as the fault state of the I2C.

7. The method according to claim 1, characterized in that The fault state includes a hung state, and the method further includes: When the BMC determines that the I2C is in a hung state, it sends an I2C reset command to the target hard disk through an in-band channel. The I2C reset command is used to instruct the power supply unit of the I2C to power off and then power on again to restore the I2C to a normal working state.

8. The method according to claim 7, characterized in that The method further comprises: After the I2C reset command is executed, the BMC obtains the hard disk information through the out-of-band channel; When the BMC obtains the hard disk information through the out-of-band channel, determining that the I2C channel is restored to a normal operating state; When the BMC cannot obtain the hard disk information through the out-of-band channel, an alarm instruction is generated.

9. A fault detection method, characterized in that: include: The target hard disk receives an acquisition command sent by a baseboard management controller BMC, wherein the acquisition command is a command for the BMC to acquire hard disk information of the target hard disk through an out-of-band channel; When the target hard disk fails to return the hard disk information to the BMC through the out-of-band channel, the serial communication bus I2C status information is returned to the BMC through the in-band channel, so that the BMC determines the fault status of the I2C according to the data line SDA level status and the clock line SCL level status in the I2C status information.

10. The method according to claim 9, characterized in that When the target hard disk fails to return the hard disk information to the BMC through the out-of-band channel, returning the serial communication bus I2C status information to the BMC through the in-band channel includes: When the target hard disk does not return the hard disk information to the BMC through the out-of-band channel, the target hard disk receives the access command sent by the BMC; After receiving the access command, the target hard disk reads the current level state of SDA and the current level state of SCL, and determines whether the current level state of SDA and the current level state of SCL are flipped relative to the level state of SDA and the level state of SCL obtained in the most recent inspection cycle; When the current level state of SDA and the current level state of SCL do not flip, the target hard disk marks the I2C state information as a hung state and writes it into the target storage area, and returns the I2C state information marked as a hung state in the target storage area to the BMC through the in-band channel.

Images