Fault processing method of server, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a server fault processing method, a storage medium and electronic equipment, and the method comprises the steps: obtaining a first detection result, and executing a first target operation on a memory with a fault when the first detection result is that the memory has the fault; in response to the fact that the first detection result is no fault or the first target operation is executed completely, continuing to execute server startup until startup is completed; in response to completion of startup of the server, obtaining a second detection result; second target operation is executed on the memory based on the second detection result, the second target operation comprises generation of reminding information, and the reminding information is used for displaying the fault information to the target user. Through the server fault detection method and device, the problem that the real-time performance of server fault detection is poor in the related technology is solved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of servers, and in particular, to a server fault handling method, a storage medium, and an electronic device. Background Art

[0002] In the related technology, before DDR5 (Double Data Rate 5 fifth-generation double data rate random access memory (SDRAM) technology), the power of the memory was provided by a separate memory power supply circuit on the motherboard. Starting from DDR5, the memory has integrated the power supply module (PMIC chip) into the memory. The PMIC chip on the DDR5 memory refers to the power management integrated circuit used to manage and control the power supply of the DDR5 memory.

[0003] The current server handles memory failures only by having the CPU access the memory's PMIC during the boot process. Under the system, the CPU can be accessed through PECI, allowing the CPU to access the PMIC. However, PECI is a single-line with a low rate, and there is no way to achieve real-time monitoring.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a server fault handling method, a storage medium, and an electronic device to at least solve the problem of poor real-time performance of server fault detection in the related art.

[0006] According to an embodiment of the present application, a method for handling faults of a server is provided, comprising: obtaining a first detection result, the first detection result comprising a fault detected by a processor on a register state of a power management integrated chip during a power-on self-test phase, the processor being used to execute BIOS firmware, and the power management integrated chip being used to manage the power supply of a memory; in response to the first detection result being a fault, performing a first target operation on the memory where the fault occurs, the first target operation comprising disabling a data channel where the memory where the fault occurs is located; in response to the first detection result being no fault or the first target operation being completed, continuing to power on the server until the power-on is completed; in response to the completion of the server power-on, obtaining a second detection result, the second detection result comprising a fault detected by a baseboard controller on a register state of a power management integrated chip during an operating system operation phase; performing a second target operation on the memory based on the second detection result, the second target operation comprising generating a reminder message, the reminder message being used to display the fault information to a target user, the fault information comprising at least one of the following: fault location, fault type, fault level, and recommended fault handling operation.

[0007] In an exemplary embodiment, before acquiring the first detection result, the method includes: inputting first control information into a multiplexer, the first control information being used to control the multiplexer to establish communication between the processor and a communication component of the power management integrated chip.

[0008] In an exemplary embodiment, before acquiring the second detection result, the method includes: inputting second control information into the multiplexer, the second control information being used to control the multiplexer to establish communication between the substrate controller and the communication component of the power management integrated chip.

[0009] In an exemplary embodiment, the method further includes: in response to the first detection result being a fault, generating location information of the faulty memory; generating a fault log based on the location information, the fault log being used to display to a target user, the fault log being generated by a baseboard controller.

[0010] In an exemplary embodiment, after performing the first target operation on the memory having failed, the method further includes: eliminating a fault exception signal generated by a fault register corresponding to the memory having failed.

[0011] In an exemplary embodiment, in response to the server power-on completion, obtaining a second detection result includes: when it is detected that the general input / output pin of the server is a high level signal, the server power-on completion.

[0012] In an exemplary embodiment, after performing the first target operation on the failed memory, the method also includes: powering on the memory in a normal state; initializing the memory after powering on; setting the power management integrated chip of all memories to a write protection mode; controlling the processor to poll the register of the power management integrated chip of the memory after initialization; in response to the processor detecting that there is a fault in the register, transmitting the fault information of the register to the baseboard controller and performing a restart power-on operation on the server; in response to the processor not detecting that there is a fault in the register, continuing to power on the server until the power-on is completed; in response to determining that the server power-on is completed, obtaining at least one of a second detection result and a third detection result, wherein the third detection result is obtained by the processor polling the temperature sensor on the memory module, and the temperature signal of the temperature sensor is used to determine the temperature of the memory; in response to the second detection result being a fault, performing the second target operation on the server; in response to determining that the temperature of the memory exceeds the temperature limit based on the third detection result, performing the second target operation on the server.

[0013] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0014] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0015] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0016] In the embodiment of the present invention, a method of dividing the detection into two times and performing the detection by the processor and the baseboard controller respectively is adopted. By detecting the fault of the processor in the power-on self-test stage and the baseboard controller in the operating system running stage respectively, the purpose of detecting the fault by the processor in the early stage of server startup and by the baseboard controller after the startup is completed is achieved, thereby achieving the technical effect of avoiding the processor and the baseboard controller from competing for the control of the power management integrated chip communication component, and the processor can accurately locate the fault in the power-on self-test stage, and in the operating system running stage, the baseboard controller detects the fault, so there is no need to frequently execute the server power-on again, thereby solving the technical problem of poor real-time performance of server fault detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a hardware structure block diagram of a server device according to a server fault handling method of an embodiment of the present application;

[0018] Figure 2 is a flow chart of a method for handling a server failure according to an embodiment of the present application;

[0019] Figure 3 is a system architecture diagram of a server fault monitoring system according to an embodiment of the present application;

[0020] Figure 4 is a hardware diagram of a baseboard controller detecting a memory power signal according to an embodiment of the present application;

[0021] Figure 5 It is a flowchart of a method for handling server failures according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0024] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a server device of a server fault handling method according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0025] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the fault handling method of the server in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0026] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0027] Specifically, the relevant nouns are explained as follows:

[0028] NGN: Next Generation Network. DDR5: Double Data Rate 5. Fifth generation double data rate random access memory (SDRAM) technology. PMIC: Power Management Integrated Circuit. CPLD: Complex Programmable Logic Device. TSOD: Temperature sensor on DIMM. RCD: Registering Clock Drive. BMC: Baseboard Management Controller. VR: Voltage Regulator. BIOS: Basic Input and Output System. PSU: Power Supply Unit. MUX: Multiplexer. POST: Power On Self-Test. PECI: Platform Environment Control Interface.

[0029] A PMIC chip (Power Management Integrated Circuit) is an integrated circuit that integrates multiple power management functions. It is commonly used in electronic devices to manage and control power supply to provide stable power to various components and subsystems. PMIC chips can implement functions such as power switching, voltage regulation, current control, battery charge and discharge management, and temperature monitoring to ensure the normal operation and safety of the equipment. The design and selection of PMIC chips have an important impact on the power consumption, efficiency, and performance of electronic equipment.

[0030] Before DDR5 (Double Data Rate 5 fifth-generation double data rate random access memory (SDRAM) technology), the power of the memory was provided by a separate memory power supply circuit on the motherboard. Starting from DDR5, the memory has integrated the power supply module (PMIC chip) into the memory. The PMIC chip on the DDR5 memory refers to the power management integrated circuit used to manage and control the power supply of the DDR5 memory. Specifically, the PMIC chip of the DDR5 memory is mainly responsible for providing a stable power supply, power management and monitoring of the DDR5 memory, and protecting the DDR5 memory from power fluctuations and overload problems. Different DDR5 memory products may use different PMIC chips. Common PMIC chip manufacturers include Infineon, Texas Instruments, ADI, etc.

[0031] I3C is a new type of serial bus interface, the full name is Improved Inter-Integrated Circuit (Enhanced Inter-IC Communication Protocol). It was developed by the MIPI Alliance (Mobile Industry Processor Interface) to replace the I2C (Inter-Integrated Circuit) and SMBus (System Management Bus) interfaces. While maintaining backward compatibility with I2C devices, the I3C interface provides higher data transfer rates, lower power consumption, and more functions. It can be used to connect various sensors, memories, and other peripherals, and is widely used in mobile devices, the Internet of Things, automotive electronics, and other fields. The rate of the I3C interface can vary depending on the specific implementation and device support. In general, the maximum rate of the I3C interface can reach 12.5Mbps, which is several times faster than the traditional I2C interface. In addition, I3C also supports multi-host parallel transmission and high-speed data mode, which can further increase the data transfer rate.

[0032] In this embodiment, a method for handling server failure is provided. Figure 2 is a flow chart of a method for handling a server failure according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0033] Step S102, obtaining a first detection result, the first detection result includes a fault detected by the processor on the register state of the power management integrated chip during the power-on self-test phase, the processor is used to execute the BIOS firmware, and the power management integrated chip is used to manage the power supply of the memory; fault detection includes multiple types of detection, such as monitoring the voltage, current, temperature value, etc. of each module of the PMIC, and when at least one of the above parameters exceeds a set threshold, an alarm is issued and a fault is detected. Optionally, the first detection result can also be obtained by the processor polling the temperature sensor (TSOD) of the DIMM during the power-on self-test phase to perform fault detection.

[0034] Step S104, in response to the first detection result being a fault, performing a first target operation on the memory where the fault occurs, the first target operation including disabling a data channel where the memory where the fault occurs is located;

[0035] Step S106, in response to the first detection result being no fault or the first target operation being completed, continuing to start up the server until the startup is completed;

[0036] Step S108, in response to the completion of the server startup, a second detection result is obtained, the second detection result includes a fault detected by the baseboard controller on the register state of the power management integrated chip during the operating system operation stage; fault detection includes multiple types of detection, such as monitoring the voltage, current, temperature value, etc. of each PMIC module, and when at least one of the above parameters exceeds a set threshold, an alarm is issued and a fault is detected. Optionally, the second detection result can also be obtained by the baseboard controller polling the DIMM temperature sensor (TSOD) during the operating system operation stage to perform fault detection.

[0037] Step S110, performing a second target operation on the memory based on the second detection result, the second target operation includes generating reminder information, the reminder information is used to display fault information to the target user, the fault information includes at least one of the following: fault location, fault type, fault level, and recommended fault handling operation.

[0038] Through the above steps, the method of dividing the detection into two times and the two detections are respectively performed by the processor and the baseboard controller, and the fault detection of the processor in the power-on self-test stage and the baseboard controller in the operating system running stage is respectively carried out, so that the purpose of detecting the fault by the processor in the early stage of server startup and detecting the fault by the baseboard controller after the startup is completed is achieved, thereby achieving the technical effect of avoiding the processor and the baseboard controller from competing for the control of the power management integrated chip communication component, and the processor can accurately locate the fault in the power-on self-test stage, and in the operating system running stage, the baseboard controller detects the fault, so there is no need to frequently execute the server power-on again, thereby solving the technical problem of poor real-time performance of server fault detection.

[0039] BIOS (Basic Input / Output System): During the boot phase, BIOS is mainly responsible for hardware initialization and basic function detection. It reads the PMIC registers through the I3C bus to check whether there are any fault records. Once a fault is found, BIOS will take action, such as isolating the faulty memory channel to avoid instability when the system is powered on. BIOS will also set the PMIC registers to write protection mode to prevent unauthorized access. POST (Power-On Self-Test) is a series of hardware check programs executed by BIOS (Basic Input / Output System) after the boot phase to verify whether the basic functions of the server hardware are normal, such as CPU, memory, hard disk, etc. During the POST phase, the BIOS firmware is executed by the processor to access the PMIC registers through the I3C bus, check the PMIC status, and identify and handle potential hardware faults, such as PMIC failure or memory overheating. BIOS, the full name of Basic Input / Output System, is a program stored in a ROM (Read Only Memory) chip on the server motherboard. It contains a series of basic hardware initialization codes and self-test programs, and is the first software to run when the PC system starts. The main responsibilities of BIOS include: BIOS is responsible for identifying and initializing the hardware resources in the system, including CPU, memory, disk, display adapter, etc. BIOS performs the POST process to check the system hardware and verify whether it is in normal working condition. If a hardware failure is found, BIOS will notify the user through a specific error code or sound signal. If the POST process passes, BIOS will read the boot program (usually the boot loader of the operating system) from the preset boot device (such as the hard disk) and hand over control to the boot program to start the operating system.

[0040] The CPU is the "brain" of the computer, responsible for executing instructions and performing calculations. During the POST phase, the CPU's role is mainly reflected in: the CPU reads the BIOS program from the motherboard ROM and executes the initialization and self-test code therein. The CPU will access and test hardware resources according to the BIOS instructions, such as reading the PMIC register through the I3C bus to check the power management status of the memory. If the BIOS detects a hardware failure during the POST process, the CPU will follow the instructions of the BIOS and may stop the boot process or skip the initialization of the faulty hardware so that the system can continue to boot to a usable state.

[0041] BMC (Baseboard Management Controller): After POST is completed and the OS phase is entered, BMC takes over the hardware monitoring responsibilities of the server, including real-time monitoring of the PMIC status register through the I3C bus, and monitoring the temperature information of the TSOD (temperature sensor). BMC monitoring focuses more on the real-time and continuity during operation. It can continuously check the voltage, current, temperature and other indicators of the PMIC. Once an over-threshold situation is found, it will immediately record and alarm. If necessary, it can also notify the CPU through the PECI interface to take temperature-related protection measures.

[0042] When a PMIC fails, the computer will shut down. The current Intel design of BMC (mainboard management controller) cannot accurately detect which memory POWER fail (power failure, power supply failure). The current Intel design only allows the CPU to access the PMIC of the memory during the boot process. Under the system, the CPU can be accessed through PECI to allow the CPU to access the PMIC, but PECI is a single line with a low rate, and there is no way to achieve real-time monitoring. Based on this, this application proposes a method for Intel server monitoring DDR5 PMIC, which can monitor the PMIC of all memories on the server and report the specific location of the failed PMIC without AC reboot (AC power restart).

[0043] like Figure 3 The system architecture diagram of the server fault monitoring system is shown. All devices on the DDR5 DIMM (power management integrated circuit [PMIC], thermal sensor [TS] / temperature sensor [TSOD], register clock driver [RCD]) are connected to the SPD HUB through local I3C*. The SPD HUB is connected to the MUX through the I3C* bus. The MUX is connected to the CPU on one side and the BMC on the other side. The device select of the MUX is connected to the CPLD. In this way, all local DIMM devices (including the SPD HUB) are visible as terminal devices on the SPD I3C* bus. The SPD controller and BMC in the CPU are I3C* master devices, and the control of the CPU and BMC is switched by the CPLD.

[0044] SPD HUB can be understood as Serial Presence Detect Hub, which is a key component on DDR5 DIMM and is used to manage the communication between various devices connected through the I3C bus on the memory module. The role of SPD HUB is somewhat similar to a small communication management center. It receives data from various devices on the DIMM (such as PMIC, TSOD, RCD, etc.), integrates this data and sends it to the CPU or BMC on the motherboard through the I3C bus for processing.

[0045] On DDR5 DIMM, SPD HUB connects to these devices through the local I3C bus. The "local" here means that these connections and communications are limited to the inside of the DIMM module. The I3C bus is an improved I2C bus that can provide higher data transfer rates and richer functions, making it very suitable for high-speed communication between devices inside the module.

[0046] The MUX (multiplexer) acts as a switch for the communication channel, allowing the CPU or BMC to control the I3C bus at different points in time to access the devices on the DIMM. Specifically:

[0047] A port of the MUX that is connected to the CPU on the motherboard. During the POST (Power On Self-Test) phase, the CPU accesses devices such as the PMIC and TSOD on the DIMM through this port for initialization, configuration, and fault detection.

[0048] Another port of the MUX is connected to the BMC (Baseboard Management Controller). During the OS (Operating System) operation phase, the control is switched from the CPU to the BMC, which monitors the PMIC status on the DIMM in real time through this port, including parameters such as voltage, current, and temperature, to ensure the stability and security of the server hardware during operation.

[0049] This design allows the CPU and BMC to use the I3C bus at different time points, avoiding possible communication conflicts when they access the devices on the DIMM at the same time, while also ensuring that the devices on the DIMM can be monitored and managed efficiently and in real time. The switching of the MUX is controlled by the CPLD (complex programmable logic device), further improving the flexibility and reliability of the system's management of DIMM devices.

[0050] Current servers (such as Intel platform servers) obtain memory temperature by polling the TSOD (temperature sensor) of the memory in real time through the CPU. This is automatically implemented on the CPU internal hardware. If the BMC (mainboard management controller) polls the PMIC on the memory through I3C, it will conflict with the CPU. A MUX (Multiplexer) is needed to solve this problem. Since the memory is read very quickly, the MUX has high requirements and is prone to problems, resulting in the CPU or BMC not being able to read the memory temperature. Based on this patent, a method for the BMC to monitor the PMIC and memory temperature is proposed.

[0051] like Figure 4The hardware diagram of BMC detecting memory power signal is shown. The hardware design of current servers (such as Intel platform servers) is that the POWER GOOD signals of every four DIMMs are connected together to form a POWER GOOD signal connected to the CPLD, and the CPLD is connected to the BMC, so that the BMC can detect the POWER GOOD signal of the memory. When the memory is normal, the POWER GOOD signal of the memory is high. When the PMIC fails, the POWER GOOD will be pulled low, and the BMC can detect that the POWER GOOD signal of the memory is pulled low. However, the BMC cannot accurately know which DIMM's memory POWER has a problem.

[0052] In the related art, one limitation of BMC is that when a memory power supply anomaly is detected, it is impossible to specifically locate which memory module (DIMM) has a power failure. This may be because the power good (POWERGOOD) signals of multiple memory modules are aggregated together, and only one overall signal is provided to the BMC, resulting in that when a fault occurs, the BMC can only know that there is a problem with the power supply of a memory module, but cannot tell which specific module it is. The technical solution of the present application realizes the precise location of the fault by using BIOS and BMC for fault detection, which is then used to remind users to quickly replace or repair.

[0053] The communication mechanism between BMC (Baseboard Management Controller) and PMIC (Power Management Integrated Circuit) is, specifically, through the I3C (Improved Inter-Integrated Circuit Communication) bus. In traditional designs, the CPU will periodically poll the health status of the PMIC through the I3C bus, but during system operation, if the PMIC fails and causes unstable memory power supply, AC reboot is usually required to reinitialize the system.

[0054] In the technical solution of the present application, the control ownership of the I3C bus is changed by introducing the control strategy of MUX (multiplexer) and CPLD (complex programmable logic device). In the POST (power-on self-test) stage, the I3C control is held by the CPU, and the CPU detects the status of the PMIC through the BIOS; in the OS (operating system) running stage, the control is switched to the BMC, which can directly poll the registers of each PMIC through the I3C bus to monitor the status of the PMIC in real time, including the voltage, current and temperature of each module. When it is detected that the parameters of a PMIC exceed the preset threshold, indicating that the PMIC has failed, the BMC can immediately record the fault information and locate the specific memory module without the need for an AC reboot of the entire server.

[0055] This method improves the real-time and accuracy of fault detection and reduces the need for AC reboot, thereby avoiding data loss, service interruption, and waste of reboot time that may be caused by AC reboot, providing important technical support for the stable operation and maintenance of the server.

[0056] The PMIC specification stipulates that 11 events will trigger the internal VR disable (voltage regulator shutdown). When the VR disable is triggered, the POWER GOOD signal of the memory will be pulled low, and the CPLD will detect the abnormal voltage of the memory. At this time, the CPLD will shut down the server. When the server is running normally, if the PMIC of a certain memory fails, the server will shut down immediately. If you send a command or manually press the power button to start the server at this time, the server cannot start normally because the POWER GOOD fail state of the memory has not been released and does not conform to the normal power-on logic. Therefore, the server will still shut down at a certain stage of the startup. In the current system design, since the power good signals of the four memory DIMMs are connected together, the BMC has no way to detect and locate which DIMM has a PMIC failure. With the technical solution of the present application, the monitoring of PMIC is divided into two stages. The first is the startup stage (also known as the post stage, power-on self-test). The CPLD gives control to the CPU, and the BIOS detects whether the PMIC has a fault. If there is a fault, it is isolated and reported to the memory. The second stage is the OS stage (immediately following the post stage, which is the operating system running stage). In this stage, the CPLD switches control to the BMC, which monitors the PMIC registers through I3C. This setting enables the CPU and BMC to monitor the PMIC registers in real time. The CPU can accurately locate the address where the fault occurs (generally refers to the slot where the fault occurs), and then shield the channel of the faulty memory to ensure normal boot, while the BMC can implement fault detection in the OS stage without frequent server AC reboots. In DDR5 memory, multiple memory sticks (DIMMs) usually share one or more data channels. If the PMIC of a DIMM fails, for the sake of system stability and performance, it is necessary to disable (that is, stop using) the channel where the faulty DIMM is located, rather than the entire memory system. This is because: the unstable power supply of the faulty memory may cause data read and write errors. By disabling the channel, data corruption or system instability can be prevented. Although disabling a channel will reduce the available memory bandwidth, this method can keep the system running to a certain extent and avoid greater performance loss compared to a complete shutdown or restart. Disabling the faulty memory channel can reduce unnecessary power load and help the system's thermal management and energy efficiency. Through this series of operations, the system can safely isolate and handle power supply failures of the DDR5 memory while minimizing the impact on system performance and ensuring that the server or computer can continue to operate stably.

[0057] In an exemplary embodiment, before acquiring the first detection result, the method includes: inputting first control information into a multiplexer, the first control information being used to control the multiplexer to establish communication between the processor and a communication component of the power management integrated chip.

[0058] In an exemplary embodiment, before acquiring the second detection result, the method includes: inputting second control information into the multiplexer, the second control information being used to control the multiplexer to establish communication between the substrate controller and the communication component of the power management integrated chip.

[0059] A MUX (Multiplexer) is an electronic device or circuit that is mainly used to select one signal from multiple input signals and transmit it to a single output channel. In the hardware architecture of computers and servers, the role of a MUX is to switch the communication path so that multiple devices can share the same bus or communication line, thereby improving the utilization of hardware resources.

[0060] When the MUX is working, it will select which input signal to send to the output according to the control signal. The control signal is an instruction used to specify which input source the MUX should currently select. It is generally a set of binary digits, each of which corresponds to the selection state of an input terminal. For example, in an 8-to-1 MUX, the control signal can be a 3-bit binary number. Depending on the different combinations (000 to 111), the MUX will select one of the 8 input terminals to transmit to the output terminal.

[0061] In the server's PMIC monitoring system, the main role of MUX is to solve the problem of sharing the I3C bus between the CPU and BMC. Both the CPU and BMC need to access the I3C bus to monitor and manage the PMIC and other devices, but due to limited bus resources, simultaneous access may cause data conflicts or communication failures. Using MUX, the control of the I3C bus can be allocated on demand: MUX can ensure that at any given time, only one master device (CPU or BMC) can control the I3C bus, thereby avoiding data conflicts. In the POST stage, the control signal of MUX is set by CPLD (complex programmable logic device), so that the CPU initializes and detects faults of PMIC through the BIOS program. After POST is completed, the control signal switches, and MUX hands control of the I3C bus to the BMC, allowing the BMC to directly monitor the status of the PMIC. Through MUX, the CPU and BMC can achieve orderly access to the I3C bus, ensuring the stability of communication and the integrity of data.

[0062] MUX provides a mechanism in the hardware architecture that enables multiple devices to share limited communication resources, but under the requirements of high performance and high stability, its own limitations and potential problems also need to be noted. Through improved design, such as the BMC monitoring solution proposed in this application, the functions of BMC can be effectively utilized, the complexity and risks introduced by MUX can be reduced, and more efficient and real-time hardware monitoring can be achieved.

[0063] Specifically, POST, short for Power-On Self-Test, is an important stage in the server startup process. It is mainly used to detect and initialize hardware devices to ensure that they are in a stable and operational state. When the server starts, the CPU performs a POST check through the Basic Input and Output System (BIOS). This process includes:

[0064] 1. Hardware detection: BIOS will detect all key hardware components in the system, such as CPU, memory, chipset on the motherboard, input and output ports, hard disk and drive, etc., to ensure that these components can work properly.

[0065] 2. Initialize devices: BIOS will initialize the devices in the system, including setting the memory's operating mode, frequency and voltage, initializing the CPU's cache and bus, etc. At this stage, BIOS will communicate with PMIC (power management integrated circuit) to ensure that the memory's power supply and management are correct.

[0066] 3. Error detection and reporting: BIOS will check the operating status of hardware devices. If any abnormality is found, such as abnormal PMIC status, it will record the error and may report it to the user through BEEP sound, screen information or log file.

[0067] 4. System configuration: The POST process also sets the basic configuration of the system, such as adjusting the system clock, activating security features, etc., in preparation for the operating system to load.

[0068] During the POST phase, I3C control is usually held by the CPU through the BIOS, because the main task of this phase is for the CPU and BIOS to initialize and detect the hardware to ensure that the hardware is in a correct and healthy working state before the server enters the operating system.

[0069] When the POST process is completed, the BIOS hands over control to the operating system (OS), and the server enters normal operation. At this stage:

[0070] 1. Application running: Users can start various applications and services, and the operating system is responsible for scheduling these resources.

[0071] 2. Hardware resource management: The operating system takes over the management of hardware resources, including memory, processor, disk, network interface, etc., and dynamically allocates resources according to the needs of the application.

[0072] 3. Real-time monitoring and maintenance: During the operation of the server, the BMC (baseboard management controller) comes into play. The BMC is an independent microprocessor used to monitor and maintain the environmental conditions and hardware status of the server. During the OS operation phase, the BMC obtains access to the PMIC registers through the I3C bus, can poll the status of the PMIC in real time, monitor its voltage, current and temperature parameters, and ensure the stability and security of the hardware during operation.

[0073] 4. Fault response and recovery: If the BMC detects an abnormality in the PMIC, such as voltage instability, current abnormality, or overheating, it can immediately record the specific fault information, including the precise location of the faulty memory, and alert the administrator through the network or system log. In some cases, the BMC may also take measures to isolate the faulty component to prevent it from affecting the operation of the entire system.

[0074] 5. Remote management and maintenance: BMC supports remote management functions. Even when the OS is not running, the server can be accessed through the network for status monitoring, fault diagnosis and system maintenance.

[0075] During the OS operation phase, since the hardware has been initialized and the system is in a stable working state, the BMC can monitor more carefully and in real time. It replaces the CPU as the master device of the I3C bus, responsible for continuous monitoring and maintenance of the hardware, while the CPU focuses on executing the tasks of the application and operating system. This division of labor improves the overall stability and management efficiency of the system, especially in an environment such as a server that requires 24 / 7 continuous operation, the real-time monitoring function of the BMC is particularly important.

[0076] In an exemplary embodiment, the method further includes: in response to the first detection result being a fault, generating location information of the faulty memory; generating a fault log based on the location information, the fault log being used to display to a target user, the fault log being generated by a baseboard controller.

[0077] In an exemplary embodiment, after performing the first target operation on the memory having failed, the method further includes: eliminating a fault exception signal generated by a fault register corresponding to the memory having failed.

[0078] In an exemplary embodiment, after performing the first target operation on the failed memory, the method also includes: powering on the memory in a normal state; initializing the memory after powering on; setting the power management integrated chip of all memories to a write protection mode; controlling the processor to poll the register of the power management integrated chip of the memory after initialization; in response to the processor detecting that there is a fault in the register, transmitting the fault information of the register to the baseboard controller and performing a restart power-on operation on the server; in response to the processor not detecting that there is a fault in the register, continuing to power on the server until the power-on is completed; in response to determining that the server power-on is completed, obtaining at least one of a second detection result and a third detection result, wherein the third detection result is obtained by the processor polling the temperature sensor on the memory module, and the temperature signal of the temperature sensor is used to determine the temperature of the memory; in response to the second detection result being a fault, performing the second target operation on the server; in response to determining that the temperature of the memory exceeds the temperature limit based on the third detection result, performing the second target operation on the server.

[0079] DIMM is the abbreviation of "Dual In-line Memory Module", which is a memory module standard used in computers and servers. In DDR5 technology, DIMM refers to a memory module that complies with the DDR5 specification, which not only contains DRAM chips (dynamic random access memory) for storing data, but also integrates other important components and circuits to achieve higher performance, lower power consumption and better signal integrity.

[0080] All devices on a DDR5 DIMM typically include the following components: 1. DRAM (Dynamic Random Access Memory) chip: This is the main storage component on the DIMM, responsible for storing and reading data. 2. PMIC (Power Management Integrated Circuit): PMIC is an important new component on the DDR5 DIMM. It is responsible for managing the power supply of the DRAM chip, controlling the voltage and current to meet the strict requirements of the DDR5 specification while reducing power consumption and improving energy efficiency. 3. TSOD (Temperature Sensor): Located on the memory module, it is used to monitor the temperature of the DRAM chip to prevent overheating from damaging the hardware or affecting performance. 4. RCD (Register Clock Driver): Used to improve signal quality and distribution of clock signals to ensure accuracy and high speed of data transmission. 5. SPD (Serial Presence Detect) chip: Stores configuration information about the memory module, such as capacity, speed, and manufacturer data, so that the system BIOS can read this information at startup and configure the memory correctly. 6. I3C bus interface: I3C is an improved I2C (Inter-Integrated Circuit) bus technology used to transmit data and control signals between devices on the DIMM and the motherboard. I3C provides higher transfer speeds and better signal integrity, supporting communication between devices on the DIMM and the motherboard management controller or CPU.

[0081] These components and circuit designs of DDR5 DIMM together form a complex memory module system that works closely together to support the high-speed data transmission, low power consumption and high reliability of DDR5 technology. By integrating PMIC, TSOD and RCD on DIMM, DDR5 technology can manage the power and signal of memory more efficiently, reduce the circuit complexity on the motherboard, and provide a more stable memory environment.

[0082] like Figure 5 As shown, a flowchart of a method for handling server failures is provided.

[0083] During the boot phase, BIOS needs to access PMIC and other devices on the memory, such as TSOD, so the control of I3C must be handed over to the CPU. After the boot is completed and the OS is entered, the CPU no longer accesses the PMIC, and the control can be handed over to the BMC, which monitors the PMIC and memory. The main steps of BIOS monitoring the PMIC during the boot process are as follows:

[0084] Step 1: Power on the AC, start the server, and the PMIC enters the configuration mode, which is not the write protection mode.

[0085] Step 2: BIOS polls the PMIC registers of all DIMMs through I3C to check whether there is a PMIC failure. Because after a PMIC failure, the error information will be recorded in the non-volatile registers, so by checking these registers, it can be determined whether there is a PMIC failure.

[0086] Specifically, polling is a method of computer communication and data acquisition in which the central processing unit (CPU) or other master device periodically or continuously checks the status or data of slave devices (such as sensors, memory, peripherals, etc.) to determine whether there is new information or events that need to be processed. In polling mode, the master device actively initiates queries instead of waiting for the slave device to send an interrupt signal to notify the status change.

[0087] During operation, the CPU will periodically access the temperature sensor (TSOD) on the memory to read the current temperature information. TSOD is a temperature sensor embedded in the DDR memory module, which is used to monitor the temperature of the memory to prevent performance degradation or hardware damage caused by overheating.

[0088] In order to optimize resource utilization and avoid bus conflicts between the CPU and BMC, a MUX (multiplexer) is used to switch control of the I3C bus at different stages, allowing the CPU to poll during the POST stage, while the BMC performs real-time monitoring and polling during the OS running stage.

[0089] In this way, both the CPU and BMC can effectively monitor memory temperature, but the BMC's monitoring is more real-time and detailed, and can respond to temperature anomalies in a timely manner without interrupting the normal operation of the system, thereby improving the stability and security of the system.

[0090] In an optional embodiment, the CPU may poll the TSOD of the memory in real time, and the specific steps may include:

[0091] 1. Initialization: When the system starts, the CPU will initialize the communication of TSOD, including setting bus parameters, confirming the address and configuration status of the sensor.

[0092] 2. Query cycle: The CPU sets a query cycle, which can be fixed, such as once per second, or dynamically adjusted according to system load and temperature changes.

[0093] 3. Query command: In each query cycle, the CPU sends a query command to the TSOD through I2C (Inter-Integrated Circuit), I3C or other communication buses.

[0094] 4. Data reading: After receiving the command, TSOD will return the current temperature data to the CPU through the bus.

[0095] 5. Status assessment: After receiving the temperature data, the CPU will compare it with the preset threshold to determine whether the temperature is normal. If the temperature is outside the normal range, the CPU may need to take measures such as adjusting the cooling system, reducing system performance or generating a warning message.

[0096] 6. Loop Repeat: The CPU will continue to repeat this process until the system is shut down or the query mechanism is interrupted by other higher priority tasks.

[0097] Step 3: If a certain memory PMIC is determined to be faulty, the BIOS will send the specific information of the memory where the PMIC fault occurred and the PMIC register information to the BMC. After receiving the information, the BMC will generate a log and issue an alarm. In this way, the user can locate the specific slot and replace the memory. The BIOS sets the faulty PMIC register, removes the power good abnormal state, and disables the channel where the faulty memory is located to isolate the faulty memory.

[0098] Step 4: If there is no fault or the BIOS has shielded the faulty memory, the BIOS enables the VR of all normal memory so that the memory can be powered on normally.

[0099] Step 5: Initialize memory.

[0100] Step 6: Initialize all normal memories. After initialization is complete, set the PMIC of all memories to write protection mode.

[0101] Step 7: BIOS polls the PMIC registers of all memories again to check if there is any fault in the memory PMIC. If there is a fault, the information of the faulty memory is sent to the BMC, and the BMC performs an AC reboot and starts again from step 1. If there is no fault, continue to boot up.

[0102] Step 7: When BIOS POST is completed, set the GPIO (general purpose input output pin). This GPIO is set to low level by default. When POST is completed, BIOS sets this GPIO to high level as a sign of BIOS POST completion.

[0103] Step 8: When CPLD detects that GPIO is high, CPLD switches the control of I3C to BMC. From this point on, BMC starts to monitor PMIC.

[0104] After POST completes and enters OS, BMC monitors PMIC.

[0105] When POST is completed, the control of I3C is switched from the CPU to the BMC, and the BMC starts polling the PMIC and TSOD to monitor the PMIC registers and obtain the memory temperature.

[0106] According to the JEDEC specification, PMIC has 256-byte registers (0x0-0xFF), of which 0-0x3F is the host region, 0x40-0x6F is the DIMM Vendor region, and 0x70-0xFF is the Vendor Specific Region.

[0107] The host region is an area that the host can access and can read and write registers.

[0108] DIMM Vendor region and Vendor Specific Region are PMIC manufacturer-specific protection areas. Registers in these two areas cannot be read or written without a password.

[0109] What BMC needs to monitor is the Status Register of the host region. The registers in this region are divided into four types:

[0110] 1. Global History of Error Log Register: This register records the history of the PMIC status for each abnormal power-off cycle. When the PMIC generates a VRDisable command due to a fault, the PMIC will write to this register.

[0111] 2. Error Log Registers: These registers record the PMIC status for each power-off sequence. When the PMIC self-generates a VR Disable command due to a fault, it writes to this register by itself. These registers will be updated during the power-off cycle.

[0112] 3. Real-time Status Registers: Once any event occurs, the corresponding real-time register bit will be written to 1. If the event is resolved, the corresponding register bit will remain 1 and will not be cleared until the PMIC receives a command to clear the register.

[0113] 4. Periodic Status Registers: These registers are dynamically updated at a specific frequency by the PMIC, and they indicate the current status, mainly the current, voltage, and temperature status of each module inside the PMIC.

[0114] BMC mainly monitors Periodic Status Registers and Error Log Registers. BMC polls all memories in real time, obtains the voltage, current, and temperature of each module inside the PMIC, and reads the thresholds of voltage, current, and temperature from the registers. When an indicator exceeds the threshold, a log is recorded and an alarm is issued. When an event that triggers VR disable is detected, a serious alarm is recorded on the BMC web interface, and the user is prompted to replace the faulty memory of the PMIC. By monitoring the voltage, current, and temperature status of each PMIC module in real time, various information about the faulty memory of the PMIC can be reported immediately without performing an AC reboot.

[0115] When POST is completed and enters the OS, in addition to polling the memory's PMIC, the memory's TSOD must also be polled in real time, that is, the memory temperature must be obtained in real time. When overtemperature occurs, the CPU memory temperature-related registers are actively written through PECI. When the overtemperature is relieved, the CPU memory temperature-related registers are actively cleared through PECI.

[0116] By adopting the technical solution of the present application, a MUX is added to the hardware to switch the control rights of the CPU and BMC I3C. The control right is given to the CPU in the POST stage and to the BMC after the POST is completed. The BMC trains the memory TSOD and PMIC in turn through I3C to avoid the risks that may be caused by the CPU and BMC competing for the control right of I3C. The BMC monitors the voltage, current and temperature values ​​of each PMIC module by polling the memory PMIC. When the threshold is exceeded, an alarm is issued and the detailed information is recorded. In this way, when a PMIC failure occurs, the fault information can be accurately recorded, and there is no need to perform an AC reboot again. The BIOS detects the failure and reports it to the BMC. This method has better real-time performance and is more convenient.

[0117] In an optional embodiment, at the initial stage of server startup, after AC is powered on, PMIC will enter non-write protection mode, i.e., configuration mode, allowing BIOS to perform initialization and configuration. Non-write protection mode means that the registers of PMIC can be read and written, which is a prerequisite for BIOS to check and set the status of PMIC.

[0118] BIOS polls PMIC registers via I3C: BIOS calls the CPU to access the PMIC registers of each DIMM via the I3C bus to check if any fault information is recorded. The "BIOS" here refers to the BIOS program running on the CPU and communicating with the PMIC through the CPU.

[0119] Fault diagnosis and reporting: If a PMIC fault is detected, the BIOS reports the detailed information of the faulty memory and the PMIC register status to the BMC. The BMC then records a log and generates an alarm to guide the user to locate the faulty memory.

[0120] Fault isolation and normal memory configuration: BIOS sets the faulty PMIC registers, removes the abnormal state (such as removing the disabled voltage regulator), and disables or isolates the channel where the faulty memory is located to ensure that the system can run stably. Then, BIOS enables the voltage regulators of all normal memories, initializes and powers on the normal memories.

[0121] The POST completion and control switching process is as follows: GPIO set: When the BIOS completes POST (power-on self-test), it sets the GPIO pin state to a high level as a sign of POST completion. CPLD control switching: After the CPLD detects that the GPIO is high, it switches the control of I3C from the CPU to the BMC, turning on the BMC's real-time monitoring of the PMIC.

[0122] The monitoring process of BMC on PMIC during OS operation phase is as follows: After entering OS operation phase, BMC becomes the master controller of I3C bus and is responsible for real-time monitoring of PMIC status and memory temperature.

[0123] The monitoring process of BMC's PMIC monitoring is as follows: BMC polls PMIC's Periodic Status Registers and Error Log Registers through the I3C bus. These registers contain the real-time status of the PMIC module's voltage, current, and temperature, as well as past error information. BMC reads the voltage, current, and temperature thresholds in the registers. Once an indicator exceeds the threshold, it will immediately record the log and issue an alarm to the user. In particular, if an event that will cause VR disable is detected, BMC will record a serious alarm in its web interface and prompt the user to replace the faulty PMIC memory.

[0124] In addition to PMIC monitoring, the BMC also polls the TSOD (temperature sensor) on the memory in real time to obtain memory temperature information. If overtemperature is detected, the BMC will send a signal to the CPU through the PECI (Platform Environment Control Interface) bus to trigger the CPU's temperature protection mechanism, such as reducing system performance or shutting down the system to prevent the device from overheating and damage. When the overtemperature state is removed, the BMC will also clear the temperature-related protection state in the CPU through PECI and restore normal system operation.

[0125] Through the above process, the server can ensure that the PMIC and memory temperatures are effectively monitored and managed during the startup and operation stages, thereby improving the stability and security of the system.

[0126] The execution subject of the above steps may be a server, a terminal, etc., but is not limited thereto.

[0127] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0128] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0129] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0130] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0131] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0132] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0133] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0134] The embodiments of the present application also provide a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any one of the above method embodiments.

[0135] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0136] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0137] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for handling server failures, characterized in that: include: Obtaining a first detection result, the first detection result comprising a fault detected by a processor on a register state of a power management integrated chip during a power-on self-test phase, the processor being used to execute BIOS firmware, and the power management integrated chip being used to manage power supply to a memory; In response to the first detection result being a fault, performing a first target operation on the memory where the fault occurs, wherein the first target operation includes disabling a data channel where the memory where the fault occurs is located; In response to the first detection result being no fault or the first target operation being completed, continuing to start up the server until the startup is completed; In response to determining that the server is powered on, obtaining a second detection result, wherein the second detection result includes a fault detected by the baseboard controller on the register state of the power management integrated chip during the operation of the operating system; Based on the second detection result, a second target operation is performed on the server, and the second target operation includes generating reminder information, and the reminder information is used to display fault information to the target user. The fault information includes at least one of the following: fault location, fault type, fault level, and recommended fault handling operation.

2. The method according to claim 1, characterized in that Before obtaining the first detection result, the method includes: First control information is input to the multiplexer, where the first control information is used to control the multiplexer to establish communication between the processor and the communication component of the power management integrated chip.

3. The method according to claim 1 or 2, characterized in that: Before obtaining the second detection result, the method further includes: Second control information is input to the multiplexer, and the second control information is used to control the multiplexer to establish communication between the substrate controller and the communication component of the power management integrated chip.

4. The method according to claim 1, characterized in that: The method further comprises: In response to the first detection result being that there is a fault, generating location information of a memory having the fault; A fault log is generated based on the location information. The fault log is used to be displayed to a target user. The fault log is generated by a baseboard controller.

5. The method according to claim 1, characterized in that After performing the first target operation on the failed memory, the method further includes: Eliminate the fault exception signal generated by the fault register corresponding to the faulty memory.

6. The method according to claim 1, characterized in that In response to determining that the server startup is complete, obtaining a second detection result includes: When it is detected that the general input / output pin of the server is a high level signal, it is determined that the server startup is completed.

7. The method according to claim 1, characterized in that After performing the first target operation on the failed memory, the method further includes: Power on the memory in normal state; Initialize the memory after power-on; Set all memory power management integrated chips to write protection mode; The control processor polls the register of the power management integrated chip of the memory after the initialization is completed; In response to the processor detecting that a register has a fault, transmitting fault information of the register to the baseboard controller and performing a restart and power-on operation on the server; In response to the processor not detecting that there is a fault in the register, continuing to power on the server until the power on is completed; In response to determining that the server is powered on, obtaining at least one of the second detection result and the third detection result, wherein the third detection result is obtained by the processor polling a temperature sensor on the memory module, and a temperature signal of the temperature sensor is used to determine the temperature of the memory; In response to the second detection result being a fault, performing the second target operation on the server; In response to determining that the temperature of the memory exceeds the temperature limit based on the third detection result, the second target operation is performed on the server.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images