Server failure processing method, storage medium, and electronic device

Abstract

Embodiments of the present application provide a server fault processing method, a storage medium and an electronic device, and the method comprises: obtaining a first detection result, in response to the first detection result being with a fault, performing a first target operation on a memory with a fault; in response to the first detection result being without a fault or the first target operation being completed, continuing to perform server startup to completion of startup; in response to completion of the server startup, obtaining a second detection result; and performing a second target operation on the memory based on the second detection result, the second target operation comprising generating a prompt information, the prompt information being used to show fault information to a target user. Through the present application, the problem of poor real-time performance of server fault detection in the related art is solved.

Description

Technical Field

[0001] This application relates to the field of servers, and more specifically, to a server fault handling method, storage medium, and electronic device. Background Technology

[0002] In related technologies, prior to DDR5 (Double Data Rate 5, fifth generation double data rate random access memory (SDRAM) technology), the power of the memory was provided by a separate memory power supply circuit on the motherboard. Starting with DDR5, the power supply module (PMIC chip) is integrated into the memory. The PMIC chip on DDR5 memory refers to the power management integrated circuit used to manage and control the power supply of DDR5 memory.

[0003] Current servers handle memory failures only by having the CPU access the memory's PMIC during boot. Under system conditions, the CPU can be accessed via PECI, allowing the CPU to access the PMIC. However, PECI is single-threaded, has low speed, and cannot achieve real-time monitoring.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a server fault handling method, storage medium, and electronic device to at least solve the problem of poor real-time performance of server fault detection in related technologies.

[0006] According to one embodiment of this application, a server fault handling method is provided, comprising: acquiring a first detection result, the first detection result including a fault detected by the processor during the power-on self-test phase based on the register state of the power management integrated chip, the processor being used to execute BIOS firmware, and the power management integrated chip being used to manage the power supply to the memory; in response to the first detection result indicating a fault, performing a first target operation on the faulty memory, the first target operation including disabling the data channel where the faulty memory is located; in response to the first detection result indicating no fault or the first target operation being completed, continuing to power on the server until power-on is complete; in response to the server power-on being complete, acquiring a second detection result, the second detection result including a fault detected by the baseboard management controller during the operating system running phase based on the register state of the power management integrated chip; and performing a second target operation on the memory based on the second detection result, the second target operation including generating a reminder message, the reminder message being used to display fault information to a target user, the fault information including at least one of the following: fault location, fault type, fault level, and suggested fault handling operation.

[0007] In an exemplary embodiment, before obtaining a first detection result, the method includes: inputting first control information to a multiplexer, the first control information being used to control the multiplexer to establish communication between the processor and a communication component of a power management integrated chip.

[0008] In one exemplary embodiment, before obtaining the second detection result, the method includes: inputting second control information to a multiplexer, the second control information being used to control the multiplexer to establish communication between the substrate management controller and the communication components of the power management integrated chip.

[0009] In one exemplary embodiment, the method further includes: generating location information of the faulty memory in response to a first detection result indicating a fault; generating a fault log based on the location information, the fault log being displayed to a target user, the fault log being generated by a baseboard management controller.

[0010] In one exemplary embodiment, after performing the first target operation on the faulty memory, the method further includes: eliminating the fault exception signal generated by the fault register corresponding to the faulty memory.

[0011] In one exemplary embodiment, in response to the server power-on completion, a second detection result is obtained, including: the server power-on is completed when a high-level signal is detected on a general-purpose input / output pin of the server.

[0012] In an exemplary embodiment, after performing a first target operation on the faulty memory, the method further includes: powering on the memory in a normal state; initializing the powered-on memory; setting the power management integrated circuits of all memory modules to write-protected mode; controlling the processor to poll the registers of the power management integrated circuits of the memory modules after initialization; in response to the processor detecting a fault in the register, transmitting the fault information of the register to the baseboard management controller and performing a restart power-on operation on the server; in response to the processor not detecting a fault in the register, continuing to power on the server until power-on is complete; in response to determining that the server power-on is complete, acquiring at least one of a second detection result and a third detection result, wherein the third detection result is obtained by the baseboard management controller polling the temperature sensor on the memory module, and the temperature signal of the temperature sensor is used to determine the temperature of the memory; in response to the second detection result indicating a fault, performing a second target operation on the server; and in response to determining that the temperature of the memory exceeds a temperature limit based on the third detection result, performing a second target operation on the server.

[0013] According to yet another embodiment of this application, a computer-readable storage medium is also provided, in which a computer program is stored, wherein the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0014] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein a computer program is stored in the memory and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0015] According to yet another embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0016] In this embodiment of the invention, a method of two separate detections, one by the processor and the other by the baseboard management controller, is adopted. By detecting faults in the processor during the power-on self-test (POST) phase and in the baseboard management controller during the operating system operation phase, the goal of detecting faults by the processor in the early stage of server startup and by the baseboard management controller after startup is achieved. This avoids the processor and baseboard management controller competing for control of the power management integrated chip communication components. Furthermore, the processor can accurately locate faults during the POST phase, and the baseboard management controller detects faults during the operating system operation phase, thus eliminating the need for frequent server power-on re-energization and solving the technical problem of poor real-time performance in server fault detection. Attached Figure Description

[0017] Figure 1 This is a hardware structure block diagram of a server device for a server fault handling method according to an embodiment of this application; Figure 2 This is a flowchart of a server fault handling method according to an embodiment of this application; Figure 3 This is a system architecture diagram of a server fault monitoring system according to an embodiment of this application; Figure 4 This is a hardware diagram of a baseboard management controller detecting a memory power signal according to an embodiment of this application; Figure 5 This is a flowchart of a server fault handling method according to an embodiment of this application. Detailed Implementation

[0018] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0020] The methods and embodiments provided in this application can be executed on a server device or a similar computing device. Taking running on a server device as an example, Figure 1 This is a hardware structure block diagram of a server device for a server fault handling method according to an embodiment of this application. For example... Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the image. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The server device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the server equipment described above. For example, the server equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0021] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the server fault handling method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the server device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0022] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the server device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0023] Specifically, the relevant terms are explained as follows: NGN: Next Generation Network. DDR5: Double Data Rate 5 (SDRAM) technology. PMIC: Power Management Integrated Circuit. CPLD: Complex Programmable Logic Device. TSOD: Temperature sensor on DIMM. RCD: Registering Clock Drive. BMC: Baseboard Management Controller. VR: Voltage Regulator. BIOS: Basic Input and Output System. PSU: Power Supply Unit. MUX: Multiplexer. POST: Power On Self-Test. PECI: Platform Environment Control Interface.

[0024] A PMIC (Power Management Integrated Circuit) is an integrated circuit that integrates multiple power management functions. It is commonly used in electronic devices to manage and control the power supply, providing a stable power source to various components and subsystems. PMIC chips can perform functions such as power switching, voltage regulation, current control, battery charge / discharge management, and temperature monitoring to ensure the normal operation and safety of the device. The design and selection of PMIC chips have a significant impact on the power consumption, efficiency, and performance of electronic devices.

[0025] Prior to DDR5 (Double Data Rate 5 SDRAM), memory power was supplied by a separate memory power supply circuit on the motherboard. Starting with DDR5, the power supply module (PMIC chip) is integrated into the memory module. The PMIC chip on DDR5 memory refers to the power management integrated circuit used to manage and control the power supply to the DDR5 memory. Specifically, the DDR5 memory PMIC chip is mainly responsible for providing a stable power supply, managing and monitoring the power supply to the DDR5 memory, and protecting the DDR5 memory from power fluctuations and overloads. Different DDR5 memory products may use different PMIC chips; common PMIC chip manufacturers include Infineon, Texas Instruments, and Analog Devices.

[0026] I3C is a new type of serial bus interface, short for Improved Inter-Integrated Circuit. Developed by the MIPI (Mobile Industry Processor Interface) consortium, it aims to replace I2C (Inter-Integrated Circuit) and SMBus (System Management Bus) interfaces. While maintaining backward compatibility with I2C devices, the I3C interface offers higher data transfer rates, lower power consumption, and more functionality. It can be used to connect various sensors, memories, and other peripherals, and is widely used in mobile devices, the Internet of Things (IoT), automotive electronics, and other fields. The speed of the I3C interface varies depending on the specific implementation and device support. Generally, the maximum speed of the I3C interface can reach 12.5 Mbps, several times faster than the traditional I2C interface. Furthermore, I3C supports multi-master parallel transmission and high-speed data modes, which can further improve data transfer rates.

[0027] This embodiment provides a server fault handling method. Figure 2 This is a flowchart of a server fault handling method according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps: Step S102: Obtain the first detection result. The first detection result includes faults detected by the processor during the power-on self-test (POST) phase by checking the register status of the power management integrated chip (PMIC). The processor is used to execute the BIOS firmware, and the PMIC is used to manage the power supply to the memory. Fault detection includes various types of detection, such as monitoring the voltage, current, and temperature values ​​of each module of the PMIC. When at least one of the above parameters exceeds a set threshold, an alarm is triggered and a fault is detected. Optionally, the first detection result can also be obtained by the processor polling the temperature sensor (TSOD) of the DIMM during the POST phase for fault detection.

[0028] Step S104: In response to the first detection result indicating a fault, a first target operation is performed on the faulty memory. The first target operation includes disabling the data channel where the faulty memory resides. Step S106: In response to the first detection result being fault-free or the first target operation being completed, continue executing the server startup until startup is complete; Step S108: In response to the server power-on completion, a second detection result is obtained. The second detection result includes faults detected by the baseboard management controller during the operating system runtime by monitoring the register status of the power management integrated chip. Fault detection includes various types of detection, such as monitoring the voltage, current, and temperature values ​​of each module of the PMIC. When at least one of the above parameters exceeds a set threshold, an alarm is triggered and a fault is detected. Optionally, the second detection result can also be obtained by the baseboard management controller polling the temperature sensor (TSOD) of the DIMM during the operating system runtime to perform fault detection.

[0029] Step S110: Perform a second target operation on the memory based on the second detection result. The second target operation includes generating a reminder message. The reminder message is used to display fault information to the target user. The fault information includes at least one of the following: fault location, fault type, fault level, and suggested fault handling operation.

[0030] Through the above steps, a two-stage fault detection method is adopted, with the processor and the baseboard management controller performing the detections separately. By detecting faults in the processor during the power-on self-test (POST) phase and in the baseboard management controller during the operating system runtime phase, the goal of detecting faults by the processor before the server boots up and by the baseboard management controller after booting up is achieved. This avoids the processor and baseboard management controller competing for control of the power management integrated chip communication components. Furthermore, the processor can accurately locate faults during the POST phase, and the baseboard management controller detects faults during the operating system runtime, thus eliminating the need for frequent server power-on re-energization and solving the technical problem of poor real-time server fault detection.

[0031] BIOS (Basic Input / Output System): During the power-on phase, the BIOS is primarily responsible for hardware initialization and basic function testing. It reads the PMIC registers via the I3C bus to check for fault records. If a fault is detected, the BIOS takes action, such as isolating faulty memory channels to prevent system instability upon power-up. The BIOS also sets the PMIC registers to write-protected mode to prevent unauthorized access. POST (Power-On Self-Test) is a series of hardware checks executed by the BIOS after the power-on phase. It verifies the basic functionality of the server hardware, such as the CPU, memory, and hard drive. During POST, the BIOS firmware is executed by the processor to access the PMIC registers via the I3C bus, check the PMIC status, identify and handle potential hardware faults, such as PMIC failures or memory overheating. The BIOS, short for Basic Input / Output System, is a program stored in a ROM (Read-Only Memory) chip on the server motherboard. It contains a series of basic hardware initialization codes and self-test programs and is the first software to run when the PC system starts. The main responsibilities of the BIOS include: Identifying and initializing system hardware resources, including the CPU, memory, disk, and display adapter. The BIOS performs the POST (Power-On Self-Test) process to check the system hardware and verify its normal operating condition. If a hardware fault is detected, the BIOS will notify the user through a specific error code or audible signal. If the POST process succeeds, the BIOS will read the bootloader (usually the operating system's boot loader) from the preset boot device (such as the hard drive) and transfer control to the bootloader, thereby starting the operating system.

[0032] The CPU is the "brain" of a computer, responsible for executing instructions and performing calculations. During the POST (Power-On Self-Test) phase, the CPU's main functions are: The CPU reads the BIOS program from the motherboard ROM and executes its initialization and self-test code. The CPU accesses and tests hardware resources according to BIOS instructions, such as reading the PMIC registers via the I3C bus to check the memory's power management status. If the BIOS detects a hardware fault during POST, the CPU will follow the BIOS's instructions, potentially halting the boot process or skipping the initialization of the faulty hardware, allowing the system to continue booting to a usable state.

[0033] BMC (Baseboard Management Controller): After POST (Power-On Post-Processing) completes and enters the OS phase, the BMC takes over the server's hardware monitoring responsibilities. This includes real-time monitoring of the PMIC's status registers via the I3C bus and monitoring the temperature information from the TSOD (Temperature Sensor). The BMC's monitoring focuses on real-time and continuous operation. It continuously checks the PMIC's voltage, current, and temperature, and immediately records and alarms any exceeding of thresholds. If necessary, it can also notify the CPU to implement temperature-related protection measures via the PECI interface.

[0034] When a PMIC malfunctions, the system shuts down. Current Intel designs cannot accurately detect which memory module's power failure (POWER Failure) is being detected. Current Intel designs only allow the CPU to access the memory PMIC during startup. Under system load, the CPU can access the PMIC via PECI, but PECI is single-wire and has low speed, making real-time monitoring impossible. Therefore, this application proposes a method for monitoring DDR5 PMICs in Intel servers. This method can monitor all memory PMICs on the server and report the specific location of the faulty PMIC without requiring an AC reboot.

[0035] like Figure 3 The system architecture diagram of the server fault monitoring system is shown. All devices on the DDR5 DIMM (Power Management IC [PMIC], Thermal Sensor [TS] / Temperature Sensor [TSOD], Register Clock Driver [RCD]) are connected via a local I3C. Connect to the SPD HUB. The SPD HUB connects via I3C. The bus connects to the MUX, which connects to the CPU on one side and the BMC on the other. The MUX's device select is connected to the CPLD. In this way, all local DIMM devices (including the SPD hub) function as SPD I3Cs. The terminal devices on the bus are visible. The SPD controller and BMC in the CPU are I3C. The main device allows the CPLD to switch control of the CPU and BMC.

[0036] SPD HUB can be understood as Serial Presence Detect Hub. It is a key component on DDR5 DIMMs, used to manage communication between various devices connected to the memory module via the I3C bus. The SPD HUB functions somewhat like a small communication management center. It receives data from various devices on the DIMM (such as PMIC, TSOD, RCD, etc.), integrates this data, and sends it to the CPU or BMC on the motherboard for processing via the I3C bus.

[0037] On DDR5 DIMMs, the SPD HUB connects to these devices via a local I3C bus. "Local" here means that these connections and communications are limited to within the DIMM module. The I3C bus is an improved version of the I2C bus, offering higher data transfer rates and richer functionality, making it ideal for high-speed communication between devices within the module.

[0038] The MUX (Multiplexer) acts as a switcher for communication channels, allowing the CPU or BMC to control the I3C bus at different times to access devices on the DIMM. Specifically: One of the ports of the MUX connects to the CPU on the motherboard. During the POST (Power On Self-Test) phase, the CPU uses this port to access devices such as the PMIC and TSOD on the DIMM for initialization, configuration, and fault detection.

[0039] Another port of the MUX connects to the BMC (Baseboard Management Controller). During OS (Operating System) operation, control switches from the CPU to the BMC. The BMC uses this port to monitor the PMIC status on the DIMM in real time, including parameters such as voltage, current, and temperature, to ensure the stability and security of the server hardware during operation.

[0040] This design allows the CPU and BMC to use the I3C bus at different times, avoiding communication conflicts that may occur when they simultaneously access devices on the DIMM. It also ensures that devices on the DIMM can be efficiently and in real-time monitored and managed. Controlling the switching of the MUX via a CPLD (Complex Programmable Logic Device) further enhances the system's flexibility and reliability in managing DIMM devices.

[0041] Current servers (such as Intel platform servers) obtain memory temperature by the CPU polling the memory's TSOD (Temperature Sensor Depth) in real time; this is automatically implemented by the CPU's internal hardware. If the BMC (Baseboard Management Controller) polls the PMIC on the memory via I3C, it will conflict with the CPU, requiring a MUX (Multiplexer) to resolve this issue. Because memory access is very fast, the MUX has high requirements and is prone to problems, potentially causing the CPU or BMC to fail to read the memory temperature. Based on this, this patent proposes a method for monitoring PMIC and memory temperature using the BMC.

[0042] like Figure 4 The diagram shows the hardware diagram for the BMC to detect memory power signals. Current server hardware designs (such as Intel platform servers) typically connect the power signals of four DIMMs together to form a single power signal, which is then connected to the CPLD. The CPLD is in turn connected to the BMC, allowing the BMC to detect the memory's power signal. When the memory is functioning correctly, the memory's power signal is high. When the PMIC malfunctions, the power signal is pulled low, which the BMC can detect. However, the BMC cannot pinpoint exactly which DIMM's power signal is faulty.

[0043] One limitation of related technologies is that when a memory power supply abnormality is detected, the BMC cannot pinpoint which memory module (DIMM) has experienced a power failure. This may be because the POWERGOOD signals from multiple memory modules are aggregated, providing only a general signal to the BMC. Consequently, when a fault occurs, the BMC can only know that the power supply of a memory module is faulty, but cannot identify the specific module. The technical solution in this application utilizes both the BIOS and BMC for fault detection, achieving precise location of the fault and thus alerting the user to perform rapid replacement or repair.

[0044] The communication mechanism between the BMC (Baseboard Management Controller) and PMIC (Power Management Integrated Circuit) is specifically through the I3C (Improved Inter-Integrated Circuit Communication) bus. In traditional designs, the CPU periodically polls the health status of the PMIC via the I3C bus. However, if a PMIC malfunctions during system operation, causing unstable power supply to the memory, an AC reboot is usually required to reinitialize the system.

[0045] In this application's technical solution, the control of the I3C bus is altered by introducing a control strategy using a MUX (Multiplexer) and CPLD (Complex Programmable Logic Device). During the POST (Power-On Self-Test) phase, I3C control is held by the CPU, which monitors the PMIC's status via the BIOS. However, during the OS (Operating System) runtime phase, control switches to the BMC (Browser Control Center). The BMC can directly poll the registers of each PMIC via the I3C bus, monitoring the PMIC's status in real time, including the voltage, current, and temperature of each module. When a PMIC's parameter exceeds a preset threshold, indicating a PMIC failure, the BMC can immediately record the fault information and locate the specific memory module without requiring an AC reboot of the entire server.

[0046] This approach improves the real-time performance and accuracy of fault detection, reduces the need for AC reboot, and thus avoids data loss, service interruption, and wasted restart time that may result from AC reboot, providing important technical support for the stable operation and maintenance of the server.

[0047] The PMIC specification defines 11 events that trigger internal VR disable (voltage regulator shutdown). When VR disable is triggered, the memory's POWER GOOD signal is pulled low, and the CPLD detects the abnormal memory voltage, causing the CPLD to shut down the server. When the server is running normally, if a memory module's PMIC fails, the server will immediately shut down. If a command is sent or the power button is pressed manually to power on at this point, the server will fail to boot because the memory's POWER GOOD fail state is not cleared, violating normal power-on logic. Therefore, the server will shut down again at some point during the boot process. In the current system design, because the power good signals of the four memory DIMMs are connected together, the BMC cannot detect and locate which DIMM has a PMIC failure. However, the technical solution in this application divides PMIC monitoring into two phases. The first is the boot phase (i.e., the post-test phase), where the CPLD transfers control to the CPU, and the BIOS detects PMIC faults. If a fault is found, it is isolated and reported to the memory. The second stage is the OS stage (following the post-stage, the operating system runtime stage). In this stage, the CPLD switches control to the BMC, which monitors the PMIC registers via I3C. This setup allows the CPU and BMC to monitor the PMIC registers in real time. The CPU can accurately locate the address where the fault occurred (generally referring to the slot), and then disable the channel of the faulty memory to ensure normal boot. The BMC can perform fault detection in the OS stage without frequent server AC reboots. In DDR5 memory, multiple memory modules (DIMMs) typically share one or more data channels. If the PMIC of a DIMM fails, for system stability and performance, the channel containing the faulty DIMM should be disabled, rather than the entire memory system. This is because: unstable power supply to the faulty memory can lead to data read / write errors; disabling the channel prevents data corruption or system instability. Although disabling a channel reduces available memory bandwidth, compared to a complete shutdown or restart, this method can maintain system operation to some extent and avoid greater performance loss. Disabling the faulty memory channel reduces unnecessary power load, contributing to system thermal management and energy efficiency. Through this series of operations, the system can safely isolate and handle power supply failures of DDR5 memory, while minimizing the impact on system performance and ensuring the continuous and stable operation of the server or computer.

[0048] In an exemplary embodiment, before obtaining a first detection result, the method includes: inputting first control information to a multiplexer, the first control information being used to control the multiplexer to establish communication between the processor and a communication component of a power management integrated chip.

[0049] In one exemplary embodiment, before obtaining the second detection result, the method includes: inputting second control information to a multiplexer, the second control information being used to control the multiplexer to establish communication between the substrate management controller and the communication components of the power management integrated chip.

[0050] A multiplexer (MUX) is an electronic device or circuit primarily used to select one signal from multiple input signals for transmission to a single output channel. In computer and server hardware architectures, the role of a multiplexer is to improve the utilization of hardware resources by switching communication paths, enabling multiple devices to share the same bus or communication line.

[0051] When a MUX is working, it selects which input signal to send to the output based on control signals. Control signals are instructions that specify which input source the MUX should currently select; they are typically a set of binary numbers, each corresponding to the strobe state of an input. For example, in an 8-to-1 MUX, the control signal could be a 3-bit binary number, and depending on different combinations (000 to 111), the MUX will select one of the eight input signals to transmit to the output.

[0052] In a server's PMIC monitoring system, the primary role of the MUX (Multi-User Access Controller) is to resolve the sharing issue of the I3C bus between the CPU and BMC (Browser Control Center). Both the CPU and BMC need access to the I3C bus to monitor and manage the PMIC and other devices. However, due to limited bus resources, simultaneous access can lead to data conflicts or communication failures. Using a MUX allows for on-demand allocation of I3C bus control: the MUX ensures that only one master device (CPU or BMC) can control the I3C bus at any given time, thus avoiding data conflicts. During the POST (Post-Processing) phase, the MUX's control signals are set by the CPLD (Complex Programmable Logic Device), enabling the CPU to initialize and perform fault detection on the PMIC through the BIOS program. After POST is complete, the control signals switch, and the MUX relinquishes control of the I3C bus to the BMC, allowing the BMC to directly monitor the PMIC's status. Through the MUX, ordered access to the I3C bus by the CPU and BMC is achieved, ensuring communication stability and data integrity.

[0053] MUX provides a mechanism in hardware architecture that allows multiple devices to share limited communication resources. However, its limitations and potential problems need to be considered when high performance and stability are required. By improving the design, such as the BMC monitoring scheme proposed in this application, the functionality of BMC can be effectively utilized, reducing the complexity and risks introduced by MUX, and achieving more efficient and real-time hardware monitoring.

[0054] Specifically, POST, short for Power-On Self-Test, is a crucial stage in the server startup process. It primarily detects and initializes hardware devices, ensuring they are in a stable and operational state. When the server starts, the CPU performs POST checks via the Basic Input / Output System (BIOS). This process includes: 1. Hardware detection: The BIOS will detect all critical hardware components in the system, such as the CPU, memory, chipset on the motherboard, input / output ports, hard drive and drives, etc., to ensure that these components are working properly.

[0055] 2. Device Initialization: The BIOS initializes the devices in the system, including setting the memory's operating mode, frequency, and voltage, and initializing the CPU's cache and bus. During this stage, the BIOS communicates with the PMIC (Power Management Integrated Circuit) to ensure proper power supply and management of the memory.

[0056] 3. Error detection and reporting: The BIOS checks the operating status of hardware devices. If it finds any abnormalities, such as an abnormal PMIC status, it will record the error and may report it to the user through BEEP sound, screen information or log file.

[0057] 4. System Configuration: The POST process also sets the basic system configuration, such as adjusting the system clock and activating security features, in preparation for the operating system to load.

[0058] During the POST phase, I3C control is usually held by the CPU through the BIOS, because the main task of this phase is for the CPU and BIOS to initialize and test the hardware, ensuring that the hardware is in a correct and healthy working state before the server enters the operating system.

[0059] Once the POST process is complete, the BIOS transfers control to the operating system (OS), and the server enters normal operation. During this stage: 1. Application operation: Users can launch various applications and services, and the operating system is responsible for scheduling these resources.

[0060] 2. Hardware Resource Management: The operating system takes over the management of hardware resources, including memory, processor, disk, network interface, etc., and dynamically allocates resources according to the needs of applications.

[0061] 3. Real-time Monitoring and Maintenance: During server operation, the BMC (Baseboard Management Controller) comes into play. The BMC is an independent microprocessor used to monitor and maintain the server's environmental conditions and hardware status. During OS operation, the BMC gains access to the PMIC registers via the I3C bus, allowing it to poll the PMIC's status in real time and monitor its voltage, current, and temperature parameters, ensuring hardware stability and security during operation.

[0062] 4. Fault Response and Recovery: If the BMC detects an anomaly in the PMIC, such as unstable voltage, abnormal current, or overheating, it can immediately record specific fault information, including the precise location of the faulty memory, and issue an alert to the administrator via the network or system logs. In some cases, the BMC may also take measures to isolate the faulty component to prevent it from affecting the operation of the entire system.

[0063] 5. Remote Management and Maintenance: BMC supports remote management functions, allowing users to access the server via the network even when the OS is not running, to perform status monitoring, fault diagnosis, and system maintenance.

[0064] During the OS runtime phase, since the hardware has been initialized and the system is in a stable working state, the BMC's monitoring can be more detailed and real-time. It replaces the CPU as the master device on the I3C bus, specifically responsible for continuous hardware monitoring and maintenance, while the CPU focuses on executing application and operating system tasks. This division of labor improves the overall stability and management efficiency of the system, especially in server environments that require 24 / 7 continuous operation, where the BMC's real-time monitoring capabilities are particularly important.

[0065] In one exemplary embodiment, the method further includes: generating location information of the faulty memory in response to a first detection result indicating a fault; generating a fault log based on the location information, the fault log being displayed to a target user, the fault log being generated by a baseboard management controller.

[0066] In one exemplary embodiment, after performing the first target operation on the faulty memory, the method further includes: eliminating the fault exception signal generated by the fault register corresponding to the faulty memory.

[0067] In an exemplary embodiment, after performing a first target operation on the faulty memory, the method further includes: powering on the memory in a normal state; initializing the powered-on memory; setting the power management integrated circuits of all memory modules to write-protected mode; controlling the processor to poll the registers of the power management integrated circuits of the memory modules after initialization; in response to the processor detecting a fault in the register, transmitting the fault information of the register to the baseboard management controller and performing a restart power-on operation on the server; in response to the processor not detecting a fault in the register, continuing to power on the server until power-on is complete; in response to determining that the server power-on is complete, acquiring at least one of a second detection result and a third detection result, wherein the third detection result is obtained by the baseboard management controller polling the temperature sensor on the memory module, and the temperature signal of the temperature sensor is used to determine the temperature of the memory; in response to the second detection result indicating a fault, performing a second target operation on the server; and in response to determining that the temperature of the memory exceeds a temperature limit based on the third detection result, performing a second target operation on the server.

[0068] DIMM is an abbreviation for "Dual In-line Memory Module," a standard for memory modules used in computers and servers. In DDR5 technology, DIMM refers to a memory module that conforms to the DDR5 specification. It not only contains DRAM chips (Dynamic Random Access Memory) for storing data but also integrates other important components and circuitry to achieve higher performance, lower power consumption, and better signal integrity.

[0069] All devices on a DDR5 DIMM typically include the following components: 1. DRAM (Dynamic Random Access Memory) chips: This is the main storage component on the DIMM, responsible for data storage and retrieval. 2. PMIC (Power Management Integrated Circuit): The PMIC is an important new component on DDR5 DIMMs. It manages the power supply to the DRAM chips, controlling voltage and current to meet the stringent requirements of the DDR5 specification, while reducing power consumption and improving energy efficiency. 3. TSOD (Temperature Sensor): Located on the memory module, it monitors the temperature of the DRAM chips to prevent overheating from damaging the hardware or affecting performance. 4. RCD (Register Clock Driver): Used to improve signal quality and clock signal distribution, ensuring accurate and high-speed data transmission. 5. SPD (Serial Presence Detect) chip: Stores configuration information about the memory module, such as capacity, speed, and manufacturer data, allowing the system BIOS to read this information during startup and correctly configure the memory. 6. I3C bus interface: I3C is an improved version of I2C (Inter-Integrated Circuit) bus technology used to transfer data and control signals between devices on the DIMM and the motherboard. I3C offers higher transmission speeds and better signal integrity, supporting communication between devices on the DIMM and the board management controller or CPU.

[0070] These components and circuitry of a DDR5 DIMM collectively form a complex memory module system. They work closely together to support the high-speed data transfer, low power consumption, and high reliability of DDR5 technology. By integrating the PMIC, TSOD, and RCD onto the DIMM, DDR5 technology can more efficiently manage memory power and signals, reduce circuit complexity on the motherboard, and provide a more stable memory environment.

[0071] like Figure 5 The diagram shows a flowchart of a server fault handling method.

[0072] During the boot process, the BIOS needs to access the PMIC and other devices on the memory, such as TSOD. Therefore, during boot, control of the I3C is transferred to the CPU. After booting and entering the OS, the CPU no longer accesses the PMIC, and control is transferred to the BMC, which monitors the PMIC and memory. The main steps of the BIOS monitoring the PMIC during boot are as follows: Step 1: Power on the AC, turn on the server, and the PMIC enters configuration mode, which is non-write-protected mode.

[0073] Step 2: The BIOS polls the PMIC registers of all DIMMs via I3C to check for PMIC faults. Since error messages are recorded in non-volatile registers after a PMIC fault occurs, checking these registers can help determine if a PMIC fault exists.

[0074] Specifically, polling is a computer communication and data acquisition method in which the central processing unit (CPU) or other master device periodically or continuously checks the status or data of slave devices (such as sensors, memory, peripherals, etc.) to determine if there is any new information or event that needs to be processed. In polling mode, the master device actively initiates queries instead of waiting for slave devices to send interrupt signals to notify of status changes.

[0075] During operation, the CPU periodically accesses the temperature sensor (TSOD) on the memory to read the current temperature information. The TSOD is a temperature sensor embedded in the DDR memory module, used to monitor the memory temperature and prevent overheating that could lead to performance degradation or hardware damage.

[0076] To optimize resource utilization and avoid bus conflicts between the CPU and BMC, a MUX (multiplexer) is used to switch control of the I3C bus at different stages, allowing the CPU to poll during the POST stage and the BMC to perform real-time monitoring and polling during the OS runtime stage.

[0077] In this way, both the CPU and BMC can effectively monitor memory temperature, but the BMC's monitoring is more real-time and detailed, and can respond to temperature anomalies in a timely manner without interrupting normal system operation, thereby improving system stability and security.

[0078] In an optional embodiment, the CPU can poll the TSOD of the memory in real time, and the specific steps may include: 1. Initialization: When the system starts, the CPU initializes the communication of TSOD, including setting bus parameters and confirming the sensor address and configuration status.

[0079] 2. Query cycle: The CPU sets a query cycle, which can be fixed, such as querying once per second, or dynamically adjusted according to system load and temperature changes.

[0080] 3. Query command: In each query cycle, the CPU sends a query command to TSOD via I2C (Inter-Integrated Circuit), I3C or other communication buses.

[0081] 4. Data Reading: After receiving the command, TSOD will return the current temperature data to the CPU via the bus.

[0082] 5. Status Assessment: After receiving temperature data, the CPU compares it with a preset threshold to determine if the temperature is normal. If the temperature exceeds the normal range, the CPU may need to take measures such as adjusting the cooling system, reducing system performance, or generating a warning message.

[0083] 6. Repeated loop: The CPU will continue to repeat this process until the system shuts down or the polling mechanism is interrupted by other higher priority tasks.

[0084] Step 3: If a memory module's PMIC is determined to be faulty, the BIOS sends the specific information of the faulty memory and its PMIC register information to the BMC. Upon receiving this information, the BMC generates a log and issues an alarm. This allows the user to pinpoint the specific memory slot and replace the memory. The BIOS also configures the faulty PMIC register, clears the "power good" exception, and disables the channel containing the faulty memory, isolating it.

[0085] Step 4: If there is no fault or the BIOS has disabled the faulty memory, the BIOS enables VR for all normal memory, allowing the memory to power on normally.

[0086] Step 5: Initialize memory.

[0087] Step 6: Initialize all normal memory. After initialization, set the PMIC of all memory to write-protected mode.

[0088] Step 7: The BIOS polls the PMIC registers of all memory modules again to check for any remaining memory PMIC faults. If a fault is found, information about the faulty memory is sent to the BMC, and the BMC performs an AC reboot, starting from step one again. If there are no faults, the system continues booting.

[0089] Step 8: After the BIOS POST is complete, set the GPIO (General Purpose Input / Output) pin. This GPIO is set to low by default. After POST is complete, the BIOS sets this GPIO to high as a sign that the BIOS POST is complete.

[0090] Step 9: When the CPLD detects that the GPIO is high, the CPLD switches control of the I3C to the BMC. From this moment on, the BMC starts monitoring the PMIC.

[0091] After POST is completed and the system enters the OS, the BMC monitors the PMIC.

[0092] After POST is complete, I3C control switches from the CPU to the BMC. The BMC then starts polling the PMIC and TSOD to monitor the PMIC registers and obtain memory temperature.

[0093] According to the JEDEC specification, the PMIC has 256 bytes of registers (0x0-0xFF), where 0-0x3F is the host region, 0x40-0x6F is the DIMM Vendor region, and 0x70-0xFF is the Vendor Specific Region.

[0094] The host region is the area that the host can access, and it can read and write registers.

[0095] The DIMM Vendor region and Vendor Specific Region are manufacturer-specific protected areas for PMICs. Registers in these two areas cannot be read or written without a password.

[0096] The BMC needs to monitor the host region's Status Register, which is divided into four types: 1. Global History of Error Log Register: This register records the historical status of the PMIC during each abnormal power outage cycle. When the PMIC generates a VRDisable command due to a fault, it will write the command to this register.

[0097] 2. Error Log Registers: These registers record the PMIC status for each power outage sequence. When a VR Disable command is generated automatically due to a fault, the PMIC writes to these registers, which are updated during the power outage cycle.

[0098] 3. Real-time Status Registers: Once any event occurs, the corresponding bit in the real-time register will be written to 1. If the event is resolved, the corresponding bit in the register will remain 1 and will not be cleared until the PMIC receives a command to clear the register.

[0099] 4. Periodic Status Registers: These registers are dynamically updated by the PMIC at a specific frequency, indicating the current status. They primarily reflect the current, voltage, and temperature of the various modules within the PMIC.

[0100] The BMC primarily monitors Periodic Status Registers and Error Log Registers. The BMC polls all memory modules in real time to obtain the voltage, current, and temperature of each module within the PMIC, and reads the threshold values ​​for voltage, current, and temperature from the registers. When any indicator exceeds the threshold, a log is logged and an alarm is triggered. If an event that could cause VR disable is detected, a critical alarm is logged on the BMC web interface, prompting the user to replace the faulty memory module in the PMIC. By monitoring the voltage, current, and temperature status of each module in the PMIC in real time, various information about the faulty memory module can be reported immediately without requiring an AC reboot.

[0101] After POST is completed and the system enters the OS, in addition to polling the memory PMIC, it is also necessary to poll the memory TSOD in real time, that is, to obtain the memory temperature in real time. When overheating occurs, the CPU memory temperature-related registers are actively written through PECI. When the overheating is resolved, the CPU memory temperature-related registers are actively cleared through PECI.

[0102] The technical solution of this application adds a hardware MUX to switch control between the CPU and BMC I3C. Control is given to the CPU during the POST phase and to the BMC after POST completion. The BMC then polls the memory TSOD and PMIC via the I3C, avoiding the risks that might arise from the CPU and BMC competing for I3C control. The BMC monitors the voltage, current, and temperature of each module of the PMIC by polling it. When thresholds are exceeded, an alarm is triggered, and detailed information is recorded. This allows for accurate recording of fault information in the event of a PMIC failure, eliminating the need for an AC reboot. The BIOS detects the fault and reports it to the BMC. This method offers better real-time performance and is more convenient.

[0103] In an optional embodiment, during the initial server startup, after the AC is powered on, the PMIC enters a non-write-protected mode, also known as configuration mode, allowing the BIOS to perform initialization and configuration tasks. Non-write-protected mode means that the PMIC's registers can be read and written, which is a prerequisite for the BIOS to check and set the PMIC state.

[0104] The BIOS polls the PMIC registers via the I3C bus: the BIOS calls the CPU to access the PMIC register of each DIMM via the I3C bus to check if any fault information has been recorded. Here, "BIOS" refers to the BIOS program running on the CPU, communicating with the PMIC through the CPU.

[0105] Fault diagnosis and reporting: If a PMIC fault is detected, the BIOS will report the detailed information of the faulty memory and the status of the PMIC registers to the BMC. The BMC will then log the information and generate an alarm to guide the user in locating the faulty memory.

[0106] Fault isolation and normal memory configuration: The BIOS configures the faulty PMIC register, clears the abnormal state (such as disabling the voltage regulator), and disables or isolates the channel where the faulty memory is located to ensure stable system operation. Then, the BIOS enables the voltage regulators of all normal memory, initializes and powers on the normal memory.

[0107] The POST completion and control switchover process is as follows: GPIO setting: After the BIOS completes POST (Power-On Self-Test), it sets the GPIO pin to a high level as a sign of POST completion. CPLD control switchover: After the CPLD detects that the GPIO is high, it switches the control of the I3C from the CPU to the BMC, enabling the BMC to monitor the PMIC in real time.

[0108] During the OS runtime phase, the BMC monitors the PMIC as follows: After entering the OS runtime phase, the BMC becomes the master controller of the I3C bus, responsible for real-time monitoring of the PMIC status and memory temperature.

[0109] The BMC's PMIC monitoring process is as follows: The BMC polls the PMIC's Periodic Status Registers and Error Log Registers via the I3C bus. These registers contain real-time status information such as voltage, current, and temperature of the PMIC module, as well as past error information. The BMC reads the voltage, current, and temperature thresholds from the registers. Once any indicator exceeds the threshold, it immediately logs the information and sends an alarm to the user. In particular, if an event that could disable VR is detected, the BMC will log a critical alarm on its web interface and prompt the user to replace the faulty PMIC memory.

[0110] In addition to PMIC monitoring, BMC also polls the TSOD (Temperature Sensor) on the memory in real time to obtain memory temperature information. If overheating is detected, BMC sends a signal to the CPU via the PECI (Platform Environment Control Interface) bus to trigger the CPU's temperature protection mechanism, such as reducing system performance or shutting down the system, to prevent device overheating damage. Once the overheating state is resolved, BMC also clears the temperature-related protection states in the CPU via PECI, restoring normal system operation.

[0111] Through the above process, the server can ensure that the PMIC and memory temperatures are effectively monitored and managed during the startup and operation phases, thereby improving system stability and security.

[0112] The entities that perform the above steps can be servers, terminals, etc., but are not limited to these.

[0113] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0114] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0115] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0116] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0117] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0118] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0119] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0120] The embodiments described herein also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0121] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0122] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0123] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for handling server faults, characterized in that, include: Obtain a first detection result, which includes a fault detected by the processor during the power-on self-test phase by checking the register status of the power management integrated chip. The processor is used to execute BIOS firmware, and the power management integrated chip is used to manage the power supply to the memory. In response to the first detection result indicating a fault, a first target operation is performed on the faulty memory, the first target operation including disabling the data channel where the faulty memory resides; In response to the first detection result being no fault or the first target operation being completed, the server startup process continues until startup is complete. In response to determining that the server has completed power-on, a second detection result is obtained, the second detection result including faults detected by the baseboard management controller in the register state of the power management integrated chip during the operating system operation phase; Based on the second detection result, a second target operation is performed on the server. The second target operation includes generating a reminder message, which is used to display fault information to the target user. The fault information includes at least one of the following: fault location, fault type, fault level, and suggested fault handling operation.

2. The method according to claim 1, characterized in that, Before obtaining the first detection result, the method includes: Input first control information to the multiplexer, the first control information being used to control the multiplexer to establish communication between the processor and the communication component of the power management integrated chip.

3. The method according to claim 1 or 2, characterized in that, Before obtaining the second detection result, the method further includes: Input second control information to the multiplexer, the second control information being used to control the multiplexer to establish communication between the baseboard management controller and the power management integrated chip communication components.

4. The method according to claim 1, characterized in that, The method further includes: In response to the first detection result indicating a fault, the location information of the faulty memory is generated; A fault log is generated based on the location information. The fault log is used to display to the target user. The fault log is generated by the baseboard management controller.

5. The method according to claim 1, characterized in that, After performing the first target operation on the faulty memory, the method further includes: Eliminate the fault signals generated by the fault register corresponding to the faulty memory.

6. The method according to claim 1, characterized in that, In response to confirming that the server has completed booting, a second detection result is obtained, including: When a high-level signal is detected on a general-purpose input / output pin of the server, it is determined that the server has completed power-on.

7. The method according to claim 1, characterized in that, After performing the first target operation on the faulty memory, the method further includes: Power on the memory that is in normal condition; Initialize the memory after power-on; Set all memory power management integrated chips to write-protected mode; The registers of the power management integrated chip that control the processor to poll the memory after initialization; In response to the processor detecting a fault in a register, the fault information of the register is transmitted to the baseboard management controller and a restart power-on operation is performed on the server; In response to the processor not detecting a fault in a register, the server boot process continues until boot is complete. In response to determining that the server has completed booting, at least one of the second detection result and the third detection result is acquired, wherein the third detection result is obtained by the baseboard management controller polling the temperature sensor on the memory module, and the temperature signal of the temperature sensor is used to determine the temperature of the memory; In response to the second detection result indicating a fault, the second target operation is performed on the server; In response to determining, based on the third detection result, that the temperature of the memory exceeds the temperature limit, the second target operation is performed on the server.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1 to 7.

Images