Method and system for handling server fatal errors

Abstract

This application proposes a method and system for handling fatal errors in servers, relating to the field of server fault handling technology. When applied to an ATF (Automatic Server Fault Tolerance), the method includes: in response to receiving a system restart request sent by the operating system when a kernel crash is triggered, obtaining error severity information pre-stored for the kernel crash; in response to the error severity information being classified as Level 1, notifying the SCP (Application Processor Center) to take all application processor cores offline, so that after confirming that all application processor cores are offline, the SCP notifies the MCP (Machine Programming Center) to collect and transmit the collected diagnostic data related to the kernel crash to a designated non-volatile storage medium within a preset delay time window; and when the delay time window expires or the diagnostic data transmission is completed ahead of schedule, causing the SCP to execute the corresponding hardware reset operation according to the restart policy specified by the operating system. This solution can effectively and completely retain the diagnostic data of fatal errors while ensuring safe system termination.

Description

Technical Field

[0001] This application relates to the field of server fault handling technology, and more specifically, to a method and system for handling fatal server errors. Background Technology

[0002] In ARM-based server systems, when the hardware detects fatal errors such as uncorrectable memory errors or high-level PCIe device errors, the standard error handling process is typically handled collaboratively by the Trusted Firmware (ATF) and the System Controller Processor (SCP). Specifically, after the operating system determines the error to be fatal based on the error severity field, it triggers a kernel crash and sends a reboot request to the ATF via the Power State Coordination Interface / Security Monitor call interface. Upon receiving the request, the ATF instructs the SCP to shut down all application processor cores and immediately perform a hardware reset via the System Control and Management System Interface.

[0003] However, the existing error handling process often encounters situations where diagnostic data cannot be effectively obtained: after a single fatal error occurs, the related diagnostic data is often not fully saved or is even completely lost; in the case of multiple consecutive fatal errors, not only may the diagnostic data of the previous errors be completely lost, but even the last error may only have some data recorded, resulting in a complete lack of historical error information, which seriously affects the completeness and accuracy of root cause analysis.

[0004] To address the aforementioned issues, existing solutions attempt to extend the time window before the operating system terminates by modifying the operating system kernel code, such as suppressing kernel crash timeout parameters or adding data collection hook functions, thereby improving the integrity of diagnostic data retention. However, such modifications not only require adaptation for different operating system versions, making implementation complex and maintenance costly, but also potentially introducing kernel stability risks. More importantly, extending system runtime for data collection could be exploited by attackers, making it difficult to strike a balance between data integrity and data leakage prevention.

[0005] Therefore, how to effectively and completely preserve the diagnostic data of fatal errors while ensuring the safe termination of the system has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this application is to provide a method and system for handling fatal errors in servers, which can effectively and completely retain the diagnostic data of fatal errors while ensuring the safe termination of the system.

[0007] This application is implemented as follows: Firstly, this application provides a method for handling fatal errors in servers, applied to Trusted Firmware (ATF). The method includes the following steps: In response to receiving a system restart request sent by the operating system when a kernel crash is triggered, obtaining error severity information pre-stored for the kernel crash, wherein the error severity information characterizes the severity level of the hardware error causing the kernel crash. In response to the error severity information being classified as level one, notifying the System Controller Processor (SCP) to take all application processor cores offline, so that after confirming that all application processor cores are offline, the SCP notifies the Management Controller Processor (MCP) within a preset delay time window to collect and transmit the collected diagnostic data related to the kernel crash to a designated non-volatile storage medium. Upon the expiration of the delay time window or the premature completion of the diagnostic data transmission, the SCP executes a corresponding hardware reset operation according to the restart strategy specified by the operating system.

[0008] Secondly, this application provides a method for handling fatal errors in a server, using a System Control Processor (SCP). This method includes the following steps: receiving a first instruction from a Trusted Firmware (ATF). The first instruction is given by the ATF in response to a system restart request sent by the operating system when a kernel crash is triggered. After obtaining error severity information pre-stored for the kernel crash, and given that the error severity information is classified as a first level, the ATF notifies the SCP to take all application processor cores offline. The error severity information represents the severity level of the hardware error that caused the kernel crash. In response to the first instruction, all application processor cores are taken offline. After confirming that all application processor cores are offline, within a preset delay time window, the Management Control Processor (MCP) is notified to collect diagnostic data related to the kernel crash and transmit the collected diagnostic data to a designated non-volatile storage medium. When the delay time window expires or the diagnostic data transmission is completed ahead of schedule, a corresponding hardware reset operation is performed according to the restart strategy specified by the operating system.

[0009] Thirdly, this application provides a server fatal error handling system, comprising: a management control processor (MCP), configured to, in response to a system hardware detecting a fatal error and reporting the complete context of the collected hardware error to the operating system, simultaneously save error severity information to a storage area accessible by the trusted firmware (ATF); and configured to, based on a notification sent by the system control processor (SCP) to collect diagnostic data, collect diagnostic data related to kernel crashes within a preset delay time window, and transmit the diagnostic data to a designated non-volatile storage medium; and an operating system, configured to, upon receiving the complete context of the hardware error reported by the management control processor (MCP), trigger a kernel crash and send a system restart request to the trusted firmware (ATF), the system restart request carrying... The system includes a specified reboot policy; a Trusted Firmware (ATF) for responding to a received system reboot request, acquiring error severity information pre-stored for the kernel crash, the error severity information representing the severity level of the hardware error causing the kernel crash, and for sending a second instruction to the System Controller Processor (SCP) in response to the error severity information being classified as a first level; and the SCP for responding to the received second instruction, taking all application processor cores offline, and sending a notification to the SCP to collect diagnostic data after confirming that all application processor cores are offline, and for performing a corresponding hardware reset operation according to the reboot policy specified by the operating system when the delay time window expires or the diagnostic data transmission is completed ahead of schedule.

[0010] Compared with the prior art, this application has at least the following advantages or beneficial effects: In the server fatal error handling method proposed in this application, the delay window occurs after the System Controller Processor (SCP) safely takes the application processor core offline. At this time, the operating system is in a completely frozen state with no executable code, fundamentally eliminating the risk of data leakage and avoiding security issues introduced by extending runtime for data collection. Secondly, the entire process is completely transparent to the operating system, requiring no modification to any operating system kernel code, thus avoiding the high maintenance costs and potential kernel stability risks associated with existing improvement solutions that require adaptation to different versions. Thirdly, through the preset delay window, this application allows the SCP to calmly notify the Management Controller Processor (MCP) to collect complete diagnostic data and transmit it to a non-volatile storage medium independent of dynamic random access memory, ensuring that historical error information is completely preserved even in scenarios with multiple consecutive fatal errors. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of an embodiment of a server fatal error handling method according to this application; Figure 2 This is a flowchart of yet another embodiment of a method for handling fatal server errors according to this application; Figure 3 This is a flowchart of another embodiment of a server fatal error handling method according to this application; Figure 4 This is a structural block diagram of an embodiment of a server fatal error handling system according to this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0014] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0015] To facilitate understanding of the technical solutions provided in this application, some concepts will be introduced below.

[0016] Trusted Firmware (ATF): In this application, it refers to the underlying firmware running in the secure world of the ARM processor, responsible for core functions such as system startup, runtime security monitoring, and receiving reboot requests from the operating system.

[0017] System Control Processor (SCP): In this application, it is an independent processor specifically responsible for processor core power management and system control. It receives instructions from the Trusted Firmware (ATF) and performs low-level hardware control operations such as taking the application processor core offline and starting the delay timer.

[0018] Management Control Processor (MCP): In this application, it is an independent processor responsible for hardware error monitoring and diagnostic data collection. When a fatal error occurs in the system, it collects error context information and transmits diagnostic data to non-volatile storage media within a delay window.

[0019] In the process of developing this application, the inventors discovered that the reason why existing error handling processes often encounter situations where diagnostic data cannot be effectively obtained is because the firmware layer immediately performs a hardware reset after the application core is taken offline. The diagnostic data is cleared before it has been transferred to other non-volatile storage media, resulting in the complete loss of single error data and the overall loss of historical information in multiple consecutive error scenarios.

[0020] In addition, while existing solutions attempt to modify the operating system kernel code to extend the time window before termination, they have drawbacks such as needing to adapt to different versions and introducing kernel stability risks. Furthermore, extending the runtime may be exploited by attackers, making it difficult to strike a balance between data integrity and leakage prevention.

[0021] To address this, this application implements a "safe shutdown first, delayed restart" design at the firmware layer. This provides a deterministic time window for diagnostic data collection while ensuring safe system termination, thus effectively and completely preserving diagnostic data for fatal errors even under safe system termination conditions. The "safe shutdown first, delayed restart" design means that the Trusted Firmware (ATF) does not directly perform a hardware reset. Instead, it first acquires error severity information pre-stored for kernel crashes and makes intelligent branch decisions accordingly. When this information indicates a first-level (i.e., fatal) error, the ATF sends a notification to the System Controller Processor (SCP), instructing it to shut down all application processor cores. After confirming that all application processor cores have been safely shut down, the SCP initiates a preset delay time window. Within this window, it notifies the Management Controller Processor (MCP) to collect diagnostic data related to the kernel crash and transmits the data to a designated non-volatile storage medium. When the delay time window expires or the data transmission is completed ahead of schedule, the SCP then performs the corresponding hardware reset operation according to the restart strategy specified by the operating system.

[0022] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings. Unless otherwise specified, the various embodiments and features described below can be combined with each other.

[0023] Please see Figure 1 The method for handling fatal server errors, applied to trusted firmware ATF, specifically includes the following steps: Step S101: In response to receiving a system restart request sent by the operating system when a kernel crash is triggered, obtain the error severity information pre-stored for the kernel crash, wherein the error severity information represents the severity level of the hardware error that caused the kernel crash.

[0024] It should be noted that in a server system, when the hardware detects a fatal error (such as an uncorrectable memory error, a high-level PCIe device error, or a serious failure of another hardware controller), the Management Control Processor (MCP) or the Trusted Firmware ATF collects the error information and reports it to the operating system (for example, this can be done through existing standard interfaces such as the ACPI standard interface GHESv2). After the operating system determines the error severity level as fatal, it triggers a kernel panic (the action taken by the operating system when it detects an internal fatal error and cannot safely handle it) and completes the kernel panic process. Subsequently, it calls the Power State Coordination Interface / Security Monitor Call (PSCI / SMC) interface according to standard specifications to send a system reboot request to the Trusted Firmware ATF. As the lowest-level firmware in the system, the Trusted Firmware ATF is the first to receive this reboot request.

[0025] Meanwhile, when the Management Control Processor (MCP) reports an error to the operating system, it simultaneously saves the corresponding error severity information (i.e., the value of the error severity field) to a storage area accessible by the Trusted Firmware ATF (such as reserved memory or registers). Therefore, upon receiving a system reboot request, the Trusted Firmware ATF does not immediately perform a hardware reset, but first reads the error severity information pre-stored for this kernel crash from this accessible storage area. This information specifically characterizes the severity level of the hardware error that caused the current kernel crash, such as whether it is fatal.

[0026] In other words, by reading pre-stored error severity information, the Trusted Firmware ATF can determine the severity of the current error without relying on additional information from the operating system, laying the foundation for subsequent intelligent branch decisions. Furthermore, this mechanism can be implemented entirely based on existing standard interfaces (such as ACPI GHESv2) without modifying the operating system's reporting logic or adding new hardware signals, ensuring compatibility with existing systems.

[0027] Step S102: In response to the error severity information being characterized as Level 1, the system controller (SCP) is notified to take all application processor cores offline. After confirming that all application processor cores are offline, the SCP notifies the management controller (MCP) to collect and transmit the collected diagnostic data related to the kernel crash to a designated non-volatile storage medium within a preset delay time window. When the delay time window expires or the diagnostic data transmission is completed ahead of schedule, the SCP executes the corresponding hardware reset operation according to the restart strategy specified by the operating system.

[0028] First, it should be noted that although step S102 logically includes calls and coordination with the system control processor SCP and the management control processor MCP, these subsequent actions are derived actions completed by other entities after the trusted firmware ATF executes the corresponding actions.

[0029] Specifically, after obtaining error severity information, the Trusted Firmware ATF (ATF) determines the severity level. If the error severity is classified as Level 1 (i.e., fatal, such as the Fatal level with a value of 1 as defined in the ACPI GHESv2 specification), the controlled delayed restart process in step S102 is initiated. If it is not Level 1, the standard immediate restart process is executed. That is, in some implementations of this application, the application further includes the following step: in response to the error severity being classified as not Level 1, a process is executed in which the Trusted Firmware ATF or the System Control Processor (SCP) directly triggers a hardware reset to restart the operating system without initiating a delay time window. Of course, the Trusted Firmware ATF's response to the error severity being classified as Level 1 could also be that another entity determines the severity level of the error and informs the Trusted Firmware ATF of its classification, after which the Trusted Firmware ATF performs the subsequent response.

[0030] Next, when the Trusted Firmware (ATF) responds to an error severity level of 1, it can send a notification to the System Controller Processor (SCP) via the System Control and Management Interface (SCMI) command, instructing it to shut down all application processor cores. Upon receiving the notification, the SCP, as a processor specifically responsible for power management and core control, immediately performs a safe shutdown operation. This may include, for example, disabling instruction execution on all application processor cores, clearing sensitive data from the application processor core caches, disabling the direct memory access engine, and isolating access permissions to input / output devices. By shutting down all application processor cores, the operating system enters a safe termination state where execution flow is completely frozen. Although no hardware reset is performed, no code is executable, fundamentally eliminating the risk of data leakage.

[0031] It should be noted that, as described in step S101, before sending a notification to the System Controller Processor (SCP) to instruct it to shut down all application processor cores, the Management Controller Processor (MCP) or the Trusted Firmware (ATF) has already saved the complete error-related context information (such as the program counter, exception link register, fault status register, etc.) to reserved shared memory or dedicated registers for subsequent analysis. The storage and access of this data are not affected by the shutdown of the application processor cores. After the application processor cores are shut down, uncore data (non-core area data) is collected, such as memory controller status and Fast Peripheral Interconnect (HPI) error logs. This data is not affected by the core shutdown.

[0032] Next, after confirming that all application processor cores have been safely taken offline, the System Control Processor (SCP) (which can also be started by other entities) will initiate a preset delay window (e.g., 10 seconds by default, configurable). Within this delay window, the SCP notifies the Management Control Processor (MCP) to begin collecting diagnostic data related to the current kernel crash, including CPU context, error register status, timestamps, etc. The MCP transmits the collected diagnostic data to a designated non-volatile storage medium. For example, this designated non-volatile storage medium is a non-volatile storage medium of operating system-independent dynamic random access memory (e.g., Baseboard Management Controller (BMC)), ensuring that the data remains accessible after a subsequent hardware reset.

[0033] The termination conditions for the delay window can include three scenarios: the window timer expires naturally, diagnostic data transmission is completed ahead of schedule (notified to the system controller processor SCP by the management control processor MCP), or a forced exit is performed according to the restart policy passed by the operating system through the security monitor. When any exit condition is met, the system controller processor SCP will execute the corresponding hardware reset operation according to the restart policy specified by the operating system. For example, the system-specified restart policy can include hardware reset restart or system freeze suspension. When the system controller processor SCP performs the final operation according to the restart policy specified by the operating system when sending the restart request, it includes: if the restart policy is hardware reset restart, then perform hardware reset; if the restart policy is system freeze suspension, then maintain the current frozen state (applicable to debugging scenarios).

[0034] In summary, step S102, through the timing design of "safe shutdown first, then delayed restart," provides a deterministic time window for diagnostic data collection while ensuring the safe termination of the operating system, thus solving the data loss problem in existing technologies. Furthermore, its safe shutdown mechanism ensures the operating system is in an absolutely safe state during the delay period, eliminating potential security risks introduced by extended runtime. Simultaneously, its multi-level exit mechanism within the delay window balances data integrity with system recovery efficiency. In addition, the entire process is completely transparent to the operating system; the operating system still sends restart requests according to standard procedures without any code modification, avoiding the adaptation costs and stability risks associated with intrusive modifications.

[0035] In summary, in steps S101 and S102, the trusted firmware (ATF) does not immediately reset upon receiving a reboot request from the operating system. Instead, it assesses the severity of the error. If it is fatal, a special procedure is initiated to allow time for the diagnostic data to be saved. This special procedure solves the problem of data loss without modifying the operating system code. Furthermore, by taking the core offline first, it ensures absolute system security during the delay period, leaving no opportunity for attackers.

[0036] Specifically, in this special process, when a fatal error occurs in the server hardware, the operating system completes the standard kernel crash procedure and sends a system reboot request to the Trusted Firmware Controller (ATF). Upon receiving this request, the ATF first reads pre-stored error severity information. This information, synchronously saved when the Management Controller Processor (MCP) reported the error, tells the ATF how severe the hardware error that caused the kernel crash is. If the information indicates a Level 1 (fatal) error, the ATF instructs the System Controller Processor (SCP) to take all application processor cores offline. After the SCP safely takes all cores offline, the operating system enters a completely frozen state. At this point, within a preset delay window, it instructs the MCP to collect diagnostic data and transfer this data to non-volatile storage. Once the delay window expires or the data transfer is completed prematurely, the SCP performs a hardware reset according to the pre-defined reboot strategy of the operating system. This ensures the safe termination of the operating system while effectively and completely preserving the diagnostic data of the fatal error.

[0037] Based on the aforementioned scheme, in some implementations of this application, the process of generating the system restart request includes: after the management control processor (MCP) responds to the system hardware detecting a fatal error, collects the complete context of the hardware error, and reports an error report containing an error severity field to the operating system, the operating system, based on the received error report, triggers and completes the kernel crash process, and then sends a system restart request to the trusted firmware (ATF).

[0038] It's important to note that existing practices typically rely on Trusted Firmware (ATF) to collect hardware error information. However, ATF runs alongside the main operating system, and its collection efforts are affected if the main operating system crashes or becomes unstable. In the implementation described above, the Management and Control Processor (MCP) collects the complete hardware error context. Because the MCP has its own independent micro-operating system, separate from the server's main operating system, it can continue operating independently even if the main operating system crashes. This ensures that hardware error information is reliably collected and reported. After collecting the information, the MCP reports an error report, including a severity field, to the operating system via a standard interface. Upon receiving this report and seeing that the severity field indicates a fatal level, the operating system triggers a kernel crash according to standard procedures. Only after the kernel crash process is complete does it send a system reboot request to the ATF.

[0039] In summary, the above implementation replaces the trusted firmware ATF (Automatic Test Foundation) that follows the main operating system with a Management and Control Processing Unit (MCP) independent of the main operating system for error collection. This ensures that error information can still be reliably obtained after a system crash, and fully complies with existing standards and specifications. Each step—the MCP reporting errors, the operating system triggering a kernel crash, and finally requesting a reboot from the trusted firmware ATF—is a standard behavior at the operating system and firmware levels, requiring no additional modifications. This guarantees that the proposed solution can seamlessly integrate with existing operating systems without requiring modifications to the operating system's code to obtain diagnostic data, thus avoiding the adaptation difficulties and stability risks that might arise from kernel modifications.

[0040] Based on the aforementioned scheme, in some implementations of this application, the error severity information pre-stored for the kernel crash is information synchronously saved to the accessible storage area of ​​the trusted firmware ATF when the management control processor (MCP) reports the complete context of the collected hardware error to the operating system in response to the system hardware detecting a fatal error.

[0041] In the above implementation, when the Management and Control Processor (MCP) reports the complete context of a hardware error to the operating system in response to a fatal error detected by the system hardware, it simultaneously stores a separate copy of the error severity information in a storage area accessible to the Trusted Firmware ATF. This storage area can be reserved memory or a specific register; the key is to ensure that the Trusted Firmware ATF can independently read it when needed, without relying on the operating system for transmission.

[0042] In this way, the Management and Control Processor (MCP) stores the information simultaneously with the error report, without adding any extra process overhead. When the operating system crashes and issues a reboot request later, the Trusted Firmware (ATF) can directly read this pre-stored information to determine the severity of the error. The entire process is completely transparent to the operating system, ensuring reliable information acquisition while avoiding modification of the operating system code.

[0043] For example, the accessible storage area of ​​this trusted firmware ATF can be a non-volatile storage medium of dynamic random access memory (DRAM) independent of the operating system (such as the Baseboard Management Controller (BMC), which, similar to the Management Control Processor (MCP), also has an independent micro-operating system, separate from the server's main operating system). This ensures that data remains accessible after a subsequent hardware reset. DRAM, commonly known as RAM, loses its data upon power failure or reset, making it extremely prone to diagnostic data loss in existing solutions. Non-volatile storage, however, retains its data even after power loss, and if it is an DRAM independent of the operating system, the diagnostic data stored there remains intact even if a hardware reset clears the RAM. Regardless of how many times the server restarts, this data can be retained for maintenance personnel to analyze, effectively preventing diagnostic data loss.

[0044] Based on the aforementioned scheme, in some implementations of this application, after the step of causing the system control processor SCP to perform the corresponding hardware reset operation according to the restart policy specified by the operating system, the processing method further includes: clearing the pre-stored error severity information to avoid state residue affecting the error severity judgment during the next startup.

[0045] In the above implementation, after the system controller (SCP) completes the hardware reset operation, it performs an additional task: clearing the previously stored error severity information. This information is stored when the error occurred. If it is not deleted, and the error occurs again after the next operating system startup, the trusted firmware (ATF) might mistake the old information for new error information, leading to incorrect judgment. This ensures that each error judgment is based on the latest context, preventing old information from interfering with the current processing.

[0046] Based on the same concept as the server fatal error handling method applied to Trusted Firmware ATF, this application also provides a server fatal error handling method applied to System Control Processor (SCP). Please refer to [link to relevant documentation]. Figure 2 The processing method specifically includes: Step S201: Receiving a first instruction from the Trusted Firmware ATF, wherein the first instruction is an instruction from the Trusted Firmware ATF in response to receiving a system restart request sent by the operating system when a kernel crash is triggered, obtaining the error severity information pre-stored for the kernel crash, and in response to the error severity information being characterized as level 1, notifying the System Controller Processor (SCP) to take all application processor cores offline, wherein the error severity information represents the severity level of the hardware error that caused the kernel crash; Step S202: In response to the first instruction, taking all application processor cores offline; Step S203: After confirming that all application processor cores have been taken offline, notifying the Management Controller Processor (MCP) to collect diagnostic data related to the kernel crash within a preset delay time window, and transmitting the collected diagnostic data to a designated non-volatile storage medium; Step S204: When the delay time window expires or the diagnostic data transmission is completed ahead of schedule, executing the corresponding hardware reset operation according to the restart strategy specified by the operating system.

[0047] It is understood that the server fatal error handling method applied to the system control processor (SCP) in this application and the aforementioned server fatal error handling method applied to trusted firmware (ATF) are based on the same inventive concept, and they correspond to each other in execution flow and interaction logic. For the specific implementation process of this method, please refer to the description in any of the aforementioned implementations of the server fatal error handling method applied to trusted firmware (ATF), and will not be repeated here.

[0048] To enable those skilled in the art to more intuitively understand this application, a specific example will be provided below. This example is an exemplary demonstration combining the overall technical paradigm of this application with some optional implementation details. It should be noted that the following demonstration is intended to aid understanding and does not constitute an exhaustive list of all embodiments of this application, nor does it imply that this application must include all the details described below in its specific implementation.

[0049] Specifically, please refer to Figure 3 The data processing flow in this example includes the following: Step 1: Hardware error occurs. The system hardware detects a fatal error, including an uncorrectable memory error, a high-level error in a PCIe device, or a serious failure of another hardware controller, triggering a synchronization exception signal.

[0050] Step 2: The Management and Control Processor (MCP) reports the error to the operating system via the ACPI standard interface GHESv2. The MCP collects complete context information of the hardware error, including the error address, timestamp, and error type, and reports the error to the operating system via the ACPI standard interface GHESv2. The error severity field is explicitly set to the first level (set to 1, i.e., Fatal level), indicating that the platform cannot be recovered and the system must terminate.

[0051] Step 3: The operating system triggers the standard kernel panic procedure. After receiving the Level 1 error reported by GHESv2, the operating system triggers a kernel panic according to the standard procedure, including terminating all user processes, disabling interrupts and the scheduler, and printing error logs. The entire process maintains the original implementation of the operating system without any code modification.

[0052] Step 4: The operating system calls the PSCI / SMC (Power State Coordination Interface / Security Monitor) to request a system reboot. After the kernel crash process is completed, the operating system sends a system reboot request to the Trusted Firmware ATF through the standard firmware interface (PSCI / SMC). This call is standard operating system behavior and conforms to the reboot process defined by the ACPI specification (Advanced Configuration and Power Management Interface).

[0053] Step 5: The Trusted Firmware ATF reads the error severity field for judgment. After receiving the reboot request, the Trusted Firmware ATF independently reads the error severity field value pre-stored by the Management Control Processor (MCP). This value has been synchronously stored in the reserved memory area accessible by the Trusted Firmware ATF when the error is reported to the operating system, without requiring additional information or interface from the operating system.

[0054] Step 6: Trusted Firmware ATF Intelligent Branch Decision. The Trusted Firmware ATF makes a branch decision based on the error severity field value: if the value is not the first level, the standard immediate restart process is executed; if the value is the first level, the controlled delayed restart process is entered (steps 7-12), realizing differentiated handling of different error types.

[0055] Step 7: Shut down all application processor cores to ensure safe termination. The Trusted Firmware (ATF) notifies the System Controller (SCP) to shut down all application processor cores via SCMI (Standard System Control and Management Interface) commands. This includes disabling instruction execution, clearing sensitive data from the cache, disabling the Direct Memory Access (DMA) engine, and isolating I / O device access permissions. This puts the operating system into a safe termination state where the execution flow is completely frozen, fundamentally eliminating the risk of data leakage.

[0056] Step 8: Start a delay timer to provide a data collection time window. After confirming that all application processor cores have been safely taken offline, the system control processor SCP starts a delay timer, which is set to 10 seconds by default and can be configured to provide a deterministic time window for diagnostic data collection. This delay occurs after the system has been safely terminated and does not affect security.

[0057] Step 9: Notify the Management Control Processor (MCP) to begin collecting and transmitting diagnostic data. The System Control Processor (SCP) notifies the MCP to begin collecting diagnostic data, including the CPU context, error register status, timestamps, etc., via a hardware mailbox or interrupt mechanism, and transmits it to the Baseboard Management Controller (BMC)'s non-volatile memory or flash memory to ensure that the data remains accessible after a system reboot.

[0058] Step 10: Wait for data transmission to complete and support flexible exit. The System Control Processor (SCP) continuously monitors the data transmission status during the delay period and supports three exit conditions: delay timer timeout, early exit when diagnostic data is ready, or forced exit based on the restart policy transmitted by the operating system via SCMI, balancing data integrity and system recovery efficiency.

[0059] Step 11: Perform final operations and clear status flags. After the delay ends, the System Control Processor (SCP) performs final operations according to the restart policy specified by the operating system, including hardware reset and restart or maintaining system freeze and suspension. Subsequently, it automatically clears the error status flags to prepare for the next restart judgment and avoids misjudgment caused by residual status.

[0060] Step 12: Persistent storage of diagnostic data for subsequent analysis. During the delay period, the Management Control Processor (MCP) completes the transfer of diagnostic data to the Baseboard Management Controller (BMC)'s non-volatile memory or flash memory. The data is stored in a non-volatile medium independent of the system's dynamic random access memory, ensuring that even if the system experiences multiple Level 1 errors or fails to restart, historical error information can still be exported by maintenance personnel for root cause analysis.

[0061] Please see Figure 4This application provides a system for handling fatal errors in a server, comprising a Management and Control Processor (MCP), an operating system, a Trusted Firmware Array (ATF), and a System Control Processor (SCP). The MCP is configured to, in response to a fatal error detected by the system hardware, simultaneously save error severity information to a storage area accessible by the ATF. It is also configured to, within a preset delay time window, collect diagnostic data related to kernel crashes based on a notification from the SCP, and transfer the diagnostic data to a designated non-volatile storage medium. The operating system, upon receiving the complete context of the hardware error reported by the MCP, triggers a kernel crash and sends a system restart request to the ATF, the system restart request carrying a specified restart strategy. The ATF, in response to the received system restart request, obtains error severity information pre-stored for the kernel crash, the error severity information representing the severity level of the hardware error causing the kernel crash, and further, in response to the error severity information representing a first level, sends a second instruction to the SCP. The system control processor (SCP) is used to respond to the received second instruction to take all application processor cores offline, and after confirming that all application processor cores have been taken offline, send a notification to the system control processor (SCP) to collect diagnostic data, and perform the corresponding hardware reset operation according to the restart strategy specified by the operating system when the delay time window expires or the diagnostic data transmission is completed ahead of schedule.

[0062] Understandably, in a server fatal error handling system, the Management and Control Processor (MCP) is responsible for storing information and receiving data, the operating system follows standard procedures, the Trusted Firmware (ATF) makes decisions, and the System Control Processor (SCP) executes the decisions. Through their coordinated efforts, diagnostic data is preserved, the operating system code is not modified, and the security of the entire process is guaranteed. For a detailed explanation of the specific implementation process of the aforementioned server fatal error handling system, please refer to any of the above embodiments for a server fatal error handling method; it will not be elaborated upon here.

[0063] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this application. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for handling fatal errors in a server, characterized in that, The processing method, applied to trusted firmware ATF, includes: In response to receiving a system restart request sent by the operating system when a kernel crash is triggered, the system obtains the error severity information pre-stored for the kernel crash, wherein the error severity information represents the severity level of the hardware error that caused the kernel crash; In response to the error severity information being classified as Level 1, the System Controller Processor (SCP) is notified to take all application processor cores offline. After confirming that all application processor cores are offline, the SCP will, within a preset delay time window, notify the Management Controller Processor (MCP) to collect and transmit the collected diagnostic data related to the kernel crash to a designated non-volatile storage medium. When the delay time window expires or the diagnostic data transmission is completed ahead of schedule, the SCP will execute the corresponding hardware reset operation according to the restart policy specified by the operating system.

2. The processing method according to claim 1, characterized in that, The process of generating the system restart request includes: After the Management and Control Processor (MCP) responds to the system hardware detecting a fatal error, collects the complete context of the hardware error, and reports an error report containing the error severity field to the operating system, the operating system, based on the received error report, triggers and completes the kernel crash process, and then sends a system reboot request to the Trusted Firmware (ATF).

3. The processing method according to claim 1, characterized in that, The error severity information pre-stored for the kernel crash is information that is synchronously saved to the accessible storage area of ​​the Trusted Firmware ATF when the Management Control Processor (MCP) reports the complete context of the collected hardware error to the operating system in response to the system hardware detecting a fatal error.

4. The processing method according to claim 1, characterized in that, The step of notifying the system control processor (SCP) to take all application processor cores offline includes: The System Controller Processor (SCP) is instructed to perform at least one of the following operations via the Standardized Power Management Interface (SCMI) command: disable instruction execution on all application processor cores, clear sensitive data from the application processor core cache, disable the direct memory access engine, and isolate access permissions for input / output devices.

5. The processing method according to claim 1, characterized in that, The processing method further includes: In response to the error severity information being classified as not being at the first level, a process is executed in which the trusted firmware ATF or the system control processor SCP directly triggers a hardware reset to restart the operating system without initiating a delay time window.

6. The processing method according to claim 1, characterized in that, The specified non-volatile storage medium is a non-volatile storage medium of dynamic random access memory independent of the operating system.

7. The processing method according to claim 1, characterized in that, The restart strategy specified by the operating system includes hardware reset restart or system freeze suspension.

8. The processing method according to claim 1, characterized in that, After the step of causing the system controller processor (SCP) to perform the corresponding hardware reset operation according to the restart policy specified by the operating system, the processing method further includes: Clear the pre-stored error severity information to prevent residual state from affecting the error severity judgment during the next startup.

9. A method for handling fatal errors in a server, characterized in that, The processing method, applied to the system control processor (SCP), includes: The system receives a first instruction from the Trusted Firmware ATF. The first instruction is an instruction from the Trusted Firmware ATF in response to receiving a system restart request sent by the operating system when a kernel crash is triggered, after obtaining the error severity information pre-stored for the kernel crash, and in response to the error severity information being characterized as level 1, to notify the System Controller Processor (SCP) to take all application processor cores offline. The error severity information represents the severity level of the hardware error that caused the kernel crash. In response to the first instruction, all application processor cores are taken offline; After confirming that all application processor cores have been taken offline, the management control processor (MCP) is notified within a preset delay time window to collect diagnostic data related to the kernel crash and transmit the collected diagnostic data to a designated non-volatile storage medium. When the delay time window expires or the diagnostic data transmission is completed ahead of schedule, the corresponding hardware reset operation is performed according to the restart strategy specified by the operating system.

10. A system for handling fatal errors in servers, characterized in that, The processing system includes: The Management and Control Processor (MCP) is used to synchronously save error severity information to a storage area accessible by the Trusted Firmware ATF when the system hardware detects a fatal error and reports the complete context of the collected hardware error to the operating system. It is also used to collect diagnostic data related to kernel crash within a preset delay time window according to the notification sent by the System Control Processor (SCP) to collect diagnostic data, and transfer the diagnostic data to a specified non-volatile storage medium. The operating system is configured to trigger a kernel crash upon receiving a full context of a hardware error reported by the management control processor (MCP) and send a system reboot request to the trusted firmware (ATF), the system reboot request carrying a specified reboot strategy. Trusted firmware ATF is used to obtain error severity information pre-stored for the kernel crash in response to the received system reboot request, the error severity information representing the severity level of the hardware error that caused the kernel crash, and is also used to send a second instruction to the system controller processor SCP in response to the error severity information being represented as a first level. The system control processor (SCP) is used to respond to the received second instruction to take all application processor cores offline, and after confirming that all application processor cores have been taken offline, send a notification to the system control processor (SCP) to collect diagnostic data, and perform the corresponding hardware reset operation according to the restart strategy specified by the operating system when the delay time window expires or the diagnostic data transmission is completed ahead of schedule.

Images