Computer system and method for detecting bias and non-transitory computer-readable medium

Abstract

Disclosed is a computer system and method and non-transitory computer readable medium for detecting a deviation. The system and method are used to identify, analyze, and repair a deviation in a power-on self-test (POST) routine of a computer system. The computer system includes a basic input / output system (BIOS) that includes a POST routine that is executed between boot-up and hand-off to an operating system. A memory stores a golden image file POST code sequence. A controller and the BIOS receive a sequence of POST codes from execution of the POST routine. The sequence of POST codes is compared to the golden image file POST code sequence. A deviation in the POST routine is identified based on at least one POST code in the sequence of POST codes that does not match the golden image file POST code sequence. The identified deviation can be analyzed for further information in a message. The message can be used to recover from the deviation.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to a startup routine of a computing device. More specifically, aspects of the present disclosure relate to identifying, diagnosing, and repairing a system that encounters a deviation in a power-on self-test (POST) routine of a startup routine. BACKGROUND

[0002] Servers are heavily utilized in high demand applications, such as network-based systems or data centers. The advent of cloud computing has increased the demand for data centers. A data center has many servers that store data and execute applications accessed by remote connected computer device users. A typical data center has a physical rack structure with power and communication connections. Each rack can hold multiple computing servers and storage servers.

[0003] Servers in a data center benefit many commercial services, including executing applications, providing virtualization services, and facilitating network commerce. As businesses rely more on the services provided by data center servers, uptime becomes more important and valuable. When servers fail to operate properly, productivity slows or stops completely, resulting in loss of business profits. Although data center managers can take preventive measures against the disruption caused by errors in servers, the problem of server downtime is difficult to completely eliminate. The main causes of server downtime can include server hardware errors, performing hardware upgrades, performing software upgrades, and unexpected conditions during server maintenance. Given the high cost of potential downtime, data center managers usually have plans to deal with the various possible causes of downtime, however, when recovering servers, the servers need to be restarted.

[0004] Individual servers, when starting up, usually begin operation with a power-on self-test (POST) routine. During the POST routine of a computer system (such as a processor core on a server), the processor core executes a basic input output system (BIOS) to initialize the hardware of the server. After the POST routine is successfully completed, control is then handed over to the operating system of the server.

[0005] The server's boot is a complex process. The Unified Extensible Firmware Interface (UEFI) BIOS firmware must go through a number of steps in the POST routine to initialize and enable all hardware components until the operating system of the server platform is ready. These steps include executing individual programs in sequence to initialize hardware components, enable software functions, perform server security activities, and execute routines for platform error recovery. In general, each software program in the POST routine typically holds important checkpoints to narrate the purpose, behavior, outcome, and status of the software program. Checkpoints are outputted with visible messages that can be displayed on the terminal device's screen, specific LCD panel, LED indicator lights, and / or console redirection. Administrators can also audit the checkpoints by reviewing system error log files at in-band (non-volatile memory of the server) and out-of-band (system error logs stored by the baseboard management controller (BMC)) locations.

[0006] When the server is running in a reliable environment (e.g., stable power supply, cool temperature, and proper humidity), the sequence of checkpoint records will be the same each time the server boots, unless the user has new firmware settings or has adjusted the hardware configuration. Most of the changes in the sequence of checkpoint records are normal operation and expected behavior, such as reference environment parameters and logical operating software programs, thus performing different routines. However, in some cases, it is difficult to predict when, how, and why an interruption will occur. Thus, such events ultimately extend server downtime, impacting the services provided by the data center.

[0007] Therefore, there is a need for a system that includes an audit based on a sequence of POST codes to identify specific deviations in the POST routine. There is also a need for a system that can diagnose deviations in the POST routine. There is also a need for a system that suggests recovery actions for deviations in the POST routine. SUMMARY

[0008] One example disclosed is a computer system to detect a deviation in a power-on self-test (POST) routine. The computer system includes a basic input / output system (BIOS) that includes a power-on self-test (POST) routine having a plurality of stages executed between boot-up and hand-off to an operating system. A controller is in communication with the BIOS. A memory is coupled to the controller and the BIOS. The memory stores a golden image file of a POST code sequence. The controller and the BIOS can receive a sequence POST code from the POST routine. The sequence POST code is generated from execution of the plurality of stages. The sequence POST code is compared to the golden image file of the POST code sequence. A deviation in the POST routine is identified based on at least one POST code in the sequence POST code that does not match the golden image file of the POST code sequence.

[0009] Further implementations of the example system are that the controller is an embodiment of a baseboard management controller. In another implementation, the computer system is a server. In another implementation, the computer system includes a network interface and the sequence POST code is transmitted to a remote management server. In another implementation, the deviation is a detection of a defect in a security function stage, a corruption in a security function stage, or an untrusted condition of the POST routine. In another implementation, the deviation is a hardware device not discovered by an exploratory step in the POST routine, a failed hardware interface, or a hardware speed degradation after the POST routine initializes the hardware device. In another implementation, the deviation is a failed software service, an unavailable software service, or an unsupported software service. In another implementation, the memory is a flash memory that stores the POST code. In another implementation, the golden image file of the POST code sequence is updated by server management software that transmits a golden image file of the POST image file to the controller via the network interface. In another implementation, the controller and the BIOS are further operable to: based on the identified deviation, perform a deviation analysis; store a message that includes results of the deviation analysis; and after the deviation analysis is complete, power down and power up the computer system. In another implementation, the deviation analysis includes a recovery method to repair the deviation and the BIOS and the controller are further operable to automatically perform the recovery method after the power down and power up.

[0010] Another example disclosed is a method of identifying a deviation in a POST routine executed when starting a computer system. The computer system includes a basic input / output system (BIOS). The BIOS includes a power-on self-test (POST) routine having a plurality of stages executed between starting and handing over to an operating system. The computer system includes a controller in communication with the BIOS, and a memory coupled to the controller and the BIOS. A golden image file POST code sequence is stored in the memory. The POST routine of the BIOS is executed. A sequence POST code is received from the POST routine. The sequence POST code is generated from execution of the plurality of stages. The sequence POST code is compared to the golden image file POST code sequence. A deviation in the POST routine is identified based on at least one POST code in the sequence POST code that does not match the golden image file POST code sequence.

[0011] In another implementation of the example method, the controller is a baseboard management controller, and the computing device is a server. In another implementation, the method further includes transmitting the comparison to a remote management server via a network interface. In another implementation, the deviation is a detection of a defective security function stage, a compromised security function stage, or an untrusted condition of the POST routine. In another implementation, the deviation is a hardware device not discovered by an exploratory step in the POST routine, a failed hardware interface, or a hardware speed degradation after the POST routine initializes the hardware device. In another implementation, the deviation is a failed software service, an unavailable software service, or an unsupported software service. In another implementation, the method further includes performing a deviation analysis based on the identified deviation, storing a message including results of the deviation analysis, and powering off and powering on the computer system after the deviation analysis is completed. In another implementation, the method further includes repairing the deviation via data in the message.

[0012] Another example disclosed is a non-transitory computer readable medium storing software instructions that, when executed by a processor in a computer system, cause the processor to execute a plurality of stages of a power-on self-test (POST) routine of the computer system. The instructions further cause the processor to output a sequence POST code during execution of the POST routine. The instructions further cause the processor to compare the sequence POST code to a golden image file POST code sequence. The instructions further cause the processor to identify a deviation in the POST routine based on at least one POST code in the sequence POST code that does not match the golden image file POST code sequence.

[0013] Another example disclosed is a computer system that analyzes a bias in a power-on self-test (POST) routine. The computer system includes a basic input / output system (BIOS) having a power-on self-test (POST) routine that has a plurality of stages performed between boot-up and hand-off to an operating system. The computer system includes a controller in communication with the BIOS. The controller and the BIOS are operable to receive a bias identified in the POST routine in the form of a POST code. The controller and the BIOS perform a bias analysis based on the identified bias. The controller and the BIOS store a message in a destination device upon completion of the bias analysis, the message including a result of the bias analysis.

[0014] Further implementations of the example system are a controller that is an embodiment of a baseboard management controller. In another implementation, the computer system is a server. In another implementation, the bias analysis is enabled by setting a UEFI variable in the BIOS. In another implementation, the message includes an automatic recovery of the POST from the bias. In another implementation, the recovery is manual, the controller alerts an administrator and powers down the computer system. In another implementation, the message includes a human-readable summary and a technical summary. In another implementation, the human-readable summary includes a definition of the bias, a cause of the bias, a method of recovery, and a risk assessment. In another implementation, the technical summary includes metadata of various lengths and content to describe the bias for a particular service routine. In another implementation, the recovery routine indicated by the message is performed by the controller or BIOS firmware. In another implementation, the recovery routine includes optimizing hardware component initialization, clearing suspect data structures, or reloading custom golden settings. In another implementation, the bias analysis routine is written specifically for the identified hardware or software component based on files provided by the hardware or software developer. In another implementation, the destination device is a system memory, a storage device, or a flash memory accessible by the controller.

[0015] The foregoing summary is not intended to represent every implementation of or aspect of the present disclosure. To the contrary, the foregoing summary is provided merely for purposes of reciting certain novel concepts and features of some implementations included herein. Other features and advantages of the present disclosure, as well as the BRIEF DESCRIPTION OF DRAWINGS

[0016] The present disclosure will be better understood from the following descriptive description of representative embodiments and modes presently contemplated for the implementation of the application, together with the attached drawings.

[0017] Figure 1is a block diagram of a computer system in accordance with certain aspects of the present disclosure, the computer system having components that execute an example POST routine that includes modules to identify, diagnose, and repair deviations in the POST routine.

[0018] Figure 2 is a program diagram in accordance with certain aspects of the present disclosure, showing different steps in an example POST routine and corresponding POST codes.

[0019] Figure 3 is a program diagram in accordance with certain aspects of the present disclosure, showing a diagnosis from an example module of a POST routine.

[0020] Figures 4A-4C is a flow diagram of a function in accordance with certain aspects of the present disclosure to identify, diagnose, and repair an example module of a POST routine.

[0021] Figure 5 is a code sequence for a secure boot service in a POST routine in accordance with certain aspects of the present disclosure, the POST routine including a pass state service routine of a diagnostic function of an example module.

[0022] Figure 6 is a flow diagram of a function service routine in accordance with certain aspects of the present disclosure to collect Figure 5 a state of a function call routine.

[0023] Figure 7 is a flow diagram of a function call service routine in accordance with certain aspects of the present disclosure to expose Figure 5 a message of a function call routine.

[0024] Figure 8 is a table of possible recovery mechanisms in accordance with certain aspects of the present disclosure, the possible recovery mechanisms being based on error diagnoses outputtable by a recovery function of an example module.

[0025] The present disclosure is susceptible to numerous modifications and alternative forms. Certain representative embodiments have been shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the application is not to be limited to the particular embodiments described, as these can vary. It is to be understood that the application is not to be limited to particular

[0026] NOTATION AND APPENDICES

[0027] 100: remote system

[0028] 102: service station

[0029] 104: network

[0030] 106: server

[0031] 108: technician

[0032] 110: baseboard management controller (BMC)

[0033] 112: network interface card or network interface controller

[0034] 120: basic input / output system (BIOS)

[0035] 122: flash memory

[0036] 124: golden file

[0037] 126: system error log (SEL)

[0038] 130: central processing unit

[0039] 132: hardware component

[0040] 134: software program

[0041] 140: operating system (OS)

[0042] 200: program diagram

[0043] 210: security phase

[0044] 212: pre-EFI initialization (PEI) phase

[0045] 214: driver execution environment (DXE) phase

[0046] 216: boot device selection (BDS) phase

[0047] 218: transient system load (TSL) phase

[0048] 220: prevalidator

[0049] 222: Ucode patching

[0050] 224: security routine

[0051] 230: ultra path interconnect (UPI) / interface to interface memory interface (XGMI) topology discovery program

[0052] 232: start memory reference encoding program

[0053] 234: initial platform port program

[0054] 240: driver

[0055] 242: Peripheral Component Interconnect (PCI) scan and resource allocation program

[0056] 244: Establish boot and run-time services program

[0057] 246: Publish Advanced Configuration and Power Interface (ACPI) and System Management BIOS (SMBIOS) table program

[0058] 250: Start dispatcher program

[0059] 252: Enable secure boot program

[0060] 254: Load boot loader program

[0061] 260: Short operating system boot loader program

[0062] 262: Operating system handoff program

[0063] 270: POST code

[0064] 272: First offset

[0065] 274: Second offset

[0066] 300: Program diagram

[0067] 310: Series of messages

[0068] 312: First message

[0069] 314: Second message

[0070] 402-434: Steps

[0071] 436-466: Steps

[0072] 470-498: Steps

[0073] 500: Code segment

[0074] 510: Gather status routine

[0075] 512: Publish message routine

[0076] 520: Message

[0077] 522: Human-readable schema

[0078] 524: Technical schema

[0079] 530: Deliver status routine

[0080] 610-614: Steps

[0081] 620-644: Steps

[0082] 710-714: Steps

[0083] 720-728: Steps

[0084] 800: Table

[0085] 810, 820: Columns DETAILED DESCRIPTION

[0086] The present application can be embodied in many different forms. Representative embodiments are shown in the drawings and will be described in detail herein. The present disclosure is an exemplification of the principles of the application and is not intended to limit the broad aspects of the present application to the embodiments depicted. Elements and limitations of the presently disclosed embodiments are to be understood to be among the many possible embodiments for carrying out the principles of the application and examples set forth herein. In this regard, the disclosure is to be regarded as illustrative, rather than restrictive, of the broad concepts of the application. Elements and limitations of the presently disclosed embodiments are to be understood to be among the many possible embodiments for carrying out the principles of the application and examples set forth herein. In this regard, the disclosure is to be regarded as illustrative, rather than restrictive, of the broad concepts of the application. The mere fact that an element is not recited in the detailed description does not indicate that the element is not of importance. Additionally, reference throughout this security specification to "one embodiment" or "an embodiment" can mean that a particular element described will be present in some embodiments, but not necessarily in others. Thus, on a continuing basis, this description is intended to ensure that at least some embodiments of the present application are fairly described. Moreover, the description is intended to cover any adaptations or variations of the specific implementa- tions discussed herein. It is intended that the application encompass all such adaptations or variations.

[0087] Examples disclosed herein include a firmware module that can be executed in UEFI BIOS firmware and BMC firmware to assist in identifying and responding to deviations in the power-on self-test (POST) routine in a computer device. The example module identifies anomalies in the sequence checkpoints output by the POST routine by comparing the output POST code to the POST code of a golden record POST routine each time the server is booted. The routine can determine potentially abnormal and harmful boot sequences on the server that can reduce performance, increase downtime, and perform non-secure functions. Identifying anomalies in the sequence checkpoints from the POST routine requires tracing the execution path of the POST routine and providing the necessary data structures to the operating system (OS). The routine outputs the state of the secure function and the results of the motherboard hardware initialization through the sequence POST code. The routine includes novel modules to determine deviations in the sequence checkpoints in the POST routine, determine the cause of each divergence, estimate the risk of the divergence, and then suggest appropriate manual or automated recovery mechanisms.

[0088] Figure 1A remote system 100 is shown that enables management of remote devices, such as servers in a data center. The remote system 100 includes a remote server management and service station 102 accessible by a data center operator, and a network 104, and remote computer systems, such as a server 106. The remote server management and service station 102 allows the data center operator to access a management interface that allows the operator to perform remote diagnostics of the server 106. As will be explained below, the service station 102 also receives operational data from the server 106. Other personnel, such as a technician 108, can perform human intervention based on a schedule provided by the service station 102, such as replacing a faulty hardware component on the server 106.

[0089] The server 106 includes a baseboard management controller (BMC) 110. The BMC 110 includes a network interface card or controller 112 that is coupled to the network 104. The BMC 110 is coupled to a boot firmware, such as a Unified Extensible Firmware Interface (UEFI) basic input output (BIOS) firmware 120. The BMC 110 uses a flash memory 122 to store relevant operational data about the server 106, as well as other firmware executed by the BMC 110. In this example, the flash memory 122 stores a golden file 124 and a system error log (SEL) 126. In this example, the golden file 124 can be downloaded by the BMC from the service station 102. As will be explained below, the golden file 124 includes an expected POST code sequence resulting from the execution of a POST routine. The server 106 includes hardware components 130 and 132 that can perform functions such as storage, computation, and switching. For example, the hardware component 130 can be a central processing unit (CPU). Other hardware components 132 can be other processors, memory devices (e.g., dual inline memory modules (DIMMs), serial advanced technology attachment (SATA) devices, and similar devices), peripheral component interconnect express (PCIe) devices, and the like. It should be understood that there can be many hardware components on a typical server.

[0090] The UEFI BIOS firmware 120 performs a POST routine that optimizes the initialization of the hardware components of the server 106, such as the hardware components 130 and 132. The BMC 110 can also perform hardware diagnostics of the hardware components of the server 106, such as the hardware components 130 and 132. The BMC 110 can also monitor the health status of the hardware components of the server 106, such as the hardware components 130 and 132. The POST routine can also interact with the POST software program 134 to enable debug messages and collect all status and data structure information from such program. Once the POST routine of the UEFI BIOS firmware 120 has completed, the operating system (OS) 140 begins the operation of the server 106.

[0091] In the present example, the UEFI BIOS firmware 120 includes an example module in the POST routine to report anomalies in the sequence checkpoints output by the POST routine. The example module in the POST routine identifies deviations of the sequence checkpoints of the POST routine from the checkpoints of the stored golden record. Identifying, analyzing, and recovering from the deviations makes the server 106 more resilient and efficient. Thus, the example module provides an analysis of the anomalies in a human-readable metadata format, such as a Redfish representational state transfer (REST) application programming interface (API) format of JavaScript object notation (JSON), to the BMC 110, which can then be collected by server management software executed by the service station 102. The manager / IT engineer can provide policy definitions to address the deviations. For example, the policy can instruct the server 106 to immediately perform an automatic recovery procedure or to wait for a scheduled time for human intervention by the technician 108.

[0092] In general, recording the POST codes output by the UEFI BIOS firmware 120 between the booting of the server 106 and the handoff to the OS 140 is a very useful software forensics method. Software developers use unique American Standard Code for Information Interchange (ASCII) codes for POST codes for a variety of purposes. The ASCII coded POST codes are embedded at important execution points such as the start, end, and branching of firmware such as the UEFI BIOS firmware 120. The UEFI BIOS firmware 120 redirects the output POS codes to accessible destinations. The POST codes are sequentially recorded from the first output code to the last output code as a history of the UEFI BIOS firmware boot sequence of the server 106.

[0093] Figure 2 is a program diagram 200 showing different sequence stages of a POST routine from initial boot to handoff of control to the operating system 140. Here, the first column 210 represents the security stage. The second column 212 represents the pre-extensible framework interface initialization environment (pre-EFI; PEI) stage. The third column 214 represents the driver execution environment (DXE) stage. The fourth column 216 represents the boot device selection (BDS) stage. The fifth column 218 represents the transient system load (TSL) stage.

[0094] The security stage 210 is initiated by a pre-verifier 220. The pre-verifier 220 applies a Ucode patch 222 and starts a silicon-based security routine 224. The PEI stage 212 includes an ultra path interconnect (UPI) / socket-to-socket global memory interface (XGMI) topology discovery routine 230, a start memory reference code routine 232, and an initial platform porting routine 234.

[0095] A set of drivers 240, such as device drivers, bus drivers, or service drivers, implement the DXE phase 214. Each driver continues to execute until the DXE phase 214 ends. This phase includes peripheral component interconnect (PCI) scan and resource allocation routines 242, establish boot and runtime services routines 244, and publish advanced configuration and power interface (ACPI) and system management BIOS (SMBIOS) table routines 246.

[0096] The boot device selection phase 216 includes a boot dispatcher routine 250, enable secure boot routines 252, and load boot loader routines 254. The transient system load (TSL) phase 218 includes transient operating system boot loader routines 260, and operating system handoff routines 262. At the end of the transient system load phase 218, control is handed over to the OS boot loader, such as Linux GRUB, and the operating system 140 boots and begins.

[0097] As each of the routines of phases 210, 212, 214, 216, and 218 are initiated and completed, the POST routine outputs a set of sequence POST codes 270. In the present example, the UEFI BIOS firmware image file 120 has a complete table of POST codes to describe the function definition of each code. Thus, the code definition of each code can assist an administrator or developer to understand the execution of each step of the POST routine. A normal sequence of POST codes for the security phase 210 is 0x00-03, a normal sequence of POST codes for the PEI phase 212 is 0x10-13, a normal sequence of POST codes for the DXE phase 214 is 0x20-23, a normal sequence of POST codes for the boot device selection phase 216 is 0xA0-A3, and a normal sequence of POST codes for the TSL phase 218 is 0xB0-B3.

[0098] If one of the programs encounters a deviation, an alternative POST code is output and in the POST code sequence. In this example, the first deviation 272 replaces the POST code 0x12 of the initial platform port program 234 with the POST code 0xF1, indicating an error in the platform porting. Thus, the POST code 0x12 is part of the service routine "initial platform port" that performs specific hardware initialization, such as setting up the output port settings, and writing and read tests to determine if they are successful. If the read test is not correct, the POST code 0xF1 is sent out instead of the normal POST code 0x12. The second deviation 274 replaces the normal POST code Al of the secure boot enable program 252 with the POST code FF, indicating an error in the program. The POST code 0xA1 indicates hardware initialization of the trusted platform module (TPM) to perform secure functions, and the controller's write and read status are successful. When the read status is not correct, the POST code 0xFF is sent out instead of the normal POST code 0xA1. In this example, these deviations affect the ability to hand over control to the operating system 140 because the operating system 140 can not function due to a specific port or controller malfunction.

[0099] The example modules in the example POST routine have three main functions to assist in ensuring an efficient POST routine even in the presence of deviations in the POST routine. These three functions include deviation identification, deviation analysis, and deviation recovery. The deviation analysis function is automatically invoked after a harmful deviation is identified from the sequence POST codes. Harmful deviations can include conditions such as security functions of the server being disabled or reduced, hardware communication interface failure or degradation, software protocol or service invalid or unsupported, and the like.

[0100] The deviation identification function thus compares the records of the POST sequence codes output by the UEFI BIOS firmware 120 during the POST routine to the POST sequence codes of the golden file image 124. The deviation analysis function makes intelligible messages from any anomalies in the sequence checkpoints. The messages output include human-readable and technical schemas related to the deviation. The deviation analysis function also diagnoses the suspect hardware components and collects additional information from the server 106. The deviation recovery function performs different recovery steps to mitigate the deviation, such as optimizing hardware component initialization, clearing suspect data structures in the non-volatile memory, and reloading BIOS firmware self-tailored golden settings.

[0101] Figure 3is a program map 300 indicating information provided by the example module in the POST routine. The module outputs the POST code sequence 270, which is compared to Figure 1 the POST sequence of the golden image file 124 in the

[0102] The example module performs anomaly analysis and outputs a series of messages 310. In this example, the anomaly analysis function outputs messages that include human-readable summaries and technical summaries. The human-readable summaries include a definition of the POST code anomaly, a cause of the anomaly, whether the anomaly is recoverable, a corresponding recovery method, and a risk assessment. For example, based on the anomaly analysis, the risk assessment can be a score between 1 and 5. In this example, the risk assessment score is determined from a table. In this example, a risk score of 1 is given when a hardware function fails, or a software function fails and cannot be immediately re-enabled. A risk score of 2 is given when a security function fails without permission, but the function can be re-enabled. A risk score of 3 is given when a hardware function is downgraded, but a retry mechanism (such as a speed reduction or link width reduction) can be employed. A risk score of 4 is given when a software function is disabled without permission, but the function can be re-enabled. A risk score of 5 is given when a software data structure or layout is incorrect, but a retry mechanism (such as a data buffer overflow) can be employed.

[0103] As shown in Figure 3 , the first message 312 output by the example module includes human-readable information related to the anomaly in the initial platform port 234. The second message 314 includes human-readable information related to the anomaly in the secure boot enabler 252.

[0104] In this example, the anomaly detection function compares the POST code corresponding to the golden image file of the UEFI BIOS firmware 120 to the actual POST code. To detect anomalies in the POST code of the UEFI BIOS firmware 120, a golden file of the sequence POST code representing the available configuration of the server 106 (such as the golden file 124 in Figure 1 ) is stored in a non-volatile memory, such as the flash memory 122 of the BMC 110. The contents of the golden file 124 represent only the essential sequence POST code that is executed during the most critical parts of the initialization sequence. For example, 0xA0, 0xA1, 0xA2 are essential sequence POST codes output by the program of the power-on device selection phase 216.

[0105] The "Rack Server Correct POST Code Maintenance" function enabled by the administrator redirects each POST code output by the UEFI BIOS firmware 120 during boot-up to be stored in an accessible destination, such as the flash memory 122 or other BMC memory. Thus, a remote server, such as the management station 102, can access the POST code output. In the present example, the BMC 110 executes a module of the POST routine to compare whether the POST code output by the POST routine of the server 106 matches the sequence of POST codes in the contents of the golden file 124 of POST codes. If the comparison of POST codes shows a POST code that is different from what is expected, thereby detecting a harmful deviation, the BMC 110 enables the "Deviation Analysis" and "Deviation Recovery" intervention service routines in the UEFI BIOS firmware 120 and the BMC 110.

[0106] Figures 4A-4C is a flowchart showing three phases of the routine performed by the example module. The three phases include the deviation identification, deviation analysis, and deviation recovery functions. Figure 4A shows the overall routine, the deviation identification function performed by the UEFI BIOS 120 and the BMC 110 in Figure 1 In the present example, the UEFI BIOS 120 starts recording the POST routine at step 402 to the system error log (SEL) 126 of the BMC 110. Thus, the initial POST code (0xE0) is recorded. Then, the UEFI BIOS 120 initializes the processor, memory, and other components on the motherboard at step 404. Then, the UEFI BIOS 120 initializes the input / output hardware at step 406. Then, the UEFI BIOS 120 installs the boot services at step 408. Then, the UEFI BIOS 120 installs the runtime services at step 410. Each of the POST steps 402, 404, 406, 408, 410 triggers the sending of a POST code 270 to the BMC 110 to indicate the start and completion of the various steps.

[0107] Then, the UEFI BIOS 120 enables secure boot at step 412. In the present example, a deviation occurs that is reflected in the POST codes, such as Figure 2POST code 274 in the POST code sequence 270. After secure boot, the UEFI BIOS 120 records the end of POST to the SEL of the BMC 110 at step 414 by POST code (0xE1). The UEFI BIOS 120 then determines whether the results of the POST code and bias analysis should be enabled from the BMC 110 at step 416. If no bias is found, the POST routine ends at step 418, handing over to the OS 140. If a bias is found, such as the POST code 274 generated by the secure boot procedure, then the BIOS bias analysis is initiated at step 420.

[0108] For example, certain hardware biases can occur that cause specific POST codes to be output in the POST code sequence. Such hardware biases can occur due to a hardware device not being discovered from the discovery step in the POST routine, a hardware interface failing after initialization, or a hardware device failing at a reduced speed after initialization. In such a case, the bias would cause an error or warning code to be output in the POST code sequence. Another example of a bias in the POST routine can be a security type bias, such as a failure of a security function that can occur due to the absence of a Root of Trust. In such a case, the bias would cause an error or warning code to be output in the POST code sequence. Another security bias can be the detection of an untrusted condition, such as the loading of an unsigned UEFI driver. Such a bias can be indicated by an incomplete POST sequence, as the unsigned driver would not have a POST code output. Another example bias is a software based bias, such as an unsupported software protocol from an installation step, or an unavailable data structure from an initialization step. In such a case, the bias causes an error or warning code to be output in the POST code sequence.

[0109] During the deviation discrimination function in FIG. 4, the BMC 110 determines at step 422 whether the POST code maintenance that allows the service routine to be performed is enabled by the administrator. If the POST code maintenance is not enabled, the BMC 110 does not make further movement. If the POST code maintenance is enabled, the BMC 110 receives at step 424 the POST codes 270 output by the POST routine of the UEFI BIOS 120 and stores the POST codes 270 in a POST code log file in a memory, such as the system memory or the flash memory 122. After receiving each POST code, the BMC 110 determines at step 426 whether the received POST code indicates the end of the POST routine. If the received POST code does not indicate the end of the POST routine, the BMC 110 goes back to step 424 to continue receiving and storing the POST codes. If the received POST code indicates the end of the POST routine, the BMC 110 loads at step 428 the POST code golden file 124 from the flash memory 122. Then, the BMC 110 compares at step 430 the POST code log file with the POST code sequence from the POST code golden file. At step 432, the BMC 110 determines whether a harmful deviation is found. If no harmful deviation is found, the BMC 110 ends the routine. If a harmful deviation is found, the BMC 110 starts at step 434 the BMC deviation analysis and BMC deviation recovery routines. The BMC 110 also notifies the UEFI BIOS 120 about the finding of the harmful deviation.

[0110] Since the software algorithm of the initialization program in the POST routine can add, delete, or modify steps with new POST codes, the definition of the golden image file POST code sequence can be upgraded by the management software running on the service station 102. When the server 106 has a new UEFI BIOS firmware release, such management software sends the new golden image file to the BMC 110. The upgrade of the golden POST code sequence avoids the incorrect indication of deviations from the old golden POST code file. The server management software can also invalidate the old version of the golden POST code file in the database of the server BMC. In the present example, the database has multiple pairs of BIOS image files and golden files and can be managed by the service station 102. When the administrator decides to upgrade the firmware version of the server 106, the administrator will upgrade the corresponding golden file. This procedure can be achieved by sending a new firmware package containing the firmware image file, the golden file, and the script file through an upgrade protocol, such as the Redfish firmware upgrade protocol.

[0111] Alternatively, the deviation can be a known, accepted compromise. For example, a POST code deviation can result from an accepted problem by the administrator, such as using low cost dual in-line memory modules that have a reduced frequency problem, thus causing a deviation in the POST code from the golden file POST code. The administrator can choose to leave the modules in place, thus resulting in a continuing POST code deviation.

[0112] Once a harmful deviation is detected, the BMC 110 begins executing the "Deviation Analysis" portion of the example module in the UEFI BIOS firmware 120 and the BMC 110. In the present example, the BMC 110 requests the UEFI BIOS firmware 120 to enable a UEFI variable ("Deviation Analysis") that allows the suspicious software program of the POST routine to send a message to an accessible memory destination regarding the deviation. Since this message is built into the compiler and linker of the UEFI BIOS firmware 120 and only takes effect when the specific UEFI deviation analysis variable is enabled, the output message does not change based on the status of the deviation in the sequence POST code in future boots. The UEFI BIOS firmware 120 will ask the BMC 110 if it wants to run the "Deviation Analysis" routine before handing over to the operating system 140. Once the specific case has been confirmed from the deviation analysis between the UEFI BIOS firmware 120 and the BMC 110, another UEFI variable is enabled. Thus, the BMC 110 does not hand over to the operating system 140, but instead powers off and powers on the server to run the deviation analysis.

[0113] Figure 4B The deviation analysis function of the example module is shown. At step 434, the BMC 110 begins deviation analysis based on the identification of the deviation. At step 436, the BMC 110 requests the UEFI BIOS 120 to set the UEFI variable "Deviation Analysis." Then, the BMC 110 checks at step 438 if the UEFI BIOS 120 has granted permission. If the permission has not been granted, the BMC 110 continues to check at step 438 if the UEFI BIOS 120 has granted permission. If the permission has been granted, the BMC powers off and powers on the server at step 440.

[0114] In response to the request from the BMC, the UEFI BIOS 120 sets the UEFI variable "Bias Analysis" at step 442 to initiate the bias analysis. The UEFI BIOS 120 then authorizes the BMC 110 to begin the bias analysis function at step 444. The UEFI BIOS 120 then waits for the BMC to power the server off and back on at step 446. Once the server 106 begins the power off and back on initiated by the BMC 110 at step 440, the POST routine begins and the UEFI BIOS 120 logs the occurrence of the bias analysis to the SEL 126 in the flash memory 122 at step 448. The UEFI BIOS 120 then exposes the additional information received from the BMC in the bias analysis at step 450. Because the entire UEFI BIOS initialization sequence is rerun, the UEFI BIOS 120 exposes the additional information after the power off and back on. Because the UEFI variable "Bias Analysis" is enabled, the UEFI BIOS POST routine will collect more useful hardware component health status, software data structures, software retry counts, and recovery mechanism data. The POST routine translates the above data into additional information with human-readable data and machine data that is delivered to a physical destination, such as the flash memory 122 that is accessible to the BMC 110.

[0115] The UEFI BIOS 120 then clears the UEFI variable "Bias Analysis" at step 452. The UEFI BIOS 120 then waits for the BMC 110 to begin bias recovery in the case of an automatic recovery and waits for the server 106 to power off in the case of human intervention at step 454.

[0116] At step 456, the BMC 110 monitors the server 106 and determines whether the POST routine has begun after initiating the power off and back on. If the POST routine has not begun, the BMC 110 continues to monitor at step 456 to determine whether the POST routine has begun. If the POST routine has begun, the BMC 110 collects data from the server regarding the bias analysis at step 458. The routine to collect the data needed for the bias analysis is explicit for the hardware and software that is identified as potentially biased. The bias analysis routine can be written by the developer of the UEFI BIOS firmware based on the specifications and recommendations provided by the hardware or software vendors for the components of the server 106.

[0117] At step 460, the BMC 110 checks the deviation analysis to determine if the POST sequence can be recovered to fix any or all of the identified deviations. In the present example, the administrator can choose to recover each deviation separately, or to recover all deviations at once. If the POST sequence cannot be recovered from the deviations, the BMC 110 sends an alert or alarm to the management software on the service station 102 over the network 104 at step 462. Then, because human intervention is required, the BMC 110 powers down the server at step 464. As described above, the software of the service station 102 can schedule a time for a technician to check and repair the server 106, and / or provide the administrator the ability to decide on other repair actions. If, based on the deviation analysis, the deviations can be handled and thus the POST sequence can be automatically recovered, the BMC 110 starts the deviation recovery procedure at step 466.

[0118] The service sub-routine, referred to in the present example as "DeliverStatus," performs built-in analysis and can be included in any critical initialization service routine of the UEFI BIOS firmware 120 of the mother layer. In the present example, the DeliverStatus sub-routine is added to the hardware device detection service routine of the secure boot service routine of the UEFI BIOS firmware 120. The DeliverStatus service routine only functions when the UEFI variable "Deviation_Analysis" is set to true at step 442 in Figure 4B The DeliverStatus service routine has two purposes. First, the DeliverStatus service routine collects the health status of the hardware devices of the server by running diagnostic routines. Second, the DeliverStatus routine determines which software information to expose and then stores the information to an accessible memory destination, such as a host memory region, or a partition of a storage device accessible by the BMC 110 (e.g., the flash memory 122).

[0119] The fault analysis information is stored in the software messages output by the delivery status routine. Each message contains a generic header with human-readable categories, and a complex payload with technical categories related to the fault. In the present example module, the generic header uses generic categories with human-readable data and can be easily adopted by the fault recovery function of the example module, the technical categories are used to describe the exact status of a particular service routine in the order of execution by using custom metadata with various lengths and contents. In this way, the technical categories provide service engineer data for in-depth analysis of the fault cause.

[0120] Figure 5 is Figure 4B The code segment 500 of the example of the "DeliverStatus()" service routine run in the software service routine "SecureBoot_Start()" of the POST routine executed by the UEFI BIOS firmware 120 in the fault analysis in the expose message step 450 in the present disclosure. In the present example, the secure boot is triggered by receiving the POST code 0xA0. The secure boot code segment 500 accesses two mandatory service routines of the delivery status service routines: the collect status routine 510 and the expose message routine 512. The service routines 510 and 512 are built-in in the example "DeliverStatus" routine 530 of the sequence POST code fault analysis. In the present example, when the UEFI variable "Fault Analysis" is true, the routine collects the device health status based on a particular function call according to a sub-routine named "Collect_Status_Of_Function_Call". The function call is written based on a particular service routine with POST code fault identification supported by the sequence of POST codes originated from the golden file. For example, the function call for a particular hardware component can support reading the health status register of the hardware component associated with the POST code. The collected health status is stored in a particular memory location. In the present example, the particular function call sub-routine causes the collected health status data to be stored in a readable format, such as the JSON format, by executing the service routine "Expose_Message_of_Function_Call".

[0121] The starting point for a safe boot is indicated by the POST routine outputting the POST code 0xA0. The pass status routine 530 is only invoked if the hardware detection routine determines that the POST code output by the POST routine deviates from the corresponding POST code in the golden file POST code sequence. The pass status routine 530 loads the UEFI variable "Deviation_Analysis", as well as the address of the hardware device detection. This causes the routine to return for an explicit function call for the deviating POST code and pass status data. In the case where the UEFI variable "Deviation_Analysis" is set to true, the pass status routine 530 executes the function call again and then the bit deviation message build data structure.

[0122] For example, the message 520 can be generated and stored to a physical destination, such as host memory, storage, or BMC flash. The message 520 includes a human readable section 522 and a technical section 524. In this example, the human readable section 522 is in JOSN format and includes the definition of the deviation, the cause of the deviation, whether recovery is possible, a proposed fix, and a risk factor reflecting the severity of the deviation. The information in the human readable section is output by a specific function call attached to a specific firmware or software component. In this example, the technical section 524 includes a register dump from a specific memory allocation range of the device, resource allocation data, additional sequence messages, function settings, configuration settings, and data structures built by the service routines.

[0123] As described above, the service routines 510 and 512 collect hardware device health status and expose messages for software function calls. The service routines do not interfere with the original behavior of the POST run order or modify any UEFI parameters. The collection of hardware device health status "Collect_Status_Of_FunctionCall" 510 is used to collect relevant technical information on the hardware device health status, system resource allocation of the hardware device, and software data structures. The other service routine "Expose_Message_Of_FunctionCall" 623 is used to expose or store a message containing the data collected by the collect status service routine 510 to an accessible destination. For example, Figure 5 The pass status routine is an extremely important service routine in the hardware device detection service routine of

[0124] The pass status sub-routine can be added to any UEFI BIOS firmware service routine that is extremely important and runs on a different execution path, such asFigure 5 The DIMM device detection service routine detects the service. Alternatively, the pass status sub-routine can be added to the platform memory initialization service routine that invokes the DIMM device detection sub-routine. The status detection of the DIMM can trigger an alternate POST path. If the detection sub-routine does not find a DIMM device, a smaller platform memory size is reported. Thus, initialization routines that require a large amount of memory are changed, resulting in an alternate POST path. The pass status routine can pass the device health status as a result of the bias of the POST code.

[0125] Figure 6 is a flow chart of the collect status routine 510 that is a function call service of the pass status routine. The pass status routine is Figure 5 the secure boot routine in the normal POST execution path. During the period of bias analysis, the collect status routine 510 re-executes the hardware device detection service routine in the POST diagnostic execution path. The address is passed so that the device detection service routine can access the function call routine. Thus, the collect status routine 510 first gets the address of the function call routine from the input parameters of the pass status routine of the mother layer in step 610. Then, the routine initializes the data buffer in step 612. Then, the routine re-executes the function call in step 614.

[0126] In step 620, the collect status routine exits to the function call routine that audits the parameters for whether a bias has occurred. Based on the parameters, the routine determines in step 622 whether the function call needs to be re-executed for diagnostic analysis and thereby collect additional diagnostic data. If not, the routine follows the normal POST R execution path in step 624. Then, the function call routine returns the status of the function call to normal in step 626 and continues the POST path in step 628.

[0127] If the function call routine needs to be executed again, the routine begins diagnosing the execution path in step 630. In step 632, the routine obtains the address of the data buffer from the input parameters. Then, in step 634, the routine performs hardware diagnostics, checking the status of health registers based on the specific hardware or software causing the deviation. If, based on the analysis, the deviation is recoverable, the routine sets a recoverable flag in step 636. Then, in step 638, the routine reads hardware registers and retrieves resource allocation data associated with the hardware. Then, in step 640, the routine writes the cause of the deviation, risk factors, whether the deviation is recoverable, recommended recovery methods, and other information collected from registers and other data sources to the data buffer. Then, in step 642, the routine converts the data on the cause of the deviation, risk factors, whether the deviation is recoverable, and recommended recovery methods into a human-readable outline in a human-readable format. Then, in step 644, the routine converts other selected data into a technical outline.

[0128] Figure 7 This is a flowchart of the public message routine 512 for the function call service. In step 710, the public message routine 512 first obtains the address of the data buffer from the input parameters set in the collection status routine 510. Then, the routine identifies the destination of the message output in step 712. In step 714, by determining the destination, the message is stored to re-execute the hardware diagnostic routine. Then, the routine sets the destination to physical storage in step 720, or to flash memory 122 in step 722, or to physical storage, such as system memory, in step 724. Then, in step 728, the routine transmits the data buffer to the designated destination via a transport protocol.

[0129] In the diagnostic routine shown in Figure 4, BMC 110 is... Figure 5 The system searches for human-readable content in message 520, which is received from the UEFI BIOS firmware 120 running the public messaging service routine 512. In step 460, the BMC 110 can identify whether harmful analysis is recoverable from the data in the human-readable content. Based on this data, the BMC 110 determines whether it should initiate the "deviation recovery" function of the example module or freeze the server by sending an alert message to the server management software.

[0130] Deviation recovery mechanism is Figure 5UEFI BIOS firmware specific function call service routine 510 and integrity method of the supported hardware devices. For example, the developers of the UEFI BIOS firmware 120 pair appropriate recovery methods to specified function calls. The specified function calls have the "pass information" service routine 500 installed. Once the recovery method is executed, the administrator / IT engineer observes whether the recovery mechanism reversed the bias and returned to the normal functioning POST routine, thereby outputting the correct POST code into the POST code sequence. In this example, the recovery method is a design developed by firmware engineers for various hardware components. Similar function calls can be paired to different software recovery methods initiated by the POST routine.

[0131] Figure 8 Table 800 is a table of certain example recovery methods for different hardware and software components that can be recovered by different function calls. Figure 8 Column 810 lists different errors of discoverable hardware or software of the server 106. Another column 820 lists recovery mechanisms that can be performed by the server management software running on the server 106.

[0132] The course of bias recovery first refers to the automatic or manual recovery strategy determined by the bias analysis routine in FIG. 4. The automatic strategy setting allows the UEFI BIOS firmware 120 and the BMC 110 on the server 106 to automatically reverse the effects of the bias identified on the POST routine. This can involve Figure 8 The movement shown in the table in FIG. 8 can be set by firmware developers. The manual strategy setting allows for manual intervention at a scheduled time, such as alerting a technician to replace or update a hardware component, or to restart software. The strategy setting can be set by the server management software on the remote management station, such as the service station 102 in FIG. 1. The strategy setting is sent to the BMC 110 for reference by the example module in the POST routine. Figure 1

[0133] The automatic bias recovery function includes the BMC 110 initiating the bias recovery procedure by using recovery mechanisms built into the UEFI BIOS firmware 120 or recovery mechanisms performed by the BMC 110. The automatic recovery mechanisms can allow the server 106 to correct the POST sequence, but can not necessarily completely remove all biases. Therefore, a technician can need to schedule a time for manual inspection in the message and for manual intervention for complete server diagnosis.

[0134] As Figure 4C ​As shown, deviation recovery is initiated by the BMC 110 at step 466. The BMC 470 first determines whether the recovery is manual or automatic by passing the message output by the status routine 500 at step 470. If the recovery is manual, the BMC 110 allows the recovery routine to be scheduled at a selected time based on a server maintenance schedule arranged by an administrator at step 472. If the recovery is automatic, the BMC determines whether the recovery is performed by the BMC 110 at step 474. If the recovery is performed by the BMC 110, the BMC 110 performs the recovery routine according to specific hardware or software at step 476. For example, the BMC 110 can reset, and enable or disable physical hardware interfaces. The BMC 110 can power on and off to force hardware logic back to a stable state. The BMC 110 can use a Non-Volatile Memory Express (NVMe) management interface to reset NVMe devices.

[0135] If the recovery is not performed by the BMC, or once the BMC completes the recovery routine, the routine determines whether the recovery is performed by the UEFI BIOS firmware 120 at step 478. If the recovery is not performed by the UEFI BIOS firmware 120, the routine powers the server off and on at step 480. If the recovery is performed by the UEFI BIOS firmware 120, the BMC 110 requests the UEFI BIOS to set a UEFI variable "Deviation_Recovery" at step 482. At step 484, the BMC 110 monitors whether an authorization from the UEFI BIOS 120 is received. If the authorization is not received, the BMC 110 continues to request the UEFI BIOS to set the UEFI deviation recovery variable at step 482. If the BMC 110 receives the authorization from the UEFI BIOS 120, the BMC 110 powers the server off and on at step 480.

[0136] When the UEFI BIOS 120 receives the request from the BMC 110 at step 482, the UEFI BIOS 120 sets a UEFI variable "Deviation_Recovery" at step 486. Then, the UEFI BIOS 120 authorizes the BMC 110 to start the deviation recovery at step 488. Then, the UEFI BIOS 120 stops and waits for the BMC to power the server off and on at step 490.

[0137] Once the shutdown and restart is initiated, the UEFI BIOS firmware 120 logs the deviation recovery to the SEL 126 stored by the flash memory 122 at step 492. Then, the UEFI BIOS firmware 120 performs a recovery mechanism at step 494. For example, the BIOS firmware can force the inter-integrated circuit (I2C) host controller to pull the data / clock signal low to force the slave device to reset itself to the original idle state. As another example, the BIOS firmware powers on and off devices that support hot plug. After the recovery mechanism is complete, the UEFI BIOS 120 clears the UEFI variable "Deviation_Recovery" at step 496. Then, the UEFI BIOS firmware 120 restarts the server at step 498.

[0138] Figures 6-7 The flowcharts in Figure 5 The machine readable instructions in the pass state routine of Figures 6-7 implemented in software stored by a tangible medium, such as flash memory, CD-ROM, floppy disk, hard disk, digital versatile disk (DVD), or other memory devices, but those skilled in the art will readily appreciate that the algorithms can alternatively be implemented in hardware, firmware, and / or software such as by an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable logic device (FPLD), a field programmable gate array (FPGA), discrete logic, etc. For example, any or all of the components of the interface can be implemented by software, hardware, and / or firmware. Also, certain or all of the machine readable instructions represented by the flowcharts can be implemented manually. Further, although the example algorithms are described with reference to the flowcharts illustrated in Figures 6-7

[0139] As used in this application, the terms "component," "module," "system" or the like are generally intended to refer to a computer-related entity, either hardware (e.g., a circuit), a combination of hardware and software, software, or an entity related to an operational machine with one or more specific functionalities. For example, a component can be, but is not limited to being, a process running on a processor (e.g., a digital signal processor), a processor, an object, an executable, a thread of execution, a program, and / or a computer. By way of illustration, both an application running on a controller and the controller itself can be a component. One or more components can reside within a process and / or thread of execution and a component can be localized, partially localized, or distributed across two or more computers or other processing devices.

[0140] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Further, to the extent that "comprising" is used in either the detailed description or the claims, such term is used in the sense of "including" rather than other- wise.

[0141] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an overly literal or overly formal sense unless expressly so defined herein.

[0142] While various embodiments of the present application have been described above, it should be understood that they have been presented by way of example only, and not limitation. Although the present application has been described with reference to one or more embodiments, persons of ordinary skill in the art will readily appreciate that various modifications and changes can be applied to the above description and equiva- lent embodiments, and in addition, individual features of device implementations can be selected and combined in a manner different from illustrated embodiments and that numerous changes and modifications within the scope of the application can be made. In addition, while a particular feature of the application can have been disclosed with respect to only one of several embodiments, such feature can be combined with one or more other features of the same or different embodiments. It is therefore intended that the scope of the application should not be limited by the described embodiments.

Claims

1. A computer system for detecting deviations, comprising: a basic input / output system (BIOS) including a power-on self-test (POST) routine having a plurality of phases executed between boot-up and hand-off to an operating system; a controller in communication with the BIOS; and a memory coupled to the controller and the BIOS, the memory storing a golden image POST code sequence, wherein the controller and the BIOS are operable to: receive a sequence POST code from the POST routine, the sequence POST code resulting from execution of the plurality of phases; compare the sequence POST code to the golden image POST code sequence; and identify a deviation in the POST routine based on at least one POST code in the sequence POST code that does not match the golden image POST code sequence.

2. The system of claim 1, wherein the deviation is one of: a flaw in a security function phase; a compromise of a security function phase; and a detection of an untrustworthy condition of the POST routine.

3. The system of claim 1, wherein the deviation is one of: a hardware device in a plurality of hardware devices not discovered by an exploration step in the POST routine; a failed hardware interface; and a hardware speed degradation after the POST routine initializes a hardware device.

4. The system of claim 1, wherein the deviation is one of: a failed software service; an unavailable software service; and an unsupported software service.

5. The system of claim 1, wherein the controller and the BIOS are further operable to: perform a deviation analysis based on the identified deviation; store a message including results of the deviation analysis; and power down and power up the computer system after the deviation analysis is complete; wherein the deviation analysis includes a recovery method to fix the deviation, and wherein the BIOS and the controller are further operable to automatically perform the recovery method after the power down and power up.

6. A method for detecting deviations, the method performed on a power-on self-test (POST) routine during boot-up of a computer system, wherein: the computer system comprises: a basic input / output system (BIOS) including a power-on self-test (POST) routine having a plurality of phases executed between boot-up and hand-off to an operating system; a controller in communication with the BIOS; and a memory coupled to the controller and the BIOS, the method comprising: storing a golden image POST code sequence in the memory; executing the POST routine of the BIOS; receiving a sequence POST code from the POST routine, the sequence POST code resulting from execution of the plurality of phases; comparing, by the controller, the sequence POST code to the golden image POST code sequence; and identifying a deviation in the POST routine based on at least one POST code in the sequence POST code that does not match the golden image POST code sequence. ​ identifying a deviation in the POST routine based on at least one POST code in the sequence POST code that does not conform to the golden image file POST code sequence.

7. The method of claim 6, wherein the deviation is one of: a flaw in a security function phase; a compromise in a security function phase; and a detection of an untrusted condition of the POST routine.

8. The method of claim 6, wherein the deviation is one of: a hardware device not discovered by an exploration step in the POST routine; a failed hardware interface; and a hardware speed degradation after the POST routine initializes a hardware device.

9. The method of claim 6, wherein the deviation is one of: a failed software service; an unavailable software service; and an unsupported software service.

10. A non-transitory computer readable medium storing software instructions that, when executed by a processor in a computer system, cause the processor to perform the method of detecting a deviation of any one of claims 6-9.

Images