System and method for detecting problem code changes in firmware

Abstract

An information handling system includes a storage device and a baseboard management controller. The storage device stores a current firmware version of the information handling system. The baseboard management controller generates a set of breakpoints for code differences from a previous firmware version to a current firmware version and executes a plurality of test cases for the current firmware version. Based on execution of the test case, the baseboard management controller stores a breakpoint hit event of the test case in the storage device. The baseboard management controller filters and ranks the breakpoint hit events, and maps the filtered and ranked breakpoint hit events to code differences. The baseboard management controller stores, in the storage device, a reduced list of code differences that are highly related to failed test cases of a current firmware version.

Description

Technical Field

[0001] The present disclosure relates generally to information handling systems, and more particularly to detecting problem codes in information handling systems. Background Art

[0002] As the value and use of information continue to grow, individuals and businesses seek additional ways to process and store information. One option is an information handling system. An information handling system typically processes, compiles, stores or transmits information or data for commercial, personal or other purposes. Technology and information handling needs and requirements may vary between different applications. Therefore, information handling systems may also differ in terms of what information is handled, how the information is handled, how much information is processed, stored or transmitted, and how quickly and efficiently the information can be processed, stored or transmitted. The changes in information handling systems allow information handling systems to be general or configured for specific users or specific purposes (such as financial transaction processing, airline ticketing, enterprise data storage or global communications). In addition, an information handling system may include a variety of hardware and software resources that can be configured to process, store and transmit information, and may include one or more computer systems, graphical interface systems, data storage systems, networking systems and mobile communication systems. Information handling systems can also implement various virtualization architectures. Data and voice communications in information handling systems can be carried out via a network, which is wired, wireless or a combination thereof. Summary of the invention

[0003] An information handling system includes a storage device and a baseboard management controller. The storage device can store a current firmware version of the information handling system. The baseboard management controller can generate a breakpoint set for code differences from a previous firmware version to a current firmware version, and execute multiple test cases for the current firmware version. Based on the execution of the test case, the baseboard management controller can store the breakpoint hit event of the test case in the storage device. The baseboard management controller can filter and rank the breakpoint hit events, and map the filtered and ranked breakpoint hit events to code differences. The baseboard management controller can store a condensed list of code differences that are highly correlated with failed test cases of the current firmware version in the storage device. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] It should be understood that for simplicity and clarity of illustration, the elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of some elements may be exaggerated relative to other elements. Embodiments incorporating the teachings of the present disclosure are shown and described with respect to the drawings herein, in which:

[0005] Figure 1 is a block diagram of a portion of an information handling system according to at least one embodiment of the present disclosure;

[0006] Figure 2 is a diagram of breakpoints for multiple threads executed during a test case in accordance with at least one embodiment of the present disclosure;

[0007] Figure 3 is a diagram of grouped failed test case breakpoint hit events according to at least one embodiment of the present disclosure;

[0008] Figure 4 is a diagram of a system and method for reducing the scope of bad code in accordance with at least one embodiment of the present disclosure.

[0009] Figure 5 is a flow chart of a method for determining a set of code differences associated with a test case failure in accordance with at least one embodiment of the present disclosure;

[0010] Figure 6 is a flow chart of a method for determining a set of code differences associated with a test case failure in accordance with at least one embodiment of the present disclosure;

[0011] Figure 7 is a flowchart of a method for ranking test case breakpoints according to at least one embodiment of the present disclosure; and

[0012] Figure 8 is a block diagram of a general information handling system according to an embodiment of the present disclosure.

[0013] The use of the same reference numbers in different drawings indicates similar or identical items. DETAILED DESCRIPTION

[0014] The following description in conjunction with the accompanying drawings is provided to help understand the teaching disclosed herein. The description focuses on the specific implementation and implementation of the teaching and is provided to help describe the teaching. This focus should not be interpreted as limiting the scope or applicability of the teaching.

[0015] Figure 1A portion of an information handling system 100 according to at least one embodiment of the present disclosure is shown. For purposes of the present disclosure, an information handling system may include any tool or collection of tools operable to calculate, compute, determine, classify, process, transmit, receive, retrieve, initiate, switch, store, display, transmit, indicate, detect, record, reproduce, process, or utilize any form of information, intelligence, or data for commercial, scientific, control, entertainment, or other purposes. For example, an information handling system may be a personal computer (such as a desktop computer or laptop computer), a tablet computer, a mobile device (such as a personal digital assistant (PDA) or a smart phone), a server (such as a blade server or a rack server), a network storage device, or any other suitable device, and may vary in size, shape, performance, functionality, and price. An information handling system may include random access memory (RAM), one or more processing resources (such as a central processing unit (CPU) or hardware or software control logic), ROM, and / or other types of non-volatile memory. Additional components of an information handling system may include one or more disk drives, one or more network ports for communicating with external devices, and various input and output (I / O) devices (such as a keyboard, mouse, touch screen, and / or video display). The information handling system may also include one or more buses operable to transmit communications between the various hardware components.

[0016] Information handling system 100 includes processor 102, baseboard management controller (BMC) 104, graphics processing unit (GPU) 106, memory 108, storage 110, and operating system (OS) 112. In one example, OS 112 may include one or more applications or firmware 120 that may be executed by processor 102, BMC 104, or GPU 106. When executing application 120, processor 102, BMC 104, or GPU 106 may access memory 108. In addition, processor 102, BMC 104, or GPU 106 may write / read data from storage 110. In one example, application or firmware version 120 may be written to storage 110. Information handling system 100 may include additional components without departing from the scope of the present disclosure.

[0017] In one example, BMC 104 may provide a multi-threaded architecture for information handling system 100. For example, BMC 104 may execute or run multiple background threads to monitor sensors and schedule tasks within information handling system 100. During the execution of a large number of background threads, certain breakpoints in instructions may occur or be hit regardless of how the test cases are executed during the device under test (DUT). In one example, these breakpoints may be hit in all test cases.

[0018] The execution of the information handling system 100 and BMC 104 to applications (such as background applications 120) can be improved by determining significant breakpoints of failed test cases during the DUT. In some examples, significant breakpoints may cause one or more test cases to fail, while other test cases may pass without impact. During the development phase of the next version of the software, a large amount of code differences may accumulate during the development phase. As used herein, code differences refer to the changes in code from a previous version to the current version. In some examples, the development phase of a software version may take days or weeks.

[0019] In previous information handling systems, when a test case failure is determined by using a newly released software, an individual always has to manually confirm that the failure is a problem and opens a vulnerability. During the debugging process in previous information handling systems, analysis, debugging, and reproduction of the problem can be performed to find the root cause and identify which code change introduced the problem. In previous information handling systems, root cause analysis tools can help find failed test cases caused by software version changes. Root cause analysis tools cannot determine which code change introduced the problem. In previous information handling systems, breakpoint mechanisms in debugging tools can probe the runtime behavior of the DUT software (threads, call stacks, break locations) without rebuilding the DUT software.

[0020] In some examples, the information handling system 100 can be improved via fully automated operations to narrow down the scope of problematic code changes in firmware black box testing. In one example, the determination of problematic code changes can be performed by the BMC 104 mapping breakpoints as probes of sampled runtime information of the DUT to code changes. During the operation of the test case, the BMC 104 can perform one or more suitable operations to evaluate the impact of the breakpoints and determine potential code changes associated with the problem. Figures 2 to 4 The operations performed by the BMC 104 are described below.

[0021] Figure 2 A plurality of threads 202, 204, and 206 are shown executing during a DUT software runtime 210 in accordance with at least one embodiment of the present disclosure. In one example, the threads 202, 204, and 206 may be associated with a test case 220 executing within a test script runtime 230. The threads 202, 204, and 206 may include a plurality of basic code blocks with breakpoints 240. The test case 220 may include steps 1, 2, 3, 4, and 5. In some examples, the BMC 104 may determine a suspicious range 250 of a code block, as will be described below.

[0022] In one example, the BMC 104 can segment the code differences into basic code blocks 240 and insert breakpoints into the code blocks. In some examples, breakpoint hits can occur based on four different factors: breakpoint location; hit time; thread identifier; and function call stack. Different code blocks 240 of threads 202, 204, and 206 can be associated with different steps 1, 2, 3, 4, and 5 of the test case 220. In some examples, each step of the test case 220 can be executed in a different time period when the test script is running 230. For example, step 1 can be executed during the time period from T0 to T1, step 2 can be executed during the time period from T1 to T2, step 3 can be executed during the time period from T2 to T3, step 4 can be executed during the time period from T3 to T4, and step 5 can be executed during the time period from T4 to T5.

[0023] In some examples, the execution of the DUT software runtime 210 and the test script runtime 230 may be asynchronous, so that the reaction in the DUT software runtime may not be consistent with the test steps in the test script runtime. For example, step 1 of the test case 220 may be associated with different code blocks / breakpoints 240, even if these code blocks are not executed before the end of step 1 (such as time T1). In one example, step 1 may be associated with breakpoint IDs 1, 2, and 5 of thread 206 and breakpoint IDs 3 and 3 in thread 204. In this example, hitting these breakpoints may be caused by the operation in test step 1. Similarly, hitting breakpoint IDs 1 and 2 of thread 206 may be caused by the operation in test step 2. Hitting breakpoint IDs 3, 4, 3, and 4 of thread 204 may be caused by the operation in test step 3. Hitting breakpoint ID 5 of thread 206 may be caused by the operation in test step 4, and hitting breakpoint ID 6 of thread 206 may be caused by the operation in test step 5. In some examples, a breakpoint ID that is not connected to one of the steps of the test case 220 is hit (eg, Figure 2 The line in the figure may be caused by factors other than the operation in the test case.

[0024] In one example, BMC 104 may detect a failure of step 3 in test case 220. Based on the failure of step 3, BMC 104 may determine a suspicious range 250 of code blocks and breakpoints 240 within threads 202, 204, and 206. Figure 2As shown, the suspicious range 250 may extend from time T0 to T3. In some examples, one or more code blocks / breakpoints 240 within the suspicious range 250 may cause the failure of step 3. The BMC 104 may filter the breakpoints 240 to include only breakpoints hit earlier than time T3, so that the breakpoints are within the suspicious range 250. In one example, the BMC 104 may utilize term frequency-inverse document frequency (TF-IDF) to evaluate the impact of each breakpoint and identify potential code changes in threads 202, 204, and 206 that introduce new issues, such as the code changes about Figure 3 Descriptive.

[0025] Figure 3 304, a total breakpoint hit event 306, a group of suspected breakpoint hit events 308, and a group of total breakpoint hit events 309 according to at least one embodiment of the present disclosure. The failure case list 302 includes a plurality of failed test cases 310, 312, and 314, and each failed test case can be associated with a DUT software runtime (such as Figure 2 The failure case list 302 may include any suitable number of failure cases without departing from the scope of the present disclosure.

[0026] In one example, the BMC 104 may determine a set of breakpoint hit events 306 associated with non-failed test cases in the failed test cases 310, 312, and 314. For example, the failed case 310 may include a breakpoint sequence 320. In some examples, the breakpoints within the breakpoint sequence 320 may be arranged in the order in which the breakpoints are hit within the thread corresponding to the failed test case 310. The BMC 104 may determine a breakpoint hit that occurred before the test failure 322. In response to determining the location of the test failure 322, the BMC 104 may determine a suspicious breakpoint hit 324 that occurred before the test failure 322. The BMC 104 may provide the breakpoint hits associated with the failed test case 310 to the grouped failed test case, and separate or filter the suspicious breakpoint hit events 308 from the total breakpoint hit events 309.

[0027] In some examples, the breakpoints within the breakpoint sequence 330 may be arranged in the order in which the breakpoints were hit within the thread corresponding to the failed test case 312. The BMC 104 may determine a breakpoint hit that occurred before the test failure 332. In response to determining the location of the test failure 332, the BMC 104 may determine a suspected breakpoint hit 334 that occurred before the test failure 332. The BMC 104 may provide the breakpoint hits associated with the failed test case 312 to the grouped failed test case 304 and separate or filter the suspected breakpoint hit events 308 from the total breakpoint hit events 309.

[0028] In one example, the breakpoints within the breakpoint sequence 341 may be arranged in the order in which the breakpoints are hit within the thread corresponding to the failed test case 314. The BMC 104 may determine a breakpoint hit that occurred before the test failure 342. In response to determining the location of the test failure 342, the BMC 104 may determine a suspected breakpoint hit 344 that occurred before the test failure 342. The BMC 104 may provide the breakpoint hits associated with the failed test case 314 to the grouped failed test case 304, and separate or filter the suspected breakpoint hit events 308 from the total breakpoint hit events 309.

[0029] Figure 4 FIG. 4 shows a series of steps 400 for reducing the scope of bad code according to at least one embodiment of the present disclosure. In some examples, the operations of the series of steps 400 can be performed by Figure 1 At step 402, the BMC 104 may generate or determine breakpoint hits for the application. In one example, the breakpoint hits may be those based on code differences between a previous version and a current version of the application. At step 404, the BMC 104 may list or compile all breakpoint hits within the application.

[0030] At step 406, the BMC 104 may determine the breakpoint hit associated with the failed test case and remove all other breakpoint hits from the breakpoint hit list. At step 408, the BMC 104 may remove the breakpoint hit associated with the background operation from the breakpoint hit list. At block 410, the BMC 104 may filter the list of breakpoint hits based on the timestamp of the failed test case. In one example, the BMC 104 may determine the end timestamp of the failed test case and remove or filter out all breakpoint hits that occur after the end timestamp of the failed test case.

[0031] At step 412, the BMC 104 may score and rank all breakpoint hits to find the most significant breakpoint locations that cause the test case to fail. In one example, the BMC 104 may rank the breakpoint hits based on the scores generated via TF-IDF to determine the impact or relevance of a particular breakpoint on the application code. For example, the TF-IDF score for breakpoint hit three may be 1.145, the TF-IDF for breakpoint hit one may be 0.84, the TF-IDF for breakpoint hit two may be 0.31, the TF-IDF for breakpoint hit four may be 0.03, and the TF-IDF for breakpoint hit seven may be 0.02. In this example, breakpoint three may have the greatest impact, followed by breakpoint one, and then breakpoints two, four, and seven. At step 414, the BMC 104 may filter the ranked breakpoint hits to determine the most significant breakpoint hit that causes the test case to fail. For example, you can filter breakpoints by a top TF-IDF score, a threshold TF-IDF score, etc.

[0032] At step 416, the BMC 104 may map the highest ranked breakpoint hits to code differences in the application. For example, breakpoint three may be mapped to code difference X, breakpoint one may be mapped to code difference Y, and breakpoint two may be mapped to code difference Z. At step 418, the BMC 104 may utilize the code differences to determine code differences that may have introduced anomalies in the application. In some examples, the operations performed by the BMC 104 to minimize the number of breakpoints in the suspicious range of the test case do not require rebuilding for the version in which the partial code was submitted, and do not require rerunning the regression test case for each rebuilt version.

[0033] Figure 5 is a flow chart of a method 500 for determining a set of code differences associated with a test case failure in accordance with at least one embodiment of the present disclosure, the method beginning at block 502. In one example, the method 500 may include, but is not limited to, Figure 1 It should be readily understood that not every method step set forth in the flowchart is always necessary, but rather that certain steps of the method may be combined, performed simultaneously, in a different order, or may be omitted without departing from the scope of the present disclosure.

[0034] At block 504, a set of breakpoints is generated based on all code differences in the application. In one example, code differences may exist between two DUT firmware versions. At block 506, the generated breakpoints are deployed in the test cases of the current firmware version. At block 508, an automatic regression of the test cases is performed. In one example, the automated regression of the test cases can be performed as a probe of the sampled DUT firmware runtime context without interrupting the operation of the information handling system. In some examples, the test cases can be executed without modifying the source code to add debug information and build a debug image.

[0035] At block 510, breakpoint hit events are recorded. In one example, breakpoint hit events can be for all test cases, including both failed test cases and passed test cases. At block 512, all failed test cases and associated breakpoint hit events are grouped.

[0036] At block 514, the test case breakpoint hits are filtered and ranked. In one example, the breakpoint hits are filtered to include only the breakpoints of the failed test cases. These filtered breakpoint hits are ranked based on the importance of the breakpoint hits with respect to the failed test cases. At block 516, the important or significant breakpoint hits are mapped to code differences. At block 518, a reduced amount of code differences that are highly correlated with test case failures are generated or provided, and the process ends at block 520.

[0037] Figure 6 is a flow chart of a method 600 for determining a set of code differences associated with a test case failure in accordance with at least one embodiment of the present disclosure, the method beginning at block 602. In one example, the method 600 may include, but is not limited to, Figure 1 In some examples, method 600 may be substantially similar to Figure 5 The operations performed in block 514 of . It should be readily understood that not every method step set forth in the flowchart is always necessary, but rather that certain steps of the method may be combined, performed simultaneously, in a different order, or may be omitted without departing from the scope of the present disclosure.

[0038] At block 604, breakpoint hit events associated with background operations are removed from the grouped failed test case breakpoints. At block 606, timestamps of the failed test cases are extracted. In one example, the timestamp may be an end timestamp of each of the failed test cases. At block 608, breakpoint hit events whose hit times are before the end timestamp of the failed test cases are determined.

[0039] At block 610, test case suspicious breakpoints are extracted from all breakpoint hit events. At block 612, a different breakpoint target factor number is determined for each extracted breakpoint. At block 614, the suspicious breakpoints are ranked, and the process ends at block 616. In one example, the breakpoint hits can be ranked based on the scores generated via TF-IDF to determine the impact or relevance of a particular breakpoint on the application code.

[0040] Figure 7 is a flow chart of a method 700 for ranking test case breakpoints according to at least one embodiment of the present disclosure, the method starting at 702. In one example, the method 700 may include, but is not limited to, Figure 1 It should be readily understood that not every method step set forth in the flowchart is always necessary, but rather that certain steps of the method may be combined, performed simultaneously, in a different order, or may be omitted without departing from the scope of the present disclosure.

[0041] At block 704, code differences are received. In one example, code differences between a previous version of an application or firmware of the DUT and a current version of the application or firmware may be determined. After receiving the code differences, the flow may continue at blocks 706 and 712. At block 706, a file name associated with the application or firmware is extracted from the code. At block 708, the source file is parsed. At block 710, a breakpoint target factor number is determined.

[0042] At block 712, the line of code (LoC) of the breakpoint is extracted. At block 714, it is determined whether the LoC is within a specific range. In one example, the range may be before the end timestamp of the suspected test case block. If the LoC is not within the specific range, the process ends at block 716. If the LoC is within the specific range, the process continues at block 718 and the breakpoint location is optimized based on different factors. For example, the optimization may be performed based on any combination of objects, functions, macros, global variables, etc.

[0043] At block 720, it is determined whether the LoC is a macro variable or a global variable. If the LoC is a macro or a global variable, then at block 722, the macro and global variable name strings are searched, and flow continues as described above with respect to blocks 706 and 712. If the LoC is not a macro or a global variable, then at block 724, the basic code block (BCB) of the LoC is analyzed.

[0044] At block 726, it is determined whether the LoC is within range. If the LoC is not within range, the flow ends at block 728. If the LoC is within range, the start of the BCB is marked at block 730. At block 732, it is determined whether another BCB is within the LoC. If another BCB is within the LoC, the flow continues as described above with respect to block 724. If another BCB is not within the LoC, a target breakpoint list is created at block 734 and the flow ends at block 728.

[0045] Figure 8 A generalized embodiment of an information handling system 800 according to an embodiment of the present disclosure is shown. For the purposes of the present disclosure, an information handling system may include any tool or tool set operable to calculate, classify, process, transmit, receive, retrieve, generate, switch, store, display, indicate, detect, record, reproduce, handle or utilize any form of information, intelligence or data for commercial, scientific, control, entertainment or other purposes. For example, the information handling system 800 may be a personal computer, a laptop computer, a smart phone, a tablet device or other consumer electronic device, a network server, a network storage device, a switch, a router or another network communication device, or any other suitable device, and may differ in size, shape, performance, function and price. In addition, the information handling system 800 may include processing resources for executing machine executable code, such as a central processing unit (CPU), a programmable logic array (PLA), an embedded device such as a system on a chip (SoC), or other control logic hardware. The information handling system 800 may also include one or more computer-readable media for storing machine executable code such as software or data. Additional components of information handling system 800 may include one or more storage devices that can store machine-executable code, one or more communication ports for communicating with external devices, and various input and output (I / O) devices such as a keyboard, a mouse, and a video display. Information handling system 800 may also include one or more buses operable to transfer information between the various hardware components.

[0046] Information handling system 800 may include devices or modules that embody one or more of the following devices or modules, and operate to perform one or more of the following methods. Information handling system 800 includes processors 802 and 804, input / output (I / O) interface 810, memory 820 and 825, graphics interface 830, basic input and output system / universal extensible firmware interface (BIOS / UEFI) module 840, disk controller 850, hard disk drive (HDD) 854, optical disk drive (ODD) 856, disk emulator 860 connected to external solid state drive (SSD) 862, I / O bridge 870, one or more extended resources 874, trusted platform module (TPM) 876, network interface 880, management device 890, and power supply 895. Processors 802 and 804, I / O interface 810, memory 820, graphics interface 830, BIOS / UEFI module 840, disk controller 850, HDD 854, ODD 856, disk emulator 860, SSD 862, I / O bridge 870, expansion component resources 874, TPM 876, and network interface 880 operate together to provide a host environment for information handling system 800, which operates to provide the data processing functionality of the information handling system. The host environment operates to execute machine executable code, including platform BIOS / UEFI code, device firmware, operating system code, application programs, programs, etc., to perform data processing tasks associated with information handling system 800.

[0047] In a host environment, processor 802 is connected to I / O interface 810 via processor interface 806, and processor 804 is connected to I / O interface via processor interface 808. Memory 820 is connected to processor 802 via memory interface 822. Memory 825 is connected to processor 804 via memory interface 827. Graphics interface 830 is connected to I / O interface 810 via graphics interface 832, and provides video display output 836 to video display 834. In a particular embodiment, information handling system 800 includes separate memories dedicated to each of processors 802 and 804 via separate memory interfaces. Examples of memories 820 and 830 include random access memory (RAM), such as static RAM (SRAM), dynamic RAM (DRAM), non-volatile RAM (NV-RAM), etc., read-only memory (ROM), another type of memory, or a combination thereof.

[0048] The BIOS / UEFI module 840, the disk controller 850, and the I / O bridge 870 are connected to the I / O interface 810 via an I / O channel 812. Examples of the I / O channel 812 include a peripheral component interconnect (PCI) interface, a PCI-Extension (PCI-X) interface, a high-speed PCI-Express (PCIe) interface, another industry standard or proprietary communication interface, or a combination thereof. The I / O interface 810 may also include one or more other I / O interfaces, including an Industry Standard Architecture (ISA) interface, a Small Computer Serial Interface (SCSI) interface, an Inter-Integrated Circuit (IIC) interface, and a SCSI interface. 2 C) interface, system packet interface (SPI), universal serial bus (USB), another interface, or a combination thereof. BIOS / UEFI module 840 includes BIOS / UEFI code that is operable to detect resources within information handling system 800, provide drivers for resources, initialize resources, and access resources. BIOS / UEFI module 840 includes code that is operable to detect resources within information handling system 800, provide drivers for resources, initialize resources, and access resources.

[0049] The disk controller 850 includes a disk interface 852, which connects the disk controller to the HDD 854, the ODD 856, and the disk emulator 860. Examples of the disk interface 852 include an integrated drive electronics (IDE) interface, an advanced technology attachment (ATA) such as a parallel ATA (PATA) interface or a serial ATA (SATA) interface, a SCSI interface, a USB interface, a proprietary interface, or a combination thereof. The disk emulator 860 allows the SSD 864 to be connected to the information handling system 800 via an external interface 862. Examples of the external interface 862 include a USB interface, an IEEE 4394 (Firewire) interface, a proprietary interface, or a combination thereof. Alternatively, the solid state drive 864 can be disposed within the information handling system 800.

[0050] The I / O bridge 870 includes a peripheral interface 872 that connects the I / O bridge to an extended resource 874, a TPM 876, and a network interface 880. The peripheral interface 872 may be an interface of the same type as the I / O channel 812, or may be an interface of a different type. Thus, when the peripheral interface 872 and the I / O channel 812 are of the same type, the I / O bridge 870 extends the capabilities of the I / O channel, and when they are of different types, the I / O bridge converts information from a format suitable for the I / O channel to a format suitable for the peripheral bridge 872. The extended resource 874 may include a data storage system, an additional graphics interface, a network interface card (NIC), a sound / video processing card, another extended resource, or a combination thereof. The extended resource 874 may be located on a main circuit board, on a separate circuit board or expansion card disposed within the information handling system 800, on a device external to the information handling system, or a combination thereof.

[0051] The network interface 880 represents a NIC, which is arranged in the information handling system 800, located on the main circuit board of the information handling system, integrated into another component such as the I / O interface 810, located in another suitable location or a combination thereof. The network interface device 880 includes network channels 882 and 884 that provide interfaces to devices outside the information handling system 800. In a specific embodiment, the network channels 882 and 884 are of different types from the peripheral channels 872, and the network interface 880 converts information from a format suitable for the peripheral channels to a format suitable for external devices. Examples of network channels 882 and 884 include InfiniBand channels, Fibre Channel channels, Gigabit Ethernet channels, proprietary channel architectures or a combination thereof. Network channels 882 and 884 can be connected to external network resources (not shown). Network resources can include another information handling system, a data storage system, another network, a grid management system, another suitable resource or a combination thereof.

[0052] Management device 890 represents one or more processing devices that operate together to provide a management environment for information handling system 800, such as a dedicated baseboard management controller (BMC) system-on-chip (SoC) device, one or more associated memory devices, one or more network interface devices, complex programmable logic devices (CPLD), etc. Specifically, management device 890 is connected to various components of the host environment via various internal communication interfaces (such as low pin count (LPC) interface, inter-integrated circuit (I2C) interface, PCIe interface, etc.) to provide an out-of-band (OOB) mechanism to retrieve information related to the operation of the host environment, provide BIOS / UEFI or system firmware updates, and manage non-processing components of information handling system 800, such as system cooling fans and power supplies. Management device 890 may include a network connection to an external management system, and the management device may communicate with the management system to report status information of information handling system 800, receive BIOS / UEFI or system firmware updates, or perform other tasks for managing and controlling the operation of information handling system 800.

[0053] The management device 890 can operate off a separate power plane from the components of the host environment, such that the management device receives power to manage the information handling system 800 when the information handling system is otherwise shut down. Examples of the management device 890 include commercially available BMC products or other devices that operate in accordance with the Intelligent Platform Management Initiative (IPMI) specification, the Web Services Management (WSMan) interface, the Redfish application programming interface (API), another Distributed Management Task Force (DMTF), or other management standards, and may include an integrated Dell Remote Access Controller (iDRAC), an embedded controller (EC), etc. The management device 890 may also include associated memory devices, logic devices, security devices, etc., as needed or desired.

[0054] Although only a few exemplary embodiments are described in detail herein, those skilled in the art will readily appreciate that many modifications may be made to the exemplary embodiments without materially departing from the novel teachings and advantages of the embodiments of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the embodiments of the present disclosure as defined by the appended claims. In the claims, means-plus-function clauses are intended to cover structures described herein as performing the recited function, and not only structural equivalents, but also equivalent structures.

Claims

1. An information processing system, comprising: a storage device, the storage device being used to store a current firmware version of the information handling system; as well as A baseboard management controller communicating with the storage device, the baseboard management controller being used for: generating a set of breakpoints for code differences from a previous firmware version to the current firmware version; executing a plurality of test cases for the current firmware version; Based on the execution of the test case, storing the breakpoint hit event of the test case in the storage device; Filtering and ranking the breakpoint hit events; mapping the filtered and ranked breakpoint hit events to the code differences; as well as A condensed list of code differences that are highly relevant to the failing test cases of the current firmware version is stored in the storage device. 2 . The information handling system of claim 1 , wherein filtering and ranking the breakpoint hit events comprises the baseboard management controller further: removing breakpoint hit events associated with background operations.

3. The information handling system of claim 1 , wherein filtering and ranking the breakpoint hit events comprises the baseboard management controller further: Extract the end timestamp of the failed test case; determining a breakpoint hit event occurring before the end timestamp; and Filter out breakpoint hit events that did not occur before the end timestamp. 4 . The information handling system of claim 3 , wherein filtering and ranking the breakpoint hit events comprises the baseboard management controller further: ranking the breakpoint hit events occurring before the end timestamp. 5 . The information handling system of claim 4 , wherein the baseboard management controller ranks the breakpoint hitting events occurring before the end timestamp using term frequency-inverse document frequency. 6 . The information handling system of claim 4 , wherein filtering and ranking the breakpoint hit events comprises the baseboard management controller further: filtering out breakpoint hit events with ranking scores lower than a threshold score. 7 . The information handling system of claim 1 , wherein the set of breakpoints for code differences are located in a plurality of threads of the current firmware version. 8 . The information handling system of claim 1 , wherein the baseboard management controller further: associates different breakpoint hit events with different steps of the test case.

9. A method comprising: storing a current firmware version of an information handling system in a storage device of the information handling system; generating, by a baseboard management controller of the information handling system, a set of breakpoints for code differences from a previous firmware version to the current firmware version; executing a plurality of test cases for the current firmware version; Based on the execution of the test case, storing the breakpoint hit event of the test case in the storage device; Filtering and ranking the breakpoint hit events; mapping, by the baseboard management controller, the filtered and ranked breakpoint hit events to the code differences; as well as A condensed list of code differences that are highly relevant to the failing test cases of the current firmware version is stored in the storage device.

10. The method according to claim 9, wherein filtering and ranking the breakpoint hit events comprises: Remove the breakpoint hit event associated with the background operation.

11. The method according to claim 9, wherein filtering and ranking the breakpoint hit events comprises: Extract the end timestamp of the failed test case; Determine a breakpoint hit event that occurs before the end timestamp; as well as Filter out breakpoint hit events that did not occur before the end timestamp.

12. The method according to claim 11, wherein filtering and ranking the breakpoint hit events comprises: The breakpoint hit events occurring before the end timestamp are ranked. 13 . The method according to claim 12 , further comprising ranking the breakpoint hit events occurring before the end timestamp using term frequency-inverse document frequency.

14. The method according to claim 12, wherein filtering and ranking the breakpoint hit events comprises: Filter out breakpoint hit events whose ranking scores are lower than a threshold score.

15. The method of claim 9, wherein the set of breakpoints for code differences are located in multiple threads of the current firmware version.

16. The method according to claim 9, wherein the method further comprises: Different breakpoint hit events are associated with different steps of the test case.

17. An information processing system comprising: a storage device, the storage device being used to store a current firmware version of the information handling system; as well as A baseboard management controller, the baseboard management controller is used for: generating a set of breakpoints for code differences from a previous firmware version to the current firmware version; executing a plurality of test cases for the current firmware version; Based on the execution of the test case, storing the breakpoint hit event of the test case in the storage device; Remove the breakpoint hit event associated with the background operation; Extract the end timestamp of the failed test case; Determine a breakpoint hit event that occurs before the end timestamp; Filter out breakpoint hit events that have not occurred before the end timestamp; Mapping the filtered breakpoint hit events to the code differences; as well as A condensed list of code differences that are highly relevant to the failing test cases of the current firmware version is stored in the storage device.

18. The information handling system of claim 17, wherein the baseboard management controller ranks the breakpoint hit events occurring before the end timestamp. 19 . The information handling system of claim 18 , wherein the baseboard management controller ranks the breakpoint hitting events occurring before the end timestamp using term frequency-inverse document frequency.

20. The information handling system of claim 18, wherein the baseboard management controller further: filters out breakpoint hit events with a ranking score lower than a threshold score.

Images