An automated testing method and computer device for server BMC boot performance

Abstract

The application provides a kind of server BMC starting performance automated testing method and computer device, wherein, method includes establishing FTP server and creating shared directory, at least one version of BMC to be measured is stored by shared directory;The power-on and power-off of server are controlled by programmable power supply distribution unit;Establish communication connection with server BMC serial port, for each BMC to be measured, the following sub-steps are cyclically executed: the power-off and power-on operation of server is executed by programmable power supply distribution unit, and BMC restart is triggered;Real-time capture and record the serial port information output in this BMC starting process by server BMC serial port;Analysis serial port information, identify multiple preset key events and their corresponding starting stage;Calculate the time point of each starting stage and the time consumption of each stage, and construct BMC starting timeline;Based on BMC starting timeline, calculate BMC starting performance index, and compare performance index with preset threshold, to judge whether BMC performance meets the standard.

Description

Technical Field

[0001] This invention relates to the field of server performance testing technology, and in particular to an automated testing method and computer device for server BMC startup performance. Background Technology

[0002] The Baseboard Management Controller (BMC), as the core management unit of the server, is responsible for key functions such as remote monitoring, fault diagnosis, and firmware updates. The startup performance of the BMC directly affects the availability and operational efficiency of the server, especially in large-scale deployment scenarios in data centers, where the speed of BMC startup determines the time window from power-on to a manageable state.

[0003] Currently, testing server BMC startup performance mainly relies on manual operation or semi-automated scripts. Traditional testing methods typically involve testers manually uploading multiple BMC firmware versions to be tested, manually controlling the server's power-on and power-off via the power management module, manually capturing BMC startup logs using serial port debugging tools, and then manually reading the logs or writing simple scripts to statistically analyze key time points during the startup process. This testing approach has the following shortcomings: First, manual operation is time-consuming and labor-intensive, making it difficult to conduct large-scale cyclic stress tests on multiple BMC versions, resulting in low testing efficiency; second, different testers have subjective differences in the criteria for dividing the startup phase and identifying key events, leading to poor consistency in test results; third, existing methods cannot accurately quantify the time consumed in each stage of the BMC startup process, making it impossible to accurately locate startup performance bottlenecks; fourth, there is a lack of statistical analysis capabilities for multiple test data, making it impossible to effectively evaluate the stability and consistency of BMC startup performance. Summary of the Invention

[0004] This invention provides an automated testing method and computer device for server BMC startup performance, aiming to solve at least one of the technical problems existing in the prior art.

[0005] The technical solution of this invention is an automated testing method for server BMC startup performance, which includes: Establish an FTP server and create a shared directory to store at least one version of BMC to be tested. The power-on and power-off of the server are controlled by a programmable power distribution unit; Establish a communication connection with the server BMC serial port. For each version of the BMC under test, execute the following sub-steps in a loop: The programmable power distribution unit performs a power-off and power-on operation on the server, triggering a BMC restart. The serial port information output by the server BMC during this startup process is captured and recorded in real time through the server BMC serial port. Analyze the serial port information to identify multiple preset key events and their corresponding startup phases; Calculate the time points of each startup phase and the duration of each phase to construct the BMC startup timeline; Based on the BMC startup timeline, BMC startup performance indicators are calculated, and these performance indicators are compared with preset thresholds to determine whether BMC performance meets the standards.

[0006] According to some embodiments of the present invention, it further includes: Set a preset number of cycles, and perform a preset number of power-down and power-on operations on the server through the programmable power distribution unit to trigger a preset number of BMC restarts; Record the serial port information output during each BMC restart process; Statistical analysis is performed on the serial port information output during the BMC restart process for a preset number of loops. The total BMC startup time, the time consumed in each startup stage of the BMC, the average startup time of the BMC tested for a preset number of loops, the standard deviation of the BMC startup time, and the maximum percentage error between the single startup time of the BMC and the average startup time of the BMC are calculated. A performance comparison test report containing the best version, version ranking, and test results is generated.

[0007] According to some embodiments of the present invention, it further includes: During the automated test of BMC startup performance for each version of BMC to be tested, a preset cooling-off time is set between two adjacent automated tests of BMC startup performance. When the programmable power distribution unit is detected to be powered off, the automated test of BMC startup performance is re-executed after a preset time interval.

[0008] According to some embodiments of the present invention, the step of analyzing the serial port information and identifying multiple preset key events and their corresponding startup phases includes: The BMC startup process is divided into multiple consecutive startup stages, and key fields in the serial port information output by the server BMC serial port are matched using regular expressions. Based on the key fields, the corresponding key events are determined, and the identified key events are categorized into the corresponding startup phases; The multiple consecutive startup phases include: U-Boot boot phase, kernel loading phase, kernel startup phase, hardware detection phase, file system phase, and system service phase.

[0009] According to some embodiments of the present invention, determining the corresponding key events based on the key fields and classifying the identified key events into the corresponding startup phases includes: Identify the key fields of the serial port information output during the BMC startup process; When the key fields are identified to include bootloader initialization, memory detection, and environment variable loading, the key events corresponding to the key fields are classified into the U-Boot stage; When the key fields are identified to include kernel image verification, device tree configuration, and RAMDISK preparation, the key events corresponding to the key fields are classified into the kernel loading stage. When the key fields are identified to include Linux kernel initialization, CPU identification, and console activation, the key events corresponding to the key fields are classified into the kernel startup phase. When the key fields are identified to include hardware initialization of system controller, SPI, I2C, and network interface, the key events corresponding to the key fields are classified into the hardware probing phase. When the key fields are identified to include root file system mounting and init process startup, the key events corresponding to the key fields are classified into the file system stage. When the key fields are identified as including systemd service startup, network configuration, and BMC service readiness, the key events corresponding to the key fields are classified into the system service phase.

[0010] According to some embodiments of the present invention, the step of calculating the time points of each startup phase and the duration of each phase to construct the BMC startup timeline includes: When a startup phase corresponding to a key field is matched, the current time is immediately recorded as the time point when the corresponding startup phase occurred. Calculate the time difference between the occurrence of two adjacent startup phases to obtain the time consumption of each startup phase; The BMC startup timeline is constructed based on the time points of each startup phase and the duration of each startup phase.

[0011] According to some embodiments of the present invention, the step of calculating BMC startup performance indicators based on the BMC startup timeline and comparing the performance indicators with a preset threshold to determine whether the BMC performance meets the standard includes: Based on the time points of each startup phase in the BMC startup timeline and the duration of each startup phase; Calculate the time consumed in the automated test of BMC startup performance for the same startup phase in a preset number of cycles. If the percentage deviation between the time consumed in the startup phase and the average time consumed in a certain automated test of BMC startup performance exceeds the preset threshold, it is determined that the startup phase is abnormal or the BMC startup performance index is abnormal and fails to meet the standard.

[0012] According to some embodiments of the present invention, controlling the power-on and power-off of the server via a programmable power distribution unit includes: The controller of the programmable power distribution unit communicates with the programmable power distribution unit via an HTTP POST request; The controller of the programmable power distribution unit uses JSON format to transmit power_off(outlet) control commands, power_on(outlet) control commands, and power_cycle(outlet, delay_seconds) control commands to realize a complete power restart process, thereby controlling the power-on and power-off of the server.

[0013] According to some embodiments of the present invention, establishing an FTP server and creating a shared directory, through which at least one version of BMC to be tested is stored, includes: Define the core function of setup_ftp_server, and deploy an FTP server dedicated to BMC storage and distribution through the core function; Create a shared directory using os.makedirs to store at least one version of the BMC to be tested; Create an FTP authenticator using DummyAuthorizer and add a user account with full permissions, including the ability to list, delete, upload, and download. Configure the FTP authenticator and start the FTPServer to continuously listen for connection requests on the specified ports of all network interfaces; When a connection request is detected, the corresponding version of BMC to be tested is automatically retrieved.

[0014] The present invention also relates to a computer device, including a memory and a processor, wherein the processor performs the above-described method when executing a computer program stored in the memory.

[0015] The present invention also relates to a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0016] The automated testing method and computer device for server BMC startup performance provided in this invention have at least one of the following advantages or beneficial effects: By establishing an FTP server and creating a shared directory, centralized storage and standardized distribution of multiple BMC versions under test are achieved; by controlling the power-on and power-off of the server through a programmable power distribution unit, precise programmatic control of the server power status is achieved, ensuring the consistency of electrical initial conditions and timing repeatability for each test. Establishing a communication connection with the server BMC serial port constructs a real-time data channel for the startup process, making transparent collection of BMC underlying logs possible. For each BMC version under test, a power-off and power-on operation is repeatedly executed to trigger a BMC restart, achieving stress verification of BMC startup stability, transforming intermittent failures into statistically probabilistic data, and significantly improving the credibility of reliability assessment. By capturing and analyzing serial port information in real time, key events and their startup stages are identified, transforming unstructured log streams into structured time-series data, and establishing a hierarchical parsing framework for the startup process. By calculating the time points and duration of each stage and constructing a startup timeline, a transformation from black-box startup to visualized and quantifiable startup was achieved, supporting precise location of performance anomalies and digital twins of startup phase behavior. Performance metrics were calculated based on the startup timeline and compared with preset thresholds, enabling automated judgment of BMC startup quality and transforming human experience into objective standards. The synergistic effect of various technologies constructed a complete automated testing method for server BMC startup performance, forming a complete closed loop of firmware deployment, stress testing, data collection, and analysis and evaluation. This significantly improved BMC testing efficiency and product quality assurance capabilities, while achieving unattended operation throughout the entire process from firmware management and power control to performance evaluation.

[0017] Furthermore, additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0018] Figure 1 This is a flowchart of the automated testing method for server BMC startup performance provided in this embodiment of the invention. Figure 2 This is a detailed flowchart of the first type of automated testing method for server BMC startup performance provided in this embodiment of the invention; Figure 3 This is a detailed flowchart of step S610 in the automated testing method for server BMC startup performance provided in this embodiment of the invention; Figure 4 This is a detailed flowchart of step S700 in the automated testing method for server BMC startup performance provided in this embodiment of the invention; Figure 5This is a detailed flowchart of step S100 in the automated testing method for server BMC startup performance provided in this embodiment of the invention. Detailed Implementation

[0019] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with the embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention.

[0020] It should be noted that, unless otherwise specified, when a feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. The singular forms "a," "described," and "the" used herein are also intended to include the plural forms, unless the context clearly indicates otherwise. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and not for limiting the invention. The term "and / or" as used herein includes any combination of one or more of the associated listed items.

[0021] It should be understood that although the terms first, second, third, etc., may be used to describe various elements in this disclosure, these elements should not be limited to these terms. These terms are used only to distinguish elements of the same type from one another. For example, a first element may also be referred to as a second element without departing from the scope of this disclosure, and similarly, a second element may also be referred to as a first element. Any and all instances or exemplary language (“e.g.,” “such as,” etc.) provided herein are intended only to better illustrate embodiments of the invention and, unless otherwise required, do not impose a limitation on the scope of the invention.

[0022] This invention provides an automated testing method and computer device for server BMC startup performance, which can realize automated testing of BMC startup performance, accurately quantify the time consumed in each stage, and support multi-version comparative analysis.

[0023] Please refer to the following. Figures 1 to 5 The automated testing method and computer device for server BMC startup performance provided in this embodiment of the invention will be further described.

[0024] Reference Figure 1 As shown, Figure 1 This is a general flowchart of the automated testing method for server BMC startup performance provided in this embodiment of the invention. The automated testing method for server BMC startup performance includes, but is not limited to, steps S100 to S800. Specifically, S100: Establish an FTP server and create a shared directory to store at least one version of BMC to be tested. S200: Controls the power-on and power-off of the server via a programmable power distribution unit; S300: Establish a communication connection with the server BMC serial port. For each version of the BMC under test, execute the following sub-steps in a loop: S400: The programmable power distribution unit performs a power-down and power-on operation on the server, triggering a BMC restart. S500: Captures and records the serial port information output by the BMC during this startup process in real time via the server BMC serial port; S600: Analyzes serial port information to identify multiple preset key events and their corresponding startup phases; S700: Calculate the time points of each startup phase and the duration of each phase to construct the BMC startup timeline; S800: Based on the BMC startup timeline, calculate the BMC startup performance indicators and compare the performance indicators with preset thresholds to determine whether the BMC performance meets the standards.

[0025] In this embodiment of the invention, an FTP server is established and a shared directory is created, allowing multiple versions of BMC to be stored uniformly in the shared directory, forming a structured firmware version library. This centralized storage method enables the testing system to automatically obtain specific firmware versions on demand, without manual intervention in the file transfer process, thus supporting the automated operation of the entire testing process. Simultaneously, the standardized and widely supported nature of the FTP protocol ensures the compatibility and reliability of firmware distribution, laying the foundation for subsequent multi-version comparative testing.

[0026] By controlling the server's power-on and power-off via a programmable power distribution unit (PDU), precise programmable control of the server's power status is achieved. The PDU receives control commands through a network interface and can remotely execute power-on and power-off operations, enabling unattended execution of the testing process and eliminating the delays and uncertainties associated with manual power-on / off operations. The timing precision of the PDU's control ensures the consistency of the time interval between each power-off and power-on, guaranteeing the repeatability of test conditions. The PDU also works in conjunction with subsequent serial port monitoring to capture the entire BMC startup process under precisely controlled power conditions, preventing data loss during critical startup phases due to improper timing of manual operations.

[0027] A communication connection was established with the server's BMC serial port, constructing a real-time data channel for the BMC boot process. As the standard interface for BMC's underlying output, the serial port continuously outputs runtime logs from various levels, including the bootloader, kernel, and drivers, during the boot process. After establishing a stable serial communication connection, the test system can receive this log data in real-time streaming, enabling transparent monitoring of BMC boot behavior. This real-time capture capability ensures the integrity and timing accuracy of key event information, providing the raw data foundation for subsequent stage division and time consumption calculations.

[0028] Then, for each version of BMC to be tested, the following sub-steps are executed repeatedly: First, by performing a power-down and power-on operation on the server through a programmable power distribution unit, a BMC restart was triggered, achieving stress verification of BMC startup stability. By repeatedly executing the power-on and power-off operation, multiple restart scenarios that the server might encounter in a real-world operating environment can be simulated, thereby exposing sporadic startup timing issues. The cyclical test can statistically analyze the startup success rate and identify startup failures or performance degradation phenomena that only occur under specific conditions, significantly improving the credibility of the BMC quality assessment.

[0029] Real-time capture and recording of serial port information during the BMC startup process enables full data retention throughout the startup process. Complete recording of serial port output makes each startup a traceable and reproducible data asset. When an anomaly occurs during a test, testers can review the complete startup log to pinpoint the specific startup stage where a timeout or error occurred, thus optimizing the accuracy of critical event identification rules.

[0030] By analyzing serial port information, multiple pre-defined key events and their corresponding startup phases were identified, enabling semantic parsing of unstructured log data. The serial port information output during BMC startup is a continuous text stream lacking inherent structural boundaries. By using pre-defined key events and their corresponding regular expression matching rules, event points with phase-specific significance, such as U-Boot startup, kernel loading start, and file system mounting completion, can be automatically identified from the text stream. The effectiveness of this semantic parsing technique lies in transforming raw, difficult-to-analyze time-series data into a structured event sequence with clear business meaning. The accuracy of key event identification directly determines the accuracy of startup phase division and is the core technical aspect of the entire testing method.

[0031] By calculating the time points of each startup phase and the duration of each phase, a BMC startup timeline is constructed, enabling a visualized and quantitative representation of the BMC startup process. By recording the occurrence time of each key event, the originally black-box startup process is transformed into a complete timeline containing multiple time nodes and phase durations, supporting precise tracking.

[0032] Based on the BMC startup timeline, BMC startup performance metrics are calculated and compared with preset thresholds to determine whether BMC performance meets standards, thus achieving automated judgment of BMC startup quality. By setting preset thresholds (e.g., 5%), it can automatically determine whether the time taken for each stage of each startup is within a reasonable fluctuation range, eliminating the need for manual review of the timeline data for each test. The advantage of this automated judgment is that it significantly improves the efficiency of large-scale testing. When the time taken for a startup stage exceeds the preset threshold, the startup stage is automatically marked as abnormal, triggering an alarm and forming a closed-loop process for test issues.

[0033] The automated testing method for server BMC boot performance provided in this invention uses an FTP server to centrally manage multiple versions of BMC firmware, utilizes a programmable power distribution unit to achieve precise cyclic power-on / off control, and combines serial port real-time monitoring to capture boot logs. This method divides the BMC boot process into multiple key boot stages, automatically identifies events in each stage, calculates the time consumed in each stage, and constructs a complete boot timeline. Performance compliance is determined by setting preset thresholds for error judgment, achieving unattended operation throughout the entire process from firmware deployment and stress testing to performance evaluation, effectively improving BMC testing efficiency and the accuracy of reliability assessment.

[0034] Reference Figure 2 As shown, Figure 2 This is a detailed flowchart of the first embodiment of the automated testing method for server BMC startup performance provided by the present invention. The automated testing method for server BMC startup performance also includes, but is not limited to, steps S900 to S920. Specifically, S900: Set a preset number of cycles, and perform a preset number of power-down and power-on operations on the server through the programmable power distribution unit to trigger a preset number of BMC restarts; S910: Records the serial port information output during each BMC restart process; S920: Performs statistical analysis on the serial port information output during the BMC restart process for a preset number of loops, calculates the total BMC startup time, the time consumed in each startup stage of the BMC, the average startup time of the BMC tested for a preset number of loops, the standard deviation of the BMC startup time, and the maximum percentage error between the single startup time of the BMC and the average startup time of the BMC, and generates a performance comparison test report including the best version, version ranking, and test results.

[0035] In this embodiment of the invention, a preset number of cycles (e.g., 10 cycles) is set and a cycle of power-down and power-on operations is performed, thereby achieving quantitative stress verification of BMC startup stability. By setting a fixed number of cycles, multiple repeated startup tests can be performed on the same version of the BMC, transforming sporadic startup failures into statistically probabilistic data.

[0036] Serial port information is recorded for each BMC restart process, forming a fine-grained raw dataset and establishing a complete test data archive. This supports in-depth traceability analysis for single abnormal startups, and this sequential recording method preserves the complete timing information of the BMC startup process. The technical effect of statistical analysis of multiple serial port information achieves the transformation from raw data to performance indicators. By aggregating and calculating test data for a preset number of loops, multi-dimensional indicators such as total startup time, stage duration, average startup time, standard deviation, and maximum error percentage are generated. This statistical analysis integrates scattered single test results into a representative performance profile, comprehensively depicting the central tendency and dispersion of BMC startup behavior, enabling precise quantification and objective comparison of performance differences between versions. The maximum error percentage is compared with a preset threshold to determine whether the startup performance of that version of the BMC meets the standard; the automated test loop and statistical analysis are executed for all BMC firmware versions under test.

[0037] Based on average startup time, the performance of each tested BMC version is ranked, generating a performance comparison test report that includes a best version recommendation, a version ranking list, and detailed test data for each version. The performance comparison test report contains three levels of information output: The best version recommendation layer identifies the BMC version with the shortest average startup time that passes the threshold check, serving as the current optimal choice. The version ranking list layer sorts all tested versions in descending order of overall performance score, displaying a comparison of core metrics for each version. The detailed data appendix layer provides complete raw test data for each version, a stage time breakdown table, timeline charts, and other supporting materials for technical personnel to review and verify in depth.

[0038] In some embodiments of the present invention, a communication connection is established with the server BMC serial port, and the BMC serial port information is processed in stages, including: The BMCSerialReader class: captures and analyzes BMC startup logs in real time via serial port connection, constructing a complete startup timeline. Specifically, connect() - Establishes a serial port connection read_boot_timeline(timeout) - Core method, monitors the boot process and identifies key events; _classify_phase(event_name) - An internal method that classifies critical events into the appropriate startup phase; get_phase_times() - Analyzes the time consumed in each startup phase; disconnect() - Closes the serial port connection Configure the serial port, specifically, # Serial port configuration 'serial_port': ' / dev / ttyUSB0', # Path to the serial port device in the Linux system # 'serial_port': 'COM3', # The serial port device name in Windows. By configuring the serial port connection parameters, you can monitor the serial port output information during the BMC startup process.

[0039] It is understood that, in some embodiments of the present invention, while the programmable power distribution unit performs the power-on operation, a communication connection with the server BMC serial port is initiated to continuously read the serial port output data stream at a set baud rate. A multi-threaded or asynchronous I / O mechanism is employed to ensure that real-time capture of serial port data is not lost. The captured raw byte stream is converted to character encoding and then stored in a circular buffer or directly written to a log file, forming a complete record of the startup process.

[0040] The testing system parses the captured serial port logs line by line, using preset regular expression patterns to match key event identifier strings. Whenever a key event is successfully matched, the system immediately records the current time and categorizes the event into the corresponding startup phase. After traversing the complete log, the system calculates the time difference between adjacent key events to obtain the time consumed by each startup phase, and then sums the times consumed by each phase to obtain the total startup time, ultimately constructing a complete timeline data structure for this startup.

[0041] Event timeline recording: A timeline file is created for each identified key event, recording its offset time relative to the start point, forming the time coordinates of the event sequence. This timeline data is not only used for stage duration calculation, but also supports the drawing of Gantt charts or time sequence diagrams of the startup process, enabling a visual presentation of the startup process.

[0042] Event Statistics Count: Counts the total number of successfully matched critical events during this test and compares it with a preset complete event list to calculate the event capture coverage. If the critical event missing rate exceeds a preset proportion, the test data is marked as abnormal, indicating possible serial port data loss or abnormal termination of the startup process.

[0043] The following is a detailed index system for statistical calculations provided in the embodiments of the present invention: Average startup time calculation: The arithmetic mean of the total startup times of all successful tests is used as a typical performance representative for this version of BMC. The average calculation supports an optional strategy to remove outliers to improve statistical robustness.

[0044] Standard deviation calculation: Based on the total startup time and average value of each test, the sample standard deviation is calculated to quantify the degree of fluctuation in startup time. The smaller the standard deviation, the more stable the BMC startup performance and the less affected it is by random factors.

[0045] Maximum error percentage calculation: Iterate through the total startup time of each test, find the single test with the largest deviation from the average, and calculate the percentage of its absolute deviation relative to the average. This indicator identifies the extent of performance degradation in the worst case.

[0046] Threshold check judgment: The maximum error percentage is compared with a preset threshold (e.g., 5%). If it does not exceed the threshold, the startup performance of this version of BMC is determined to meet the standard; otherwise, it is marked as unstable performance, triggering an alarm or in-depth analysis process.

[0047] In some embodiments of the present invention, the automated testing method for server BMC startup performance further includes, but is not limited to, steps S930 to S940, specifically, S500: During the automated test of BMC startup performance for each version of BMC to be tested for a preset number of cycles, a preset cooling wait time is set between two adjacent automated tests of BMC startup performance. S510: When the programmable power distribution unit is detected to be powered off, wait for a preset time interval before re-executing the automated test of BMC startup performance.

[0048] In this embodiment of the invention, a preset cooling wait time is set to achieve the periodic recovery of the thermal state of the test environment. Inserting the preset cooling wait time between two adjacent automated tests of BMC startup performance allows the server hardware to dissipate heat sufficiently after continuous operation, returning to an initial thermal state close to a cold start. The technical effect of this thermal state reset is to eliminate test deviations caused by the cumulative effect of hardware temperature, ensuring a high degree of consistency in the starting conditions of each test.

[0049] When the server is overheated, the internal temperature sensor of the BMC may trigger a frequency reduction protection mechanism, or certain hardware initialization processes that rely on temperature calibration may experience timing deviations, thereby lengthening the startup time or introducing random fluctuations. In this embodiment of the invention, a preset cooling wait time of 30 seconds is set. This 30-second preset cooling wait time keeps the hardware temperature variation between multiple tests within an acceptable range, ensuring that the statistically calculated average startup time, standard deviation, and other indicators truly reflect the performance characteristics of the BMC firmware itself, rather than being influenced by thermal decay. It should be noted that the preset cooling wait time can be dynamically configured according to the server's heat dissipation capacity, ambient temperature, and test accuracy requirements, balancing test efficiency and data reliability.

[0050] Upon detecting that the programmable power distribution unit (PDU) has shut down, the automated BMC startup performance test is re-executed after a preset time interval, achieving a complete reset verification of the BMC power state. By detecting the PDU's shutdown operation and extending the wait time beyond that, the system ensures that the BMC power supply capacitors are fully discharged and the internal registers and volatile memory are completely cleared. This deep power-down retention technique ensures that each BMC restart starts from a true zero state, rather than a pseudo-reset state relying on residual charge.

[0051] As an embedded system, the BMC's startup timing is extremely sensitive to the rising edge of the power supply and the internal initial state. If the configuration residue or cached data from the previous run is not cleared, it may lead to abnormal startup path, distorted stage time consumption, or even startup failure.

[0052] In this embodiment of the invention, a preset time interval of 10 seconds is set. This 10-second interval provides sufficient discharge time for the hardware circuit, ensuring that the startup process triggered by subsequent power-on operations has repeatable electrical initial conditions. This technique, combined with the active power-down control of the programmable power distribution unit, forms a reliable reset mechanism at the hardware level, significantly improving the consistency of startup behavior and the effectiveness of fault exposure during stress testing.

[0053] In some embodiments of the present invention, step S600 in the automated testing method for server BMC startup performance includes, but is not limited to, steps S610 to S620. Specifically, S610: Divide the BMC startup process into multiple consecutive startup stages, and use regular expressions to match key fields in the serial port information output by the server BMC serial port; S620: Based on key fields, determine the corresponding key events and classify the identified key events into the corresponding startup stages; among them, multiple consecutive startup stages include: U-Boot boot stage, kernel loading stage, kernel startup stage, hardware detection stage, file system stage, and system service stage.

[0054] This invention divides the BMC boot process into six consecutive stages: U-Boot booting, kernel loading, kernel startup, hardware detection, file system, and system services. It establishes a hierarchical boot analysis framework, achieving a refined breakdown from general boot time to stage-level granular data. By matching key fields in serial port information using regular expressions, it automatically identifies key events and categorizes them into corresponding stages, completing the semantic transformation from unstructured logs to structured data. This technique allows for precise location of performance bottlenecks; for example, abnormal file system mounting time can be identified separately. It also provides timeline points for the boot process; the time difference between adjacent boot stages represents the stage's duration, laying the foundation for constructing an accurate boot timeline. The fuzzy matching capability of regular expressions adapts to changes in log parameters, ensuring robustness of pattern recognition, supporting automated batch processing of large-scale test data, eliminating the subjectivity and error risks of manual review, and significantly improving the efficiency and reliability of BMC boot performance testing.

[0055] Reference Figure 3 As shown, Figure 3 This is a detailed flowchart of step S610 in the automated testing method for server BMC startup performance provided in this embodiment of the invention. Step S610 includes, but is not limited to, steps S611 to S617. Specifically, S611: Key field for identifying serial port information output during the BMC startup process; S612: When the key fields are identified to include bootloader initialization, memory detection, and environment variable loading, the key events corresponding to the key fields will be classified into the U-Boot stage; S613: When the key fields are identified to include kernel image verification, device tree configuration, and RAMDISK preparation, the key events corresponding to the key fields are classified into the kernel loading stage. S614: When the key fields are identified to include Linux kernel initialization, CPU identification, and console activation, the key events corresponding to the key fields will be classified into the kernel startup phase. S615: When the key field is identified as containing hardware initialization of system controller, SPI, I2C, and network interface, the key event corresponding to the key field is classified into the hardware probing phase. S616: When the key fields are identified to include root file system mounting and init process startup, the key events corresponding to the key fields are classified into the file system stage. S617: When the key fields are identified as including systemd service startup, network configuration, and BMC service ready, the key events corresponding to the key fields are classified into the system service phase.

[0056] In this embodiment of the invention, this class uses regular expression pattern matching to subdivide the BMC startup process into six key stages: U-Boot Phase: Bootloader initialization, memory detection, environment variable loading Kernel loading phase: kernel image verification, device tree configuration, RAMDISK preparation Kernel boot phase: Linux kernel initialization, CPU identification, console startup Hardware probing phase: Hardware initialization of system controller, SPI, I2C, network interface, etc. File system phases: root file system mounting, init process startup System service phase: systemd service startup, network configuration, BMC service readiness. The list of relevant key fields is as follows: key_events = { # U-Boot Phase Events 'U-Boot startup': r'Reserving.*Bytes for IRQ stack', 'Memory initialization check': r'MiB.*capacity:.*MiB', 'SPI Flash Detection': r'SF: Detected mx66l', 'Loading Environment variables': r'Loading Environment from SPI Flash', 'Board Information': r'Model: Intel server board with AST2600', 'Network initialization': r'Net:.*eth.*ftgmac', 'NCSI detection': r'ftgmac100_probe.*NCSI detected', 'Automatic Startup Countdown': r'Hit any key to stop autoboot:.*0', # Kernel loading stage 'Kernel FIT loading begins': r'## Loading kernel from FIT Image', 'Device tree configuration': r"Using 'conf-.*\.dtb' configuration", 'Kernel Hash Verification': r'Verifying Hash Integrity.*OK', 'Kernel image information': r'Description:.*Linux kernel', 'RAMDISK loading': r'## Loading ramdisk from FIT Image', 'Device tree loading': r'## Loading fdt from FIT Image', 'Kernel boot preparation': r'Booting using the fdt blob', # Kernel boot phase 'Kernel startup': r'Starting kernel', 'Linux version': r'Linux version.*SMP.*UTC', 'CPU identification': r'CPU: ARMv7 Processor', 'Memory Management': r'Memory:.*available.*reserved', 'Machine model': r'Machine model: Intel Beechnut City BMC', 'Console enabled': r'console \[ttyS4\] enabled', # Hardware Probe Phase 'System Controller': r'ASPEED AST2600 rev', 'SMP start': r'smp: Brought up.*processors', 'Device tree initialization': r'devtmpfs: initialized', 'Serial port driver': r'Serial:.*driver.*ports', 'SPI controller': r'spi-aspeed-smc.*spi', 'Flash chip': r'spi-nor spi.*mx66l.*Kbytes', 'MTD partition creation': r'Creating.*MTD partitions on', 'Network Interface Registration': r'ftgmac100.*ethernet.*irq', 'I2C bus registration': r'aspeed-i2c-bus.*registered', 'I3C controller': r'dw-i3c-master.*registered', 'Video driver': r'aspeed-video.*registered', # File system stage 'File system driver': r'squashfs: version|jffs2: version', 'RAMDISK loading progress': r'RAMDISK: Loading.*KiB.*ram disk', 'Root file system mounted': r'VFS: Mounted root.*squashfsfilesystem', 'Init process startup': r'Run / sbin / init as init process', 'EXT4 file system': r'EXT4-fs.*mounted filesystem', # System Service Phase 'systemd start': r'systemd.*running in system mode', 'Hostname settings': r'Hostname set to', 'Watchdog configuration': r'Watchdog running with a timeout', 'Service Slice': r'Created slice Slice', 'Socket Listening': r'Listening on.*Socket', 'File system mount': r'Mounted.*File System', 'Kernel module loading': r'Finished Load Kernel Module', 'Network Time Synchronization': r'Started Network Time Synchronization', 'DNS resolution service': r'Started Network Name Resolution', 'D-Bus Started': r'Started D-Bus System Message Bus', 'BMC service started': r'Started Phosphor.*Manager', 'IPMI Service': r'Started Phosphor Inband IPMI', 'System Ready': r'Reached target Multi-User System'} This invention identifies key fields in the serial port information during the BMC boot process, accurately categorizing critical events into six consecutive stages: U-Boot booting, kernel loading, kernel startup, hardware detection, file system, and system services. This technique establishes a mapping mechanism from raw logs to structured stage data, ensuring each critical event has a clear stage affiliation and timing. This stage categorization based on key fields enables automated, layered parsing of the boot process, providing a data foundation for accurate calculation of stage durations and performance bottleneck location and diagnosis, significantly improving the automation level of BMC boot performance testing and the efficiency of fault analysis.

[0057] Reference Figure 4 As shown, Figure 4 This is a detailed flowchart of step S700 in the automated testing method for server BMC startup performance provided in this embodiment of the invention. Step S700 includes, but is not limited to, steps S710 to S730. Specifically, S710: When a startup phase corresponding to a key field is matched, the current time is immediately recorded as the time point when the corresponding startup phase occurs. S720: Calculate the difference between the time points of two adjacent startup phases to obtain the time consumption of each startup phase; S730: Construct the BMC startup timeline based on the time points of each startup phase and the duration of each startup phase.

[0058] This invention achieves precise timing reconstruction of the BMC startup process by recording the time points of key field matching in real time. When a regular expression matches a key field of a startup stage, the current system time is immediately captured as the occurrence time of that stage. This event-driven timing mechanism ensures the accuracy and real-time nature of time recording, avoiding timing deviations caused by post-analysis. By calculating the difference between the time points of adjacent startup stages, the precise time consumption of each stage is obtained, quantifying the continuous startup process into discrete time segments, transforming the originally black-box startup behavior into visualized timing data.

[0059] The BMC startup timeline, constructed based on the time points and durations of each stage, forms a complete digital twin of the startup process. It includes both macroscopic metrics of the total startup time and microscopic details of stage durations. This technology supports the creation of Gantt charts of the startup process, visually displaying the time distribution and parallel relationships of each stage; and identifies the location of faults by identifying timeline breaks or missing stages. The structured data format of the startup timeline facilitates storage, retrieval, trend analysis, and intelligent diagnostics, providing robust data support for BMC firmware performance optimization and quality assurance.

[0060] In some embodiments of the present invention, the automated testing method for server BMC startup performance includes, but is not limited to, steps S810 to S820, specifically, S810: Based on the time points of each startup phase in the BMC startup timeline and the duration of each startup phase; S820: Calculate the average time consumed in the BMC startup performance automated test with a preset number of cycles for the same startup phase. If the percentage deviation between the time consumed in the startup phase and the average time consumed in a certain BMC startup performance automated test exceeds a preset threshold, it is determined that the startup phase is abnormal or the BMC startup performance indicator is abnormal and fails to meet the standard.

[0061] This invention achieves intelligent anomaly detection in startup performance based on stage time points and time consumption data in the BMC startup timeline. By calculating the time consumption sequence of the same startup stage in a preset number of test cycles, a performance benchmark distribution for that stage is established. The average time consumption is used as a typical value reference, and the percentage deviation is used as an anomaly measure, forming a quantitative stability evaluation system.

[0062] When the percentage deviation of a certain stage's time from the average time exceeds a preset threshold (e.g., 5%) in a test, the system automatically determines that there is an anomaly in that stage or that the overall performance is substandard. This enables stage-level anomaly localization, accurately identifying whether the problem occurred in a specific stage such as U-Boot booting, kernel loading, or file system mounting, rather than simply reporting a boot failure. This threshold determination mechanism transforms human experience into an objective standard, eliminating the arbitrariness of subjective judgment. Furthermore, this method also supports early warning, detecting abnormal fluctuations in stage time before the overall boot time deteriorates significantly, and capturing potential performance degradation trends.

[0063] In some embodiments of the present invention, step S200 in the automated testing method for server BMC startup performance includes, but is not limited to, steps S210 to S220. Specifically, S210: The controller of the programmable power distribution unit communicates with the programmable power distribution unit via an HTTP POST request; S220: The controller of the programmable power distribution unit uses JSON format to transmit power_off(outlet) control commands, power_on(outlet) control commands, and power_cycle(outlet, delay_seconds) control commands to implement a complete power restart process, thereby controlling the power-on and power-off of the server.

[0064] In this embodiment of the invention, a remote communication channel based on the HTTP protocol is established between the controller of the programmable power distribution unit and the programmable power distribution unit, enabling precise programmatic control of the server's power status. HTTP POST requests are used as the communication carrier, and three types of control commands—power_off, power_on, and power_cycle—are encapsulated in JSON format, forming a standardized power management interface. This Web API-based control method overcomes the physical distance limitations of traditional serial ports or dedicated buses, supporting remote power operation across networks. This allows the test system and server to be deployed in different physical locations, improving the flexibility and scalability of the test environment.

[0065] The JSON-formatted instruction structure offers excellent readability and scalability, ensuring complete BMC discharge while avoiding excessive waiting that could impact testing efficiency. The mature HTTP communication ecosystem provides robust error handling, timeout retries, and security authentication mechanisms, guaranteeing reliable transmission and execution confirmation of control commands. This technology, in conjunction with serial port monitoring, establishes the start-up timing zero point at the moment the programmable power distribution unit's power-on command is issued, achieving precise synchronization between power control and data acquisition. This eliminates time jitter caused by manual operation and provides a reliable time reference for accurately constructing the start-up timeline.

[0066] In some embodiments of the present invention, the automated testing method for server BMC startup performance further includes: configuring the programmable power distribution unit, specifically, # Programmable Power Distribution Unit (PDU) Configuration 'pdu_ip': '192.168.1.100', # IP address of the PDU device 'pdu_username': 'admin', # PDU login username 'pdu_password': 'password', # PDU login password 'pdu_outlet': 1, # The port number of the power outlet to be controlled The power switch of the server is automatically controlled by configuring a programmable power distribution unit (PDU).

[0067] Reference Figure 5 As shown, Figure 5 This is a detailed flowchart of step S100 in the automated testing method for server BMC startup performance provided in this embodiment of the invention. Step S100 includes, but is not limited to, steps S110 to S150. Specifically, S110: Defines the core function of setup_ftp_server, which deploys an FTP server dedicated to BMC storage and distribution; S120: Create a shared directory using os.makedirs to store at least one version of the BMC under test; S130: Use DummyAuthorizer to create an FTP authenticator and add a user account with full permissions, including list, delete, upload, and download permissions; S140: Configure the FTP authenticator and start the FTPServer to continuously listen for connection requests on the specified ports of all network interfaces; S150: When a connection request is detected, automatically pull the corresponding version of BMC to be tested.

[0068] This invention, through defining the core function `setup_ftp_server`, achieves rapid automated deployment of BMC firmware storage and distribution services. It utilizes `os.makedirs` to dynamically create shared directories, ensuring the immediate availability of firmware storage paths and avoiding the tedious manual directory configuration and the risk of path errors. By employing `DummyAuthorizer` to build an FTP authenticator and configuring user accounts with full permissions for listing, deleting, uploading, and downloading, fine-grained access control is achieved. This satisfies the test system's requirement for automated firmware access permissions while preventing unauthorized access through the authentication mechanism. The FTPServer continuously listens for connection requests on designated ports of all network interfaces, supporting concurrent access from multiple clients and providing service support for large-scale distributed testing environments.

[0069] Upon detecting a legitimate connection request, the firmware retrieval process is automatically triggered, achieving a seamless transition from firmware storage to test acquisition. This technology encapsulates the entire process of FTP server deployment, configuration, operation, and response into reusable function modules, forming a lightweight, ready-to-use file service solution. The implementation based on the standard FTP protocol ensures broad compatibility with various BMC upgrade tools and testing frameworks, supports multiple firmware versions coexisting and rapid switching, and provides an efficient file distribution infrastructure for subsequent multi-version comparative testing and regression verification.

[0070] Understandably, the firmware is uploaded using the following command: cp bmc_firmware_v1.0.bin / opt / bmc_firmware / cp bmc_firmware_v1.1.bin / opt / bmc_firmware / cp bmc_firmware_v2.0.bin / opt / bmc_firmware / It should be understood that the method steps in the embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium. The method can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if necessary, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, for this purpose, the program can run on a programmed application-specific integrated circuit (ASIC).

[0071] Furthermore, the procedures described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by the context. The procedures described herein (or variations and / or combinations thereof) may be executed under the control of one or more computer systems configured with executable instructions, and may be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that commonly executes on one or more processors. The computer program comprises a plurality of instructions executable by one or more processors.

[0072] Furthermore, the method can be implemented in any suitable type of computing platform, including but not limited to personal computers, minicomputers, mainframes, workstations, networked or distributed computing environments, standalone or integrated computer platforms, or in communication with charged particle tools or other imaging devices, etc. Aspects of the invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. Furthermore, the machine-readable code, or portions thereof, can be transmitted via wired or wireless networks. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media comprises instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. When programmed according to the methods and techniques described in the invention, the invention may also include the computer itself.

[0073] A computer program can be applied to input data to perform the functions described herein, thereby transforming the input data to generate output data stored in non-volatile memory. The output information can also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including specific visual depictions of physical and tangible objects generated on the display.

[0074] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention, as long as they achieve the technical effects of the present invention by the same means, should be included within the scope of protection of the present invention. Within the scope of protection of the present invention, the technical solutions and / or implementation methods can have various modifications and variations.

Claims

1. An automated testing method for server BMC startup performance, characterized in that, include: Establish an FTP server and create a shared directory to store at least one version of BMC to be tested. The power-on and power-off of the server are controlled by a programmable power distribution unit; Establish a communication connection with the server BMC serial port. For each version of the BMC under test, execute the following sub-steps in a loop: The programmable power distribution unit performs a power-off and power-on operation on the server, triggering a BMC restart. The serial port information output by the server BMC during this startup process is captured and recorded in real time through the server BMC serial port. Analyze the serial port information to identify multiple preset key events and their corresponding startup phases; Calculate the time points of each startup phase and the duration of each phase to construct the BMC startup timeline; Based on the BMC startup timeline, BMC startup performance indicators are calculated, and these performance indicators are compared with preset thresholds to determine whether BMC performance meets the standards.

2. The automated testing method for server BMC startup performance according to claim 1, characterized in that, Also includes: Set a preset number of cycles, and perform a preset number of power-down and power-on operations on the server through the programmable power distribution unit to trigger a preset number of BMC restarts; Record the serial port information output during each BMC restart process; Statistical analysis is performed on the serial port information output during the BMC restart process for a preset number of loops. The total BMC startup time, the time consumed in each startup stage of the BMC, the average startup time of the BMC tested for a preset number of loops, the standard deviation of the BMC startup time, and the maximum percentage error between the single startup time of the BMC and the average startup time of the BMC are calculated. A performance comparison test report containing the best version, version ranking, and test results is generated.

3. The automated testing method for server BMC startup performance according to claim 2, characterized in that, Also includes: During the automated test of BMC startup performance for each version of BMC to be tested, a preset cooling-off time is set between two adjacent automated tests of BMC startup performance. When the programmable power distribution unit is detected to be powered off, the automated test of BMC startup performance is re-executed after a preset time interval.

4. The automated testing method for server BMC startup performance according to claim 1, characterized in that, The analysis of the serial port information to identify multiple preset key events and their corresponding startup phases includes: The BMC startup process is divided into multiple consecutive startup stages, and key fields in the serial port information output by the server BMC serial port are matched using regular expressions. Based on the key fields, the corresponding key events are determined, and the identified key events are categorized into the corresponding startup phases; The multiple consecutive startup phases include: U-Boot boot phase, kernel loading phase, kernel startup phase, hardware detection phase, file system phase, and system service phase.

5. The automated testing method for server BMC startup performance according to claim 4, characterized in that, The step of determining the corresponding key events based on the key fields and classifying the identified key events into the corresponding startup phase includes: Identify the key fields of the serial port information output during the BMC startup process; When the key fields are identified to include bootloader initialization, memory detection, and environment variable loading, the key events corresponding to the key fields are classified into the U-Boot stage; When the key fields are identified to include kernel image verification, device tree configuration, and RAMDISK preparation, the key events corresponding to the key fields are classified into the kernel loading stage. When the key fields are identified to include Linux kernel initialization, CPU identification, and console activation, the key events corresponding to the key fields are classified into the kernel startup phase. When the key fields are identified as including hardware initialization of system controller, SPI, I2C, and network interface, the key events corresponding to the key fields are classified into the hardware probing phase. When the key fields are identified to include root file system mounting and init process startup, the key events corresponding to the key fields are classified into the file system stage. When the key fields are identified as including systemd service startup, network configuration, and BMC service readiness, the key events corresponding to the key fields are classified into the system service phase.

6. The automated testing method for server BMC startup performance according to claim 1, characterized in that, The calculation of the time points of each startup phase and the duration of each phase, constructing the BMC startup timeline, includes: When a startup phase corresponding to a key field is matched, the current time is immediately recorded as the time point when the corresponding startup phase occurred. Calculate the time difference between the occurrence of two adjacent startup phases to obtain the time consumption of each startup phase; The BMC startup timeline is constructed based on the time points of each startup phase and the duration of each startup phase.

7. The automated testing method for server BMC startup performance according to claim 1, characterized in that, The step of calculating BMC startup performance metrics based on the BMC startup timeline and comparing the performance metrics with preset thresholds to determine whether BMC performance meets the standards includes: Based on the time points of each startup stage in the BMC startup timeline and the duration of each startup stage; Calculate the time consumed in the automated test of BMC startup performance for the same startup phase in a preset number of cycles. If the percentage deviation between the time consumed in the startup phase and the average time consumed in a certain automated test of BMC startup performance exceeds the preset threshold, it is determined that the startup phase is abnormal or the BMC startup performance index is abnormal and fails to meet the standard.

8. The automated testing method for server BMC startup performance according to claim 1, characterized in that, The control of the server's power-on and power-off via the programmable power distribution unit includes: The controller of the programmable power distribution unit communicates with the programmable power distribution unit via an HTTP POST request; The controller of the programmable power distribution unit uses JSON format to transmit power_off(outlet) control commands, power_on(outlet) control commands, and power_cycle(outlet, delay_seconds) control commands to realize a complete power restart process, thereby controlling the power-on and power-off of the server.

9. The automated testing method for server BMC startup performance according to claim 1, characterized in that, The step of establishing an FTP server and creating a shared directory, through which at least one version of the BMC to be tested is stored, includes: Define the core function of setup_ftp_server, and deploy an FTP server dedicated to BMC storage and distribution through the core function; Create a shared directory using os.makedirs to store at least one version of the BMC to be tested; Create an FTP authenticator using DummyAuthorizer and add a user account with full permissions, including the ability to list, delete, upload, and download. Configure the FTP authenticator and start the FTPServer to continuously listen for connection requests on the specified ports of all network interfaces; When a connection request is detected, the corresponding version of BMC to be tested is automatically retrieved.

10. A computer device comprising a memory and a processor, characterized in that, When the processor executes a computer program stored in the memory, it performs the method as described in any one of claims 1 to 9.

Images