A server firmware testing method and electronic device

Abstract

The application discloses a server firmware test method and electronic equipment, and relates to the technical field of server firmware automatic test.The application determines a risk score based on a double-driving model of code semantics and historical data, can accurately locate a high-frequency risk module, and generates a targeted edge scene test case, thereby improving edge scene coverage, effectively solving the problem that manual design of a test case is difficult to cover an implicit logic vulnerability, and determining the execution order of the test case, preferentially executing the test case of the high-frequency risk module, and filtering inefficient test cases, thereby improving test efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of server firmware automatic testing, and in particular to a server firmware testing method and an electronic device. BACKGROUND

[0002] With the wide application of cloud computing and large-scale server deployment, the demand for servers is increasing, and effective server firmware testing can guarantee the performance and quality of servers, improve operation and maintenance efficiency, and reduce fault risks. Related technologies rely on manual experience to design test cases, and automatic tools and platforms based on rules or scripts execute preset test cases, but they lack dynamic optimization capabilities and cannot mine high-frequency vulnerability areas based on historical data. In addition, related testing platforms rely on fixed rules or only focus on test case execution, and cannot dynamically learn new version features through data. The closed-loop link of "data collection-model training-case generation-result feedback" is not established, the testing strategy is disconnected from the evolution of firmware, related testing tools lack data modeling capabilities, and historical data is only used for report generation, not for optimizing testing strategies. Or combined with deep learning technology, it is mainly used for fault diagnosis and cannot be combined with test case generation to replace manual decision-making. SUMMARY

[0003] The present application provides a server firmware testing method and an electronic device to at least solve the problems of low server firmware testing efficiency and accuracy and high labor cost in related technologies.

[0004] The present application provides a server firmware testing method, comprising:

[0005] Obtaining first log data of a server firmware, preprocessing the first log data to extract a structured feature vector corresponding to the first log data, wherein the first log data is real-time log data;

[0006] Obtaining a first code segment of the server firmware, inputting the structured feature vector and the first code segment into a multi-modal deep learning model, outputting a risk score, and determining a target risk module in the server firmware based on the risk score, wherein the first code segment is a real-time code segment;

[0007] Determining a test case associated with the target risk module and an execution order of the test case, and generating a test path according to the execution order of the test case;

[0008] Based on the test case and the test path, an executable test script is generated, and a priority is marked to test the server firmware.

[0009] The present application also provides an electronic device, comprising a memory for storing a computer program and a processor for executing the computer program to implement the following steps of the server firmware testing method:

[0010] obtaining first log data of the server firmware, preprocessing the first log data to extract a structured feature vector corresponding to the first log data, wherein the first log data is real-time log data;

[0011] obtaining a first code segment of the server firmware, inputting the structured feature vector and the first code segment into a multi-modal deep learning model, outputting a risk score, and determining a target risk module in the server firmware based on the risk score, the first code segment being a real-time code segment;

[0012] determining a test case associated with the target risk module and an execution order of the test case, and generating a test path according to the execution order of the test case;

[0013] generating an executable test script based on the test case and the test path, and marking a priority to test the server firmware.

[0014] The application also provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the following steps of the server firmware testing method:

[0015] obtaining first log data of the server firmware, preprocessing the first log data to extract a structured feature vector corresponding to the first log data, wherein the first log data is real-time log data;

[0016] obtaining a first code segment of the server firmware, inputting the structured feature vector and the first code segment into a multi-modal deep learning model, outputting a risk score, and determining a target risk module in the server firmware based on the risk score, the first code segment being a real-time code segment;

[0017] determining a test case associated with the target risk module and an execution order of the test case, and generating a test path according to the execution order of the test case;

[0018] generating an executable test script based on the test case and the test path, and marking a priority to test the server firmware.

[0019] The application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the following steps of the server firmware testing method:

[0020] obtaining first log data of the server firmware, preprocessing the first log data to extract a structured feature vector corresponding to the first log data, wherein the first log data is real-time log data;

[0021] The first code segment of the server firmware is acquired, the structured feature vector and the first code segment are input into a multi-modal deep learning model, a risk score is output, and based on the risk score, a target risk module in the server firmware is determined, and the first code segment is a real-time code segment;

[0022] A test case associated with the target risk module is determined, and an execution order of the test case is determined, and a test path is generated according to the execution order of the test case;

[0023] Based on the test case and the test path, an executable test script is generated, and a priority is marked, so as to test the server firmware.

[0024] The application determines the risk score based on the double driving model of code semantics and historical data, can accurately locate the high-frequency risk module, and generates targeted edge scene test cases, improves the edge scene coverage rate, effectively solves the problem that the artificial design of test cases is difficult to cover the implicit logic vulnerability, determines the execution order of the test case, preferentially executes the test case of the high-frequency risk module, and improves the test efficiency by filtering the inefficient test cases. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0026] Figure 1 An application environment diagram of a server firmware test method provided by an embodiment of the application is provided;

[0027] Figure 2 A whole flowchart of a server firmware test method provided by an embodiment of the application is provided;

[0028] Figure 3 A system module whole structure diagram provided by an embodiment of the application is provided;

[0029] Figure 4 A test case generation flowchart provided by an embodiment of the application is provided;

[0030] Figure 5 A closed-loop optimization mechanism flowchart provided by an embodiment of the application is provided;

[0031] Figure 6 An internal structure diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0032] With reference to the drawings and embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0033] It should be noted that, in the description of the present application, the terms “comprise”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0034] It should be noted that the terms “S1”, “S2” and the like are only used for the purpose of describing the steps, and do not specifically refer to the order or sequence, nor do they limit the present application. They are only used to facilitate the description of the method of the present application, and cannot be understood as indicating the sequence of the steps. In addition, the technical solutions of various embodiments can be combined with each other, but must be based on the fact that a person of ordinary skill in the art can implement it. When the combination of technical solutions contradicts each other or cannot be implemented, it should be considered that the combination of technical solutions does not exist, nor is it within the protection scope claimed by the present application.

[0035] According to the background art, the related art relies on manual experience to design test cases, and the automatic tools and platforms based on rules or scripts execute preset test cases, but they lack dynamic optimization capability and cannot mine high-frequency vulnerability areas according to historical data. In addition, the related test platform relies on fixed rules or only focuses on test case execution, cannot dynamically learn new version characteristics through data, does not establish a closed-loop link of “data collection-model training-case generation-result feedback”, the test strategy is out of touch with firmware evolution, the related test tool lacks data modeling capability, only uses historical data for report generation, and is not used for optimizing test strategy, or combines deep learning technology for fault diagnosis, is not combined with test case generation, cannot replace manual decision-making, and specifically:

[0036] (1) The related art relies on manual experience to design test cases, which has two problems:

[0037] Insufficient coverage: potential fault scenarios of complex firmware (such as BMC-baseboard management controller supporting multi-node communication) (such as power timing exception, cross-version compatibility conflict) are difficult to cover by manual enumeration, especially edge scenarios and implicit logic vulnerabilities;

[0038] Inefficiency: To achieve high coverage, testers need to write a large number of redundant use cases, resulting in long test cycles and resource waste (e.g., GPU - graphics processor server stress testing needs to repeatedly execute similar processes);

[0039] That is, rule-based or script-based automation tools can only execute pre-set use cases, lack dynamic optimization capabilities, and cannot mine high-frequency vulnerability areas based on historical data.

[0040] (2) Test strategy lag caused by frequent version iteration, i.e., server firmware is updated frequently (e.g., BMC iterates 1-2 times per month), and related test solutions have the following problems:

[0041] Use case failure: static test scripts are prone to failure due to firmware interface changes (e.g., IPMI - Intelligent Platform Management Interface Instruction Set upgrade), and need to be manually re-adapted;

[0042] Strategy rigidity: potential risk points of new version firmware (e.g., new PCIe - high-speed serial computer expansion bus standard hot plug function) are not included in the test range, resulting in missed testing;

[0043] That is, related test platforms (e.g., UI - user interface automation tools based on DOM elements) rely on fixed rules and cannot dynamically learn new version features through data, resulting in a disconnect between test strategies and firmware evolution.

[0044] (3) Historical test data value is not fully mined, i.e., the massive test data accumulated by enterprises (e.g., BUG - vulnerability distribution, log error code, test execution path) is not systematically analyzed, and the following problems exist:

[0045] Repeated testing: the same module is repeatedly tested due to the lack of historical vulnerability pattern recognition (e.g., multiple redundant verifications of the BIOS - Basic Input Output System memory initialization module);

[0046] Blind area left: cross-module coupling faults (e.g., BMC firmware and BIOS startup process interaction anomalies) are not discovered through data correlation analysis;

[0047] That is, related test tools lack data modeling capabilities and only use historical data for report generation, not for optimizing test strategies.

[0048] (4) Insufficient intelligence level leads to high dependence on manpower, i.e., test solution design highly depends on engineers' experience, and the following problems exist:

[0049] Experience limitations: use cases designed by novice engineers may miss critical scenarios (e.g., BIOS startup timeout under specific hardware configurations);

[0050] Labor costs: complex testing requires multiple people to collaborate (e.g., BMC multi-node communication testing), and team collaboration costs skyrocket.

[0051] (5) Related deep learning technologies (such as CNN-convolutional neural network, LSTM-long short-term memory network) are mostly used for fault diagnosis, cannot replace manual decision-making, lack closed-loop feedback test optimization mechanism, and the related test process is one-way execution, which has the following problems:

[0052] Low efficiency of iteration: the test results are not used to dynamically optimize the model, and the same type of vulnerability may reappear in the new version;

[0053] Coverage rate decay: after the expansion of firmware functions, the original test scheme is not updated synchronously, and the coverage rate decreases over time;

[0054] That is, the related test platform (such as a distributed test framework) only focuses on the execution of test cases and does not establish a closed-loop link of "data collection-model training-test case generation-result feedback".

[0055] To solve the above technical problems, the server firmware test method and electronic device provided by the present application determine the risk score based on the double-driven model of code semantics and historical data, can accurately locate high-frequency risk modules, and generate targeted edge scene test cases, improve the edge scene coverage rate, effectively solve the problem that manual design of test cases cannot cover implicit logical vulnerabilities, determine the execution order of test cases, preferentially execute test cases of high-frequency risk modules, and improve test efficiency by filtering inefficient test cases.

[0056] In order to make the person skilled in the art better understand the present application scheme, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0057] The server firmware test method provided by the present application can be applied to the application environment as shown in Figure 1 . Among them, the terminal 102 communicates with the data processing platform set on the server 104 through the network, wherein the terminal 102 can be but not limited to various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, and the server 104 can be realized by an independent server or a server cluster composed of multiple servers.

[0058] As shown in Figure 2 , the embodiment of the present application provides a server firmware test method, which is applied to the terminal in Figure 1 as an example for description, including the following steps:

[0059] S1: Obtain the first log data of the server firmware, and pre-process the first log data to extract the structured feature vector corresponding to the first log data, wherein the first log data is real-time log data.

[0060] It should be noted that the server firmware is a bottom layer software program embedded in the server hardware, responsible for controlling and managing hardware components (such as processors, memories, storage devices, etc.), providing basic running support for the operating system, usually including core components such as BIOS (Basic Input / Output System) / UEFI (Unified Extensible Firmware Interface), BMC (Baseboard Management Controller), controller firmware, etc. The server firmware log data refers to the detailed operation recorded by the server bottom layer firmware (such as BMC, BIOS) during runtime, which is automatically generated by the firmware itself, and records hardware status, management instruction execution, error and warning events in real time, and the main content includes:

[0061] (1) Hardware events: power state switching, CPU / memory / disk abnormalities, temperature, voltage, fan speed, and sensor readings exceeding thresholds;

[0062] (2) Operation instructions: all commands executed through IPMI (Intelligent Platform Management Interface) and other out-of-band management interfaces and their results;

[0063] (3) Power-on self-test information: hardware faults or configuration problems detected by BIOS during the startup phase;

[0064] (4) Error codes: specific codes and descriptions of various errors, timeouts, and check failures that occur during system operation.

[0065] The first log data is real-time server firmware log data, and the structured feature vector refers to a set of standardized, machine-readable numerical arrays obtained after preprocessing the original unstructured log data.

[0066] S2: Obtain the first code segment of the server firmware, input the structured feature vector and the first code segment into the multi-modal deep learning model, output the risk score, and based on the risk score, determine the target risk module in the server firmware, and the first code segment is a real-time code segment.

[0067] It should be noted that the code snippet refers to a key code part in the server firmware that is running, which contains context information and execution logic; the multi-modal deep learning model can effectively identify abnormal behaviors and potential risks in firmware running by analyzing the structured feature vector and the first code snippet, and the output risk score reflects the possible impact of the risk module. After determining the target risk module, the system can perform further detailed detection or repair operations on it to improve the stability and security of the server; the multi-modal deep learning model is composed of a deep learning model based on an attention mechanism and a long short-term memory network, i.e., a Transformer model and an LSTM network; the risk module in the server firmware refers to a server firmware functional component with a high probability of having potential defects or faults, such as the I2C module of the BMC.

[0068] S3: Determine the test case associated with the target risk module and the execution order of the test case, and generate a test path according to the execution order of the test case.

[0069] It should be noted that the test case is a set of execution conditions, steps, input data and expected results written to verify whether a specific software or hardware function meets the design requirements. In server firmware testing, a test case will clearly describe how to operate the system under test (such as sending specific instructions through IPMI, setting BIOS configuration options), in what state to operate, and the correct response of the system (such as returning a success code, specific events appearing in the log, and hardware state changing as expected). The core purpose is to efficiently and repeatedly expose defects, and by systematically executing a large number of test cases, the behavior of the firmware in different scenarios can be evaluated to ensure that its functionality, performance, reliability and stability meet the predetermined standards, which is the basis for ensuring product quality; the test path refers to the execution order sequence of a group of test cases intelligently planned in the automated testing process, which defines the specific route of test operation.

[0070] S4: Based on the test case and the test path, generate an executable test script and mark the priority to test the server firmware.

[0071] It should be noted that the executable test script is a program file (such as Python, Robot Framework script) output by the test generation engine, which can be directly run by the test framework or command line parsing, and it converts abstract test cases into a series of specific and automated operation instructions (such as sending IPMI commands, checking logs, and asserting results), which realizes the automation from "test plan" to "test action"; The priority is the importance level label (such as P0, P1, P2, etc.) marked on each test case or script block, such as P0 (highest): must be executed, corresponding to the core function or high-risk module; P1 (medium): recommended execution, corresponding to important functions or medium-risk modules; P2 (low): optional execution, corresponding to general functions or low-risk scenarios.

[0072] In the above embodiments, the application determines the risk score based on the double driving model of code semantics and historical data, can accurately locate high-frequency risk modules, and generates targeted edge scenario test cases, improves the edge scenario coverage rate, effectively solves the problem that manual design of test cases is difficult to cover implicit logic vulnerabilities, determines the execution order of test cases, executes the test cases of high-frequency risk modules first, and filters inefficient test cases to improve test efficiency.

[0073] In some specific embodiments, before obtaining the first log data of the server firmware, the method further comprises:

[0074] Receiving a server firmware test request;

[0075] In response to receiving the server firmware test request, obtaining the first log data of the server firmware, and preprocessing the first log data.

[0076] In the above embodiments, before obtaining the first log data, it is determined whether there is a server firmware test request to determine whether to perform the server firmware test operation. Through the pre-positioned request confirmation mechanism, it is ensured that the log grabbing, model calculation, test execution and a series of high-overhead operations are only started when there is a test demand, thereby reducing the invalid energy consumption and calculation load of the system, and at the same time ensuring that the first log data collected has clear purpose and timeliness, providing a high-quality data starting point for subsequent analysis and decision-making, and improving the efficiency and economy of the entire test process from the source.

[0077] In some specific embodiments, preprocessing the first log data to extract the structured feature vector corresponding to the first log data comprises:

[0078] In response to the first log data being unstructured log data, the unstructured log data is parsed to obtain structured log data. For example, the unstructured log data is a BMC kernel log.

[0079] Feature extraction is performed on the structured log data.

[0080] The extracted features are numerically encoded to generate a numerical vector of a preset dimension, and the numerical vector is normalized to obtain a structured feature vector.

[0081] In some embodiments, before the structured feature vector and the first code segment are input into the multi-modal deep learning model and the risk score is output, the method further includes:

[0082] The second log data of the server firmware and the second code segment are obtained, wherein the second log data is historical log data, and the second code segment is a historical code segment;

[0083] In response to the second log data being unstructured log data, the unstructured log data is parsed to obtain structured log data.

[0084] Feature extraction is performed on the structured log data.

[0085] The extracted features are numerically encoded to generate a numerical vector of a preset dimension, and the numerical vector is normalized to obtain a structured feature vector.

[0086] Based on a preset ratio, the structured feature vector and the second code segment are respectively divided into a training set and a test set, and the target type data in the training set is oversampled to obtain a target training set. The preset ratio can be set according to actual needs, such as 7:3.

[0087] Based on the target training set and the test set, the initial multi-modal deep learning model is trained and tested respectively to determine a target multi-modal deep learning model. The target multi-modal deep learning model includes a deep learning model based on an attention mechanism and a long short-term memory network.

[0088] Specifically, for example, Figure 3As shown, the collected log data can include data in the historical test database, such as stored firmware versions (e.g., BMC v2.1), BUG types (e.g., power management timeout), test case execution logs (e.g., test case name, execution result), and real-time test feedback: current test generated logs (e.g., error code of failed cases, firmware state snapshot); further, the data preprocessing process includes three core steps of log parsing, feature extraction, and numerical encoding, to convert unstructured logs (e.g., BMC kernel logs) into structured feature vectors (e.g., power state switching times, IPMI instruction response time), first, log parsing, for log data with fixed format, regular expression matching is used to define log pattern templates, extract fixed format content, for log data without fixed format, similar patterns can be identified through clustering algorithm to generate unified templates; then feature extraction, based on business requirements, define quantitative indicators, according to analysis target, define feature type and calculation logic, for example, count the number of state switching events within a certain period of time, extract the timestamp difference between instruction sending and response, convert power state, error type (timeout / verification failure) into One-Hot vector, use sliding window to count event combinations (e.g., "power switching -> temperature alarm -> fan acceleration"); finally, numerical encoding, integrate the extracted indicators into a fixed-dimensional numerical vector, then perform normalization processing to avoid the influence of dimension difference on model training; data augmentation: oversample sparse BUG types (e.g., PCIe link training failure) to balance the training set; input the structured feature vector and firmware code segment (e.g., C code of BIOS power management module) into a multi-modal deep learning model to train the model, the multi-modal deep learning model includes two branches, the first branch is used for code analysis, i.e., using a Transformer model to extract code semantic features (e.g., function call relationship), the second branch is used for data-driven, i.e., using an LSTM network to analyze the time series distribution of historical BUGs, through the model, a high-risk module probability distribution (e.g., BMC communication stack risk weight is 0.92) can be output, i.e., a risk score (e.g., I2C module risk score 85 / 100).

[0089] In the above embodiment, through the preprocessing step, unstructured massive logs are converted into machine-readable structured feature vectors, solving the problem that the model cannot directly process text, laying the foundation for analysis, and the multi-modal model combines log time series features (analyzing BUG time series rules through LSTM) and code semantic features (extracting function call relationships through Transformer), i.e., through multi-modal correlation modeling, breaking through the limitations of single dependence on artificial rules or static data (i.e., single data source), and can more comprehensively capture hidden, cross-module complex defect patterns in firmware, thereby generating highly targeted test schemes, greatly improving the precision of testing and vulnerability detection rate.

[0090] In some embodiments, the structured feature vector and the first code segment are input into a multi-modal deep learning model, a risk score is output, and based on the risk score, it is determined that the target risk module in the server firmware includes:

[0091] The structured feature vector and the first code segment are input into a target multi-modal deep learning model, a risk score is output, and the structured feature vector and the first code segment correspond to a functional module in the server firmware.

[0092] In response to the risk score being greater than a preset threshold, it is determined that the functional module corresponding to the structured feature vector and the first code segment is the target risk module, wherein the preset threshold can be set according to actual needs, and the target risk module is a high-risk module.

[0093] Specifically, the code repository address of the current server firmware version to be tested (such as BMC v3.0) is input, the system automatically pulls the code and parses the module dependency relationship (such as the BMC communication module depends on the I2C driver), the model compares historical data (such as the BMC v2.5 once had an I2C bus timeout bug), and combines the current code changes (such as the v3.0 modified the I2C clock configuration), and outputs a risk score (such as the I2C module risk score is 85 / 100). When the risk score is greater than a preset threshold (such as 80), the I2C module is determined to be the target risk module.

[0094] In the above embodiments, through the quantitative risk score output by the model, the system can accurately locate the most vulnerable and most likely to fail specific module (such as the BMC communication stack and the BIOS power management) in the firmware, greatly improving the efficiency and defect detection rate of the test, avoiding the waste of redundant test resources on low-risk modules, and improving the test efficiency.

[0095] In some embodiments, the method further includes:

[0096] Obtaining the number of occurrences of the target risk module of the server firmware in a plurality of time periods, wherein the length of the time period can be set according to actual needs, such as 24 hours, and the number of time periods can be set according to actual needs;

[0097] In response to the number of occurrences being greater than a first preset value, determining the time nodes at which the target risk module occurs each time, and according to the time nodes, counting the number of same time nodes, wherein the first preset value can be set according to actual needs;

[0098] In response to the number of same time nodes being greater than a second preset value, marking the time nodes and the target risk module respectively, and generating a corresponding mapping relationship and saving it to a database, wherein the second preset value can be set according to actual needs;

[0099] In response to reaching the time node of the mark within the target time period, the target risk module corresponding to the time node is directly determined based on the mapping relationship without performing risk scoring, so as to directly determine the test case associated with the target risk module.

[0100] In the above embodiments, by directly determining the risk module, the test efficiency can be further improved on the basis of ensuring the test accuracy.

[0101] In some specific embodiments, the test case associated with the target risk module and the execution order of the test case are determined, and a test path is generated according to the execution order of the test case.

[0102] Based on the association relationship between the risk module and the test case, the test case associated with the target risk module is determined, wherein the association relationship is pre-stored in the database;

[0103] Based on the reinforcement learning mechanism, a reward function is constructed to maximize the test coverage and minimize the test execution time as the optimization target. The reinforcement learning mechanism is to construct the reward function based on the Q-Learning algorithm. The algorithm can be R = w1A + w2Q, wherein R represents the reward value, w1 and w2 represent weight coefficients for adjusting the preference between coverage and test time, A represents the number of BUGs found after performing the action, and Q represents the time estimation value saved by skipping low priority and / or covered tests;

[0104] Based on the reward function, the risk score of the target risk module, the logical dependency relationship of the server firmware and the current test state, the execution order of the test case is determined;

[0105] According to the execution order of the test case, a test path is generated.

[0106] Specifically, retrieve test cases associated with the target risk module (such as "I2C bus stress test_100,000 read and write") from the use case library, and filter out inefficient test cases (such as "I2C basic function test" that has been verified by history), further, first determine the test case priority of the target risk module through the risk score of the target risk module, then dynamically select the test case execution order through the reward function, that is, preferentially execute the test case corresponding to the high probability fault module predicted by the model (such as BIOS memory initialization test), and then adjust the execution order according to the firmware logical dependency relationship (such as testing BMC firmware startup first, and then testing IPMI instruction set), that is, if the test case of the high-risk module has dependent test cases, the test cases and the test case of the high-risk module are tested preferentially, further, an optimal test path is generated using a reinforcement learning algorithm, including state space: tested module set, remaining test time; action space: select the next test case (such as "I2C bus stress test" or "BMC firmware startup delay test"); reward function: reward +10 for each bug found, and reward +5 for each hour of test time saved.

[0107] In the above embodiment, the test path is dynamically optimized by the reinforcement learning strategy (Q-Learning algorithm), the reward function is "maximizing coverage" and "minimizing time", the high-risk module test case is preferentially executed and the inefficient verification (such as filtering the basic function test that has passed history) is skipped, which can shorten the test period and reduce redundant test resources (such as repeated execution of GPU stress test).

[0108] In some specific embodiments, as shown in Figure 4 Based on the test case and the test path, an executable test script is generated, and a priority is marked, so as to test the server firmware, including:

[0109] Obtain a test path, the test path being used to describe an execution order of a plurality of test cases;

[0110] Obtain a basic test script template corresponding to the target test case, wherein the basic test script template is a pre-defined, parameterized test instruction code framework, each template corresponds to a specific test scenario or function point (such as "memory stress test" and "BMC firmware upgrade"), and the basic test script template includes core logic and commands required for executing the test (such as codes for calling ipmitool to send instructions, parsing responses, and recording logs), but specific parameters of the object under test (such as target server IP, firmware version number, and test duration) are reserved as variables, when the final script is generated, the system only needs to fill the specific parameters in the optimized test path into the corresponding template, and then perform serialization splicing, so as to quickly generate a large and reliable automated test script without the need to write from scratch, ensuring efficiency and consistency;

[0111] According to the execution order of the test cases in the test path, the basic test script template is serialized and spliced to generate a main execution script;

[0112] According to a preset allocation rule, a priority label is allocated to the target test case in the test path, wherein the preset allocation rule includes determining the priority label based on the model output result, such as P0: must execute, P1: optional execution;

[0113] The priority label is injected into the corresponding code segment in the main execution script, and an executable test script is generated by the adaptive test engine, which is in Python / Robot Framework format.

[0114] In some specific embodiments, testing the server firmware includes:

[0115] The server firmware is deployed to a test environment, and it is determined that the server firmware to be tested and the management terminal network have been successfully connected, wherein the management terminal refers to a special host or virtual machine responsible for centralized control, scheduling and execution of test tasks in the server firmware automated test environment;

[0116] The executable test script is sent to the management terminal, and the executable test script is run on the management terminal to obtain the test results corresponding to the executable test script;

[0117] Based on the test results, a structured test report is generated, i.e. the script automatically sends a set of test instructions to the server in sequence, captures and records the response data of the server to the instructions and the firmware running state in real time, automatically determines the result of each test as pass or fail, and integrates all test results to automatically generate a structured test report.

[0118] In the above embodiments, by automatically converting the optimized test path into an immediately executable script, the delay and errors of manual conversion are eliminated, and at the same time, the embedded priority label enables flexible determination of test depth according to real-time resources and progress, realizes optimal dynamic allocation of test resources, improves the adaptability and efficiency of the overall test process, and automatically integrates the test results, further improving the test efficiency.

[0119] In some specific embodiments, after generating the structured test report, the method further includes:

[0120] Based on the data in the test report and the preset time period, incremental training is performed on the multimodal deep learning model. The preset time period can be set according to actual needs, such as 24 hours, which means that the model is fine-tuned daily to update the weight parameters to adapt to the new firmware version. Incremental training refers to an optimization method that allows the model to update its parameters by gradually adding new data based on existing knowledge without having to train from scratch.

[0121] Based on the incremental training results, the weight parameters in the multimodal deep learning model are updated to obtain the target multimodal deep learning model.

[0122] Specifically, such as Figure 5 As shown, this step specifically includes: Data Feedback: After test execution, the system automatically extracts the following data: New BUG characteristics: such as error type (timeout / data validation failure), triggering module (BMC network stack); Test case validity: Test cases that fail to detect BUGs are marked as "inefficient test cases" (their priority needs to be reduced); Incremental training: Training strategy: Online learning is adopted, and the model is updated after each data feedback; Loss function: Cross-entropy loss (predicted BUG distribution vs. actual distribution); Optimizer: AdamW, the learning rate decays with the number of iterations; Model validation: A / B testing: The test plans generated by the new and old models are compared, and the evaluation indicators include: Coverage improvement rate: The new plan covers I2C abnormal scenarios not covered by the old plan; Manpower saving rate: Reduced manual configuration time.

[0123] In the above implementation, a closed-loop incremental optimization mechanism is set up, that is, the "inefficient use case" label is defined and combined with online learning. The model fine-tuning is triggered by real-time test result feedback (such as the AdamW optimizer dynamically adjusting the weights). This enables the test strategy to be updated adaptively with firmware version iterations. In other words, the incremental training of the model (online learning + AdamW optimizer) is driven by real-time feedback of test results (such as new bug features and inefficient use case labels). This allows the system to adapt to firmware function expansion (such as the addition of PCIe hot-plug support) and code changes (such as IPMI instruction set upgrades). This can significantly reduce the recurrence rate of vulnerabilities and avoid the coverage decay problem caused by the rigidity of traditional static testing schemes.

[0124] In the server firmware test method, first log data of the server firmware is obtained, the first log data is preprocessed to extract a structured feature vector corresponding to the first log data, the first log data is real-time log data; a first code segment of the server firmware is obtained, the structured feature vector and the first code segment are input into a multi-modal deep learning model, a risk score is output, and based on the risk score, a target risk module in the server firmware is determined, the first code segment is a real-time code segment; a test case associated with the target risk module is determined, and an execution order of the test case is determined, and a test path is generated according to the execution order of the test case; based on the test case and the test path, an executable test script is generated, and a priority is marked, so as to test the server firmware. The application determines the risk score based on the double driving model of code semantics and historical data, can accurately locate the high-frequency risk module, and generates the targeted edge scene test case, improves the edge scene coverage rate, effectively solves the problem that the artificial design test case is difficult to cover the implicit logic vulnerability, determines the execution order of the test case, preferentially executes the test case of the high-frequency risk module, and filters the inefficient test case, and improves the test efficiency.

[0125] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment.

[0126] It should be understood that, although Figures 2-5 The steps in the flowchart of the application are displayed in sequence according to the arrows, but these steps are not necessarily executed in sequence according to the arrows. Unless otherwise stated herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, Figures 2-5 At least part of the steps in the flowchart of the application can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with other steps or sub-steps or stages of other steps.

[0127] In one embodiment, an electronic device, which can be a terminal, is provided, and an internal structure diagram of the electronic device can be as shown in Figure 6As shown in the figure. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the electronic device is used to communicate with the external terminal through the network connection. The computer program is executed by the processor to implement a server firmware testing method. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0128] Those skilled in the art can understand that, Figure 6 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0129] The embodiments of the present application provide an electronic device, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in the server firmware testing method embodiment, including:

[0130] S1: obtaining first log data of the server firmware, preprocessing the first log data to extract a structured feature vector corresponding to the first log data, wherein the first log data is real-time log data;

[0131] S2: obtaining a first code segment of the server firmware, inputting the structured feature vector and the first code segment into a multi-modal deep learning model, outputting a risk score, and determining a target risk module in the server firmware based on the risk score, the first code segment is a real-time code segment;

[0132] S3: determining a test case associated with the target risk module and an execution order of the test case, and generating a test path according to the execution order of the test case;

[0133] S4: generating an executable test script based on the test case and the test path, and labeling a priority to test the server firmware.

[0134] The embodiment of the application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program.

[0135] S1: obtaining first log data of the server firmware, pre-processing the first log data to extract a structured feature vector corresponding to the first log data, wherein the first log data is real-time log data;

[0136] S2: obtaining a first code segment of the server firmware, inputting the structured feature vector and the first code segment into a multi-modal deep learning model, outputting a risk score, and determining a target risk module in the server firmware based on the risk score, wherein the first code segment is a real-time code segment;

[0137] S3: determining a test case associated with the target risk module and an execution order of the test case, and generating a test path according to the execution order of the test case;

[0138] S4: generating an executable test script based on the test case and the test path, and marking a priority, so as to test the server firmware.

[0139] In an example embodiment, the computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0140] The embodiment of the application further provides a computer program product, and the computer program product includes a computer program, and the computer program is executed by a processor to implement the steps in the server firmware testing method embodiment, including:

[0141] S1: obtaining first log data of the server firmware, pre-processing the first log data to extract a structured feature vector corresponding to the first log data, wherein the first log data is real-time log data;

[0142] S2: obtaining a first code segment of the server firmware, inputting the structured feature vector and the first code segment into a multi-modal deep learning model, outputting a risk score, and determining a target risk module in the server firmware based on the risk score, wherein the first code segment is a real-time code segment;

[0143] S3: determining a test case associated with the target risk module and an execution order of the test case, and generating a test path according to the execution order of the test case;

[0144] S4: generating an executable test script based on the test case and the test path, and marking a priority, to test the server firmware.

[0145] Embodiments of the present application also provide another computer program product, comprising a non-volatile computer readable storage medium, the non-volatile computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps in the server firmware testing method embodiments, comprising:

[0146] S1: obtaining first log data of the server firmware, pre-processing the first log data to extract a structured feature vector corresponding to the first log data, wherein the first log data is real-time log data;

[0147] S2: obtaining a first code segment of the server firmware, inputting the structured feature vector and the first code segment into a multi-modal deep learning model, outputting a risk score, and determining a target risk module in the server firmware based on the risk score, the first code segment being a real-time code segment;

[0148] S3: determining a test case associated with the target risk module and an execution order of the test case, and generating a test path according to the execution order of the test case;

[0149] S4: generating an executable test script based on the test case and the test path, and marking a priority, to test the server firmware.

[0150] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0151] The above describes in detail a server firmware testing method, device, electronic equipment and storage medium provided by the present application. The principles and implementation modes of the present application are described by applying specific examples. The above description of the examples is only applicable to help understand the method and core idea of the present application. It should be noted that, for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the present application.

Claims

1. A server firmware testing method, characterized by, The method comprises: obtaining first log data of server firmware, preprocessing the first log data to extract a structured feature vector corresponding to the first log data, wherein the first log data is real-time log data; obtaining a first code segment of the server firmware, inputting the structured feature vector and the first code segment into a multi-modal deep learning model, outputting a risk score, and determining a target risk module in the server firmware based on the risk score, the first code segment being a real-time code segment, the first code segment containing context information and execution logic; determining a test case associated with the target risk module and an execution order of the test case, and generating a test path according to the execution order of the test case, the execution order of the test case being determined based on a reward function, a risk score of the target risk module, a logical dependency relationship of the server firmware, and a current test state, the reward function being constructed based on a reinforcement learning mechanism and comprising: R = w1A + w2Q, wherein R represents a reward value, w1 and w2 represent weight coefficients, A represents the number of BUGs found after executing an action, and Q represents an estimated time value saved by skipping low-priority and / or covered tests; generating an executable test script based on the test case and the test path, and labeling a priority to test the server firmware; inputting the structured feature vector and the first code segment into the multi-modal deep learning model to output a risk score, and determining a target risk module in the server firmware based on the risk score comprises: inputting the structured feature vector and the first code segment into the target multi-modal deep learning model to output a risk score, the structured feature vector and the first code segment corresponding to a functional module in the server firmware, the target multi-modal deep learning model being obtained by training and testing an initial multi-modal deep learning model based on second log data and a second code segment of the server firmware, and comprising a deep learning model based on an attention mechanism and a long short-term memory network; in response to the risk score being greater than a preset threshold, determining that the functional module corresponding to the structured feature vector and the first code segment is a target risk module.

2. The server firmware testing method of claim 1, wherein, Before obtaining the first log data of the server firmware and preprocessing the first log data, the method further comprises: receiving a server firmware test request; in response to receiving the server firmware test request, obtaining the first log data of the server firmware and preprocessing the first log data.

3. The server firmware testing method of claim 1, wherein, The preprocessing of the first log data to extract a structured feature vector corresponding to the first log data comprises: in response to the first log data being unstructured log data, parsing the unstructured log data to obtain structured log data; performing feature extraction on the structured log data; performing numerical encoding processing on the extracted features to generate a numerical vector of a preset dimension, and performing normalization processing on the numerical vector to obtain the structured feature vector.

4. The server firmware testing method of claim 1, wherein, Before inputting the structured feature vector and the first code segment into a multi-modal deep learning model and outputting a risk score, the method further comprises: obtaining second log data and a second code segment of the server firmware, wherein the second log data is historical log data, and the second code segment is a historical code segment; in response to the second log data being unstructured log data, parsing the unstructured log data to obtain structured log data; performing feature extraction on the structured log data; performing numerical encoding processing on the extracted features to generate a numerical vector of a preset dimension, and performing normalization processing on the numerical vector to obtain a structured feature vector; based on a preset ratio, dividing the structured feature vector and the second code segment into a training set and a test set respectively, and oversampling target type data in the training set to obtain a target training set; based on the target training set and the test set, training and testing an initial multi-modal deep learning model respectively to determine a target multi-modal deep learning model.

5. The server firmware testing method of claim 1, wherein, determining test cases associated with the target risk module and the execution order of the test cases, and generating a test path according to the execution order of the test cases comprises: determining test cases associated with the target risk module based on the association relationship between the risk module and the test case; based on a reinforcement learning mechanism, constructing a reward function with the optimization goal of maximizing test coverage and minimizing test execution time; based on the reward function, the risk score of the target risk module, the logical dependency relationship of the server firmware and the current test state, determining the execution order of the test cases; generating a test path according to the execution order of the test cases.

6. The server firmware testing method of claim 1, wherein, based on the test cases and the test path, generating an executable test script and labeling a priority to test the server firmware comprises: obtaining the test path, which is used to describe the execution order of a plurality of test cases; obtaining a basic test script template corresponding to the target test case; serializing and splicing the basic test script template according to the execution order of the test cases in the test path to generate a main execution script; according to a preset allocation rule, assigning a priority label to the target test case in the test path; injecting the priority label into the corresponding code segment in the main execution script to generate the executable test script.

7. The server firmware testing method of claim 6, wherein, testing the server firmware comprises: deploying the server firmware to a test environment and determining that the server firmware and the management terminal network have been successfully connected; sending the executable test script to the management terminal and running the executable test script on the management terminal to obtain a test result corresponding to the executable test script; based on the test result, generating a structured test report.

8. The server firmware testing method of claim 7, wherein, after generating the structured test report, the method further comprises: based on the data in the test report and a preset time period, incrementally training the multi-modal deep learning model; Based on the incremental training result, weight parameters in the multi-modal deep learning model are updated to obtain a target multi-modal deep learning model.

9. An electronic device, comprising: Comprise: a memory for storing a computer program; a processor for implementing the steps of the server firmware testing method according to any one of claims 1 to 8 when executing the computer program.

Images