Firmware program testing method and device, equipment and storage medium
By combining gray box fuzzing and symbol execution processing methods, new data sets are generated to improve the test coverage of firmware programs, and the problem of insufficient test coverage in the prior art is solved, achieving deeper and more extensive testing.
Patent Information
- Application Number
- CN202411356169.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2025-02-11
AI Technical Summary
In the prior art, the test coverage of firmware programs is insufficient, resulting in limited testing depth and breadth.
Using a method combining gray box fuzzing and symbol execution processing, a new data set is generated to improve code coverage information until preset requirements are met.
By alternately performing gray box fuzzing and symbol execution processing, it is possible to trigger code branches that are difficult to trigger gray box fuzzing, thereby significantly improving the test coverage of firmware programs.
Smart Images
Figure CN119415384A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of software testing technology, and in particular to a method, device, equipment and storage medium for testing a firmware program. Background Art
[0002] Control units play a core role in the fields of smart home, industrial automation control, and vehicle control. The control unit is embedded with firmware programs to enable the control unit to implement certain specific functions. Like computer programs, the firmware programs of control units also contain vulnerabilities and need to be tested for vulnerabilities.
[0003] In the related art, the traditional fuzz testing method is used to test the firmware program. The traditional fuzz testing method has the problem of insufficient test coverage, resulting in limited depth and breadth of the firmware program test.
[0004] Therefore, how to improve the test coverage of firmware programs has become a technical problem that needs to be solved urgently. Summary of the invention
[0005] The main technical problem solved by the present application is to provide a firmware program testing method, apparatus, device and computer-readable storage medium, which can improve the test coverage of the firmware program.
[0006] To solve the above technical problems, a technical solution adopted in the present application is: to provide a testing method for a firmware program, the method comprising: generating a first data set of the firmware program to be tested; performing a gray-box fuzz test on the firmware program to be tested using the first data set to obtain first code coverage information of the firmware program to be tested; wherein the first code coverage information is used to characterize the execution status of each first code branch of the firmware program to be tested; judging whether the first code coverage information meets the preset requirements; in response to the first code coverage information not meeting the preset requirements, performing symbolic execution processing on the firmware program to be tested to construct a new first data set, and re-executing the gray-box fuzz test on the firmware program to be tested using the first data set and its subsequent steps until the latest first code coverage information meets the preset requirements.
[0007] Optionally, symbolic execution processing is performed on the firmware program to be tested to construct a new first data set, including: identifying unexecuted code branches in each first code branch; generating path constraints corresponding to the unexecuted code branches; and solving data that meets the path constraints to obtain a new first data set.
[0008] Optionally, the method further includes: in response to the path constraint condition being unable to be resolved, marking the corresponding unexecuted code branch with a preset identifier; wherein the preset identifier is used to indicate that the unexecuted code branch is unreachable.
[0009] Optionally, the firmware program to be tested is in binary format, and a gray box fuzz test is performed on the firmware program to be tested using a first data set to obtain first code coverage information of the firmware program to be tested, including: performing a first decompilation process on the firmware program to be tested to obtain a first source program corresponding to the firmware program to be tested; performing a first code stub process on the first source program, and converting the first source program after the first code stub process into a first binary firmware program; wherein the first binary firmware program includes a plurality of first code branches and first code segments inserted corresponding to each first code branch; inputting the first data set as an initial input data set into the first binary firmware program; selecting a plurality of first fault input data that causes the first binary firmware program to run abnormally from the input data set, and adding the plurality of first fault input data into the first binary firmware program; A first fault data set; performing mutation processing on the first fault data set to obtain a new first fault data set; determining the current code coverage information of the first binary firmware program based on the execution status of each first code segment; judging whether a first preset iteration condition is satisfied; wherein the first preset iteration condition includes: the number of mutation processing times is greater than or equal to a preset number threshold, or the current code coverage information is the same as the historical code coverage information; in response to not satisfying the first preset iteration condition, inputting the new first fault data set as a new input data set into the first binary firmware program, and re-executing the steps of selecting a number of first fault input data that cause the first binary firmware program to run abnormally from the input data set and its subsequent steps; in response to satisfying the first preset iteration condition, using the latest current code coverage information as the first code coverage information.
[0010] Optionally, based on the execution status of each first code segment, the current code coverage information of the first binary firmware program is determined, including: counting the number of executed code segments in each first code segment; determining the current code coverage information based on the number of executed code segments; and / or, the method also includes: in response to the latest first code coverage information meeting preset requirements, outputting a first test result; wherein the first test result includes all first fault input data.
[0011] Optionally, before generating the first data set of the firmware program to be tested, the method also includes: determining whether the firmware program to be tested has been reinforced; in response to the firmware program to be tested not being reinforced, executing the steps of generating the first data set of the firmware program to be tested and its subsequent steps; in response to the firmware program to be tested being reinforced, generating the second data set of the firmware program to be tested; and, using the second data set to perform gray box fuzz testing on the firmware program to be tested.
[0012] Optionally, the hardening process includes any one of obfuscation, packing, data encryption and dynamic loading.
[0013] Optionally, before generating the first data set of the firmware program to be tested, the method also includes: constructing an initial virtual model; running the firmware program to be tested using the initial virtual model; determining whether the firmware program to be tested is operating normally; in response to abnormal operation of the firmware program to be tested, adjusting model parameters of the initial virtual model, and re-executing the operation of the firmware program to be tested using the initial virtual model and subsequent steps; in response to the firmware program to be tested operating normally, using the latest initial virtual model as the target virtual model of the firmware program to be tested.
[0014] Optionally, constructing the initial virtual model includes: acquiring hardware architecture information and peripheral interface information of the plurality of control units; and constructing the initial virtual model based on the hardware architecture information and peripheral interface information of the plurality of control units.
[0015] To solve the above technical problems, another technical solution adopted in the present application is: to provide a testing device for a firmware program, the device comprising: a data generation module, used to generate a first data set of the firmware program to be tested; a first testing module, used to perform a gray-box fuzz test on the firmware program to be tested using the first data set, and obtain first code coverage information of the firmware program to be tested; wherein the first code coverage information is used to characterize the execution status of each first code branch of the firmware program to be tested; to determine whether the first code coverage information meets the preset requirements; in response to the first code coverage information not meeting the preset requirements, to perform symbolic execution processing on the firmware program to be tested to construct a new first data set, and to re-execute the gray-box fuzz test on the firmware program to be tested using the first data set and its subsequent steps until the latest first code coverage information meets the preset requirements.
[0016] To solve the above technical problems, another technical solution adopted in the present application is: to provide an electronic device, comprising a memory and a processor coupled to each other, the memory storing program instructions; the processor is used to execute the program instructions stored in the memory to implement the above firmware program testing method.
[0017] In order to solve the above technical problems, another technical solution adopted by the present application is: providing a computer-readable storage medium, which is used to store program instructions, and the program instructions can be executed by a processor to implement the testing method of the above firmware program.
[0018] In the above scheme, the first data set is used to perform gray-box fuzz testing on the firmware program to be tested, and when the first code coverage information of the firmware program to be tested does not meet the preset requirements, the firmware program to be tested is subjected to symbolic execution processing to construct a new first data set, and the new first data set is used to perform gray-box fuzz testing on the firmware program to be tested again, and this cycle is repeated until the latest first code coverage information meets the preset requirements. By combining gray-box fuzz testing and symbolic execution processing, and performing gray-box fuzz testing and symbolic execution processing alternately, the first code branch that is difficult to trigger by gray-box fuzz testing can be triggered, thereby improving the test coverage rate of the firmware program to be tested. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a flowchart of an embodiment of a method for testing a firmware program provided by the present application;
[0020] Figure 2 It is a flowchart of another embodiment of the firmware program testing method provided by the present application;
[0021] Figure 3 It is a schematic diagram of a framework of an embodiment of a testing device for a firmware program provided by the present application;
[0022] Figure 4 It is a schematic diagram of a framework of an embodiment of an electronic device provided by the present application;
[0023] Figure 5 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail below with reference to the accompanying drawings and examples.
[0025] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.
[0026] In this article, the term "and / or" is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone; the term "multiple" means two or more than two; the term "at least one" means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B and C; the term "several" means at least one; the terms "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0027] See also Figure 1 , Figure 1 1 is a flowchart of an embodiment of a method for testing a firmware program provided by the present application. The method can be executed by a terminal device, which is any device that can load and run a firmware program. For example, the terminal device can be a desktop computer, a laptop computer, a tablet computer, etc., which is not specifically limited in the present embodiment. It should be noted that if there are substantially the same results, the method of the present application is not limited to the terminal device. Figure 1 The process sequence shown is limited.
[0028] like Figure 1 As shown, the method comprises the following steps:
[0029] S101: Generate a first data set of a firmware program to be tested.
[0030] The firmware program to be tested is used to be embedded in the control unit. The firmware program to be tested is usually a program file in binary format, which includes a series of machine instruction sets for enabling the control unit to implement certain specific functions. Specifically, the firmware program to be tested includes initialization code, hardware drivers, middleware, and possible application layer logic to manage various tasks of the control unit, such as input and output control, communication protocol processing, data calculation, etc. Exemplarily, the control unit can be a controller such as MCU (Microcontroller Unit), FPGA (Field-Programmable Gate Array), PLC (Programmable Logic Controller), etc., which is not specifically limited in this embodiment.
[0031] The first data set includes a plurality of first test data for testing the firmware program to be tested, each of which includes relevant input data required for the firmware program to be tested to run. Specifically, the first data set that meets the input data constraint rules of the firmware program to be tested can be randomly generated.
[0032] S102: Performing a gray box fuzzy test on the firmware program to be tested using the first data set to obtain first code coverage information of the firmware program to be tested.
[0033] Fuzz testing is an automated software testing technology. Its core idea is to input a large amount of random or semi-randomly generated unexpected data into the firmware program to be tested, while monitoring the operation of the firmware program to be tested, aiming to discover the vulnerabilities and potential security holes of the target program under abnormal input. On the basis of fuzz testing, according to the degree of understanding of the internal program structure and implementation details of the firmware program to be tested, it can be further divided into black box fuzz testing, white box fuzz testing and gray box fuzz testing. Among them, black box fuzz testing can be used for testing without knowing the internal program structure and implementation details of the firmware program to be tested; white box fuzz testing can be used for testing with the internal program structure and implementation details of the firmware program to be tested; gray box fuzz testing is between black box fuzz testing and white box fuzz testing, and can be used for testing with the understanding of part of the internal program structure and part of the implementation details of the firmware program to be tested. In this embodiment, the gray box fuzz testing method is specifically used to test the firmware program to be tested.
[0034] The first code coverage information is used to characterize the execution status of each first code branch of the firmware program to be tested. The higher the value of the first code coverage information, the more first code branches are triggered during the gray box fuzz test of the firmware program to be tested (i.e., the more first code branches are executed); the lower the value of the first code coverage information, the fewer first code branches are triggered during the gray box fuzz test of the firmware program to be tested (i.e., the fewer first code branches are executed).
[0035] S103: Determine whether the first code coverage information meets preset requirements.
[0036] Specifically, the preset requirement includes: the value of the first code coverage information is greater than or equal to the first preset threshold. When the value of the first code coverage information is greater than or equal to the first preset threshold, it is determined that the preset requirement is met. When the value of the first code coverage information is less than the first preset threshold, it is determined that the preset requirement is not met. The first preset threshold is set according to actual needs.
[0037] S104: In response to the first code coverage information not meeting the preset requirements, symbolic execution processing is performed on the firmware program to be tested to construct a new first data set, and gray-box fuzz testing and subsequent steps are re-executed on the firmware program to be tested using the first data set until the latest first code coverage information meets the preset requirements.
[0038] Symbolic execution is a software verification technique that uses symbols (rather than specific numerical values) to analyze program behavior for each operation on the execution path of the firmware program under test. This method allows exploration of multiple possible execution paths of the program, and is particularly suitable for detecting logical errors, security vulnerabilities, and potential code defects in conditional statements. During the symbolic execution process, input data is represented as symbolic variables, and path constraints are generated when the program is executed. These path constraints are then used to guide the search for input data that meets specific conditions (such as conditions that cause errors or vulnerabilities). Since the input data generated by gray-box fuzz testing is random, it is difficult to trigger complex first code branches, resulting in the first code coverage information not meeting the preset requirements, that is, the test coverage rate is low. Therefore, in this embodiment, when the first code coverage information obtained by gray-box fuzz testing does not meet the preset requirements, the symbolic execution technology is further combined to construct a new first data set for triggering the unexecuted first code branch, and then the new first data set is used to perform gray-box fuzz testing on the firmware program to be tested, and the obtained new first code coverage information is further judged. If the new first code coverage information still does not meet the preset requirements, it is further combined with the compliance execution technology to construct a new first data set. This cycle is repeated to trigger more first code branches that are difficult to trigger during the gray-box fuzz testing process until the latest first code coverage information of the firmware program to be tested meets the preset requirements.
[0039] In this embodiment, the firmware program to be tested is subjected to gray-box fuzz testing using the first data set, and when the first code coverage information of the firmware program to be tested does not meet the preset requirements, the firmware program to be tested is subjected to symbolic execution processing to construct a new first data set, and the firmware program to be tested is subjected to gray-box fuzz testing again using the new first data set, and this cycle is repeated until the latest first code coverage information meets the preset requirements. By combining gray-box fuzz testing and symbolic execution processing, and performing gray-box fuzz testing and symbolic execution processing alternately, the first code branch that is difficult to trigger by gray-box fuzz testing can be triggered, thereby improving the test coverage of the firmware program to be tested.
[0040] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of another embodiment of the firmware program testing method provided by the present application. Figure 2 As shown, the method comprises the following steps:
[0041] S201: Obtain a firmware program to be tested.
[0042] For example, a storage unit of the terminal device stores several firmware programs, and a firmware program can be selected from the stored firmware programs as the firmware program to be tested.
[0043] For another example, the terminal device may communicate with other devices to obtain the firmware program to be tested from the other devices.
[0044] For another example, the terminal device may also download the firmware program to be tested from a web page.
[0045] S202: Generate a target virtual model corresponding to the firmware program to be tested.
[0046] The target virtual model is used to provide a running environment for the firmware program to be tested. Specifically, the target virtual model can be generated through the following sub-steps:
[0047] Sub-step 1: Construct the initial virtual model.
[0048] In one embodiment, the step of constructing the initial virtual model includes: obtaining hardware architecture information and peripheral interface information of a plurality of control units; and constructing the initial virtual model based on the hardware architecture information and peripheral interface information of the plurality of control units.
[0049] Among them, the several control units include control units with relatively high usage rates in the market, that is, in this embodiment, the hardware architecture information and peripheral interface information of each control unit with relatively high usage rates in the market are summarized.
[0050] The initial virtual model includes the simulator sub-model and the peripheral interface sub-model around the simulator sub-model. The simulator sub-model is an abstract representation of the core processor, interrupt control unit, Flash, RAM, and registers required for the execution of each firmware program. The peripheral interface sub-model is the general pattern and convention followed by each firmware program when accessing the peripheral interface, as well as the acceptable input.
[0051] Exemplarily, the hardware architecture information used to build the simulator sub-model includes information such as the register type, access mode, access processing logic, interrupt mechanism and memory layout of each control unit.
[0052] Exemplarily, the peripheral interface information used to build the peripheral interface sub-model includes information such as the data format, transmission speed specification, and communication protocol standard of the peripheral interface of each control unit.
[0053] The peripheral interface of the control unit refers to the physical and logical interfaces on the control unit used to communicate with external devices. These interfaces allow the control unit to exchange data with various sensors, actuators, storage devices or other electronic components.
[0054] Exemplarily, the peripheral interface includes a USB (Universal Serial Bus) interface, an SPI (Serial Peripheral Interface), an I 2C (Inter-Integrated Circuit) interface, UART (Universal Asynchronous Receiver / Transmitter) interface, CAN (Controller Area Network) interface, PWM (Pulse Width Modulation) interface, ADC (Analog-to-Digital Converter) interface, DAC (Digital-to-Analog Converter) interface, etc. Among them, the USB interface is usually used to connect external devices such as keyboards, mice, storage devices, etc., and the USB interface supports multiple transmission rates. For example, the maximum transmission rate of USB2.0 is 480Mbps, and the maximum transmission rate of USB3.0 is increased to 5Gbps. SPI is a synchronous, full-duplex, master-slave interface, which is often used to connect external devices such as sensors, EEPROMs, and display screens. I 2 The C interface, also known as I-squared-C or IsquaredC, is a simple, bidirectional, multi-host, multi-slave serial communication bus suitable for short-distance communication, and is commonly used for device configuration and reading sensor data. The UART interface provides full-duplex asynchronous serial communication and is commonly used for device communication compatible with standards such as RS-232 and RS-485. The CAN interface is a reliable communication protocol designed for automotive and industrial applications, supporting multi-master and multi-slave, error detection and correction functions. The PWM interface is used to output PWM control signals, which are commonly used to control motor speed, LED brightness, etc., and adjust the output power by changing the width of the PWM signal. The ADC interface is used to convert analog signals into digital signals, and the DAC interface is used to convert digital signals into analog signals.
[0055] Sub-step 2: Using the initial virtual model to run the firmware program to be tested.
[0056] Sub-step three: Determine whether the firmware program to be tested runs normally.
[0057] The running result of the firmware program to be tested can be obtained, and the running result of the firmware program to be tested can be used to determine whether the firmware program to be tested is running normally.
[0058] If the firmware program to be tested runs abnormally, then execute sub-step 4. If the firmware program to be tested runs normally, then execute sub-step 5.
[0059] Sub-step 4: Adjust model parameters of the initial virtual model.
[0060] When the firmware program to be tested runs abnormally, it means that the model parameters of the constructed initial virtual model are missing, which causes the initial virtual model to be unable to run the firmware program to be tested normally, so the model parameters of the initial virtual model need to be adjusted. Exemplarily, the model parameters of the initial virtual model can be adjusted by adding preset reasonable default values, model parameters, etc. to the database corresponding to the constructed initial virtual model to improve the initial virtual model.
[0061] Furthermore, after adjusting the model parameters of the initial virtual model, sub-step 2 is re-executed to run the firmware program to be tested using the adjusted initial virtual model.
[0062] Sub-step 5: Using the latest initial virtual model as the target virtual model of the firmware program to be tested.
[0063] When the firmware program to be tested can run normally, it means that the model parameters of the constructed initial virtual model are relatively complete and no further adjustment is needed. The initial virtual model that has been adjusted multiple times (i.e., the latest initial virtual model) is used as the target virtual model of the firmware program to be tested. The target virtual model can be used to run the firmware program to be tested subsequently.
[0064] In sub-steps 1 to 5, on the one hand, by running the firmware program to be tested through the constructed target virtual model, the test of the firmware program to be tested can be independent of the actual hardware device and not limited by hardware resources. On the other hand, by continuously iteratively optimizing the initial virtual model based on whether the firmware program to be tested can run normally, activating and personalizing the initial virtual model to obtain the target virtual model (i.e., instantiating the initial virtual model to obtain the target virtual model), it can ensure accurate mapping of the operating environment of the firmware program to be tested and improve the reliability of the target virtual model. Moreover, as the number of firmware programs to be tested increases, the target virtual model is continuously upgraded and optimized, which can improve the accuracy, comprehensiveness and versatility of the target virtual model, and improve the compatibility and high matching of the target virtual model for testing other firmware programs in the future.
[0065] S203: Determine whether the firmware program to be tested has been hardened.
[0066] If the firmware program to be tested has not been reinforced, step S204 is executed. If the firmware program to be tested has been reinforced, step S209 is executed.
[0067] Performing hardening processing on the firmware program to be tested can ensure the security of the firmware program to be tested. Exemplarily, the hardening processing includes any one of obfuscation, shelling, data encryption and dynamic loading.
[0068] S204: Generate a first data set of the firmware program to be tested.
[0069] Specifically, a first data set of the firmware program to be tested can be randomly generated according to an instance of the peripheral interface sub-model of the constructed target virtual model to simulate various interaction scenarios in the real world. The first data set includes a number of first test data for testing the firmware program to be tested, and each first test data includes relevant input data required when the firmware program to be tested is running, such as configuration data of relevant registers.
[0070] S205: Performing a gray box fuzz test on the firmware program to be tested using the first data set, and obtaining first code coverage information of the firmware program to be tested.
[0071] Specifically, step S205 may include the following sub-steps:
[0072] Sub-step 1: performing a first decompile process on the firmware program to be tested to obtain a first source program corresponding to the firmware program to be tested.
[0073] The first source program obtained after the first decompilation process is performed on the firmware program to be tested may be an assembly code or other intermediate codes with high readability.
[0074] Sub-step two: performing a first code stub processing on the first source program, and converting the first source program after the first code stub processing into a first binary firmware program.
[0075] Code stub processing is a software engineering technology that refers to inserting additional code segments (i.e. "probes" or "stubs") into a program without changing the program logic. These inserted code segments are mainly used to collect information when the program is running, such as performance monitoring (such as the number of function calls and running time), memory usage analysis, code coverage analysis, or debugging assistance. The inserted code segments can help developers understand the actual execution of the program, track errors and vulnerabilities, and detect and prevent potential security issues.
[0076] Specifically, fine-grained code stubs can be performed in loops, conditional branch structures, and other locations of the first source program. The first binary firmware program includes a number of first code branches and first code segments inserted into the first code branches. It can be understood that the execution status of each first code segment can reflect the execution status of the corresponding first code branch.
[0077] In sub-step 1 and sub-step 2, since the decompiled first source program is more conducive to understanding the structure, algorithm and logic of the program, code stubbing is performed on the decompiled first source program to improve the accuracy of code stubbing.
[0078] Sub-step three: inputting the first data set as an initial input data set into the first binary firmware program.
[0079] Sub-step 4: selecting a number of first fault input data that causes the first binary firmware program to run abnormally from the input data set, and adding the first fault input data to the first fault data set.
[0080] Sub-step five: performing mutation processing on the first fault data set to obtain a new first fault data set.
[0081] Specifically, each first fault input data in the first fault data set is subjected to mutation processing to obtain a new first fault data set.
[0082] In one implementation, each first fault input data may be varied by changing a data format, a data length, or a keyword of each first fault input data.
[0083] In another embodiment, each first fault input data may be input into a large language model or a trained neural network model, and each first fault input data may be mutated using the large language model or the trained neural network model.
[0084] Sub-step six: determining current code coverage information of the first binary firmware program based on the execution status of each first code segment.
[0085] Specifically, the number of executed code segments in each first code segment is counted; and based on the number of executed code segments in each first code segment, the current code coverage information is determined.
[0086] In one implementation, the number of executed code segments in each first code segment is directly used as the current code coverage information.
[0087] In another implementation, the ratio of the number of executed code segments in each first code segment to the total number of first code segments is used as the current code coverage information.
[0088] Sub-step seven, determining whether the first preset iteration condition is met.
[0089] In one embodiment, the first preset iteration condition includes: the number of mutation processes is greater than a preset number threshold. The preset number threshold can be set according to actual needs.
[0090] In another embodiment, the first preset iteration condition includes: the current code coverage information of the first binary firmware program is the same as the historical code coverage information. That is, the current code coverage information of the first binary firmware program no longer changes, and no matter how it changes, it cannot trigger a new first code branch.
[0091] Sub-step eight, in response to the first preset iteration condition not being met, inputting the new first fault data set as a new input data set into the first binary firmware program, and re-executing sub-step four and subsequent sub-steps of sub-step four.
[0092] When the first preset iteration condition is not met, a new first fault data set is input into the first binary firmware program as a new input data set, and several new first fault input data that cause the first binary firmware program to run abnormally are selected from the new input data set to obtain a new first fault data set, and the new first fault data set is mutated so that a new first code branch can be triggered subsequently.
[0093] Sub-step nine: in response to satisfying the first preset iteration condition, taking the latest current code coverage information as the first code coverage information.
[0094] S206: Determine whether the first code coverage information meets preset requirements.
[0095] The preset requirement includes: a value of the first code coverage information is greater than or equal to a first preset threshold.
[0096] If the first code coverage information does not meet the preset requirement, step S207 is executed. If the first code coverage information meets the preset requirement, step S208 is executed.
[0097] S207: Perform symbolic execution processing on the firmware program to be tested to construct a new first data set.
[0098] Further, after executing step S207, step S205 is re-executed to perform gray box fuzz testing on the firmware program to be tested using the new first data set.
[0099] In one embodiment, performing symbolic execution processing on the firmware program to be tested to construct a new first data set may include the following sub-steps:
[0100] Sub-step one: identifying unexecuted code branches in each first code branch.
[0101] Sub-step 2: Generate path constraints corresponding to unexecuted code branches.
[0102] Sub-step three, solving data that meets the path constraint conditions of the unexecuted code branch to obtain a new first data set.
[0103] Specifically, the constraint solver can be used to solve the data that meets the path constraint conditions of the unexecuted code branch, thereby obtaining a new first data set. The constructed new first data set is used for further exploration and verification by the grey box fuzz test.
[0104] Furthermore, considering that unsolvable path constraints may be encountered during symbolic execution, a preset flag can be used to mark the corresponding unexecuted code branches. The preset flag is used to indicate that the unexecuted code branches are unreachable, so as to avoid wasting resources to try to explore them in the subsequent testing process.
[0105] S208: Output the first test result.
[0106] The first test result includes all the first fault input data in the aforementioned test process. Outputting the first test result can provide a detailed basis for subsequent vulnerability analysis and repair.
[0107] S209: Generate a second data set of the firmware program to be tested.
[0108] S210: Performing a gray box fuzz test on the firmware program to be tested using the second data set.
[0109] Specifically, step S210 may include the following sub-steps:
[0110] Sub-step 1: performing a second decompile process on the firmware program to be tested to obtain a second source program corresponding to the firmware program to be tested.
[0111] Sub-step two: performing a second code plugging process on the second source program, and converting the second source program after the second code plugging process into a second binary firmware program.
[0112] Specifically, code stubbing can be performed in loops, conditional branch structures, and key functions of the second source program. The second binary firmware program includes a number of second code branches and second code segments inserted corresponding to each second code branch. It should be noted that since the firmware program to be tested in step S205 has not been reinforced, and the firmware program to be tested in step S210 has been reinforced, the program logic of the firmware program to be tested in step S205 is clearer, and the program logic of the firmware program to be tested in step S210 is more ambiguous, resulting in the number of first code segments inserted in step S205 being greater than the number of second code segments in step S210.
[0113] Sub-step three: inputting the second data set as an initial input data set into the second binary firmware program.
[0114] Sub-step 4: selecting a number of second fault input data that causes the second binary firmware program to run abnormally from the input data set, and adding the number of second fault input data to the second fault data set.
[0115] Sub-step five: performing mutation processing on the second fault data set to obtain a new second fault data set.
[0116] Specifically, each second fault input data in the second fault data set is subjected to mutation processing to obtain a new second fault data set. The relevant contents of the mutation processing can be referred to sub-step 5 in step S205, which will not be described in detail here.
[0117] Sub-step six: determining current code coverage information of the second binary firmware program based on the execution status of each second code segment.
[0118] Specifically, the number of executed code segments in each second code segment is counted; and based on the number of executed code segments in each second code segment, the current code coverage information is determined.
[0119] In one implementation, the number of executed code segments in each second code segment is directly used as the current code coverage information.
[0120] In another implementation, the ratio of the number of executed code segments in each second code segment to the total number of second code segments is used as the current code coverage information.
[0121] Sub-step seven, determining whether the second preset iteration condition is met.
[0122] In one embodiment, the second preset iteration condition includes: the number of mutation processes is greater than a preset number threshold.
[0123] In another embodiment, the second preset iteration condition includes: the current code coverage information of the second binary firmware program is the same as the historical code coverage information. That is, the current code coverage information of the second binary firmware program no longer changes, and no matter how it changes, it cannot trigger a new second code branch.
[0124] In another embodiment, the second preset iteration condition includes: the latest current code coverage information is greater than or equal to a second preset threshold value. The second preset threshold value can be set according to actual needs.
[0125] Sub-step eight, in response to the second preset iteration condition not being met, inputting the new second fault data set into the second binary firmware program as a new input data set, and re-executing sub-step four and subsequent sub-steps of sub-step four.
[0126] When the second preset iteration condition is not met, a new second fault data set is input into the second binary firmware program as a new input data set, and several new second fault input data that cause the second binary firmware program to run abnormally are selected from the new input data set to obtain a new second fault data set, and the new second fault data set is mutated so that a new second code branch can be triggered subsequently.
[0127] Sub-step nine: generating a second test result in response to satisfying a second preset iteration condition.
[0128] The second test result includes all the second fault input data in the aforementioned test process.
[0129] S211: Output the second test result.
[0130] Outputting the second test result can provide a detailed basis for subsequent vulnerability analysis and repair.
[0131] In step S203 to step S211, a differentiated fuzzy testing strategy is flexibly formulated based on whether the firmware program to be tested has been reinforced, and precise testing schemes are designed in the gray box and white box testing dimensions respectively, which further improves the test coverage of the firmware program to be tested, ensures the depth and accuracy of the test results, and provides strong protection for software security.
[0132] See also Figure 3 , Figure 3 FIG. 3 is a schematic diagram of a framework of an embodiment of a test device for a firmware program provided by the present application. In this embodiment, the test device 30 for a firmware program includes: a data generation module 31 and a first test module 32 .
[0133] The data generating module 31 is used to generate a first data set of the firmware program to be tested.
[0134] The first testing module 32 is used to perform gray-box fuzz testing on the firmware program to be tested using the first data set to obtain first code coverage information of the firmware program to be tested; wherein the first code coverage information is used to characterize the execution status of each first code branch of the firmware program to be tested; determine whether the first code coverage information meets the preset requirements; in response to the first code coverage information not meeting the preset requirements, perform symbolic execution processing on the firmware program to be tested to construct a new first data set, and re-execute the gray-box fuzz testing of the firmware program to be tested using the first data set and its subsequent steps until the latest first code coverage information meets the preset requirements.
[0135] Optionally, the first test module 32 is used to identify unexecuted code branches in each first code branch; generate path constraints corresponding to the unexecuted code branches; and solve data that meets the path constraints to obtain a new first data set.
[0136] Optionally, the first test module 32 is further configured to, in response to the path constraint condition being unable to be resolved, mark the corresponding unexecuted code branch with a preset identifier; wherein the preset identifier is used to indicate that the unexecuted code branch is unreachable.
[0137] Optionally, the firmware program to be tested is in binary format. The first test module 32 is used to perform a first decompile process on the firmware program to be tested to obtain a first source program corresponding to the firmware program to be tested; perform a first code stub process on the first source program, and convert the first source program after the first code stub process into a first binary firmware program; wherein the first binary firmware program includes a plurality of first code branches and first code segments inserted corresponding to each first code branch; input the first data set as the initial input data set into the first binary firmware program; select a plurality of first fault input data that causes the first binary firmware program to run abnormally from the input data set, and add the plurality of first fault input data to the first fault data set; perform a mutation process on the first fault data set to obtain a new A first fault data set; based on the execution status of each first code segment, determining the current code coverage information of the first binary firmware program; judging whether a first preset iteration condition is met; wherein the first preset iteration condition includes: the number of mutation processing times is greater than or equal to a preset number threshold, or the current code coverage information is the same as the historical code coverage information; in response to not meeting the first preset iteration condition, inputting a new first fault data set as a new input data set into the first binary firmware program, and re-executing the steps of selecting a number of first fault input data that cause the first binary firmware program to run abnormally from the input data set and its subsequent steps; in response to meeting the first preset iteration condition, using the latest current code coverage information as the first code coverage information.
[0138] Optionally, the first test module 32 is used to count the number of executed code segments in each first code segment; determine the current code coverage information based on the number of executed code segments; and / or, the fuzzy test module is also used to output the first test result when the latest first code coverage information meets the preset requirements; wherein the first test result includes all first fault input data.
[0139] Optionally, the firmware program testing device 30 further includes a judgment module 33 and a second test module 34. The judgment module 33 is used to judge whether the firmware program to be tested has been reinforced. In the case where the firmware program to be tested has not been reinforced, the firmware program to be tested is tested by the first test module 32. In the case where the firmware program to be tested has been reinforced, the firmware program to be tested is tested by the second test module 34. Specifically, the data generation module 31 is also used to generate a second data set of the firmware program to be tested; the second test module 34 is used to perform a gray box fuzzy test on the firmware program to be tested using the second data set.
[0140] Optionally, the hardening process includes any one of obfuscation, packing, data encryption and dynamic loading.
[0141] Optionally, the firmware program testing device 30 also includes a model building module 35, which is used to build an initial virtual model; use the initial virtual model to run the firmware program to be tested; determine whether the firmware program to be tested is running normally; in response to abnormal operation of the firmware program to be tested, adjust the model parameters of the initial virtual model, and re-execute the operation of the firmware program to be tested using the initial virtual model and its subsequent steps; in response to the firmware program to be tested running normally, use the latest initial virtual model as the target virtual model of the firmware program to be tested.
[0142] Optionally, the model building module 35 is used to obtain hardware architecture information and peripheral interface information of several control units; and build an initial virtual model based on the hardware architecture information and peripheral interface information of the several control units.
[0143] It should be noted that the device of this embodiment can execute the steps in the above method. For detailed description of the relevant content, please refer to the above method part, which will not be repeated here.
[0144] See also Figure 4 , Figure 4 4 is a schematic diagram of a framework of an electronic device according to an embodiment of the present application. In this embodiment, the electronic device 40 includes a memory 41 and a processor 42 .
[0145] The processor 42 may also be referred to as a CPU (Central Processing Unit). The processor 42 may be an integrated circuit chip having signal processing capabilities. The processor 42 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor 42 may also be any conventional processor 42, etc.
[0146] The memory 41 in the electronic device 40 is used to store program instructions required for the processor 42 to run.
[0147] The processor 42 is used to execute program instructions to implement the testing method of the firmware program in the present application.
[0148] See also Figure 5 , Figure 5It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 50 of the embodiment of the present application stores a program instruction 51, and when the program instruction 51 is executed, the test method of the firmware program provided by the present application is implemented. Among them, the program instruction 51 can form a program file and be stored in the above-mentioned computer-readable storage medium 50 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) executes all or part of the steps of each implementation method of the present application. The aforementioned computer-readable storage medium 50 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, and a tablet.
[0149] In the above scheme, the first data set is used to perform gray-box fuzz testing on the firmware program to be tested, and when the first code coverage information of the firmware program to be tested does not meet the preset requirements, the firmware program to be tested is subjected to symbolic execution processing to construct a new first data set, and the new first data set is used to perform gray-box fuzz testing on the firmware program to be tested again, and this cycle is repeated until the latest first code coverage information meets the preset requirements. By combining gray-box fuzz testing and symbolic execution processing, and performing gray-box fuzz testing and symbolic execution processing alternately, the first code branch that is difficult to trigger by gray-box fuzz testing can be triggered, thereby improving the test coverage rate of the firmware program to be tested.
[0150] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0151] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.
[0152] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and systems can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0153] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0154] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.
[0156] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A method for testing a firmware program, characterized in that: The method comprises: generating a first data set of a firmware program to be tested; Using the first data set to perform a gray box fuzzy test on the firmware program to be tested, to obtain first code coverage information of the firmware program to be tested; wherein the first code coverage information is used to characterize the execution status of each first code branch of the firmware program to be tested; Determining whether the first code coverage information meets preset requirements; In response to the first code coverage information not meeting the preset requirements, symbolic execution processing is performed on the firmware program to be tested to construct a new first data set, and the gray-box fuzz testing of the firmware program to be tested using the first data set and subsequent steps are re-executed until the latest first code coverage information meets the preset requirements.
2. The method according to claim 1, characterized in that The performing symbolic execution processing on the firmware program to be tested to construct a new first data set includes: Identifying unexecuted code branches in each of the first code branches; Generate a path constraint condition corresponding to the unexecuted code branch; Solve the data that meets the path constraint condition to obtain the new first data set.
3. The method according to claim 2, characterized in that The method further comprises: In response to the path constraint condition being unable to be solved, marking the corresponding unexecuted code branch with a preset identifier; The preset identifier is used to indicate that the unexecuted code branch is unreachable.
4. The method according to claim 1, characterized in that: The firmware program to be tested is in binary format, and the gray box fuzzy test is performed on the firmware program to be tested by using the first data set to obtain first code coverage information of the firmware program to be tested, including: Performing a first decompile process on the firmware program to be tested to obtain a first source program corresponding to the firmware program to be tested; Performing a first code stub processing on the first source program, and converting the first source program after the first code stub processing into a first binary firmware program; wherein the first binary firmware program includes a plurality of first code branches and first code segments inserted corresponding to each of the first code branches; inputting the first data set as an initial input data set into the first binary firmware program; Selecting a plurality of first fault input data that causes the first binary firmware program to run abnormally from the input data set, and adding the plurality of first fault input data to the first fault data set; Performing mutation processing on the first fault data set to obtain a new first fault data set; Determining current code coverage information of the first binary firmware program based on execution status of each of the first code segments; Determine whether a first preset iteration condition is met; wherein the first preset iteration condition includes: the number of mutation processing is greater than or equal to a preset number threshold, or the current code coverage information is the same as the historical code coverage information; In response to the first preset iteration condition not being met, inputting a new first fault data set as a new input data set into the first binary firmware program, and re-performing the steps of selecting a plurality of first fault input data that cause the first binary firmware program to run abnormally from the input data set and subsequent steps; In response to satisfying the first preset iteration condition, the latest current code coverage information is used as the first code coverage information.
5. The method according to claim 4, characterized in that The determining, based on the execution status of each of the first code segments, current code coverage information of the first binary firmware program includes: Counting the number of executed code segments in each of the first code segments; Based on the quantity, determining the current code coverage information; And / or, the method further comprises: In response to the latest first code coverage information meeting the preset requirement, a first test result is output; wherein the first test result includes all the first fault input data.
6. The method according to claim 1, characterized in that Before generating the first data set of the firmware program to be tested, the method further includes: Determining whether the firmware program to be tested has been reinforced; In response to the firmware program to be tested not having undergone the hardening process, executing the step of generating the first data set of the firmware program to be tested and subsequent steps thereof; In response to the firmware program to be tested undergoing the hardening process, a second data set of the firmware program to be tested is generated; and the second data set is used to perform a gray box fuzzy test on the firmware program to be tested.
7. The method according to claim 6, characterized in that The hardening process includes any one of obfuscation, packing, data encryption and dynamic loading.
8. The method according to claim 1, characterized in that Before generating the first data set of the firmware program to be tested, the method further includes: Build an initial virtual model; Running the firmware program to be tested using the initial virtual model; Determining whether the firmware program to be tested operates normally; In response to the abnormal operation of the firmware program to be tested, adjusting the model parameters of the initial virtual model, and re-executing the operation of the firmware program to be tested using the initial virtual model and subsequent steps; In response to the firmware program to be tested running normally, the latest initial virtual model is used as the target virtual model of the firmware program to be tested.
9. The method according to claim 8, characterized in that The constructing of the initial virtual model comprises: Obtain hardware architecture information and peripheral interface information of several control units; The initial virtual model is constructed based on the hardware architecture information and the peripheral interface information.
10. A firmware program testing device, characterized in that: The device comprises: A data generation module, used to generate a first data set of the firmware program to be tested; A first testing module is configured to perform a gray-box fuzz test on the firmware program to be tested using the first data set to obtain first code coverage information of the firmware program to be tested; wherein the first code coverage information is used to characterize the execution status of each first code branch of the firmware program to be tested; determine whether the first code coverage information meets a preset requirement; in response to the first code coverage information not meeting the preset requirement, perform symbolic execution processing on the firmware program to be tested to construct a new first data set, and re-execute the gray-box fuzz test on the firmware program to be tested using the first data set and its subsequent steps until the latest first code coverage information meets the preset requirement.
11. An electronic device, characterized in that: comprising a memory and a processor coupled to each other, The memory stores program instructions; The processor is used to execute the program instructions stored in the memory to implement the method according to any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program instructions, and the program instructions can be executed by a processor to implement the method according to any one of claims 1 to 9.